1

Has anyone classified the range of questions as a function of user score? Is it possible to find themes according to score bins and thus define some hierarchy of philosophical reasoning (from low scorers to high scorers), or concurrently discover pervasive themes across all score ranges?

  • I haven't done much with the Data Explorer yet. I'd definitely be interested to see the results of these queries, though! :) – Joseph Weissman Jan 29 '14 at 1:19
  • 1
    @JosephWeissman there you go! – Lucas Apr 4 '14 at 20:17
4

So, I've only done some basic investigations. First of all, there is an issue with correlations with user score. The way the database works is that there are separate tables for users and posts. Each post has a user id, and each user has a score. You can cross-reference the tables, but you wont get the score at the time of posting, only their score now. Of course, one could take a small recent time window and this would remedy it, but the number of posts is quite low on this site, there are only 2000 odd questions in total. Hopefully, when you see the dubiousness of the statistics of the whole population of questions it will be clear why this might not be a worthwhile exercise.

What I have done, is pulled the question scores and tags. There are some things to bear in mind here:

  • Tags do not form categories that are independent or well-assigned to. For example, there are two tags "positivism" and "logical-positivism" whose statistics are quite different.

  • Scores are produced by a complex non-linear system that is contingent on many factors. By their popularity questions can be viewed more times and accumulate a greater score, people vote to counter down-votes, downvoting and upvoting are used quite differently and effect the user who does it in different ways, and so forth...

First of all, the score histogram for all questions

enter image description here

As you can see

  • Positive and negative scoring is quite different
  • There are a number of questions that get upvoted disproportionally

This makes statistics such as the mean score quite difficult to interpret. Variance too. The higher order moments are dubious because of lack of data. The median isn't so bad, but the most illuminating statistic I have looked at is much much simpler, the percentage of positive votes i.e. what percentage are considered by the community to be good questions. I think this reflects peoples voting behaviour much better than some of the more go-to statistics I have mentioned. It's very easy to interpret. I have ignored all tags with less than five questions. Without further ado, the first of two tables: sorted by fraction of acceptance, then by fraction of questions with 4 or more votes:

RANK                   TAG       > -1      > 3                
1)                kierkegaard: 1         0.875     (8 posts)
2)               human-rights: 1         0.85714   (7 posts)
3)         logical-positivism: 1         0.81818   (11 posts)
4)             animal-welfare: 1         0.71429   (7 posts)
5)                     hobbes: 1         0.66667   (6 posts)
6)   philosophy-of-technology: 1         0.66667   (12 posts)
7)              formal-theory: 1         0.66667   (6 posts)
8)            critical-theory: 1         0.66667   (9 posts)
9)        analytic-philosophy: 1         0.58824   (34 posts)
10)                  socrates: 1         0.58824   (17 posts)
11)            existentialism: 1         0.5625    (16 posts)
12)               linguistics: 1         0.55556   (9 posts)
13)                 causation: 1         0.55      (20 posts)
14)                 descartes: 1         0.51852   (27 posts)
15)                empiricism: 1         0.5       (16 posts)
16)               john-searle: 1         0.5       (6 posts)
17)                pragmatism: 1         0.5       (12 posts)
18)                 induction: 1         0.5       (14 posts)
19)          bertrand-russell: 1         0.5       (18 posts)
20)                    qualia: 1         0.5       (10 posts)
21)                  epicurus: 1         0.46667   (15 posts)
22)                     plato: 1         0.45161   (62 posts)
23)            utilitarianism: 1         0.45      (20 posts)
24)                    goedel: 1         0.42857   (21 posts)
25)               rationalism: 1         0.4       (10 posts)
26)                   atheism: 1         0.4       (15 posts)
27)               computation: 1         0.38889   (18 posts)
28)                    sartre: 1         0.375     (8 posts)
29)   critique-of-pure-reason: 1         0.375     (8 posts)
30)               probability: 1         0.375     (8 posts)
31)                    rights: 1         0.375     (8 posts)
32)                  ayn-rand: 1         0.36364   (11 posts)
33)         philosophy-of-law: 1         0.34783   (23 posts)
34)               modal-logic: 1         0.34375   (32 posts)
35)                   leibniz: 1         0.33333   (6 posts)
36)                  stoicism: 1         0.33333   (9 posts)
37)               physicalism: 1         0.33333   (9 posts)
38)         german-philosophy: 1         0.3       (10 posts)
39)                 democracy: 1         0.28571   (7 posts)
40)             argumentation: 1         0.28      (25 posts)
41)     philosophy-of-physics: 1         0.28      (25 posts)
42)                      hume: 1         0.27778   (18 posts)
43)                     zizek: 1         0.25      (8 posts)
44)                  identity: 1         0.25      (12 posts)
45)                      marx: 1         0.23077   (26 posts)
46)                metaethics: 1         0.21429   (14 posts)
47)                   spinoza: 1         0.2       (10 posts)
48)                      duty: 1         0.16667   (6 posts)
49)                 heidegger: 1         0.16667   (18 posts)
50)             individualism: 1         0.16667   (6 posts)
51)              reductionism: 1         0.16667   (6 posts)
52)                     locke: 1         0.16667   (6 posts)
53)           possible-worlds: 1         0.14286   (7 posts)
54)           social-contract: 1         0.125     (8 posts)
55)               determinism: 1         0.083333  (12 posts)
56)                   concept: 1         0         (8 posts)
57)              presocratics: 1         0         (7 posts)
58)                  homework: 1         0         (7 posts)
59)                  rhetoric: 1         0         (6 posts)
60)                complexity: 1         0         (8 posts)
61)                   marxism: 1         0         (8 posts)
62)                 nietzsche: 0.98039   0.47059   (51 posts)
63)                      kant: 0.97917   0.4375    (96 posts)
64)                   physics: 0.97222   0.61111   (36 posts)
65)                 free-will: 0.97222   0.44444   (36 posts)
66)                 aristotle: 0.96825   0.44444   (63 posts)
67)         reference-request: 0.96386   0.4759    (166 posts)
68)               metaphysics: 0.96139   0.37066   (259 posts)
69)      political-philosophy: 0.96117   0.36893   (103 posts)
70)               terminology: 0.95726   0.35043   (117 posts)
71)                skepticism: 0.95652   0.47826   (23 posts)
72)    continental-philosophy: 0.95652   0.30435   (46 posts)
73)                 semantics: 0.95652   0.26087   (23 posts)
74)                  ontology: 0.95283   0.39623   (106 posts)
75)                    popper: 0.95      0.65      (20 posts)
76)             phenomenology: 0.94737   0.47368   (19 posts)
77)              epistemology: 0.94444   0.46181   (288 posts)
78)                 sociology: 0.94444   0.38889   (18 posts)
79) philosophy-of-mathematics: 0.9433    0.42268   (194 posts)
80)                    ethics: 0.94158   0.36426   (291 posts)
81)                     death: 0.94118   0.35294   (17 posts)
82)                 evolution: 0.94118   0.23529   (17 posts)
83)     philosophy-of-science: 0.93774   0.44358   (257 posts)
84)               rationality: 0.93333   0.53333   (15 posts)
85)                   meaning: 0.93333   0.2       (15 posts)
86)                     logic: 0.93111   0.38      (450 posts)
87)           human-condition: 0.92857   0.42857   (14 posts)
88)     history-of-philosophy: 0.92806   0.44604   (139 posts)
89)                psychology: 0.925     0.125     (40 posts)
90)   contemporary-philosophy: 0.92308   0.61538   (13 posts)
91)            psychoanalysis: 0.92308   0.15385   (13 posts)
92)                 knowledge: 0.91935   0.40323   (62 posts)
93)                relativism: 0.91667   0.41667   (12 posts)
94)             postmodernism: 0.91667   0.33333   (12 posts)
95)               materialism: 0.91667   0.25      (12 posts)
96)   philosophy-of-economics: 0.91667   0.20833   (24 posts)
97)                     hegel: 0.91667   0.20833   (24 posts)
98)      deleuze-and-guattari: 0.91667   0.083333  (12 posts)
99)        philosophy-of-mind: 0.91209   0.32967   (91 posts)
100)   philosophy-of-language: 0.91176   0.42647   (68 posts)
101)                fallacies: 0.90909   0.27273   (44 posts)
102)       ancient-philosophy: 0.90909   0.27273   (33 posts)
103)               aesthetics: 0.90909   0.27273   (22 posts)
104)                 idealism: 0.90909   0.18182   (11 posts)
105)                    truth: 0.90698   0.44186   (43 posts)
106)             wittgenstein: 0.90625   0.5625    (32 posts)
107)                existence: 0.90566   0.26415   (53 posts)
108)        scientific-method: 0.90476   0.47619   (21 posts)
109)              objectivism: 0.9       0.5       (10 posts)
110)           metaphilosophy: 0.9       0.43333   (30 posts)
111)                solipsism: 0.9       0.4       (10 posts)
112)                mind-body: 0.9       0         (10 posts)
113)          quantum-physics: 0.89286   0.5       (28 posts)
114)       cognitive-sciences: 0.88889   0.33333   (9 posts)
115)                  deleuze: 0.88889   0         (9 posts)
116)                 theology: 0.88542   0.44792   (96 posts)
117)  artificial-intelligence: 0.88462   0.38462   (26 posts)
118)                  reality: 0.88      0.36      (25 posts)
119)                    proof: 0.875     0.40625   (32 posts)
120)                     self: 0.875     0.375     (8 posts)
121)                cosmology: 0.875     0.33333   (24 posts)
122)                     time: 0.87097   0.3871    (31 posts)
123)            consciousness: 0.87097   0.27419   (62 posts)
124)                 buddhism: 0.86207   0.2069    (29 posts)
125)                whitehead: 0.85714   0.42857   (7 posts)
126)            good-and-evil: 0.84211   0.31579   (19 posts)
127)              translation: 0.83333   0.33333   (6 posts)
128)       eastern-philosophy: 0.83333   0.22222   (18 posts)
129)               naturalism: 0.83333   0.16667   (6 posts)
130)             christianity: 0.83333   0.16667   (6 posts)
131)                 hinduism: 0.83333   0         (6 posts)
132)               positivism: 0.83333   0         (6 posts)
133)                 language: 0.8125    0.3125    (16 posts)
134)   philosophy-of-religion: 0.81159   0.27536   (69 posts)
135)                   badiou: 0.8       0.4       (10 posts)
136)                  paradox: 0.8       0.36667   (30 posts)
137)             subjectivity: 0.76923   0.15385   (13 posts)
138)            social-ethics: 0.76923   0.15385   (13 posts)
139)                education: 0.76471   0.41176   (17 posts)
140)                  society: 0.74286   0.2       (35 posts)
141)                 infinity: 0.73333   0.26667   (15 posts)
142)                happiness: 0.71429   0.28571   (7 posts)
143)                  dualism: 0.66667   0.33333   (6 posts)
144)                     life: 0.57895   0.21053   (19 posts)
145)      social-epistemology: 0.5       0         (6 posts)

The first thing to conclude is that we're pretty good at upvoting (I've looked SO's data and we do very well in comparison). But are there any trends here? It's pretty hard to say, but the tags near the bottom are probably a bit "fluffier" than the others, you'll find the religion tags near the bottom, as well as things that popular philosophy would address, such as infinity, society, QM and solipsism. Still, not that strong a trend.

The other thing that produced interesting results (I tried lots of things) was simple a measure of confusingness/non-informedness/non-unanimity - the fraction of questions that got no up or down votes.

These are questions

  • which were found really uninteresting
  • where as a community we had mixed feelings
  • where people did not feel they had the authority to upvote or downvote on the topic

I think this is a reasonable interpretation on this site, where questions don't seem to buried that often, but I admit I might be playing psychologist more than is warranted.

This conforms to stereotypes fairly nicely...

RANK                TAG          FRACTION == 0
1)                    deleuze: 0.555555555556(9 posts)
2)                   hinduism: 0.5(6 posts)
3)       deleuze-and-guattari: 0.416666666667(12 posts)
4)                       duty: 0.333333333333(6 posts)
5)              individualism: 0.333333333333(6 posts)
6)               reductionism: 0.333333333333(6 posts)
7)               christianity: 0.333333333333(6 posts)
8)                  evolution: 0.294117647059(17 posts)
9)                   homework: 0.285714285714(7 posts)
10)           possible-worlds: 0.285714285714(7 posts)
11)                 whitehead: 0.285714285714(7 posts)
12)                aesthetics: 0.272727272727(22 posts)
13)                psychology: 0.25(40 posts)
14)               materialism: 0.25(12 posts)
15)                     zizek: 0.25(8 posts)
16)   philosophy-of-economics: 0.25(24 posts)
17)                complexity: 0.25(8 posts)
18)                     hegel: 0.25(24 posts)
19)                      self: 0.25(8 posts)
20)            utilitarianism: 0.25(20 posts)
21)                  buddhism: 0.241379310345(29 posts)
22)                  socrates: 0.235294117647(17 posts)
23)              subjectivity: 0.230769230769(13 posts)
24)                      hume: 0.222222222222(18 posts)
25)                 semantics: 0.217391304348(23 posts)
26)        ancient-philosophy: 0.212121212121(33 posts)
27)         german-philosophy: 0.2(10 posts)
28)                   spinoza: 0.2(10 posts)
29)                  infinity: 0.2(15 posts)
30)                 solipsism: 0.2(10 posts)
31)                     plato: 0.193548387097(62 posts)
32)                      marx: 0.192307692308(26 posts)
33)   artificial-intelligence: 0.192307692308(26 posts)
34)                 descartes: 0.185185185185(27 posts)
35)                 fallacies: 0.181818181818(44 posts)
36)                  ayn-rand: 0.181818181818(11 posts)
37)    continental-philosophy: 0.173913043478(46 posts)
38)                   society: 0.171428571429(35 posts)
39)                 existence: 0.169811320755(53 posts)
40)                   leibniz: 0.166666666667(6 posts)
41)               determinism: 0.166666666667(12 posts)
42)                   dualism: 0.166666666667(6 posts)
43)               translation: 0.166666666667(6 posts)
44)                     locke: 0.166666666667(6 posts)
45)                 cosmology: 0.166666666667(24 posts)
46)                      time: 0.161290322581(31 posts)
47)                   reality: 0.16(25 posts)
48)     philosophy-of-physics: 0.16(25 posts)
49)                      life: 0.157894736842(19 posts)
50)            psychoanalysis: 0.153846153846(13 posts)
51)            animal-welfare: 0.142857142857(7 posts)
52)                metaethics: 0.142857142857(14 posts)
53)           human-condition: 0.142857142857(14 posts)
54)              human-rights: 0.142857142857(7 posts)
55)                  epicurus: 0.133333333333(15 posts)
56)                     logic: 0.133333333333(450 posts)
57)    philosophy-of-language: 0.132352941176(68 posts)
58)                skepticism: 0.130434782609(23 posts)
59)               terminology: 0.128205128205(117 posts)
60)               metaphysics: 0.127413127413(259 posts)
61)                   concept: 0.125(8 posts)
62)                  language: 0.125(16 posts)
63)   critique-of-pure-reason: 0.125(8 posts)
64)               probability: 0.125(8 posts)
65)           social-contract: 0.125(8 posts)
66)             argumentation: 0.12(25 posts)
67)                    ethics: 0.116838487973(291 posts)
68)                     truth: 0.116279069767(43 posts)
69)                  ontology: 0.11320754717(106 posts)
70)                  stoicism: 0.111111111111(9 posts)
71)               physicalism: 0.111111111111(9 posts)
72)                 heidegger: 0.111111111111(18 posts)
73)                 sociology: 0.111111111111(18 posts)
74)        cognitive-sciences: 0.111111111111(9 posts)
75)          bertrand-russell: 0.111111111111(18 posts)
76)                   physics: 0.111111111111(36 posts)
77)        philosophy-of-mind: 0.10989010989(91 posts)
78) philosophy-of-mathematics: 0.10824742268(194 posts)
79)     history-of-philosophy: 0.107913669065(139 posts)
80)      political-philosophy: 0.106796116505(103 posts)
81)             phenomenology: 0.105263157895(19 posts)
82)    philosophy-of-religion: 0.101449275362(69 posts)
83)                    badiou: 0.1(10 posts)
84)                 mind-body: 0.1(10 posts)
85)            metaphilosophy: 0.1(30 posts)
86)                    qualia: 0.1(10 posts)
87)             consciousness: 0.0967741935484(62 posts)
88)                 knowledge: 0.0967741935484(62 posts)
89)         reference-request: 0.0963855421687(166 posts)
90)                      kant: 0.09375(96 posts)
91)                  idealism: 0.0909090909091(11 posts)
92)              epistemology: 0.0902777777778(288 posts)
93)     philosophy-of-science: 0.0894941634241(257 posts)
94)                relativism: 0.0833333333333(12 posts)
95)                pragmatism: 0.0833333333333(12 posts)
96)  philosophy-of-technology: 0.0833333333333(12 posts)
97)             postmodernism: 0.0833333333333(12 posts)
98)                  identity: 0.0833333333333(12 posts)
99)                 aristotle: 0.0793650793651(63 posts)
100)                nietzsche: 0.078431372549(51 posts)
101)            social-ethics: 0.0769230769231(13 posts)
102)              rationality: 0.0666666666667(15 posts)
103)                  meaning: 0.0666666666667(15 posts)
104)               empiricism: 0.0625(16 posts)
105)           existentialism: 0.0625(16 posts)
106)                    proof: 0.0625(32 posts)
107)                education: 0.0588235294118(17 posts)
108)              computation: 0.0555555555556(18 posts)
109)                free-will: 0.0555555555556(36 posts)
110)       eastern-philosophy: 0.0555555555556(18 posts)
111)                 theology: 0.0520833333333(96 posts)
112)                causation: 0.05(20 posts)
113)                   popper: 0.05(20 posts)
114)                   goedel: 0.047619047619(21 posts)
115)        philosophy-of-law: 0.0434782608696(23 posts)
116)                  paradox: 0.0333333333333(30 posts)
117)              modal-logic: 0.03125(32 posts)
118)                   hobbes: 0.0(6 posts)
119)             presocratics: 0.0(7 posts)
120)                happiness: 0.0(7 posts)
121)                    death: 0.0(17 posts)
122)        scientific-method: 0.0(21 posts)
123)            good-and-evil: 0.0(19 posts)
124)                 rhetoric: 0.0(6 posts)
125)              john-searle: 0.0(6 posts)
126)              kierkegaard: 0.0(8 posts)
127)      social-epistemology: 0.0(6 posts)
128)  contemporary-philosophy: 0.0(13 posts)
129)                induction: 0.0(14 posts)
130)                   sartre: 0.0(8 posts)
131)               naturalism: 0.0(6 posts)
132)              rationalism: 0.0(10 posts)
133)            formal-theory: 0.0(6 posts)
134)          quantum-physics: 0.0(28 posts)
135)                   rights: 0.0(8 posts)
136)              objectivism: 0.0(10 posts)
137)       logical-positivism: 0.0(11 posts)
138)               positivism: 0.0(6 posts)
139)      analytic-philosophy: 0.0(34 posts)
140)                democracy: 0.0(7 posts)
141)              linguistics: 0.0(9 posts)
142)                  marxism: 0.0(8 posts)
143)          critical-theory: 0.0(9 posts)
144)             wittgenstein: 0.0(32 posts)
145)                  atheism: 0.0(15 posts)

All in all, not hugely conclusive, make of it what you like. The only thing I can be sure of is that most posts get an upvote!

  • 1
    I appreciate the work! Thank you for sharing. However I'm not entirely understanding everything. Would you mind giving an example of what you mean? Say, for each table, by chosing a line and stating what it is we are looking at? For example, in table 1 the tag kierkegard ranks first with 1 (>-1) and 0.875 (>3) given 8 posts...or something similar etc. Thank you again. – val Apr 4 '14 at 23:50
  • @val In the first table, posts tagged kirkegard were the most reliable point scorers: 100% got zero or more points, 87.5% (7 8ths) got more four or more points. In the second table deluze was the most ambiguously rated, with 55% posts not getting any (net) votes at all. – Lucas Apr 5 '14 at 0:51
  • @val and val's comment upvoter. Did my comment help? – Lucas Apr 5 '14 at 4:31
  • OK i think i get it. If I understand correctly, a question can have zero votes yet 5 are upvotes an 5 are downvotes (but we wouldn't see that in the front end). Also how would you normalize by the number of posts, since it's different for every tag. I wonder if a PCA on tags vs questions would do something similar. – val Apr 5 '14 at 14:13
  • @val Yeah, you don't get anything that you can't get by browsing about (except for internal ids used for cross-referencing between database tables). It's all normalised by number of votes (it a fraction of votes for that tag), I included the number of votes because that would affect the variance of the statistic: if there are a small number of questions for a tag then the statistic would be quite variable, whereas, for large numbers it would be quite reliable. The variance is given (approximately) by p(1-p)/n, where p is the score in column and n is the number of posts. – Lucas Apr 5 '14 at 17:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .