16
$\begingroup$

The data speaks for itself:

$$\begin{array}{c|c|c|c} \hline \text{Time span} & \text{Number of questions} & \text{Number of questions closed} & \text{Close percentage} \\ \hline \pu{90 d} & 2672 & 1560 & \mathbf{58.38 \%} \\ \pu{30 d} & 960 & 537 & \mathbf{55.94 \%} \\ \pu{14 d} & 453 & 249 & \mathbf{54.97 \%} \\ \pu{7 d} & 226 & 121 & \mathbf{53.54 \%} \\ \hline \end{array}$$

This raises some questions:

  1. Are the questions simply bad, or have there been cases of unwarranted closure?

  2. Are exceptionally high closure rates a problem to begin with, i.e. do we need to do anything about it?

  3. If yes, what can we do about it, as individuals and as a community?

and perhaps a tougher, but related question: what is the direction you want this site to head in, or what would you like to see this site become?


Edit: Because I was asked for data, percentage of closed or deleted questions, since graduation.

enter image description here

"But you've been mass-deleting questions because of Spring Cleaning!"

Nope, this doesn't affect the overall trend and certainly not the incredible increase in the last few months. I believe this graph reflects the percentage of questions posted within a time span which are closed or deleted. In any case, the "deleted questions" graph is almost flat, with a variation of at most 2 to 3 percentage points.

"But seasonal homework askers, can't you see the trend, stupid mod"

Yeah, that doesn't mean the peak height should go up so drastically from one year to the next.

$\endgroup$
  • 1
    $\begingroup$ By way of comparison: chemistry.meta.stackexchange.com/a/3305/7951 $\endgroup$ – Loong Nov 12 '17 at 11:07
  • $\begingroup$ Did you use a SEDE query, the 10k tools or something else? I wonder if one can go and generate more fine-tuned statistics. $\endgroup$ – Jan Nov 12 '17 at 14:16
  • 2
    $\begingroup$ @Jan 10 000 tools $\endgroup$ – orthocresol Nov 12 '17 at 15:54
  • $\begingroup$ Most creative way to overcome the minimum comment length I’ve seen yet =D $\endgroup$ – Jan Nov 12 '17 at 15:54
  • $\begingroup$ What do you have in mind, @Jan? $\endgroup$ – hBy2Py Nov 12 '17 at 17:51
  • 1
    $\begingroup$ Without any other data to compare this to, it seems a pointless question. Are these percentages significantly higher than the same period 1/2/3 years ago? Are these percentages in line with other SE sites of similar size/theme? $\endgroup$ – NotEvans. Nov 12 '17 at 21:49
  • 1
    $\begingroup$ @NotEvans., I think there's no real need for comparison. Anything above 50% is quite obviously not good. Literally more questions get closed than those which remain open - this cannot be a healthy sign. But if you want data... $\endgroup$ – orthocresol Nov 12 '17 at 22:00
  • 1
    $\begingroup$ @NotEvans. See edit. $\endgroup$ – orthocresol Nov 12 '17 at 22:10
  • 2
    $\begingroup$ The point of spring cleaning was to get rid of stub questions, i.e. closed questions with low scores and bad answers. This wouldn't really affect the statistics much, since they would be counted either way. $\endgroup$ – Martin - マーチン Nov 13 '17 at 4:38
  • 1
    $\begingroup$ Can you break down the percentages by close reason? It might help to see how many are duplicates, how many off-topic, how many homework, etc. $\endgroup$ – R.M. Nov 13 '17 at 19:53
  • $\begingroup$ Also, does this count closed-but-then-later-reopened questions? Theoretically, question closure should be an opportunity to improve questions. If closed questions later get reopened in an improved state, that somewhat mitigates the jaw drop on the 50+% closure rate. (Yeah, I too doubt that closed-but-then-later-reopened questions affect the numbers appreciably, but it's worth checking to make sure.) $\endgroup$ – R.M. Nov 13 '17 at 19:58
  • 6
    $\begingroup$ Yes, there are lots of bad questions of late. Most of them seem to be of the homework variety. At some point it may stabilize and people will ask good questions. One can hope. $\endgroup$ – Jon Custer Nov 13 '17 at 20:20
  • $\begingroup$ @R.M., here is the information which you want, taken from the last 90 days. (The close percentage is slightly different from that in the meta post, because one day has passed.) In total, 38 questions were reopened, which is 2.4% of all closed questions and 1.4% of all questions. This brings the effective close rate in the last 90 days down to 56.4%. Similarly insignificant changes are seen for the last 30 days, etc. (Also, this information is available to users with ≥10k rep at chemistry.stackexchange.com/tools/question-close-stats/.) $\endgroup$ – orthocresol Nov 14 '17 at 0:22
  • 2
    $\begingroup$ I'm with @NotEvans. here, I think. We really need something to compare these numbers to. I'm happy not to have cruft around, and we have plenty of very, very good questions that serve the internet well. $\endgroup$ – jonsca Nov 14 '17 at 0:35
  • 3
    $\begingroup$ @jonsca I agree that there is some need for comparison, so I pulled that graph from mod analytics. That said, there are three possible scenarios: (1) the community is closing things that shouldn't be closed, which is obviously problematic; (2) the average question that we receive is so lousy that it is more likely to be closed than not, which is also very discouraging - if this upward trend continues, the good contributors (askers and answerers) will leave. (3) Both 1 and 2. And I think it is both. $\endgroup$ – orthocresol Nov 14 '17 at 0:49
7
$\begingroup$

I have struggled with deciding to put more thought for public consumption, but since this is going to come up again, I don't want to brake my brain again and write some of my thoughts down. This is not at all intended to be complete or even conclusive, these are just my personal experiences from the last few weeks.

I have noticed this trend some time ago and was not very happy about it. (Remember, remember, ...) Eventually that was also one of the reason to completely rewrite the homework policy - and we all remember how much fun these discussions were.

Moving on, orthocresol asked some questions:

  1. Are the questions simply bad, or have there been cases of unwarranted closure?

#Yes, and #yes.
Today was almost a good day - at least while I was monitoring it from the chat. Not so many questions coming in earned themselves a downey-votey.
Usually, at least every second question coming in is some kind of meh. And I'm only saying it this way, because I dislike using strong language. That obviously ranges from copy and pasted exercises, to incomprehensible jibber-jabber, lazy broad questions, to completely off-topic whatever.
Usually more; you simply cannot run away or escape from these \$#^&!(% questions.

That's basically what we inherited with the fame of becoming graduated, getting this shiny new design and being generally a great resource.

Just because a question is bad, it doesn't mean it's off-topic. Hence I also think there are plenty a many unwarranted closures or at least ambiguous ones. Or closures with a wrong/misleading reason. I ever so more often personally disagree with the usage of the homework reason. (Oh well, tell us something we don't already know.) I simply find it not at all instructive and only inna negligible number of cases they're reopened. Let's not walk down that argumentation. Changing that is something that I have given up all hope to do.

  1. Are exceptionally high closure rates a problem to begin with, i.e. do we need to do anything about it?

Yes, maybe, no (?).
It can't be good though. Judging all that comes in and the quality of such, even some accidental closures may have a bad effect long term. Yes, we're still retaining more open questions daily than when we graduated. The question here is, whether that is good enough for us.

I think some more leniency would benefit us. There simply is not much you can do about this. The stone is rolling fast now and it is hard to change direction - we tried, we failed. We need baby steps.

  1. If yes, what can we do about it, as individuals and as a community?

As individuals we need to follow through the actual closing. It's worth not the effort of the click without guidance. And it's even less helpful if votes are not used effectively. There are users here having fewer (up + down) votes than close reviews.
If you want to discourage a certain kind of question you have to down-vote.
Closing it is not enough. Only with a negative score makes it disappear from the front page, the thing new users are probably seeing first.

As a community, I don't know. We have tried changing the rules. We have been talking a plenty of times. We're manually cleaning up old unhelpful stuff that would set bad examples. That's incredibly slow though.


Now some others had some more to say...

Without any other data to compare this to, it seems a pointless question. Are these percentages significantly higher than the same period 1/2/3 years ago? Are these percentages in line with other SE sites of similar size/theme? – NotEvans.

Orthocresol already added the data, comparing the periods last years. Note that the percentage is rising quite steadily, topping out at about 65%. I find the data of similar sites more than irrelevant, we're neither administering physics, nor math, or biology, or anything else.
I even find the data from last year only tangentially relevant. Closing every other question simply cannot be a good sign. There is no positive spin on it.

Can you break down the percentages by close reason? It might help to see how many are duplicates, how many off-topic, how many homework, etc. – R.M.

The top reason is homework, second is unclear, then comes broad. Duplicate closure comes in fourth with about 10% of the closed questions. And often when something is closed as duplicate I have a hard time recognising the link between the questions. (But that's just me, I guess.)

Also, does this count closed-but-then-later-reopened questions? Theoretically, question closure should be an opportunity to improve questions. If closed questions later get reopened in an improved state, that somewhat mitigates the jaw drop on the 50+% closure rate. (Yeah, I too doubt that closed-but-then-later-reopened questions affect the numbers appreciably, but it's worth checking to make sure.) – R.M.

The number of reopened questions is tiny, typically below 5% of the closed ones. It's barely a drop in the ocean.

Yes, there are lots of bad questions of late. Most of them seem to be of the homework variety. At some point it may stabilize and people will ask good questions. One can hope. – Jon Custer

While I generally agree with that notion that it is a seasonal thing. It won't stabilise as much as before the hike. Additionally the delete rate is rising at a much slower rate, which means we're retaining the stuff that we have decided doesn't fit our site.

I'm with @NotEvans. here, I think. We really need something to compare these numbers to. I'm happy not to have cruft around, and we have plenty of very, very good questions that serve the internet well. – jonsca

Yes indeed, we have plenty of good stuff around, but it is getting harder and harder to find. And there are plenty of new users coming too, but old ones leaving. That's natural. And the biggest problem is that our dedicated user base is simply not growing as fast as the amount of substandard content.


In conclusion, I think there is a general problem with too much substandard questions coming in, and there is the danger that the not quite sub standard stuff is getting closed. There is the real problem that too much of the unhelpful stuff is being retained instead of consequently cleaned up.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .