-6
$\begingroup$

Checking the count of open, closed and deleted questions of the site, I found the following stat:

enter image description here

What we can see, is a roughly exponential growth of the site until around middle 2015. At the time, also the count of the closed/deleted questions has shown a sharp increase, until the point that the new questions surviving the review processes are stagnating.

A healthy site has life.

New questions, new users, interesting problems, interesting review cases, and so on. And, continuous expansion.

Don't expel the new users! Teach them - and continuously motivate them to improve.

Having such a nice stat of new questions is not typical for most SE sites. The typical is a slow, linear expansion. Having an exponential increase, it is not a problem source. It is a gift.

You, avid ChemSE users, should not simply expel the "fresh meat" (both in the sense of posts and users). At least, if you are interested on a successful, living, growing site, and not in a static one, working actively to avoid its own success.

Be lenient. Invest effort to fix the fixable posts. Invest effort to "fix" the "fixable users" ;-).

The ones what you can't fix (both in the sense of posts and users), it is okay to expel (closures, downvotes), but if you avoid your growth with it, you are surely on the bad track.

It is particularly so for the first posters. They don't know too much about the site, their first experiences are crucial in their future participation. Do everything what you can, to make them feeling welcomed.

$\endgroup$
  • 5
    $\begingroup$ You fail to account for the fact that many of our posts are from "one off" unregistered accounts who, by that very nature, have no plans to return whether or not their questions remain open. I think you are cherry picking one metric here and being a bit myopic. $\endgroup$ – jonsca♦ Jun 3 '18 at 3:09
  • 2
    $\begingroup$ @jonsca So is it going everywhere. First the user posts something, not knowing the rules, not wanting to register, only posts a question. Or an answer. Or an edit. If they feels welcomed, you have a higher chance that they come back. If you want to make your site success, you should do everything to increase this chance. This stat is a proof that the VtC review is working against the success of the site, I think it deserves more than a "cherry pick" label. $\endgroup$ – peterh Jun 3 '18 at 3:13
  • 8
    $\begingroup$ I'm not sure where you're getting the notion that we're not successful from, honestly. From beta to our (very rapid) graduation and to this day, we've always been extremely cognizant of question quality, and I'm very glad that the community has kept that up over the years. I think we're right about where we need to be. $\endgroup$ – jonsca♦ Jun 3 '18 at 3:23
  • $\begingroup$ @jonsca This is how you feel. And there is what the stats show. Investing effort to kill all, even a little bit problematic questions, is not a good long-term strategy for the survival. You should invest more effort to fix what is fixable. Run the same query for different sites with similar topic. Your closure ratio is extreme. You didn't get a pile of crap, you got resources. You should convert these resources into a bigger, better, more successful Chemistry SE. It is your (=avid ChemSE users, reviewers) decision, 1) what is the quality level what you accept, $\endgroup$ – peterh Jun 3 '18 at 3:31
  • $\begingroup$ @jonsca 2) what is the effort what you invest into new users & new posts. | The point is that if you decide behind such a quality level & invested effort which avoids your own growth, this collective decision is bad. $\endgroup$ – peterh Jun 3 '18 at 3:32
  • 5
    $\begingroup$ Based on my experience on other sites, I have tried welcoming them even for off-topic questions (commenting while close-voting them). The result? They didn't come back anymore. They just want the answers, seeing this site as a "personal help desk". The growth metric doesn't indicate if those questions are good or bad, but I hope those surviving questions are all in good or better quality. $\endgroup$ – Andrew T. Jun 3 '18 at 8:03
  • 4
    $\begingroup$ Optimizing For Pearls, Not Sand $\endgroup$ – Loong♦ Jun 3 '18 at 12:12
  • 3
    $\begingroup$ What evidence do you have that our closed questions are salvageable? What about the quality of our unclosed questions? We do have a bit of a close problem right now, and your advice is clearly well-intentioned, but we have bigger issues than this. $\endgroup$ – pentavalentcarbon Jun 3 '18 at 12:32
  • $\begingroup$ @Loong Generally, I have no problem with a vehement QA, I said only that your current vehemence looks self-mutilating or suicidal. If the count of the new questions doubles while the count of the accepted ones stagnates, it is obviously over the Jeff Atwood's blogspot. $\endgroup$ – peterh Jun 3 '18 at 12:46
  • $\begingroup$ @pentavalentcarbon I don't know. I only know that doubling input results the same output, then a fixed-throughput system is a more realistic hyphotesis than a well-filtered one. $\endgroup$ – peterh Jun 3 '18 at 13:10
  • 6
    $\begingroup$ From my experience, you post meta full of hyperbole and exaggeration, cherry pick stats (that is, in the best cases) to make very bold assumptions, make a mountain out of a molehill of a problem, and you're only interested in your opinion. No matter how much the topic is discussed, you keep rephrasing your stance. That discourages me from participating in an otherwise valid discussion. IMHO, the only purpose of this vague, impractical post is to worry or depress. Nothing constructive. $\endgroup$ – M.A.R. ಠ_ಠ Jun 3 '18 at 14:45
  • 2
    $\begingroup$ We have a user who's gone off for a bit. Gaurang Tandon. He would engage one to one with every new user and guide them pretty well.His suggestions we're always constructive and always ended with a thank you. Yet I haven't seen a new user stick around and I don't think one can be more welcoming than that. $\endgroup$ – Avnish Kabaj Jun 3 '18 at 15:37
  • 5
    $\begingroup$ A better question here is: why do you even care? Your profile says you don't ask questions and don't provide answers on this site... I'm open to improvements, but I'd rather it come from members of the community or at least, people who use the site... $\endgroup$ – Zhe Jun 4 '18 at 15:18
  • $\begingroup$ @Zhe Don't worry, my own personal experiences with the ChemSE were mainly positive. Although it might be so because I know, how to ask and I inserted the required effort. I do this because I think the site would be better if it would be more inclusive, so you can see that as good-willing community work in the interest of lesser experienced SE users. $\endgroup$ – peterh Jun 4 '18 at 20:59
  • $\begingroup$ @AvnishKabaj I think exactly this type of people could be very useful on your site, I suggest to support them and their attitude. $\endgroup$ – peterh Jun 4 '18 at 21:06
8
$\begingroup$

From my perspective: Both the question and fbueckert's answer suffer from (different) errors in their premises.

The question assumes that a high closure/deletion rate is ipso facto evidence of unhealth in the site community. It doesn't take into account any metric of whether the questions being closed/deleted "deserve" to be closed/deleted. I would argue that, as Chem.SE has grown and gained both popularity and visibility, it has attracted more and more questions I would argue are genuinely "low quality": those described in penta's answer here, where what appears to be a homework or test problem is copy/pasted or screenshotted into the question box, and that's it.

Of course, in the community's aggregate ... enthusiasm, I guess? ... to cull these sorts of questions, other questions that are either obviously (to me) or at least debatably not pure cut-and-paste questions are also getting closed/deleted.

Are we closing too much? Probably.

Are we closing way too much? Probably not.

fbueckert's answer and its main conclusion are essentially a restatement of the Stack Exchange mission:

Lifetime, though, it looks like the closure/deletion rate is right around 2%. That makes it look like if the question survives the initial encounter and scrutiny, it's going to be around for the long haul.

Of course questions that survive initial screening will stay around for the long haul. That's the entire point of Stack Exchange: curating a durable, long-lived repository of quality Q&A.

$\endgroup$
  • $\begingroup$ "Of course questions that survive initial screening will stay around for the long haul." You probably don't see it much, since you don't really spend as much time in the review queues, but once in a great while very old questions come up, and they aren't all better than the current ones. Unfortunately, age doesn't correlate perfectly well with quality. $\endgroup$ – pentavalentcarbon Jun 7 '18 at 21:33
  • 1
    $\begingroup$ So, 1. some old questions should be closed that aren't, 2. some new questions shouldn't be closed that are, 3. some new questions should be edited that aren't, and do the opposite of 2 and 3 to give 4 and 5. Perhaps miraculously, we live. $\endgroup$ – pentavalentcarbon Jun 7 '18 at 21:34
  • $\begingroup$ @pentavalentcarbon Heh, point, there should have been a most in there. And, yep: that's a really good summary of why the community hasn't been able to agree on a rules change -- everybody has varying opinions on (1)-(5), and so there's no consensus possible on any of it, and the inertia of past decisions holds firm. :-) $\endgroup$ – hBy2Py Jun 7 '18 at 22:07
  • 1
    $\begingroup$ Even if the high closure rate is justified, it’s still a bad sign, as it shows that a large proportion of questions that come in are bad enough to be closed. It’s like a school, say, where the average score in final exams is 30%. Maybe the students’ answers were all genuinely rubbish, but the institution should probably review their teaching quality, their admissions process, or both. $\endgroup$ – orthocresol♦ Jun 8 '18 at 12:16
  • 2
    $\begingroup$ @orthocresol "but the institution should probably review their ... admissions process". Right, yeah, we should totally rework our system for letting people post here. :-P $\endgroup$ – hBy2Py Jun 8 '18 at 13:46
  • 3
    $\begingroup$ It is perhaps worth contemplating that, since more undergraduates take chemistry vs physics, that we might indeed get more poor questions in the first place vs Physics SE. As a member of both Chemistry and Physics, I'd say there are more poor questions here on Chemistry. And I can also say that, having taken way more physics than chemistry, if a chemistry question looks bad to me, it probably is pretty bad... $\endgroup$ – Jon Custer Jun 8 '18 at 14:03
  • 1
    $\begingroup$ @JonCuster In contemplating this question, I've been thinking right along these same lines -- chemistry to me seems like it's an introductory subject that is ~uniquely liable to these sorts of bad-for-SE questions. Biology is descriptive, and so easier to grasp in many ways. Physics is basically all equations, and relatively few equations at that, so there's less tendency to confusion about the right approach to take. Incorrect conceptions about pure math seem like they often lead to interesting thought experiments, where in chem they're just ... nonsensical. $\endgroup$ – hBy2Py Jun 8 '18 at 14:07
  • 2
    $\begingroup$ Not to say that it's a bad thing that people are asking these questions -- only way to learn! -- it's just that SE Q&A is not a good forum for answering such questions. Most such askers need tutoring, to help guide them away from misconceptions and into a correct understanding; not just information plus explanation, which is what the Q&A provides. $\endgroup$ – hBy2Py Jun 8 '18 at 14:10
  • 1
    $\begingroup$ I agree with you, on both comments. Chemistry is kind of between a rock and a hard place, with lots of hand waving explanations early on that only become clear once you have driven down several levels. And, from many of the poor questions I see, chemistry seems to be taught fairly poorly in many places. But, for example, I never understood thermodynamics from an introductory chemistry viewpoint, but once I hit statistical mechanics in physics it all became clear... $\endgroup$ – Jon Custer Jun 8 '18 at 14:14
6
$\begingroup$

Bit of a disclaimer: I am not a member of Chemistry. I have no stake in this community. I just like stats. I like seeing how numbers stack up.

The initial analysis rubbed me the wrong way when I first saw them, although I couldn't really put my finger on it at first. It seems somewhat suspect to compare how many questions were asked versus how many were closed or deleted; they don't tell the story very well; it just shows dips and spikes, without really telling you much about them.

Methodology

So, what I did was create a running total of all questions, all closed, and all deleted. That gave me a decent idea of what it was we're looking at.

Total Lifetime posts

From there, it's easy to tell how many questions are actually eligible for closure or deletion; can't very well close or delete a question that's already there.

Lifetime posts with remaining

Results

With the base query going, I could finally show some real stats. Here's what I came up with:

Question closure and deletion rate

This merits a bit of explanation, so here's what each line means:

  • PercentClosed - The number of questions closed that month, versus the total questions available that could be closed.
  • PercentDeleted - The number of questions deleted that month, versus the total questions available that could be deleted.
  • ClosedAs%OfNew - The number of questions closed, versus the number of new questions asked.
  • DeletedAs%OfNew - The number of questions deleted, versus the number of new questions asked.

From my reading of this, it looks like about 50% of new questions get closed and deleted. Lifetime, though, it looks like the closure/deletion rate is right around 2%. That makes it look like if the question survives the initial encounter and scrutiny, it's going to be around for the long haul.

Whether that's good or not, I can't say; I don't know enough about your site to say one way or the other. For comparison, here's a couple other site graphs:

Physics
Math
Skeptics
And, of course, SO itself

I think I got the math right, but if anyone wants to double-check or see if I missed something, you can find the query here.

$\endgroup$
  • $\begingroup$ Because the close/delete events happen mainly on the new questions, your stat integrates away the problem. $\endgroup$ – peterh Jun 6 '18 at 19:49
  • 5
    $\begingroup$ You think its a problem. I think the site's community should see, and decide, for itself. $\endgroup$ – fbueckert Jun 6 '18 at 20:15
  • $\begingroup$ If you think it is not a problem, then why did you generate a stat to deny the existence of the phenomenon? $\endgroup$ – peterh Jun 7 '18 at 6:40
  • 1
    $\begingroup$ @peterh Because I dislike it when misleading stats are used to push a narrative. To be clear, I used your query as the base of these statistics. I really don't like how you're trying to manipulate them to tell your story. I much prefer explaining what I did, how I got there, and a summary of the results, and then let others judge what it means. $\endgroup$ – fbueckert Jun 7 '18 at 11:39
  • $\begingroup$ No, you manipulated my query by integrating the total count of questions, with the goal to hide, that the site currently closes more than 50% of the newly posted questions, which currently stops its growth. My question is: why do you need this little "transformation", if you think that closing more than 50% of the newly posted questions is perfectly okay? $\endgroup$ – peterh Jun 7 '18 at 11:48
  • $\begingroup$ @peterh Where, exactly, do I hide that? That's one of the primary stats in the graph. I literally have an explanation of exactly that. Stopping the site's growth is a purely subjective opinion, and not worth entertaining. I also said nothing about it being okay. What I did do, though, is provide additional information to allow readers to make up their own minds. $\endgroup$ – fbueckert Jun 7 '18 at 12:09
  • $\begingroup$ Yes, your second graph shows it, but this stat doesn't show the total number of the new questions (per month). Overall, your stat is not so bad. What is bad, in my opinion, is that you are neglecting the obvious fact: if the review mechanism is working against the success of the site, then the review mechanism is bad, and not the stat which shows it. $\endgroup$ – peterh Jun 7 '18 at 12:26
  • 4
    $\begingroup$ Your opinion is exactly that. As someone who has no stake or familiarity with the site, I can't even begin to judge that the review queues are bad. What I can judge is your misleading stats and your analysis. I believe those who actually use the site should be making that judgement. $\endgroup$ – fbueckert Jun 7 '18 at 13:19
  • $\begingroup$ There was nothing "misleading" in my stat - exactly this is the reason, why was it so simple. There is nothing what could be unclear in it. And your stat shows essentially the same, while you call my stat bad, yours of course not. Furthermore, integrating anything gives typically false results on a sample where the majority of the activities happen to the new elements. $\endgroup$ – peterh Jun 7 '18 at 13:32
  • $\begingroup$ However, what is still unclear for me: what are you exactly denying now? The existence of the phenomenon I've pointed? Or its harmfulness? Please, react clearly. $\endgroup$ – peterh Jun 7 '18 at 13:34
  • 1
    $\begingroup$ @peterh No, I think I'm going to let this rest and let the community decide for itself. I'm done arguing with you. If anyone else wants to double-check my work or more clarification, I'll see what I can do. $\endgroup$ – fbueckert Jun 7 '18 at 13:36
  • $\begingroup$ I think it was a simple, clear question and it had deserved a simple, clear answer. $\endgroup$ – peterh Jun 7 '18 at 13:38
  • $\begingroup$ Thanks for the plots. The data looks in line with what I've heard about Physics and Math. Your comment about old questions also holds true, since I almost never see them appear in the review queues. I think that starts getting at the heart of a real discussion. $\endgroup$ – pentavalentcarbon Jun 7 '18 at 21:30
2
$\begingroup$

First things first: you fail to mention in your graph that each data point represents only the questions posted in that month! This is a rather important point, because the label AllQuestions reads as if it were total questions—which it is not.

If total questions were suddenly plummeting, I would indeed be concerned. But for your data I offer an alternative view: the significant valleys in all three curves occur predictably around July and to a lesser extent December. This is exactly the time when most northern hemisphere and western universities have summer or christmas holidays or low-intensity periods.

The site then receives a new bump in questions every time a term starts or exams draw near—whichever one it is. This pattern starts appearing in the second half of 2014 but doesn’t become strongly noticable until a year later.

For regular users of the site, they know this as the time when the homework wave builds up—and many lose their motivation due to the many, as penta mentioned, low-quality, zero-effort questions that we have to deal with and that form the bulk of the closed and deleted questions (as per my experience). Notice however, that the absolute number of new questions not closed or deleted seems more constant: the closure and deletion bumps align well with the new question bumps.

Finally, I want to draw the attention to the total question stats fbueckert posted. These show that after an initial exponential growth phase, this site is growing in a more linear manner. That can have a number of reasons but one rather reasonable assumption is saturation of the target audience. While exponential growth isn’t bad per se linear growth isn’t either. In fact, linear growth allows people to still get a rather good overview of what is posted with a relatively constant effort put in.

Personally, I don’t want to reach a state like StackOverflow is in where you are more likely to miss than see a question.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .