9
$\begingroup$

The experiment is over. Here are the stats, beautifully crafted with heavy MathJax with stats topping$^1$.

Here's my take. Nothing really changed where we worried about change. You can play with this SEDE query to compare$^2$. We didn't get an influx of stuff that didn't end up answered. We didn't get flooded with homework. We only got a little more than we used to, and that might have been because schools started in some parts of the world.

What we did get, however, was a focused meta. No complaints, no rants, no nothing. Just rigorous meta discussion.

So if you ask this member of the community, I'd say we just keep it that way. But I'm just a single, albeit talkative member here, and we didn't have the chance to solicit feedback during the experiment. So this post is meant to gather your feedback on how the experiment went on. Do you have a complaint to file? The answer part is your chance.

Should homework closure criteria remain what they were during the experiment$^3$? Let us know in the answers below.

1: Thanks Ortho!
2: Thanks rene!
3: This doesn't mean we won't be changing anything anymore. I'm starting to gather ideas for a third custom close reason. Stay tuned.

$\endgroup$
2
  • 2
    $\begingroup$ Well, since it worked out OK for a month, I think we shall continue closing only "blatantly obvious homework questions". Let's see how it works in a long run. $\endgroup$
    – Wildcat
    Commented Oct 1, 2016 at 17:56
  • 2
    $\begingroup$ My vision is to rework that reason to include questions asking for computations and stuff, and have a third complementary custom close reason Wild kitty. The only obvious thing here is a close reason shouldn't be as broad as HW used to be. $\endgroup$
    – M.A.R.
    Commented Oct 1, 2016 at 18:01

3 Answers 3

Reset to default
7
$\begingroup$

My analysis of the stats

Nothing has really changed at all during the experiment.

My interpretation of the above

Most HW closures are done "correctly", i.e. for blatant homework dumps. I looked into the most recent questions closed as homework (20 or so, out of a total of 133; this information is available from the 10k tools). All of them are directly copy-pasted questions.

So, does that mean it was pointless?

Well, no.

The questions that this experiment sought to not close are, in fact, few and far between. I do recall, however, seeing one such question: Why do SN2 reactions of alkyl halides proceed differently with KCN and AgCN?.

In its original form, it was arguably lousy enough to warrant closure, with a total of zero effort shown. However, we left it open, and it got two answers from two of the most qualified organic chemists around here. (I'll just pretend the third answer doesn't exist.) getafix and myself polished it up a little bit, and now it is sitting at a pretty good question score of +12/0.

Conclusion

The effect has not been large. But just for the few cases like the one I linked above, I feel that it's worth making our experiment a permanent fixture.

Additional, tangential comments

For what it's worth, I did feel that in the past the homework question was getting unfairly slapped on some questions.

Sometimes, I felt like people thought: "hmm, I don't like this question, let's just flag/close it as HW". And the vagueness of the HW policy meant that it was very open to interpretation, and nearly everything could be closed under it. It became a catch-all reason to close any question that five people with 3k+ rep didn't like.

And I must admit, I was also guilty of misusing it in this way on some occasions. In hindsight, I think that if there are questions that we (as a community) dislike and want to close, then they should be closed. I am not trying to say that our dislike for them is unjustified. But, we do need to figure out the correct close reason for them. And that will be another discussion altogether, separate from this post.

In the meantime, let's keep the homework close reason limited to homework copy-pastes with no effort.

$\endgroup$
3
  • $\begingroup$ Criticisms: 1) The cyanide question was in fact from August so pre-experiment. 2) All the polishing happened in August. 3) On 29th August alone, it got 7 upvotes. Only one additional upvote came from the experiment month. I severly contest that that question falls under the experiment. $\endgroup$
    – Jan
    Commented Oct 2, 2016 at 18:33
  • 1
    $\begingroup$ shrugs I think it still counts. It was posted after the meta post and I would like to believe that while it was technically only supposed to start 1 Sept, people bore it in mind as they voted on that question. I cannot speak for everybody but without the experiment I know I would have VTCed that question as HW. $\endgroup$
    – orthocresol
    Commented Oct 2, 2016 at 18:57
  • $\begingroup$ @Jan I said the experiment would take effect beginning on Sep 1st, but people started it after the meta post. $\endgroup$
    – M.A.R.
    Commented Oct 2, 2016 at 20:36
4
$\begingroup$

The results are very clear. I'm rather surprised at how little everything changed. That indicates the experiment was a success, right? Well, what defines "success"?

We will only close blatantly obvious homework questions. We won't close easily Googleable, or rudimentary questions.

I am hesitant to pass judgment here. Certainly my opinion is not worth as much since I am not in a position to judge the content of most questions that would appear as obvious homework, such as those in . Rarely does this come up under proper or questions, for example, which makes sense considering the number of students and practitioners in each field.

Regarding the "closed as HW then edited", "closed as HW then reopened", and "closed as HW, edited, then reopened", I assume that the reason those values decreased from before is that they weren't caught in the "closed as HW" net to begin with. So while those decreased numbers are initially disappointing, perhaps it means those questions were finally given more than just some cursory attention long enough to flag them.

So, as a relative outsider, I would say the experiment was a good thing, as long as that means people are more willing to attempt working on improving a question enough to make it useful and/or interesting. I am not going to say it was a success or failure because

The questions that this experiment sought to not close are, in fact, few and far between.

My opinion is that this experiment inadvertently resulted in

people are more willing to attempt working on improving a question enough to make it useful and/or interesting

and that is where the focus should be. I would love for some proof to back up my claim.

$\endgroup$
4
$\begingroup$

To be honest, I’m not entirely sure about the outcome of the experiment. I have questions that I cannot answer easily with the stats presented:

  • How many new questions did we get (okay, that one can be answered; 90).

  • How many of those would we have closed in a different month?

  • How many of those that we would have closed received answers?

  • How many of the others received answers?

  • What is the average score of these questions’ answers?

  • How many questions receive answers at all?

  • Before experiment September, given a question that we would have closed how likely would it be for the user to stay (and gain rep)?

  • During experiment September, same question?

All in all, I didn’t see a massive influx of homework, good or bad; I’m with you there. However, I also didn’t see a massive improvement.

We might be asking ourselves the wrong question, to be honest. Instead of ‘Does it help not closing stuff as homework?’, what about ‘How can we make mediocre homework questions that have potential good?’ I feel, the experiment didn’t help us in that respect.

Also, as I commented I contest the cyanide selectivity question of being a benefitor of the experiment. Most of what was important — most of the votes, the good answers — happened before the experiment started in August. I would not have closed that question experiment or not. I’m not convinced it is homework. In fact, it is not tagged as such.

$\endgroup$
8
  • 1
    $\begingroup$ What if we weren't in for much of an improvement, stats-wise? I just managed to breathe in the meta, and the worry that we're not doing anything went away. We also got a step back, to gain some insight on how to deal with Close Shenanigans. I guess it was more of an excuse to rethink our closing ways, and get back to the spirit of editing more. $\endgroup$
    – M.A.R.
    Commented Oct 2, 2016 at 19:21
  • $\begingroup$ @Rubisco Tbh, except for the August influx (or so I perceived it), most of the time I’m bored in chemistry meta, so … $\endgroup$
    – Jan
    Commented Oct 2, 2016 at 19:22
  • $\begingroup$ Well, I guess it's really boring if you're literally living here, but take a look back a the previous eight or nine meta posts and see how focused it managed to became. We didn't give ourselves fake reasons to look active, and a site this size doesn't need more than a meta question every couple of days. (I keep thinking to myself let's wreck havoc before the election and the excitement comes, but meh . . . ) BTW, you running for mod election? The site might throw a German Mod Overflow exception. $\endgroup$
    – M.A.R.
    Commented Oct 2, 2016 at 19:26
  • $\begingroup$ @Rubisco I mentioned in chat a few times: I cannot afford to spend even more time on these sites which I would if I were mod. I’m in a very biased balance between grabbing Fanatic on a few sites where I don’t have it yet atm, but expect my activity to redecline after that. $\endgroup$
    – Jan
    Commented Oct 2, 2016 at 19:31
  • $\begingroup$ Oh. So dramatic. I'm shedding artificial tears. $\endgroup$
    – M.A.R.
    Commented Oct 2, 2016 at 19:35
  • 1
    $\begingroup$ In principle I agree with you. If there is any effect, it is extremely small, and to be honest no other examples come to mind. (I still think cyanide was an example, but let's agree to differ on that, because let's be honest, 1 question vs 0 questions - hardly any difference.) The matter is somewhat complicated because in my experience, homework questions are often not tagged with homework, and rarely is the tag edited on. Usually they have some rubbish like everyday-chemistry. For the most part these questions are just entirely ignored until they are closed. $\endgroup$
    – orthocresol
    Commented Oct 2, 2016 at 19:42
  • 1
    $\begingroup$ But in spirit I think that it was a good initiative. My perception was that HW was being misused and I think that even though perhaps we did not need an entire month's worth of trial run for it, this at least helped to refocus our standpoint on what HW really was. I am not trying to imply that you or anybody in particular misused it. Just a general feeling I got. And yes, we definitely need more discussion and more perspectives on the matter. $\endgroup$
    – orthocresol
    Commented Oct 2, 2016 at 19:44
  • $\begingroup$ @orthocresol (Just feeling the need to tell you that the first comment upvotes on those two were mine) $\endgroup$
    – Jan
    Commented Oct 2, 2016 at 20:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .