18
$\begingroup$

I think the recent experiment of suspending the homework close reason has already produced a very valuable result. However, I really feared that this will happen, and I hope I can steer our thinking into a different direction with this post.
To partially quote Wildcat

And now I think, it is quite apparent that we need a close reason for zero-effort questions to simply save out time, because the absolute majority of questions that were previoulsy closed as homework ones are now closed with this very custom reason (zero-effort).

I would like to back that up with some numbers (10k+ tools). In the past seven days there were 213 questions asked, of which 70 were closed (33%). Of these 22 were closed with custom reasons (10%). From the summary of the close reasons, they are all a variation of "no/any effort shown".

While I agree with most of you that we need a system to protect our site from dilution the great content with questions of the zero-effort kind, I don't think that effort itself is a good metric. Just as Wildcat says, zero-effort is easy to identify and closing them on sight would be easy. But I think it wouldn't be enough to sort out other questions, too. I am of the opinion, that just showing effort doesn't necessarily ensure that it is a good, or even useful question.

I personally would like to have conceptual questions and answers, that in principle can be adapted to various situations. One of these approaches certainly is the tiny list of duplicates. If such questions include very concrete examples, I think that's fine (if not, then it would probably not be off-topic).

Please note, that this is an attempt of replacing the current homework policy to a more generalised and more objective version.

As a (very first) draft for discussion, I would like to propose a custom close reason of the following form:

At Chemistry.se we aim for questions that are helpful to as many visitors as possible. Therefore you should identify the specific chemistry concepts and the problems you are facing with it. For more information how to improve your question see: How can I make my question conceptual?

(demonstration link only; 400 total characters allowed, 334 used)

The above certainly is far, farn from perfect, but I hope it will provide a new angle on the whole closing issue. The experiment will continue for another two weeks (roundabout); please be vocal about your opinion. And if you are thinking about commenting now, maybe write an answer instead.

$\endgroup$
  • 3
    $\begingroup$ Personally, I'm skeptical of the rationale "no effort shown" as a close reason. I've found that people can rather be picky and capricious about how much effort counts as "enough". I'd be interested in seeing just what counts as "no effort" to the people casting close votes. (Though I understand if you don't want to link the questions in questions specifically.) I'm not saying some variation of "no effort" wouldn't be valid, but I'm wondering if labeling the rationale as "no effort" is an oversimplification/mischaracterization of what's really going on. $\endgroup$ – R.M. Jun 16 '17 at 16:46
  • $\begingroup$ @R.M. The current homework policy is based on the effort argument. With this post I'm trying to change that. The reason why I'm not linking to some zero-effort-posts is that I don't think they are relevant for this kind of change. $\endgroup$ – Martin - マーチン Jun 16 '17 at 17:14
  • 3
    $\begingroup$ Okay, but the experiment shows that people really like the "no effort" rationale. If you're arguing it's a poor and unclear rationale, I agree with you, but others apparently do not. The reason I suggested looking at what people think are "no effort" posts is that I'm guessing there's an XY problem going on here. There's something that people don't like about the questions, and "no effort" is just a convenient label to use. If we could look closer at the posts (and the expressed rationales), we might start to tease apart what exactly people don't like and how we should phrase the replacement. $\endgroup$ – R.M. Jun 16 '17 at 18:18
  • 1
    $\begingroup$ @R.M. We don't like anything that someone with a Bachelor's in chemistry could answer. On the other side of the same coin, any question asked by a veteran is automatically a good one. We might also not like everyday-chemistry, but I'm not as sure about that. $\endgroup$ – pentavalentcarbon Jun 20 '17 at 15:20
10
$\begingroup$

If we would like to move away from the "effort" angle, then the tooltip that pops up when hovering over a downvote button should be changed.

enter image description here

$\endgroup$
  • 3
    $\begingroup$ (down)-voting is not the same as closing as off-topic. The former is your personal choice how helpful the question is, the latter should determine which kind of questions are within our scope, and I would like it to be a bit more objective. $\endgroup$ – Martin - マーチン Jun 14 '17 at 13:20
  • 3
    $\begingroup$ @Martin-マーチン I know. What I want to say is that if voting is a personal preference, then the text should be removed entirely, since right now it sends the wrong message about why one should vote. $\endgroup$ – pentavalentcarbon Jun 14 '17 at 17:34
  • $\begingroup$ While I don't really understand your point, this would need to be discussed on a network wide basis at Meta Stack Exchange, as the text is the same across all networks.There is nothing we can do about it from here. $\endgroup$ – Martin - マーチン Jun 15 '17 at 4:02
7
$\begingroup$

My own opinion is broadly the same - I think that "effort" is a poor criterion for judging question quality. I have discussed this before. In my opinion, the most important bit is this:

In one of the worst case scenarios, OP's "efforts" are way off the mark and simply become useless noise for future readers. In another bad scenario, OP's efforts are entirely correct and the answerer is reduced to saying "yes, you're right".

So, to begin with, I don't think that having effort is necessarily a good thing. Neither is lacking effort necessarily a bad thing, as will be discussed below. But, not only is it a bad criterion; I'd argue we don't even follow it.


Double standards

In the post linked above, I also wrote

The HW policy is subjective and open to interpretation. There is occasional "misuse" of the HW close reason - not exactly abuse, but different people have different quality standards. Furthermore, what counts as "effort" is unclear. If somebody simply adds a line at the bottom of their question "I tried to look in my textbook, but couldn't find anything", does that count as effort?

This subjectiveness, perhaps, leads to what I would perceive as some degree of a double standard. Basic questions with zero-effort get closed, but advanced questions with zero-effort don't get closed: see Exhibit A, for example.*

In order to further test my perception of a double standard, I created a new account and asked a graduate/advanced undergraduate-level organic chemistry question with next to no effort: Exhibit B.† In 24 hours since it was posted, it got six upvotes and one favourite. No close votes, and no comments saying to show effort. Clearly, people think it is a good and useful question for the site. But other, less advanced, questions have been closed even with more "effort" given.


Aims of the site

At this moment I'd also like to remind everybody of what Stack Exchange aims to be. This is quoted from the very first part of the tour:

With your help, we're working together to build a library of detailed answers to every question about chemistry.

I don't think that this library can only be built if people show their effort. I think the questions that I linked to above are examples which show that so-called no-effort questions can indeed be beneficial. Marko's question (Exhibit A) was pretty interesting and it received a good answer which it fully deserved. I hope that nobody disagrees with me when I say that the site benefitted from that Q+A. I am glad that the community recognises the worth of such questions, but our behaviour is clearly not congruent with the supposed "no effort" requirement.

Even my sock's question is fairly decent, in terms of its value to the site. It could certainly have a bit more context, and I will edit it in in due time, after we are done with this discussion. (I purposely made it as low-quality as I could).

The point still stands, though; I strongly disagree that effort is a good criterion for judging the quality of a question.


$\Large \textbf{What should we do, then?}$ Well, my 2c:

  1. Get rid of the effort criterion for closing questions.

  2. Replace it with a conceptual criterion. This means that even the worst homework questions will not be closed as "off-topic", as long as they clearly identify the concept/issue that is being asked about. However, they may still be closed as duplicates of older questions.

  3. Focus on editing and improving questions, if possible. If you think that a not-so-good question has an interesting concept behind it, and can potentially receive a good answer, then by all means, edit it so that it looks like a good question.

  4. Make some canonical Q&As in order to address common homework-type queries - this should be a long-term project; for obvious reasons, this cannot happen overnight.

Martin and I think that a conceptual criterion is the best kind, and AFAIK Physics.SE adopts this viewpoint too. Maybe the community thinks that a conceptual criterion isn't the best idea, which is fine - it's really okay (and it's great) to disagree - but we would really like to hear some alternatives.

Maybe you simply want to get rid of all high school chemistry questions. To be honest, I get this feeling sometimes, but it's obviously an untenable position. I feel that the best way to deal with this is to efficiently close as duplicates (there are only so many high school concepts one can ask about!), and get users to ask more high-level questions, as this recent meta post by NotEvans suggests.

Or maybe the idea is that we don't want to be doing people's homework for them. I can understand why you may have this sentiment, but in this case I'd like to suggest that we stop viewing questions as homework or not - remember this isn't a homework help site and also it isn't a site where homework is forbidden - and judge it solely based on whether you think it could possibly receive an interesting answer which would contribute to the site as a whole.


Footnotes

* Before people accuse me of not unilaterally closing that as a moderator: you know that I already do not think that effort is a good criterion. Furthermore, we cannot always be mod-closing everything; otherwise, there will never be any chance for an organic shift in community consensus. If I had closed this, then I would not have this example to share with you today.

† Obviously, I haven't done anything with my sock that my own account could not do. I also did not upvote that question with my main. It is locked for the time being, only because I want to avoid the so-called "meta effect". In due time this account will be merged with my main account.

In response to hBy2Py's comment about when we should feel free to edit a post, the answer is: always, as long as it is done in good faith (i.e. not deliberately distorting OP's query, for example). Quoting from Section 3 of the Terms of Service:

You agree that all Subscriber Content that You contribute to the Network is perpetually and irrevocably licensed to Stack Exchange [...]

$\endgroup$
  • $\begingroup$ Naughty Victor! In all seriousness though, the more advanced the question is the harder it is to simply google the answer, assuming you even know what you're searching for. Exhibit B vs Exhibit X (generic stoichiometry question) are very different... kettles of fish, surely $\endgroup$ – NotEvans. Jun 19 '17 at 18:07
  • $\begingroup$ Great suggestions @orthocresol. Your idea of replacing effort with concept seems very nice. I had once mention this in my post: Should we completely ban all homework questions?, so I feel that this is a great way to approach things. $\endgroup$ – Pritt Balagopal Jul 12 '17 at 13:01
7
$\begingroup$

This very much isn’t a fully thought out answer, but it was a lot longer than I could fit into a comment. Just my two english cent. . .

Effort as a metric for closure

Broadly speaking, I agree that effort (or lack thereof) in itself isn’t a particularly great metric for whether or not a question deserves help.

When we’ve been discussing this, the tex.SE approach (and other code based sites) springs to mind. Users shouldn’t necessarily need to attempt to answer the question (as ortho points out theres really no point in someone making an awful attempt at some derivation), but there should be a minimum standard of effort involved.

On tex.SE, they have the idea of a ‘minimum working example’, this obviously doesn’t apply here, but there could be a minimum standard we’re willing to accept. If you think about the worst of the homework questions, they’re the wonky screenshots of some obscure, poorly worded, indian chemistry exam.

On chem.SE, our ‘MWE’ could therefore look something like: a fully typed out question (not just a picture), some context about the level of the question, and an indication of why the question is difficult to them (why isn’t it like the 400 other stoichiometry problems you’ve gone through).

Conceptuality as a metric

Whilst the idea of questions being ‘conceptual’ is a good one (i.e. questions are a lot more interesting when they’re a little more abstract rather than heavily set in a particular set of numbers), and one I agree fairly strongly with. I’m mindful of the fact that actually, chem.SE isn’t that busy relative to many other sites on the network.

The issue I take with having a set of stock canonical answers is that the old answers aren’t necessarily the best, and people are fairly unwilling to re-answer old questions that already have accepted answers/lots of up-votes (it seems counter productive).

[As an extreme example of this, I could in theory sit here and write a question and associated answer related to every section of Atkins’ physical chemistry, and we’d never need to actually answer another question again].

$\endgroup$
  • $\begingroup$ The next steps would be to define 1. what constitutes a good conceptual question (what's the "cutoff"), and 2. what less-conceptual questions are acceptable? Otherwise, the same trap of how to define homework will appear again. $\endgroup$ – pentavalentcarbon Jun 20 '17 at 15:23
  • 1
    $\begingroup$ @pentavalentcarbon I don't think that this is the same trap, because we are not talking about closing conceptual questions, we are talking about closing non-conceptual questions. As such it is easy to deal with "bad" and "good" conceptual questions: Vote on them. $\endgroup$ – Martin - マーチン Jun 21 '17 at 6:53
  • $\begingroup$ (+1) though "indian chemistry exam." really are there so many of them? o.O $\endgroup$ – Gaurang Tandon Apr 18 '18 at 9:00
4
$\begingroup$

10% of posted questions is way to much.

Yet I don't think a closing-policy would do any good. I think there are two kinds of bad questions. The first kind are questions that are bad for different reasons. From what I see, often there the asker is willing to edit their question so it gets better. I don't think we need to do anything about those questions.

The second kind, the zero-effort questions are a problem.

The biggest problem with the whole zero-effort situation is that if you ask the OP to add effort of his own, you never hear back from them as they abandon the question. These guys write that question probably in chem.SE, but also on quora, gutefrage.net etc pp.. Then they just wait where they find their answer and then they're gone (Are there statistics available with 1-rep accounts with 1 question that is closed?).

So I think that we will always have those questions, and the quality of them will not change.

I can think of these options:

  1. Try to only advertise this site to "the right audience" (very difficult and kind of elitist/arrogant)
  2. Close-and-forget the zero-effort questions
  3. Implement hurdles for asking questions (very bad idea imho)
  4. Stick with trying to be nice and get the OP to add effort and get more frustrated every day
  5. Try your model

I don't think that posting a "identify the concept behind your question and conceptualize it" comment would change anything, since the zero-effort-zombies don't read it, and they also don't read it on other questions since they are hunting for a quick no-brainer answer to their homework/whatever.

I personally like your proposition, and I would be all up for trying it, but I honestly don't have a lot of hope.

I don't think getting a new closing-policy will solve the problem, but only mask it. If we want to make this site better, we need to be active, not reactive. A proposition like @ortho's in the comments might be a better way, although it would result in probably a lot of work for a couple determined people.

$\endgroup$
  • 1
    $\begingroup$ To be clear, I don't think we are trying to "solve the homework problem" in the sense of getting rid of HW. As you said, this will always happen, no matter what we do. I feel that it is more of an issue of what we do with it once it lands on our doorsteps. $\endgroup$ – orthocresol Jun 13 '17 at 10:48
  • 4
    $\begingroup$ Yeah I know. I don't think generally hw-questions should be banned either, but Martin proposes a way to try to get people to write more conceptualized questions. I think you cannot achieve this with trying to communicate with the zero-effort-zombies since they are neither reading that nor are they the target audience for those concept-questions. $\endgroup$ – Fl.pf. Jun 13 '17 at 10:53
  • 3
    $\begingroup$ Ah, I see what you mean. I agree, it is difficult to get these people to edit their questions. Back when I left comments on nearly every homework post telling people to include "effort", the success rate was <20%, and that's probably quite a generous estimate. However, we can always edit it for them into something more conceptual ourselves, especially if the question holds some promise. $\endgroup$ – orthocresol Jun 13 '17 at 10:54
  • $\begingroup$ @orthocresol That seems to be a good idea! The question is good, so we edit it to make it better and it gets an answer! $\endgroup$ – Pritt Balagopal Jun 13 '17 at 14:19
  • 1
    $\begingroup$ A policy (separate from this discussion) on when the community should feel comfortable "hijacking" a low-quality question to improve it might be useful. E.g.: Any question with 0 score or less that has been untouched by OP for more than 30 days can be freely edited to improve it (say, to make it more consistent with an excellent answer posted). $\endgroup$ – hBy2Py Jun 14 '17 at 4:06
  • $\begingroup$ Are there statistics available with 1-rep accounts with 1 question that is closed? I put together a SEDE query to investigate this. If my queries and interpretation are correct, about 24% of 1-rep users created more than 30 days ago with exactly 1 posted question have had that question closed/deleted (268 out of 1114). $\endgroup$ – hBy2Py Jun 14 '17 at 4:50
  • $\begingroup$ (I think this ^^ might exclude users whose accounts have been deleted, though.) $\endgroup$ – hBy2Py Jun 14 '17 at 5:00
  • $\begingroup$ @hBy2Py A question with a score lower than 0 has an expiration date of 30 days. It'll be deleted by scripts. $\endgroup$ – Martin - マーチン Jun 14 '17 at 5:24
  • 3
    $\begingroup$ Just for the record: As far as I understood your answer, I disagree with it. I do not think that you can safe all 33% (or 50% if we take the old numbers) by editing them into a good (whatever that may mean) question, and I do not think that it would be beneficial in the first place. Re: the asker is willing to edit their question - that barely ever happens. In the last 14 days it was twice. If we do not close questions, we have no quality control. $\endgroup$ – Martin - マーチン Jun 14 '17 at 5:37
  • $\begingroup$ @Martin-マーチン Right, that's why I drew from PostsWithDeleted and checked for questions with either .ClosedDate or .DeletionDate as NOT NULL. $\endgroup$ – hBy2Py Jun 14 '17 at 10:53
3
$\begingroup$

What is a conceptual question?

That's a serious question, and one I think gets to the heart of the issue with redoing the homework policy as a "conceptual question" policy.

I'll be honest and say I'm not really sure what Martin and orthocresol mean when they say "conceptual question". Yes, we can probably find some clear examples on both sides where a question is clearly "conceptual" and where the question is clearly "non-conceptual", but it's that middle region - that intermediate gray area - that causes issues.

If all we cared about were the clear examples, the current homework policy would be fine - yes, the fuzzy screenshots of poorly worded textbook problems are bad, and the advanced stuff is fine. But the current homework policy is dissatisfactory because of the gray area. It's those questions on the edge we're not quite handling correctly that are at issue.

That's my unease with the change: it's all well and good to redefine the criteria to say "conceptual: yes; non-conceptual: no", but if the definition of conceptual is "I know it when I see it", it's going to fall apart when the rank and file (who apparently like the current "effort" standard) disagree with Martin and orthocresol (and each other) as to what counts as "conceptual" or not.

Not knowing what "conceptual" means here (or more particularly, what makes a question "non-conceptual"), I can't necessarily agree that switching to a "conceptual" standard is going to adequately replace the (desired) functionality of the current homework close. It might have potential, but there's still details that need to get worked out.

$\endgroup$
  • 1
    $\begingroup$ Honestly, I agree, and I'm aware that it's not a perfect solution. It will still be somewhat subjective, and I think that that is why we still need 5 votes to close a question. // You could call my thinking idealistic, but I think that in principle, "conceptual" makes much more sense than "has effort". The implementation of both might not be ideal. However, I really just want to push our closure policy in a different direction. We have been sitting on "homework/no effort" for so long, and gotten so comfortable with it, perhaps a little too comfortable. $\endgroup$ – orthocresol Jun 22 '17 at 15:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .