Let's talk about effort, shall we?

Question

I think the recent experiment of suspending the homework close reason has already produced a very valuable result. However, I really feared that this will happen, and I hope I can steer our thinking into a different direction with this post.
To partially quote Wildcat

And now I think, it is quite apparent that we need a close reason for zero-effort questions to simply save out time, because the absolute majority of questions that were previoulsy closed as homework ones are now closed with this very custom reason (zero-effort).

I would like to back that up with some numbers (10k+ tools). In the past seven days there were 213 questions asked, of which 70 were closed (33%). Of these 22 were closed with custom reasons (10%). From the summary of the close reasons, they are all a variation of "no/any effort shown".

While I agree with most of you that we need a system to protect our site from dilution the great content with questions of the zero-effort kind, I don't think that effort itself is a good metric. Just as Wildcat says, zero-effort is easy to identify and closing them on sight would be easy. But I think it wouldn't be enough to sort out other questions, too. I am of the opinion, that just showing effort doesn't necessarily ensure that it is a good, or even useful question.

I personally would like to have conceptual questions and answers, that in principle can be adapted to various situations. One of these approaches certainly is the tiny list of duplicates. If such questions include very concrete examples, I think that's fine (if not, then it would probably not be off-topic).

Please note, that this is an attempt of replacing the current homework policy to a more generalised and more objective version.

As a (very first) draft for discussion, I would like to propose a custom close reason of the following form:

At Chemistry.se we aim for questions that are helpful to as many visitors as possible. Therefore you should identify the specific chemistry concepts and the problems you are facing with it. For more information how to improve your question see: How can I make my question conceptual?

(demonstration link only; 400 total characters allowed, 334 used)

The above certainly is far, farⁿ from perfect, but I hope it will provide a new angle on the whole closing issue. The experiment will continue for another two weeks (roundabout); please be vocal about your opinion. And if you are thinking about commenting now, maybe write an answer instead.

Answer 1

My own opinion is broadly the same - I think that "effort" is a poor criterion for judging question quality. I have discussed this before. In my opinion, the most important bit is this:

In one of the worst case scenarios, OP's "efforts" are way off the mark and simply become useless noise for future readers. In another bad scenario, OP's efforts are entirely correct and the answerer is reduced to saying "yes, you're right".

So, to begin with, I don't think that having effort is necessarily a good thing. Neither is lacking effort necessarily a bad thing, as will be discussed below. But, not only is it a bad criterion; I'd argue we don't even follow it.

---

Double standards

In the post linked above, I also wrote

The HW policy is subjective and open to interpretation. There is occasional "misuse" of the HW close reason - not exactly abuse, but different people have different quality standards. Furthermore, what counts as "effort" is unclear. If somebody simply adds a line at the bottom of their question "I tried to look in my textbook, but couldn't find anything", does that count as effort?

This subjectiveness, perhaps, leads to what I would perceive as some degree of a double standard. Basic questions with zero-effort get closed, but advanced questions with zero-effort don't get closed: see Exhibit A, for example.*

In order to further test my perception of a double standard, I created a new account and asked a graduate/advanced undergraduate-level organic chemistry question with next to no effort: Exhibit B.† In 24 hours since it was posted, it got six upvotes and one favourite. No close votes, and no comments saying to show effort. Clearly, people think it is a good and useful question for the site. But other, less advanced, questions have been closed even with more "effort" given.

---

Aims of the site

At this moment I'd also like to remind everybody of what Stack Exchange aims to be. This is quoted from the very first part of the tour:

With your help, we're working together to build a library of detailed answers to every question about chemistry.

I don't think that this library can only be built if people show their effort. I think the questions that I linked to above are examples which show that so-called no-effort questions can indeed be beneficial. Marko's question (Exhibit A) was pretty interesting and it received a good answer which it fully deserved. I hope that nobody disagrees with me when I say that the site benefitted from that Q+A. I am glad that the community recognises the worth of such questions, but our behaviour is clearly not congruent with the supposed "no effort" requirement.

Even my sock's question is fairly decent, in terms of its value to the site. It could certainly have a bit more context, and I will edit it in in due time, after we are done with this discussion. (I purposely made it as low-quality as I could).

The point still stands, though; I strongly disagree that effort is a good criterion for judging the quality of a question.

---

$\Large \textbf{What should we do, then?}$ Well, my 2c:

1. Get rid of the effort criterion for closing questions.

2. Replace it with a conceptual criterion. This means that even the worst homework questions will not be closed as "off-topic", as long as they clearly identify the concept/issue that is being asked about. However, they may still be closed as duplicates of older questions.

3. Focus on editing and improving questions, if possible. If you think that a not-so-good question has an interesting concept behind it, and can potentially receive a good answer, then by all means, edit it so that it looks like a good question.^‡

4. Make some canonical Q&As in order to address common homework-type queries - this should be a long-term project; for obvious reasons, this cannot happen overnight.

Martin and I think that a conceptual criterion is the best kind, and AFAIK Physics.SE adopts this viewpoint too. Maybe the community thinks that a conceptual criterion isn't the best idea, which is fine - it's really okay (and it's great) to disagree - but we would really like to hear some alternatives.

Maybe you simply want to get rid of all high school chemistry questions. To be honest, I get this feeling sometimes, but it's obviously an untenable position. I feel that the best way to deal with this is to efficiently close as duplicates (there are only so many high school concepts one can ask about!), and get users to ask more high-level questions, as this recent meta post by NotEvans suggests.

Or maybe the idea is that we don't want to be doing people's homework for them. I can understand why you may have this sentiment, but in this case I'd like to suggest that we stop viewing questions as homework or not - remember this isn't a homework help site and also it isn't a site where homework is forbidden - and judge it solely based on whether you think it could possibly receive an interesting answer which would contribute to the site as a whole.

---

Footnotes

* Before people accuse me of not unilaterally closing that as a moderator: you know that I already do not think that effort is a good criterion. Furthermore, we cannot always be mod-closing everything; otherwise, there will never be any chance for an organic shift in community consensus. If I had closed this, then I would not have this example to share with you today.

† Obviously, I haven't done anything with my sock that my own account could not do. I also did not upvote that question with my main. It is locked for the time being, only because I want to avoid the so-called "meta effect". In due time this account will be merged with my main account.

^‡ In response to hBy2Py's comment about when we should feel free to edit a post, the answer is: always, as long as it is done in good faith (i.e. not deliberately distorting OP's query, for example). Quoting from Section 3 of the Terms of Service:

You agree that all Subscriber Content that You contribute to the Network is perpetually and irrevocably licensed to Stack Exchange [...]

Answer 2

This very much isn’t a fully thought out answer, but it was a lot longer than I could fit into a comment. Just my two english cent. . .

Effort as a metric for closure

Broadly speaking, I agree that effort (or lack thereof) in itself isn’t a particularly great metric for whether or not a question deserves help.

When we’ve been discussing this, the tex.SE approach (and other code based sites) springs to mind. Users shouldn’t necessarily need to attempt to answer the question (as ortho points out theres really no point in someone making an awful attempt at some derivation), but there should be a minimum standard of effort involved.

On tex.SE, they have the idea of a ‘minimum working example’, this obviously doesn’t apply here, but there could be a minimum standard we’re willing to accept. If you think about the worst of the homework questions, they’re the wonky screenshots of some obscure, poorly worded, indian chemistry exam.

On chem.SE, our ‘MWE’ could therefore look something like: a fully typed out question (not just a picture), some context about the level of the question, and an indication of why the question is difficult to them (why isn’t it like the 400 other stoichiometry problems you’ve gone through).

Conceptuality as a metric

Whilst the idea of questions being ‘conceptual’ is a good one (i.e. questions are a lot more interesting when they’re a little more abstract rather than heavily set in a particular set of numbers), and one I agree fairly strongly with. I’m mindful of the fact that actually, chem.SE isn’t that busy relative to many other sites on the network.

The issue I take with having a set of stock canonical answers is that the old answers aren’t necessarily the best, and people are fairly unwilling to re-answer old questions that already have accepted answers/lots of up-votes (it seems counter productive).

[As an extreme example of this, I could in theory sit here and write a question and associated answer related to every section of Atkins’ physical chemistry, and we’d never need to actually answer another question again].

Answer 3

If we would like to move away from the "effort" angle, then the tooltip that pops up when hovering over a downvote button should be changed.

Answer 4

10% of posted questions is way to much.

Yet I don't think a closing-policy would do any good. I think there are two kinds of bad questions. The first kind are questions that are bad for different reasons. From what I see, often there the asker is willing to edit their question so it gets better. I don't think we need to do anything about those questions.

The second kind, the zero-effort questions are a problem.

The biggest problem with the whole zero-effort situation is that if you ask the OP to add effort of his own, you never hear back from them as they abandon the question. These guys write that question probably in chem.SE, but also on quora, gutefrage.net etc pp.. Then they just wait where they find their answer and then they're gone (Are there statistics available with 1-rep accounts with 1 question that is closed?).

So I think that we will always have those questions, and the quality of them will not change.

I can think of these options:

1. Try to only advertise this site to "the right audience" (very difficult and kind of elitist/arrogant)
2. Close-and-forget the zero-effort questions
3. Implement hurdles for asking questions (very bad idea imho)
4. Stick with trying to be nice and get the OP to add effort and get more frustrated every day
5. Try your model

I don't think that posting a "identify the concept behind your question and conceptualize it" comment would change anything, since the zero-effort-zombies don't read it, and they also don't read it on other questions since they are hunting for a quick no-brainer answer to their homework/whatever.

I personally like your proposition, and I would be all up for trying it, but I honestly don't have a lot of hope.

I don't think getting a new closing-policy will solve the problem, but only mask it. If we want to make this site better, we need to be active, not reactive. A proposition like @ortho's in the comments might be a better way, although it would result in probably a lot of work for a couple determined people.

Answer 5

What is a conceptual question?

That's a serious question, and one I think gets to the heart of the issue with redoing the homework policy as a "conceptual question" policy.

I'll be honest and say I'm not really sure what Martin and orthocresol mean when they say "conceptual question". Yes, we can probably find some clear examples on both sides where a question is clearly "conceptual" and where the question is clearly "non-conceptual", but it's that middle region - that intermediate gray area - that causes issues.

If all we cared about were the clear examples, the current homework policy would be fine - yes, the fuzzy screenshots of poorly worded textbook problems are bad, and the advanced stuff is fine. But the current homework policy is dissatisfactory because of the gray area. It's those questions on the edge we're not quite handling correctly that are at issue.

That's my unease with the change: it's all well and good to redefine the criteria to say "conceptual: yes; non-conceptual: no", but if the definition of conceptual is "I know it when I see it", it's going to fall apart when the rank and file (who apparently like the current "effort" standard) disagree with Martin and orthocresol (and each other) as to what counts as "conceptual" or not.

Not knowing what "conceptual" means here (or more particularly, what makes a question "non-conceptual"), I can't necessarily agree that switching to a "conceptual" standard is going to adequately replace the (desired) functionality of the current homework close. It might have potential, but there's still details that need to get worked out.