I think the FAQ already succinctly captures the main points; the problems likely are due to those questions where askers either did not read the FAQ or failed to apply them to their questions. For those people (and others watching), a much more visible signal is how questions are treated which fail to conform to these guidelines, and why.
First, some philosophical thoughts to motivate what follows. Clearly, SciComp should not and cannot be a replacement for a dedicated support forum, but it can be a very valuable complement. In particular, the main advantages of the former over the latter are
- Long-term discoverability: Solved problems are much easier to find, either with Google or a site-specific search. As a corollary, it's very convenient for pointing users with already solved questions to. (This is the reason developers want to have questions on a StackExchange site.)
- Scientific expertise: Ideally, users will not only get a solution to their problem, but also an explanation why the problem is solved this way (i.e., the why together with the how). (This should be the reason developers want to have questions on this StackExchange site. I want to make clear that I'm not saying the developers don't have the necessary expertise; just that there is more of it (and broader) here due to the larger audience, and there is already a culture of tackling questions in this context.)
If the question does not make use of (at least!) one of these advantages, it is either better posed somewhere else or posed incorrectly. (More on that below.)
Regarding how to treat specific questions, I would hope to be as inclusive as possible -- for the sake of the askers and of the site -- in the sense that no topic is a priori off-topic as long as sufficient effort was made to leverage the above advantages (i.e., without sacrificing quality). For example, this means the question should
- Make it about a problem, not your problem. Provide code, but make it minimal so that it contains what and only what is necessary for this question (as brief as possible, as specific as necessary).
- Provide (scientific) context. This helps the answerer select the most appropriate from multiple possible solutions, and also catch cases where something is not working because the whole approach is wrong. In particular, it should be made clear that a valid answer to the question "How do I do X in package Z?" would be "Don't do that in Z, use Y; here's how." (If this answer is not acceptable to the asker, it's a sign he's using the wrong forum.)
Clearly, the more software-related the question, the higher the bar on these points. Just as clearly, the burden is on the asker to convince the community in this respect. (I think that is already the practice in the community; there have been questions that were as software-specific as some closed questions but received up-votes and excellent answers because they were well-posed. This could be made explicit in the FAQ.)
There is also the issue of how to treat questions that failed to convince the community. The difficulty here is that closing a question can mean two different things: The question is not welcome here (as in "reject") or that it is not posed appropriately (as in "major revision") but will be reopened if suitably modified. In principle, the difference shows in the reason for closing, but that is not always so obvious, especially to a new user coming from outside the SE network (especially, support forums). We should be much more explicit about this difference, both in the FAQ and the comments. (And my personal feeling -- at least from the latter -- is now that we do too much of the former, and not enough of the latter, which lead to some frustration. We could also use a "minor revision" option: Down-vote and leave a comment asking for improvement, but (publicly) give a grace period (e.g., 24 hours) before voting to close.)
On a related note, if the asker is not willing (or, more importantly, does not possess the necessary scientific, programming or language skills) to make his question pass those bars but someone (e.g., developers) thinks the question should be answered for the sake of the (software-specific) community, that should be acceptable as long as the question is also edited accordingly by them. (Of course, that might send wrong signals; probably deserves a separate question on meta.)
To end with a concrete suggestion: In addition to the general guidelines, the FAQ could also link to example questions that are considered good and bad, respectively, together with comments on what makes them so:
- "The following question was up-voted and received several helpful answers because..."
- "The following question was closed as too localized because it only asked about a technical problem with the software..."
- "The following question was closed as off-topic because it asked about a specific implementation question without context" or "did not show that it wasn't already answered in the documentation", possibly adding "(but would have been on-topic if ...)"
(This would have the added benefit of making visible the results the asker can expect for his question -- carrot and stick, if you will.)