Should we close questions just because their answers may be lists?

Question

Several questions have recently been closed because they are so-called "list questions" and hence "not a good fit" for Stackexchange. There seem to be a small number of vocal and active people behind this. I'd like to know if the general community is in agreement with it.

The latest example is https://scicomp.stackexchange.com/questions/1307/examples-of-physicists-and-astronomers-real-computer-science-usage.

I think that the rationale being used here is inconsistent. If https://scicomp.stackexchange.com/questions/1307/examples-of-physicists-and-astronomers-real-computer-science-usage is a "list question", then so are the following:

• Which linear algebra texts should I read before learning numerical linear algebra?
• What are some applications which require interval arithmetic?
• What kinds of problems lend themselves well to GPU computing?
• Databases of results for numerical codes
• Recommendations for a usable, fast C++ matrix library?
• Algebraic Multigrid Code
• Are open-source codes available to study protein folding?
• Adding deliberate imperfection to RNG output - toolkits?
• Mathematical Libraries for OpenCL?
• Sparse hermitian eigensystems: are there better techniques than Arpack or TRLan?
• Where can I find a database of simple chemical structures in XYZ format?
• Texture analysis methods modern survey paper
• Is there a high quality nonlinear programming solver for Python?
• Are there any open source inverse-based multilevel ILU implementations?
• Is there any open-source or easy-to-access software that can simplify algebraic expressions like $x^{2}+2x+3, x=\sqrt{2}t-1$?
• What guidelines should I use when searching for good preconditioning methods for a specific problem?
• https://scicomp.stackexchange.com/questions/1120/is-there-a-good-site-for-holding-online-discussions-of-scientific-papers
• Where do dense matrices occur?
• What simple methods are there for adaptively sampling a 2D function?
• Venues for publishing papers that emphasize software

I could go on. Why haven't these all been closed?

Furthermore, the criteria apparently being applied on our site are not consistent with other SE sites. Take a look at this question on Stackoverflow:

https://stackoverflow.com/questions/1981526/what-should-every-developer-know-about-databases

and compare with

What core skills should every computational scientist have?

They are practically identical except for subject matter, yet the latter was closed while the former is open with 128 upvotes!

Answer 1

The guidelines that work on one SE site don't necessarily work on another, so just because other sites discourage list questions, it doesn't mean we necessarily need to do the same here.

Take Stack Overflow, for instance. SO is about simple coding questions, so it's expected that whatever someone asks, there will be some (probably fairly standard) algorithm to do it, or some library function that can be offered up as an answer, perhaps with a short usage example. On the other hand, this site is more about library recommendation questions, because the kind of things that one needs to accomplish as a computational scientist usually can't be done with 10 lines of code from the standard library of your language of choice. You usually need to find something that's already been written and packaged up. In other words, I see a lot of the value of this site coming from its ability to direct people to libraries of pre-written code that they might not have known about otherwise. Those kinds of questions inherently tend to attract more and more varied answers than the ones on simple coding, because there is much more freedom in how one can implement, say, a Monte Carlo integrator than in how one can implement (or even just use) quicksort.

But beyond that, I think even the definition of list questions has gotten a bit out of hand. After all, any question could be construed as being a list question since it will necessarily attract a list of answers (except for the occasional question that actually does have exactly one correct answer). So perhaps "list questions" isn't even the best term to use. As I see it, the thing that makes the formerly-known-as-list questions bad is that there is no metric for evaluating the answers other than popularity. This kind of goes along with what Geoff wrote about good questions having wrong answers. Stack Exchange's niche is that it provides a very potent system for differentiating good (correct) answers from bad (wrong) ones. If people ask questions which are simply looking to collect a set of options, but where there is little sense in labeling one option as better than another, it's an abuse of the system. Those kinds of questions are better suited to a forum or poll site. I think that when the SE FAQ says "practical, answerable questions," this is what it's getting at.

Also, consider what Kev wrote in the passage Geoff quoted, specifically this paragraph:

The question is forgotten about and 5-10 more "Best sites" questions are asked, no-one can remember the canonical question and these new questions either get closed as "not constructive" or slip through the radar being upvoted and answered by users who should know better.

Since a good question inherently contains some criteria by which answers can be judged as good or bad, if the question is asked multiple times, you'd expect to get the same answer(s) rising to the top pretty much every time. On the other hand, the rating of answers to a "list question" is pretty arbitrary, depending on what people reading the answers tend to like. So if it's asked multiple times, you might get a whole different set answers from one instance to the next, and that sows confusion.

Basically, I want to make three points:

1. This site may be able to get away with slightly more list-like questions than other SE sites
2. I don't think all the questions you've mentioned here are actually list questions, or at least they're not all bad for the site
3. Whether a question is answerable, in particular whether it differentiates good from bad answers in a reliable manner, is a better criterion for identifying good vs. bad questions than is just classifying them as list or non-list.

Answer 2

The Short Answer

I think the short(er) answer to that question is that half of those questions were put up before we even discovered that we were writing questions that may not be well-suited to the Stack Exchange format. Even Stack Overflow has these types of questions when they started out, but from what I understand, it got to be a problem because curating them became unmanageable. Now that the issue of "list questions" has been broached, it's becoming a topic of discussion while we figure out what that phrase even means.

Voting down and closing list-type questions makes the site cleaner

Before I even get to the detailed answer (by which point people will TL;DR), one problem with list-type questions is that they're going to be very popular. By their very nature, they make it very easy to contribute. You're going to get good answers, and you're going to get crap. In addition to considering the community's input, really take a good look at decide whether you and the rest of the moderators, plus all of the other high rep users, are going to want to maintain the answers to these types of questions by cleaning out all of the crap, deleting old links, culling duplicates, getting people to update their answers, and so on.

A good example of the life cycle of a list-type questions is Kev's answer in "Exceptional cases for list questions" on Meta Stack Overflow:

Here's what happens:

The first 6 to 10 answers start out mostly ok because they're visible on the first page of the question and everyone can see if they're about to post a dupe link or not.

The next 11 - 20 answers provide maybe one or two new additions but because no-one can be bothered reading the first 6 to 10 answers duplicates creep in

After a month or so links start to rot on say 30% of the answers. Most of the folk who linked to dead destinations never respond to comments to fix the links. Mods are flagged to clean up these dead answers.

When the question is flagged and closed there's a big fuss made over it on the meta site. Moderators and sensible users, who see these questions for what they are, spend valuable time defending the closure and re-explaining the folly of such posts.

Peer pressure sees the question re-opened because of faithful promises to keep the question well maintained and as a canonical reference. The reality is that this doesn't happen, the question accretes more duplicate links and more links rot over time. Users and mods spend more time flagging, deleting, chasing to have links fixed, and what have you.

The question is forgotten about and 5-10 more "Best sites" questions are asked, no-one can remember the canonical question and these new questions either get closed as "not constructive" or slip through the radar being upvoted and answered by users who should know better.

And on and on and on it goes...until eventually after about two years the question is finally closed, locked and then thankfully dispatched, and you know what? Nobody cared or noticed.

What a huge waste of time.

These questions appear to be popular because they're easy to answer, everyone has an opinion and a favourite they want to show off.

As the months and years pass they degrade into sad, forgotten and unloved content just cluttering up search engine results with their uselessness.

I mean, do you, as a potential maintainer of the site, really want to deal with that? I've badgered people already about answers to:

• Open-access journals in Computational Science
• Which journals should I read to keep up on advances in solving PDEs numerically?

and while I don't expect people to change their answers right away, I can see that checking periodically on each question like this might become a losing battle.

Luckily, we're early on in the life cycle of an SE site, so we have the chance to deal with this sort of hassle before it gets too too ugly.

The Long Answer

Now, in the spirit of The Great Question Deletion Audit of 2012, let's go down the list:

• Which linear algebra texts should I read before learning numerical linear algebra?: Keep. There's a small number of defensible answers here by experts. Could it be posed better so that it looks less like a list question? Yeah, probably. I'm not a fan of the answer that data dumped a list without any explanation. The key criteria I'd use here is: there are definitely wrong answers here, in that there are a small number of highly recommended linear algebra books, and to recommend anything other than those books would require a lot of evidence to back up said recommendation.

• What are some applications which require interval arithmetic?: Keep. Again, there are wrong answers, and it's not as if anyone can just chime in (as opposed to the dense matrix question further down the page). Interval analysis is a pretty specialized topic within computational science.

• What kinds of problems lend themselves well to GPU computing?: Keep. Not anyone in computational science can just chime in here. Only people who actually know stuff about GPUs could answer this question in a way that makes sense.

• Databases of results for numerical codes: Borderline. I'd keep it, but maybe reorganize the answers into a community wiki resource. There are definitely wrong answers to this question, but the barrier to entry is high (i.e., come up with a data set not mentioned, and have it be a bad data set with standard results).

• Recommendations for a usable, fast C++ matrix library?: Borderline. I wrote this question, and originally, I set very specific guidelines on what I wanted, and then I wrote a long answer figuring that hey, people might want a summary of what I found out after doing research for a day. A lot of people like it, and it generated me a lot of rep (thank you for all who voted). I thought there weren't that many C++ matrix libraries out there in popular use, but the answer has started to attract some good links and some bad links. We could wikify it, but I'd rather take myself out of that decision and leave it to other people because I'm biased about my own question. The problem I could see with closing this question on the grounds that it's a software question is going to be that there are so many software-type questions out there, and they make up a substantial fraction of the site. I think the more defensible reason would be, "Anyone can wrap a 1-D array in a class to simulate a matrix, post it on their web site, and call it a 'Matrix Library'." So, in other words, it could be that this software request is too basic and that there are virtually no wrong answers, in which case, I should've asked a better question, and can try to edit it to save the question by eliminating upstarts and focusing on codes that have an institution behind them; this qualification should, in theory, make it easier for there to be wrong answers.

• Algebraic Multigrid Code: Keep. Another software-type question. I didn't think there are that many AMG codes out there, but perhaps I'm wrong?

• Are open-source codes available to study protein folding?: Keep. Are there that many protein-folding codes? (Seriously, it's not my field, so I'm curious if there are.)

• Adding deliberate imperfection to RNG output - toolkits?: Keep. I don't even know of any software that does anything like adding deliberate imperfections to RNG output toolkits, which makes me think that there are indeed wrong answers.

• Mathematical Libraries for OpenCL?: Edit. We should edit this one. Everything but the first sentence of this question is pretty specific in its scope; those sentences are limited to ViennaCL, cusp, and OCLTools. The problem is in the first sentence, "I am looking for information from anyone that has tried to use OpenCL in their scientific code." It's too easy for someone to post about casual experience with OpenCL, and I don't think that's the intent of the question.

• Sparse hermitian eigensystems: are there better techniques than Arpack or TRLan?: Keep. Either there are packages that are competitive or better than ARPACK and TRLan for the poster's problems, or there aren't.

• Where can I find a database of simple chemical structures in XYZ format?: Borderline. I could see this question going either way. Either there aren't that many databases for chemical structures, or the barrier to entry of this question is too low, and anyone can say something about structure conversion. I'm tempted to say that we should keep this one, on the grounds that I doubt there's are a lot of databases.

• Texture analysis methods modern survey paper: Keep. No one's even answered this question, so I'm not too worried about it anyone being able to contribute a correct answer.

• Is there a high quality nonlinear programming solver for Python?: Keep. Yes, the question is subjective, but there are really only so many high quality nonlinear programming solvers out there (maybe 10?), and the rest are just implementations of some sequential quadratic programming method or active set method or interior point method, etc. A lot of people have implemented nonlinear programming methods, but implementing a method doesn't make it high quality, and the people who stepped forward with research quality libraries were pretty up front about it.

• Are there any open source inverse-based multilevel ILU implementations?: Keep. Sadly, the answer to this question seems to be no, because no one's said anything. I'm not expecting a flood of answers here.

• Is there any open-source or easy-to-access software that can simplify algebraic expressions like $x^{2}+2x+3, x=\sqrt{2}t-1$?: Keep. There are a handful of commonly used open source CAS software packages, so I don't think it'd be terrible to list all of them.

• What guidelines should I use when searching for good preconditioning methods for a specific problem?: Keep. Looks like a question that isn't easy to answer. I thought JackPoulson did a good job here. I don't think it's a list question, because there are wrong answers.

• https://scicomp.stackexchange.com/questions/1120/is-there-a-good-site-for-holding-online-discussions-of-scientific-papers: Keep for now. Somewhat surprisingly, this post hasn't gotten an answer. I think there won't be many answers until a bunch of sites like HiveMined, and so on, come online, and then there might be a lot of them. We might want to edit this question, but this question isn't in imminent danger of being swamped with answers.

• Where do dense matrices occur?: Close. I think the potential for abuse of this question is too great, because I don't think the number of applications in which dense matrices occur is small, therefore it's too easy for this question to become a list of applications because the pool of answers is too large.

• What simple methods are there for adaptively sampling a 2D function?: Keep. I feel like this question is limited in scope. I don't think it would be easy for anyone in the scientific community to come in and post an answer to that question.

• Venues for publishing papers that emphasize software: Keep for now. Ah, this question. I like this question, but I'd badger some people about making their answers better. One of the problems I have with journal questions is that the potential for abuse is too great, which is why I wanted to close it. There are a lot of journals out there, and any exhaustive list would be long, which makes me think it's easy fodder for newcomers to the site. Part of me is a little concerned that the vote system could in some way be construed as a popularity indicator of journals. All of these reasons are treading over old ground, though.

Answer 3

I'd like to defend "Are there any open source inverse-based multilevel ILU implementations?".

For one, I read David's reply in one of the Scope Meta questions about Journals as : ".....I think such questions should be allowed. They have a factual answer (a list of all journals on a given topic) which may not be easy to find anywhere else....." Assuming we still stand by this, then this implies we would want questions which lead to limited factual lists devoid of opinion. How is this different for ILU? I agree for my question (on dense matrices occurrence) that it deserves to be closed (In fact, I voted for it to be closed a minute ago) because it would demand opinions but not the ILU question.

In fact, it leaves lesser margin for opinion as compared to the Journal question. One might argue that the SIAM Journal for Matrix Analysis might be relevant to SciComp and there could be a debate. However, for an ILU implementation: either something is or is not. Whether or not this specific question invites answers is another issue but can't we let list based questions with factual & "limited" lists (or in other words, lists with no margin for opinion) be allowed? Or atleast be converted to Community Wikis?

Horrible lists have become Wikis elsewhere.

And Lastly, my question on "Which LA before NLA" was not a "suggest me textbooks" question. As I mentioned in the question, I merely wanted an opinion on which side to tilt : Rigor or No Rigor (or Golub). I don't know why people started recommending books as answers.