Have story identification questions "overtaken" SF&F?

Question

I recently asked about whether identification questions would be on-topic for a proposed beta.

One of the replies originally stated that identification questions were some of the least productive on the site.

I followed up with:

As for productivity, when I look at SciFi's tags I see that Story-Identification is their most-used tag (more than the next 2 tags combined).

And the reply was:

We cannot guarantee there's a site (or an audience) for every question, but in my opinion (and many others), identify-this-* questions are not generally a very useful part of most sites. Unfortunately, showing me how it has all-but overtaken SciFi (and our Movie site) does not enamor me to it any moreso.

From this, I get the feeling that there's a large number of people who dislike identification questions, despite their very apparent popularity.

Is the popularity of these questions something that the stack has only begrudgingly allowed?

---

Some data:

After seeing a quote from Jeff Atwood on the Guessing Games blog post:

Well, look at the data. https://scifi.stackexchange.com/search?q=views%3A500 571 questions with views of 500 or more. Of those ONLY SIXTEEN, 2.8% are [story-identification]. https://scifi.stackexchange.com/search?q=%5Bstory-identification%5D+views%3A500 That’s a little odd, don’t you think, considering [story-identification] is the second most common tag on the site? Like I was saying.. “The goal of Stack Exchange is not to construct un-findable single-serving questions that only help one person, but that’s exactly what guessing game questions tend to do.” I know they’re fun in a way, but a) we’re not shy about demoting fun when it isn’t producing useful artifacts and b) you have to take a cold, hard look at the utility of the artifacts that are produced.

I wrote this query: Story ID question with > 500 views

And found that there were 1131 rows returned, which is just under 24% of all such questions. If you bump it up to > 1000 views, it's just over 7%.

I provide this only as evidence that the questions themselves are popular to view, not just to be asked, but I won't make any normative statement about the questions in general.

Answer 1

Inspired by Napoleon Wilson's post on the M&TV Meta, I thought I'd look at some of the statistics on identification questions.

A few notes before I get started:

1. I don't want to diminish anyone's experience. All of the numbers and graphs and I'm going to spit at you all in this answer are averages. There are certainly some exceptional users who were drawn here by ID questions, or who stay here because of ID questions. I think that's wonderful. Your personal experiences may not match with the average, but that's why averages are average

2. We're more than just story-id. But not much more; we have seven (non-aliased) identification tags: story-identification, character-identification, episode-identification, author-identification, actor-identification, object-identification, and music-identification, but only story-id has more than 100 questions

3. Caching is (still) a problem. I'm going to use SEDE as much as possible, so my numbers may be slightly out of date. At time of writing, SEDE was last updated four days ago.

I'm also going to use another frequently-used tag as a baseline; harry-potter is our second most-used tag, so that'll do.

Question Numbers

There are currently 5,403 non-deleted ID questions, and 2,677 Harry Potter questions out of 25,574 non-deleted total questions. So 21.1% of all non-deleted questions have been identification questions, and 10.5% have been Harry Potter questions.

Of those ID questions, 287 of them have been closed; that's 5.3% of all ID questions. In accordance with our policy of closing duplicate ID questions, the vast majority of those closures are duplicates; I can't check this with SEDE (or, I don't know how to), but from looking on the main site: out of 293 closed ID questions (coming from a later time), 210 (71.7%) of them were closed as duplicates. With 2,191 total closed questions, ID questions represent 13.1% of all closures.

Of the Harry Potter questions, 328 have been closed, 12.3% of all HP questions; that's 15% of all closed questions. Out of 331 closed questions, 276 (83.4%) of them were closed as duplicates.

Unfortunately I can't comment on deleted questions; SEDE strips tags from deleted questions, and I don't have the mod power to search for them on the main site. Undoubtedly some of the poorer examples get sucked by the roomba, and are lost to my stats.

Question Quality

With the acknowledgement that quality is a hard thing to measure statistically, ID questions are lower in all stats we can measure (note that these are averages, and only includes positive-scoring answers):

I think the average views/question is the number that shocks me the most; ID questions are viewed an average of five times less than non-ID questions. But, though it's less dramatic, I also find it noteworthy that both the questions and their answers are scored noticeably lower than non-ID questions.

Question Growth

ID questions are growing quite rapidly; the left graph shows absolute number of questions (Yellow is total, green is non-ID, blue is ID, red is HP), and the right shows growth as a percentage of total questions (yellow is ID, blue is HP):

Who asks them?

75% of all ID questions are a user's first question, compared with 20.9% of HP questions. When considering only a user's first post, those drop to 71.1% and 16.2%, respectively.

Of all first posts, 26.9% of them are ID posts; when a first post is a question, 50.1% of the time it's an ID question, and when the first post is an answer, it's an ID answer just 10.8% of the time.

In contrast, 8.3% of first posts are HP posts; 5.5% of first-post questions, and 10.2% of first-post answers.

So ID questions are clearly driving new users to the site.

But they're not staying, at least not in droves (red are users whose first question wasn't an ID question; green is for HP, yellow for all askers, blue for ID):

31.1% of users whose first question was an ID question were still visiting 30 days later. Compare to 47.7% for the site average, 59.1% for HP askers, and 63.8% for general non-ID askers.

20.2% of users whose first question was an ID question have any more posts. 16.8% have another post that's not an ID question. Compare to 55.5% and 55.3% for HP-askers, or 34.2% and 32.3% for the site as a whole. Users whose first question is an ID question appear to participate less, in general.

My Thoughts

So obviously I have some statements about the utility and quality of ID questions peppered throughout my analysis above. But despite how I may have interpreted the numbers, I don't see ID questions as a problem.

It's true that they've "overtaken" the site, in the sense that they tend to be of less-high quality and utility, as judged by question score and view count, and are abandoned more often (judged by likelihood of acceptance).

But I don't think that's a bad thing for the site. The fact that my numbers are as positive as they are proves that we've been able to stem the tide of really objectionable ID questions (and objectionable questions of all sorts). As long as the ones that are left don't diminish the quality of the site (and they don't; they just increase it by a smaller amount), I have no problem with them.

Likewise, I don't think it's a problem that a third of ID askers keep visiting the site, and that 15-25% of them continue to contribute. I'd rather have those people here, and contributing, then not have them.

At best they're the reason some people keep coming back to the site. At worst, they're a mild nuisance to some frequent users. Yes, I count myself in the "nuisance" group, but the minor inconvenience they cause me is outweighed, in my mind, by the benefit they bring to other users.

Answer 2

I have no personal interest in identification questions either, because I'm nowhere near well-read enough to answer any of them, and I already have more things to read/watch than I'll ever get around to so I get nothing out of reading them. But I find the meta issue of ID questions very interesting, especially how much it seems to vary from site to site, so here's a text wall of the various random observations I can offer.

tl;dr: In my experience, the acceptability of ID questions on a given site comes from the natural quality level that ID questions have there when they're allowed without restriction, and this is a product of 1) how their enthusiasts stumble across things they want identified, and 2) whether their enthusiasts remember the kinds of details that help distinguish one work from another.

For instance, on Anime & Manga you get a lot of "What anime is this image from?" which can often be answered by a trivial reverse image search. This is probably because in the anime/manga communities it's very normal to browse imperfectly-tagged internet archives of images from potentially anywhere, and many of the people doing this simply don't know that reverse image search is a thing.

Over on Arqade, ID questions were underspecified so often that they had to narrow the scope to ID questions with an "audiovisual artifact". Presumably this is because it's hard to write a textual description of any video game that clearly distinguishes it from any other; most mechanics, weapons, enemy types, plot devices, etc are shared by several different games (unless you throw in details like exact cooldown times that no one would normally remember). You might remember playing a game that had this cool gravity manipulation power, but just saying "gravity manipulation" doesn't narrow it down as much as you might think.

This site seems to be one of the few SEs on which simply allowing any identification question tends to get a lot of decently-asked, genuinely answerable questions that are not a blatant "lack of research effort", nor are there so many of them they risk overtaking the site. I suspect this is because 1) most of our story-ID askers are not just "I saw this thing on Google, what is it from?" but people who actually read or watched the work a long time ago and are trying to find it again, and 2) the things we remember a sci-fi/fantasy story for are usually the very same things that make it easily identifiable ("the alien burst out of his chest", "dystopia where it turned out the food was made of people", etc). Note that we also have a lot of character-ID questions, but those tend to be motivated by very specific things like "Who was that character at the end of the last Marvel movie?" which are--unlike most ID questions--potentially useful to other people!

Answer 3

Yes and no.

Yes, we have heaps and heaps of them. Like you said, more than the next two tags combined.

No, since we're not too bothered by them. They're easy to ignore if you want to, but often fun to answer.

Answer 4

I don't like story ID questions at all. But, yes, I have the option to filter these questions away -- I've just been too focused on other activities (Q&A, chat, etc.) to put the filter in place. Since we have the option of filtering by tag, I kind of feel like posts that ask "Do we have too many of "X" questions are redundant -- as long as the tag is on topic for SFF. If the tag is on topic, and it bothers you so greatly, simply put a filter into place. Problem solved! We all like different things, so the filter is a great tool.

Answer 5

I personally find the story identification queries very useful, not only for my own asked questions, but also because I occasionally find it's already been answered. In addition, I do find answering the questions entertaining both for the challenge and for the fact that it exposes me to new works I might never have thought to look for (A Ghost and his Ventriloquist hunt down Nazis!, A Boy and his Friends are haunted by serial killer memorabilia!, A Boy is thought a Monster because he can swim!, Young boy goes from Page to Wizard!, Aliens attack over Thanksgiving!, The Space between Perforations is the new super-technology!, Opera-singing Multi-dimensional Secret Agents!, Children turned to Butterflies by a Tree and then Dissected!, The Battle between Wicca and Yeast!, Death by Swordfish in Satan's Triangle! and many others).

Yes, Story Identification is as much about fun as anything else, as well as huge geek cred for being able to answer, but honestly, isn't that entirely what the Code Golf Stack Exchange is about?

Answer 6

I like them....my major annoyance with them (which is implied by Jason Baker's wonderful data and analysis) is when a first time user pops in, asks one, I answer it correctly, and then they never show up again to accept it. Grrr. Ironically, my highest voted answer on this site is one of these. But I still like them.

PS, I came to the site to ask one (which was answered) and have stayed.