7

I recently asked about whether identification questions would be on-topic for a proposed beta.

One of the replies originally stated that identification questions were some of the least productive on the site.

I followed up with:

As for productivity, when I look at SciFi's tags I see that Story-Identification is their most-used tag (more than the next 2 tags combined).

And the reply was:

We cannot guarantee there's a site (or an audience) for every question, but in my opinion (and many others), identify-this-* questions are not generally a very useful part of most sites. Unfortunately, showing me how it has all-but overtaken SciFi (and our Movie site) does not enamor me to it any moreso.

From this, I get the feeling that there's a large number of people who dislike identification questions, despite their very apparent popularity.

Is the popularity of these questions something that the stack has only begrudgingly allowed?


Some data:

After seeing a quote from Jeff Atwood on the Guessing Games blog post:

Well, look at the data. https://scifi.stackexchange.com/search?q=views%3A500 571 questions with views of 500 or more. Of those ONLY SIXTEEN, 2.8% are [story-identification]. https://scifi.stackexchange.com/search?q=%5Bstory-identification%5D+views%3A500 That’s a little odd, don’t you think, considering [story-identification] is the second most common tag on the site? Like I was saying.. “The goal of Stack Exchange is not to construct un-findable single-serving questions that only help one person, but that’s exactly what guessing game questions tend to do.” I know they’re fun in a way, but a) we’re not shy about demoting fun when it isn’t producing useful artifacts and b) you have to take a cold, hard look at the utility of the artifacts that are produced.

I wrote this query: Story ID question with > 500 views

And found that there were 1131 rows returned, which is just under 24% of all such questions. If you bump it up to > 1000 views, it's just over 7%.

I provide this only as evidence that the questions themselves are popular to view, not just to be asked, but I won't make any normative statement about the questions in general.

  • 1
    They've always been on-topic as far as I know, because I've had the tag on ignore for a long time. I absolutely hate them, but they are extremely popular on here and some people love them. – KutuluMike Aug 6 '15 at 20:06
  • 1
    Indeed. If you visit scifi.stackexchange.com/users/preferences and set the tag to "ignore", you'll never see another one again – Valorum Aug 6 '15 at 20:10
  • 1
    Interesting to ask this on the one site where those questions don't seem to be a huge problem. ;-) – TARS says Reinstate Monica Aug 6 '15 at 20:15
  • @TARS Well, it was the one referenced in my discussion, and the only one I frequent. It's huge popularity seems to be evidence that it's not a problem. But then again, the first two comments are by long-time users who seem to very much dislike those questions. I wonder what a graph of people using the story-id tag based on account length would look like. – user31178 Aug 6 '15 at 20:18
  • I'm hardly a "long-time user". And @RobertCartaino is one of the employees and they're certainly likely to have strong opinions about things. Two people not liking ID questions doesn't mean that a community can't decide that they're OK... if Arts and Crafts wants them, they can, as a community, downvote our answers and upvote the "yes" answers... it doesn't mean that users who don't like ID questions can't try to prevent them showing up on other sites. – Catija Aug 6 '15 at 20:27
  • @Catija I think he was referring to the commenters here on this very question and not the question on Area51 when he talked about "long-time users". – TARS says Reinstate Monica Aug 6 '15 at 20:31
  • @TARS Ah, that makes sense. I still think the rest of my comment is valid but if someone wants to excise it, that's fine. – Catija Aug 6 '15 at 20:32
  • @Catija I was referring to Richard and Michael. My goal isn't to push the A&C thing, or call anyone out, which is why I didn't link. I'm perfectly fine with the answers received. I just wondered if there was some history I'm missing, as I've only been on SE actively for less than a year. I often feel like there's stuff I've missed out on that shaped the communities. – user31178 Aug 6 '15 at 20:34
  • Is "overtaken" in the title a (Google Translate?) mistranslation? In English, "overtake" != "take over", and it's not clear how "overtaken" makes sense in the present context. – user14111 Nov 13 '15 at 20:02
  • @user14111 Grown to be a problem, rather than a benefit – user31178 Nov 13 '15 at 20:52
  • So you meant "taken over"? – user14111 Nov 13 '15 at 21:15
  • @user14111 Well I was quoting someone else, so I don't know about them. Both work. – user31178 Nov 13 '15 at 21:19
16

Inspired by Napoleon Wilson's post on the M&TV Meta, I thought I'd look at some of the statistics on identification questions.

A few notes before I get started:

  1. I don't want to diminish anyone's experience. All of the numbers and graphs and I'm going to spit at you all in this answer are averages. There are certainly some exceptional users who were drawn here by ID questions, or who stay here because of ID questions. I think that's wonderful. Your personal experiences may not match with the average, but that's why averages are average

  2. We're more than just story-id. But not much more; we have seven (non-aliased) identification tags: , , , , , , and , but only story-id has more than 100 questions

  3. Caching is (still) a problem. I'm going to use SEDE as much as possible, so my numbers may be slightly out of date. At time of writing, SEDE was last updated four days ago.

I'm also going to use another frequently-used tag as a baseline; is our second most-used tag, so that'll do.

Question Numbers

There are currently 5,403 non-deleted ID questions, and 2,677 Harry Potter questions out of 25,574 non-deleted total questions. So 21.1% of all non-deleted questions have been identification questions, and 10.5% have been Harry Potter questions.

Of those ID questions, 287 of them have been closed; that's 5.3% of all ID questions. In accordance with our policy of closing duplicate ID questions, the vast majority of those closures are duplicates; I can't check this with SEDE (or, I don't know how to), but from looking on the main site: out of 293 closed ID questions (coming from a later time), 210 (71.7%) of them were closed as duplicates. With 2,191 total closed questions, ID questions represent 13.1% of all closures.

Of the Harry Potter questions, 328 have been closed, 12.3% of all HP questions; that's 15% of all closed questions. Out of 331 closed questions, 276 (83.4%) of them were closed as duplicates.

Unfortunately I can't comment on deleted questions; SEDE strips tags from deleted questions, and I don't have the mod power to search for them on the main site. Undoubtedly some of the poorer examples get sucked by the roomba, and are lost to my stats.

Question Quality

With the acknowledgement that quality is a hard thing to measure statistically, ID questions are lower in all stats we can measure (note that these are averages, and only includes positive-scoring answers):

enter image description here

I think the average views/question is the number that shocks me the most; ID questions are viewed an average of five times less than non-ID questions. But, though it's less dramatic, I also find it noteworthy that both the questions and their answers are scored noticeably lower than non-ID questions.

Question Growth

ID questions are growing quite rapidly; the left graph shows absolute number of questions (Yellow is total, green is non-ID, blue is ID, red is HP), and the right shows growth as a percentage of total questions (yellow is ID, blue is HP):

enter image description here

Who asks them?

75% of all ID questions are a user's first question, compared with 20.9% of HP questions. When considering only a user's first post, those drop to 71.1% and 16.2%, respectively.

Of all first posts, 26.9% of them are ID posts; when a first post is a question, 50.1% of the time it's an ID question, and when the first post is an answer, it's an ID answer just 10.8% of the time.

enter image description here

In contrast, 8.3% of first posts are HP posts; 5.5% of first-post questions, and 10.2% of first-post answers.

So ID questions are clearly driving new users to the site.

But they're not staying, at least not in droves (red are users whose first question wasn't an ID question; green is for HP, yellow for all askers, blue for ID):

enter image description here

31.1% of users whose first question was an ID question were still visiting 30 days later. Compare to 47.7% for the site average, 59.1% for HP askers, and 63.8% for general non-ID askers.

enter image description here

20.2% of users whose first question was an ID question have any more posts. 16.8% have another post that's not an ID question. Compare to 55.5% and 55.3% for HP-askers, or 34.2% and 32.3% for the site as a whole. Users whose first question is an ID question appear to participate less, in general.

My Thoughts

So obviously I have some statements about the utility and quality of ID questions peppered throughout my analysis above. But despite how I may have interpreted the numbers, I don't see ID questions as a problem.

It's true that they've "overtaken" the site, in the sense that they tend to be of less-high quality and utility, as judged by question score and view count, and are abandoned more often (judged by likelihood of acceptance).

But I don't think that's a bad thing for the site. The fact that my numbers are as positive as they are proves that we've been able to stem the tide of really objectionable ID questions (and objectionable questions of all sorts). As long as the ones that are left don't diminish the quality of the site (and they don't; they just increase it by a smaller amount), I have no problem with them.

Likewise, I don't think it's a problem that a third of ID askers keep visiting the site, and that 15-25% of them continue to contribute. I'd rather have those people here, and contributing, then not have them.

At best they're the reason some people keep coming back to the site. At worst, they're a mild nuisance to some frequent users. Yes, I count myself in the "nuisance" group, but the minor inconvenience they cause me is outweighed, in my mind, by the benefit they bring to other users.

  • 3
    This is a great write up, and I think it gives us much to think about. – user31178 Nov 12 '15 at 23:27
  • 3
    The fact that so many story-ID questions are closed as duplicates belies the BS claim that they are "single-serving questions that only help one person". – user14111 Nov 13 '15 at 21:11
  • What are the stats on users whose first post was a story-ID answer? – user14111 Nov 13 '15 at 21:12
  • 1
    @user14111, given the policy for closing ID questions as duplicates, it means the answer helped more than one person, but the question clearly didn't. – Mark Nov 13 '15 at 22:34
  • 1
    @Mark So you're saying that answers are more helpful than questions? I can't argue with that, though I'm not sure how that applies to story-ID questions any more than to other types of questions. – user14111 Nov 13 '15 at 22:47
  • 2
    What I'm getting from that is that we're becoming known as a place where you can get your Story-ID question answered (presumably via word of mouth) and that more than a third of Story-ID questioners stay with the site afterwards. Both are highly encouraging. – Valorum Nov 13 '15 at 23:24
  • @user14111 I'm starting to add stats for first answers; based on the table I just added, it doesn't look as though ID answers are much different from answers in any other popular tag – Jason Baker Nov 13 '15 at 23:27
  • 1
    @Richard There's ups and downs. Story-id definitely attracts new eyeballs, but it doesn't grow the community as effectively as other subjects (admittedly Harry Potter was perhaps not the best choice of baseline in that regard; the HP fanbase can be a bit rabid at times) – Jason Baker Nov 13 '15 at 23:42
  • @JasonBaker - 20% future uptake sounds excellent to me. – Valorum Nov 13 '15 at 23:55
  • 4
    I see a lot of complaints here about story-ID Q&A, which as far as I can see is practically the only redeeming feature of this site. Does no one ever complain about the plague of Harry Potter questions? – user14111 Nov 14 '15 at 0:02
  • 1
    @user14111 -- Yes, they do. Complain about Harry Potter questions, that is. I'm unclear why it's so hard for people who don't answer Harry Potter questions to just ignore HP questions, but HP questions do continue to be and area of contention. – Slytherincess Dec 10 '15 at 17:42
  • 1
    @user14111 - what Slytherincess said. I see complaints about HP questions. Sometimes SW questions. Sometimes about "too many 'how many of XYZ' questions". Sometimes about "too many 'first use of XYZ' questions". Etc.... etc.... People, in general, like to complain :) – DVK-on-Ahch-To Dec 15 '15 at 19:58
13

I have no personal interest in identification questions either, because I'm nowhere near well-read enough to answer any of them, and I already have more things to read/watch than I'll ever get around to so I get nothing out of reading them. But I find the meta issue of ID questions very interesting, especially how much it seems to vary from site to site, so here's a text wall of the various random observations I can offer.

tl;dr: In my experience, the acceptability of ID questions on a given site comes from the natural quality level that ID questions have there when they're allowed without restriction, and this is a product of 1) how their enthusiasts stumble across things they want identified, and 2) whether their enthusiasts remember the kinds of details that help distinguish one work from another.

For instance, on Anime & Manga you get a lot of "What anime is this image from?" which can often be answered by a trivial reverse image search. This is probably because in the anime/manga communities it's very normal to browse imperfectly-tagged internet archives of images from potentially anywhere, and many of the people doing this simply don't know that reverse image search is a thing.

Over on Arqade, ID questions were underspecified so often that they had to narrow the scope to ID questions with an "audiovisual artifact". Presumably this is because it's hard to write a textual description of any video game that clearly distinguishes it from any other; most mechanics, weapons, enemy types, plot devices, etc are shared by several different games (unless you throw in details like exact cooldown times that no one would normally remember). You might remember playing a game that had this cool gravity manipulation power, but just saying "gravity manipulation" doesn't narrow it down as much as you might think.

This site seems to be one of the few SEs on which simply allowing any identification question tends to get a lot of decently-asked, genuinely answerable questions that are not a blatant "lack of research effort", nor are there so many of them they risk overtaking the site. I suspect this is because 1) most of our story-ID askers are not just "I saw this thing on Google, what is it from?" but people who actually read or watched the work a long time ago and are trying to find it again, and 2) the things we remember a sci-fi/fantasy story for are usually the very same things that make it easily identifiable ("the alien burst out of his chest", "dystopia where it turned out the food was made of people", etc). Note that we also have a lot of character-ID questions, but those tend to be motivated by very specific things like "Who was that character at the end of the last Marvel movie?" which are--unlike most ID questions--potentially useful to other people!

10

Yes and no.

Yes, we have heaps and heaps of them. Like you said, more than the next two tags combined.

No, since we're not too bothered by them. They're easy to ignore if you want to, but often fun to answer.

  • 2
    Seconded. I quite like them, they attract users and they're often challenging/fun. – Valorum Aug 6 '15 at 21:53
  • As fun as they are, I don't see 'fun/challenging' as good metric for quality. Regardless, I think they are good content. By their nature media that is difficult to identify is either so obscure few people are looking for it, or actually obvious so few people are looking for it. By that token they'll never be 'view' magnets, but I bet if you could see how many people come to the site because of them it would be pretty high up their. – AncientSwordRage Aug 7 '15 at 6:29
  • 5
    @Pureferret Personally, I'd be more interested in seeing how many come because of them and stay – Jason Baker Aug 7 '15 at 7:46
  • What does an upvote or downvote mean on a meta answer that says "yes and no"? – user14111 Aug 8 '15 at 5:47
  • 6
    @JasonBaker I joined the site to ask a story identification question, and now have over 7000 rep and counting, answering many questions (more often than not in the Marvel Cinematic Universe tag). I imagine that I am not the only person like that. – Dr R Dizzle Aug 10 '15 at 11:52
  • 1
    @DrRDizzle I don't imagine you are either; but for every story like yours, how many never come back once their question has been answered? Or how many see a story-id question somebody else asked, and then think "I want to become a part of this community"? – Jason Baker Aug 10 '15 at 18:29
  • 1
    @user14111 - It means you disagreed with their vacillation. – Valorum Aug 10 '15 at 18:44
8

I don't like story ID questions at all. But, yes, I have the option to filter these questions away -- I've just been too focused on other activities (Q&A, chat, etc.) to put the filter in place. Since we have the option of filtering by tag, I kind of feel like posts that ask "Do we have too many of "X" questions are redundant -- as long as the tag is on topic for SFF. If the tag is on topic, and it bothers you so greatly, simply put a filter into place. Problem solved! We all like different things, so the filter is a great tool.

  • 5
    It's totally relevant when new users don't realise this isn't a 'guess the movie site'. But I genuinely don't see us at that point yet – AncientSwordRage Aug 7 '15 at 6:26
8

I personally find the story identification queries very useful, not only for my own asked questions, but also because I occasionally find it's already been answered. In addition, I do find answering the questions entertaining both for the challenge and for the fact that it exposes me to new works I might never have thought to look for (A Ghost and his Ventriloquist hunt down Nazis!, A Boy and his Friends are haunted by serial killer memorabilia!, A Boy is thought a Monster because he can swim!, Young boy goes from Page to Wizard!, Aliens attack over Thanksgiving!, The Space between Perforations is the new super-technology!, Opera-singing Multi-dimensional Secret Agents!, Children turned to Butterflies by a Tree and then Dissected!, The Battle between Wicca and Yeast!, Death by Swordfish in Satan's Triangle! and many others).

Yes, Story Identification is as much about fun as anything else, as well as huge geek cred for being able to answer, but honestly, isn't that entirely what the Code Golf Stack Exchange is about?

5

I like them....my major annoyance with them (which is implied by Jason Baker's wonderful data and analysis) is when a first time user pops in, asks one, I answer it correctly, and then they never show up again to accept it. Grrr. Ironically, my highest voted answer on this site is one of these. But I still like them.

PS, I came to the site to ask one (which was answered) and have stayed.

  • My 2 highest voted answers on this site : sobs: – Organic Marble Feb 29 '16 at 1:53

You must log in to answer this question.