23

Context: Unpinning the accepted answer from the top of the list of answers

Stack Overflow has just stopped pinning the accepted answer to the top of the list of answers, instead ordering answers purely by score. Company staff are now calling on all individual sites to let them know whether we want to (a) follow Stack Overflow in unpinning accepted answers or (b) keep the old/current system of pinning accepted answers to the top.

Would you like to have the accepted answer unpinned on your site?

We can change the way the engine sorts answers in site settings. We would like to hear from you all if it is something you want to see on your site. (Please let me acknowledge in advance that we will not be able to run a test on each site.)

SFF has one of the most active meta communities on the network (I think), and this is a rare opportunity for us to decide ourselves what to do about a built-in site tooling feature, so let's get our discussion juices flowing! Do we want the accepted answer pinned or not? Why or why not?


Outcome

Based on voting as of October 12, 2021, we have asked to unpin the accepted answer. We have been told by a CM that we can ask to pin the accepted answer again if we don't like it, so the plan is to have the accepted answer unpinned for a month and then re-open the discussion for a final vote. The new discussion will be open for a month to collect votes, so the accepted answer will be unpinned for at least two months.

You can see the list of affected questions using this SEDE query. Since many users are specifically concerned about this change for story identification questions, you can see the list of affected story ID questions using this SEDE query and may choose to vote for the accepted answer in order to ensure it is at the top whether or not the pinned option is chosen.

20
  • 4
    Does someone with serious SEDE-fu feel like crafting a query that would show how often the highest-voted answer is not the accepted answer (but another answer is accepted)? This feels right, but I can't tell if it's an actual problem we have. It's not like PKD is going to re-write Do Androids... every 2 or 3 years and keep invalidating the answers to any questions we may have about it.
    – DavidW
    Sep 8 at 17:15
  • 2
    @DavidW Not sure if that last comment is tongue in cheek, but there's plenty of "franchises in progress" where the correct answer is going to change over time (how many regenerations does the Doctor have left, suggested reading order for the Cosmere, etc.)
    – Rand al'Thor Mod
    Sep 8 at 17:17
  • It was maybe partially tongue-in-cheek, but while ongoing franchises may (well, realistically "will") suffer from drifting canon, there are a lot of (probably a majority of) works that are basically complete as they stand, and we already tend to distinguish different versions of a work. (e.g. total-recall vs. total-recall-2012)
    – DavidW
    Sep 8 at 17:33
  • 1
    I would say yes. Thinking about later answers to old questions that are more "correct" or fleshed out than the current accepted answer: potential to be accepted but because they're newer they've received fewer votes, so the "correct" answer isn't highlighted. That being said, I can see and appreciate the other side of the coin
    – fez
    Sep 8 at 17:35
  • 11
    Would be pretty bad for the story-IDs answered and accepted months/years later with little to no upvotes
    – Jenayah
    Sep 8 at 18:14
  • 2
    @DavidW posted a few queries for our Politics site here. Feel free to fork / copy them to run on the SciFi SEDE. :)
    – JJJ
    Sep 8 at 19:12
  • @Randal'Thor Did we ever have a policy agains questions whose "correct answer is going to change over time"? Was that covered by "too localized"?
    – user14111
    Sep 8 at 20:04
  • 3
    @user14111 I'm not sure if we had a "too localised" close reason back in the mists of time, but at least in my 7 years on the site, we haven't been closing questions whose answer changes over time. That would disallow a whole lot of questions about ongoing franchises, which we don't.
    – Rand al'Thor Mod
    Sep 8 at 20:13
  • 3
    @JJJ Thank you! For everyone else, I've cloned it here. Glancing through the results, I've found a lot of cases where acceptance should be on the higher-scoring answer, but there are also cases where the lower scoring accepted answer (mostly because it's later) is the better answer.
    – DavidW
    Sep 8 at 21:27
  • 2
    @Jenayah I was thinking those, and our "Earliest X" questions
    – AncientSwordRage Mod
    Sep 9 at 8:59
  • 5
    Why would we even need accepting answers if they weren't pinned on top?
    – VienLa
    Sep 9 at 18:02
  • 4
    @VienLa pinning accepted answers to the top has always been a side effect of an accept, not the reason for doing it.
    – TheLethalCarrot Mod
    Sep 9 at 21:22
  • 3
    Pinning an answer to the top helps in the case of giving attention to a late better answer.
    – ibid
    Sep 10 at 20:31
  • 3
    Damn, I just upvoted every moderator answer on a post on Meta. Either I'm growing soft in my old age and no longer qualify as punk, or the quality of moderator team significantly improved. Sep 13 at 22:40
  • 2
    This has been added to the Community Team's backlog and we'll update you once someone picks it up and evaluates your request.
    – JNat StaffMod
    5 hours ago
13

I think this would be a neutral-to-positive change for us.

(answer format and a lot of content taken from doppelgreener, here, and reworded to be specific to this site)

The accepted answer exists to signal something specific: “this is the solution that I went with, that worked for me.” It originated on Stack Overflow because only one person is in the position to confirm that the code they were given compiled & fixed their issue.

SFF.SE inherited that behaviour simply because we inherited the same Q&A code, regardless of whether that feature solves specific issues for us or not. We don't have code to compile; we have questions about characters and plot, stories to identify, etc. This meta gives us an opportunity to consider from a blank slate: which option actually makes sense in the scope of Sci-Fi and Fantasy Q&A?

On SFF.SE specifically, I think the most helpful thing is to not pin the accepted solution, and just let the highest voted answer float to the top always.

Even though we have some exception (more of that later), it's often the case that our most popular answers are the best fit for the rest of the community. It's all too easy for the answer to be accepted, but have a much better answer appear later on, or rise in popularity later on. That's not to say there aren't exceptions, in that case people seemed to like the Gif, but it wasn't enough to sway the OP.

There are 2455 (as of Sept 5th( affected answer, out of the total of 37,398 accepted answers. Thats some 6%.

Without going through the whole list, it's hard to know how many exceptions there are, but I'd wager the lowest quality yet highly upvoted answers are some of the shortest/easiest to skip as you scroll though.

Stack Overflow's arguments against pinning the first answer generally apply to us too.

Please unpin the accepted answer from the top (on Meta SO) made a case against several commonly provided reasons for pinning the accepted answer under the heading “The argument against unpinning”. They generally apply here too.

I'll mainly compare against rules & social questions, which are the biggest two categories we see on our site.

  • [Reason to keep it pinned:] The accepted answer actually works for one person

    [Counter-argument:] But the Stack Overflow answers are not ultimately there for that one person. They are there for the countless visitors who come later. Ninety-nine percent of the time that I upvote a non-accepted answer, I am also signifying that it solved my problem or at least helped me get one step closer to solving it.

The counter-argument, which is in favour of not pinning, mostly applies here.

For questions, the highest-scored answer will be whichever the community recommends as the most correct answer with the most effective explanation as to why that correctly identified the story. While whatever the querent accepted is what they remember being correct. There're only 174, and in the top few most of the more highly upvoted answers are still good answers.

For , they are often a bit of a mixed bag/mess. Thankfully only 62 questions will be affected. I'd strongly suggest we should address the nature of these question at a later date to figure out the best way to curate these.

  • [Reason to keep it pinned:] The accepted answer is tested (*)

    [Counter-argument:] It is true that many people will upvote answers without testing them. However, if a highly voted answer contains a mistake, it will start collecting comments saying so. If that mistake is not corrected, other people start adding answers that are correct. The voting system starts moving those correct answers to the top. This seems fair to me.

The counter-argument, which is in favour of not pinning, applies here.

In general, all answers are expected to be backed up by citations whether they be from the work being asked about or supplementary information. Meanwhile, the highest voted is the one filtered through community knowledge and experience that gets vetted as effective. (In bad cases, as merely an answer that seems to have truthiness, but that's when we'll start nagging someone to back their answer up.) The accepted answer status doesn't really add anything to this—we should always present what the community's vetting.

Maybe an answer found one specific approach solved their specific situation, but again, what we're upvoting is what we'd recommend generally.

  • [Reason to keep it pinned:] The accepted answer solves the actual problem described in the question (*)

    [Counter-argument:] Fine, but when I use Stack Overflow I don't care if my problem is exactly the same in every detail as the OP. Sometimes the OP uses a title that leads search engines and thousands of future visitors to expect a different answer than the one that solved the OP's problem (example). There is no need to keep those kind of answers on top anymore. If I want to find them I can scroll down.

The counter-argument, which is in favour of not pinning, applies here.

In general, we're recommending that the most upvoted answer will solve peoples' problems, an we can filter that through our own knowledge or experience.

I'm beginning to repeat myself already and I'm only three points in, so I'll move on from this section.

Summary: not pinning the accepted answer means we present the one the community recommends, and that's good.

Pinning the accepted answer isn't a value-add. We don't have code to compile. It's valuable to indicate what worked for the querent personally, but anytime the accepted answer and the highest-voted answer are out of sync, it's almost certainly more worthwhile to present the community's overall recommended solution.

This way, readers are given, as a priority, the general case the community would recommend.

In cases the highest voted answer and accepted answer are different, The specific case that worked for the querent is not necessarily the most helpful to present to readers—it's just the specific case that worked for one person. We should prefer presenting the option the community generally preferred overall first.

Tangentially: Pinning a non-top answer is sometimes a valuable feature, but the accepted answer serves that purpose poorly

One of the things the accepted answer does for us is resolve situations where the top answer is, somehow, actually not the most advisable one: maybe HNQ caused us trouble; maybe that answer's obsolete and a new or better one has arisen. There's value in being able to pin an answer that isn't the top-rated one. However, I'd raise the question:

  • Why do we provide the power to pin that answer to exclusively one user (where even the mods can't override it) on a site where the userbase curates everything? That one user might not even be here anymore.
  • Why do we conflate a “this is the answer you should look at” pin with the “this is the answer the OP went with” marker? Why should that user be expected to switch the marker with answers they didn't go with? Isn't this overloading the pin to do multiple jobs?
  • When this situation occurs, doesn't that mean the Stack should be rethinking the way it handles votes (especially in light of obsolescence)? Isn't this a prompt to the Stack to rethink how it handles the impact of HNQ, or Fastest Gun in the West, or other related voting problems?

I think that in a world the accepted answer is no longer being pinned by default, we're either unaffected or better off in the majority of cases, and in the rest, the Stack should finally look to implementing a feature for pinning a recommended answer that's finally controlled by the community, not just by the post author.


Because of the overlap, much of the text above is verbatim from DG's answer, but they provide a lot of wisdom here. My initial concerns about and have been put to rest, except where it's highlighted separate issues.

Let me know if there are any comments, criticisms or corrections from the community - this is not policy, and I'd like to know how people react to the above knowledge.

4
  • Is there a way for mods to pin/unpin an answer on a question-by-question basis?
    – Spencer
    Sep 14 at 16:11
  • 1
    @Spencer I believe we have to go via CM if we want to do that.
    – AncientSwordRage Mod
    Sep 14 at 16:53
  • 2
    FWIW there's "currently" an experiment on SO about outdated answers. Presumably depending on the results from that a change might be coming in the next 6-8 weeks which could address some of the concerns over the "highlighting new better answers" issue.
    – TheLethalCarrot Mod
    Sep 15 at 9:17
  • True. Unpin it.
    – Crazy Frog
    Sep 19 at 4:41
11

I think this site should continue with the established practice of showing accepted answers first, if only for the sake of the story identification questions. For those kinds of questions, only the user who asks the question can really know whether an answer is correct or not.

6
  • 3
    How many story id questions would be affected by a change (i.e. the accepted answer is not the highest scoring one)?
    – Null Mod
    Sep 9 at 17:57
  • 3
    @Null 174 compared to the 2455 total. In other words ~7% of the potentially affected posts and ~0.9% of story id posts will be affected.
    – TheLethalCarrot Mod
    Sep 9 at 19:26
  • 3
    If you agree that many of these answers should be the top answer, consider upvoting them. (Some of them in the query may be self answers, which wouldn’t be pinned either.) Even if only a few people do this, we could easily cut down on the number of answers like this by over 50.
    – Laurel
    Sep 10 at 13:00
  • 3
    @TheLethalCarrot And not all of the 174 are really affected -- in some cases the highest voted and the accepted answer are the same work, so it's not like a "wrong" answer would be put on top. Also, there are some self-answers in those 174 which would also be unaffected.
    – Null Mod
    Sep 10 at 14:26
  • 3
    This is a perfect objection. I agree. Can we have this rule tag-based?
    – Crazy Frog
    Sep 19 at 4:42
  • 2
    @TimeLord I'd be very surprised if SE made this tag level configurable. However, you may want to post this on the main meta post so the staff can see your idea/question.
    – TheLethalCarrot Mod
    Sep 24 at 8:54
4

Unpinning the accepted answer empowers the community to bring the best answer to the top

My initial thought was that unpinning the accepted answer would be a good thing for all tags except . In general, the community is often better at picking the best answer since a single user (often a relatively new one and/or not particularly invested in the curation of the site) can easily choose to accept an answer that is not the best. This may be due to a variety of reasons, including but not limited to cases where the accepted answer is:

  • out of date
  • objectively wrong but accepted because it fits the querent's "head-canon" better than the objectively correct answer
  • not up to the community's standards, especially when compared to another answer

In such cases the community's only hope to change the top answer is to convince the querent (who may not even be around anymore) to change the accepted answer -- the querent essentially has a supervote to override the community's vote for any reason, and the community is powerless if the querent chooses poorly. The only case where the querent should have a supervote is since in that case only the querent can really know the correct answer even if other stories fit just as well as described.

According to an SEDE query for story ID questions with an accepted answer that is not the highest scoring, there are 174 questions on the site that would theoretically be negatively affected by this change. This is only 0.9% of all story ID questions, so it's not as significant a problem as I'd initially feared. The community is able to (and should) correct the scores for the affected questions by upvoting the accepted answer -- the vast majority of the affected questions could be fixed by only a couple dozen users upvoting the accepted answer.

Furthermore, it turns out that many of the 174 "affected" questions are not really affected by the change. The past few days I've been going through the list and upvoting the accepted answers on these questions, but in many cases the accepted answer and the highest scoring answer cite the same work (e.g., this, this, this, this, this) or the accepted answer was posted by the querent (in which case the accepted answer is already not pinned to the top). The number of questions that would be affected by the change is even smaller than estimated, and the community has the power to reduce that number to 0.

16
  • 5
    I think expecting the community to upvote a late answer above an early incorrect HNQ-d answer is not realistically going to happen. There's a significantly higher chance of the OP logging back in, seeing the new answer, and switching the checkmark.
    – ibid
    Sep 14 at 6:49
  • 2
    @ibid The data for the affected story ID questions shows that it would not be too difficult for the community to upvote a late "correct" answer above an earlier, highly scoring answer. I also doubt that there's a higher chance of the querent switching the accepted answer based on the number of drive-by users I've seen. Even if you are correct, though, you need the chance of the querent returning to be 100% to ensure that the best quality answer is at the top, whereas if the accepted answer is unpinned then there is always a chance for the community to get the best answer to the top.
    – Null Mod
    Sep 14 at 12:38
  • It's a little presumptuous to suggest all questions have an objectively correct answer. But just replace "the" with "an" or "any" and we're good.
    – Spencer
    Sep 14 at 16:13
  • 4
    @null - yeah it's probably true for story-id's, but not really the other tags, which are more likely to hit hnq. I've seen it happen where an OP who hadn't used the site in a few years was encouraged to log back it and change the checkmark. I've never seen a late answer beat an hnq-upvoted answer. Any new attention the question gets necessary for the new answer to get upvotes will help the top answer more than the new answer. It'll go from +50 vs +0 to +75 vs +10.
    – ibid
    Sep 14 at 16:30
  • @ibid Again, with the accepted answer pinned to the top you need every OP to log back in and change the checkmark. With it unpinned you just need the community to upvote the best answer -- that may be difficult if it is late and/or another answer benefitted from the HNQ, but it is not impossible. As for questions that aren't story IDs, why should the OP get a supervote that outweighs the entire community (which probably contains multiple users with more expertise than the OP to judge which is truly the best answer)?
    – Null Mod
    Sep 14 at 20:07
  • 1
    @Null - When it's a late answer coming in months or years after the HNQ, then it's not the OP going against the community, because the community never even had a chance to vote on the new answer, and votes are worthless for determining the best answer. Ideally someone would be able to just trigger a vote reset, but I understand that's even more controversial. In lack of that, an accepted answer is currently the only way realistically possible to get the new answer on top.
    – ibid
    Sep 14 at 20:19
  • @ibid In the event that a new, better answer arrives to compete with a highly upvoted answer then, yes, accepting the answer is the easiest way to get the best answer to the top provided with OP is willing to cooperate. In the absence of the OP's cooperation, though, it is currently completely impossible to replace a poor, accepted answer at the top. I'd rather give the community a chance to vote the best answer to the top rather than hope that the OP cooperates.
    – Null Mod
    Sep 14 at 20:35
  • 1
    @Null - OP isn't judging the best answer. They're choosing "an answer that worked for them personally."
    – Valorum
    Sep 15 at 6:34
  • 4
    @Valorum Which is all the more reason not to pin the accepted answer to the top (except for story ID) -- the answer that worked the best for the community (as indicated by votes) would most likely help future visitors and therefore should be on top, not the answer that happened to help the one querent at the time of asking.
    – Null Mod
    Sep 15 at 12:46
  • 2
    Under what circumstances do you envision 'the community' finding out about this new answer and outvoting the existing one?
    – Valorum
    Sep 17 at 17:55
  • @Valorum The Q&A is bumped to the top of the page when the new answer arrives or when edited, users will naturally come across the Q&A if it is useful in the long term, users can bring up the Q&A on meta and ask the community to upvote a better but lower scored answer, etc...
    – Null Mod
    Sep 17 at 18:35
  • 3
    @Null - I really don't know how well it would be received for a user to post a meta asking people to outvote an existing answer. Perhaps you could try it and see?
    – Valorum
    Sep 17 at 19:03
  • 2
    @Null - In my experience here any increased attention to the question will help the top voted answer more then it'll help the newer correct answer. The more you keep on editing or linking to it, the more this new answer will go from being "far behind" to being "very far behind".
    – ibid
    Sep 17 at 19:51
  • 2
    @ibid - That was my thought. Also any attempt to use meta to encourage voting in a particular direction could easily backfire.
    – Valorum
    Sep 18 at 9:32
  • 2
    Here is but one example where a late answer eventually overtook the accepted answer. It took more than two years, but eventually it got more votes.
    – Alex
    Sep 20 at 1:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .