Does SFF need/have an objective way of scoring tags?

Question

I've been noticing a lot of meta questions asking whether or not some tag should be created/destroyed are including a "tag score sheet" copied from this meta post. That score sheet seems to have been copied from another Stack Exchange site and slightly modified for the effort described in that question. However, the logic for coming up with that specific score sheet or if it should be used in perpetuity was never the intention of that original meta question.

My question here is; does the Science Fiction and Fantasy Stack Exchange need an objective way of scoring tags, and if so, what is that scoring method?

I would expect answer to this question to be of the form:

Yes - this is why it is necessary. Here is a proposed scoring sheet and how to use it.

No - this is why it is unnecessary.

I'd also point out that no mention of scoring a tag is mentioned in the General tagging practices (admittedly, an old question).

Answer 1

No. It's unnecessary.

1. It's nearly impossible to make objective. We've already seen where our current scoring method gets different scores based on different users. Coming up with a more objective system isn't likely.
2. It's tedious. Tagging was never meant to be this much of a pain in the ass to make decisions on. Unless a tag is atrociously bad or almost unused, it may as well stay.
3. It's not likely to be effective as the site grows, as evidenced by earlier tagging policies. It would need to be revisted in the future, making it a bit pointless (possibly by actively undermining future tagging policies, where we burninate or prevent tags we later believe would be useful).
4. Any tag we prevent from being made or burninate actually destroys our ability to track the actual usage of tags, to develop policies based around what our users naturally feel is an effective way to tag questions.
5. It discourages tagging in general more than it encourages good tagging. (My opinion, anyway)

Answer 2

That checklist is pretty sketchy in general, but might perhaps be applied successfully to a very strictly defined subject.

Sci-fi and fantasy is not a very strictly defined subject.

In particular, question #1 is worthless: there are experts in every topic imaginable if you look hard enough; at best then, a "no" answer to #1 is useful for identifying a lazy judge whose further opinions you can safely ignore.

Test #7 is also worthless; there is actual data for this, although it isn't necessarily public. If you were going to base a decision on it, it would be worth pulling real numbers instead of trying to guess "objectively".

The second-most damning problem with this checklist is that it repeatedly confuses tags with topics... On some sites, those sets overlap quite a bit, but not this one.

But the single most damning problem with that list is that it has nine distinct tests. Ain't nobody got time for that.

see also: When to burninate

Answer 3

Yes - It is necessary.

• Tagging is hard and an inexact science
• Having some well thought out rules/guidelines would help make it a little easier
• Having a standard list of questions to ask about a tag helps jump-start the discussion as to whether or not a tag should be kept

Here is a proposed tag score card and how to use it:

Tag Scoring (blatantly stolen and slightly adapted from Chemistry.SE's tagging discussions)

For each tag, answer the following questions with a "Yes", "No", or "Sometimes" e.g. maybe it's "yes but only in in some cases". (I've modified this a bit based on discussions with it's author based on Chemistry's useage patterns vs. ours; please feel free to leave feedback in the comments on this, as it's just a proposal!)

1. Does it make sense to be an "expert* in proposed"?
2. Does it make sense for a question to be tagged only with proposed"?
3. Does proposed have a single, universally-unambiguous meaning?
4. Is proposed likely to be used correctly just based on it's name?
5. Are there "enough" (> 15) but not "too many" (> 10% site-wide) questions that qualify for proposed?
6. Are people like to use proposed to find questions to answer?
7. Are there likely some users (be objective!) who will favorite or ignore proposed?
8. Could proposed be reasonably used to feed questions to a specialized chat room?
9. Can proposed be used to search for questions (for any reason) in a way that keyword searching cannot accomplish?

^* _{The meaning of "expert" in the context of SF/F may not be obvious. For our purposes, being an "expert" in a tag means: 1) if the only thing someone knew about a question was that it was tagged proposed, that would be enough information for them to feel confident about trying to answer it, and 2) it's reasonable for someone to have a lot of knowledge about proposed as a separate subject, and not only because it's a subset of a larger topic (e.g. would someone know a lot about light-sabers on their own, only because they're an expert on everything about star-wars)}

For each "Yes", score +2. For each "Sometimes", score +1; for each "No", score -1. The goal of this score is to identify tags that are "multi-purpose"; since different people use tags for different things, we should strive for tags that cover all the bases.

Very roughly speaking, tags that score > 12 are "good" tags, tags that score < 8 are "terrible" tags, others are likely good but may need some clarification/renaming/etc.

Quote taken from: Tag Wars Episode I: Harry Potter and the Tag Cleanup