Do we want Smoke Detector to auto flag spam?

Question

TL;DR: there's a system called "Smoke Detector" that has been automatically flagging spam here for the past year. You can view a list of posts that have been flagged by clicking this text (warning: some vulgar / offensive content if you click through to the flagged posts in that list.) Right now, the folks running this system are trying to decide if they should delete this sort of thing even faster - so if you think you might have an opinion on this, read that list, form your opinion, then come back here and share it.

---

As some of you may have seen already there is this post on mother meta. To save a bad explanation by me here's the TL;DR from the post.

Charcoal is the organisation behind SmokeDetector. Since January 2017, we've been casting up to 3 flags automatically on posts that our systems are confident are spam. We'd like to increase that to 5 automatic flags to reduce the time spam spends alive on the network.

As it stands this proposal is network wide but there has been some discussion with the Charcoal team whether this can be on a per site basis. Therefore, this is a preemptive discussion on whether we'd like auto flagging of spam on our site and how we'd want it handled. The following are the questions we'd need to address if we are able to change things on a per site basis.

1. Do we want auto spam flagging?
2. How many auto spam flags would we want?
3. Who should be able to auto spam flag? (Anyone, active users, users >N rep, SmokeDetector account, etc.)

---

_{If you have anymore questions to discuss in relation to this feel free to edit them into the list though if there are any answers already pinging them for an update would be a nice courtesy.}

Answer 1

Clarifying Smokey

Let me start with a very brief summary of what Smoke Detector does, since there's been some confusion over this. (Charcoal people, please correct me in comments if I get anything wrong.)

• Posts which contain certain blacklisted content (swearwords, particular link patterns, etc.) are reported into chat for user review. Users then decide whether or not to flag based on their own appraisal of the reported post. Reports have a relatively high false positive rate: in the last 12 months on SFF, ~15% of reported posts have been false positives, or ~25% counting posts that were delete-worthy but not spam/abusive. Smokey tries to optimise for finding as much spam as possible rather than ensuring all reports are spam.

• Posts which are almost certainly spam/abusive are autoflagged by the Smokey bot, using the accounts of users who've signed up as autoflaggers. This "almost certainly" is defined by tweaking the blacklist parameters. Auto-flagging has a low false positive rate: in the last 12 months on SFF, ~2% of reported posts (a single one among 47) have been false positives. To be auto-flaggable, a post needs to be far more 'spammy/abusive' than it does to be reportable.

• There's a proposal currently open to increase the autoflags per post from 3 (the current number) to 5 (one less than the number needed to nuke a post). If this happens, the parameters would be tweaked even further, to the point where a post must be (at least) 99.99% likely to be spam before flagging, rather than the current 99.75%. Thus, this would have a very low false positive rate, to balance the greater risk of accidentally nuking a post which shouldn't be nuked.

I've seen some users getting them mixed up. Hopefully this clarifies the differences between reporting, autoflagging, and 5-flag autoflagging. These are three separate things, with three different thresholds required to make them happen (if the latter even happens at all).

---

Getting to the actual question

It's possible for the Smokey devs to turn off any or all of the Smokey behaviours on a per-site basis. So I'm going to take this meta as the canonical "what do we want Smokey doing on our site?" post, and answer it as such. This isn't an official response from the mod team, but my own personal thoughts - please let me know in comments what you all think.

• Do we want Smokey reports for SFF?

  I'd say YES. SFF doesn't get much weird content which would be considered spam/abusive on other sites but is OK here, so there's not much danger of non-SFF Smokey people manually flagging things wrongly. Smokey reports help our site to get rid of spam quicker (even though it's not always actual site users getting rid of it for us). Making people aware of potential spam or abusive posts is a benefit for the site, and I can't really see any major downsides. Occasionally over-zealous flaggers might cast wrong flags, but site mods can decline these as necessary.

• Do we want Smokey autoflags for SFF?

  I could go either way on this one. On the one hand, there are moral/philosophical objections to letting a bot clean up our site for us - flags should be cast by people, whether SFF people or Smokey people (or both) - and SFF has more than enough active users to manually flag spam and nuke it in short order anyway. On the other hand, it's been working pretty well here for a year or so - I bet most of the SFF community didn't even notice it, or indeed notice the spam posts it eliminated for us - and even autoflagged posts still have to be reviewed by humans in order to be nuked, since the maximum number of autoflags currently cast is 3.

• Do we want Smokey 5-autoflags for SFF?

  As far as I understand, the motivation for enabling 5-autoflagging is to speed up the deletion time of spam. On some sites, with low activity and few flaggers, that might actually be a noticeable gain, but here on SFF, we have enough actively flagging users that no clearly spam/abusive post is going to last long anyway. I'm reasonably sure that if you go through this list and look at the posts which took longest to be deleted, most will have some post-specific issues or ambiguity. So I can't see any real benefit for our site in 5-autoflagging.

  The risk, however, is clear. Even with all the safeguards in the book, and a 99.99%+ likelihood of spam, it's always possible that some false positive will slip through and get autoflagged. That's not so bad now, when 3 human flags are still required to nuke it, but a 5-autoflagged post only needs one person flagging incorrectly to delete and lock it. And unfortunately we can't rely on people flagging sensibly - you should see some of the flags I've had to decline on this site ;-) Even without the risk of a single bad human flag, 5 autoflags are enough to get a post's score down to -5 and hide it from the front page of the site, making it harder for people to find in order to reverse the damage.

  With the risks outlined above, and the lack of tangible benefits, I'd have to say NO.

Answer 2

To give you a bit of background, I'm the user that posted the current post about increasing the number of automatic flags, the previous one a year ago, and the automated comment flagger over on Stack Overflow. That should give you the idea that I'm in favor of automation.

1. Do we want auto spam flagging?

Yes. For the same reason that you want spam protection on your email account. You really don't want to see it and when you do, it's an annoyance. Charcoal has automatically flagged thousands of posts, accurately, across the entire network.

2. How many auto spam flags would we want?

Honestly, I'd love to have all of this handled before it even makes it onto the site. We can't get that early in the process though, so the next best thing is to remove it as quickly as possible. There is no benefit to having spam sit around while we wait for users to come around and flag it. Automatic flags offer to take most of that work on. By allowing 5 automatic flags to the most egregious of posts, we can effectively cut the time a piece of spam survives on the site to a handful of seconds.

3. Who should be able to auto spam flag? (Anyone, active users, users >N rep, SmokeDetector account, etc.)

Right now, anyone who has permission to flag a post is eligible to auto-flag the post. That means the user is trusted, at some level, on the network. There are no other restrictions across the network on who can flag. It's based on reputation level of the user.

---

Bots, and automation of certain tasks, are not the problem. Poor oversight is a problem. SmokeDetector has oversight. It has a chat room dedicated to monitoring it. It has an open dashboard for any one to look at.

As was nicely pointed out in the comments by Valorum:

For the record, we've now spent more time discussing this bot than we spent deleting every single spam post on SFF:SE over the past year.

They are correct. This site doesn't receive a ton of spam. That's great for the site. However, it's also important to point out that a lot of that spam that has been removed has been removed in seconds to minutes thanks to the existing effort of automatic flags. That should continue.

Answer 3

Oddly, while I have argued for the flagging (And intend on using it on the sites that I moderate), I am not 100% convinced that it is needed on a site such as SFF. (Yes, this is a change from my earlier stance.)

There are enough very active, high rep users here that the flagging and deletion of spam is taken care of in extremely short order. Increasing the chance of false positives, I'm just not sure is worth the risk.

Now, on sites with lesser traffic (and higher relative spam incidence), such as fitness and health, I believe it is of much higher value. So while I still support the initiative, I am not 100% sure it is needed on the really high traffic sites with a broad high rep user base.