Context and repetition

The signal that is not in the message: the same thing arriving forty-seven times, and who sent it.

Everything else here judges one piece of content on its own. That is the honest way to start, and it leaves the most reliable spam signal there is on the table: the same message arriving forty-seven times in six minutes. No classifier reading one message can see that, however good it is, because it is not a property of the message.

Repetition

Nothing to switch on and nothing to send. Every call is counted as it arrives, and when the same text has turned up enough times recently it becomes a spam signal in its own right:

{
  "decision": "block",
  "flagged": ["spam"],
  "signals": [
    { "category": "spam", "score": 0.85, "detector": "repetition",
      "reason": "This exact text has arrived 20 times recently on this account." }
  ],
  "context": { "repeats": 20, "similar": 20 }
}

Copies	Score
3–4	0.40	Below the line. Somebody posted twice in the wrong tab.
5–9	0.55	`review`.
10–19	0.72
20+	0.85	`block`. Somebody's tool.

similar counts the same message rewritten, with the link swapped, a word changed or an emoji added, which is what a campaign looks like as soon as whoever is sending it notices that identical copies get caught. It uses a fingerprint where a small change to the text moves a small number of bits, so it is a comparison rather than a search.

Counted per account, never across them. Two customers receiving the same newsletter are not a campaign. And only for content over about forty characters: a forum where two hundred people write thanks! is a forum, not an attack.

actor

Send your own id for whoever wrote the thing, on any endpoint. A user id, a hash, whatever you call them:

{ "content": "...", "actor": "user_8412", "reference": "comment_9931" }

On its own it is filed with the verdict, so you can ask later what this user has been posting. With reputation switched on in a policy, it also moves the line:

"context": {
  "actor": "user_8412",
  "repeats": 0,
  "similar": 0,
  "history": { "seen": 812, "blocked": 0, "reviewed": 3, "adjustment": 0.05 }
}

A user with eight hundred clean messages should not be held for a borderline one, and a user with three blocks today should not get the same benefit of the doubt. That is what every moderator does by hand and what a stateless API cannot do.

What a record is not allowed to do

This is the one thing in the product that is about behaviour rather than about content, and the categories are emphatic that we judge the second and not the first. So the boundary is written into how it works, not into a promise:

It never becomes a signal	There is no category for it, and it cannot appear in `flagged`.
It never blocks anything alone	It moves each line by at most 0.10, and clean content is clean at any line.
It never leaves your account	The same `actor` at another customer is another person to us, and we do not join them.
It is always reported	`adjustment` is in the answer, every time. A decision changed by something you cannot see is the kind of moderation this exists not to be.
It is off by default	A policy has to ask for it.

A record under twenty messages is ignored entirely. Judging somebody on five messages is judging them on nothing, and it would make the first thing a new user does decide the rest.

How this interacts with caching

Verdicts are cached, and a count that changes the answer has to be part of the question, or the forty-seventh copy of a message would be answered with the first one's verdict for ever. The counts go into the cache key in buckets, so the first few copies share an entry and crossing into 3, 5, 10, 20 or 50 produces a fresh verdict. You will see cached: false exactly when the number started to mean something new.