Check a conversation

Every other endpoint here judges one piece of content on its own. Some of the worst things a moderation system has to catch do not exist in a single message, and they are built that way on purpose.

A pile-on is thirty accounts each writing a rude but ordinary sentence. An approach to a child is a completely innocuous conversation that becomes something else over a week. No classifier reading one message can see either, however good it is, because what is wrong is a property of the sequence.

curl https://toxicfilter.com/api/v1/conversation \
  -H "Authorization: Bearer tf_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "author": "u1", "content": "menudo idiota eres, no tienes ni idea" },
      { "author": "u2", "content": "eres tonto o te lo haces" },
      { "author": "u3", "content": "qué inútil eres de verdad" },
      { "author": "u5", "content": "no tienes ni idea de esto, de verdad que no" }
    ],
    "locales": ["es"],
    "reference": "msg_9931"
  }'

The last message is the one being judged

Everything before it is context and gets no verdict of its own. Send up to fifty; past that, more history stops adding anything. author is your own opaque id for whoever wrote each one, the same kind of thing as actor and never a name. It is what makes "four different people" a question we can answer.

The message is also checked for everything a normal call is checked for: it is still text. Same price, same categories, same policy.

The thread is never stored. It is somebody's private conversation, and the same rule applies to it as to everything else here, if anything more so. What is filed is the verdict on the last message.

When the model reads a message

Every new message is a new call, so reading all of them with the model would make a forty message thread forty model calls. In a conversation the model reads when it has something to add: the free detectors found something to weigh, a lead type is half recognised, or it is the fifth message, when the whole thread is read again. A single /v1/text call is always read when you ask for it.

So a polite message with nothing recognisable in it, arriving between those points, is not read by the model, and the answer says which of the two happened rather than leaving you to work it out from the bill:

{
  "decision": "allow",
  "used_ai": false,
  "model": { "asked": true, "read": false, "why": "conversation_sampling" }
}

That call is billed as the cheap one, one credit, because that is what ran. Nothing failed: an outage is degraded and a different field. Both are described in the response.

Pile-ons

The same sentence, judged twice:

Counted by distinct author, so one furious person writing fifteen messages is one furious person. That is an argument, and an argument is not a pile-on.

Approaches to a child

This is the most dangerous thing here to get wrong in either direction, so the rules are structural rather than intentions:

The same shapes between two adults, with nobody's age stated, score 0.30 which is deliberately below the shipped review line of 0.35, so nothing new fires for anybody who has not asked for it. A platform with children on it lowers minor_safety to 0.25 in its policy and starts seeing them.

With an age stated it blocks, and it is the one safety category that does. The reasoning that keeps self_harm from ever blocking runs the other way here: the person harmed by deleting a self-harm post is its author, and the person harmed by delivering this one is a child.

Treat a hit as a person's job, immediately, and not as something to resolve automatically. And treat a clean answer as "nothing recognisable in these words": this catches shapes it has seen, in the languages it has patterns for, and an approach that does not take one of those shapes will pass it.

Borrowed accounts

"You apply with your account and go to the interviews, I do all the work, we split it 50/50." It reaches freelance platforms, job boards and anybody with a public profile, and it is fraud against two people at once: the client pays somebody they never hired, and whoever lends the account is the one who loses it. It rarely arrives in one message. It arrives as an introduction, then the account, then the split, so the conversation is read one person at a time, and the verdict lands on the message that completes the offer.

"With your account" and "in your name" are also how a helpdesk talks about an invoice, so ordinary words count only towards the whole offer. Where anybody says they are from is not a factor, and never will be: the shape of the proposal is what gives it away.

Deals taken off the platform

What a marketplace loses its commission to, and the step before most marketplace fraud: once the payment is outside, nobody has the protection the platform was giving them. Filed as spam, beside a chat handle, because it is the same behaviour.

Romances that turn into money

Affection, then another app, then an investment platform an uncle taught them about, or a customs fee for somebody on an oil rig. Affection and moving to another app are how people who genuinely like each other talk, so neither counts on its own or together. What makes the shape is money or an investment inside that conversation, from the same person: an investment beside affection, or a request for money beside affection and a story nobody can check, is scam and blocks by default.

All of these read the same person across the conversation when you send one, and the message alone on /v1/text. To tell what kind of lead is writing (free work for equity, no budget, a sales pitch) and discard the ones you do not want, see lead types.