Languages
Which languages the word lists cover, which the model covers, and what a clean answer means outside both.
Two different things run on your content, and they do not cover the same languages. It is worth knowing which is which before you decide what a clean answer means.
| What it reads | |
|---|---|
The model ("ai": true) | Whatever it reads, which is far more than the list below and includes scripts we have no word lists for at all. It is the part that understands a sentence rather than matching one. |
| The free detectors (every call) | The word and phrase lists below. Structural checks are language-independent and work everywhere: links, wallet addresses, card and IBAN checksums, repetition, image fingerprints. |
The word lists
English, Spanish, Portuguese, French, Italian, German, Catalan and Dutch. That covers
the vocabulary side of toxicity, harassment, hate,
sexual, violence, self_harm, scam,
the pile-on check, the minor
safety patterns, and the scam shapes: borrowed accounts, deals taken off the platform, rentals, jobs, romances, fake reviews and discriminatory job ads.
Say this one out loud: a language that is not on that list is not half-covered, it is not covered at all by the free path. The answer comes back clean, and a clean answer from a detector that has no words for the language is not evidence that there was nothing to find. On a site in a language we do not list, send
"ai": truefor anything that matters.
What you do not have to spell out
Within a covered language, the spellings are handled for you and you should not add them
to any list. f*ck, f u c k, fυck with a Greek
upsilon, ｆｕｃｋ, a zero-width space in the middle, leetspeak,
missing accents: all of it folds to the same word before anything is searched for. That
applies to your own policy terms too:
write competidor once, not eleven spellings of it.
The fold is careful in the direction that matters: it never transliterates Chinese, Arabic, Hebrew, Japanese or Korean, and it unmasks a word only when the word itself is mixing scripts. Otherwise ordinary Russian would come back reported as maximally disguised.
Telling us what to expect
locales is not what selects the word lists. Those are all searched, always,
because a comment in Portuguese arrives on a Spanish forum every day. It is what makes
two other checks possible: a wall of an unexpected alphabet, and a message in a language
your site is not in. Both exist to be quiet, with a fifteen-character alphabet floor and
a sixty-character floor for the language check, because a confident guess about a short
message is worse than no guess.
Your own words, and your own language
Anything the lists miss, you can add yourself, and it lands in the same place with the same normalisation:
| Policy terms | Your block, review and allow lists, per account, in any language. This is the answer for a community whose language is not listed above: the words that matter to you, in your words. |
| Topics | Subjects rather than harms, defined by the words that name them. |
On a self-hosted deployment the shipped lists themselves are two data files,
terms.php and phrases.php, both keyed by language, and
MODERATION_TERMS_PATH and MODERATION_PHRASES_PATH point at
replacements kept outside the repository. Adding a language is a column in those files,
not a change to any detector.