Marketplaces

Keep the deal on your platform, and the scammers off it

Listings, buyer and seller messages and reviews, checked as they are written. ToxicFilter catches the offer to pay outside, the rented account, the phone number in a listing and the replica sold as real, and tells you why in a sentence you can show the seller.

What goes wrong on platforms like yours

Deals that leave the platform

A buyer and a seller agree on your site and finish somewhere else: a phone number in the listing, a "message me on WhatsApp", a "pay me by transfer and we skip the fee". You lose the commission, and the buyer loses every protection you offer the moment the money moves outside.

Scams written like normal listings

A flat that does not exist with a deposit to pay first, a job that charges you to start, a freelancer renting out the account that passed your checks. Each of them reads like an honest post one sentence at a time, and only the shape of the whole offer gives it away.

Personal data published by accident

Sellers paste their phone, their IBAN or a card number into a listing without thinking. Refusing the whole listing for it loses a sale; publishing it exposes your user and makes you the one who did.

Replicas and reviews nobody wrote honestly

"1:1", "AAA grade", "wholesale, ships worldwide": counterfeits announce themselves in a vocabulary of their own. So do reviews written by the seller about the seller, or paid for with a discount.

What it decides, on content like yours

Judged right now by the engine the API uses, under the template for this kind of platform, with no model involved.

  1. 01 An offer to pay outside
  2. 02 A rented account
  3. 03 A phone and an IBAN in a listing
  4. 04 A replica sold as real
  5. 05 An honest listing

An offer to pay outside POST /v1/conversation

buyer Hi! Is the bike still available? I can pick it up on Saturday.

seller Yes it is. To avoid the fees, let's continue outside the platform: message me on WhatsApp and pay me directly by bank transfer.

block 25 ms
  • Proposes taking the payment outside the platform, where neither side is protected by it.

Taking the talk elsewhere and paying directly are each how honest people speak. Together, from the same person, they are the move off your platform, and the marketplace template refuses it.

The answer, abridged
{
  "decision": "block",
  "flagged": [
    "spam"
  ],
  "scores": {
    "spam": 0.7
  },
  "signals": [
    {
      "category": "spam",
      "score": 0.7,
      "reason": "Proposes taking the payment outside the platform, where neither side is protected by it.",
      "evidence": [
        "outside the platform",
        "message me on whatsapp",
        "pay me directly",
        "avoid the fees"
      ]
    }
  ],
  "model": {
    "read": false
  },
  "took_ms": 25
}

A rented account POST /v1/text

Hi! You attend the client interviews with your Upwork account, I do all the work, and we split the payment 50/50.

block 13 ms
  • Asks the reader to lend their account or identity while somebody else does the paid work, for a share of the money.

Lending the identity, doing the work and splitting the money: the whole offer, so it is refused. Any one of the three alone is a normal sentence and passes.

The answer, abridged
{
  "decision": "block",
  "flagged": [
    "scam"
  ],
  "scores": {
    "scam": 0.85
  },
  "signals": [
    {
      "category": "scam",
      "score": 0.85,
      "reason": "Asks the reader to lend their account or identity while somebody else does the paid work, for a share of the money.",
      "evidence": [
        "your upwork account",
        "i do all the work",
        "we split the",
        "split the payment",
        "50/50"
      ]
    }
  ],
  "model": {
    "read": false
  },
  "took_ms": 13
}

A phone and an IBAN in a listing POST /v1/text with redact: true

Cargo bike, barely used, 900 €. Call me on +44 7700 900123 or send the deposit to ES91 2100 0418 4502 0005 1332.

review 7 ms
  • Contains what looks like a phone number.
  • Contains 1 bank account number(s) (IBAN). Publishing one invites fraud against whoever owns it.

Held rather than refused, and the listing comes back with the phone and the IBAN masked, so the rest of it can still be published.

The answer, abridged
{
  "decision": "review",
  "flagged": [
    "personal_data"
  ],
  "scores": {
    "personal_data": 0.9
  },
  "signals": [
    {
      "category": "personal_data",
      "score": 0.5,
      "reason": "Contains what looks like a phone number.",
      "evidence": [
        "447•••••••23"
      ]
    },
    {
      "category": "personal_data",
      "score": 0.9,
      "reason": "Contains 1 bank account number(s) (IBAN). Publishing one invites fraud against whoever owns it.",
      "evidence": [
        "ES••••••••••••••••••1332"
      ]
    }
  ],
  "redacted": "Cargo bike, barely used, 900 €. Call me on [redacted] or send the deposit to [redacted].",
  "model": {
    "read": false
  },
  "took_ms": 7
}

A replica sold as real POST /v1/text

Rolex Submariner 1:1 replica, AAA grade, wholesale price, worldwide shipping. More models on my yupoo album.

block 7 ms

Counterfeits are a subject and not a harm, so they act only where a rule says so. The marketplace template says so; a sneaker forum teaching how to spot fakes is left alone.

The answer, abridged
{
  "decision": "block",
  "flagged": [
    "topic:counterfeit"
  ],
  "topics": {
    "counterfeit": 0.98
  },
  "signals": [],
  "model": {
    "read": false
  },
  "took_ms": 7
}

An honest listing POST /v1/text

Selling my road bike, 54 cm frame, serviced in May. Collection in Valencia or shipping at the buyer's cost.

allow 8 ms

Nearly every listing looks like this. It is settled by the free checks in about a millisecond, and nothing is paid for.

The answer, abridged
{
  "decision": "allow",
  "flagged": [],
  "signals": [],
  "model": {
    "read": false
  },
  "took_ms": 8
}

What it decides, on content like yours

Judged right now by the engine the API uses, under the template for this kind of platform, with no model involved.

An offer to pay outside POST /v1/conversation

buyer Hi! Is the bike still available? I can pick it up on Saturday.

seller Yes it is. To avoid the fees, let's continue outside the platform: message me on WhatsApp and pay me directly by bank transfer.

block 25 ms
  • Proposes taking the payment outside the platform, where neither side is protected by it.

Taking the talk elsewhere and paying directly are each how honest people speak. Together, from the same person, they are the move off your platform, and the marketplace template refuses it.

The answer, abridged
{
  "decision": "block",
  "flagged": [
    "spam"
  ],
  "scores": {
    "spam": 0.7
  },
  "signals": [
    {
      "category": "spam",
      "score": 0.7,
      "reason": "Proposes taking the payment outside the platform, where neither side is protected by it.",
      "evidence": [
        "outside the platform",
        "message me on whatsapp",
        "pay me directly",
        "avoid the fees"
      ]
    }
  ],
  "model": {
    "read": false
  },
  "took_ms": 25
}

A rented account POST /v1/text

Hi! You attend the client interviews with your Upwork account, I do all the work, and we split the payment 50/50.

block 13 ms
  • Asks the reader to lend their account or identity while somebody else does the paid work, for a share of the money.

Lending the identity, doing the work and splitting the money: the whole offer, so it is refused. Any one of the three alone is a normal sentence and passes.

The answer, abridged
{
  "decision": "block",
  "flagged": [
    "scam"
  ],
  "scores": {
    "scam": 0.85
  },
  "signals": [
    {
      "category": "scam",
      "score": 0.85,
      "reason": "Asks the reader to lend their account or identity while somebody else does the paid work, for a share of the money.",
      "evidence": [
        "your upwork account",
        "i do all the work",
        "we split the",
        "split the payment",
        "50/50"
      ]
    }
  ],
  "model": {
    "read": false
  },
  "took_ms": 13
}

A phone and an IBAN in a listing POST /v1/text with redact: true

Cargo bike, barely used, 900 €. Call me on +44 7700 900123 or send the deposit to ES91 2100 0418 4502 0005 1332.

review 7 ms
  • Contains what looks like a phone number.
  • Contains 1 bank account number(s) (IBAN). Publishing one invites fraud against whoever owns it.

Held rather than refused, and the listing comes back with the phone and the IBAN masked, so the rest of it can still be published.

The answer, abridged
{
  "decision": "review",
  "flagged": [
    "personal_data"
  ],
  "scores": {
    "personal_data": 0.9
  },
  "signals": [
    {
      "category": "personal_data",
      "score": 0.5,
      "reason": "Contains what looks like a phone number.",
      "evidence": [
        "447•••••••23"
      ]
    },
    {
      "category": "personal_data",
      "score": 0.9,
      "reason": "Contains 1 bank account number(s) (IBAN). Publishing one invites fraud against whoever owns it.",
      "evidence": [
        "ES••••••••••••••••••1332"
      ]
    }
  ],
  "redacted": "Cargo bike, barely used, 900 €. Call me on [redacted] or send the deposit to [redacted].",
  "model": {
    "read": false
  },
  "took_ms": 7
}

A replica sold as real POST /v1/text

Rolex Submariner 1:1 replica, AAA grade, wholesale price, worldwide shipping. More models on my yupoo album.

block 7 ms

Counterfeits are a subject and not a harm, so they act only where a rule says so. The marketplace template says so; a sneaker forum teaching how to spot fakes is left alone.

The answer, abridged
{
  "decision": "block",
  "flagged": [
    "topic:counterfeit"
  ],
  "topics": {
    "counterfeit": 0.98
  },
  "signals": [],
  "model": {
    "read": false
  },
  "took_ms": 7
}

An honest listing POST /v1/text

Selling my road bike, 54 cm frame, serviced in May. Collection in Valencia or shipping at the buyer's cost.

allow 8 ms

Nearly every listing looks like this. It is settled by the free checks in about a millisecond, and nothing is paid for.

The answer, abridged
{
  "decision": "allow",
  "flagged": [],
  "signals": [],
  "model": {
    "read": false
  },
  "took_ms": 8
}

What it looks for

Each a score of its own, with its own lines, and the reason in a sentence whenever one acts.

The template you start from

Pick it when you create a policy and these rules are written for you, ready to edit. Everything it does not mention keeps following our defaults.

  • Scam holds at 0.30 · refuses at 0.60
  • Spam holds at 0.35 · refuses at 0.65
  • Filter evasion holds at 0.35 · refuses at 0.70
  • Gibberish holds at 0.45 · never refuses
  • Counterfeit goods holds at 0.50 · refuses at 0.80
  • Personal data holds at 0.45 · refuses at 0.95

Set by the template

One call before you publish

Send the text with where it will appear, and act on the decision.

The request

curl https://toxicfilter.com/api/v1/text \
  -H "Authorization: Bearer $TOXICFILTER_KEY" \
  -d content="Hi! Is the bike still available? I can pick it up on Saturday." \
  -d surface=listing
The full reference →

A month, in numbers

Items checked
150,000
Read by the model
12,000
Images
8,000
Credits, about
314,000

Fits in the Max plan. See the plans →

A guide to moderating a marketplace

What to check, when to check it and how to set the rules, for listings, messages and reviews.

How to moderate a marketplace without slowing it down

Call the API before a listing or a message is published, and act on one of three answers: publish it, hold it for a person, or refuse it. Most of a marketplace's traffic is someone describing a bike or asking whether it is still available, and the free checks settle that in about a millisecond. The model only reads what they leave open, so moderation adds no noticeable wait to posting and costs a credit for most items. What arrives in review waits in a queue, in your dashboard or through the API, and a signed webhook tells your site when somebody approves it.

How to stop off-platform payments

The move off your platform is a pair: an invitation to continue somewhere else and an offer to pay outside. An offer to pay directly is ordinary on its own ("no commission on this price"), so a filter that fires on it is switched off within a week. ToxicFilter files the move under spam, beside the phone numbers and chat handles that usually come with it, and scores it by how much of the pair it sees from the same person, in one message or across a conversation. By default both halves together are held for review. The marketplace template is stricter: it also holds an invitation to take the deal outside the platform on its own, and refuses the full pair.

How to detect scam listings before they go live

Scams on a marketplace are written to look like normal posts, so no single word is the evidence. What gives them away is the shape of the offer: the owner who is abroad and wants the deposit before a viewing, the job that asks for money to start, the account lent to someone else to pass your checks. Each shape is a set of phrase families, and a listing is only flagged when it carries enough of them. Reporting a scam ("is this a scam?") never counts as running one.

Personal data in listings: mask it, do not refuse it

A seller who writes their phone number into a listing has not done anything wrong, and refusing the listing loses the sale. With redact, the answer carries the same text with the phone, the email, the IBAN or the card masked, and the rest can be published as it is. Cards and IBANs are checked by their checksum, and a Spanish DNI by its letter, so a sixteen digit order number is not reported as a card.

Counterfeits: a rule of your house, not a crime

Talking about replicas is fine; selling them is your rule. That is why counterfeits are a measured subject and not a category of harm: every answer carries the score, and nothing happens until a rule gives it a line. The marketplace template does, so a listing centred on a replica is refused, while a buyer asking whether a watch is authentic is left alone.

Choosing the thresholds for your marketplace

Start from the marketplace template, which lowers the lines for scams, spam and disguised text and adds one for counterfeits. Then try any change as a second policy running in parallel with the one in force: both verdicts are computed on your own traffic, only one acts, and the dashboard shows where they disagree before you switch.

Frequently asked questions

How do you stop users taking payments off the platform?

ToxicFilter reads each message, and the same person across a conversation, for the two halves of the move: taking the deal off the platform ("outside the platform", "off Upwork") and paying outside it ("pay me directly", "avoid the fees"). By default, both from the same person are held for review. The marketplace template is stricter: it holds an invitation to move outside on its own and refuses the full pair. A phone number or a chat handle in a listing is reported on its own, as personal data or spam.

Can it detect scam listings before they are published?

Yes, as long as you call it before publishing, which takes about a millisecond for a listing the free checks settle. It recognises the shapes of rental scams, job scams, romance scams and rented accounts in eight languages, and in a conversation it reads the same person across several messages, because these offers usually arrive in instalments.

What happens to a listing with a phone number or a bank account in it?

It is held for review rather than refused, and if you ask for redact the answer carries the same text with the phone, the IBAN or the card masked. Card and IBAN numbers are validated by their checksum, so an order reference with sixteen digits is not mistaken for a card.

Does it work for listings in Spanish and other languages?

The instant checks carry word lists and phrase patterns in English, Spanish, Portuguese, French, Italian, German, Catalan and Dutch. The model reads many more. Your own word lists work in any language.

How much does it cost to moderate a marketplace?

A check costs one credit, and when the model reads it the tokens it used are added, about eight in all for a message. Most listings and messages are settled by the free checks, so the bill follows how much of your traffic is ambiguous, not how much there is.

Do I have to explain to a seller why a listing was removed?

In the EU, yes: article 17 of the Digital Services Act requires a statement of reasons for every removal or restriction, whatever the size of the platform. ToxicFilter can write it for you with every blocked listing.

Try it on your own traffic

2,000 credits a month on the free plan, no card. Enough to send a week of your own content and see what it says about it.