How to Document Your Moderation Decisions for an Audit

At some point a regulator, a litigant, or a journalist is going to ask you the same question: "How did you decide to remove that content?" If your answer is "the AI flagged it", you have not answered the question. You have introduced three more.

This post is a checklist for logging and documentation that will hold up under scrutiny. It is the difference between a two-hour conversation with your lawyer and a two-year legal process.

What "audit-ready" actually means

It means that for any specific moderation decision you have ever made, within your retention window, you can produce:

1. The exact content that was moderated.
2. The decision taken (block, remove, warn, queue, allow).
3. The rule or policy that applied.
4. The evidence that triggered the decision: model category, confidence, user report, and so on.
5. Whether a human was involved and, if so, who and when.
6. The notification sent to the affected user.
7. The outcome of any appeal.

Reproduce these seven items and you can answer almost any regulator. Skip any one of them and you cannot.

The moderation decision log, field by field

A minimum viable schema:

decision_id           UUID
content_id            FK to the content (post, message, etc.)
content_snapshot      the content as it was at moderation time
user_id               author of the content
decided_at            timestamp, UTC
decision              enum: allow, warn, restrict, remove, queue, escalate
policy_reference      versioned policy ID the decision maps to
model_provider        e.g. "toxicfilter", "perspective", "internal"
model_version         exact model version
categories            array of categories triggered
confidence            float per category
automation_level      enum: auto, human_review, human_override
reviewer_id           nullable, FK to moderator
reviewer_note         optional free text from the moderator
notification_sent_at  timestamp of reasoned statement to user
appeal_id             nullable, FK to appeal record

Two fields deserve extra attention:

content_snapshot

Content changes. Users edit comments. Moderation targets the content as it existed at decision time. If you only log a pointer, you cannot reconstruct what you were looking at. Store the snapshot inline, or in immutable storage with the decision.

policy_reference

Policies change too. "Removed for hate speech" under v3 of your policy is different from "removed for hate speech" under v5. Version your policy like code, and reference the specific version each decision is grounded in.

Explainability is not optional anymore

Several frameworks now require "meaningful information about the logic involved" when automated decisions affect individuals:

• GDPR Article 22 on automated decision-making.
• DSA Article 17 on statements of reasons.
• EU AI Act transparency requirements for high-risk systems.

"Our AI returned a score of 0.87" is not meaningful. "Our hate-speech classifier flagged your message as targeting a protected group, primarily based on the phrase [quoted phrase]" is. This means your model layer has to produce not just a score but a reason token or span attribution, meaning the specific part of the content that drove the decision. Modern moderation APIs (ToxicFilter included) return these by default; older ones do not.

What to keep, for how long

Retention is a trade-off. Longer retention helps with audits and repeat-offender tracking; shorter retention reduces privacy risk and data-breach exposure. A reasonable default:

Document the retention policy. Automate deletion. Nothing is more damaging in an audit than "we meant to delete these but did not".

The reasoned statement to the user

Under the DSA, every enforcement action needs a written explanation to the affected user. A template:

Hi [user],

On [date] we removed your [post/message/comment] for the following reason:
[specific policy violation, e.g. "targeted harassment of another user"].

This decision was made by [our automated moderation system / a human reviewer / our automated system and confirmed by a human].

If you believe this was a mistake, you can appeal here: [link]. Appeals are reviewed by a human and resolved within [X] days.

The content that was removed: [quoted content or link to it].

The full policy is available here: [link to versioned policy].

This one template, filled in correctly and sent consistently, satisfies most regulatory transparency obligations.

Test your audit posture

Once a year, have someone (internal counsel, an external consultant, even a capable engineer outside the moderation team) run a "mock regulator" exercise:

1. Pick 20 moderation decisions from the last quarter at random.
2. For each, ask: what was removed, why, by what, under which policy, was the user told, did they appeal, what happened?
3. Time how long it takes to produce the answer. Note what was missing.

If the exercise takes more than an afternoon or leaves any of the seven items unanswered, you know exactly what to fix. Better to find it yourself than have a regulator find it for you.