How to Moderate User-Uploaded Images Before Saving Them to S3

The default AWS tutorial teaches you to upload images directly to S3, then process them. For media moderation, that ordering is exactly backwards. By the time you scan the image, it already lives on your infrastructure, under your bucket policy, with your retention rules, and for certain categories of content, that can be an actual legal problem.

Why "upload first, moderate later" is a problem

• CSAM. In most jurisdictions, possessing this content is a crime regardless of intent. "We were going to scan it" is not a defense. The window between upload and detection is a compliance hole.
• Non-consensual intimate imagery (NCII). Same category of problem, same legal risk.
• Brand safety. Even when the content is merely gross (not illegal), its existence in your storage, even for minutes, appears in audit logs, backups, and can surface in discovery.
• Attack surface. Malicious content in your bucket is a vector. Bucket misconfiguration incidents happen. Do not store what you have not approved.

Pattern 1: Pre-signed upload with moderation gateway

The cleanest approach: the client uploads to a gateway (Lambda + API Gateway, or a lightweight server), which runs moderation in memory and only forwards to S3 if approved.

// Lambda pseudo-code
export const handler = async (event) => {
    const imageBuffer = Buffer.from(event.body, 'base64');

    // Moderation in memory, never touches storage
    const result = await toxicfilter.moderateImage({
        data: imageBuffer,
        categories: ['nudity', 'violence', 'csam', 'weapons']
    });

    if (result.isBlocked) {
        return {
            statusCode: 422,
            body: JSON.stringify({
                error: 'Content not allowed',
                reason: result.reason
            })
        };
    }

    // Approved, now write to S3
    const key = `uploads/${uuid()}.jpg`;
    await s3.putObject({
        Bucket: 'prod-user-uploads',
        Key: key,
        Body: imageBuffer,
        ContentType: 'image/jpeg'
    });

    return { statusCode: 200, body: JSON.stringify({ key }) };
};

Pros: content never lands in S3 unless approved. Clean compliance story.

Cons: doubles the inbound bandwidth cost. Lambda has payload size limits (6MB sync, 256KB async); you need a streaming setup for larger files.

Pattern 2: Quarantine bucket

For larger files where streaming through a Lambda is impractical, use a two-bucket pattern:

1. Client uploads to uploads-quarantine (a bucket with strict access, no public read, short TTL).
2. An S3 event triggers a moderation worker.
3. If approved, the worker copies to uploads-approved (the bucket your app reads from).
4. The quarantine bucket has a lifecycle rule that deletes objects after 24 hours regardless.

Key details:

• The approved bucket is the only one your app queries. Presigned read URLs only come from there.
• The quarantine bucket has no public access. Ever. Not for convenience, not for debugging.
• Rejected content is deleted immediately on rejection, not waiting for the lifecycle rule.
• For illegal content (CSAM), you have a mandatory reporting path to NCMEC or equivalent. The moderation vendor should provide a documented flow for this.

Pattern 3: Hash pre-check (cheap and fast)

Before running full image classification, check the image hash against known-bad databases. PhotoDNA, PDQ, and similar perceptual hashes can identify content that has been flagged before in under 10ms. This is particularly effective for CSAM and known terrorist imagery.

The order of operations we recommend:

1. Hash check (~10ms) rejects known bad material immediately.
2. Fast classifier (~50ms) catches most novel content.
3. Deep classifier (~200ms) for anything the fast classifier was uncertain about.
4. Human review for the narrow band of content where the deep model was still uncertain.

What about EXIF and stripping metadata?

Moderation and metadata stripping are different concerns but often get bundled. Either way, you probably want to strip GPS coordinates and personal EXIF from user uploads before storage, for privacy reasons, not moderation. Do this in the same gateway step. Our preference: strip EXIF, re-encode to a canonical format, and then store.

The part most teams get wrong

They moderate the image they store. They do not moderate the thumbnails, resized variants, or cropped versions. If your pipeline generates 5 resized copies of every upload, you either need to moderate all of them (wasteful) or trust that the approved original implies all derivatives are safe (usually fine, but not always: aggressive crops can land on specific unsafe regions).

The safe default: moderate the original before generating any derivatives. If the original is approved, generate variants. If the original is rejected, no variants are ever created. Simple and auditable.