Media & Evidence package

User-generated media moderation

Uploads arrive by API: AI safety analysis of images and videos, threshold rules that auto-approve or reject clear cases, a moderator queue for the rest, and the uploader is told.

Connects to Google Gemini

OfficialHeadless / APIIntermediate~20 min setupv1.0.0
Download .utpkg 16 KBHow to install

About this package

What it does

User-generated media moderation starts when your site or app posts an upload to the workflow: its id, a download URL, whether it is an image or a video, the uploader and the caption.

Analyze Image (or Analyze Video for clips) describes the upload and answers a structured moderation question: a risk score from 0 to 1, the top category (nudity, violence, hate, harassment, self-harm, drugs, weapons, spam), a possible child safety concern, whether people are visible and a one-sentence reason. The Moderation thresholds decision table auto-approves low risk, auto-rejects high risk and sends borderline items, self-harm and any child safety concern to a person.

Moderators get a task with the AI reason clearly marked as a triage aid and approve or reject with the guideline that applies. Your platform is called to publish or hide the upload (placeholder REST API), the uploader is emailed with how to appeal, and every decision is logged with a one-year retention policy.

Headless / API

Headless: your platform calls the start API (or an inbound webhook) for each new upload.

Another system starts it over REST or an inbound webhook. No UI is needed; people only appear where a decision needs a human.

The flow

Step by step

  1. 1

    Upload received

    Your platform posts the upload id, media URL and type, uploader and caption.

  2. 2

    AI safety analysis

    Analyze Image or Analyze Video returns a description and a structured verdict: risk score, category, child safety concern, people visible, reason.

  3. 3

    Thresholds

    The Moderation thresholds decision table approves low risk, rejects high risk and queues the rest for a person.

  4. 4

    Moderator queue

    Borderline uploads become tasks: the moderator approves, or rejects with the guideline.

  5. 5

    Publish or hide

    The platform API publishes or hides the upload and the uploader is emailed (with how to appeal).

  6. 6

    Log

    Every check and decision is logged with who decided and why, under a one-year retention policy.

Upload moderation — An image or video upload from your platform is checked by AI for safety, the moderation thresholds approve or reject the clear cases, borderline ones go to a moderator, the uploader is told and the platform publishes or hides it. The same canvas you get in the designer after installing.

Forms

Inputs and task forms

The inputs a caller passes when it starts a run, and the forms people complete when the process needs a decision.

Start inputs

Upload moderation

Passed by the calling system

Upload id * upload_id · textThe id of the upload in your platform
Media URL * media_url · urlhttps://…Where the image or video can be downloaded (signed URL)
Media type * media_type · select
imagevideo
image or video
Uploader email uploader_email · emailname@company.comWho to tell about the decision
Uploader name uploader_name · textShown in the emails
Caption caption · textThe caption or title the uploader gave
Channel channel · textWhere it was uploaded (community, reviews, profile …)

User Task

Moderate the upload

{{priority}}: {{upload_id}} ({{verdict.category}}, risk {{verdict.risk_score}}) · 2-column grid · own header and theme

Guideline (for a rejection) guideline · choice
Nudity or sexual contentViolence or goreHate or harassmentSelf-harmIllegal goodsSpam or scam+1
Note to the uploader note · textarea
ApproveReject

Headless

Call it from your systems

Every workflow can be started over REST, MCP or the CLI, whatever its trigger, with the sample input below. Runs start straight away on the published version; replace <workflow id> with the id the install created.

REST

shell
curl -X POST "$USERTASKS_URL/api/v1/workflows/start" \
  -H "Authorization: Bearer $USERTASKS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"workflow_id":"<workflow id>","variables":{"upload_id":"UP-20261030-0042","media_type":"image","uploader_email":"river.example@example.com","uploader_name":"River","caption":"My weekend poster design","channel":"community"}}'

CLI

shell
usertasks workflows inputs <workflow id>          # the start form: keys, types, required
usertasks workflows run <workflow id> --input upload_id=UP-20261030-0042 --input media_type=image --input uploader_email=river.example@example.com --input uploader_name=River … --wait

Inbound webhook

shell
usertasks webhooks create <workflow id>          # prints the hook slug and secret once
curl -X POST "$USERTASKS_URL/api/webhooks/inbound?hook=<slug>" \
  -H "Authorization: Bearer <webhook secret>" -H "Content-Type: application/json" \
  -d '{"upload_id":"UP-20261030-0042","media_type":"image","uploader_email":"river.example@example.com","uploader_name":"River","caption":"My weekend poster design","channel":"community"}'

What's included

10 assets, ready to adapt

TypeNameDetails
Email templateUpload not publishedTells the uploader why their upload was not published and how to appeal.
Email templateUpload publishedTells the uploader their upload is live.
Data tableModeration logEvery upload checked: AI risk, categories, the decision, who made it and why.
IntegrationContent platformYour site or app that holds the uploads: approved items are published, rejected ones hidden. Replace with your CMS, community platform or storage API.
IntegrationGoogle Gemini (media AI)Media AI provider for transcription, image and video analysis and finding what to redact. Paste your Gemini API key on install (aistudio.google.com). To use OpenAI, Azure AI or AWS instead, point the media steps at that integration.
API referencePublish uploadPOST /uploads/{{upload_id}}/publish — Publishes an approved upload.
API referenceReject uploadPOST /uploads/{{upload_id}}/reject — Hides a rejected upload and records the reason.
Decision tableModeration thresholds5 rules — Automatic approve and reject bands for the AI risk score, and categories that always go to a person. Tune them to your community guidelines.
Retention policyModeration log: 1 yearModeration decisions are kept one year for appeals and transparency reports, then purged.
WorkflowUpload moderation22 steps — An image or video upload from your platform is checked by AI for safety, the moderation thresholds approve or reject the clear cases, borderline ones go to a moderator, the uploader is told and the platform publishes or hides it.

Moderation log · 1 sample rows

UploadUploaderTypeChannelRiskTop categoryAI reasonDecisionDecided byReasonDecided

Requirements

What you connect it to

Credentials never travel in a package. The install wizard asks for what your environment must provide, or installs it incomplete until you finish setup.

Needs

  • Media AI provider — An API key for a media AI provider (Google Gemini by default; OpenAI, Azure AI, AWS and others can be swapped in) that transcribes, analyses and finds what to redact.
  • REST API — A REST API integration: set its base URL and credentials on install.
  • Assignees — People in your workspace to receive the User Tasks (picked on install).
  • Inbound webhook (optional) — An inbound webhook created for the workflow, so other systems can start it.
  • Email (optional) — Email delivery configured for the instance (EMAIL_PROVIDER), used for notifications.

After installing

  1. Google Gemini (media AI): paste your Gemini API key. (For image safety categories from a moderation endpoint, add an OpenAI or AWS integration and choose it on Analyze Image with the Safety feature.)
  2. Content platform: set the base URL and token of the API that publishes and hides uploads, and adjust the Publish upload / Reject upload paths.
  3. Assignees: pick the moderator (escalates to their manager after 8 hours).
  4. Tune the Moderation thresholds to your guidelines. Route possible child safety issues to your legal reporting process: this package only queues them for a person.
  5. Create an inbound webhook (or call the start API) from your platform for each upload; media URLs must be reachable from UserTasks.

Asked for on install

  • Integration “Content platform”Required

    • Tokenconfig.auth_config.token
    • Base urlconfig.base_urldefault https://content-platform.example.com/api/v1· optional

  • Integration “Google Gemini (media AI)”Required

    • Key valueconfig.auth_config.key_value
    • Base urlconfig.base_urldefault https://generativelanguage.googleapis.com· optional

  • Workflow “Upload moderation” assigneesRequired

    • User for step “Moderate the upload”nodes.11.config.assignee_id

Install

Install it in your workspace

Works on UserTasks cloud and on any self-hosted instance. Installing creates the assets above; nothing runs until you complete setup and start it.

In the app

  1. Download ugc-media-moderation-1.0.0.utpkg.
  2. Open Automations → Packages and drag the file onto the page (or choose Import package). Browse gallery there lists every gallery package without downloading by hand.
  3. Pick the file. The wizard verifies the checksum, every file hash and the signature, and shows the readme.
  4. Review the changes (what is created) and the data rows.
  5. Fill in the requirements, or tick Keep incomplete and finish later with Complete setup.
  6. Done: the workflow is published and active once setup is complete.

With the CLI

shell
curl -fLO https://usertasks.com/gallery/ugc-media-moderation-1.0.0.utpkg
usertasks packages inspect ugc-media-moderation-1.0.0.utpkg       # offline: checksum, hashes, signature
usertasks packages install ugc-media-moderation-1.0.0.utpkg --dry-run
usertasks packages install ugc-media-moderation-1.0.0.utpkg       # prompts for each requirement
usertasks packages install ugc-media-moderation-1.0.0.utpkg --keep-incomplete --yes   # in CI; finish with:
usertasks packages complete ugc-media-moderation -f values.yaml

Verify the download

File SHA-256
f7b807aa3c782696f268af78e3272c84bfb7ddfc4a848bed8603a1f5fc88387e
Package checksum (what is signed)
sha256:6c24e21f21a3067dcebe0f57ba66f7d35ff00f96022dad0c09a706daf7a28178
Signer
UserTasks Gallery · 49:f3:04:90:47:02:4d:07:28:61:3d:dc:bb:aa:db:11

shasum -a 256 ugc-media-moderation-1.0.0.utpkg should print the file SHA-256. How gallery packages are signed.

Versions

Version 1.0.0

  1. v1.0.02026-10-30First release.

Installing a newer version upgrades in place: local credentials and requirement values are kept, and changes you made are flagged first. How packages work

More packages

Related

Browse the whole gallery