usertasksDocs
Sign in

Media (images, audio, video)

Media provider setup

Where to get an API key for OpenAI, Google Gemini, Azure AI Speech and Vision, AWS, Deepgram, AssemblyAI, ElevenLabs and Stability AI, which connector to add and what each can do.

The media steps call an AI provider with your own API key. Add the provider once as an integration (Integrations → Connectors, then the provider), paste the key, and choose that connection in the step's AI provider field. The key is stored encrypted and loaded only for the workspace's own runs. See Connectors.

A provider can only do what it supports — see the capability matrix. For example, OpenAI transcribes, describes images, checks safety, generates and edits images and speaks, but cannot return face or text boxes, so Redact Image needs Google Gemini, Azure AI Vision or AWS.

OpenAI

  • API key: platform.openai.com → API keys.
  • Connector: Integrations → Connectors → OpenAI (connector key openai).
  • The form asks for: API Key, Organization ID (optional), Project ID (optional), Default Model (optional), Default Max Tokens (optional).
  • Steps: Transcribe, Analyze Image, Analyze Video, Generate / Edit Image, Text to Speech.
  • Default models: transcribe: whisper-1, vision: gpt-4.1-mini, video: gpt-4.1-mini, image: gpt-image-2, tts: gpt-4o-mini-tts (change them per step with Model).
  • Notes: Transcription uploads up to 25 MB per request (longer media is split into chunks). Timed segments with whisper-1 (the default); speaker labels switch to gpt-4o-transcribe-diarize; gpt-4o-transcribe / gpt-4o-mini-transcribe return text without timings. No face or text boxes (redaction needs a provider with boxes). Video is analysed from sampled frames.

Google Gemini

  • API key: aistudio.google.com → Get API key.
  • Connector: Integrations → Connectors → Google Gemini (connector keys google_gemini, gemini).
  • The form asks for: API Key, Default Model (optional).
  • Steps: Transcribe, Analyze Image, Analyze Video, Redact Image, Redact Video.
  • Default models: transcribe: gemini-3.8-flash, vision: gemini-3.8-flash, video: gemini-3.8-flash (change them per step with Model).
  • Notes: Understands whole videos (summary, scenes with timestamps). Face, object and text boxes are located by the model (approximate) — check redactions before relying on them.

Azure AI Speech

  • API key: Azure portal → your Speech (or AI services) resource → Keys and Endpoint.
  • Connector: Integrations → Connectors → Azure AI Speech (connector key azure_speech).
  • The form asks for: Key, Region, Endpoint (optional), Default voice (optional).
  • Steps: Transcribe, Text to Speech.
  • Default models: tts: en-US-AvaMultilingualNeural (change them per step with Model).
  • Notes: Fast transcription API (synchronous; diarisation, word timings). Files under 500 MB / 5 hours.

Azure AI Vision

  • API key: Azure portal → your Computer Vision (or AI services) resource → Keys and Endpoint.
  • Connector: Integrations → Connectors → Azure AI Vision (connector key azure_ai_vision).
  • The form asks for: Key, Endpoint.
  • Steps: Analyze Image, Analyze Video, Redact Image, Redact Video.
  • Notes: Image Analysis 4.0 (caption, tags, objects, Read OCR with word boxes). Face boxes use the Face API detection on the same resource (detection only — no identification). Caption is available in some regions only.

AWS (Transcribe + Rekognition)

  • API key: AWS IAM → an access key for a user / role allowed transcribe:TranscriptionJob, rekognition:Detect, and s3:PutObject / GetObject / DeleteObject on the staging bucket.
  • Connector: Integrations → Connectors → AWS Transcribe + Rekognition (connector key aws_media).
  • The form asks for: Access Key ID, Secret Access Key, Session token (optional), AWS Region, Staging bucket.
  • IAM permissions: transcribe:StartTranscriptionJob, transcribe:GetTranscriptionJob, transcribe:DeleteTranscriptionJob; rekognition:DetectFaces, rekognition:DetectLabels, rekognition:DetectText, rekognition:DetectModerationLabels; s3:PutObject, s3:GetObject, s3:DeleteObject on the staging bucket. Media to transcribe is staged in the bucket under usertasks-media/ and deleted when the job ends.
  • Steps: Transcribe, Analyze Image, Analyze Video, Redact Image, Redact Video.
  • Notes: Transcribe runs as an asynchronous job: media is staged in your S3 bucket and deleted afterwards. Rekognition images up to 5 MB (larger images are downsized).

Deepgram

  • API key: console.deepgram.com → API Keys.
  • Connector: Integrations → Connectors → Deepgram (connector key deepgram).
  • The form asks for: API Key.
  • Steps: Transcribe, Text to Speech.
  • Default models: transcribe: nova-3, tts: aura-2-thalia-en (change them per step with Model).
  • Notes: Synchronous pre-recorded transcription with diarisation and word timings.

AssemblyAI

  • API key: assemblyai.com/dashboard → API Keys.
  • Connector: Integrations → Connectors → AssemblyAI (connector key assemblyai).
  • The form asks for: API Key, Data region (optional).
  • Steps: Transcribe.
  • Default models: transcribe: universal-3-5-pro (change them per step with Model).
  • Notes: Asynchronous: upload, then the step polls (long recordings run in the background).

ElevenLabs

  • API key: elevenlabs.io → Profile → API keys.
  • Connector: Integrations → Connectors → ElevenLabs (connector key elevenlabs).
  • The form asks for: API Key, Default voice ID (optional).
  • Steps: Transcribe, Text to Speech.
  • Default models: transcribe: scribe_v2, tts: eleven_multilingual_v2 (change them per step with Model).

Stability AI

  • API key: platform.stability.ai → API keys.
  • Connector: Integrations → Connectors → Stability AI (connector key stability_ai).
  • The form asks for: API Key, Default model (optional).
  • Steps: Generate / Edit Image.
  • Default models: image: core (change them per step with Model).

Choosing a provider

  • Meetings and calls: any transcription provider. Deepgram and Azure AI Speech answer in one request; AssemblyAI and AWS work as jobs (long recordings run in the background). Speaker labels: turn on Label speakers.
  • Redaction: Azure AI Vision and AWS return measured face and text boxes; Google Gemini's boxes are located by the model and approximate — check the results.
  • Whole-video understanding: Google Gemini watches the video itself; other providers look at sampled frames (needs FFmpeg).
  • Images and speech: OpenAI or Stability AI generate images; OpenAI, Azure AI Speech, Deepgram and ElevenLabs speak.