Video detection

The video detection API, POST /v1/harness/video-detection, finds the object you describe in every frame of a video. You describe the object in plain words, for example “Player with the ball” or “the red car that runs the light”, and get its bounding box in each frame.

How it works

The video is read at 5 frames a second. A tracking model follows every object that could match your description, for example every player. Once per second of video, Strom watches the last 4 seconds with those objects numbered and picks the one your whole description fits. Each frame gets that object’s box. When the description fits a different object later in the video, the box moves to it: in a football match, when the ball is passed, the box moves to the player who now has it.

Starting a job

A video takes a while to process, so the API works in two steps. First, start a job with the video and the description:

curl https://api.uprelic.com/v1/harness/video-detection \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "video": {"url": "https://example.com/match.mp4"},
    "description": "Player with the ball"
  }'

video.url must be a public http(s) URL. The video can be MP4, MOV, WebM or another common format, at most 500 MB and 10 minutes long. The response is a 202 with the job’s id:

{
  "id": "5f0c6b2e-8d1a-4c3e-9b7a-2e4f6a8c0d12",
  "status": "queued",
  "progress": 0.0,
  "description": "Player with the ball",
  "frames": null,
  "usage": null,
  "error": null,
  "created_at": "2026-11-01T09:30:00+00:00"
}

Getting the result

Then get the job with its id every few seconds, until status is done or failed. A minute of video takes about 30 seconds. While the job runs, progress goes from 0 to 1.

curl https://api.uprelic.com/v1/harness/video-detection/5f0c6b2e-8d1a-4c3e-9b7a-2e4f6a8c0d12 \
  -H "Authorization: Bearer YOUR_API_KEY"

{
  "id": "5f0c6b2e-8d1a-4c3e-9b7a-2e4f6a8c0d12",
  "status": "done",
  "progress": 1.0,
  "description": "Player with the ball",
  "frames": [
    {
      "t": 0.0,
      "box": {"left": 0.4103, "top": 0.2981, "right": 0.4702, "bottom": 0.5215},
      "confidence": 0.81
    },
    {
      "t": 0.2,
      "box": {"left": 0.4155, "top": 0.3012, "right": 0.4761, "bottom": 0.5254},
      "confidence": 0.81
    },
    ...
    {"t": 9.0, "box": null, "confidence": null},
    ...
  ],
  "usage": {"video_seconds": 20.0, "cost": 0.0293, "currency": "eur"},
  "error": null,
  "created_at": "2026-11-01T09:30:00+00:00"
}

When the job is done, frames has one entry for every 0.2 seconds of video, from the start:

• t: the frame’s time in seconds.
• box: where the object is in the frame. left and right are fractions of the frame’s width, top and bottom fractions of its height, measured from the top left corner. null when the object isn’t in the frame.
• confidence: how sure Strom is that the box is around the object you described, from 0 to 1. null when box is.

You can get a job’s frames for as long as your plan keeps request contents (7 days on Free). After that, frames and description are null.

The Python SDK starts the job and waits for it in one call:

from uprelic import Uprelic

client = Uprelic()
job = client.video_detection(
    description="Player with the ball",
    video="https://example.com/match.mp4",
)

for frame in job.frames:
    if frame.box:
        print(frame.t, frame.box.left, frame.box.top, frame.box.right, frame.box.bottom)

Cost and limits

• 0.10 USD per minute of video, billed by the second and charged in your account’s currency. A 20-second video costs about 0.033 USD. The cost is in usage.cost once the video is downloaded.
• A job that fails costs nothing.
• At most 10 jobs per account can be queued or running at once.

Errors

• 400: video.url isn’t an http(s) URL.
• 402: your balance is too low. A job also fails if the balance doesn’t cover the whole video once it’s downloaded.
• 404: there is no job with this id on your account.
• 429: you already have 10 jobs queued or running.
• A job that can’t be finished has status failed and the reason in error, for example when the video can’t be downloaded or read, or is longer than 10 minutes.