Images, Video and Object Detection

Overview

All four Torki models understand images and video. They can describe and compare them, read the text in them (signs, documents, screenshots, subtitles), answer questions about them and find objects with their positions. Send images and video in user messages; they can't go in system or assistant messages.

Images

client.chat.completions.create(model="torki-orbit", messages=[{"role": "user", "content": [
    {"type": "text", "text": "What is in this photo?"},
    {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
]}])

The url can be:

• A public link (https:// or http://) on the standard port, 80 or 443. Torki downloads it: 20 MB at most, within 30 seconds.
• A data URI, such as data:image/png;base64,… — 20 MB at most once decoded.

To send a local file as a data URI in Python:

import base64

with open("photo.jpg", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()

image_part = {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64," + b64}}

Formats: JPEG (phone photos included), PNG, WebP and GIF (the first frame of an animation), up to 25 megapixels. Torki turns the image upright, removes its metadata and resizes it to fit detail:

detail	Longer side at most	Tokens (about)
low	512 px	200–260
auto (default)	1,536 px	up to 2,300
high	2,048 px	up to 4,100

An image costs about one input token per 32 × 32 pixels, billed at the model's input price and counted in usage.prompt_tokens. A 640 × 480 photo is about 300 tokens; a phone photo at the default detail is about 1,750 tokens, about $0.0005 on Pulse.

Video

client.chat.completions.create(model="torki-orbit", messages=[{"role": "user", "content": [
    {"type": "text", "text": "What happens in this video? Quote any on-screen text."},
    {"type": "video_url", "video_url": {"url": "https://example.com/clip.mp4"}},
]}])

A video_url part is a Torki addition to the OpenAI format; in the OpenAI Python SDK, pass the message as a plain dictionary, as above. It takes a public link or a data URI (data:video/mp4;base64,…).

• Formats: MP4, MOV, M4V, WebM, MKV and AVI. Up to 10 minutes and 100 MB, with frames up to 8,192 pixels on a side.
• Size of a request: a whole request can be at most 50 MB, so a video sent as base64 can be about 35 MB. Send a link for anything larger.
• How it is read: the model sees up to 8 frames sampled evenly across the video, each labelled with its timestamp, such as [Frame 3/8 at 1:15]. Add "frames": 1 to 8 inside video_url to use fewer.
• Sound: the audio track is not used.
• Cost: a frame is at most 768 px on its longer side, about 340 input tokens for a 16:9 video (8 frames ≈ 2,700 tokens) and up to about 580 for a square one.

Object detection

There is no separate detection endpoint: ask the model for objects and their boxes as JSON. Coordinates come back on a 0–1000 scale relative to the image (x from the left, y from the top), whatever the image's size, so multiply by your image's width and height and divide by 1,000 to get pixels.

import json

resp = client.chat.completions.create(model="torki-orbit", temperature=0, messages=[{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "https://example.com/street.jpg"}},
    {"type": "text", "text": 'Detect every car and person. Return only JSON: '
                             '[{"label": str, "bbox_2d": [x1, y1, x2, y2]}]'},
]}])
text = resp.choices[0].message.content
boxes = json.loads(text[text.index("["): text.rindex("]") + 1])
width, height = 1920, 1080                      # your image's size in pixels
for b in boxes:
    x1, y1, x2, y2 = b["bbox_2d"]
    print(b["label"], [x1 * width / 1000, y1 * height / 1000, x2 * width / 1000, y2 * height / 1000])

Name the objects you want for the best results, and set a max_tokens that fits the number of boxes you expect (about 20 tokens a box).

Content moderation

Images and video frames are screened by Torki's content moderation, like text. A refused image returns 400 content_policy_violation and is not charged. In a video, a few flagged frames are left out (the model never sees them) and the rest is answered; if more than half the frames are flagged, the video is refused. If the screening service is busy, the API returns 503 overloaded: retry after a few seconds. Nothing unscreened ever reaches a model.

Errors

• 400 invalid_image or image_too_large — not a JPEG, PNG, WebP or GIF Torki can read, damaged, over 20 MB or over 25 megapixels.
• 400 invalid_video or video_too_large — not a supported video, unreadable, longer than 10 minutes or over 100 MB.
• 400 invalid_media_url or media_fetch_failed — the link isn't a public http(s) address on port 80 or 443, or downloading it failed or took over 30 seconds.
• 400 too_many_images — more than 8 images, video frames included.
• 503 overloaded — image and video processing is busy. Retry after a few seconds.

FAQ

Can the models generate images?

No. The Torki API reads images and video; it answers in text.

Does the model hear the video's soundtrack?

No. Only frames are used, so ask about what can be seen, including on-screen text and subtitles.

My image link fails with media_fetch_failed.

The link must be reachable from the public internet on port 80 or 443, without a sign-in, and download within 30 seconds. For a private image, send it as a data URI instead.