Streaming, Tool Calls and JSON Output

Overview

The Torki API follows the OpenAI chat completions format, so streaming, tool (function) calling and JSON output work the way your OpenAI SDK expects. This article covers what is supported and where Torki differs.

Streaming

Send stream: true to receive the answer as server-sent events. Add stream_options.include_usage to get the token counts in the last chunk.

stream = client.chat.completions.create(
    model="torki-cosmos",
    messages=[{"role": "user", "content": "Plan a three-day trip to Kyoto."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
    if chunk.usage:
        print(chunk.usage)

• On Stellar and Cosmos, the thinking streams first, in delta.reasoning_content.
• A stream counts as one request in flight until it ends.
• If a stream is interrupted, the tokens produced up to that point are charged.
• An error after the stream has started arrives as a final data: {"error": {...}} event followed by data: [DONE].

Tool calling

Describe your functions in tools. When the model wants one, the response has tool_calls; run the function yourself and send the result back in a tool message.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

resp = client.chat.completions.create(
    model="torki-orbit",
    messages=[{"role": "user", "content": "Do I need an umbrella in Mumbai today?"}],
    tools=tools,
)
calls = resp.choices[0].message.tool_calls

• tool_choice: "auto" and "none" behave as in the OpenAI API. "required" and a named function are best effort: the model is told a tool call is required, but may still answer in text. Check for tool_calls before you rely on one.
• parallel_tool_calls is supported.
• Up to 128 tools per request.
• On Stellar and Cosmos, max_tokens limits the tool-call arguments as well as the answer.

JSON output

Set response_format to {"type": "json_object"}, or to json_schema with your schema (text is the default). Say in the prompt what the JSON should contain as well, and validate the result in your code before you use it.

resp = client.chat.completions.create(
    model="torki-pulse",
    response_format={"type": "json_object"},
    messages=[{"role": "user", "content": 'Extract the name and city as JSON: "Asha lives in Pune."'}],
)

Other request fields

Supported: model, messages (roles system, developer, user, assistant, tool), stream, stream_options.include_usage, max_tokens / max_completion_tokens, temperature, top_p, presence_penalty, frequency_penalty, seed, stop, response_format, tools, tool_choice, parallel_tool_calls and thinking (Torki).

• n must be 1.
• stop is a string or up to 4 strings.
• temperature is 0–2, top_p 0–1 and the penalties −2 to 2. Without a temperature, each model uses its own default.
• max_tokens is capped at the model's max output and lowered automatically so the request fits the context window. When it is lowered, the response carries x-torki-max-tokens-clamped with the value used.
• System and developer messages must come before the conversation; several at the start are combined into one.

The full field list is in the API reference.

FAQ

Can I use the Playground to build a request?

Yes. The Playground's Code view shows the request for any OpenAI SDK, and Import reads a chat completions call you paste in (Python or JavaScript OpenAI SDK, or a curl / JSON body).

Is the Responses API or the Assistants API supported?

No. The Torki API serves chat completions and the model list. See API Request Limits and What Is Coming.