Structured Outputs

Sutro’s SDK allows you to enforce a JSON schema for the outputs of your inference job. This can be useful if you want to ensure that the outputs of your inference job are in a specific format, or if you want to extract specific information from the outputs.

To specify a schema for the outputs of your inference job, you can pass a Pydantic Model or a JSON schema to the output_schema parameter in the SDK. If you pass a JSON schema, it must follow the json-schema.org specification.

On rare occasions, the model may not adhere to the provided schema and will produce invalid JSON; in this case, Sutro will return the raw, unparsed text. If you need help improving JSON adherence, please contact us.

Example with Pydantic Model

import sutro as so
from pydantic import BaseModel

texts = [
    "The airplane is flying at an altitude of 30,000 feet.",
    "The capital of France is Paris.",
    "The best way to cook a steak is on a grill."
]

class AviationOutput(BaseModel):
    is_aviation_related: bool
    answer_justification: str

results = so.infer(texts, system_prompt="Is the following sentence aviation related? Please provide a justification for your answer.", output_schema=AviationOutput)

print(results)

Example with JSON Schema

import sutro as so

texts = [
    "The airplane is flying at an altitude of 30,000 feet.",
    "The capital of France is Paris.",
    "The best way to cook a steak is on a grill."
]

json_schema = {
    "type": "object",
    "properties": {
        "is_aviation_related": {
            "type": "boolean"
        },
        "answer_justification": {
            "type": "string"
        }
    },
    "required": ["is_aviation_related", "answer_justification"]
}

results = so.infer(texts, system_prompt="Is the following sentence aviation related? Please provide a justification for your answer.", output_schema=json_schema)

print(results)