Quickstart

Create a deployment key in the API Keys panel of your Sutro UI, then configure the deployment before running these examples: If the panel is not visible, contact the Sutro team at [email protected] to create a key for you.

export SUTRO_API_URL="https://your-sutro-deployment.example.com"
export SUTRO_API_KEY="sk_..."

The SDK accepts either the Sutro deployment base URL or the same URL with /v1 appended. You can persist the same values with sutro login instead.

Using the Python SDK

Here’s the simplest way to run a job with Sutro - just pass a list of inputs and a system prompt.

import sutro as so

user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful.",
]

results = so.infer(
    user_reviews,
    system_prompt="Classify the review as positive, neutral, or negative."
)

print(results)

This outputs a list preserving the original ordering:

[
    'This review is positive.',
    'This review is neutral.',
    'This review is negative.'
]

Below are some more complex examples that show different ways to use Sutro!

Structuring outputs

In the above example, we’re trying to perform a simple classification task. In such cases, we may want structured outputs. We can accomplish this by passing in a Pydantic model or JSON schema using the output_schema parameter. The model will strictly adhere to this schema in its output content.

import sutro as so
from pydantic import BaseModel

user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]

class ReviewClassification(BaseModel):
    classification: str

results = so.infer(
    user_reviews,
    system_prompt="Classify the review as positive, neutral, or negative.",
    output_schema=ReviewClassification
)

print(results)

Now we should obtain the following output:

[
    {"classification": "positive"},
    {"classification": "neutral"},
    {"classification": "negative"}
]

Structured outputs also work well with reasoning models, since the model has the token space to go through a reasoning process before its outputs get constrained to the output schema.

import sutro as so
from pydantic import BaseModel
from typing import List

reviews = [...]

class ReviewAnalysis(BaseModel):
    sentiment: str
    rating: int
    key_aspects: List[str]
    would_recommend: bool

system_prompt = """Analyze the review and extract structured insights.
Reflect and conisider the implications of what the customer is stating
and how that may affect your analysis. Rate from 1-5."""

results = so.infer(
    data=reviews,
    system_prompt=system_prompt,
    output_schema=ReviewAnalysis,
    model='qwen-3-30b-a3b-thinking'
)

# Note the `content` and `reasoning_content` fields
print(results[0])
# >> {"content": {"sentiment": ... }, "reasoning_content": "Ok, our tasks is to..."}

Working with DataFrames and Sampling Parameters

This example shows how to work with DataFrames, customize sampling parameters, and wait for job completion.

import sutro as so
import polars as pl

# Load your data
df = pl.read_csv('customer_feedback.csv')

# Run inference with custom sampling parameters
results_df = so.infer(
    data=df,
    column='feedback_text',
    output_column='sentiment_analysis',
    model='llama-3.1-70b',
    system_prompt='Analyze sentiment and extract key themes',
    sampling_params={
        'temperature': 0.3,
        'top_p': 0.9,
        'max_tokens': 200
    },
)

print(results_df)

Multi-Model Comparison

Run the same inputs across multiple models to compare outputs and quality.

import sutro as so

prompts = [
    "Explain quantum computing in simple terms",
    "What are the benefits of renewable energy?",
    "How does photosynthesis work?"
]

# Run same inputs across multiple models for comparison
job_ids = so.infer_per_model(
    data=prompts,
    models=['gemma-3-27b-it', 'qwen-2.5-32b-instruct', 'gpt-oss-20b'],
    names=['gemma-27b-run', 'qwen-32b-run', 'gpt-oss-run'],
    system_prompt='Provide a concise, accurate explanation',
)

# Retrieve results from each model
for model_name, job_id in zip(['gemma-27b', 'qwen-32b', 'gpt-oss'], job_ids):
    results = so.await_job_completion(job_id)
    print(f"\n{model_name} results:")
    print(results)

Cost Estimation

Before submitting production work, inspect your row and token quotas with so.get_quotas() or sutro quotas. See Increasing Quotas if the input exceeds your priority-1 limits. You can request an estimate with dry_run=True. The SDK prints the estimate and returns the estimate job ID; it does not return the estimate value.

import sutro as so
import polars as pl

# Load a large dataset
df = pl.read_csv('large_dataset.csv')

# The estimate is displayed automatically; retain its job ID for audit/support.
estimate_job_id = so.infer(
    data=df,
    column='text_column',
    model='gemma-3-27b-it',
    system_prompt='Summarize this text',
    job_priority=1,
    dry_run=True,
)

print(estimate_job_id)

Using Files

You can also use files to pass in data. We currently support CSV, Parquet, and TXT files. If you’re using a TXT file, each line should represent a single input. If you’re using a CSV or Parquet file, you must specify the column name that contains the inputs using the column parameter.

import sutro as so

job_id = so.infer(
    data='my_file.csv',
    column='reviews',
    system_prompt='Classify the review as positive, neutral, or negative.',
)

print(job_id)

You can view the full details of the SDK at here.

Moving to Production

So far we’ve shown prototyping jobs (priority 0, the default). For a production job, use priority 1 and give Sutro a presigned HTTPS GET URL for a private Parquet object in Amazon S3. This keeps the submission request small and avoids loading the full production dataset into the SDK process. Install Boto3 if your signing environment does not already include it:

python -m pip install boto3

Have your producer upload to a unique, immutable key. In production, use a separate read-only signing identity with s3:GetObject; the producer identity needs s3:PutObject. If bucket versioning is enabled, capture the uploaded version ID and follow the version-pinning example in Presigned S3 Inputs.

import boto3
import sutro as so

bucket = "my-production-inputs"
key = "sutro/reviews/2026-07-29/reviews.parquet"
bucket_region = "us-west-2"  # Replace with the bucket's actual AWS region.

signer = boto3.client("s3", region_name=bucket_region)

input_url = signer.generate_presigned_url(
    ClientMethod="get_object",
    Params={
        "Bucket": bucket,
        "Key": key,
    },
    # One hour is illustrative. Use the shortest lifetime that covers your
    # worst-case scheduling and pickup window, with a safety buffer.
    ExpiresIn=60 * 60,
)

job_id = so.infer(
    data=input_url,
    column="reviews",
    system_prompt="Classify the review as positive, neutral, or negative.",
    job_priority=1,
    stay_attached=False,
)

The job ID is returned immediately. Persist it before continuing. For a production-sized result set, poll without loading the complete JSON result into the SDK process:

import sutro as so

so.await_job_completion(
    job_id,
    timeout=6 * 60 * 60,
    obtain_results=False,
)

if so.get_job_status(job_id) != "SUCCEEDED":
    raise RuntimeError("Job did not complete successfully")

obtain_results=False returns None for success, failure, cancellation, and timeout, so confirm SUCCEEDED as shown. Then request a Parquet download URL from GET /jobs/{job_id}/results-url?format=parquet and download urls.get; see Presigned Results URLs. S3 evaluates expiration when a request starts, and a download started before expiration can finish afterward. Sutro currently issues one full-object GET and does not resume a failed input download; inspect the original job before creating a fresh URL and replacement job. Treat the full URL as a temporary credential, keep it out of logs and source control, and retain the immutable source object until results are reconciled. For version pinning, KMS permissions, retention, and troubleshooting guidance, see Presigned S3 Inputs.

Using the CLI to view job progress and results

Once you’ve submitted jobs via the SDK or API, you can use the CLI to view the status of the job and retrieve the results. Viewing current and past jobs:

sutro jobs list

Retrieving job status:

sutro jobs status <job_id>

Retrieving job results:

sutro jobs results <job_id>

You can view the full details of the CLI at here.

API Usage Example

You can accomplish the same tasks using the API directly. You’ll need to preprocess data as a list or array and pass it in via the parameters in the JSON body.

import requests
import json
import os

api_url = os.environ["SUTRO_API_URL"].rstrip("/")
if not api_url.endswith("/v1"):
    api_url = f"{api_url}/v1"

user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]

params = {
    "model": "llama-3.1-8b",
    "inputs": user_reviews,
    "system_prompt": "Classify the review as positive, neutral, or negative.",
}
headers = {
    "Authorization": f"Key {os.environ['SUTRO_API_KEY']}",
    "Content-Type": "application/json"
}

response = requests.post(f"{api_url}/batch-inference", json=params, headers=headers)
results = response.json()

For more details on using the API directly, refer to the Batch API Reference.

Getting Started

Core Concepts

Common Patterns

Examples

Quickstart

Quickstart

Using the Python SDK

Structuring outputs

Working with DataFrames and Sampling Parameters

Multi-Model Comparison

Cost Estimation

Using Files

Moving to Production

Using the CLI to view job progress and results

API Usage Example

​Quickstart

​Using the Python SDK

​Structuring outputs

​Working with DataFrames and Sampling Parameters

​Multi-Model Comparison

​Cost Estimation

​Using Files

​Moving to Production

​Using the CLI to view job progress and results

​API Usage Example

Quickstart

Using the Python SDK

Structuring outputs

Working with DataFrames and Sampling Parameters

Multi-Model Comparison

Cost Estimation

Using Files

Moving to Production

Using the CLI to view job progress and results

API Usage Example