Run Functions at scale

Sutro Batch is the production execution path for published Functions. Use it to apply the same optimized prompt, model, schema, and runtime settings to one row or millions of rows asynchronously. Batch jobs cannot search the web or call tools. Inputs must contain all context the Function needs.

Install and authenticate

pip install sutro
export SUTRO_API_URL="https://your-sutro-deployment.example.com"
export SUTRO_API_KEY="sk_..."

Submit a Function

import polars as pl
import sutro as so

reviews = pl.DataFrame({
    "review": [
        "The battery life is terrible.",
        "Great camera and build quality.",
    ]
})

job_id = so.batch_run_function(
    name="review-sentiment",
    data=reviews,
    job_priority=1,
    stay_attached=False,
)

results = so.await_job_completion(job_id)
print(results)

name is the published Function name. Input fields must match its declared inputs; the published Function supplies its prompt, model, output schema, and sampling defaults. Submission returns a job ID, and successful results return as a DataFrame.

Job priority and SLA

Job priority selects the workload class and service level for each submission. It is not a request to move one job ahead of another.

Priority	Use	Completion target
`0`	Prototyping on smaller samples	Within 10 minutes
`1`	Production execution at scale	Within one hour

Priority 1 is the production SLA for full-dataset Function runs and has higher row and token quotas. Completion time still depends on input size and model. Check the quotas configured for your deployment with sutro quotas or so.get_quotas() before submitting a large job. Sutro can handle large workloads, often in the millions of rows or billions of tokens. Reach out to [email protected] to request higher quotas for your deployment.

Supported inputs

Lists of dictionaries
Pandas and Polars DataFrames
Local CSV and Parquet files
CSV and Parquet datasets in Amazon S3, Cloudflare R2, or other S3-compatible storage, submitted with a presigned HTTPS GET URL

Batch does not accept s3:// URIs directly. For large production datasets, generate a read-only presigned HTTPS GET URL, preserve a stable ID column, and follow Production Batch for estimation, monitoring, and resumable downloads.

Custom models and embeddings

Batch can also execute custom open-source generation and embedding models for specialized workloads. Contact [email protected] if you need this capability enabled for your deployment.

​Install and authenticate

​Submit a Function

​Job priority and SLA

​Supported inputs

​Custom models and embeddings

Install and authenticate

Submit a Function

Job priority and SLA

Supported inputs

Custom models and embeddings