Direct model inference

Production Batch workloads normally execute a published Function with batch_run_function(). Use infer() when you intentionally want to call an available model ID directly and supply the prompt, schema, and runtime settings yourself.

infer(
    data,
    model="gpt-oss-20b",
    name=None,
    description=None,
    column=None,
    output_column="inference_result",
    job_priority=0,
    output_schema=None,
    sampling_params=None,
    system_prompt=None,
    dry_run=False,
    stay_attached=None,
    random_seed_per_input=False,
    truncate_rows=True,
    id_column=None,
)

Parameter	Description
`data`	List, Pandas/Polars DataFrame, local CSV/Parquet/TXT path, or HTTPS CSV/Parquet URL.
`model`	Model ID shown in the Sutro app.
`name`, `description`	Optional job metadata.
`column`	DataFrame/file input column, or a list of columns and literal separators to concatenate.
`output_column`	Name used when results are materialized.
`job_priority`	`0` for the prototyping service level or `1` for the one-hour production SLA.
`output_schema`	Pydantic model or JSON Schema for structured output. Not supported by embedding models.
`sampling_params`	Overrides merged with the model defaults.
`system_prompt`	Optional system instruction.
`dry_run`	Create an estimate job instead of the full job.
`stay_attached`	Defaults to `True` for priority 0 and `False` for priority 1.
`random_seed_per_input`	Use a different random seed for each row.
`truncate_rows`	Truncate rows that exceed the model context window.
`id_column`	Preserve an ID column from a remote CSV/Parquet input.

import sutro as so

job_id = so.infer(
    data=["Great product", "Would not buy again"],
    model="gpt-oss-20b",
    system_prompt="Classify sentiment.",
    stay_attached=False,
)
results = so.await_job_completion(job_id)

infer() returns a job ID. In attached mode it also streams progress and prints a result preview.

Run multiple models

infer_per_model() submits one job per model and returns their job IDs. It accepts the same data, column, schema, sampling, prompt, priority, estimate, seed, truncation, and ID options as infer(), plus optional parallel names and descriptions lists.

job_ids = so.infer_per_model(
    data=["Explain photosynthesis."],
    models=["gpt-oss-20b", "gpt-oss-120b"],
)

Batch cannot search the web or call tools. Inputs must include all required context.