> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sutro.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Direct model inference

> Call one or more model IDs directly instead of using a published Function.

# Direct model inference

Production Batch workloads normally execute a published Function with
[`batch_run_function()`](/reference/python-sdk/functions). Use `infer()` when
you intentionally want to call an available model ID directly and supply the
prompt, schema, and runtime settings yourself.

Configure your [deployment URL and API key](/reference/authentication) first.
Direct model inference uses the same deployment's `/v1/batch-inference` endpoint
as published Functions.

```python theme={null}
infer(
    data,
    model="gpt-oss-20b",
    name=None,
    description=None,
    column=None,
    output_column="inference_result",
    job_priority=0,
    output_schema=None,
    sampling_params=None,
    system_prompt=None,
    dry_run=False,
    stay_attached=None,
    random_seed_per_input=False,
    truncate_rows=True,
    id_column=None,
)
```

| Parameter | Description |
| - | - |
| `data` | List, Pandas/Polars DataFrame, local CSV/Parquet/TXT path, or HTTPS CSV/Parquet URL. |
| `model` | Model ID shown in the Sutro app. |
| `name`, `description` | Optional job metadata. |
| `column` | DataFrame/file input column, or a list of columns and literal separators to concatenate. |
| `output_column` | Name used when results are materialized. |
| `job_priority` | `0` for the prototyping service level or `1` for the [one-hour production SLA](/batch/run-model#job-priority-and-sla). |
| `output_schema` | Pydantic model or JSON Schema for structured output. Not supported by embedding models. |
| `sampling_params` | Overrides merged with the model defaults. |
| `system_prompt` | Optional system instruction. |
| `dry_run` | Create an estimate job instead of the full job. |
| `stay_attached` | Defaults to `True` for priority 0 and `False` for priority 1. |
| `random_seed_per_input` | Use a different random seed for each row. |
| `truncate_rows` | Truncate rows that exceed the model context window. |
| `id_column` | Preserve an ID column from a remote CSV/Parquet input. |

```python theme={null}
import sutro as so

job_id = so.infer(
    data=["Great product", "Would not buy again"],
    model="gpt-oss-20b",
    system_prompt="Classify sentiment.",
    stay_attached=False,
)
results = so.await_job_completion(job_id)
```

`infer()` returns a job ID. In attached mode it also streams progress and
prints a result preview.

## Run multiple models

`infer_per_model()` submits one job per model and returns their job IDs. It
accepts the same data, column, schema, sampling, prompt, priority, estimate,
seed, truncation, and ID options as `infer()`, plus optional parallel `names`
and `descriptions` lists.

```python theme={null}
job_ids = so.infer_per_model(
    data=["Explain photosynthesis."],
    models=["gpt-oss-20b", "gpt-oss-120b"],
)
```

Batch cannot search the web or call tools. Inputs must include all required
context.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.