> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sutro.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Creating a Batch Inference Job

> Run batch inference on inline inputs, a dataset, a download URL, or a published Sutro Function.

<Warning>Using the API directly is not recommended for most users. Instead, we recommend using the [Python SDK](/python-sdk/setup).</Warning>

Run batch inference over a list of inputs, a dataset, or an HTTP(S) CSV/Parquet download URL.

<Note>
  For large production workloads, use an HTTPS presigned S3 GET URL with priority `1`. See [Presigned S3 Inputs](/python-sdk/presigned-s3-inputs) for the file contract, URL lifetime, security guidance, and end-to-end examples.
</Note>

## Using a Sutro Function as `model`

Set `model` to the published Sutro Function name and send rows whose keys match that Function's inputs.

<Note>
  Use the Function name only. Do not include a namespace, owner, or revision in `model`.

  Sutro resolves the Function namespace from the authenticated API key's user account and loads the currently published revision through that Function's `latest.json` pointer.
</Note>

When `model` is a Sutro Function name:

* every required Function input field must be present; optional fields may be absent and extra fields are ignored
* input values are converted to strings, with null values rendered as empty strings
* string rows are treated as already-rendered prompts
* HTTP(S) CSV/Parquet download URLs are read as row objects whose columns match the Function inputs
* `system_prompt` and `json_schema` should be omitted because they come from the published Function
* request-level `sampling_params` are merged on top of the Function/runtime defaults
* dataset IDs such as `dataset-<uuid>` are not supported

<Warning>
  Only text Functions are supported through the Batch API today. Image, PDF, and other multimodal Functions are not supported here yet.
</Warning>

## Request Body

<ParamField body="inputs" type="string[]|object[]|string" required>
  Accepts one of the following input forms:

  * **Array** — an array of strings, or object rows for a Sutro Function/custom model
  * **Dataset ID** — a dataset ID such as `dataset-<uuid>`
  * **Download URL** — an HTTP(S) CSV or Parquet download URL, including a presigned Amazon S3 GET URL

  Direct standalone model runs (i.e. `model="gpt-oss-20b"`) expect string rows. Sutro Function runs expect object rows whose keys match the Function inputs, already-rendered string rows, or a CSV/Parquet download URL with matching columns.

  An `s3://` URI is not accepted. For production S3 inputs, generate an HTTPS presigned GET URL with enough lifetime for the worker to start its one full-object download. Sutro does not currently resume or retry a failed input download.
</ParamField>

<ParamField body="column_name" type="string" default="None">
  Column name to use when `inputs` is a dataset ID or a download URL for standalone/base-model inference.

  Dataset IDs require a `column_name` to be passed indicating which column to use. For presigned download URLs, `column_name` selects the column to run; if omitted, the first column is used. Specify it explicitly in production so upstream column reordering cannot change the model input.

  Omit `column_name` when `model` is a Sutro Function or a custom model with a declared request schema. Those URL inputs are matched against the declared input fields and rendered by Sutro.
</ParamField>

<ParamField body="id_column_name" type="string" default="None">
  Column containing user-provided row IDs to carry into job results.

  When using the Python SDK, pass this as `id_column`.

  This field is supported only when `inputs` is an HTTP(S) CSV or Parquet download URL. The column must exist, must differ from every inference column, and cannot use the reserved `SKYSIGHT_` prefix or the result column names `inputs`, `outputs`, `confidence_score`, or `cumulative_logprobs`.

  The ID column is returned with every result format even when `include_inputs` is `false`, so results can be joined back to the source table. Sutro preserves ID values but does not guarantee preservation of the source file's physical integer type.
</ParamField>

<ParamField body="model" type="string" default="gpt-oss-20b">
  Standalone model ID, custom model name, or published Sutro Function name.

  If the value is not an available standalone model, Sutro treats it as a Function name and resolves the correct model to use based on the Function's latest spec.
</ParamField>

<ParamField body="system_prompt" type="string" default="None">
  System prompt for standalone model batch inference.

  Omit this field when `model` is a Sutro Function name.
</ParamField>

<ParamField body="json_schema" type="object" default="None">
  Structured output schema for standalone model batch inference.

  Omit this field when `model` is a Sutro Function name.
</ParamField>

<ParamField body="sampling_params" type="object" default="None">
  Sampling parameters for the batch job. See [Sampling Parameters](/concepts/sampling-parameters).

  For Sutro Function jobs, most users should omit this and use the published defaults. If provided, these values override the Function/runtime defaults for that job.
</ParamField>

<ParamField body="job_priority" type="integer" default="0">
  Batch priority level. Priorities `0` and `1` are supported.

  Dataset IDs require priority `1`.
</ParamField>

<ParamField body="cost_estimate" type="boolean" default="false">
  If `true`, create an estimate job instead of launching the normal full job. Priority-1 estimates at or above the sampling threshold run inference on a prefix sample of approximately 1 million input tokens, which can include every row for workloads near that threshold. The submission response contains the estimate job ID; fetch that job after it succeeds to read `cost_estimate`. See [Cost Estimates](/concepts/cost-estimates/) for more information.
</ParamField>

<ParamField body="random_seed_per_input" type="boolean" default="false">
  If `true`, generate a random seed per input row.
</ParamField>

<ParamField body="truncate_rows" type="boolean" default="true">
  If `true`, rows that exceed the selected model's context window are truncated to fit. Truncation removes the minimal amount of text such that the token count of (input text + prompt text + max output tokens) is less than the model's context window length. If `false`, jobs with rows that exceed the context window will be marked as FAILED.
</ParamField>

<ParamField body="name" type="string" default="None">
  Optional job name for metadata and experiment tracking. Maximum length is 45 characters.
</ParamField>

<ParamField body="description" type="string" default="None">
  Optional job description for metadata and experiment tracking. Maximum length is 512 characters.
</ParamField>

## Headers

<ParamField header="Authorization" type="string" required>
  Your Sutro API key using the Key authentication scheme.

  Format: `Key YOUR_API_KEY`

  Example: `Authorization: Key sk_live_abc123...`
</ParamField>

## Response

Returns the created job ID in both `metadata.job_id` and `results`.

<Note>
  The response confirms job creation, not successful ingestion or quota validation. Download, row-quota, token-quota, and cost-quota checks run asynchronously. Check priority-1 quotas before submission and monitor the returned job ID.
</Note>

<Warning>
  Job creation is not idempotent. Before retrying an ambiguous or timed-out submission, inspect recent jobs so you do not create duplicate work.
</Warning>

<ResponseField name="metadata" type="object">
  Metadata for the created job. Contains `job_id` and `message`.
</ResponseField>

<ResponseField name="results" type="string">
  Job ID for the created batch inference job. This is the same value as `metadata.job_id`.
</ResponseField>

<ResponseExample>
  ```json Response theme={null}
  {
    "metadata": {
      "job_id": "job-12345678-1234-1234-1234-1234567890ab",
      "message": "Job created successfully"
    },
    "results": "job-12345678-1234-1234-1234-1234567890ab"
  }
  ```
</ResponseExample>

## Code Examples

### Standalone model with array inputs

<CodeGroup>
  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.sutro.sh/batch-inference",
      headers={
          "Authorization": "Key YOUR_SUTRO_API_KEY",
          "Content-Type": "application/json",
      },
      json={
          "model": "gpt-oss-20b",
          "inputs": [
              "What is the capital of France?",
              "Explain quantum computing in simple terms.",
              "Write a haiku about programming.",
          ],
          "system_prompt": "You are a helpful assistant.",
          "job_priority": 0,
      },
  )

  result = response.json()
  print(f"Job created: {result['results']}")
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.sutro.sh/batch-inference \
    -H "Authorization: Key YOUR_SUTRO_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "gpt-oss-20b",
      "inputs": [
        "What is the capital of France?",
        "Explain quantum computing in simple terms.",
        "Write a haiku about programming."
      ],
      "system_prompt": "You are a helpful assistant.",
      "job_priority": 0
    }'
  ```
</CodeGroup>

### Published Sutro Function with object rows

Replace `lead-qualifier` and the input field names with your published Function name and schema.

<CodeGroup>
  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.sutro.sh/batch-inference",
      headers={
          "Authorization": "Key YOUR_SUTRO_API_KEY",
          "Content-Type": "application/json",
      },
      json={
          "model": "lead-qualifier",
          "inputs": [
              {
                  "query": "Find cybersecurity leaders evaluating AI vendors.",
                  "region": "APAC",
              },
              {
                  "query": "Find sales operations leaders replacing manual enrichment.",
                  "region": "EMEA",
              },
          ],
          "job_priority": 0,
          "name": "lead-qualifier-smoke",
      },
  )

  result = response.json()
  print(f"Job created: {result['results']}")
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.sutro.sh/batch-inference \
    -H "Authorization: Key YOUR_SUTRO_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "lead-qualifier",
      "inputs": [
        {
          "query": "Find cybersecurity leaders evaluating AI vendors.",
          "region": "APAC"
        },
        {
          "query": "Find sales operations leaders replacing manual enrichment.",
          "region": "EMEA"
        }
      ],
      "job_priority": 0,
      "name": "lead-qualifier-smoke"
    }'
  ```
</CodeGroup>

### Published Sutro Function with a download URL

The CSV or Parquet file must contain columns matching the Function inputs. For this example, the file contains `query`, optionally `region`, and a user-provided `lead_id`.

<CodeGroup>
  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.sutro.sh/batch-inference",
      headers={
          "Authorization": "Key YOUR_SUTRO_API_KEY",
          "Content-Type": "application/json",
      },
      json={
          "model": "lead-qualifier",
          "inputs": "https://your-bucket.s3.amazonaws.com/leads.parquet?X-Amz-Algorithm=...",
          "id_column_name": "lead_id",
          "job_priority": 1,
          "name": "lead-qualifier-file-run",
      },
  )

  result = response.json()
  print(f"Job created: {result['results']}")
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.sutro.sh/batch-inference \
    -H "Authorization: Key YOUR_SUTRO_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "lead-qualifier",
      "inputs": "https://your-bucket.s3.amazonaws.com/leads.parquet?X-Amz-Algorithm=...",
      "id_column_name": "lead_id",
      "job_priority": 1,
      "name": "lead-qualifier-file-run"
    }'
  ```
</CodeGroup>

### Standalone model with download URL input

The URL must be presigned for `GET` and remain valid until Sutro starts its one full-object download. Priority `1` with an unwrapped Parquet object is recommended for large production jobs. Sutro first downloads and retains the complete source object; for standalone/base-model jobs, tokenization then proceeds in bounded slices.

<CodeGroup>
  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.sutro.sh/batch-inference",
      headers={
          "Authorization": "Key YOUR_SUTRO_API_KEY",
          "Content-Type": "application/json",
      },
      json={
          "model": "gpt-oss-20b",
          "inputs": "https://your-bucket.s3.amazonaws.com/data.parquet?X-Amz-Algorithm=...",
          "column_name": "prompt",
          "id_column_name": "row_id",
          "system_prompt": "You are a helpful assistant.",
          "job_priority": 1,
      },
  )

  result = response.json()
  print(f"Job created: {result['results']}")
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.sutro.sh/batch-inference \
    -H "Authorization: Key YOUR_SUTRO_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "gpt-oss-20b",
      "inputs": "https://your-bucket.s3.amazonaws.com/data.parquet?X-Amz-Algorithm=...",
      "column_name": "prompt",
      "id_column_name": "row_id",
      "system_prompt": "You are a helpful assistant.",
      "job_priority": 1
    }'
  ```
</CodeGroup>

### Standalone model with dataset input

<CodeGroup>
  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.sutro.sh/batch-inference",
      headers={
          "Authorization": "Key YOUR_SUTRO_API_KEY",
          "Content-Type": "application/json",
      },
      json={
          "model": "gpt-oss-20b",
          "inputs": "dataset-8be01234-abcd-5678-ef90-1234567890ab",
          "column_name": "prompt",
          "system_prompt": "You are a helpful assistant.",
          "job_priority": 1,
      },
  )

  result = response.json()
  print(f"Job created: {result['results']}")
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.sutro.sh/batch-inference \
    -H "Authorization: Key YOUR_SUTRO_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "gpt-oss-20b",
      "inputs": "dataset-8be01234-abcd-5678-ef90-1234567890ab",
      "column_name": "prompt",
      "system_prompt": "You are a helpful assistant.",
      "job_priority": 1
    }'
  ```
</CodeGroup>

### Cost estimate

<CodeGroup>
  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.sutro.sh/batch-inference",
      headers={
          "Authorization": "Key YOUR_SUTRO_API_KEY",
          "Content-Type": "application/json",
      },
      json={
          "model": "lead-qualifier",
          "inputs": [
              {
                  "query": "Find cybersecurity leaders evaluating AI vendors.",
                  "region": "APAC",
              }
          ],
          "job_priority": 0,
          "cost_estimate": True,
      },
  )

  result = response.json()
  print(f"Estimate job created: {result['results']}")
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.sutro.sh/batch-inference \
    -H "Authorization: Key YOUR_SUTRO_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "lead-qualifier",
      "inputs": [
        {
          "query": "Find cybersecurity leaders evaluating AI vendors.",
          "region": "APAC"
        }
      ],
      "job_priority": 0,
      "cost_estimate": true
    }'
  ```
</CodeGroup>