Run a Function

A published Function can be run two ways:

Real time. One input, one synchronous answer. Use it for single requests and low-QPS integrations, and for Functions with image or PDF inputs.
Batch. A job over a table, dataframe, or file. Use it for larger datasets, from thousands to millions of rows. Note that it is text-only.

Both run the same published prompt, model, output schema, and sampling defaults, so the answers agree. Open Integrate on a ready Function to see code snippets to integrate into your pipeline or script.

Install and authenticate

pip install sutro
export SUTRO_API_URL="https://your-sutro-deployment.example.com"
export SUTRO_API_KEY="sk_..."

Use the origin of the deployment where you published the Function and a deployment API key. Verify your connection before submitting a job. You can persist the same values with sutro login.

Run one input

import sutro as so

result = so.run_function(
    "support-escalation",
    {"text": "The customer reports an unrecognized charge."},
)

print(result.output)      # {"label": "billing", "reasoning": "..."}
print(result.confidence)  # 0.8
print(result.usage)       # {"input_tokens": 412, "output_tokens": 38, "cost_usd": 0.0016}

The same call over HTTP:

curl https://YOUR-SUTRO-DEPLOYMENT/v1/functions/support-escalation/run \
  -H "Authorization: Key YOUR_SUTRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": {"text": "The customer reports an unrecognized charge."}
  }'

Or from the command line:

sutro functions run support-escalation --input '{"text": "..."}'
sutro functions run support-escalation --input @request.json

Request

POST /v1/functions/{name}/run takes exactly one body field, input. Its keys must match the Function’s configured input fields; every field is required, and unknown fields are rejected. Values that are not strings are converted the same way Batch converts a row: null becomes an empty string, objects and lists become JSON, and anything else becomes its plain string form. A bare string is accepted in place of the object when the Function has exactly one text input. Sampling parameters cannot be overridden per request. The Function’s published settings are what make its answers reproducible.

Response

{
  "request_id": "rt_01j7wnq3f8k2c9x4",
  "function": {
    "name": "support-escalation",
    "model": "claude-sonnet-4-5",
    "model_source": "model-sweep"
  },
  "output": {"label": "billing", "reasoning": "..."},
  "confidence": 0.8,
  "usage": {"input_tokens": 412, "output_tokens": 38, "cost_usd": 0.0016}
}

Field	Meaning
`request_id`	This request’s ID. It also appears in your Function’s Monitoring tab.
`function.model`	The base model that answered.
`function.model_source`	`model-sweep` when a model was set active from a model sweep. `sutro-default` when none has been and the Function ran its latest iteration’s prompt on the Sutro default model.
`output`	The answer, parsed against the Function’s output schema. A string when the Function has no schema.
`confidence`	A confidence score between 0 and 1 for this answer. `null` when scoring is turned off.
`usage`	Token counts and cost for the whole request.

Confidence scores

confidence reports how settled the answer is, using the same technique as the Batch API. Answers at or below 0.6 are the ones worth a human look. Treat the score as a routing signal rather than a probability: send low-confidence answers to a review step, a fallback path, or a person, and let the rest through. The inputs behind them are also the most useful ones to label in the next iteration of the Function. Scoring adds cost and latency to a request. If you will not act on the score, turn it off per request with confidence_scoring=False in the SDK or "confidence_scoring": false in the request body. confidence is then null. The answer still appears in Monitoring but is never added to Needs Review.

result = so.run_function(
    "support-escalation",
    {"text": "The customer reports an unrecognized charge."},
    confidence_scoring=False,
)
print(result.confidence)  # None

Image and PDF inputs

Functions with an image or PDF input field are real-time only, since Batch is currently text-only. An asset field takes one of three values.

import sutro as so

result = so.run_function(
    "invoice-reader",
    {
        "note": "check page 2",
        "scan": so.Asset.from_path("invoices/april.pdf"),   # inline bytes
        "logo": so.Image.from_url("https://cdn.example.com/logo.png"),
        "prior": so.Asset.from_name("march-invoice.pdf"),   # already uploaded
    },
)

{
  "input": {
    "note": "check page 2",
    "scan": {"type": "pdf", "base64": "JVBERi0...", "mime_type": "application/pdf"},
    "logo": {"type": "image", "url": "https://cdn.example.com/logo.png"},
    "prior": {"name": "march-invoice.pdf"}
  }
}

Value	Shape
Inline bytes	`{"base64": "...", "mime_type": "image/png"}`
URL	`{"url": "https://..."}` — `https` only, public hosts only (no private, loopback, or link-local addresses), streamed with a 10 second timeout and the same 20 MB cap
Stored asset	`{"name": "march-invoice.pdf"}`, or the name as a bare string

PNG, JPEG, WebP, and PDF are accepted, up to 20 MB per asset. The bytes are sniffed, so a declared type or mime_type that disagrees with the actual content is rejected rather than passed to the model.

Limits

Limit	Default
Requests per API key	10 per second, bursting to 20
In-flight requests per model	64 per deployment
Request timeout	90 seconds
Input size	200,000 characters, counted after the input fields are joined into the prompt the model receives
Asset size	20 MB each

A rate-limited request returns 429 with a Retry-After header; wait that many seconds before retrying. The SDK surfaces it as SutroRateLimitError.retry_after. Deployment operators can change these defaults with the HARMONIZE_REALTIME_* environment variables.

Errors

Failures return {"detail": ..., "code": ..., "request_id": ...}.

Status	`code`	Cause
401	`unauthorized`	Missing or invalid API key.
404	`function_not_found`	No Function by that name in this deployment.
409	`no_runnable_prompt`	The Function has no published or completed prompt to run yet.
409	`deployment_unpublished`	The latest deploy or completed iteration failed to publish to Batch, so real time refuses to serve a prompt Batch does not. Retry the publish.
413	`payload_too_large`	The request body is over 32 MB.
422	`invalid_input`	Missing, unknown, or oversized input, or an unreadable asset.
422	`not_servable`	The Function’s active model cannot serve it in real time. The message says why.
429	`rate_limited`	Your key’s request rate. See `Retry-After`.
429	`at_capacity`	The deployment has no room for another request on this model right now. Retry after `Retry-After`.
429	`provider_rate_limited`	The model provider rate limited the request. See `Retry-After`.
502	`provider_error`	The model provider failed or returned an unusable answer.
503	`provider_not_configured`	No API key is configured for the Function’s provider. Add one in the app’s provider settings.
503	`inference_spend_circuit_open`	The deployment’s paid-inference budget is paused. See `Retry-After`.
504	`timeout`	The request exceeded the 90 second budget.

Requests are never retried automatically, in the SDK or the API: a Function run costs tokens, and a caller waiting on an answer should decide for itself whether to try again.

Submit many rows instead

For anything larger than a few hundred inputs, Batch is the better developer experience. It takes a list of dictionaries, Pandas or Polars DataFrames, local CSV or Parquet files, and HTTPS download URLs. It has no request rate limits and comfortably handles hundreds of thousands to millions of rows.

import sutro as so

job_id = so.batch_run_function(
    name="support-escalation",
    data=[
        {"text": "The customer asks to reset a password."},
        {"text": "The customer reports an unrecognized charge."},
    ],
)

results = so.await_job_completion(job_id)

See batch_run_function() and Run a Function in Batch.

Run with your own provider

The OpenAI-compatible endpoints section in Integrate provides the optimized system prompt, output schema, and recommended model. Copy these artifacts into your own compatible provider when you want to run the Function’s configuration on your own infrastructure. That path runs the exported configuration: it does not invoke the published Function by name, and it does not return a confidence score.

​Run a Function

​Install and authenticate

​Run one input

​Request

​Response

​Confidence scores

​Image and PDF inputs

​Limits

​Errors

​Submit many rows instead

​Run with your own provider