Skip to main content

​
Quickstart

​
Using the Python SDK

The following is a basic example to get started with the Python SDK. 
import sutro as so

so.set_api_key("sk_******") # this can be skipped if set via the CLI with `sutro login`

user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]

results = so.infer(user_reviews, system_prompt="Classify the review as positive, neutral, or negative.")

print(results)
This will output something like: 
[
    'This review is classified as positive. The reviewer expressed a strong affection for the product ("I loved the product") and mentioned two specific benefits: it was "easy to use" and had a "great user interface".', 
    'This review is neutral.', 
    'This review is negative.'
]
which preserves the original ordering of the inputs.

​
Structuring outputs

In the above example, we’re trying to perform a simple classification task. In such cases, we may want structured outputs. We can accomplish this by passing in a pydantic model or JSON schema to guide the LLM to output the correct format using the output_schema parameter. 
import sutro as so
from pydantic import BaseModel

so.set_api_key("sk_******") # this can be skipped if set via the CLI with `sutro login`

user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]

class ReviewClassification(BaseModel):
    classification: str

results = so.infer(user_reviews, system_prompt="Classify the review as positive, neutral, or negative.", output_schema=ReviewClassification)

print(results)
Now we should obtain the following output: 
[
    {"classification": "positive"},
    {"classification": "neutral"},
    {"classification": "negative"}
]

​
Using Files

You can also use files to pass in data. We currently support CSV, Parquet, and TXT files. If you’re using a TXT file, each line should represent a single input. If you’re using a CSV or Parquet file, you must specify the column name that contains the inputs using the column parameter. 
import sutro as so

results = so.infer(
    inputs='my_file.csv', 
    column='reviews', 
    system_prompt='Classify the review as positive, neutral, or negative.',
)

print(results)
You can view the full details of the SDK at here.

​
Moving to Production

So far we’ve shown what it looks like to use prototyping jobs (priority 0, default). After working with a small amount of data using prototyping jobs, you’ll likely want to move to production jobs (priority 1) which are less expensive and have higher quotas. To do so, you simply need to set the job_priority parameter to 1: 
job_id = so.infer(inputs='my_file.csv', column='reviews', system_prompt='Classify the review as positive, neutral, or negative.', job_priority=1)
Instead of waiting for the job to finish, it will return a job ID immediately which you can pass to await_job_completion, which will poll for job status and retrieve results once complete. 
import time
import sutro as so

job_id = so.infer(
    inputs="my_file.csv",
    column="reviews",
    system_prompt="Classify the review as positive, neutral, or negative.",
    job_priority=1,
)

results = so.await_job_completion(job_id)

​
Using the CLI to view job progress and results

Once you’ve submitted jobs via the SDK or API, you can use the CLI to view the status of the job and retrieve the results. Viewing current and past jobs: 
sutro jobs
Retrieving job status: 
sutro status <job_id>
Retrieving job results: 
sutro results <job_id>
You can view the full details of the CLI at here.

​
API Usage Example

You can accomplish the same tasks using the API directly. You’ll need to preprocess data as a list or array and pass it in via the parameters in the JSON body. 
import requests
import json

user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]

params = {
    "model": "llama-3.1-8b",
    "inputs": user_reviews,
    "system_prompt": "Classify the review as positive, neutral, or negative.",
}
headers = {
    "Authorization": "Key <YOUR_API_KEY>",
    "Content-Type": "application/json"
}

response = requests.post("https://api.sutro.sh/batch-inference", json=params, headers=headers)
results = response.json()
For more details on using the API directly, refer to the Batch API Reference.