Sutro API Documentation

Sutro is a serverless batch LLM inference service. We’re currently in beta with a focus on data processing workloads, enabling use cases including synthetic data generation, classification, structured extraction, and more.

Our product is available via our hosted batch inference API and Python SDK. You can expect up to 80-90% decrease in costs relative to online inference solutions, support for very large-scale jobs, extreme throughput capabilities, and white-glove support. If you have a use case that may benefit from our tools, please reach out to us at team@sutro.sh to discuss your requirements.

Contents

• Introduction
  • Why Sutro?
    • When to use Sutro
    • When not to use Sutro
    • Not sure if Sutro is right for you, or not sure how to get started?
• Installation
  • Setting up your API key
• Quickstart
  • Using the Python SDK
  • Structuring outputs with JSON schemas
  • Using Files
  • Moving to Production
  • Using the CLI to view job progress and results
  • API Usage Example
• Examples
  • Stage Inference Example with Python SDK
• Batch API Reference
  • Create Batch Inference Jobs
    • Basic Example
  • Retrieving and Managing Job State
    • Job Status
    • Cancelling a Job
    • Retrieving Results
    • Listing All Jobs
  • Job Priority
    • Priority 0 (Prototyping)
    • Priority 1 (Production)
  • Structured Outputs
    • Example
  • Cost Estimates
  • System Prompts
    • Basic System Prompt Example
  • Cumulative Logprobs (Experimental)
  • Using Random Seeds
  • Increasing Quotas
  • Sampling Parameters
  • Batch Inference with Embedding Models
• Stage API Reference
  • Creating Stages
  • Uploading and Downloading Files
    • Uploading a File
    • Downloading a File
  • Stage Metadata
    • Listing Stages
    • Listing Files in a Stage
• Python SDK and CLI Reference
  • Python SDK
    • Basic Methods
    • Job Methods
    • Stage Methods
  • Command Line Interface (CLI)
    • Account Management
    • Job Commands
    • Stage Commands
• Available Models
• Text-to-Text Models
• Reasoning Models (text-to-text)
• Embedding Models
• Custom Models