Skip to main content

​
Build your first Function

This walkthrough creates a binary classifier in the Sutro app. The same loop applies to judges and extractors.

​
1. Prepare a dataset

Create a CSV with one row per decision. Start with representative, unlabeled production data; around 1,000 rows is a useful default.
Sutro accepts CSV, JSON, JSONL, and Parquet datasets.

​
2. Create the Function

In the Sutro app, open Functions, select Create, and configure:
  • Type: Binary classification
  • Task definition: Should this request be escalated to a human?
  • Dataset: your uploaded file
  • Input column: text
Keep the task definition short and use your own language. Your annotations will teach Sutro the detailed decision rules.

​
3. Review the hard cases

Sutro runs an ensemble over the dataset and opens an annotation workspace. Review:
  • Low confidence: cases where the models disagree
  • High confidence: cases where the models agree, included as a check
  • Held-out: fixed evaluation cases, if you enabled a held-out set
Choose the correct label and add a rationale when the decision depends on policy, context, or expert judgment.

​
4. Optimize and repeat

Complete the iteration to generate an improved prompt. Compare the prompt and metrics with the previous iteration, then run another iteration if important errors remain. When the Function is ready, use Model Sweep to compare deployment models, then open Integrate to copy a working SDK example.

Run the Function

Invoke the published Function from Python.