> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sutro.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Improve a Function

> Use disagreement, annotations, and held-out examples to refine behavior.

# Improve a Function

Each iteration surfaces a small number of cases for expert review to gather
additional context on how the task should be carried out.

## Review three kinds of examples

* **Low confidence:** samples that are ambiguous and require expert annotation.
* **High confidence:** samples where there is majority consensus, but which are
  worth auditing to ensure the task is well understood.
* **Held-out set (optional):** preserves a fixed set to ensure the task
  generalizes. These samples are not seen by the optimizer while learning the task.

Correct the label or extracted fields. Add a rationale when the prompt needs a
rule that is not obvious from the label alone.

<Tip>
  If your rationale differs from the model's, explain your decision rule directly
  to guide its task understanding. Rationales are highly valuable for transferring
  the nuances of your task preferences to the model.
</Tip>

## Run the optimizer

After review, run the optimizer. Sutro abstracts your annotations, corrections, and
rationales into generalized rules for the next iteration, helping the Function
apply your feedback to new cases rather than memorize individual examples.

## Complete an iteration

### Use metrics to track progress

**Model consensus** measures task-level consistency across various models.
**User/model agreement** measures how often reviewed predictions match your annotations.
**Optimization metrics** measure how well a model can learn the task from the annotated data.

### Review the prompt diff

Review the prompt diff and validation results before starting the next
iteration.

Sutro automatically indicates when a Function appears to have learned the task
and is ready for deployment. From there, we recommend running a model sweep.

## Select a deployment model

Use **Model Sweep** to compare supported models on your annotated rows. Sutro
reports quality and estimated cost, then lets you deploy the selected
prompt/model combination. This allows you to better understand cost and performance
trade-offs for your task, or choose between open-source and proprietary models.

## Continue improving

You can return later with new production data and continue from the same
annotation history, or choose a new model to optimize for your task.