Model Context Protocol (MCP)
Sutro MCP lets agents drive the Function workflow while keeping human review in the loop.Install from Sutro
Install MCP from within the Sutro app. The app provides the deployment-specific configuration for your agent client. Use the MCP button on your authenticated Sutro deployment to copy a native client setup command. Once connected and authenticated, use After setup, send this prompt to copy the starter prompt into your client’s chat. MCP is unavailable on public trial deployments. A custom deployment origin must be configured by its operator withHARMONIZE_BASE_URL so browser and upload links
point to the correct instance.
Discover a relevant first Function
Ask your agent to use thediscover_first_function MCP prompt. The agent reads
your repository locally, or works from context you supply; the remote MCP server
cannot scan your filesystem. Discovery does not upload repository contents.
The agent checks existing Functions and datasets, then proposes at most three
narrow candidates using evidence from READMEs, schemas, prompt calls, review
workflows, or fixtures. Each proposal should explain:
- The supporting file paths or supplied context, and why the task is useful.
- The required input columns and expected labels or output schema.
- Who can label the task consistently, its operational risk, and missing setup.
- A first offline or shadow evaluation: representative rows, a human reviewer, held-out examples, comparison with the current baseline, success criteria, and a practical cost and latency budget.
Approve the task, transfer, and paid creation
- Approve the exact task definition. After selecting a candidate, review its one- or two-sentence definition. Selecting a general idea does not approve wording you have not seen.
- Approve the exact transfer before upload or append. Review the source
file, path, hosted location, or review-queue row references; selected columns
and row count; excluded or redacted content; destination customer-specific
deployment and dataset filename or
asset collection; and whether the transfer creates, appends to, or overwrites
data. For an append, include the destination Function and existing dataset.
Explicitly approve that transfer, including the append or overwrite. This
applies to every dataset append, including continual-data tools and
resolve_review_rows(action="add")from Needs Review, as well as uploads. Selecting review rows alone does not approve their ingestion into learning data, even when those rows are already on the deployment. Reconfirm scope changes. - Approve paid Function creation. Before calling
create_function, the agent explains that it immediately starts paid ensemble calls for the approved task and dataset, then asks for explicit approval to create the Function and start those calls. Task wording and transfer approvals alone do not approve this billable operation. Reconfirm a changed creation scope.
What an agent can drive
Agents can inspect or upload data, create Functions, monitor runs, open labeling, start optimization, iterate on prompts, and run model sweeps. Changes remain visible in the Sutro app. The agent opens labeling and waits for your feedback. Backend optimizer eligibility determines whether optimization can proceed; completed labels do not justify skipping the optimizer. After optimization, inspect the prompt diff and callget_deployment_status before starting another iteration. Learning
readiness is advisory: you can choose to continue improving the Function.
Before a sweep, review the exact model configurations, prompt source, and paid
optimization scope from prepare_model_sweep. Confirmation binds those choices;
changes require fresh confirmation. Results link to the exact run in Model
Optimization, including historical runs. Failed runs cannot become completion
handoffs. Stopped runs retain partial results for review but cannot be deployed;
stale or unverified results need review and a fresh sweep before selection.
Use Deployment for integration options and current Batch or real-time
availability. A saved model choice does not prove publication succeeded. Failed
publication needs an explicitly approved retry; unknown status needs verification.
An existing published Function remains available while you keep learning. Only
choose or activate a model, or retry publication, after approving that action.
Opening a handoff URL does not start a sweep or deploy a model.
Inspect evaluation quality and changed rows
Ask your agent questions such as:- “How much has this Function improved over its seed prompt?”
- “Which examples got worse after optimization, and why?”
- “Show this row’s predictions across prompt versions and where its annotation came from.”
get_evaluation_history. It compares prompt versions against the current
annotations, evaluation model, and scoring rules, with an optional held-out-only
view. This provides a comparable view of progress; an optimizer validation score
alone does not describe overall Function quality.
get_evaluation_comparison provides label or field breakdowns and pages of
improved, regressed, or unchanged examples. get_evaluation_case opens a
specific case’s inputs, expected and actual outputs, reasoning, explanations,
annotation origin, and performance across versions. For extraction tasks,
partial matches remain visible rather than being described as exact matches.
The agent should inspect this evidence before recommending another iteration,
while keeping prompt review and deployment readiness checks in the workflow.
Reading saved evaluation results does not start model calls. Missing or stale
results remain visible. If you want to refresh them, the agent explains the
scope and paid inference before using replay_evaluation or
start_evaluation_comparison, then monitors results with get_evaluation_run
or get_evaluation_comparison. Your existing authorization for that evaluation
continues to apply. Reviewing results does not authorize changes to your labels
or rationales.