Cost Estimates
A significant benefit to batch inference is decreased costs as well as transparent pricing as inputs are known in advance. We aim to provide transparent pricing models so you know in advance how much a batch job will cost before running it.Understanding Pricing
We charge based on the count of input and output tokens that successfully complete inference. Our pricing page contains the average cost per million-token for each model, blending both input and output tokens and weighted according to typical usage patterns. However, output tokens are generally more expensive than input tokens, and we do charge differently for each.Using the Dry Run Feature
To estimate a batch job, setdry_run=True in the Python SDK. The SDK creates an estimate job, waits for completion, prints the estimate, and returns the estimate job ID. If you call the HTTP API directly, use cost_estimate=true in the request body; the submission response contains the estimate job ID, and the completed job resource contains cost_estimate.
Estimate jobs do not launch the normal full job. Priority-1 estimates at or above the sampling threshold run inference on a prefix sample of approximately 1 million input tokens. For workloads near that threshold, the sample can include every row. We recommend creating an estimate before a large job.