​
Sampling Parameters

Sampling parameters are a dictionary of parameters that are used to control the sampling process and allow for more customization of outputs. They are passed to the model as a JSON object. By default, we set sampling parameters appropriate for the selected model. You can override the default sampling parameters by passing a dictionary with overrides to the sampling_params parameter in the SDK. For example: 
import sutro as so

sampling_params = {
    "temperature": 0.75,
    "max_tokens": 1024,
    "repetition_penalty": 1.15
}

results = so.infer(
    inputs=...,
    sampling_params=sampling_params
)