Request options¶

StreamOptions contains settings shared by all protocols. Settings whose semantics differ between protocols are nested under ProtocolOptions.

temperature := 0.2
retries := 2
options := llm.StreamOptions{
    Temperature: &temperature,
    MaxTokens:   2048,
    MaxRetries:  &retries,
    Timeout:     30 * time.Second,
    Headers: map[string]string{
        "X-Request-ID": requestID,
    },
}

The shared options are:

Option	Purpose
`APIKey`	Override credential lookup for this request
`Env`	Override named environment values without mutating the process environment
`Temperature`	Override the model sampling temperature
`MaxTokens`	Cap output tokens; zero leaves the model default
`Headers`	Merge custom HTTP headers over model defaults
`Reasoning`	Request a provider-neutral reasoning effort level
`ProtocolOptions`	Carry settings for exactly one wire protocol
`MaxRetries`	Override SDK retries; `0` disables them
`Timeout`	Cap each HTTP attempt independently of context cancellation
`OnRequest`	Observe every serialized HTTP request attempt
`RewriteRequest`	Replace the serialized request body before it is sent
`OnResponse`	Observe every HTTP response attempt

Supply credentials per request¶

By default the package reads the selected provider's key from the process environment. To override that for a single request—for example a multi-tenant server holding each user's key—set APIKey directly, or supply named environment values through Env without mutating the process environment.

// A literal key for this request only.
options := llm.StreamOptions{APIKey: userKey}

// Or resolve the provider's environment variable from a per-request source.
options := llm.StreamOptions{
    Env: llm.ProviderEnv{"DEEPSEEK_API_KEY": userKey},
}

APIKey takes precedence; Env is consulted before the process environment. The full order, highest to lowest, is: StreamOptions.APIKey, a provider override key, StreamOptions.Env, override Env, then the process environment.

Credential discovery can be inspected without sending a request:

Function	Result
`APIKeyEnvVars(providerID)`	Environment-variable names configured for the provider
`FindEnvAPIKeys(providerID)`	Names whose values are currently set in the process environment
`DefaultProviderRegistry().AuthStatus(providerID, env)`	Whether a credential resolves, its source, and missing variable names

FindEnvAPIKeys returns secret values. Do not write its result to logs.

Observe HTTP requests and responses¶

The hooks are useful for logging, tracing, and debugging. Both fire once per attempt, so retries remain visible. OnRequest receives the exact request body serialized for the provider, including protocol-specific fields.

options := llm.StreamOptions{
    OnRequest: func(method, url string, body []byte) {
        log.Printf("→ %s %s\n%s", method, url, body)
    },
    OnResponse: func(status int, headers http.Header) {
        log.Printf("← %d", status)
    },
}

Hooks may expose prompts, tool arguments, credentials in URLs or headers, and provider response metadata. Redact sensitive data before sending it to logs or telemetry systems.

Hooks run synchronously in the request path. Slow hook code delays the request, and a blocking hook can prevent an attempt from completing. The callback API does not return errors; perform fallible telemetry work outside the callback or record failures separately.

Rewrite the request body¶

RewriteRequest transforms the serialized request body before it is sent, an escape hatch for provider-specific fields the typed API does not expose. It receives the same method, URL, and body as OnRequest and returns the body to send; returning nil leaves it unchanged. Like the observers it fires once per attempt and always rewrites the original body, so retries stay consistent.

options := llm.StreamOptions{
    RewriteRequest: func(method, url string, body []byte) []byte {
        var payload map[string]any
        if err := json.Unmarshal(body, &payload); err != nil {
            return nil // leave the body unchanged on error
        }
        payload["custom_provider_field"] = true
        patched, err := json.Marshal(payload)
        if err != nil {
            return nil
        }
        return patched
    },
}

See Reasoning options and Tool definitions and calls for the protocol-specific option types currently included with the package.

Use a custom HTTP client¶

The package-level functions use the default Client. To supply a custom *http.Client—for example for a proxy, transport instrumentation, or custom TLS configuration—construct an explicit client:

httpClient := &http.Client{Timeout: 45 * time.Second}
adapters := llm.NewAdapterRegistry()
if err := adapters.Register(openai.NewAdapter(httpClient)); err != nil {
    log.Fatal(err)
}
client := llm.NewClient(adapters, llm.NewBuiltInProviderRegistry())

response, err := client.Complete(ctx, model, input, options)

Client has no Close method. Resource ownership remains with the *http.Client passed to openai.NewAdapter or anthropic.NewAdapter. See Clients and registries for imports and isolated adapter and provider registries.