Skip to content

Architecture overview

About these pages

The Internals section walks through how the llm package works inside, for contributors and the curious. The public API is documented under LLM; this section is about the implementation.

or/llm is a stateless translation layer. It decides what to send for one request and how to interpret the streamed response, and leaves history storage, context compaction, and tool-loop orchestration to the caller.

Two layers

Layer Location Responsibility
Public facade llm/ Type aliases and thin forwarding so callers import one package
Internal core llm/ The real implementation, plus per-protocol adapters under providers/

Request data flow

flowchart TD
    A["llm.Complete / Stream"] --> B["Client.Stream"]
    B --> C{"registry.Get(model.Protocol)"}
    C -->|anthropic-messages| D["Anthropic adapter"]
    C -->|openai-completions| E["OpenAI adapter"]
    D --> F["convertMessages → SDK request"]
    E --> F
    F --> G["StreamWriter: Emit / Done / Fail"]
    G --> H["chan Event → caller"]

The Protocol field on the model is the discriminator: Client.Stream uses it to pick an adapter from the registry.

Reading a request end to end

1
2
3
4
5
6
func (c *Client) Stream(ctx context.Context, model Model, input Context, options StreamOptions) (<-chan Event, error) {
    // ... validation ...
    adapter, ok := c.registry.Get(model.Protocol) // (1)!
    // ... inject API key ...
    return adapter.Stream(ctx, model, input, options)
}
  1. Protocol selects the adapter. The same conversation can target either protocol; the library re-adapts the history per request.

Source: llm/client.go.

Where to go next