Models & Provider Abstraction

Models route to their Provider — header merge has a priority chain

Learning Objectives

After this chapter you will be able to:

• Explain how the Models collection unifies 40+ vendors behind a single stream() call
• Describe the 5-level header priority chain and its merge logic
• Explain why auth "resolves without making a request"
• Understand how mixed-API Providers support multiple wire protocols under one provider

Key Source Files

File	Lines	Responsibility
packages/ai/src/models.ts	705	Models collection: routing, header merge, stream delegation
packages/ai/src/types.ts	783	Provider / Model / StreamOptions type definitions
packages/ai/src/providers/all.ts	151	40+ provider registry

Design Motivation: Why a Unified Abstraction?

Problem: Every LLM vendor has a different API — Anthropic uses Messages API, OpenAI uses Completions, Google uses Generative AI, Bedrock uses Converse. If upper-layer code talks to each directly, adding one provider means changing N call sites.

Analogy: Like JDBC for databases — application code writes connection.query(), drivers adapt to MySQL/PostgreSQL/Oracle. pi-ai's Models is "JDBC for LLMs."

Design goals:

• Upper layer (agent loop) calls only models.stream(opts) → unified event stream
• Adding a provider = adding one file, zero changes upstream
• auth, headers, retry policies vary independently

The Provider Contract

interface Provider<TApi> {
  id: string                    // "anthropic" | "openai" | "bedrock" | ...
  baseUrl: string               // API endpoint
  headers: HeaderFn             // base headers (e.g. API version)
  auth: (m: Model) => AuthResult  // resolve credentials, no HTTP
  api: StreamApi<TApi>          // concrete wire protocol (→ Chapter 7)
  models: Model[]               // models owned by this provider
}

Key constraint: auth is pure resolution — reads env/credential store, makes no network call. Network I/O is deferred to inside api.stream(). This lets Models.stream() assemble the request synchronously; async only happens when HTTP actually fires.

Models.stream(): Route + Merge + Delegate

class Models {
  async stream(opts: StreamOptions): Promise<AssistantMessageEventStream> {
    // ① Route: find the provider that owns this model
    const provider = this.owner(opts.model)

    // ② Resolve auth (pure local, no request)
    const auth = await provider.auth(opts.model)

    // ③ 5-level header merge
    const headers = mergeHeaders(
      provider.headers,          // 1. provider base
      opts.model.headers,        // 2. model-level override
      opts.headers,              // 3. caller override
      opts.transformHeaders,     // 4. final transform (e.g. gateway signature)
    )

    // ④ Delegate to the concrete wire protocol
    return provider.api.stream({ ...opts, headers, auth })
  }
}

The 5-Level Header Priority Chain

Low priority ─────────────────────────────────── High priority
provider.headers → model.headers → options.headers → transformHeaders
     │                  │                │                  │
  API version      model-specific    per-request       gateway sig/proxy
  (fixed)          needs             caller override   (last resort)

Why transformHeaders? Enterprise deployments often have API gateways (AWS API Gateway, custom proxies) that require a signature header on every request. transformHeaders lets callers inject this without modifying provider code.

Mixed-API Providers

One Provider, multiple wire protocols:

Copilot Provider
  ├── model.api = "openai-completions"  → openai-completions.ts
  ├── model.api = "anthropic-messages"  → anthropic-messages.ts
  └── model.api = "codex-responses"     → openai-codex-responses.ts

Aggregators (Copilot, OpenCode, OpenRouter) proxy multiple model families behind one endpoint. pi dispatches by model.api to the correct wire protocol — provider means "who bills you," api means "what language they speak."

createProvider: Custom Providers

// Let Ollama / vLLM / enterprise proxies use the same path
const myProvider = createProvider({
  id: "my-proxy",
  baseUrl: "http://localhost:8080/v1",
  api: "openai-completions",     // reuse OpenAI wire protocol
  models: [{ id: "llama-3.1-70b", api: "openai-completions" }],
})
models.register(myProvider)

Users write zero wire-protocol code — as long as the proxy speaks OpenAI-compatible format, one createProvider call plugs it in.

Engineering Insights

1. 705 lines in models.ts: Most of it is header merging, model lookup, and error classification. The actual "routing" logic is ~20 lines — complexity lives in handling 40 vendors' quirks.

2. Model is a value object: Model<TApi> carries id, provider reference, contextWindow, maxOutput, pricing. Upper layers make decisions by model (compression threshold, cost estimation) without knowing provider internals.

3. all.ts is only 151 lines: Because each provider is a separate file (anthropic.ts, openai.ts...), all.ts just imports + registers. Adding a provider never touches existing code.

Cross-Chapter Links

Direction	Chapter	Relationship
Prerequisite	Ch01 Agent Loop	streamFn ultimately calls Models.stream()
Follow-up	Ch07 Wire Protocols	Concrete wire protocol behind provider.api
Follow-up	Ch08 Auth & OAuth	Credential resolution inside provider.auth
Follow-up	Ch14 Model Runtime	Runtime dynamic model/provider switching

Check Your Understanding

1. If auth resolution required an HTTP request (instead of being purely local), what impact would that have on callers of Models.stream()?
2. Why does header merging need 5 priority levels rather than simply "caller overrides everything"?
3. In a mixed-API Provider, what different dimensions do provider and model.api represent?

Takeaways

• One Models.stream() interface, 40 vendors — upper layer is oblivious
• auth resolves locally; network I/O deferred to actual streaming
• 5-level header priority chain: provider → model → options → transform
• Mixed-API: provider = "who bills you," api = "what language they speak"

---

This chapter is based on the main-branch source of earendil-works/pi.