Model Runtime, Provider Composition

composeModelProvider merges extension Providers with built-in

Learning Objectives

After this chapter you will be able to:

• Explain how ModelRuntime distinguishes available (authed) from all (complete catalog)
• Describe how composeModelProvider merges extension Providers with the built-in catalog
• Explain how resolveCliModel parses provider/model:thinking shorthand
• Understand the validation and merge strategy for extension registerProvider

Key Source Files

File	Lines	Responsibility
packages/coding-agent/src/core/model-runtime.ts	595	ModelRuntime: available/all, auth passthrough
packages/coding-agent/src/core/model-resolver.ts	726	resolveCliModel: shorthand parsing + diagnostics
packages/coding-agent/src/core/provider-composer.ts	548	composeModelProvider: merge extension with built-in
packages/coding-agent/src/core/auth-storage.ts	~150	auth.json backend: persist authentication state

Design Motivation: Why a Runtime Layer?

Problem: The pi-ai package (→ Chapter 6) provides a static Models collection. But coding-agent needs:

• Extensions can inject new Providers (→ Chapter 13 registerProvider)
• Users may have authed only some Providers (show only available)
• CLI shorthand needs intelligent parsing and error diagnostics
• Runtime model switching needs to notify the UI

Analogy: pi-ai's Models is the "car catalog"; ModelRuntime is the "dealership system" — the catalog lists all models, the dealership knows which are in stock (authed), which can be ordered (unauthed), and how to recommend based on customer needs (resolveCliModel).

ModelRuntime: available vs all

class ModelRuntime {
  private models: Models          // pi-ai's Models collection
  private extProviders: Provider[] // extension-injected Providers

  // Authed models — UI list shows only these
  get available(): Model[] {
    return this.allModels.filter(m => m.authed)
  }

  // All models — including unauthed (for "available after login" hints)
  get all(): Model[] {
    return [...this.models.list(), ...this.extModels]
  }

  // Auth passthrough: let UI trigger interactive login
  async getAuth(provider: string): Promise<AuthResult> {
    return this.models.getProvider(provider).auth()
  }
}

Why distinguish available/all?

• TUI model picker (Ctrl+P) shows only available — avoids selecting unusable models
• But "Settings" page shows all + "requires login" badge — guides users to authenticate
• Error diagnostics: "You want gpt-4o, but openai isn't authed. Run pi login openai first"

provider-composer: Merge Strategy

function composeModelProvider(
  builtin: ProviderConfig,
  extension: ExtensionProviderConfig
): ProviderConfig {
  // ① Validate extension Provider
  validateExtensionProvider(extension)  // id unique, baseUrl valid

  // ② Merge headers (extension overrides built-in)
  const headers = { ...builtin.headers, ...extension.headers }

  // ③ Merge flags
  const flags = {
    openaiCompletionsCompat: extension.openaiCompletionsCompat ?? builtin.openaiCompletionsCompat,
    // ...
  }

  // ④ Produce merged Provider
  return { ...builtin, ...extension, headers, flags }
}

What extensions CAN do:

• Inject private/local Providers (enterprise intranet LLM gateway)
• Override baseUrl (proxy forwarding)
• Add headers (gateway signatures)
• Declare OpenAI compat mode (route Ollama/vLLM through OpenAI wire protocol)

What extensions CANNOT do:

• Delete built-in Providers
• Override auth logic (security)
• Modify other extensions' Providers

resolveCliModel: Shorthand Parsing + Diagnostics

function resolveCliModel(spec: string): ScopedModel | DiagnosticError {
  // "openai/gpt-4o:high" → { provider: "openai", model: "gpt-4o", thinking: "high" }
  const [fullName, thinking] = spec.split(":")
  const [provider, model] = fullName.split("/")

  if (!provider || !model) {
    // Try fuzzy match: "gpt-4o" → find provider from catalog
    const match = fuzzyMatch(spec)
    if (match) return match
    return { error: `Cannot parse "${spec}", expected format: provider/model[:thinking]` }
  }

  // Validate provider exists
  if (!knownProviders.has(provider)) {
    return { error: `Unknown provider "${provider}", available: ${[...knownProviders].join(", ")}` }
  }

  // Validate model exists
  if (!providerModels(provider).includes(model)) {
    return { error: `"${provider}" has no "${model}", available: ${providerModels(provider).join(", ")}` }
  }

  return { provider, model, thinking: thinking as ThinkingLevel }
}

Diagnostics first: Not a cold "invalid input" — tells you what's wrong and what's available.

Engineering Insights

1. model-resolver.ts 726 lines: Mostly diagnostic logic — fuzzy matching, spelling suggestions ("did you mean gpt-4o?"), multi-provider same-name disambiguation. Good error messages are harder than good algorithms.

2. provider-composer.ts 548 lines: Merge logic handles deep nesting (headers as function vs object, flags with inheritance semantics). Uses structuredClone + field-by-field merge, not simple spread.

3. auth-storage.ts: Persists "which providers are authed" to ~/.pi/auth.json. ModelRuntime reads at startup, avoiding auth resolution attempts on every list call.

Cross-Chapter Links

Direction	Chapter	Relationship
Prerequisite	Ch06 Models & Providers	ModelRuntime wraps pi-ai's Models
Prerequisite	Ch08 Auth & OAuth	available based on auth state
Prerequisite	Ch13 Extensions	registerProvider injects extension Providers
Follow-up	Ch10 CLI Modes	--model flag goes through resolveCliModel
Follow-up	Ch16 Interactive TUI	Ctrl+P model picker uses available
Follow-up	Ch09 Streaming & Retry	Retry failure can trigger model downgrade

Check Your Understanding

1. If the UI showed all instead of available, what user experience problems would arise?
2. Why can't extensions override auth logic? What security risk would that create?
3. In what scenarios might resolveCliModel's fuzzy matching give wrong suggestions? How to prevent?

Takeaways

• available = authed (UI-selectable), all = complete (includes "requires login")
• composeModelProvider merges extension with built-in — extensions can add, never delete
• resolveCliModel diagnostics first — tells you what's wrong and what's available
• Runtime model switch notifies UI + writes to session (→ Chapter 11)

---

This chapter is based on the main-branch source of earendil-works/pi.