Custom provider

Your team uses an internal model serving cluster, or a provider not yet shipped as a built-in adapter. You don't need agentfootprint to support it explicitly — you just implement two methods. The library treats your custom provider identically to anthropic() or openai().

The contract

import type { LLMProvider, LLMRequest, LLMResponse, LLMChunk, LLMCallHooks } from 'agentfootprint';

interface LLMProvider {
  readonly name: string;

  /** Which roles your wire carries INSIDE the messages array. Optional. */
  readonly carriesInMessages?: readonly ('system' | 'user' | 'assistant')[];

  complete(req: LLMRequest, hooks?: LLMCallHooks): Promise<LLMResponse>;

  stream?(req: LLMRequest, hooks?: LLMCallHooks): AsyncIterable<LLMChunk>;
}

That's the whole interface. complete is required; stream and carriesInMessages are optional (the library falls back to complete if stream is absent).

Wrappers must preserve the request witness

Pass the optional hooks argument to the inner provider in both complete and stream. Forward the whole object, or spread it with { ...hooks } when adding a callback. Rebuilding only its named properties loses the library's private symbol channel. Do not depend on that channel's name or invoke it yourself.

With findings enabled, admission uses the request that produced the accepted answer. A leaf that sends its input unchanged needs no extra preparation report. But a wrapper that trims history, rewrites results or otherwise changes the request must report the actual changed request before delegating through hooks?.onReplayPrepared?.({ provider: name, model: actual.model, request: actual, omissions: [] }), then forward all hooks to the inner provider. Empty omissions means no thinking-retirement claim; never invent omission metadata. If the inner provider prepares the request again, its report must reach the caller too.

Without that report the caller can only inspect the pre-transform request. It could admit a citation to a result your wrapper removed, which the model never saw. Hook forwarding alone does not describe a wrapper's own transformation. Reports belong to the live attempt and must precede transport; do not retain or reuse callbacks after that attempt settles. The built-in retry and fallback decorators implement these boundaries.

Copy the Node-only provider-wrapper conformance helper into your tests and call await assertProviderRequestForwarding(inner => myWrapper(inner)). It is example source, not an npm export. The factory must use only the supplied scripted leaf, without keys, environment files or a live transport. It checks an unchanged or reported transformed request before transport, symbol-hook and replay forwarding, and the wrapper's separate complete/stream paths. Its returned list names only the methods exercised; a complete-only wrapper does not pass a stream test. Disable cache shortcuts for this fixture so the supplied leaf is reached.

This is one synthetic request, not a proof for every transformation, retry, fallback, cancellation or concurrency path. Add your own fixtures for those branches; an identity check never proves the cited assertion true.

carriesInMessages — declare what your wire can carry

Wires disagree about one thing in a way nothing else can detect: whether a role: 'system' message inside the messages array survives. The Anthropic-family adapters drop it (system rides a separate top-level field); the OpenAI-family adapters carry it. A slot: 'messages' injection with that role would therefore arrive on one provider and vanish on another, and nothing in the recording would tell the two apart.

So each adapter states what it carries, and the engine refuses at run start what a provider cannot deliver — naming your provider and its roles — rather than recording a message as injected and dropping it.

The element type is exported as WireRole ('system' | 'user' | 'assistant') if you want to name it in your own signatures.

If you omit the field, you are treated as carrying DEFAULT_CARRIES_IN_MESSAGES — ['user', 'assistant'], the floor every known wire supports, exported so you can assert against it in your own tests. That is deliberately the conservative reading: a missing declaration is not permission. Declare the field if your wire carries more:

class MyCustomProvider implements LLMProvider {
  readonly name = 'my-custom';
  // My API takes system messages inline, like OpenAI's does.
  readonly carriesInMessages = ['system', 'user', 'assistant'] as const;
  // …
}

'tool' is not a member of the list and never will be: a tool message answers a specific tool call, so it is never something an injection can author.

If you write a wrapper, forward the field. A wrapper that rebuilds the provider object and drops it silently narrows everything underneath to the floor — a wrapped OpenAI provider would start refusing system-role delivery it can perfectly well do. The three shipped decorators forward it; withFallback publishes the intersection of its two providers, because a role only one of them carries is a role the call might drop.

A minimal custom provider

class MyCustomProvider implements LLMProvider {
  readonly name = 'my-custom';
  private apiKey: string;

  constructor(opts: { apiKey: string }) {
    this.apiKey = opts.apiKey;
  }

  async complete(req: LLMRequest): Promise<LLMResponse> {
    const res = await fetch('https://my-llm-api.example.com/v1/chat', {
      method: 'POST',
      headers: { 'Authorization': `Bearer ${this.apiKey}` },
      body: JSON.stringify(toMyApiFormat(req)),
    });
    const json = await res.json();
    return fromMyApiFormat(json);
  }
}

The two converters (toMyApiFormat, fromMyApiFormat) are the only API-specific code. The agent loop, recorders, memory, skills — all work unchanged.

Stream implementation

For UI-streaming support:

async *stream(req: LLMRequest): AsyncIterable<LLMChunk> {
  const res = await fetch('https://my-llm-api.example.com/v1/stream', { ... });
  const reader = res.body!.getReader();
  let tokenIndex = 0;
  let accumulated = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    const text = new TextDecoder().decode(value);
    accumulated += text;
    yield { tokenIndex: tokenIndex++, content: text, done: false };
  }

  // Final chunk MUST carry the authoritative LLMResponse
  yield {
    tokenIndex,
    content: '',
    done: true,
    response: {
      content: accumulated,
      toolCalls: [],   // populate from API's terminal payload
      usage: { input: 0, output: 0 },
      stopReason: 'stop',
    },
  };
}

The agent reads response off the final chunk to drive ReAct decisioning. UI consumers read intermediate content chunks for token streaming. One round-trip, two consumers.

AbortSignal support

The agent puts the run's signal on every request as req.signal. It fires when the caller aborts the run or the run's timeoutMs passes. Hand it to your transport, so a cancelled run cancels the HTTP request itself and stops paying for tokens nobody will read:

async complete(req: LLMRequest): Promise<LLMResponse> {
  const res = await fetch(url, {
    signal: req.signal,
    // ...
  });
  // ...
}

An SDK usually takes it as a per-request option (client.create(params, { signal: req.signal })). In stream(), close the SDK stream or cancel the response body when the signal fires. If your provider ignores the signal, the agent still stops waiting for it and never records an answer that arrives after the abort. The remote call keeps running until it finishes, though, and you are billed for it. When the call fails because of the abort, throw req.signal.reason. The decorators (withRetry, withFallback, withCircuitBreaker) never retry, fall back from or count a cancelled call, whatever the error is called.

The library's resilience decorators (withRetry, withFallback) wrap your custom provider transparently — no special integration needed.

Wrapping with resilience

Drop into the standard composition:

import { withRetry } from 'agentfootprint/resilience';

const provider = withRetry(new MyCustomProvider({ apiKey: '...' }), {
  maxAttempts: 5,
});

withRetry classifies retryability by HTTP status: it reads err.status (or err.statusCode) off the thrown error, skips AbortError and 4xx client errors — except 429 Too Many Requests, which IS retried — and retries 5xx, network errors, and unknown shapes — unless the error declares retryable: false, which it never retries. So if your provider throws errors that should be retryable, surface the upstream HTTP status on the error:

async complete(req: LLMRequest): Promise<LLMResponse> {
  const res = await fetch(url, { signal: req.signal /* ... */ });
  if (!res.ok) {
    const err = new Error(`${res.status} from upstream`) as Error & { status: number };
    err.status = res.status; // withRetry reads this to decide retryability
    throw err;
  }
  // ...
}

Need a custom rule (e.g. retry a specific provider-defined code)? Pass shouldRetry:

withRetry(new MyCustomProvider({ apiKey: '...' }), {
  shouldRetry: (err, attempt) => (err as { status?: number }).status === 503,
});

What to normalize — and where the built-in providers still differ

Your fromMyApiFormat converter has one job beyond copying text: map your vendor's vocabulary onto the shared one, the same way the built-in adapters do. The agent loop reads only the normalized fields, so code that works on anthropic() works on yours.

A few differences are not normalized away, because they are real:

• Extended thinking. Agent.create(...).thinking({ budget }) puts thinking: { budget } on every request — the intent, not a wire shape. Anthropic sends the shape each model takes (a budget, or adaptive thinking with none — see Anthropic), Gemini sends its thinking budget, Ollama turns thinking on; OpenAI ignores it, because its reasoning is picked by the model id. If your API has no such knob, ignore req.thinking. If your models disagree, declare thinkingMode(model) ('budget', 'adaptive' or 'none') so the agent refuses a model that cannot think at build — and forward it from any wrapper you write.
• Forced tool choice. Keep carriesForcedToolChoice boolean. If only some models support it, add supportsForcedToolChoice: (model) => boolean; absence preserves the wire flag's behavior. Call forcedToolChoiceFor(provider, model) from agentfootprint/providers to read both declarations. Resolve your adapter's shorthand inside the predicate and forward it unbound from wrappers. A fallback pair requires both sides to support the requested model. Enforce the same restriction in direct provider calls, before sending, with UnsupportedToolChoiceError.
• System messages inside messages. Some wires carry them, some drop them — that is what carriesInMessages (above) declares.
• Model ids. They are the vendor's. A Bedrock id is not an Anthropic API id, and on Azure OpenAI or Foundry the model is your deployment name. Switching providers usually means switching the model string too.

If your application code starts branching on which provider is attached, stop and write against the normalized fields instead.

Picking a built-in provider from config — createProvider()

When the provider is chosen at runtime — an env var, a feature flag, a tenant setting — createProvider builds any built-in provider from one tagged object. kind picks the adapter; the rest of the object is that factory's own options:

import { createProvider, type CreateProviderOptions } from 'agentfootprint/providers';

const provider = createProvider({
  kind: process.env.LLM_PROVIDER ?? 'mock',   // 'anthropic', 'openai', 'bedrock', …
  apiKey: process.env.LLM_API_KEY,
  defaultModel: process.env.LLM_MODEL,
} as CreateProviderOptions);

The kinds are mock, anthropic, openai, ollama, foundry, foundry-local, bedrock, gemini, browser-anthropic, browser-openai and invoke-model-gateway (your gateway) (the type is exported as ProviderKind). An unknown kind throws, naming it.

Gotchas:

• Custom providers don't go through it — pass your instance to Agent.create({ provider }) directly.
• No Azure kind. Call azureOpenai() (or browserAzureOpenai()) directly.
• The as CreateProviderOptions cast is the price of a kind read from a string: TypeScript can't check that the options match a kind it only learns at runtime. With a literal kind: 'anthropic' you get full checking and need no cast.
• Credentials already in the environment? providerFromEnv() (main barrel) reads them and picks the provider itself — see Ollama and Microsoft Foundry for how it decides.

Anti-patterns

• Don't bake retry logic into complete. Let withRetry handle it; your provider should be a thin SDK wrapper.
• Don't swallow the upstream HTTP status. Surface it on the thrown error (err.status / err.statusCode) so withRetry's default classification works (4xx terminal vs 5xx/429 retryable).
• Don't omit usage from the response — recorders depend on it for cost tracking.

Next steps

• Anthropic — reference adapter implementation
• OpenAI — reference adapter implementation
• Resilience — your custom provider gets retry + fallback for free