Your gateway (InvokeModel)

Many organisations reach Claude through their own gateway: plain HTTPS, an API-key header, and the Bedrock InvokeModel operation. bedrock() signs with AWS credentials against the Converse operation, and providerFromEnv() has no base URL for this wire — so neither reaches it. invokeModelGateway() does, with no SDK.

Use

const provider = withRetry(  invokeModelGateway({    baseUrl: 'https://llm-gateway.example.com/bedrock',    apiKeyHeader: 'api-key',    // One key per model: this gateway scopes keys, and the wrong one is a 403.    apiKey: { [HAIKU]: 'haiku-key', [SONNET]: 'sonnet-key' },    model: HAIKU,    fetch: scriptedGateway(log), // ← drop this line to talk to the real gateway  }),); // a 429, a 5xx or a network failure is retried by withRetry, never inside the adapter

Then Agent.create({ provider, model: 'invoke-model-gateway' }). Prefer new? new InvokeModelGatewayProvider(options) is the same provider as a class. The shorthand 'invoke-model-gateway' sends the model you configured; any other model name is sent as given, so one provider can serve several models.

The options object is typed InvokeModelGatewayOptions:

A key per model

Gateways often scope a key to some models, and asking with the wrong one gets a 403. Give each model id its own key with a map (as above) or a function. A model the map does not name is refused before any request is sent, and the error lists the ids the map does name. The key itself is never in an error message.

What goes on the wire

The body is Anthropic's Messages API, with anthropic_version: "bedrock-2023-05-31" and no model field (the model is in the path). The system prompt rides the top-level system field; tool calls go out as tool_use blocks and results come back as tool_result blocks on a user turn. A forced tool (tool_choice) is supported.

An assistant turn in the history with no text and no tool calls is left out of the body. The API accepts empty content only on a final assistant message, so sending one mid-conversation would fail the whole request with a 400 that no retry can mend. The turns on either side may then share a role; the API combines consecutive same-role turns.

stream() reads the SSE the gateway sends after decoding AWS's binary framing. Bare data: lines and event: + data: pairs both work. Text arrives as chunks; tool arguments are reassembled from their input_json_delta fragments, and the final chunk carries the full response (tool calls, usage, stop reason).

Errors

Every failure is an InvokeModelGatewayError with a reason (typed InvokeModelGatewayErrorReason) and the modelId:

"Prompt is too long" becomes a ContextWindowExceededError instead, and a cancelled request is passed through as the AbortError it is.

Retries: compose withRetry

The provider makes one attempt per call and has no retry loop of its own. Every InvokeModelGatewayError carries retryable: true for a 429, a 5xx, a network failure and a timeout, false for everything else. withRetry's default policy honours the false, so it retries exactly those four:

• A 403 or 404 is an answer, not a passing fault, and fails straight away.
• A refusal raised before any request (no-key, no-model) is never repeated, and nothing waits out a backoff for it.
• A 2xx the provider could not read (unreadable-response) is never re-sent: the model may already have run, and billed.

The wait the gateway states is honoured. A throttling gateway often says how long to wait only in the 429 body — "Rate limit is exceeded. Try again in 4 seconds." — with no Retry-After header. The provider reads that wording (the header wins when both are present) and declares it as retryAfterMs; withRetry then waits the longer of its own backoff and the stated wait, capped by maxDelayMs (default 10 s). Without this, short backoffs spent every attempt inside the stated wait.

const log: string[] = [];let waited = 0;const provider = withRetry(  invokeModelGateway({    baseUrl: 'https://llm-gateway.example.com/bedrock',    apiKeyHeader: 'api-key',    apiKey: 'sonnet-key',    model: SONNET,    fetch: scriptedGateway(log, true),  }),  {    initialDelayMs: 10,    // The 429 said "Try again in 1 seconds": the wait is max(10 ms, 1000 ms),    // capped by maxDelayMs (default 10 s) so no header can stall a run.    onRetry: (_err, _attempt, delayMs) => (waited = delayMs),  },);await provider.complete({  model: 'invoke-model-gateway',  messages: [{ role: 'user', content: 'hi' }],});

withRetry retries stream() too, but only until its first chunk: every failure this provider raises before the first chunk (an HTTP status, a network error, a timeout) is retried, and nothing after it is.

Deadline: timeoutMs

With no deadline, a gateway that never answers holds the run until your own req.signal fires, if you set one. timeoutMs bounds each wait:

• the response headers;
• a complete() body;
• each stream read: the first chunk, then every gap between chunks. A stream that keeps sending is never cut off, however long it runs;
• the response body close, including a transport whose cancellation never settles.

A missed deadline aborts the request (a real fetch drops the connection) and raises reason: 'timeout' with retryable: true. Your req.signal still wins: aborting it rejects with your signal's reason (an AbortError when you gave none), never a timeout or network error. With no timeoutMs the request is sent exactly as it was before this option existed.

const provider = invokeModelGateway({  baseUrl: 'https://llm-gateway.example.com/bedrock',  apiKeyHeader: 'api-key',  apiKey: 'sonnet-key',  model: SONNET,  // Bounds the response, a complete() body, and EACH stream read (the first  // chunk and every gap) — a stream that keeps talking is never cut off.  timeoutMs: 50,  fetch: () => new Promise<Response>(() => undefined), // never answers});try {  await provider.complete({    model: 'invoke-model-gateway',    messages: [{ role: 'user', content: 'hi' }],  });  return '(answered — unexpected)';} catch (err) {  if (err instanceof InvokeModelGatewayError && err.reason === 'timeout') {    return `${err.reason}, retryable: ${err.retryable}`;  }  throw err;}

Next steps

• AWS Bedrock — bedrock(), for AWS credentials and the Converse API
• Resilience — withRetry / withFallback
• Custom provider — createProvider({ kind: 'invoke-model-gateway', ... })