Prompt caching

An agent loop sends the same system prompt and tool descriptions on every iteration. On providers with prompt caching, those repeated bytes can cost a fraction of normal input tokens — but only if the request marks which blocks are stable. agentfootprint does that marking for you — the conversation so far included.

It is on by default. Most agents never touch it.

Turning it off

import { Agent } from 'agentfootprint';
import { anthropic } from 'agentfootprint/providers';

const agent = Agent.create({
  provider: anthropic({ defaultModel: 'claude-sonnet-4-5' }),
  caching: 'off', // the only accepted value; leave unset to keep caching on
})
  .system('…')
  .build();

Turn it off when:

• running tests against mock() — there is no real provider, so caching is moot;
• debugging context assembly and you want the request exactly as built, every iteration;
• running a token-cost comparison and you need an uncached baseline.

In production, leave it on: a built-in gate (below) already stands caching down on the turns where it would cost more than it saves.

What gets cached — a policy per injection

Each piece of context says how long it stays stable. The defaults follow how each kind behaves:

Override on any injection:

import { defineSkill } from 'agentfootprint/context';

const triage = defineSkill({
  id: 'port-error-triage',
  description: 'Diagnose port errors on a storage switch.',
  body: '…',
  cache: 'always', // keep the body cached even while the skill is inactive
});

The values are 'always', 'while-active', 'never', or { until: (ctx) => boolean } — cached until the predicate returns true. The predicate sees the iteration, the iterations remaining, the user message, the last tool called and the input tokens used so far:

cache: { until: (ctx) => ctx.cumulativeInputTokens > 50_000 }

When caching stands itself down

Every iteration, before the model is called, a gate decides whether to place cache markers at all. It skips them when:

1. caching is off for the agent;
2. a supplied hit rate is below 0.3 (the agent computes none yet);
3. three or more different skills were active in the last five iterations — each switch invalidates the cache, so marking would only add write cost.

Under .findings(), stable tool definitions and immutable system instructions can be cached. The changing advisory ID list and ledger declarations are late request-only DATA, outside the cache breakpoints. See the layout and limits.

Which providers, and how

Each provider declares how its wire caches (promptCaching, a PromptCaching from agentfootprint/cache); the agent picks its strategy from that, never from the provider's name:

Bedrock support comes from the adapter's model table. A cache point below the model's minimum size does not cache.

A wrapper keeps caching when it forwards the field (withRetry, withCircuitBreaker and withFallback do):

const routed: LLMProvider = {
  name: `my-app/${inner.name}`,
  ...(inner.promptCaching !== undefined && { promptCaching: inner.promptCaching }),
  complete: (req, hooks) => inner.complete(req, hooks),
};

Without it the wrapper runs uncached, and its receipts say 'none'.

Did it pay off?

cacheRecorder() (also on agentfootprint/cache) reports what caching cost and saved on a turn — and says "not measured" rather than zero when a provider reported nothing. See The cache meter.

Gate or stuff? Both work now

Without caching, the only sensible shape for a skill-heavy agent is to gate: put only the active skill's text in the prompt. With caching, stuffing every skill into the prompt and letting the cache carry it becomes reasonable too. Measured on a 10-skill, 18-tool agent against the live Anthropic API (input tokens for one task):

Gating still uses fewer tokens and has a faster first call (no large cache write), but stuffing is now a matter of taste rather than cost.

Findings uses a stable prefix

With .findings(), library-owned _findings tool definitions stay fixed as result IDs and standings change. The fixed findings instruction can be cached; mutable declarations and the current advisory IDs are served in a final request-only DATA frame, after the last cache breakpoint. This frame is quoted model/record data, not a new human request or verified facts. Judge and argument rows are not added to it.

// The example advertises breakpoint support to test placement. In an app,// use the chosen adapter's own model-aware capability declaration.const provider = {  name: inner.name,  complete: inner.complete.bind(inner),  promptCaching: { mode: 'breakpoints' as const, maxBreakpoints: 4, reportsUsage: true },};const agent = Agent.create({ provider, model: 'mock', maxIterations: 5 })  .system('Use the synthetic fixture to answer.')  .tool(    defineTool({      name: 'read_fixture',      description: 'Read a local fixture.',      inputSchema: { type: 'object', properties: {} },      execute: () => ({ status: 'ready' }),    }),  )  .findings({ maxContextBytes: 4096 })  .build();await agent.run({ message: input });// Mutable findings DATA is request-only, after the committed history and// its cache markers. Citation admission still checks actual served results.

maxContextBytes is a host-owned UTF-8 ceiling: default 16,384, a positive whole number between 1,024 and 262,144. Whole rows or IDs that do not fit are omitted, with per-bucket counts; IDs are never shortened. A source that cannot be read as bounded plain JSON is unknown as a whole, including its row count. Missing entries must not be interpreted as a healthy result or proof of absence.

The advisory list is not a citation grant. The existing admission check still uses the actual unique executed results shown in the request producing the accepted answer. IDs found only in DATA or in the model's own statements cannot authorize a tool. Providers and wrappers must still report any private request rewrite through the existing hooks.

Global caching: 'off', host cache restrictions and unsupported models remain authoritative. Tool activation, window changes and collapsing judged historical results can still change a prefix and miss the cache. Vendor minimum sizes and pricing still apply. The offline example verifies placement and bytes only; it reports no cache-hit or money-saving measurement.

servedAt rebuilds DATA from the call's recorded layout and byte limit, even when a checkpoint is resumed by an agent with a different limit. Records without a layout discriminator use the legacy layout; unknown or masked layout evidence produces a findings-layout gap. Redaction can therefore withhold reconstruction rather than silently rebuilding private model data.