Observability
Typed event streams, recorders, and tier-3 enable.* helpers — observe what the agent did without shaping what it does. Typed events emitted during DFS traversal, no instrumentation.
Replaying a saved event stream
Live agents use runStepRecorder() through .watch(recorder), collecting
steps while the run executes. For a saved DomainEvent[], use the same
recorder's batch-ingestion method:
import { runStepRecorder } from 'agentfootprint/observe';
const recorder = runStepRecorder();
recorder.ingestDomainEvents(savedEvents);
const steps = recorder.getSteps();Pass the complete saved batch so sibling fork events are grouped together.
Use a fresh recorder (or clear()) for a different recording; use
getSteps(drillPath) to project a nested path. Version 10 removes
buildRunSteps, BuildRunStepsOptions, and RunStepGraph; this preserves
offline replay without a second interpretation of the event stream.
Logs that say "agent ran" are useless. Logs that say "agent called
redact_pii({text: '...'})and then injected the result into the messages slot before iteration 3, costing $0.0042" let you debug a hallucination at 2 AM. The difference is whether observability is structured + typed at the source or string-formatted at the sink.
Shipping it somewhere?
This page is the event taxonomy + recorders. To export the trace to a backend — AWS AgentCore Observability (CloudWatch GenAI) or OpenTelemetry — see Exporters: AgentCore & OTEL.
Three workflows from one trace
When an Anthropic adapter omits eligible completed-turn thinking,
agentfootprint.agent.thinking_omitted records each block before transport:
provider, model, message/block indices, reason and phase: 'prepared'. It
contains no thinking text or signature. Preparation does not prove the vendor
received or accepted the request. Live history and error checkpoints retain
the omission even when transport subsequently fails. Unsafe unfinished-turn
replay is refused instead; see Anthropic.
agentfootprint's observability isn't bolted on. The framework owns the loop, so every decision and execution is recorded during DFS traversal — not collected after the fact via spans. That single artifact serves three workflows:
| Mode | What you do | What the trace gives you |
|---|---|---|
| Live | Debug as you build | Exactly which injection produced which token; which predicate fired this iteration; which prefix actually got cached |
| Offline | Monitor what shipped | Replay any past run from its trace. Alert on drift. Attribute cost per injection. |
| Detailed | Improve via export | Every successful trajectory is labeled training data for SFT, DPO, or process-RL — no separate data-collection phase |
And a fourth, novel: the agent can read its own trace. Six months after the agent rejected loan #42, "why did you reject it?" answers from the recorded evidence (creditScore=580, threshold=600), not a rerun. Causal memory turns the trace into the agent's working memory.
The model
Every runner (Agent, LLMCall, Sequence, etc.) exposes the same observability surface, in three layers:
| Layer | API | When to use |
|---|---|---|
| Events | runner.on('agentfootprint.<domain>.<event>', fn) | You want a typed stream — payloads are auto-completed by your IDE |
| Recorders | runner.attach(recorder) | You want pre-built aggregation (cost totals, token counts, narrative entries) |
| Tier-3 helpers | runner.enable.liveStatus({ strategy }), runner.enable.observability({ strategy }) | You want a one-liner for the common case (status line, structured logs) |
There are 124 typed events across 27 domains: agent (incl. run_configured, the run-configuration manifest described below; agentfootprint.agent.budget_exhausted, which says a turn's budget ran out AND what the run did about it — wrapped up with one last tool-free call, or cut short; output_schema_validation_failed for Instructor-style retry and output_schema_retry for the loop-level re-ask), composition, context, stream, tools (incl. absent and coverage_declared — a tool that looked and found nothing, and a tool that named what its verdict does not rule out — and semantics_declared, the full semantic envelope a tool returned via describedResult() while the model read only the compact projection), skill, memory, cost, permission (incl. halt for sequence governance), credential (declare-and-push resolution), artifacts (the claim-check lifecycle: minted / resolved / expired / refused / presented, plus hand_over_failed — a host's own filing for a turn that did not go cleanly), eval, embedding, pause, checkin (evidence-carrying human consent), middleware (a governance chain answered), findings (the model's own standings on its tool results under .findings() — agentfootprint.findings.declared when a call carries a basis, agentfootprint.findings.standing when a previous result is called a fact, open, ruled-out or noise, and under .findings({ judge }) agentfootprint.findings.judged when a configured classifier judged a landed result — its standing, confidence, latency and tokens — or agentfootprint.findings.judge_failed when that call produced no answer, with agrees on the model's own findings.standing once a judgment exists for the result, and under .findings() beside .namesAndNumbersFromEvidence() agentfootprint.findings.contingent when the model used a value — in its answer or as a later call's argument — that came only from results it had itself declared open, noise or ruled-out: the moment, the carrier count, their distinct standings and the value's length; and, for a tool that declares askOrAssume, agentfootprint.findings.argument for each ruled argument of each call — whether the library filled the tool's declared default or the model sent the value, with the conversation turn; identities, enums and numbers only, never the model's words, the judged state or the value), tool_choice (a classifier's pick of WHICH TOOL beside every model call under .toolChoice() — agentfootprint.tool_choice.picked before the call with the pick, its confidence, how many tools were offered and served, whether the list was narrowed and the cost; agentfootprint.tool_choice.outcome after the reply with the tools the model called, firstAgrees and any missed; agentfootprint.tool_choice.failed when the classifier produced no answer; tool names are identities, never a description or the message), ontology (the declared map served on a model call under .ontology() — agentfootprint.ontology.served with the map's id, version and hash and its node, source and edge counts; identities and numbers only, never a meaning, a coverage sentence or a node name), answer (the answer's standing under .answerLayer() — agentfootprint.answer.assessed, once per answer, with the value, its word, the reason kinds and the checks that ran; never a value from the answer or a quote), error, reliability, resilience (the outputSchema fallback ladder — output_fallback_triggered and output_canned_used), fallback. Every event has a typed payload — agentfootprint.stream.llm_start, agentfootprint.stream.tool_start, agentfootprint.cost.tick, agentfootprint.composition.iteration_start, etc. The framework owns the loop and emits during DFS traversal — no instrumentation, no spans-after-the-fact.
Trust boundary observations
trustBoundaryRecorder captures selected typed events during execution,
never by reconstructing a finished tree. It observes; it does not enforce policy.
Enable the optional observer with
recordRun(agent, { trustBoundaries: { maxFacts: 100 } }), before running.
Example 93
demonstrates a real permission refusal with a mock model and independently
asserts that the synthetic tool did not execute.
RecordRunOptions.trustBoundaries accepts true or
TrustBoundaryRecorderOptions; default off. RunRecorder.trustBoundaries
exposes the TrustBoundaryRecorder handle. Standalone collection uses
trustBoundaryRecorder from agentfootprint/observe: subscribe(agent)
returns an unsubscribe function. TrustBoundaryEventSource is that narrow
subscription port. One active subscription is permitted; this typed observer
does not belong in raw engine .attach() or .watch().
Facts, not approvals
TrustBoundaryEventType selects seven agentfootprint. events:
middleware.decision, permission.check, permission.halt,
credential.requested, credential.acquired,
credential.authorization_required, and credential.failed.
Each TrustBoundaryFact carries sequence, event type, run/stage ids and source
time, plus supplied call identity/iteration/position. Its payload retains only
named decision, capability, service and error-class metadata.
The field whitelist
defines each variant.
An allow does not prove execution: middleware pass-through can allow without
invoking the user's rule. changed means transformation, not redaction;
after-tool refusal does not undo execution. Pull credential success and
permission-gate lifecycle are not covered. Sequence/Parallel .act() children
currently lack these typed security bridges, even when subscribing to the
mounted child. Missing evidence never means “allowed,” “safe” or “checked.”
Retention and lifecycle
TrustBoundarySnapshot is a TrustBoundaries recorder row with
meta.version: 1. recordRun adds it to snapshot.recorders without changing
the engine snapshot. Its data contains capture id, immutable facts,
TrustBoundaryCounters, and nullable first/last observed/retained sequences.
The accounting law is
observed = retained + evicted + invalid + oversized + pending.
Selected events reserve their sequence before inspection; rejected events
leave gaps. pending counts projections still in progress, normally zero
after dispatch finishes. Unrelated/unreadable types are ignored.
maxFacts defaults to 1,000 (integer 1–10,000). The tail retains newest facts.
Strings are limited to 512 UTF-16 units, mount paths to 32 segments and serialized
facts to 8,192 UTF-8 bytes. Oversized identities are rejected, not truncated;
malformed supplied positions reject the fact. Source positions follow
the committed-prefix contract below,
not the emitting stage's eventual state.
Capture spans subscribed run ids/resumes without implicit reset.
resetCapture() starts a new id, clearing facts and counts, including fencing
in-progress old observations. Await execution before export: stopping early
does not drain pending engine events or prove completion. With trust capture,
recordRun.stop() pins its base snapshot, retains facts and is idempotent.
The base keeps the existing by-reference contract; serialize an approved
projection to detach. A fact whose log is no longer retained stays unplaced.
Content-minimized is not PII-free
Prompts, arguments, results, rewrite values, free-text reasons, authorization URLs, session ids, principals and tenants are excluded. Declared identifiers and service/error labels can still contain sensitive text.
The surrounding event tail and state may contain raw content.
boundaryDetail: 'lean' reduces only BoundaryEvents, not the whole
recording. Review every exported channel. This observer promises neither
complete coverage nor legal compliance or tamper-proof evidence.
See Archive a run.
Which tool call did a security check describe
permission.check, permission.halt and the emitted credential lifecycle
events carry toolCallId and iteration from the call being dispatched.
The ToolCallEventIdentity type from agentfootprint/events names this pair;
the new fields are optional on public event payloads for older or caller-emitted events.
Use both alongside meta.runId to distinguish repeated calls to the same tool
or credential service. A resumed credential request keeps the paused call's
id; a nested ctx.tools failure carries that inner call's existing id.
const facts: { event: string; runId: string; toolCallId: string; iteration: number }[] = [];agent.on('*', (event) => { if ( event.type !== 'agentfootprint.permission.check' && event.type !== 'agentfootprint.credential.requested' && event.type !== 'agentfootprint.credential.acquired' ) return; const { toolCallId, iteration } = event.payload; // Old recordings and caller-emitted facts may have no call identity. // Do not guess it from a tool name, service or arrival order. if (toolCallId === undefined || iteration === undefined) return; facts.push({ event: event.type, runId: event.meta.runId, toolCallId, iteration });});await agent.run(input);assert.deepEqual( facts.map((fact) => fact.toolCallId), ['first-call', 'first-call', 'first-call', 'second-call', 'second-call', 'second-call'],);assert.ok(facts.every((fact) => fact.iteration === 1));assert.equal(new Set(facts.map((fact) => fact.runId)).size, 1);// A check's allow result is not proof that execution or later checks passed.Old recordings and caller-emitted events may lack these fields. Display that
as unknown attribution, not an allowed call. A permission allow or credential
acquired fact does not mean the tool executed: later checks may still refuse
it. These identifiers do not supply a time-travel commit position, and existing
reason text still needs its own retention policy.
Emission time and recorded positions
For events forwarded from a stage's emit channel, meta.wallClockMs is the
original emission timestamp, including under deferred delivery. runOffsetMs
uses that timestamp minus the agent run's start time. Other observer hooks
without a source timestamp still use metadata-construction time; these clocks
are not a total ordering of concurrent events.
When the engine supplies valid coordinates, meta.sourcePosition carries the
EventSourcePosition type exported by agentfootprint/events. Its
engineRunId names the emitting engine execution leg, not meta.runId.
logRunId identifies the owning log, drillPath contains actual runtime mount
ids, and committedThroughIdx is the local inclusive committed prefix at
emission. A value of -1 means only the fold base. Pending writes are excluded.
const origins: EventMeta[] = [];agent.on('agentfootprint.stream.llm_start', (event) => origins.push(event.meta));await agent.run(input);const rows = origins.map((origin) => ({ agentRunId: origin.runId, emittedAt: origin.wallClockMs, emitter: origin.runtimeStageId, position: origin.sourcePosition ?? 'unavailable',}));// sourcePosition.engineRunId belongs to the execution engine, not the agent.// A position is the already-committed prefix, NOT the emitter's final state.// No position? Do not substitute the current cursor or root commit count.This optional capability needs an engine build that supplies source positions. An older engine, manual or unaddressable emitter, or malformed carrier leaves the field absent. No dependency version or position is invented. The bridge copies only the named coordinate fields and does not inspect event payloads.
A viewer must match the saved log address and check local bounds before folding
state. Do not select the emitter's eventual commit, convert the static
subflowPath into a runtime mount path, or silently clamp a missing position.
These coordinates describe a committed prefix, not a policy approval, a complete
event ordering, or proof that nested history is available. This bridge change
does not add a Lens navigation implementation or sanitize ordinary payloads.
Which arm was this run? — agent.run_configured
Swapping a strategy is easy. Attributing an outcome to it is the hard part: the numbers (tokens, latency, iterations, routing, refusals, evidence verdicts) all carry meta.runId, but until 9.41.0 nothing said which STRATEGIES that runId was running. Grouping ten runs into two arms was your own bookkeeping, spelled differently in every study.
agentfootprint.agent.run_configured fires once, at run start, and names them:
const arms = new Map<string, string>(); // runId → arm label
const latency = new Map<string, number>();
agent.on('agentfootprint.agent.run_configured', (e) => {
const p = e.payload;
arms.set(e.meta.runId, `${p.llm.model} · ${p.memories[0]?.retrieval ?? 'no-retrieval'}`);
});
agent.on('agentfootprint.agent.turn_end', (e) => {
latency.set(e.meta.runId, e.payload.durationMs);
});
// …run the same workload under two configurations, then:
for (const [runId, ms] of latency) console.log(arms.get(runId), ms);It names the provider and model, the ReAct mode, every mounted memory (id, type, strategy kind, retrieval rule, embedder id), the window/compaction strategy, the skill graph's routing posture and classifier, the evidence gate's posture, and whether an artifact store is wired.
Two rules make it safe to keep and honest to read:
- Names and ids only. No credentials, endpoints, connection strings, keys, tenants or principals — and no config value that could be one. A manifest travels into every recording and every vendor sink, which is exactly where a secret must not go. Where the only handle on a component is a value (a store's directory, a bucket, a table), the manifest reports that the component is PRESENT and leaves it unnamed.
- Absence is a fact. A field you did not configure is missing from the payload — never
'default'or'unknown'filled in on your behalf.memories: []is the explicit "no memory is mounted", because "no memory" is itself an arm.
llm.model is the model the run STARTS with. If a .configure() resolver or per-skill brains can replace it mid-run, llm.modelOverrides names them — and agentfootprint.stream.llm_start still reports what each individual call really used.
The event is skipped entirely when nothing is subscribed, and it carries only names, so its size is bounded by your configuration rather than by the turn.
What the recording carries for a routing debugger (9.50.0)
Three facts the framework always knew at run time now land in the recording as DATA, so a viewer never parses them back out of prose:
agentfootprint.skill.graph_declared— the author's skill map, once per run, right afteragent.run_configured: every declared node (id,kind, catalogdescriptionverbatim) and edge (from—nullfor the synthetic START —to,kind, caption), straight from the built graph. Per-hoprouting[]provenance names an edge only once it fires; this event carries the COMPLETE declared topology, so a graph drawn from a recording no longer has to call itself partial. No graph mounted (or a structurally-typed graph that carries nonodes) ⇒ no event.context.evaluated.cursorMove.reachable— where the run could go next, typed on every cursor move: the declared hops out of the landed cursor plus the open skills, composed from the same resolvers that build theread_skillmenu and the refusal messages (skill.rejected.allowedis the same set, on the refusing iteration).[]means a dead end; an absent key means the mounted graph predatesreachableSkills.stream.llm_start.systemPromptText— the ASSEMBLED system prompt, byte-for-byte as sent, opt-in viarecordSystemPrompt: trueonAgent.create/LLMCall.create. The default is OFF and the default is a privacy decision: the assembled prompt carries whatever was injected into it (skill bodies, RAG passages, memory, per-user instructions) and rides into every sink and every persisted recording, so absent the dial onlysystemPromptChars— the length — is on the record.
The runnable tour is examples/features/63-recording-carries-the-map.ts: one mock skill-routed run archived with persistRecording, all three facts read back off the JSON file, and the privacy default proven on the archived bytes of a second, un-opted-in run.
Tier-3 — .enable.liveStatus() and .enable.observability()
The 80% case: a live status line ("what is the agent doing right now?") and a domain-filtered log firehose. Both are one-liners over a strategy — pick a built-in (chatBubbleLiveStatus, consoleObservability from agentfootprint/observe) or a vendor one.
// Live status line — user-facing "what's the agent doing right now".// The chat-bubble strategy maps the thinking-state machine to one line.const stopThinking = agent.enable.liveStatus({ strategy: chatBubbleLiveStatus({ onLine: (line) => console.log(` ⎈ ${line}`) }),});.enable.liveStatus({ strategy }) with chatBubbleLiveStatus({ onLine }) gives you a Claude-Code-style live status line — one string per agent state transition. Use for terminal UIs, CLI progress, or chat-UI typing indicators.
// Firehose observability via the console strategy. Swap in any vendor// strategy (Datadog, OTel, AgentCore, CloudWatch) — same call site.const stopLogging = agent.enable.observability({ strategy: consoleObservability({ logger: { log: (...args) => console.log(' [log]', ...args) } }),});.enable.observability({ strategy }) with consoleObservability() gives you structured logs of every typed event. Swap the strategy for pino, winston, or a vendor backend — same call site.
Both return a stop() function — call it on shutdown to detach the listener cleanly.
Migration (4.0.0): the old flat
.enable.thinking()/.enable.logging()one-liners were removed in favor of these uniform strategy enablers. Replaceenable.thinking({ onStatus })withenable.liveStatus({ strategy: chatBubbleLiveStatus({ onLine: onStatus }) }), andenable.logging()withenable.observability({ strategy: consoleObservability() }).
Tier-3 — .enable.observability() — grouped vendor strategies
For shipping events to a vendor backend (CloudWatch, X-Ray, OTel, AgentCore), .enable.observability(...) accepts a typed strategy + optional detach driver. The strategy is the WHERE; the detach is the HOW (sync inline vs fire-and-forget):
import { agentcoreObservability } from 'agentfootprint/observe';
import { microtaskBatchDriver } from 'footprintjs/detach';
const telemetry = agent.enable.observability({
strategy: agentcoreObservability({
region: 'us-east-1',
logGroupName: '/agentfootprint/my-agent',
}),
// Recommended for production — slow exporters never block the agent loop.
// The strategy's `exportEvent(event)` runs on the driver's schedule
// instead of inline on the dispatcher's hot path.
detach: { driver: microtaskBatchDriver, mode: 'forget' },
});What comes back — ObservabilityHandle
telemetry is an ObservabilityHandle (StrategyHandle is the same type
under its group-neutral name — enable.cost() and enable.liveStatus() return
it too). It is still the unsubscribe function it has always been, so every call
site written before 8.12.0 keeps working:
| call | what it does |
|---|---|
telemetry() | detach the subscription. Never stops your strategy — that is what lets one instance be enabled, released and enabled again. |
await telemetry.flush() | drain: the events still queued on the detach driver first, then the strategy's own buffer. That order is the whole point — the reverse ships nothing. |
telemetry.stop() | release the strategy — but only once nothing else is still subscribed to it, and at most once ever. |
await agent.shutdown() | do all of the above for everything enabled on the agent. The agent stays usable afterwards. |
Where your runtime has Symbol.asyncDispose, the handle is an AsyncDisposable,
so await using telemetry = agent.enable.observability({ strategy }) flushes and
releases at the end of the scope.
agent.run() never flushes — telemetry must not become a term in run latency.
For scripts and short-lived processes, opt into a per-run drain with
flushOn: 'run-end', which FIRES a flush when a run ends without gating run()
on it.
Detach modes
detach.mode | Returns | Use when |
|---|---|---|
omitted (no detach) | sync | Strategy is fast (< 100µs). Default, back-compat. |
'forget' (default when detach set) | void | Pure fire-and-forget telemetry. |
'join-later' | onHandle(h) callback fires per event | You want to await exports later (tests, backpressure). |
Drivers
Pick by environment — see footprintjs/detach for the full list. All exported from 'footprintjs/detach':
| Driver | Best for |
|---|---|
microtaskBatchDriver | Default. Cross-runtime. Lowest latency. |
setImmediateDriver | Node only. Yields to I/O before flushing. |
setTimeoutDriver | Cross-runtime, configurable delay. |
createSendBeaconDriver({ url }) | Browser. Survives page-unload (analytics). Factory — requires a target URL. |
createWorkerThreadDriver({ ... }) | CPU-isolated. For heavy serialization. Factory — requires worker config. |
microtaskBatchDriver, setImmediateDriver, and setTimeoutDriver ship as ready-to-use singletons. createSendBeaconDriver and createWorkerThreadDriver are factories (no default singleton) because they need required options — call the factory and pass the result as driver.
Graceful shutdown
Since 8.12.0 the handle enable.observability() returns drains itself — and it is the only
thing that can, because a detached event is sitting on the driver, not in your strategy's buffer:
const telemetry = agent.enable.observability({
strategy,
detach: { driver: microtaskBatchDriver, mode: 'forget' },
});
await telemetry.flush(); // driver queue first, then the strategy's buffer
await agent.shutdown(); // …or drain and release everything in one callhandle.flush() waits for the events THIS subscription scheduled. flushAllDetached() (from
'footprintjs/detach') remains the process-wide hammer for detached work agentfootprint did not
schedule.
Before 8.11.1, this recipe lied
flushAllDetached() used to return { done: 0, failed: 0, pending: 0 } — a clean bill of health —
while events were still in flight, because scheduling was deferred by one microtask and the
registry was empty when it looked. Upgrade; then prefer handle.flush().
Vendor adapters — agentfootprint/observe
Vendor strategies ship under one grouped subpath (parallel-providers pattern, mirrors llm-providers / tool-providers / memory-providers):
| Adapter | Status | Peer dep (optional) |
|---|---|---|
agentcoreObservability | ✅ shipped | @aws-sdk/client-cloudwatch-logs |
cloudwatchObservability | ✅ shipped | @aws-sdk/client-cloudwatch-logs |
xrayObservability | ✅ shipped | @aws-sdk/client-xray |
otelObservability | ✅ shipped | @opentelemetry/api + @opentelemetry/sdk-node |
datadogObservability | roadmap | dd-trace (use otelObservability with Datadog OTel collector for now) |
All peer deps are declared optional — consumers who never call a particular factory don't need to install its SDK. Each adapter lazy-imports its SDK at first use.
enable.cost(...) accepts the same detach option for the same reasons. enable.liveStatus and enable.flowchart deliberately stay sync — UI render must feel responsive.
Cost tracking
Pass a pricingTable to the Agent constructor and the framework emits agentfootprint.cost.tick after every LLM call with per-call and cumulative USD. Add a costBudget and you also get a one-shot cost.limit_hit the first time cumulative crosses the budget.
Each tick also says what was billed: model and provider name the call that spent the money, so a run that switched models mid-loop — a 'retry-other' failover, a compactor on a cheaper vendor — breaks down from the cost channel alone. The compactor's spend is filed under its summarizer, not the agent's provider. (provider is absent only when a third-party window strategy reports spend without declaring its billing.)
A bare number warns: the event fires with action: 'warn' and the run carries on, and you decide what to do (cancel via signal, escalate, log). Since 8.14.0 you can also ask it to stop:
Agent.create({ provider, model, pricingTable, costBudget: 0.50 }) // warns
Agent.create({ provider, model, pricingTable,
costBudget: { usd: 0.50, onExceed: 'halt' } }) // stops'halt' ends the loop at the next iteration boundary — the same boundary maxIterations uses. Never mid-call: the call that crossed the budget completes, is billed and is recorded; halting only decides there will not be another one. The event then reports action: 'abort', and agent.stoppedEarly() says what happened:
const answer = await agent.run({ message });
const cut = agent.stoppedEarly();
if (cut) console.log(`stopped at iteration ${cut.iteration}: ${cut.reason}`);// The same budget, made a STOP. Halting ends the loop at the next iteration// boundary — the same boundary maxIterations uses — so the call that crossed// the budget still completes, is still billed and is still recorded. What// halting decides is that there will not be another one.const halting = Agent.create({ provider: provider ?? exampleProvider('feature'), model: 'demo-sonnet', pricingTable: pricing, costBudget: { usd: 0.00001, onExceed: 'halt' }, // crossed on the FIRST call}) .system('') .tool({ schema: { name: 'noop', description: '', inputSchema: { type: 'object' } }, execute: () => 'ok', }) .build();const halted = await halting.run({ message: 'do the thing' });const cut = halting.stoppedEarly();if (cut) { // `AgentOutput` is a bare string, so the reason cannot ride the answer. It // rides committed state instead, where it is provable after the run rather // than only observable by whoever happened to be subscribed. console.log( `\nhalted at iteration ${cut.iteration} (${cut.reason}); ` + `${cut.pendingToolCalls} tool call(s) never ran; ` + `answer was ${cut.answerWasEmpty ? 'EMPTY' : 'partial but real'}`, );} else { console.log('\nthis run finished before the budget was crossed:', halted);}stoppedEarly() also covers maxIterations. It is set only when a limit cut the turn short — the model was still asking for tools and the run refused to run them — so a turn that used its whole budget and then genuinely finished reports nothing. When the answer that comes back is the empty string the library warns once on the console too, because an empty string reaching a user is indistinguishable from a bug.
// 'feature' kind drives the smart tool-call flow. Cost ticks fire// automatically off the per-iteration usage MockProvider estimates// (chars/4) — sufficient to demo the budget crossing.const agent = Agent.create({ provider: provider ?? exampleProvider('feature'), model: 'demo-sonnet', pricingTable: pricing, costBudget: 0.0001, // trip the warning}) .system('') .tool({ schema: { name: 'noop', description: '', inputSchema: { type: 'object' } }, execute: () => 'ok', }) .build();agent.on('agentfootprint.cost.tick', (e) => { const p = e.payload; console.log( `[tick] +$${p.estimatedUsd.toFixed(6)} — cumulative $${p.cumulative.estimatedUsd.toFixed(6)}`, );});Event taxonomy
agentfootprint.<domain>.<event>| Domain | Sample events | What it covers |
|---|---|---|
agent | turn_start · turn_end · iteration_start · iteration_end · route_decided · handoff · output_schema_validation_failed · output_schema_retry · output_contract_unmet · thinking_parse_failed | ReAct loop boundaries; output_schema_validation_failed fires inside the reliability gate when the LLM's final answer fails the agent's outputSchema (v2.13+); output_contract_unmet (8.18.0) fires once on a run that hands back an answer the schema rejects — including the default retries: 0, where nothing used to say so |
composition | enter · exit · iteration_start · iteration_exit · fork_start · branch_complete · merge_end · route_decided | Sequence / Parallel / Conditional / Loop lifecycles |
context | injected · evaluated · evicted · slot_composed · budget_pressure | Context engineering — every injection's full lifecycle. budget_pressure fires when a slot composes over its budgetCap (planAction: 'none' = nothing was truncated, the full content still went to the LLM — the signal is that the budget is not being respected). Emitted since 7.6.1; before that the event was documented but never fired |
stream | llm_start · llm_end · token · tool_start · tool_progress · tool_end · thinking_delta · thinking_end | Token-by-token + tool-call + thinking lifecycle. tool_progress is the one event a TOOL AUTHOR fires — ctx.progress(payload) from inside a still-running execute (see Tools) — so a long call says "hop 3 of 12" instead of going silent — and since 9.54.0 the live status line consumes it too, so the report reaches a person and not only the record |
tools | offered · activated · deactivated · discovery_started · discovery_completed · discovery_failed · shadowed · claim_swallowed · answered_off_wire | ToolProvider activation per iteration (skills, gating); async-provider lifecycle (start/complete/fail with durationMs); shadowed fires when two parties' contracts competed for ONE tool name on the wire — since 9.92.0 dispatch follows the offer, so schemaFrom and dispatchTo both name the party that won it; claim_swallowed names every party whose claim to a name is dead that iteration ({ toolName, lostBy, wonBy, iteration }); answered_off_wire records a call the dispatcher answered although its name was not on that call's wire — the held-out dispatch the capability law keeps ({ toolName, toolCallId, iteration, answeredBy }). All three carry names only |
skill | activated · deactivated | LLM-activated skill resolution |
memory | attached · detached · written · strategy_applied | Memory pipeline subflows |
cost | tick · limit_hit | Per-call USD + budget threshold crossings |
permission | check · gate_opened · gate_closed · halt | PermissionChecker decisions; halt fires when a checker terminates the run via sequence governance (v2.12+) |
eval | score · threshold_crossed | Custom quality / guardrail signals |
embedding | generated | Embedder calls (RAG / semantic memory) |
pause | request · resume | Human-in-the-loop checkpoints |
error | fatal · recovered · retried | fatal once per failed top-level run; retried / recovered come from the withRetry provider decorator, reported in-run — see Resilience |
fallback | triggered | Provider failover — withFallback / fallbackProvider naming which provider actually served (kind: 'provider') |
reliability | fail_fast · retried · recovered | Reliability-gate primitives (rules-based loop, distinct from decorator-shaped error.*) |
Every event is exhaustively typed via AgentfootprintEvent (a discriminated union); your IDE autocompletes the payload shape based on the event name. No string-parsing logs. No "I think this means the tool was called."
Custom aggregations
For aggregations the built-ins don't cover, subscribe to the typed events directly. runner.on(type, fn) returns an unsubscribe — the payload is fully typed by the event name:
let injections = 0;
let totalOutputTokens = 0;
const stopA = runner.on('agentfootprint.context.injected', (e) => {
// e.payload — { slot, source, contentSummary, asRole?, ... }
// `asRole` is a `ContextRole` — 'system' | 'user' | 'assistant' | 'tool',
// the role the piece carries in the conversation. Present on messages-slot
// records (which report the conversation itself), absent on system-prompt ones.
injections++;
});
const stopB = runner.on('agentfootprint.agent.turn_end', (e) => {
// e.payload — { turnIndex, finalContent, iterationCount, totalInputTokens, totalOutputTokens, durationMs }
totalOutputTokens += e.payload.totalOutputTokens;
});
// Or subscribe to a whole domain with a wildcard:
const stopC = runner.on('agentfootprint.context.*', (e) => { /* every context.* event */ });
// On shutdown:
stopA(); stopB(); stopC();On context.injected, asRole is a ContextRole — 'system' | 'user' | 'assistant' | 'tool', the role the piece carries in the conversation. It is present on messages-slot records, which report the conversation itself, and absent on system-prompt records, which have no message role to carry. The same union types LLMMessage.role, so a consumer grouping injections by role and a consumer reading the request are speaking about the same four values.
runner.attach(recorder) is the lower-level path: it accepts a footprintjs CombinedRecorder that observes the raw footprintjs stream (onWrite, onSubflowEntry, onDecision, …) — not the grouped agentfootprint.* events. For agentfootprint's typed event taxonomy, prefer runner.on(...). A CombinedRecorder discovers its handler methods via runtime method-shape detection — implement only the hooks you care about; ignore the rest.
Consumer-owned events — runner.emit()
Some domains are partly yours to fill: eval (quality scores), memory, skill. Emit into them with runner.emit(name, payload) and every subscriber — your own listeners, enable.observability strategies, OTel — receives it like a library event. Use it for a score your evaluator computed after the run, a thumbs-up from the UI, or any app event you want on the same stream.
agent.on('agentfootprint.eval.score', (e) => {
console.log(`${e.payload.metricId} = ${e.payload.value}`);
});
const answer = await agent.run({ message: 'Summarize the Q3 report' });
agent.emit('agentfootprint.eval.score', {
metricId: 'response-quality',
value: 0.85,
target: 'run',
targetRef: 'this-run',
evaluator: 'heuristic',
});A name outside the registry (say 'myapp.feedback.thumbs_up') is delivered to '*' listeners only, as an untyped event.
Gotchas:
- Nobody listening, nothing sent.
emit()returns immediately when no listener matches — subscribe first. - The payload is not type-checked.
emit()takes any object; a registry name does not make the compiler check its shape. Match the documented payload yourself. - Keep
agentfootprint.*for the registered names. Invent your own prefix for app events.
Runnable: examples/features/05-events.ts.
Custom recorder
When a listener per event is not enough — you want one object that sees scope writes, stage boundaries and emitted events together — write a recorder: an object with an id plus whichever footprintjs hooks you need (onWrite, onCommit, onStageExecuted, onDecision, onEmit, onError, …).
import { Agent, type CombinedRecorder } from 'agentfootprint';
const log: string[] = [];
const audit: CombinedRecorder = {
id: 'audit',
onWrite: (e) => log.push(`write ${e.key}`), // a stage wrote a state key
onEmit: (e) => log.push(`emit ${e.name}`), // an event emitted from a stage
};
// At build time — sees the very first run:
const agent = Agent.create({ provider, model }).system('Be helpful.').watch(audit).build();
// …or on a live agent — returns an unsubscribe you own:
const detach = agent.attach(audit);
detach();Gotchas:
attach()takes effect on the NEXT run. A recorder attached while a run is in flight sees nothing of that run..watch()on the builder avoids the question.attach()is not idempotent. Each call adds another entry; attaching per request without calling the unsubscribe is a leak. Attach once, or detach every time.onEmitsees the events stages emit, not all of them. A few events are dispatched by the runner itself (agent.run_configured,pause.request,pause.resume) and reach onlyrunner.on(...). For "every event", subscriberunner.on('*', fn).- For one domain's worth of events, a closure over
runner.on('agentfootprint.<domain>.*', fn)is simpler than a recorder — see Custom aggregations.
Built-in bridges (you don't attach these)
Each run, the agent attaches internal recorders that turn what the stages emit into the typed stream you subscribe to. You never need them to observe an Agent — runner.on(...) is how you read their output. They are exported from agentfootprint/observe for one case: wiring the typed stream onto a footprintjs executor you drive yourself. Each factory takes { dispatcher, getRunContext, id? }.
| Factory | Forwards |
|---|---|
agentRecorder | agentfootprint.agent.* |
streamRecorder | agentfootprint.stream.* |
compositionRecorder | agentfootprint.composition.* |
ContextRecorder (class) | agentfootprint.context.injected · slot_composed · evicted · budget_pressure |
contextEvaluatedRecorder | agentfootprint.context.evaluated |
costRecorder | agentfootprint.cost.* |
toolsRecorder | agentfootprint.tools.* |
skillRecorder | agentfootprint.skill.* |
memoryRecorder | agentfootprint.memory.* |
embeddingRecorder | agentfootprint.embedding.* |
permissionRecorder | agentfootprint.permission.* |
evalRecorder | agentfootprint.eval.* |
resilienceRecorder | agentfootprint.error.retried · recovered · circuit_changed, agentfootprint.fallback.triggered |
OpenTelemetry
otelObservability ships in agentfootprint/observe. It maps agentfootprint events onto OTel spans following the GenAI semantic conventions (gen_ai.* attributes) via your own @opentelemetry/api tracer. Pair with @opentelemetry/sdk-node and your favorite exporter (Jaeger, Tempo, Datadog OTel collector, etc.):
import { otelObservability } from 'agentfootprint/observe';
import { microtaskBatchDriver } from 'footprintjs/detach';
import { trace } from '@opentelemetry/api';
const otel = otelObservability({
serviceName: 'my-agent',
tracer: trace.getTracer('my-agent', '1.0.0'),
// genAiSpanNames: true, // opt-in spec span names ('chat gpt-4', 'execute_tool search', …)
});
const stop = agent.enable.observability({
strategy: otel,
detach: { driver: microtaskBatchDriver, mode: 'forget' },
});Span tree — one trace per turn: invoke_agent root (agent name, turn-total gen_ai.usage.* tokens) → iteration:N → chat spans per LLM call (gen_ai.provider.name, gen_ai.request.model, usage incl. cache tokens, gen_ai.response.finish_reasons) and execute_tool spans per tool call (gen_ai.tool.name, gen_ai.tool.call.id, parallel-safe correlation).
Explainability span events (on by default; explainability: false to opt out) carry the decision trail a compliance reviewer needs: route decisions, skill-routing provenance (decision path + unlocked tools), tool-arg validation rejections, permission checks/halts, credential lifecycle. PII discipline mirrors the validation contract — tool args appear as key names, results as a type, prompts and LLM content never.
For operator-level decide()/select() evidence (creditScore gt 700 → 750 (true)), attach the strategy's FlowRecorder bridge too:
const agent = Agent.create({ provider, model })
.watch(otel.decisionEvidenceRecorder()) // decide()/select() evidence → span events
.build();Runnable demo: examples/features/18-otel-genai.ts.
End-to-end compliance lighthouse — spans + tamper-evident audit export + causal memory feeding an OFFLINE auditor that explains a loan decline weeks later from persisted JSON alone: examples/features/20-regulated-decisioning.ts.
Anti-patterns
- ❌ Don't log every event — dozens of typed events × N iterations × M agents = your log infra hates you. Filter to the domains you actually need.
- ❌ Don't subscribe in a hot loop —
runner.on(...)returns an unsubscribe; call it on shutdown. Forgotten subscriptions leak listeners across runs. - ❌ Don't use
getNarrative()as a log sink — it's structured trace data, not log lines. Pass the entries to a real log/trace backend if you need to ship them.
Next steps
- Localize a context bug (Beta) — when the trace shows what happened but not which input caused it
- Causal memory deep dive — the trace as the agent's working memory
- Reliability gate —
agentfootprint.reliability.fail_fastevent semantics - Streaming guide — token-by-token rendering via
provider.stream()
Embedding cost — embeddingRecorder
embeddingRecorder (with EmbeddingRecorderOptions) is the always-on bridge that forwards agentfootprint.embedding.generated to the typed stream. It is attached on every run, so you only construct one yourself when wiring a recorder outside Agent.
The event separates the two halves of embedding cost through inputKind: 'document' is index time (once, scaling with corpus size) and 'query' is retrieval time (per turn, scaling with traffic).
agent.on('agentfootprint.embedding.generated', (e) => {
console.log(e.payload.inputKind, e.payload.count, e.payload.durationMs);
});