Streaming

A user types a question into your chat UI. They expect the answer to appear word-by-word like ChatGPT — not in one big block five seconds later. Behind the scenes the agent is also using that same response to decide whether to call a tool. One LLM call, two consumers: the UI streams tokens; the agent loop reads the final LLMResponse to drive ReAct decisioning.

How streaming works

Every LLM provider implements two methods:

The agent calls stream() when streaming is active. Each chunk has tokenIndex, content, and done. The final chunk has done: true AND response: LLMResponse — the full authoritative shape with toolCalls, usage, stopReason. This single round-trip serves BOTH the UI's appetite for token-by-token feedback AND the agent loop's need for the structured response.

Listen to streamed events

Every runner exposes typed events. Subscribe to agentfootprint.stream.* for the streaming surface:

agent.on('agentfootprint.stream.tool_start', (e) =>  console.log(`→ tool ${e.payload.toolName}(${JSON.stringify(e.payload.args)})`),);agent.on('agentfootprint.stream.tool_end', (e) =>  console.log(`← tool result: ${e.payload.result}`),);

The event taxonomy:

• stream.llm_start — provider call about to start (model, request shape)
• stream.token — one token chunk arrived (content, tokenIndex)
• stream.llm_end — one model call ended (content, usage, stopReason); output-governed drafts have empty content and contentWithheld: true
• stream.tool_start — agent about to dispatch a tool (name, args)
• stream.tool_progress — a long-running tool reporting mid-call (ctx.progress(payload); framework-stamped toolCallId / toolName / iteration, author's payload verbatim)
  • Since 9.54.0 this also moves the live status line: a top-level string message is shown verbatim (cut at 120 characters, cut stated), anything else renders an honest generic line — never a payload dump. See Tools → What a person sees.
• stream.tool_end — tool returned (result or error)

Every event arrives as an envelope { type, payload, meta }. e.payload is typed by the event name you subscribed to; e.meta says where it came from (runId, runtimeStageId, turnIndex, iterIndex, wallClockMs, …). .on() returns an unsubscribe function, and 'agentfootprint.stream.*' / '*' subscribe to a whole domain or to everything — see Observability for the full taxonomy.

Token events follow the provider

Without a delivery policy, token events follow the provider: a stream() provider emits text chunks, while a complete()-only provider emits the other lifecycle events without tokens. The same subscriber code works with either kind.

An output middleware policy changes delivery, not the provider call. It holds all draft tokens and withholds llm_end.content with contentWithheld: true, while retaining usage and timing. Final capture releases one accepted answer token, even for complete-only providers; an empty accepted answer has no token. A refusal releases no answer and the API boundary emits error.fatal. Declared input-only .act({ input: [...] }) rules retain normal streaming. Generic .messageMiddleware(...) may inspect both phases and therefore holds drafts too. Answer validation can also hold tokens; policy acceptance is not a validation result.

The order events arrive in

One agent.run() is one turn. A turn holds one or more iterations; each iteration is one LLM call followed by the tools it asked for. This is the ordinary, ungoverned streaming order:

agent.turn_start
  agent.iteration_start
    stream.llm_start → stream.token* → stream.llm_end
    stream.tool_start → stream.tool_end      (once per requested tool)
  agent.iteration_end
  agent.iteration_start
    stream.llm_start → stream.token* → stream.llm_end   (no tool calls: final answer)
  agent.iteration_end
agent.turn_end

A real two-iteration run that calls one tool, then answers:

agent.turn_start       { turnIndex: 0, userPrompt: 'Check order ORD-1003' }
agent.iteration_start  { turnIndex: 0, iterIndex: 1 }
stream.llm_start       { iteration: 1, provider, model, messagesCount, toolsCount }
stream.token           { iteration: 1, tokenIndex: 0, content: "I'll " }
stream.token           { iteration: 1, tokenIndex: 1, content: 'look it up' }
stream.llm_end         { iteration: 1, content: "I'll look it up", toolCallCount: 1, usage, stopReason, durationMs }
stream.tool_start      { toolName: 'lookup_order', toolCallId: 'tc-1', args: { orderId: 'ORD-1003' } }
stream.tool_end        { toolCallId: 'tc-1', result: '…', durationMs: 42 }
agent.iteration_end    { turnIndex: 0, iterIndex: 1, toolCallCount: 1 }
agent.iteration_start  { turnIndex: 0, iterIndex: 2 }
stream.llm_start       { iteration: 2, … }
stream.token …
stream.llm_end         { iteration: 2, toolCallCount: 0, content: 'Your order was denied…' }
agent.iteration_end    { turnIndex: 0, iterIndex: 2, toolCallCount: 0 }
agent.turn_end         { turnIndex: 0, finalContent: 'Your order was denied…', iterationCount: 2, totalInputTokens, totalOutputTokens, durationMs }

(All names carry the agentfootprint. prefix; it is dropped above for width.) Iterations count from 1, turns from 0. agentfootprint.error.fatal fires instead of a clean ending when a run fails.

With output middleware, there are no provider-draft token events in this sequence. The accepted final token follows the output decision and the last llm_end, before turn_end. A model call ending is not the turn ending: more tools, retries or policy checks may follow. Never close a turn's stream at llm_end.

Parallel tool calls

When the model asks for several tools in one reply, each stream.tool_start carries parallelCount — how many calls that reply requested. It is absent when there was only one. The tools run one after another in the order the model listed them, so every tool_start is followed by its own tool_end before the next begins. Key your UI rows by toolCallId (both events carry it; only tool_start carries the name):

const rows = new Map<string, { name: string; startedAt: number; endedAt?: number }>();

agent.on('agentfootprint.stream.tool_start', (e) => {
  rows.set(e.payload.toolCallId, { name: e.payload.toolName, startedAt: Date.now() });
});
agent.on('agentfootprint.stream.tool_end', (e) => {
  const row = rows.get(e.payload.toolCallId);
  if (row) row.endedAt = Date.now();
});

Pause events

When a tool asks for a human (pauseHere(...) / askHuman(...)), the stream stops mid-turn: that tool's tool_start is followed by agentfootprint.pause.request — no tool_end, no turn_end — and agent.run() returns a paused outcome ({ paused: true, checkpoint, pauseData }) instead of a string. agent.resume(...) fires agentfootprint.pause.resume, then the paused tool's tool_end and the rest of the turn stream as usual.

import { isPaused } from 'agentfootprint';

agent.on('agentfootprint.pause.request', (e) => {
  showApprovalPrompt(e.payload.questionPayload); // what the tool asked, plus toolName / toolCallId
});

const outcome = await agent.run({ message: 'Refund this order' });
if (isPaused(outcome)) {
  // store outcome.checkpoint (it is JSON), then later — even in another process:
  const answer = await agent.resume(outcome.checkpoint, { approved: true });
}

For a chat UI this means: end the SSE stream on pause.request too, not only on turn_end. Runnable end to end: examples/features/01-pause-resume.ts.

A throwing listener never breaks the run

Listeners are isolated. If an .on() handler throws, the error is caught, the other listeners for that event still fire, and the agent carries on — a bug in your UI code cannot fail the run. The flip side: nobody tells you. With footprintjs dev mode on (enableDevMode() from footprintjs) the error is logged to the console; otherwise it is swallowed silently, so wrap anything important in your own try/catch.

Listeners are also never awaited. An async listener runs in the background, and the run does not wait for it — collect the promises and await them after run() if you need them finished.

SSE for browser clients

The built-in toSSE() helper from agentfootprint/observe yields SSE-formatted strings (event: <name>\ndata: <json>\n\n). Start consuming it before starting the run, so the subscriber is already listening. For an agent whose run ends with turn_end or a run failure event:

import { toSSE } from 'agentfootprint/observe';

const frames: string[] = [];
const draining = (async () => {
  for await (const frame of toSSE(agent)) frames.push(frame);
})();
// An HTTP adapter can write each frame instead of collecting it.
await Promise.all([agent.run({ message: userInput }), draining]);

toSSE(runner, options?) accepts filter (skip events), format: 'full' | 'text' ('text' yields raw token content only — pipe straight into a chat UI), eventName (rename events), and heartbeatMs (keep-alive : ping comments for proxies). The class form new SSEFormatter(agent).stream() is identical. encodeSSE(name, payload) formats a one-off frame for app-level events outside the typed registry.

Existing helper limits: its filter runs before terminal detection, so a filter that excludes turn_end or error.fatal can leave it waiting forever. Use format: 'text' without a filter for text-only delivery. The helper currently stops on any error.fatal, including observer errors, and does not close on a pause or an input refusal that raises without a terminal event. It is not a universal run-lifecycle adapter. For those flows, use a hosting transport or own the subscriptions and close from the run's returned outcome or rejection too.

A request-owned subscription can handle those outcomes explicitly. Unsubscribe when it ends; do not leave one request listening to a later run on the same agent:

import { isPaused } from 'agentfootprint';

// Inside one HTTP request, after setting its SSE headers.
let ended = false;
function finish(event: 'done' | 'failed' | 'paused') {
  if (ended) return;
  ended = true;
  res.write(`event: ${event}\ndata: {}\n\n`);
  res.end();
}
const unsubscribe = [
  agent.on('agentfootprint.stream.token', (e) => {
    if (!ended) res.write(`data: ${JSON.stringify({ type: 'token', content: e.payload.content })}\n\n`);
  }),
  agent.on('agentfootprint.agent.turn_end', () => finish('done')),
  agent.on('agentfootprint.error.fatal', (e) => {
    if (e.payload.scope === 'run') finish('failed');
  }),
  agent.on('agentfootprint.pause.request', () => finish('paused')),
];

try {
  const outcome = await agent.run({ message: userInput });
  finish(isPaused(outcome) ? 'paused' : 'done');
} catch {
  finish('failed'); // covers API-boundary refusals without a terminal event
} finally {
  for (const off of unsubscribe) off();
}

The browser consumes via EventSource and renders tokens into the UI as they arrive.

When streaming isn't useful

• Cost-sensitive batch jobs — complete() is one less per-call cost than stream() on most providers (negligibly).
• Tool-heavy ReAct loops — tool dispatch happens at stream.llm_end, not during token chunks. If your UI doesn't render tokens, complete() is simpler.
• Tests / mocks — mock({ reply: 'X' }) works in both modes; pick whichever the test asserts on.

Anti-patterns

• Don't try to dispatch tools off stream.token. Tool calls land in the final chunk's response.toolCalls. Token chunks are content-only.
• Don't close SSE at stream.llm_end: it ends one model call, not the turn. Close on agent.turn_end, a run-scoped error.fatal, or a pause; handle the run's rejected promise too.
• Don't run two stream()s concurrently on the same agent. Build one, run one.

Next steps

• Observability guide — the full 59-event taxonomy + recorders
• Quick start — mock({ reply }) streams word-by-word for $0 dev