Fetcher Llm Streaming

---
name: fetcher-llm-streaming
description: >
  Consume Server-Sent Events and LLM token streams with `@ahoo-wang/fetcher-eventstream`: the `Response.prototype` helpers (`requiredJsonEventStream`), standalone converters, `TerminateDetector` for `[DONE]`-style sentinels, `jsonEventStreamResultExtractor` and `for await` iteration. Use for your own SSE endpoint, a non-OpenAI token stream or an SSE decorator endpoint. Do not load for GPT or OpenAI-compatible chat completions, streamed or not: fetcher-openai-client handles those.
---

# fetcher-llm-streaming

## Decisions

- **Prototype helpers vs converters**: `import '@ahoo-wang/fetcher-eventstream'` patches `Response.prototype` (`contentType`, `isEventStream`, `eventStream()`, `requiredEventStream()`, `jsonEventStream()`, `requiredJsonEventStream()`, skipping members that already exist) and polyfills `ReadableStream` async iteration. `toServerSentEventStream(response)` / `toJsonServerSentEventStream(stream, detector?)` avoid _calling_ the patched members, but importing anything from the package still runs the patch — there is no side-effect-free entry. A library that must not touch `Response.prototype` cannot depend on this package.
- **Nullable vs required**: `eventStream()` / `jsonEventStream()` return `null` for a non-SSE Content-Type; the `required*` variants throw `EventStreamConvertError` (with `.response`). A body that is already used or locked also throws `EventStreamConvertError` — a response converts once.
- **Decorator/`RequestOptions` endpoints**: `jsonEventStreamResultExtractor<T>(detector)` builds the extractor; set it per endpoint (`@post(path, { resultExtractor })`).
- **OpenAI chat completions** (`/chat/completions`, `[DONE]`, typed chunks) are done by `$fetcher-openai-client`; don't rebuild them here.

## Gotchas a capable model gets wrong

- There is **no default terminator**. Without a `TerminateDetector`, `data: [DONE]` reaches `JSON.parse` and iteration throws `SyntaxError`. This package exports no `[DONE]` detector — write `e => e.data === '[DONE]'` (`DoneDetector` lives in `@ahoo-wang/fetcher-openai`).
- The constant `JsonEventStreamResultExtractor` has **no** detector and takes no options; use the factory `jsonEventStreamResultExtractor(detector)` instead. Neither is on `ResultExtractors` from `@ahoo-wang/fetcher`.
- Items are `JsonServerSentEvent<T>`: the payload is `event.data` (not `event.token`); `id` is `''` when the server sent none; `event` defaults to `'message'`. The detector sees the raw string event before parsing, and the matching event is not yielded.
- With a detector, a stream that ends without the terminating event errors with `EventStreamIncompleteError` — a cut-off answer is not silently "complete". Mid-stream errors (`SyntaxError`, `EventStreamIncompleteError`, network errors) come from the `for await`, not from the call that returned the `Response`. `break` cancels the connection.

## Canonical patterns

```ts
import '@ahoo-wang/fetcher-eventstream';
import { fetcher } from '@ahoo-wang/fetcher';
import type { TerminateDetector } from '@ahoo-wang/fetcher-eventstream';

const untilDone: TerminateDetector = e => e.data === '[DONE]';

export async function generate(prompt: string): Promise<string> {
  const response = await fetcher.post('/generate', { body: { prompt } });
  let text = '';
  for await (const event of response.requiredJsonEventStream<{
    token: string;
  }>(untilDone)) {
    text += event.data.token;
  }
  return text;
}
```

Declared endpoint (see `$fetcher-decorator-service`):

```ts
import {
  api,
  autoGeneratedError,
  body,
  post,
} from '@ahoo-wang/fetcher-decorator';
import {
  jsonEventStreamResultExtractor,
  type JsonServerSentEventStream,
} from '@ahoo-wang/fetcher-eventstream';

interface Chunk {
  token: string;
}

@api('/llm')
export class LlmApi {
  @post('/generate', {
    resultExtractor: jsonEventStreamResultExtractor<Chunk>(
      e => e.data === '[DONE]',
    ),
  })
  generate(
    @body() req: { prompt: string },
  ): Promise<JsonServerSentEventStream<Chunk>> {
    throw autoGeneratedError(req);
  }
}
```

## References

- `references/api.md`: every export with its signature, SSE parsing rules, error semantics, the transform pipeline and `SafeTransformer`, CommonJS. Load it for raw `ServerSentEvent` streams, custom transformers or exact error behaviour.

## Related Skills

- $fetcher-openai-client: OpenAI-compatible chat completions with `[DONE]` and typing handled.
- $fetcher-decorator-service: declaring streaming endpoints with decorators.
- $fetcher-integration: the Fetcher that produces the `Response`.