dragon.ai.inference.llm_engine.StreamChunk

class StreamChunk[source]

Bases: object

A single chunk yielded during streaming generation.

Each chunk represents incremental output from the LLM, typically one or more tokens. The final chunk for a request has is_finished=True and includes complete metrics.

Parameters:

request_index (int) – Index of this request within the batch (0 for single-request streaming).
delta_text (str) – New text generated since the last chunk. Empty string for the final sentinel chunk.
accumulated_text (str) – Full text generated so far for this request.
is_finished (bool) – True when generation for this request is complete.
finish_reason (str | None) – Reason generation stopped (e.g., “stop”, “length”). Only populated when is_finished=True.
time_to_first_token (float | None) – Time in seconds from request start to first token. Populated on every chunk after the first token arrives.
metrics (dict | None) – Performance metrics. Only populated on the final chunk.

__init__(request_index: int, delta_text: str, accumulated_text: str, is_finished: bool, finish_reason: str | None = None, time_to_first_token: float | None = None, metrics: Dict[str, float] | None = None) → None

Methods

__init__(request_index, delta_text, ...[, ...])

Attributes

`finish_reason`
`metrics`
`time_to_first_token`
`request_index`
`delta_text`
`accumulated_text`
`is_finished`

request_index: int

delta_text: str

accumulated_text: str

is_finished: bool

finish_reason: str | None = None

time_to_first_token: float | None = None

metrics: Dict[str, float] | None = None

__init__(request_index: int, delta_text: str, accumulated_text: str, is_finished: bool, finish_reason: str | None = None, time_to_first_token: float | None = None, metrics: Dict[str, float] | None = None) → None