dragon.ai.inference.llm_engine.StreamChunk

class StreamChunk[source] 

Bases: object

A single chunk yielded during streaming generation.

Each chunk represents incremental output from the LLM, typically one or more tokens. The final chunk for a request has is_finished=True and includes complete metrics.

Parameters:

request_index (int ) – Index of this request within the batch (0 for single-request streaming).
delta_text (str ) – New text generated since the last chunk. Empty string for the final sentinel chunk.
accumulated_text (str ) – Full text generated so far for this request.
is_finished (bool ) – True when generation for this request is complete.
finish_reason (str | None) – Reason generation stopped (e.g., “stop”, “length”). Only populated when is_finished=True.
time_to_first_token (float | None) – Time in seconds from request start to first token. Populated on every chunk after the first token arrives.
metrics (dict | None) – Performance metrics. Only populated on the final chunk.

__init__(request_index: int , delta_text: str , accumulated_text: str , is_finished: bool , finish_reason: str | None = None, time_to_first_token: float | None = None, metrics: Dict [str , float ] | None = None) → None 

Methods

__init__(request_index, delta_text, ...[, ...])

Attributes

`finish_reason`
`metrics`
`time_to_first_token`
`request_index`
`delta_text`
`accumulated_text`
`is_finished`

request_index: int 

delta_text: str 

accumulated_text: str 

is_finished: bool 

finish_reason: str | None = None

time_to_first_token: float | None = None

metrics: Dict [str , float ] | None = None

__init__(request_index: int , delta_text: str , accumulated_text: str , is_finished: bool , finish_reason: str | None = None, time_to_first_token: float | None = None, metrics: Dict [str , float ] | None = None) → None 