dragon.ai.inference.batching.Batch

class Batch

Bases: object

A collection of items to be processed together.

__init__(items: List [BatchItem], batch_id: int , created_at: float )

Initialize a Batch instance.

Parameters:

• items (list [BatchItem]) – List of BatchItem instances.

• batch_id (int ) – Unique identifier for the batch.

• created_at (float ) – Timestamp when the batch was created.

Methods

__init__(items, batch_id, created_at)

Initialize a Batch instance.

Attributes

continue_final_message_list	Per-request continue_final_message flags.
formatted_prompts	Extract formatted prompts from batch items.
json_schema_list	Per-request JSON schema overrides for guided decoding.
latency_metrics	Extract latency metrics from batch items.
response_queues	Extract response queues from batch items.
size	Get the batch size.
tools_list	Per-request tool definitions.
user_prompts	Extract user prompts from batch items.

__init__(items: List [BatchItem], batch_id: int , created_at: float )

Initialize a Batch instance.

Parameters:

• items (list [BatchItem]) – List of BatchItem instances.

• batch_id (int ) – Unique identifier for the batch.

• created_at (float ) – Timestamp when the batch was created.

property size: int

Get the batch size.

Returns:

Number of items in the batch.

Return type:

int

property user_prompts: List [str ]

Extract user prompts from batch items.

Returns:

List of user prompts.

Return type:

list [str ]

property formatted_prompts: List [str ]

Extract formatted prompts from batch items.

Returns:

List of formatted prompts.

Return type:

list [str ]

property response_queues: List [Queue]

Extract response queues from batch items.

Returns:

List of response queues.

Return type:

list [dragon.native.Queue]

property latency_metrics: List [Tuple [float , float , float ]]

Extract latency metrics from batch items.

Returns:

List of latency metrics tuples.

Return type:

list [tuple [float , float , float ]]

property tools_list: List [List [Dict [str , Any ]] | None ]

Per-request tool definitions.

property json_schema_list: List [dict | None ]

Per-request JSON schema overrides for guided decoding.

property continue_final_message_list: List [bool ]

Per-request continue_final_message flags.