dragon.ai.inference.config.BatchingConfig

class BatchingConfig

Bases: object

Dynamic batching configuration.

__init__(enabled: bool = True, batch_type: str = 'dynamic', batch_wait_seconds: float = 0.1, max_batch_size: int = 60) → None

Methods

__init__([enabled, batch_type, ...])
validate()	Validate batching configuration.

Attributes

batch_type
batch_wait_seconds
enabled
max_batch_size

enabled: bool = True

batch_type: str = 'dynamic'

batch_wait_seconds: float = 0.1

max_batch_size: int = 60

validate() → None

Validate batching configuration.

__init__(enabled: bool = True, batch_type: str = 'dynamic', batch_wait_seconds: float = 0.1, max_batch_size: int = 60) → None