dragon.ai.inference.config.DynamicWorkerConfig
- class DynamicWorkerConfig[source]
Bases:
objectDynamic inference worker spin-up and spin-down configuration.
Controls the optional energy-saving path where extra GPU inference workers can shut down after idle periods and restart when prompt pressure rises.
- Parameters:
enabled (bool ) – Enable dynamic worker lifecycle management. If
False, the initially started workers remain active until service shutdown.min_active_workers_per_cpu (int ) – Minimum number of inference workers that remain active under each CPU-head worker.
spin_down_threshold_seconds (int ) – Idle time before a worker above the minimum active count is allowed to shut down.
spin_up_threshold_seconds (int ) – Rolling time window used to count incoming prompts for spin-up decisions.
spin_up_prompt_threshold (int ) – Number of prompts within the spin-up window required to restart an available worker.
- __init__(enabled: bool = True, min_active_workers_per_cpu: int = 1, spin_down_threshold_seconds: int = 3600, spin_up_threshold_seconds: int = 3, spin_up_prompt_threshold: int = 5) None
Methods
__init__([enabled, ...])validate()Validate dynamic worker configuration.
Attributes