Skip to content

ScheduleConfig

Module: fast_llm.engine.schedule.config

Fields

breadth_first_micro_batches — core

Type: int    Default: 1

Number of micro-batches processed breadth-first, i.e., interleaved across model stages.

depth_first_micro_batches — core

Type: int    Default: 1

Number of micro-batches processed depth-first, i.e., each runs through all model stages before the next begins. This is the standard way to perform gradient accumulation.

micro_batch_splits — performance

Type: int    Default: 1

Number of splits for each micro-batch.

debug_schedule — logging

Type: bool    Default: False

Log the whole schedule.

debug_send_recv — logging

Type: bool    Default: False

Log the pipeline-parallel operations.

profile_schedule — logging

Type: bool    Default: False

Detailed time table for the schedule execution (cpu and gpu times).

throttle_cpu — expert

Type: bool    Default: True

Avoid scheduling too many operations in advance to limit memory fragmentation and prevent costly memory cache flushes.

throttle_cpu_delay — expert

Type: int    Default: 1

Synchronize with a cuda event registered this many steps before, to avoid a full synchronization.

throttle_cpu_rate — expert

Type: int    Default: 1

Number of schedule steps between each cpu throttling.

Used in