LearningRateScheduleConfig¶

Module: fast_llm.engine.optimizer.config

Fields¶

base — core

Type: float Default: 0.0001

Base learning rate for the optimizer.

decay_iterations — feature

Type: int or None Default: None

Duration of the learning rate decay, in iterations.

decay_power — feature

Type: float Default: 1.0

Exponent for learning rate decay, applied on the decay step..

decay_style — feature

Type: str Default: "constant"

The learning rate decay formula.

minimum — feature

Type: float Default: 0.0

Learning rate at the end of decay.

warmup_iterations — feature

Type: int Default: 0

Number of iteration for the learning rate warmup.

schedule — wip

Type: str or None Default: None

Complex learning rate schedule encoded in a string (untested, replaces the other arguments.