Skip to content

ParameterConfig

Module: fast_llm.engine.config_utils.parameter

Inherits from: ModuleConfig

Fields

initialization — feature

Type: InitializationConfig    Default: (sub-fields optional)

If provided, override the default initialization method set by the parent layer.

lr_scale — feature

Type: float or None    Default: None

Scaling factor for the parameter learning rate. Combines multiplicatively with the scale set by the parent layer, if applicable.

weight_decay — feature

Type: float or bool or None    Default: None

Override the default weight decay for this parameter. Set to True or False to enable or disable the optimizer weight decay, or to a number to use a specific value.

Used in