OptionalParameterConfig

Module: fast_llm.engine.config_utils.parameter

Inherits from: ParameterConfig, ModuleConfig

Fields

initialization — feature

Type: InitializationConfig    Default: (sub-fields optional)

If provided, override the default initialization method set by the parent layer.

lr_scale — feature

Type: float or None    Default: None

Scaling factor for the parameter learning rate. Combines multiplicatively with the scale set by the parent layer, if applicable.

enabled

Type: bool or None    Default: None

Used in

• bias in AffineLinearBaseConfig
• bias in AffineLinearConfig
• bias in CausalConv1dConfig
• position_embeddings in LanguageModelEmbeddingsConfig