FixedRMSNormConfig

Module: fast_llm.layers.common.normalization.config

Variant of: NormalizationConfig — select with type: fixed_rms_norm

Inherits from: NormalizationConfig, ModuleConfig

Fields

epsilon — architecture

Type: float    Default: 1e-05

Regularizer for the division.

lr_scale — feature

Type: float or None    Default: None

Scaling factor for the layer learning rate. Combines multiplicatively with the scale set by the parent layer and individual parameters, if applicable.