FixedBlockSequenceConfig

Module: fast_llm.layers.block.config

Variant of: BlockSequenceConfig — select with type: fixed

Inherits from: BlockSequenceConfig, BlockConfig, ModuleConfig

Fields

block — architecture

Type: BlockConfig    Default: (sub-fields optional)

Common configuration for all the blocks.

num_blocks — architecture

Type: int    Default: 12

Number of blocks in the model.

lr_scale — feature

Type: float or None    Default: None

Scaling factor for the layer learning rate. Combines multiplicatively with the scale set by the parent and child layers, if applicable.