PatchEmbeddingsConfig¶

Module: fast_llm.layers.vision.config

Inherits from: BlockConfig, ModuleConfig

Fields¶

normalization — architecture

Type: NormalizationConfig Default: (sub-fields optional)

Configuration for the normalization layer.

patch_embeddings — architecture

Type: AffineLinearConfig Default: (sub-fields optional)

Configuration for the patch embedding layer.

patch_height — core

Type: int Default: 16

Height of image patches, in pixels.

patch_width — core

Type: int Default: 16

Width of image patches, in pixels.

lr_scale — feature

Type: float or None Default: None

Scaling factor for the layer learning rate. Combines multiplicatively with the scale set by the parent and child layers, if applicable.

full_precision_residual — stability

Type: bool Default: False

Store the residuals for the model in full precision (optimization_dtype).