RotaryConfig¶
Abstract
This class cannot be instantiated directly. Use one of the variants listed below.
Module: fast_llm.layers.attention.rotary.config
Inherits from: ModuleConfig
No user-configurable fields.
Variants¶
Select a variant by setting type: to one of the following values.
type value |
Class | Description |
|---|---|---|
default |
DefaultRotaryConfig | |
default_2d |
Rotary2DConfig | |
llama3 |
Llama3RotaryConfig | Llama3 scaling: https://github.com/meta-llama/llama-models/blob/baf7b01b6e62bc7126c7b558d2b67d4533142680/models/llama3/reference_impl/model.py#L45-L67 |
none |
NoRotaryConfig | |
yarn |
YarnRotaryConfig | Yarn scaling: |