Layers / Attention / Rotary¶
Classes¶
| Class | Description |
|---|---|
| DefaultRotaryConfig | |
| Llama3RotaryConfig | Llama3 scaling: https://github.com/meta-llama/llama-models/blob/baf7b01b6e62bc7126c7b558d2b67d4533142680/models/llama3/reference_impl/model.py#L45-L67 |
| NoRotaryConfig | |
| Rotary2DConfig | |
| RotaryConfig (abstract) | |
| YarnRotaryConfig | Yarn scaling: |