Layers / Decoder¶
Sections¶
Classes¶
| Class | Description |
|---|---|
| BlockWithBiasConfig (abstract) | A common interface for various blocks and block layers |
| DecoderBlockConfig | |
| MLPBaseConfig (abstract) | Abstract base configuration for MLP (feedforward) layers. Use type: mlp or type: moe to select a variant |
| MixerConfig (abstract) | Base config class for mixers |
| StochasticMixerConfig | Stochastic mixer that uniformly samples from multiple mixer options during training |