MonolithicLossConfig

Module: fast_llm.layers.language_model.loss.config

Inherits from: LanguageModelLossConfig

Fields

losses — core

Type: dict[str, LanguageModelLossConfig]    Default: dict()

The combinable losses sharing a single softmax pass. They must agree on logits_scale_factor.

weight — core

Type: float    Default: 1.0

Weight for this loss in the total loss computation.

logits_scale_factor — feature

Type: float    Default: 1.0

Extra logits scale factor applied for this loss only, stacked on top of the model's logits_scale_factor.

use_triton — expert

Type: bool or None    Default: None

Enable the triton implementation of the shared-softmax kernel. Default: use if available.