GPTTrainerConfig

Module: fast_llm.models.gpt.config

Variant of: TrainerConfig — select with type: gpt

Variant of: RunnableConfig — select with type: train_gpt

Inherits from: PretrainedGPTModelConfig, TrainerConfig, PretrainedFastLLMModelConfig

Fields

data — core

Type: GPTDataConfig    Default: (sub-fields optional)

Configuration for the dataset and model-independent preprocessing.

model — core

Type: GPTModelConfig    Default: (sub-fields optional)

Configuration for the Fast-LLM model.

optimizer — core

Type: OptimizerConfig    Default: (sub-fields optional)

Configuration for the training optimizer and learning rate schedule.

run — core

Type: RunConfig    Default: (sub-fields optional)

Global properties for the experiment.

schedule — core

Type: ScheduleConfig    Default: (sub-fields optional)

Configuration for the scheduling of each iteration.

training — core

Type: TrainingConfig    Default: (sub-fields optional)

Configuration for the training phases and global properties.

callbacks — feature

Type: dict[str, TrainerCallbackConfig]    Default: dict()

Configuration for training callbacks.

pretrained — feature

Type: CheckpointLoadConfig    Default: (sub-fields optional)

Configuration for loading the configuration and state of a pretrained model.

reference_models — feature

Type: dict[str, PretrainedGPTModelConfig]    Default: dict()

Auxiliary models used during training, ex. for knowledge distillation.

profiling — logging

Type: ProfilingConfig    Default: (sub-fields optional)

Configuration for the optional profiling of GPU and CPU CUDA operations.