Dataset Class Distribution Analysis Config

Default value: DatasetWarningsOptions()

Some thresholds can be set to modify the number of warnings in the Dataset Class Distribution Analysis.

Class DefinitionConfig Example

from pydantic import BaseModel

class DatasetWarningsOptions(BaseModel):
    min_num_per_class: int = 20 # (1)
    max_delta_representation: float = 0.05 # (2)
    max_delta_mean_tokens: float = 3.0 # (3)
    max_delta_std_tokens: float = 3.0 # (4)

1. Threshold for the first set of warnings (missing samples).
2. Threshold for the second set of warnings (representation mismatch).
3. Threshold for the third set of warnings (length mismatch).
4. Threshold for the third set of warnings (length mismatch).

{
  "dataset_warnings": {
    "min_num_per_class": 40
  }
}