tlc.integration.hugging_face.trainer#

A drop-in replacement for the 🤗 transformers Trainer.

Module Contents#

Classes#

Class

Description

TLCTrainer

TLCTrainer is meant to be used as a drop-in replacement for the 🤗 transformers Trainer, adding per-sample metrics collection on both the train and eval datasets every time .evaluate() is called.

API#

class tlc.integration.hugging_face.trainer.TLCTrainer(*args: Any, compute_tlc_metrics: Callable[..., dict[str, tlc.core.builtins.types.MetricData]] | None = None, collect_tlc_metrics_before_training: bool = False, **kwargs: Any)#

Bases: transformers.Trainer

TLCTrainer is meant to be used as a drop-in replacement for the 🤗 transformers Trainer, adding per-sample metrics collection on both the train and eval datasets every time .evaluate() is called.

To specify what metrics to collect, pass in a function to the compute_tlc_metrics argument that takes in a batch of data and returns a dictionary of per-sample metrics for the batch.

Parameters:
  • compute_tlc_metrics – A function that takes in a batch of data and returns a dictionary of metrics.

  • collect_tlc_metrics_before_training – Whether to collect metrics before training starts.

train(*args: Any, **kwargs: Any) Any#
prediction_step(*args: Any, **kwargs: Any) tuple[torch.Tensor | None, torch.Tensor | None, torch.Tensor | None]#
evaluate(eval_dataset: torch.utils.data.Dataset | None = None, ignore_keys: list[str] | None = None, metric_key_prefix: str = 'eval') dict[str, float]#