tlc.integration.super_gradients.callbacks.base_callback

Callback that logs metrics and hyperparameters from a SuperGradients Trainer in a 3LC Run, and facilitates per-sample metrics collection.

Module Contents

Classes

Class

Description

MetricsCollectionCallback

API

class MetricsCollectionCallback(
project_name: str | None = None,
run_name: str | None = None,
run_description: str | None = None,
image_column_name: str = 'image',
label_column_name: str | None = None,
metrics_collection_epochs: list[int] = [],
collect_metrics_on_train_end: bool = True,
collect_val_only: bool = False,
batch_size: int | None = 32,
pipeline_params: PipelineParams = PipelineParams(),
)

Bases: super_gradients.training.utils.callbacks.Callback

Callback that collects per-sample metrics and logs SuperGradients aggregate metrics to a 3LC run.

To collect per-sample metrics, subclasses must implement the methods compute_metrics and metrics_column_schemas, and the property label_column_name.

Parameters:

  • project_name – The name of the 3LC project to use if no active run exists.

  • run_name – The name of the 3LC run to use if no active run exists.

  • run_description – The description of the 3LC run to use if no active run exists.

  • image_column_name – The name of the column in the table that contains the images.

  • label_column_name – The name of the column in the table that contains the labels. If not provided, a task-specific default will be used.

  • metrics_collection_epochs – The zero-indexed epochs after which to collect metrics.

  • collect_metrics_on_train_end – Whether to collect metrics after training finishes.

  • collect_val_only – Whether to collect metrics only on the validation set.

  • metrics_collection_dataloader_args – Additional arguments to pass to the dataloaders used for metrics collection.

  • batch_size – The batch size to use for metrics collection.

  • pipeline_params – The pipeline parameters to use for metrics collection.

property run: Run
on_training_start(
context: super_gradients.training.utils.callbacks.PhaseContext,
) → None

Called when training starts, creates a 3LC run if no active run exists and logs hyperparameters.

Raises:

ValueError – If project_name or run_name is provided and different from those of an existing active run.

on_train_loader_end(
context: super_gradients.training.utils.callbacks.PhaseContext,
) → None

Called when the training loader ends, logs aggregate SuperGradients metrics to the 3LC run.

on_validation_loader_end(
context: super_gradients.training.utils.callbacks.PhaseContext,
) → None

Called when the validation loader ends, logs aggregate SuperGradients metrics to the 3LC run.

on_training_end(
context: super_gradients.training.utils.callbacks.PhaseContext,
) → None

Called when training ends, invokes metrics collection if configured to and updates the run status.

Parameters:

context – The PhaseContext received from the SuperGradients Trainer.

collect_metrics_direct(
model: super_gradients.training.models.sg_module.SgModule,
tables: list[Table],
constants: dict[str, Any] | None = None,
) → None

Collect metrics directly on tables.

Parameters:

  • model – The model to collect metrics from.

  • tables – The tables to collect metrics from.

  • constants – Additional constants to pass to the metrics collection.

compute_metrics(
images: list[str],
predictions: super_gradients.training.utils.predict.prediction_results.ImagesPredictions | super_gradients.training.utils.predict.prediction_results.ImagePrediction,
table: Table,
) → dict[str, Any]

Compute metrics for a batch of images and corresponding predictions.

Parameters:

  • images – A list of absolute image URLs as strings.

  • predictions – A list of predictions or a single prediction.

  • table – The table containing the images and predictions.

Returns:

A dictionary of computed metrics.

metrics_column_schemas(
table: Table,
) → dict[str, Schema]

Return the column schemas for the metrics of this callback.

property label_column_name: str

The name of the label column.