tlc.integration.super_gradients.callbacks.base_callback

Callback that logs metrics and hyperparameters from a SuperGradients Trainer in a 3LC Run, and facilitates per-sample metrics and embeddings collection.

Module Contents

Classes

Class

Description

MetricsCollectionCallback

Callback that collects per-sample metrics and logs SuperGradients aggregate metrics to a 3LC run.

API

class MetricsCollectionCallback(
project_name: str | None = None,
run_name: str | None = None,
run_description: str | None = None,
image_column_name: str = 'image',
label_column_name: str | None = None,
metrics_collection_epochs: list[int] | None = None,
collect_metrics_on_train_end: bool = True,
collect_val_only: bool = False,
batch_size: int | None = 32,
pipeline_params: PipelineParams | dict[str, Any] | None = None,
collect_predictions: bool | None = None,
collect_embeddings: bool = False,
embeddings_dim: int = 2,
embeddings_method: Literal[pacmap, umap] = 'pacmap',
inference_chunk_size: int = 5000,
)

Bases: super_gradients.training.utils.callbacks.Callback

Callback that collects per-sample metrics and logs SuperGradients aggregate metrics to a 3LC run.

To collect per-sample metrics, subclasses must implement the methods compute_metrics and metrics_column_schemas, and the property label_column_name.

Create a new metrics collection callback.

Parameters:

  • project_name – The name of the 3LC project to use if no active run exists.

  • run_name – The name of the 3LC run to use if no active run exists.

  • run_description – The description of the 3LC run to use if no active run exists.

  • image_column_name – The name of the column in the table that contains the images.

  • label_column_name – The name of the column in the table that contains the labels. If not provided, a task-specific default will be used.

  • metrics_collection_epochs – The zero-indexed epochs after which to collect metrics.

  • collect_metrics_on_train_end – Whether to collect metrics after training finishes.

  • collect_val_only – Whether to collect metrics only on the validation set.

  • metrics_collection_dataloader_args – Additional arguments to pass to the dataloaders used for metrics collection.

  • batch_size – The batch size to use for metrics collection.

  • pipeline_params – The pipeline parameters to use for metrics collection. Can be provided as a PipelineParams instance, a dictionary, or None (default) which will use a default PipelineParams instance.

  • collect_predictions – Whether to collect predictions. Default is None, which means predictions will be logged for all task-specific subclasses if not explicitly set to False. On using the base callback, no predictions will be logged by default.

  • collect_embeddings – Whether to collect embeddings. Is only applied if the model has a backbone attribute (Yolo-NAS models have this). Default is False because embeddings collection performs additional inference and therefore spends additional time.

  • embeddings_dim – The dimensionality to reduce the embeddings to, default is 2. 2 or 3 are recommended.

  • embeddings_method – The method to use for reducing the embeddings.

  • inference_chunk_size – SuperGradients Pipelines load all images in memory at once. This parameter effectively controls how many images to load in memory at once, and how many rows each metrics table will contain.

property run: Run
on_training_start(
context: super_gradients.training.utils.callbacks.PhaseContext,
) → None

Called when training starts, creates a 3LC run if no active run exists and logs hyperparameters.

Raises:

ValueError – If project_name or run_name is provided and different from those of an existing active run.

on_train_loader_end(
context: super_gradients.training.utils.callbacks.PhaseContext,
) → None

Called when the training loader ends, logs aggregate SuperGradients metrics to the 3LC run.

on_validation_loader_end(
context: super_gradients.training.utils.callbacks.PhaseContext,
) → None

Called when the validation loader ends, logs aggregate SuperGradients metrics to the 3LC run.

on_training_end(
context: super_gradients.training.utils.callbacks.PhaseContext,
) → None

Called when training ends, invokes metrics collection if configured to and updates the run status.

Parameters:

context – The PhaseContext received from the SuperGradients Trainer.

collect_metrics_direct(
model: super_gradients.training.models.sg_module.SgModule,
tables: list[Table],
constants: dict[str, Any] | None = None,
) → None

Collect metrics directly on tables.

Parameters:

  • model – The model to collect metrics from.

  • tables – The tables to collect metrics from.

  • constants – Additional constants to pass to the metrics collection.

compute_metrics(
images: list[str],
predictions: super_gradients.training.utils.predict.prediction_results.ImagesPredictions | super_gradients.training.utils.predict.prediction_results.ImagePrediction,
table: Table,
) → dict[str, Any]

Compute metrics for a batch of images and corresponding predictions.

Parameters:

  • images – A list of absolute image URLs as strings.

  • predictions – A list of predictions or a single prediction.

  • table – The table containing the images and predictions.

Returns:

A dictionary of computed metrics.

metrics_column_schemas(
table: Table,
) → dict[str, Schema]

Return the column schemas for the metrics of this callback.

property label_column_name: str

The name of the label column.