tlc.integration.detectron2.metrics_collection_hook#

Hook that collects 3LC metrics on a detectron dataset.

Module Contents#

Classes#

Class

Description

MetricsCollectionHook

Hook that periodically collects metrics on a detectron dataset.

API#

class tlc.integration.detectron2.metrics_collection_hook.MetricsCollectionHook(dataset_name: str, metrics_collectors: list[tlc.client.torch.metrics.MetricsCollector] | tlc.client.torch.metrics.MetricsCollector, predictor: torch.nn.Module | tlc.client.torch.metrics.predictor.Predictor | None = None, cfg: detectron2.config.CfgNode | None = None, collection_start_iteration: int = 0, collection_frequency: int = -1, collect_metrics_before_train: bool = False, collect_metrics_after_train: bool = False, metric_collection_batch_size: int = 8, dataset_split: str = '')#

Bases: detectron2.engine.hooks.HookBase

Hook that periodically collects metrics on a detectron dataset.

The hook will collect metrics on the dataset specified by dataset_name, which must be registered in the MetadataCatalog (see the detectron2 docs for more details). The metrics will be collected using the metrics collectors specified in metrics_collectors.

The hook can be configured to collect metrics at the beginning of training, at regular intervals during training, and at the end of training. The frequency of metric collection can be set with collection_frequency, and the iteration to start collecting metrics can be set with collection_start_iteration.

Metrics will be collected after an iteration is completed.

Example:

# Setup detectron2 trainer and register datasets
trainer = ...

# Collect metrics on the training set before and after training:

train_hook = MetricsCollectionHook(
    dataset_name="my_train_dataset",
    metrics_collectors=MyMetricsCollector(),
    collect_metrics_before_train=True,
    collect_metrics_after_train=True,
)

# Collect metrics on the validation set every 100 iterations starting at iteration 1000:
test_hook = MetricsCollectionHook(
    dataset_name="my_test_dataset",
    metrics_collectors=MyMetricsCollector(),
    collection_start_iteration=1000,
    collection_frequency=100,
)

trainer.register_hooks([train_hook, test_hook])
trainer.train()

Initializes the hook

Parameters:

  • dataset_name – The name of the dataset to collect metrics on. This name should be registered in the detectron2 MetadataCatalog.

  • metrics_collectors – The metrics collector(s) to use.

  • predictor – A model or Predictor to use for computing metrics.

  • cfg – The detectron config. If None, the config will be loaded from the trainer.

  • collection_start_iteration – The iteration to start collecting metrics on.

  • collection_frequency – The frequency with which to collect metrics. Must be greater than 0 for metrics to be collected during training.

  • collect_metrics_before_train – Whether to collect metrics at the beginning of training.

  • collect_metrics_after_train – Whether to collect metrics at the end of training.

  • metric_collection_batch_size – The batch size to use for collecting metrics.

  • dataset_split – The split of the dataset to collect metrics on. Will be prepended to the dataset name for any aggregate metric values collected by the hook.

before_train() → None#

Creates a test-dataloader from the trainer and collects metrics if required.

after_train() → None#

Collects metrics if required.

after_step() → None#

Collects 3LC metrics at regular intervals.

property model: torch.nn.Module#

Returns the model from the trainer.