tlc.client.torch.metrics.metrics_collectors.functional_metrics_collector#

Use a custom function to collect metrics.

Module Contents#

Classes#

Class

Description

FunctionalMetricsCollector

A metrics collector which uses a function to collect metrics.

API#

class tlc.client.torch.metrics.metrics_collectors.functional_metrics_collector.FunctionalMetricsCollector(collection_fn: Callable[[Any, Any, Any], dict[str, Any]], column_schemas: dict[str, tlc.core.schema.Schema] | None = None, model: Any | None = None, compute_aggregates: bool = True)#

Bases: tlc.client.torch.metrics.metrics_collectors.metrics_collector_base.MetricsCollector

A metrics collector which uses a function to collect metrics.

Parameters:
  • collection_fn – A function which takes a batch of inputs, a batch of predictions, and a dictionary of hook outputs and returns a dictionary of metrics.

  • column_schemas – A dictionary of schemas for the columns. If no schemas are provided, the schemas will be inferred from the columns.

  • model – The model to use when collecting metrics.

compute_metrics(batch: tlc.core.builtins.types.SampleData, predictions: tlc.core.builtins.types.SampleData | None = None, hook_outputs: dict[int, torch.Tensor] | None = None) dict[str, Any]#
property column_schemas: dict[str, tlc.core.schema.Schema]#