`tlc.client.torch.metrics.metrics_collectors.functional_metrics_collector`#

Collect metrics using a custom function.

Module Contents#

Class	Description
`FunctionalMetricsCollector`	A metrics collector which uses a function to collect metrics.

class tlc.client.torch.metrics.metrics_collectors.functional_metrics_collector.FunctionalMetricsCollector(collection_fn: Callable[[tlc.core.builtins.types.SampleData, tlc.client.torch.metrics.predictor.PredictorOutput], dict[str, Any]], column_schemas: dict[str, tlc.core.schema.Schema] | None = None, compute_aggregates: bool = True)#

A metrics collector which uses a function to collect metrics.

Create a new functional metrics collector.

Parameters:: collection_fn – A function for computing custom metrics. The function should take two arguments: a batch of samples, and an instance of PredictorOutput. It should return a dictionary of computed metrics, mapping the names of the metrics to a batch of their values. A trivial collection_fn might look like this:

def collection_fn(batch, predictor_output):
    return {"prediction": predictor_output.forward}

Parameters:: column_schemas – A dictionary of schemas for the columns. If no schemas are provided, the schemas will be inferred from the column data.

compute_metrics(batch: tlc.core.builtins.types.SampleData, predictor_output: tlc.client.torch.metrics.predictor.PredictorOutput) → dict[str, Any]#