tlc.client.torch.metrics.metrics_collectors.functional_metrics_collector

Collect metrics using a custom function.

Module Contents

Classes

Class	Description
FunctionalMetricsCollector	A metrics collector which uses a function to collect metrics.

API

class FunctionalMetricsCollector(

collection_fn: Callable[[tlc.core.builtins.types.SampleData, PredictorOutput], dict[str, Any]],

column_schemas: dict[str, Schema] | None = None,

compute_aggregates: bool = True,

)

Bases: tlc.client.torch.metrics.metrics_collectors.metrics_collector_base.MetricsCollector

A metrics collector which uses a function to collect metrics.

Create a new functional metrics collector.

Parameters:

collection_fn – A function for computing custom metrics. The function should take two arguments: a batch of samples, and an instance of PredictorOutput. It should return a dictionary of computed metrics, mapping the names of the metrics to a batch of their values. A trivial collection_fn might look like this:

def collection_fn(batch, predictor_output):
    return {"prediction": predictor_output.forward}

Parameters:

column_schemas – A dictionary of schemas for the columns. If no schemas are provided, the schemas will be inferred from the column data.

compute_metrics(

batch: tlc.core.builtins.types.SampleData,

predictor_output: PredictorOutput,

) → dict[str, Any]

property column_schemas: dict[str, Schema]