tlc.core.writers.metrics_writer

A class for writing metrics to persistent storage.

Module Contents

Classes

Class	Description
MetricsTableWriter	A class for writing metrics tables to runs.

API

class tlc.core.writers.metrics_writer.MetricsTableWriter(run_url: tlc.core.url.Url | str | None = None, foreign_table_url: tlc.core.url.Url | str = '', foreign_table_display_name: str = '', column_schemas: dict[str, tlc.core.schema.Schema] = {})

Bases: tlc.core.writers.table_writer.TableWriter

A class for writing metrics tables to runs.

Updating Runs with Metrics Tables

A MetricsTableWriter is a specialized TableWriter that writes tables inside a run’s directory. The MetricsTableWriter does not update the run to reference the newly written tables. To do that, call Run.update_metrics with the return value of get_written_metrics_infos.

If a foreign_table_url is supplied, the written metrics table will also be associated with the given foreign table, indicating that each metric value is associated with a specific row in the foreign table.

For this to work, each added metrics batch must contain a column called example_id. This is the foreign key that links the metrics table to the foreign table. The values of example_id are linear indices into the foreign table, starting from 0. A single metrics table can contain multiple values for the same example_id, and does not need to contain values for all example_ids in the foreign table.

Example:

from tlc import MetricsTableWriter

# Assuming a input table of length 8 exists at the url "input_table_url"

run = tlc.init()

metrics_writer = MetricsTableWriter(
    run_url=run.url,
    foreign_table_url="input_table_url",
    foreign_table_display_name="Input Table",
)

# First batch of metrics, corresponding to the first 4 rows of the foreign table
metrics_writer.add_batch({
    "loss": [0.1, 0.2, 0.3, 0.4], "example_id": [0, 1, 2, 3],
})

# Second batch of metrics, corresponding to the last 4 rows of the foreign table
metrics_writer.add_batch({
    "loss": [0.2, 0.4, 0.1, 0.5], "example_id": [4, 5, 6, 7],
})

# Finalize the metrics writer to write the metrics table to persistent storage
metrics_table = metrics_writer.finalize()

# Add a reference to the written metrics table to the run
run.update_metrics(metrics_writer.get_written_metrics_infos())

Initialize a MetricsTableWriter.

Parameters:

• run_url – The Url of the run to write metrics for. Will default to the active run if not provided.

• foreign_table_url – The Url of the dataset to write metrics for.

• foreign_table_display_name – An optional display-name of the foreign table to show in the Dashboard. If not provided the dashboard will generate one from the URL.

• column_schemas – A dictionary of column names to schema overrides. Schemas will be inferred from the data if not provided.

add_batch(metrics_batch: MutableMapping[str, tlc.core.builtins.types.MetricData]) → None

Add a batch of metrics to the internal buffer.

The metrics batches sent in to subsequent calls to add_batch on the same MetricsTableWriter must have the same metrics keys, with values of the same type and length.

Parameters:

metrics_batch – A dictionary of metrics, where the keys are the metric names and the values are lists of metric values.

Raises:

ValueError: If the added batch can’t be written to persistent storage, or is not consistent with the previous batches.

get_written_metrics_infos() → list[tlc.core.builtins.types.MetricTableInfo]

Get the list of written metrics infos.

Returns:

A list of written metrics infos. The returned Urls are relative to the run’s Url.

finalize() → tlc.core.objects.table.Table

Write all added batches to persistent storage and return the written table.

Returns:

The written metrics table.