Runs

A Run is the basic unit of computation in the 3LC ecosystem, and represents a single execution of a specific process or experiment. A Run object encapsulates information about setup, state, and results of a computation, as well as providing a handle to access and modify the underlying data and configuration.

When using one of the 3LC integration modules, Run-management is handled automatically, and you will not need to interact with Run objects directly. However, when integrating 3LC manually into e.g. your PyTorch training script, you will need to create and manage Run objects yourself. This guide provides an overview of the Run object and its methods.

Create a Run

Create a run using tlc.init().

import tlc

run = tlc.init(project_name="my-project")

The run created will automatically be assigned as the “active run”, and any subsequent operations that expect a run will use this run by default.

Accessing a Run

The active Run can be accessed using tlc.active_run().

run = tlc.active_run()

Load a Run from a location specified by a Url by using the run.from_url() method.

run = tlc.Run.from_url("...")

Alternatively, if you know the name of the Run as well as the name of the project it belongs to you can use the run.from_names() method.

run = tlc.Run.from_names("My Project", "This Run")

Setting the Active Run

The active Run will automatically be set through calls to tlc.init(). Alternatively, the active Run can be set using tlc.set_active_run().

tlc.set_active_run(run)

Hyperparameters

Keep track of hyperparameters by using the run.set_parameters() method.

config = {
    "epochs": 10,
    "batch_size": 32
}

run.set_parameters(config)

Adding Input Tables

Input values are references Tables that were used as input to the run.

run.add_input_table("/path-to-training-table")
run.add_input_table("/path-to-validation-table")

Adding Output Values

Output values are free-form dictionaries describing simple scalar-valued results of the run.

A Run keeps track of a list of such output-values, which are then used to display results in the “Projects” page of the 3LC Dashboard.

Output values should be reserved for small, simple scalar values. For more complex data, adding references to external metrics tables is recommended.

run.add_output_value({"epoch": 1, "average_loss": 0.1})
run.add_output_value({"epoch": 2, "average_loss": 0.09})

Adding a Metrics Table to a Run

A metrics table is a Table containing metrics data, where each row corresponds to a single metric record. A metrics table can be added to a run using run.add_metrics_table().

metrics_table = ... # create a metrics table, e.g. using a MetricsTableWriter
run.add_metrics_table(metrics_table)

Adding Metrics Data to a Run

For more complex metrics-collection workflows, collect_metrics() should be used. For simpler cases when you just want to record some metrics, you can use run.add_metrics_data() to add a free-form dictionary of metrics data.

metrics_data = {"epoch": [1], "loss": [0.1], "accuracy": [0.9]}
run.add_metrics_data(metrics_data)

Access a Run’s Metrics Tables

Access a Run’s metrics tables using the run.metrics_tables property.

run.metrics_tables

Modifying the Run Status

A Run can have a status, which can be set using run.set_status().

# All Runs start with the status "empty"

run.set_status_running()

# ... do some work ...

run.set_status_completed()

Deleting a Run

A run can be deleted by deleting the run’s url.

run.url.delete()

Copying a Run

A run can be copied to a new location using run.copy().

run.copy("...")