tlc.integration.pytorch_lightning.module_decorator#

Decorator for using 3LC with PyTorch Lightning.

Module Contents#

Functions#

Function

Description

lightning_module

Decorator for using 3LC with PyTorch Lightning.

API#

tlc.integration.pytorch_lightning.module_decorator.lightning_module(structure: tlc.client.sample_type._SampleTypeStructure | None = None, dataset_prefix: str = '', run_name: str | None = None, run_description: str | None = None, project_name: str | None = None, root_url: tlc.core.url.Url | str | None = None, if_dataset_exists: typing.Literal[raise, reuse, rename, overwrite] = 'reuse', if_run_exists: typing.Literal[raise, reuse, rename, overwrite] = 'rename', add_weight_column: bool = True, all_arrays_are_fixed_size: bool = True, exclude_zero_weights_train: bool = True, weighted: bool = True, use_latest: bool = True, map_functions: list[collections.abc.Callable[[object], object]] | collections.abc.Callable[[object], object] | None = None, predictor_args: tlc.client.torch.metrics.predictor.PredictorArgs = PredictorArgs(), metrics_collectors: tlc.client.torch.metrics.metrics_collectors.MetricsCollectorType | None = None, collect_metrics_on_train_end: bool = True, collect_metrics_on_train_start: bool = False, metrics_collection_interval: int = 0, metrics_collection_start: int = 0, metrics_collection_exclude_zero_weights: bool = True, metrics_collection_map_functions: list[collections.abc.Callable[[object], object]] | collections.abc.Callable[[object], object] | None = None) → collections.abc.Callable[[type[tlc.integration.pytorch_lightning.module_decorator._LightningModule]], type[tlc.integration.pytorch_lightning.module_decorator._LightningModule]]#

Decorator for using 3LC with PyTorch Lightning.

Adding this decorator to a pytorch_lightning.LightningModule subclass definition will create tables from the datasets used within the module, as well as collect metrics and create a run when used with a Trainer.

Parameters:

  • structure – The structure of a single sample in the datasets. This is used to infer the schema of the table, and perform any necessary conversions between the row representation and the sample representation of the data. If not provided, the structure will be inferred from the first sample in the dataset.

  • dataset_prefix – A prefix to be added to each split of the dataset to create dataset names.

  • run_name – The name of the run.

  • run_description – The description of the run.

  • project_name – The name of the project.

  • root_url – The root url of the table.

  • if_dataset_exists – What to do if the dataset already exists. One of “raise”, “reuse”, “rename”, “overwrite”.

  • if_run_exists – What to do if the run already exists. One of “raise”, “reuse”, “rename”, “overwrite”.

  • add_weight_column – Whether to add a column of sampling weights to the table, all initialized to 1.0.

  • all_arrays_are_fixed_size – Whether all arrays (tuples, lists, etc.) in the dataset are fixed size. This parameter is only used when generating a SampleType from a single sample in the dataset when no structure is provided.

  • exclude_zero_weights_train – If True, rows with a weight of zero will be excluded from the sampler during training. This is useful for adjusting the length of the sampler, and thus the length of an epoch, to the number of non-zero weighted rows in the table.

  • weighted – If True, the sampler will use sample weights (beyond the exclusion of zero-weighted rows) to ensure that the distribution of the sampled rows matches the distribution of the weights. When weighted is set to True, you are no longer guaranteed that every row in the table will be sampled in a single epoch, even if all weights are equal.

  • use_latest – If True, the latest versions of all tables will be used.

  • map_functions – A list of map functions to be applied to all tables, except when collecting metrics.

  • metrics_collectors – List of metrics collectors to be used to collect metrics.

  • collect_metrics_on_train_end – Whether to collect metrics at the end of training.

  • collect_metrics_on_train_start – Whether to collect metrics at the start of training.

  • metrics_collection_interval – The number of epochs between each metrics collection. A value of 0 means that metrics will not be collected.

  • metrics_collection_start – The epoch to start collecting metrics. Metrics will be collected at the end of the specified epochs.

  • metrics_collection_exclude_zero_weights – Whether to exclude zero-weighted samples when collecting metrics.

  • metrics_collection_map_functions – A list of map functions to be applied to all tables when collecting metrics.