Configuration

3LC has several configuration options, which control data discovery, logging, and network configuration.

All the options are documented in the tlcconfig.options module. The documentation for each option details the configuration file key, environment variable and command-line argument to the 3LC Object Service that can be used to set the option.

Due to 3LC’s architecture, where multiple processes are involved (the 3LC Object Service, Jupyter Notebooks, and the 3LC Dashboard run as distinct processes), there are some caveats to be aware of when configuring 3LC.

The preferred way of configuring 3LC is through the configuration file. The configuration file is located in $HOME/.config/3LC/config.yaml on Linux and in %LOCALAPPDATA%\3LC\3LC\config.yaml on Windows. The location can be overridden by the TLC_CONFIG_FILE environment variable.

This file is created, with default values, the first time 3LC is launched (either when starting the Object Service, or when invoking import tlc in a notebook). An example configuration file can be seen below.

Example config.yaml

# Configuration file for `tlc`.
#
# This YAML file contains setting for the `tlc` Python Package.
# Created at 2024-05-17 02:46:22


# Service Settings
service:

  port: 5015                                      # Port for the server.

  host: 127.0.0.1                                 # Host for the server.

  # Specify license or license file
  #  
  # The option can be either be the license key or point to a local file containing the license key.
  license: ''

  # Specify the amount of memory to use for caching, in bytes.
  #  
  # Setting the value to 0 will disable in-memory caching.
  #  
  # Default: 1073741824 (1 GB)
  cache_in_memory_size: 1073741824

  # Specify the cache item time out, in seconds.
  #  
  # Setting the value to 0 will disable cache eviction based on time.
  #  
  # Default: 3600 (1 hour)
  cache_time_out: 3600

# Indexing Settings
indexing:

  # Location for reading and writing 3LC data.
  #  
  # This option is mandatory and must point to a location (e.g. directory on disk or object store bucket) with write
  # access. The location will be created if it does not exist.
  #  
  # If the option value contains an environment variable, it will be expanded.
  project-root-url: /home/build/.local/share/3LC/projects

  # Locations to scan for 3LC objects (runs and tables) following a standard 3LC project layout.
  #  
  # Usually a 'projects' directory containing multiple projects. Default directories will be created if they do not
  # exist.
  project-scan-urls: []

  # Extra (non-recursive) locations to scan for 3LC Table objects.
  #  
  # This option allows searching individual folders/locations for 3LC Table objects outside the standard hierarchy of a
  # 3LC project structure. For example:
  #  - "C:\Users\user\Documents\3LC\tables"
  #  - "s3://my-bucket/3LC/tables".
  #  
  # Note: these locations will not be scanned recursively.
  #  
  # Indexed Tables from these extra locations can be used interchangeably with other Tables discovered by the system.
  extra-table-scan-urls: []

  # Extra (non-recursive) locations to scan for 3LC Run objects.
  #  
  # This option allows searching individual folders/locations for 3LC Run objects outside the standard hierarchy of a
  # 3LC project structure. For example:
  #  - "C:\Users\user\Documents\3LC\runs"
  #  - "s3://my-bucket/3LC/runs".
  #  
  # Note: these locations will not be scanned recursively.
  #  
  # Indexed Runs from these extra locations can be used interchangeably with other Runs discovered by the system.
  extra-run-scan-urls: []

# Logging Settings
logging:

  # Log file for the 3LC logger.
  #  
  # The directory will be created if it does not exist.
  #  
  # If the option value contains an environment variable, it will be expanded.
  logfile: /home/build/.local/state/3LC/log/3LC.log

  # Log level for the 3LC logger.
  #  
  # The `tlc` Python package adheres to the standard Python logging levels:
  #  
  #   - DEBUG:  Detailed information, typically of interest only when diagnosing problems.
  #   - INFO: Confirmation that things are working as expected.
  #   - WARNING: An indication that something unexpected happened, or indicative of some problem in the near future
  # (e.g. "disk space low"). The software is still working as expected.
  #   - ERROR: Due to a more serious problem, the software has not been able to perform some function.
  #   - CRITICAL: A serious error, indicating that the program itself may be unable to continue running.
  loglevel: WARNING

# Tlc Settings
tlc:

  # Whether to display progress bars or not.
  #  
  # The option can be either 0 or 1, where 0 means no progress bars and 1 means progress bars.
  display-progress: 1

  telemetry: 1                                    # Whether to send Telemetry or not.


# Aliases Settings
aliases: {}                                       # List of aliases.

Overriding Configuration Options

3LC follows the following precedence rules for configuration:

api-call > command-line argument > environment variable > configuration file

The rules above should be straightforward when launching the Object Service from the command, but it is more complicated when importing the tlc package in a Jupyter notebook (e.g. import tlc). Importing tlc causes the default configuration file to be read, and it is too late to modify these options after the module has been imported.

For this reason, there exists a tlcconfig-module that allows for creating and modifying an OptionLoader-instance prior to importing the main tlc-module. See the module documentation for an example of how to use this module.