3LC Python Package Version 2.8

2.8.0

Features

  • [13435] Made it so that when writing bulk data for a Table, URLs are relativized with respect to the Table if the URL is within the same project, which will make it easier to relocate and share projects without requiring the definition of an alias. Note that table_rows continues to provide a raw view of the data, which will now show these relative URLs, but that __getitem__ (used when iterating over the Table itself) will always make URLs absolute.

  • [13344] For the Object Service, allow immutable objects and external data (e.g. images) to be cached in the per user browser cache by setting the "Cache-Control" header in responses. This should improve performance for repeated access to the same project data.

  • Made it easier to control various configuration options via environment variables, in addition to the config file

    • [13373] Made it possible to override the default config directory with the environment variable TLC_DEFAULT_ROOT_DIR. This can be useful e.g. when the default directory is read-only.

    • [13428] Made it so aliases can optionally be specified via environment variables with the prefix TLC_ALIAS_, in addition to the config file

    • [13434] Made it so the options for project scan URLs, extra table scan URLs, and extra run scan URLs can be specified via environment variables TLC_CONFIG_PROJECT_SCAN_URLS, TLC_CONFIG_EXTRA_TABLE_SCAN_URLS, and TLC_CONFIG_EXTRA_RUN_SCAN_URLS, respectively, in addition to the config file.

  • [13400] Added RepeatByWeightSampler, which allows the use of sampling weights to control the relative importance of samples, but also guarantees that all samples with weight > 1 are shown every single epoch

Enhancements and Fixes

  • [13259] Set the row_count of a TableWriter set to 0 when calling finalize to allow for reusing the same TableWriter to write multiple Tables

  • [12858] Retry file delete operations on Windows to make them more robust

  • [13333] Made the error message when failing to load a table to export more specific

  • [13351] When deleting a Table as a result of overwriting an existing URL, delete the standard bulk_data directory that would be associated with the Table as well.

  • [13326] Give more helpful feedback in the case where a Table is created from an empty torch dataset with len(dataset)==0. Previously this would fail with an index error when peaking at the first element of the dataset to create/validate the SampleType.

  • [13358] Raise an error instead of allowing silent failure for <PIL.Image>.save(url) given a tlc.Url that would result in a corrupt file

  • [13369] Don’t crash in TableFromTorchDataset if __len__ is incorrectly implemented for the Torch dataset such that it returns a length that is too high for __getitem__. Instead, if we run into IndexError or StopIteration(common signals for terminating the iteration of a Torch dataset) before the specified __len__, the Table will be created from the rows extracted before the raised exception (as if __len__ was not implemented) and a helpful warning will be logged.

  • [13364] Fixed an issue preventing the possibility of adding a class to a segmentation mask

  • [13387] Made it possible to continue on error during import of optional integration packages. Previously, we would disable an integration if a required package was not importable, but raise an error if the package was importable but failed to load (i.e. the package is installed but there is some problem with the installation). Now, we log a warning if such a package fails to load, disable the integration, and continue without raising an error to calling code.

  • [13417] Made the error message when specifying an alias with an invalid key more helpful

  • [13416] Made the error message when writing empty batches to TableWriters more informative

  • [13373] Made is so configuration validation is deferred until it will be used, added an option 3lc config --validate that validates the configuration, and updated 3lc config --list to be able to run and show the current configuration even if it is invalid

  • [13413] In TableFromCoco, handle the case where the file_name stored in the annotations file is absolute and an image_folder is also specified. In this case, the image_folder is ignored.

  • [13382] Fixed an issue where importing tlc would fail if an older version of torch (<1.9) was installed, which has inference_mode in a different package than more recent versions of torch

  • [13437] Fixed an issue where use of an API key would be assumed unless TLC_LICENSE is defined, which meant that specifying a license key in the config file or command-line no longer worked

  • [13208] Ignore/discard unknown fields when edited tables are saved instead of raising an error

  • [9721] Relocated many constants to be inside modules under tlc.core.builtins.constants rather than scattered throughout the codebase

  • [13436] Fixed a bug in TableFromYolo where rows were removed when a label file existed but was empty or whitespace-only. Now they are kept with no boxes.

  • [13457] Fixed an issue in TableFromYolo where an error was incorrectly raised when numpy floats were used for bounding box coordinates

Known Issues

  • The tlc Python package does not detect, handle, or support NaN (Not-a-Number) values in tlc.Table, and their presence may lead to unpredictable behavior or inconsistencies within the system.