3LC Python Package Version 2.23

2.23.0

3lc 2.23 is the last planned minor version prior to the release of 3lc 3.0, a major release that includes breaking changes. This release anticipates many of those breaking changes by doing the following:

• Introducing new APIs for functionality that will remain present in 3.0 but be called in a different way

• Providing deprecation warnings for existing APIs that will be replaced by those new APIs and then internally forwarding calls to the new APIs

We recommend that users update to the new APIs to get rid of all deprecation warnings in their existing code. This will prepare the way for a smooth future transition to 3lc 3.0.

Features

• [16359] Rewrote the CLI to use Click

  • Changed some commands from being invoked with parameters to actually being commands:

    • 3lc config --list -> 3lc config show

    • 3lc config --to-yaml -> 3lc config show --format yaml

    • 3lc config --validate -> 3lc config validate

    • 3lc config --write -> 3lc config write

  • Parameters specific to what previously were other parameters have been moved to be parameters on the new commands. E.g. --quiet is now a parameter on 3lc config show, not a parameter on 3lc config tied to --list.

  • Old commands still work but raise a deprecation warning, until removal in 3.0

  • Added 3lc version and 3lc -V as alternative ways to get the version

  • Configuration options are available on commands, both before and after the command name. For example, one can now do 3lc --loglevel DEBUG service or 3lc service --loglevel DEBUG. If such an option is passed multiple times, an error is raised.

  • Configuration options that belong on subcommands (service.host) are now only listed on the subcommand. They are also left as deprecated options on the top level command. For example, you can now use 3lc service --host <host>, but so 3lc --host <host> service still works but emits a deprecation warning.

• [16265] Introduced ProjectLayout and ProjectHelper to centralize project logic

  • Extracted all project folder layout URL construction, classification, parsing, and listing logic into a new ProjectLayout static helper class

  • Moved several project-level operational helpers into a new ProjectHelper class

  • These classes and their methods replace the previous approach where these responsibilities were scattered across Url, Table, Run, and module-level functions

  • The old functions are still present, but they emit a deprecation warning and then delegate to the new functions

  • Made sure methods that accept project/dataset/table/run path components follow a uniform parameter order: entity name first (e.g. table_name, run_name), then context parameters (dataset_name, project_name, root_url), and also validate those parameters with common logic.

  • Some methods that previously logged warnings and silently ignored conflicting parameters now raise ValueError

  • Deprecated unused utility functions resolve_table_url, get_hash, get_cache_file_name

• [15084] Replaced Table.map() with Table.with_transform() returning a view

  • Introduced a new on-the-fly transform mechanism which is not stateful on Table instances. The new mechanism available through Table.with_transform() returns a TableView, which holds a Table and func which is applied on __getitem__.

  • Made it so collect_metrics accepts not only the new TableView, but anything adhering to a new public protocol MapDataset, which signals the existence of __len__ and __getitem__ (e.g. torch datasets). If a MapDataset, but not a Table or TableView, is provided, a foreign_table_url must be provided to know which table to associate the metrics with.

  • The old Table.map(), Table.map_collect_metrics() and collection_mode are deprecated and emit warnings ahead of removal in 3.0

• [16298] Signal the tlc package is typed with py.typed

• [17288] Added Object Service endpoints that the 3LC Dashboard can use to get self-authenticated URLs for media tags (<img>, <video>, <audio>) that can be used to fetch media directly from cloud storage rather than requiring them to be routed through the Object Service. This allows for distributed downloads and browser memory management.

  • Self-authenticated URLs are valid for a limited time (30 minutes)

  • Media URLs in cloud storage are translated to self-authenticated URLs when possible (AWS S3 presigned URLs, Azure SAS URLs, Google Cloud Storage V4 signed URLs)

  • Media on file systems or that cannot otherwise be accessed still route through the Object Service itself

  • The disable-external-media-urls config option may be used to explicitly disable this functionality for all URLs

  • See this page for more details

Enhancements and Fixes

• [16330] Gave several types new names that follow newly formalized coding conventions. Updated old names to work as aliases for the new names and emit a deprecation warning on usage.

• [14688] Restructured the core object hierarchy to decouple schema/serialization from URL persistence, enabling future lightweight inline-serializable objects that don’t need a URL. This affects types and code that is generally internal, and should have no visible impact on users.

• [17109] Added FutureWarning when COCO detection tables store segmentation data alongside bounding boxes. In 3.0, detection tables will only contain bounding boxes.

• [16383] Relocated and deprecated Table.create_sampler; added standalone sampler factory functions

• [16384] Refactored and deprecated Table.from_hugging_face into from_hugging_face_hub and from_hugging_face_dataset

• [17156] Added deprecation warnings for schema types whose names will change in 3.0

• [17156] Deprecated tlc.core.operations and tlc.client.operations modules ahead of removal in 3.0

• [17486] Deprecated the lightning_module ahead of removal in 3.0; usage should be replaced with approach in example notebook

• [16892] Do not allow errors to raise out of LoggerConfigurator.flush_handlers to avoid unexpected errors at shutdown

• [16535] Ensure extra table indexing locations are scanned for config correctly

• [16968] Show helpful error message when Object Service port is already in use

• [16961] Added copy-to-clipboard support in Object Service TUI