tlc.core.url_adapters.fsspec_url_adapter#

Base class for adapters resolving reads/writes to URLs using an implementation of fsspec

Module Contents#

Classes#

Class

Description

FSSpecUrlAdapterDirEntry

A directory entry for an FSSpecUrlAdapter

FSSpecUrlAdapter

An adapter for resolving reads/writes to URLs using an implementation of fsspec.

API#

class tlc.core.url_adapters.fsspec_url_adapter.FSSpecUrlAdapterDirEntry(ls_info: dict[str, Any])#

Bases: tlc.core.url_adapter.UrlAdapterDirEntry

A directory entry for an FSSpecUrlAdapter

abstract property name: str#
abstract property path: str#
is_dir() → bool#
is_file() → bool#
abstract mtime() → datetime.datetime#
mtime_datetime() → datetime.datetime#
class tlc.core.url_adapters.fsspec_url_adapter.FSSpecUrlAdapter#

Bases: tlc.core.url_adapter.UrlAdapterAsyncFromSync

An adapter for resolving reads/writes to URLs using an implementation of fsspec.

The basic workflow for each public method providing access to a URL is to translate the URL to a file system URL key that identifies the file system instance to use when accessing that URL, get or create a file system instance for that URL key, and then use that file system instance to access the URL.

Translation of a URL to a file system URL key is done using _get_fs_url_key. The default implementation is to return the empty string, meaning that all URLs would be mapped to the same file system instance. However, derived classes can override _get_fs_url_key to customize rules for mapping URLs based on e.g. whether they are in the same bucket, whether they require the same credentials, etc.

reset_backend() → None#
read_string_content_from_url(url: tlc.core.url.Url) → str#
read_binary_content_from_url(url: tlc.core.url.Url) → bytes#
delete_url(url: tlc.core.url.Url) → None#
make_dirs(url: tlc.core.url.Url, exist_ok: bool = False) → None#
get_file_size(url: tlc.core.url.Url) → int#
exists(url: tlc.core.url.Url) → bool#
is_dir(url: tlc.core.url.Url) → bool#
is_writable(url: tlc.core.url.Url) → bool#

Checks if a Url is writable.

Actual implementation checks if the bucket is writable.

list_dir(url: tlc.core.url.Url) → Iterator[tlc.core.url_adapters.fsspec_url_adapter.FSSpecUrlAdapterDirEntry]#