tlcurl.url_adapters.fsspec_url_adapter¶

Base class for adapters resolving reads/writes to URLs using an implementation of fsspec

Module Contents¶

Classes¶

Class

Description

FSSpecUrlAdapter

An adapter for resolving reads/writes to URLs using an implementation of fsspec.

FSSpecUrlAdapterDirEntry

A directory entry for an FSSpecUrlAdapter

Data¶

Data

Description

logger

API¶

class FSSpecUrlAdapter¶

Bases: tlcurl.url_adapter.UrlAdapterAsyncFromSync

An adapter for resolving reads/writes to URLs using an implementation of fsspec.

The basic workflow for each public method providing access to a URL is to translate the URL to a file system URL key that identifies the file system instance to use when accessing that URL, get or create a file system instance for that URL key, and then use that file system instance to access the URL.

Translation of a URL to a file system URL key is done using _get_fs_url_key. The default implementation is to return the empty string, meaning that all URLs would be mapped to the same file system instance. However, derived classes can override _get_fs_url_key to customize rules for mapping URLs based on e.g. whether they are in the same bucket, whether they require the same credentials, etc.

delete_url(
url: Url,
) → None¶
exists(
url: Url,
) → bool¶
get_file_size(
url: Url,
) → int¶
is_dir(
url: Url,
) → bool¶
is_writable(
url: Url,
) → bool¶

Checks if a Url is writable.

Actual implementation checks if the bucket is writable.

list_dir(
url: Url,
) → Iterator[FSSpecUrlAdapterDirEntry]¶
make_dirs(
url: Url,
exist_ok: bool = False,
) → None¶
read_binary_content_from_url(
url: Url,
) → bytes¶
read_string_content_from_url(
url: Url,
encoding: str = 'utf-8',
) → str¶
reset_backend() → None¶
stat(
url: Url,
) → FSSpecUrlAdapterDirEntry¶

Get status information for a file/directory.

Parameters:

url – The URL of the file/directory to get status for

Returns:

A directory entry object containing file information

Raises:

FileNotFoundError – If the URL does not exist

touch(
url: Url,
) → None¶

Update the last modified timestamp of a file to the current time. Creates the file if it doesn’t exist.

Parameters:

url – The URL of the file to touch

class FSSpecUrlAdapterDirEntry(
ls_info: dict[str, Any],
)¶

Bases: tlcurl.url_adapter.UrlAdapterDirEntry

A directory entry for an FSSpecUrlAdapter

is_dir() → bool¶
is_file() → bool¶
abstract mtime() → datetime¶
mtime_datetime() → datetime¶
abstract property name: str¶
abstract property path: str¶
logger = getLogger(...)¶