tlcurl.url_adapters.fsspec_url_adapter

Base class for adapters resolving reads/writes to URLs using an implementation of fsspec

Module Contents

Classes

Class

Description

FSSpecUrlAdapterDirEntry

A directory entry for an FSSpecUrlAdapter

FSSpecUrlAdapter

An adapter for resolving reads/writes to URLs using an implementation of fsspec.

Data

Data

Description

logger

API

logger = getLogger(...)
class FSSpecUrlAdapterDirEntry(
ls_info: dict[str, Any],
)

Bases: tlcurl.url_adapter.UrlAdapterDirEntry

A directory entry for an FSSpecUrlAdapter

abstract property name: str
abstract property path: str
is_dir() → bool
is_file() → bool
abstract mtime() → datetime
mtime_datetime() → datetime
class FSSpecUrlAdapter

Bases: tlcurl.url_adapter.UrlAdapterAsyncFromSync

An adapter for resolving reads/writes to URLs using an implementation of fsspec.

The basic workflow for each public method providing access to a URL is to translate the URL to a file system URL key that identifies the file system instance to use when accessing that URL, get or create a file system instance for that URL key, and then use that file system instance to access the URL.

Translation of a URL to a file system URL key is done using _get_fs_url_key. The default implementation is to return the empty string, meaning that all URLs would be mapped to the same file system instance. However, derived classes can override _get_fs_url_key to customize rules for mapping URLs based on e.g. whether they are in the same bucket, whether they require the same credentials, etc.

reset_backend() → None
read_string_content_from_url(
url: Url,
) → str
read_binary_content_from_url(
url: Url,
) → bytes
delete_url(
url: Url,
) → None
make_dirs(
url: Url,
exist_ok: bool = False,
) → None
get_file_size(
url: Url,
) → int
exists(
url: Url,
) → bool
is_dir(
url: Url,
) → bool
is_writable(
url: Url,
) → bool

Checks if a Url is writable.

Actual implementation checks if the bucket is writable.

list_dir(
url: Url,
) → Iterator[FSSpecUrlAdapterDirEntry]
stat(
url: Url,
) → FSSpecUrlAdapterDirEntry

Get status information for a file/directory.

Parameters:

url – The URL of the file/directory to get status for

Returns:

A directory entry object containing file information

Raises:

FileNotFoundError – If the URL does not exist

touch(
url: Url,
) → None

Update the last modified timestamp of a file to the current time. Creates the file if it doesn’t exist.

Parameters:

url – The URL of the file to touch