tlcurl.url_adapters.fsspec_url_adapter

Base class for adapters resolving reads/writes to URLs using an implementation of fsspec

Module Contents

Classes

Class	Description
FSSpecUrlAdapterDirEntry	A directory entry for an FSSpecUrlAdapter
FSSpecUrlAdapter	An adapter for resolving reads/writes to URLs using an implementation of fsspec.

Data

Data	Description
logger

API

tlcurl.url_adapters.fsspec_url_adapter.logger = getLogger(...)

class tlcurl.url_adapters.fsspec_url_adapter.FSSpecUrlAdapterDirEntry(ls_info: dict[str, Any])

Bases: tlcurl.url_adapter.UrlAdapterDirEntry

A directory entry for an FSSpecUrlAdapter

abstract property name: str

abstract property path: str

is_dir() → bool

is_file() → bool

abstract mtime() → datetime.datetime

mtime_datetime() → datetime.datetime

class tlcurl.url_adapters.fsspec_url_adapter.FSSpecUrlAdapter

Bases: tlcurl.url_adapter.UrlAdapterAsyncFromSync

An adapter for resolving reads/writes to URLs using an implementation of fsspec.

The basic workflow for each public method providing access to a URL is to translate the URL to a file system URL key that identifies the file system instance to use when accessing that URL, get or create a file system instance for that URL key, and then use that file system instance to access the URL.

Translation of a URL to a file system URL key is done using _get_fs_url_key. The default implementation is to return the empty string, meaning that all URLs would be mapped to the same file system instance. However, derived classes can override _get_fs_url_key to customize rules for mapping URLs based on e.g. whether they are in the same bucket, whether they require the same credentials, etc.

reset_backend() → None

read_string_content_from_url(url: tlcurl.url.Url) → str

read_binary_content_from_url(url: tlcurl.url.Url) → bytes

delete_url(url: tlcurl.url.Url) → None

make_dirs(url: tlcurl.url.Url, exist_ok: bool = False) → None

get_file_size(url: tlcurl.url.Url) → int

exists(url: tlcurl.url.Url) → bool

is_dir(url: tlcurl.url.Url) → bool

is_writable(url: tlcurl.url.Url) → bool

Checks if a Url is writable.

Actual implementation checks if the bucket is writable.

list_dir(url: tlcurl.url.Url) → collections.abc.Iterator[tlcurl.url_adapters.fsspec_url_adapter.FSSpecUrlAdapterDirEntry]

stat(url: tlcurl.url.Url) → tlcurl.url_adapters.fsspec_url_adapter.FSSpecUrlAdapterDirEntry

Get status information for a file/directory.

Parameters:

url – The URL of the file/directory to get status for

Returns:

A directory entry object containing file information

Raises:

FileNotFoundError – If the URL does not exist

touch(url: tlcurl.url.Url) → None

Update the last modified timestamp of a file to the current time. Creates the file if it doesn’t exist.

Parameters:

url – The URL of the file to touch