`tlcurl.url_adapter`¶

Provides a unified interface for URL-based file operations across different schemes and storage backends.

Module Contents¶

Classes¶

Class	Description
`IfExistsOption`	An Enum indicating what to do if content already exists at the target URL
`UrlAdapterDirEntry`	The base class for all directory entries.
`UrlAdapter`	The base class for all URL adapters
`UrlAdapterAsyncFromSync`	A UrlAdapter where the Async methods are implemented as Futures that call the equivalent Sync methods on the thread pool
`UrlAdapterSyncFromAsync`	A UrlAdapter where the Sync methods are implemented by calling the equivalent Async methods and then waiting on the Futures returned.

Data¶

Data	Description
`logger`

API¶

tlcurl.url_adapter.logger = getLogger(...)¶

class tlcurl.url_adapter.IfExistsOption¶

Bases: str, enum.Enum

An Enum indicating what to do if content already exists at the target URL

from tlc.core.url_adapter import IfExistsOption, UrlAdapter
from tlc.core.url import Url

url = Url("s3://my-bucket/my-file.txt")
adapter = UrlAdapter.get_adapter(url)
adapter.write_string_content_to_url(url, "Hello World!", IfExistsOption.OVERWRITE)

Initialize self. See help(type(self)) for accurate signature.

OVERWRITE = overwrite¶: If content already exists at the target URL, overwrite the content (default).

RENAME = rename¶: If content already exists at the target URL, leave that content, and write the new content to a new URL that is similar to the original target URL.

RAISE = raise¶: If content already exists at the target URL, raise an exception.

REUSE = reuse¶: If content already exists at the target URL, reuse the existing content.

class tlcurl.url_adapter.UrlAdapterDirEntry¶

The base class for all directory entries.

abstract property name: str¶: The entry’s base filename

abstract property path: str¶: The entry’s full path name

abstract is_dir() → bool¶: Return True if this entry is a directory

abstract is_file() → bool¶: Return True if this entry is a file

abstract mtime() → Any¶

Return the modification time object for this entry

The mtime object is not specified to be of any particular type, but must be comparable to other mtime objects of the same type

May raise exceptions when accessing the resources

abstract mtime_datetime() → datetime.datetime¶: Return the modification time for this entry as a datetime object

class tlcurl.url_adapter.UrlAdapter¶

Bases: abc.ABC

The base class for all URL adapters

abstract schemes() → list[tlcurl.url.Scheme]¶: Get URL schemes

abstract read_string_content_from_url(url: tlcurl.url.Url) → str¶: Read content from URL synchronously, dispatch

abstract read_string_content_from_url_async(url: tlcurl.url.Url) → concurrent.futures.Future¶: Read content from URL asynchronously

abstract read_binary_content_from_url(url: tlcurl.url.Url) → bytes¶: Read binary content from URL synchronously, dispatch

abstract read_binary_content_from_url_async(url: tlcurl.url.Url) → concurrent.futures.Future¶: Read binary content from URL asynchronously

write_string_content_to_url(url: tlcurl.url.Url, content: str, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → tlcurl.url.Url¶: Write content to URL synchronously

abstract write_string_content_to_url_async(url: tlcurl.url.Url, content: str, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → concurrent.futures.Future¶: Write content to URL asynchronously

write_binary_content_to_url(url: tlcurl.url.Url, content: bytes, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → tlcurl.url.Url¶

Write binary content to URL synchronously

Handles write options and dispatches to _write_binary_content_to_url

abstract write_binary_content_to_url_async(url: tlcurl.url.Url, content: bytes, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → concurrent.futures.Future¶: Write binary content to URL asynchronously

copy_url(source: tlcurl.url.Url, destination: tlcurl.url.Url, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → tlcurl.url.Url¶: Copy URL synchronously

abstract copy_url_async(source: tlcurl.url.Url, destination: tlcurl.url.Url, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → concurrent.futures.Future¶: Copy URL asynchronously

abstract delete_url(url: tlcurl.url.Url) → None¶: Delete URL synchronously

abstract delete_url_async(url: tlcurl.url.Url) → concurrent.futures.Future¶: Delete URL asynchronously

abstract make_dirs(url: tlcurl.url.Url, exist_ok: bool = False) → None¶

Create a leaf directory and all intermediate ones synchronously

For flat file hierarchies, this may be a no-op, see is_file_hierarchy_flat.

abstract make_dirs_async(url: tlcurl.url.Url, exist_ok: bool = False) → concurrent.futures.Future¶

Create a leaf directory and all intermediate ones asynchronously

For flat file hierarchies, this may be a no-op, see is_file_hierarchy_flat.

abstract get_file_size(url: tlcurl.url.Url) → int¶: Get the size of the file at the given URL synchronously

abstract get_file_size_async(url: tlcurl.url.Url) → concurrent.futures.Future¶: Get the size of the file at the given URL asynchronously

abstract list_dir(url: tlcurl.url.Url) → collections.abc.Iterator[tlcurl.url_adapter.UrlAdapterDirEntry]¶: List the entries belonging to the directory at the given URL synchronously

abstract list_dir_async(url: tlcurl.url.Url) → concurrent.futures.Future¶: List the entries belonging to the directory at the given URL asynchronously

abstract exists(url: tlcurl.url.Url) → bool¶: Return True if the given URL refers to an existing path synchronously

abstract exists_async(url: tlcurl.url.Url) → concurrent.futures.Future¶: Return True if the given URL refers to an existing path asynchronously

abstract is_dir(url: tlcurl.url.Url) → bool¶: Return True if the given URL refers to a directory synchronously

abstract is_dir_async(url: tlcurl.url.Url) → concurrent.futures.Future¶: Return True if the given URL refers to a directory asynchronously

abstract is_writable(url: tlcurl.url.Url) → bool¶: Return True if the given URL is writable

abstract is_writable_async(url: tlcurl.url.Url) → concurrent.futures.Future¶: Return True if the given URL is writable asynchronously

is_file_hierarchy_flat() → bool¶

Determine if the file hierarchy is flat for this adapter

Cloud storage normally has a flat hierarchy, meaning that make_dirs is a no-op.

abstract stat(url: tlcurl.url.Url) → tlcurl.url_adapter.UrlAdapterDirEntry¶

Get metadata about a file or directory at the given URL synchronously: param url: The URL to get metadata for; return: A UrlAdapterDirEntry containing metadata about the file or directory; raises FileNotFoundError: If the URL does not exist

abstract stat_async(url: tlcurl.url.Url) → concurrent.futures.Future¶

Get metadata about a file or directory at the given URL asynchronously: param url: The URL to get metadata for; return: A Future that resolves to a UrlAdapterDirEntry containing metadata; raises FileNotFoundError: If the URL does not exist

abstract touch(url: tlcurl.url.Url) → None¶

Update the last modified timestamp of a file to the current time. Creates the file if it doesn’t exist.

Parameters:: url – The URL of the file to touch

class tlcurl.url_adapter.UrlAdapterAsyncFromSync¶

Bases: tlcurl.url_adapter.UrlAdapter

A UrlAdapter where the Async methods are implemented as Futures that call the equivalent Sync methods on the thread pool

Derived classes must implement the Sync methods.

read_string_content_from_url_async(url: tlcurl.url.Url) → concurrent.futures.Future¶

read_binary_content_from_url_async(url: tlcurl.url.Url) → concurrent.futures.Future¶

write_string_content_to_url_async(url: tlcurl.url.Url, content: str, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → concurrent.futures.Future¶

write_binary_content_to_url_async(url: tlcurl.url.Url, content: bytes, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → concurrent.futures.Future¶

delete_url_async(url: tlcurl.url.Url) → concurrent.futures.Future¶

copy_url_async(source: tlcurl.url.Url, destination: tlcurl.url.Url, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → concurrent.futures.Future¶

make_dirs_async(url: tlcurl.url.Url, exist_ok: bool = False) → concurrent.futures.Future¶

get_file_size_async(url: tlcurl.url.Url) → concurrent.futures.Future¶

list_dir_async(url: tlcurl.url.Url) → concurrent.futures.Future¶

exists_async(url: tlcurl.url.Url) → concurrent.futures.Future¶

is_dir_async(url: tlcurl.url.Url) → concurrent.futures.Future¶

is_writable_async(url: tlcurl.url.Url) → concurrent.futures.Future¶

stat_async(url: tlcurl.url.Url) → concurrent.futures.Future¶

class tlcurl.url_adapter.UrlAdapterSyncFromAsync¶

Bases: tlcurl.url_adapter.UrlAdapter

A UrlAdapter where the Sync methods are implemented by calling the equivalent Async methods and then waiting on the Futures returned.

Derived classes must implement the Async methods.

read_string_content_from_url(url: tlcurl.url.Url) → str¶

read_binary_content_from_url(url: tlcurl.url.Url) → bytes¶

write_string_content_to_url(url: tlcurl.url.Url, content: str, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → tlcurl.url.Url¶

write_binary_content_to_url(url: tlcurl.url.Url, content: bytes, options: tlcurl.url_adapter.IfExistsOption = IfExistsOption.OVERWRITE) → tlcurl.url.Url¶

delete_url(url: tlcurl.url.Url) → None¶

make_dirs(url: tlcurl.url.Url, exist_ok: bool = False) → None¶

get_file_size(url: tlcurl.url.Url) → int¶

list_dir(url: tlcurl.url.Url) → collections.abc.Iterator[tlcurl.url_adapter.UrlAdapterDirEntry]¶

exists(url: tlcurl.url.Url) → bool¶

stat(url: tlcurl.url.Url) → tlcurl.url_adapter.UrlAdapterDirEntry¶

tlcurl.url_adapter¶

Module Contents¶

Classes¶

Data¶

API¶

`tlcurl.url_adapter`¶