tlc.client.utils

Module Contents

Classes

Class	Description
SubsetSequentialSampler	Samples elements sequentially from a given list of indices.
RangeSampler	Samples elements sequentially from a range
RepeatByWeightSampler	Repeats elements based on their weight.
StandardizedTransforms	A callable class that wraps transforms to take the whole sample as its only argument, rather than destructuring it.

Functions

Function	Description
bytes2str
str2bytes
take
batched_iterator
without_transforms	Ensures that, if the dataset is a Torchvision dataset, its transforms are temporarily removed.
relativize_with_max_depth	Relativize the given URL with respect to the given owner URL, up to a maximum depth.
standardized_transforms	Create a new transforms function which takes the whole sample as its only argument, rather than destructuring it.
get_column_from_pyarrow_table	Return a the specified column of the table as a pyarrow table.

API

bytes2str(

obj: bytes,

) → str

str2bytes(

s: str,

) → bytes

take(

iterator: Iterator,

batch_size: int,

) → list

batched_iterator(

iterator: Iterable,

batch_size: int,

) → Iterator[list]

class SubsetSequentialSampler(

indices: list[int],

)

Bases: torch.utils.data.sampler.Sampler[int]

Samples elements sequentially from a given list of indices.

class RangeSampler(

end: int,

start: int = 0,

step: int = 1,

)

Bases: torch.utils.data.sampler.Sampler[int]

Samples elements sequentially from a range

class RepeatByWeightSampler(

weights: list[float],

shuffle: bool = True,

random_state: Random | None = None,

)

Bases: torch.utils.data.sampler.Sampler[int]

Repeats elements based on their weight.

without_transforms(

dataset: Dataset,

) → Generator[Callable | None, None, None]

Ensures that, if the dataset is a Torchvision dataset, its transforms are temporarily removed.

Parameters:

dataset – The dataset to temporarily remove transforms from.

relativize_with_max_depth(

url: Url,

owner: Url,

max_depth: int,

) → Url

Relativize the given URL with respect to the given owner URL, up to a maximum depth.

Deprecated: Use Url.to_relative_with_max_depth instead.

class StandardizedTransforms(

transforms: Callable[..., Any],

)

A callable class that wraps transforms to take the whole sample as its only argument, rather than destructuring it.

standardized_transforms(

transforms: Callable[..., Any],

) → Callable[[Any], Any]

Create a new transforms function which takes the whole sample as its only argument, rather than destructuring it.

Parameters:

transforms – The transforms function to standardize.

Returns:

The standardized transforms function.

get_column_from_pyarrow_table(

table: Table,

name: str,

combine_chunks: bool = True,

) → Array | ChunkedArray

Return a the specified column of the table as a pyarrow table.

To get nested sub-columns, use dot notation. E.g. ‘column.sub_column’. The values in the column will be the row-view of the table. A column which is a PIL image in its sample-view, for instance, will be returned as a column of strings.

Parameters:

• name – The name of the column to get.

• combine_chunks – Whether to combine the chunks of the returned column in the case that it is a ChunkedArray. Defaults to True.

Returns:

A pyarrow array containing the specified column.

Raises:

KeyError – If the column does not exist in the table.