tlc.client.sample_type¶
Implementations of all SampleType classes.
Module Contents¶
Classes¶
Class |
Description |
|---|---|
Base class for all atomic sample types. |
|
A python bool. |
|
A COCO-like list of bounding boxes. |
|
A helper SampleType for making a dict with a single value appear as just the value itself, when provided as a sample. |
|
A categorical label. |
|
Base class for all composite sample types. |
|
A confidence score. |
|
Base class for all dimensional sample types. |
|
A floating point number. |
|
A value which should not be present in the sample. |
|
A list of fixed length and structure. |
|
A tuple of fixed length and structure. |
|
A path to an image file. |
|
A sample type for instance segmentation masks. |
|
A sample type for instance segmentation polygons. |
|
An integer. |
|
An Intersection over Union score. |
|
A |
|
A large numpy array. |
|
A large torch tensor. |
|
A list of variable length. |
|
The fallback SampleType for atomic schemas. |
|
Base class for numeric types |
|
A PIL Image. |
|
A string representing a path. |
|
Dimensional sample types whose row representation is a list. |
|
Base class for referenced atomic sample types. |
|
The base class for all sample types. |
|
A path to a semantic segmentation mask image. |
|
A torch tensor representing a segmentation mask. |
|
A single-channel PIL-image containing a segmentation mask. |
|
A sample type for handling the concept of a pseudo-scalar; a scalar which has a size0 with min=max=1. |
|
A small bytes object. |
|
A small numpy array. |
|
A small torch tensor. |
|
A python string. |
|
A dict with string keys. |
|
A base class for atomic sample types whose row representation is the same as their sample representation. |
|
A tuple of variable length. |
Functions¶
Function |
Description |
|---|---|
Create a Bytes SampleType. |
|
Create a NumpyArray SampleType. |
|
Create a TorchTensor SampleType. |
|
A decorator for registering a SampleType. |
Data¶
API¶
- class AtomicSampleType(
- name: str,
Bases:
tlc.client.sample_type.SampleType[tlc.client.sample_type.ST,tlc.client.sample_type.RT]Base class for all atomic sample types.
Atomic sample types are sample types that contain a single value. They are used to represent atomic data structures like ints, floats, strings, and images. Atomic sample types have the
valueattribute, which is aScalarValuerequired to create aSchemafor the SampleType.Subclasses of AtomicSampleType only need to implement the
sample_from_rowandrow_from_samplemethods, which define how the value should be converted to and from a row, as well as thevalueproperty.- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- abstract row_from_sample(
- sample: Any,
- abstract sample_from_row(
- row: Any,
- abstract property value: ScalarValue¶
A
ScalarValuerepresenting the SampleType.- Returns:
The
ScalarValueobject.
- class Bool(
- name: str,
Bases:
tlc.client.sample_type.TrivialAtomicSampleType[bool]A python bool.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- sample_type = bool¶
- class BoundingBoxList(
- name: str,
- format: Literal[xywh, xyxy] = 'xyxy',
- normalized: bool = False,
- classes: list[str] | dict[int, str] | dict[float, str] | dict[float, MapElement] | None = None,
Bases:
tlc.client.sample_type.StringKeyDictA COCO-like list of bounding boxes.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- class Box(
- child: tlc.client.sample_type._SampleTypeStructure,
- hidden_children: list[tlc.client.sample_type._SampleTypeStructure] | None = None,
Bases:
tlc.client.sample_type.CompositeSampleType[object]A helper SampleType for making a dict with a single value appear as just the value itself, when provided as a sample.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- sample_type = box¶
- Bytes( ) LargeBytes | SmallBytes¶
Create a Bytes SampleType.
If the maximum size of the bytes is less than or equal to a cutoff of 1,000 bytes, a SmallBytes is created. Otherwise, a LargeBytes is created.
- Parameters:
name – The name of the Bytes SampleType.
max_size – The maximum size of the bytes object (in number of bytes).
- Returns:
The created SampleType.
- class CategoricalLabel(
- name: str,
- classes: Sequence[str] | Sequence[dict[str, str]] | Sequence[MapElement] | dict[float, str] | dict[int, str] | dict[float, MapElement] | dict[int, MapElement],
Bases:
tlc.client.sample_type.TrivialAtomicSampleType[int]A categorical label.
Categorical labels are represented as ints, with the mapping from ints to class names defined by the
classesattribute, a list of strings.Create a CategoricalLabel.
- Parameters:
name – The name of the CategoricalLabel.
classes – The classes of the CategoricalLabel.
- sample_type = categorical_label¶
- property value: Int32Value¶
- class CompositeSampleType(
- name: str,
- children: list[SampleType],
Bases:
tlc.client.sample_type.SampleType[tlc.client.sample_type.ST,dict[str,object]],typing.Generic[tlc.client.sample_type.ST]Base class for all composite sample types.
Composite sample types are sample types that contain other sample types. They are used to represent composite data structures like lists, tuples, and dicts. Composite sample types have the
childrenattribute, which is a list of the sample types that they contain.Subclasses of CompositeSampleType only need to implement the
sample_from_rowandrow_from_samplemethods, which define how the children should be composed into a single sample or row.- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- class Confidence(
- name: str,
Bases:
tlc.client.sample_type.FloatA confidence score.
- Parameters:
precision – The precision of the float, in bits. Must be one of [32, 64].
normalized – Whether the value of the float is normalized between 0 and 1.
number_role – The number role of the float. This determines how the float will be displayed in the Dashboard.
- class DimensionalSampleType(
- content: tlc.client.sample_type._SampleTypeStructure,
Bases:
tlc.client.sample_type.SampleType[tlc.client.sample_type.ST,tlc.client.sample_type.RT]Base class for all dimensional sample types.
Dimensional sample types describe how a sample can be extended along a dimension. They are used to represent composite data structures whose size might vary between samples in a dataset. Dimensional sample types have the
contentattribute, which is a sample type that describes the structure of the samples along the dimension.Subclasses of DimensionalSampleType only need to implement the
sample_from_rowandrow_from_samplemethods, which define how the content should be composed into a single sample or row.The basic initializer for all DimensionalSampleType objects.
- Parameters:
content – The sample type that describes the structure of the samples along the dimension.
- abstract row_from_sample(
- sample: Any,
- abstract sample_from_row(
- row: Any,
- class Float( )¶
Bases:
tlc.client.sample_type.Number[float]A floating point number.
- Parameters:
precision – The precision of the float, in bits. Must be one of [32, 64].
normalized – Whether the value of the float is normalized between 0 and 1.
number_role – The number role of the float. This determines how the float will be displayed in the Dashboard.
- sample_type = float¶
- property value: Float32Value | Float64Value¶
- class Hidden( )¶
Bases:
tlc.client.sample_type.SampleType[object,object]A value which should not be present in the sample.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- ensure_sample_valid(
- sample: object,
Hidden values will not be present in the ‘sample view’, but this function never raises in order to allow users to write values with a “hidden” sample type in their schema to a Table using TableWriter.
- row_from_sample(
- sample: object,
While the ‘sample view’ of a Hidden sample normally does not exist, this is a useful workaround for writing values with a “hidden” sample type in their schema to a Table using TableWriter
- sample_type = hidden¶
- class HorizontalList(
- name: str,
- children: list[SampleType],
Bases:
tlc.client.sample_type.CompositeSampleType[list]A list of fixed length and structure.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- sample_type = horizontal_list¶
- class HorizontalTuple(
- name: str,
- children: list[SampleType],
Bases:
tlc.client.sample_type.CompositeSampleType[tuple]A tuple of fixed length and structure.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- sample_type = horizontal_tuple¶
- class ImagePath(
- name: str,
Bases:
tlc.client.sample_type.PathA path to an image file.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- sample_type = image_path¶
- property value: StringValue¶
- class InstanceSegmentationMasks(
- name: str,
- instance_properties_structure: dict[str, dict | tlc.client.sample_type._SampleTypeStructure] | None,
- is_prediction: bool = False,
Bases:
tlc.client.sample_type._InstanceSegmentationA sample type for instance segmentation masks.
A sample of this type is a dictionary with the fields defined by the TypedDict
SegmentationMasksDict.- Parameters:
name – The name of the sample type.
instance_properties_structure – The structure of the instance properties.
is_prediction – Whether the instance segmentation is a prediction.
- create_rles_from_sample(
- sample: SegmentationMasksDict,
- sample_from_row(
- row: tlc.core.data_formats.segmentation._InternalInstanceSegmentationDict,
- sample_type = instance_segmentation_masks¶
- class InstanceSegmentationPolygons(
- name: str,
- instance_properties_structure: dict[str, dict | tlc.client.sample_type._SampleTypeStructure],
- relative: bool = False,
- is_prediction: bool = False,
Bases:
tlc.client.sample_type._InstanceSegmentationA sample type for instance segmentation polygons.
A sample of this type is a dictionary with the fields defined by the TypedDict
SegmentationPolygonsDict.- Parameters:
name – The name of the sample type.
instance_properties_structure – The structure of the instance properties.
relative – Whether the polygons are relative to the image size.
is_prediction – Whether the instance segmentation is a prediction.
- create_rles_from_sample(
- sample: SegmentationPolygonsDict,
- sample_from_row(
- row: tlc.core.data_formats.segmentation._InternalInstanceSegmentationDict,
- sample_type = instance_segmentation_polygons¶
- class Int( )¶
Bases:
tlc.client.sample_type.Number[int]An integer.
- Parameters:
precision – The precision of the integer, in bits. Must be one of [8, 16, 32, 64].
signed – Whether the value of the integer can be negative.
number_role – The number role of the integer. This determines how the integer will be displayed in the Dashboard.
- sample_type = int¶
- property value: tlc.client.sample_type._IntegerValue¶
- class IoU(
- name: str,
Bases:
tlc.client.sample_type.FloatAn Intersection over Union score.
- Parameters:
precision – The precision of the float, in bits. Must be one of [32, 64].
normalized – Whether the value of the float is normalized between 0 and 1.
number_role – The number role of the float. This determines how the float will be displayed in the Dashboard.
- class LargeBytes(
- name: str,
Bases:
tlc.client.sample_type.ReferencedAtomicSampleType[bytes]A
bytesobject.Bytes objects with this sample type will, as opposed to
SmallBytes, be stored in a table as references to files. This is useful for large binary data.- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- extension = bin¶
- sample_type = large_bytes¶
- property value: UrlStringValue¶
- class LargeNumpyArray(
- name: str,
Bases:
tlc.client.sample_type._LargeTensor[numpy.ndarray],tlc.client.sample_type._NumpyDataTypeMixinA large numpy array.
Numpy arrays with this sample type will, as opposed to
SmallNumpyArray, be stored in a table as references to files on disk. This is useful for large arrays that would be inefficient to store in the table itself. Note that you will not be able to view or edit individual elements of the array in the 3LC Dashboard when using this sample type.- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- extension = npy¶
- sample_type = large_numpy_array¶
- class LargeTorchTensor(
- content: SampleType,
Bases:
tlc.client.sample_type._LargeTensor[torch.Tensor],tlc.client.sample_type._TorchDataTypeMixinA large torch tensor.
Torch tensors with this sample type will, as opposed to
SmallTorchTensor, be stored in a table as references to files on disk. This is useful for large tensors that would be inefficient to store in the table itself. Note that you will not be able to view or edit individual elements of the tensor in the 3LC Dashboard when using this sample type.- extension = pt¶
- sample_type = large_torch_tensor¶
- class List(
- content: tlc.client.sample_type._SampleTypeStructure,
- min_size: int = 0,
- max_size: int | None = None,
Bases:
tlc.client.sample_type.PythonContainer[list]A list of variable length.
- Parameters:
content – The sample type that describes the structure of the samples along the dimension.
min_size – The minimum size of the container.
max_size – The maximum size of the container.
- sample_type = list¶
- class NoOpSampleType(
- name: str,
- value: ScalarValue,
Bases:
tlc.client.sample_type.TrivialAtomicSampleType[object]The fallback SampleType for atomic schemas.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- sample_type = <Multiline-String>¶
- property value: ScalarValue¶
- class Number( )¶
Bases:
tlc.client.sample_type.TrivialAtomicSampleType[tlc.client.sample_type.ST]Base class for numeric types
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- NumpyArray(
- shape: tuple[int, ...] | int,
- content: SampleType,
Create a NumpyArray SampleType.
If the number of elements in the array is less than or equal to a cutoff of 1000, a SmallNumpyArray is created. Otherwise, a LargeNumpyArray is created.
- Parameters:
shape – The shape of the array.
content – A SampleType representing the content of the array. A numpy array with
dtype=f32, for instance, would have a content oftlc.Float("<Array Name>", precision=32).
- Returns:
The created SampleType.
- class PILImage(
- name: str,
Bases:
tlc.client.sample_type.ReferencedAtomicSampleType[PIL.Image.Image]A PIL Image.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- extension = png¶
- sample_type = PILImage¶
- property value: StringValue¶
- class Path(
- name: str,
Bases:
tlc.client.sample_type.AtomicSampleType[str,str]A string representing a path.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- property value: StringValue¶
- class PythonContainer(
- content: tlc.client.sample_type._SampleTypeStructure,
- min_size: int = 0,
- max_size: int | None = None,
Bases:
tlc.client.sample_type.DimensionalSampleType[tlc.client.sample_type._ContainerST,list]Dimensional sample types whose row representation is a list.
- Parameters:
content – The sample type that describes the structure of the samples along the dimension.
min_size – The minimum size of the container.
max_size – The maximum size of the container.
- RT = TypeVar(...)¶
Generic row type
- class ReferencedAtomicSampleType(
- name: str,
Bases:
tlc.client.sample_type.AtomicSampleType[tlc.client.sample_type.ST,str]Base class for referenced atomic sample types.
These are samples whose row value is a reference to a file, but whose sample value is the file content.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- ST = TypeVar(...)¶
Generic sample type
- class SampleType(
- name: str,
Bases:
abc.ABC,typing.Generic[tlc.client.sample_type.ST,tlc.client.sample_type.RT]The base class for all sample types.
A
SampleTypedefines the type of a single sample. It can be used to create aSchemafor aTable, and convert samples between their ML ‘sample’ representation and their Table ‘row’ representation. SampleType objects are structured like trees, with composite types (e.g. lists, tuples, dicts) as internal nodes and atomic types (e.g. ints, floats, strings) as leaves.SampleType objects can be created in three main ways:
From a
Schemaobject, usingSampleType.from_schema(schema).From a sample, using
SampleType.from_sample(sample).From a ‘structure’, a simple declarative description of the structure of a single sample, using
SampleType.from_structure(structure).
If you have custom types that you want to use in your
Table, you can create aSampleTypefor them by subclassing the appropriate subclass ofSampleTypeand registering it with theregister_sample_typedecorator.- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- ensure_row_valid(
- row: object,
Raises a ValueError if the row does not match this SampleType. Does nothing otherwise.
- Parameters:
row – The row to validate.
- Raises:
ValueError – If the row does not match this SampleType.
- ensure_sample_valid(
- sample: object,
Raises a ValueError if the sample does not match this SampleType. Does nothing otherwise.
- Parameters:
sample – The sample to validate.
- Raises:
ValueError – If the sample does not match this SampleType.
- classmethod from_sample( ) SampleType¶
Create a SampleType from a sample.
This method is used to create a SampleType when creating a Table from a PyTorch Dataset, and the user has not specified a SampleType or a structure. Since a SampleType is needed to convert a sample to a row, the first sample of the dataset is used as a reference to create the SampleType.
- Parameters:
sample – The sample to create the SampleType from.
name – The name of the SampleType.
all_arrays_are_fixed_size – If True, all arrays will be treated as fixed size arrays.
- Returns:
The created SampleType.
- classmethod from_schema( ) SampleType¶
Create a SampleType from a Schema.
- Parameters:
schema – The Schema to create the SampleType from.
name – An optional name for the NoOpSampleType fallback.
- Returns:
The created SampleType.
- classmethod from_structure(
- structure: tlc.client.sample_type._SampleTypeStructure,
- name: str = 'value',
Create a SampleType from a structure.
A structure is a simple declarative description of the structure of a single sample. Instead of initializing SampleType objects for composite sample types, structures can be represented as nested lists, tuples, or dicts containing either SampleType objects, or simply a subclass of SampleType where you don’t care about the names of the columns.
E.g.
((Int, Int), String)is equivalent to:HorizontalTuple("value", [ HorizontalTuple("value_0", [Int("value_0_0"), Int("value_0_1")]), String("value_1") ] )
- Parameters:
structure – The structure to create the SampleType from.
name – The name of the SampleType.
- Returns:
The created SampleType.
- rename(
- name: str,
Rename the SampleType.
- Parameters:
name – The new name of the SampleType.
- abstract row_from_sample(
- sample: Any,
Convert a sample to a row using the SampleType.
- Parameters:
sample – The sample to convert.
- Returns:
The converted row.
- abstract sample_from_row(
- row: Any,
Convert a row to a sample using the SampleType.
- Parameters:
row – The row to convert.
- Returns:
The converted sample.
- class SegmentationImagePath(
- name: str,
- classes: Sequence[str] | Sequence[dict[str, str]] | Sequence[MapElement] | dict[float, str] | dict[int, str] | dict[float, MapElement] | dict[int, MapElement],
Bases:
tlc.client.sample_type.PathA path to a semantic segmentation mask image.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- sample_type = segmentation_image_path¶
- property value: SegmentationMaskUrlStringValue¶
- class SegmentationMask(
- name: str,
- classes: Sequence[str] | Sequence[dict[str, str]] | Sequence[MapElement] | dict[float, str] | dict[int, str] | dict[float, MapElement] | dict[int, MapElement],
Bases:
tlc.client.sample_type.AtomicSampleType[torch.Tensor,str]A torch tensor representing a segmentation mask.
The tensor is expected to have shape (H, W) and contain integer values representing the class of each pixel.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- sample_type = segmentation_mask¶
- property value: StringValue¶
- class SegmentationPILImage(
- name: str,
- classes: Sequence[str] | Sequence[dict[str, str]] | Sequence[MapElement] | dict[float, str] | dict[int, str] | dict[float, MapElement] | dict[int, MapElement],
Bases:
tlc.client.sample_type.PILImageA single-channel PIL-image containing a segmentation mask.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- sample_type = segmentation_PILImage¶
- property value: SegmentationMaskUrlStringValue¶
- class Singleton(
- content: tlc.client.sample_type._SampleTypeStructure,
Bases:
tlc.client.sample_type.DimensionalSampleType[object,object]A sample type for handling the concept of a pseudo-scalar; a scalar which has a size0 with min=max=1.
This value should appear in both its sample and row representations as the scalar itself, rather than a list containing the scalar.
The basic initializer for all DimensionalSampleType objects.
- Parameters:
content – The sample type that describes the structure of the samples along the dimension.
- sample_type = singleton¶
- class SmallBytes(
- name: str,
Bases:
tlc.client.sample_type.DimensionalSampleType[bytes,bytes]A small bytes object.
Bytes objects with this sample type will be stored in the table itself. This is useful for small binary data.
The basic initializer for all DimensionalSampleType objects.
- Parameters:
content – The sample type that describes the structure of the samples along the dimension.
- sample_type = small_bytes¶
- class SmallNumpyArray( )¶
Bases:
tlc.client.sample_type._SmallTensor[numpy.ndarray],tlc.client.sample_type._NumpyDataTypeMixinA small numpy array.
Numpy arrays with this sample type will be stored in a table as a list of lists. This is useful for small arrays that can be efficiently stored in the table itself. Unlike
LargeNumpyArray, you will be able to view and edit individual elements of the array in the 3LC Dashboard when using this sample type.- Parameters:
shape – The shape of the array.
content – The sample type that describes the structure of one element in the array.
- sample_type = small_numpy_array¶
- class SmallTorchTensor( )¶
Bases:
tlc.client.sample_type._SmallTensor[torch.Tensor],tlc.client.sample_type._TorchDataTypeMixinA small torch tensor.
Torch tensors with this sample type will be stored in a table as a list of lists. This is useful for small tensors that can be efficiently stored in the table itself. Unlike
LargeTorchTensor, you will be able to view and edit individual elements of the tensor in the 3LC Dashboard when using this sample type.- Parameters:
shape – The shape of the tensor.
content – The sample type that describes the structure of one element in the tensor.
- sample_type = small_torch_tensor¶
- class String( )¶
Bases:
tlc.client.sample_type.TrivialAtomicSampleType[str]A python string.
- Parameters:
string_role – The string role of the string. This determines how the string will be displayed in the Dashboard.
- sample_type = string¶
- property value: StringValue¶
- class StringKeyDict(
- name: str,
- children: list[SampleType],
Bases:
tlc.client.sample_type.CompositeSampleType[dict[str,object]]A dict with string keys.
- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- sample_type = <Multiline-String>¶
- TorchTensor(
- shape: tuple[int, ...] | int,
- content: SampleType,
Create a TorchTensor SampleType.
If the number of elements in the tensor is less than or equal to a cutoff of 1000, a SmallTorchTensor is created. Otherwise, a LargeTorchTensor is created.
- Parameters:
shape – The shape of the tensor.
content – A SampleType representing the content of the tensor. A torch tensor with
dtype=f32, for instance, would have a content oftlc.Float("<Array Name>", precision=32).
- Returns:
The created SampleType.
- class TrivialAtomicSampleType(
- name: str,
Bases:
tlc.client.sample_type.AtomicSampleType[tlc.client.sample_type.ST,tlc.client.sample_type.ST]A base class for atomic sample types whose row representation is the same as their sample representation.
Subclasses of TrivialAtomicSampleType only need to implement the
valueproperty.- Parameters:
name – The name of the SampleType. This will be used as the column name in the Table.
- row_from_sample(
- sample: Any,
- sample_from_row(
- row: Any,
- class Tuple(
- content: tlc.client.sample_type._SampleTypeStructure,
- min_size: int = 0,
- max_size: int | None = None,
Bases:
tlc.client.sample_type.PythonContainer[tuple]A tuple of variable length.
- Parameters:
content – The sample type that describes the structure of the samples along the dimension.
min_size – The minimum size of the container.
max_size – The maximum size of the container.
- sample_type = tuple¶
- register_sample_type(
- sample_type: type[tlc.client.sample_type._SampleTypeSubclass],
A decorator for registering a SampleType.
Custom sample types must be registered with this decorator in order to be created from a Schema.
- Parameters:
sample_type – The SampleType to register.
- Returns:
The registered SampleType.