tlc.core.schema_helper#

Helper methods for working with Schemas

Module Contents#

Classes#

Class

Description

SchemaHelper

A class with helper methods for working with Schema objects

API#

class tlc.core.schema_helper.SchemaHelper#

A class with helper methods for working with Schema objects

static object_input_urls(obj: Any, schema: tlc.core.schema.Schema) list[tlc.core.url.Url]#

Returns a list of all URLs referenced by this object, from scalar strings or lists of strings

Note: the result is likely to be relative wrt the object’s URL

static from_pyarrow_datatype(data_type: pyarrow.DataType) type[tlc.core.schema.ScalarValue]#

Converts a pyarrow Schema or DataType to a schema or scalar value.

Returns the type of the scalar value that corresponds to the pyarrow data type.

static to_pyarrow_datatype(schema_or_value: tlc.core.schema.Schema | tlc.core.schema.ScalarValue) pyarrow.DataType#

Converts a schema or scalar value to a pyarrow datatype.

Currently supports scalar types, lists of scalar types, structs, and lists of structs.

static cast_scalar(value: Any, value_type: tlc.core.schema.ScalarValue) Any#

Cast a value which is a ScalarValue into its corresponding python type.

static cast_value(value: Any, value_schema: tlc.core.schema.Schema) Any#

Cast any value into its corresponding python type based on the Schema.

static default_scalar(value_type: tlc.core.schema.ScalarValue) Any#

Returns the default value for a ScalarValue.

static default_value(schema: tlc.core.schema.Schema) Any#

Returns the default value for a schema.

A schema holds either:

  • a ScalarValue (schema.value) which corresponds to a scalar type (potentially an array of scalars)

  • a dict of sub-Schemas (schema.values) corresponding compound types (potentially an array)

static is_computable(schema: tlc.core.schema.Schema) bool#

Returns True if the schema is computable.

static add_schema_to_existing_schema_at_location(added_schema: tlc.core.schema.Schema, existing_schema: tlc.core.schema.Schema, location: list[str]) None#

Adds the value to the schema at the given location.

static is_pseudo_scalar(schema: tlc.core.schema.Schema) bool#

Returns True if the schema is a pseudo-scalar.

When a schema has a size0 with min=1 and max=1, it is considered a pseudo-scalar. This is a trick we use when unrolling/rolling up tables. We want to treat table cells with 1-element lists as scalars.

static get_nested_schema(schema: tlc.core.schema.Schema, path: str) tlc.core.schema.Schema | None#

Retrieves a nested schema from a schema.

Parameters:
  • schema – The schema to retrieve the nested schema from.

  • path – The (dot-separated) path to the nested schema.

Returns:

The nested schema, or None if the path doesn’t exist.

static create_sparse_schema_from_scalar_value(path: str, scalar_value: tlc.core.schema.ScalarValue) tlc.core.schema.Schema#

Creates a sparse schema from a path and a schema.

Parameters:
  • path – The (dot-separated) path to the nested schema.

  • new_schema – The schema to create the sparse schema from.

Returns:

The sparse schema.

static url_values(schema: tlc.core.schema.Schema) list[str]#

Return a list of sub-schemas that represent atomic URL values.

Parameters:

schema – The schema to retrieve the URL values from.

Returns:

A list of sub-value keys corresponding to URL values.