Working with Geometry in 3LC

Computer vision annotations often consist of spatial data that describe shapes or regions of interest. These may include axis-aligned or oriented bounding boxes, point clouds, poses, segmentation masks, polygon lists, and more. 3LC provides a flexible format for storing and working with these annotation types, which can be used both in the 3LC Dashboard and in downstream machine learning workflows.

The articles below cover these annotation types in detail.

• Segmentation
• Keypoints
• Oriented Bounding Boxes

Data Layout

You don’t need to understand the internal data structure to effectively use geometric data in 3LC. However, knowing the details can be helpful if you have advanced or custom requirements.

A geometry column is any composite column that has the following top-level attributes:

{
  "instances": [], # list of instances
  "instances_additional_data": {}, # mapping from extra name to array of additional data for each instance
  "x_min": 0, # minimum x coordinate of the scene
  "x_max": 1, # maximum x coordinate of the scene
  "y_min": 0, # minimum y coordinate of the scene
  "y_max": 1, # maximum y coordinate of the scene
  "z_min": 0, # minimum z coordinate of the scene (optional)
  "z_max": 1, # maximum z coordinate of the scene (optional)
}

A single geometry instance may contain the following values:

{
  "vertices_{2d,3d}": [], # list of xy(z) coordinates,
  "lines": [], # list of vertex-pairs,
  "triangles": [], # list of vertex-triples,
  "vertices_{2d,3d}_additional_data": {}, # mapping from extra name to list of additional data for each vertex,
  "lines_additional_data": {}, # mapping from extra name to list of additional data for each line,
  "triangles_additional_data": {}, # mapping from extra name to list of additional data for each triangle,
  "bbs_{2d,3d}": [], # list of bbs [x_min, y_min, x_max, y_max] absolute coordinates
  "oriented_bbs_{2d,3d}": [], # list of oriented bbs [cx, cy, w, h, rotation], or [cx, cy, cz, w, h, d, yaw, pitch, roll] absolute coordinates with angles in radians
}

The vertices_{2d,3d}_additional_data, lines_additional_data, triangles_additional_data, and instances_additional_data attributes are optional and can be used to store additional metadata for each vertex, line, triangle, or instance. For example, you can store the visibility of each vertex, the name of each line, the role of each triangle, or the label of each instance.

The Geometry2DSchema and Geometry3DSchema classes are used to define the schema of the geometry data. This is where you can provide additional metadata such as default skeleton topologies, keypoint names, etc.

The classes Geometry2DInstances and Geometry3DInstances are used to store geometry data in 3LC. They are utilized to convert geometry data to and from the Table format.