tlc.metrics.semantic_segmentation¶

Offline (Python-side) semantic-segmentation metrics, built on the C x C confusion matrix.

Every standard semantic-segmentation metric — pixel accuracy, per-class recall/accuracy, per-class IoU, mean IoU, frequency-weighted IoU, Dice/F1 — is a cheap array reduction over a single C x C pixel confusion matrix (C = the real classes; void excluded). This module computes that primitive and the readouts, mirroring the UI virtual-column math so the training-time headline and the interactive dashboard agree.

Aggregation duality (the gotcha to encode explicitly):

  • Cumulative — accumulate the confusion matrix over all pixels of all images, then compute IoU. This is what every benchmark reports.

  • Per-image then averaged — noisier and biased toward small objects.

3LC’s per-sample story wants per-image readouts (to sort / curate / filter), but the headline must be cumulative. So semantic_segmentation_metrics() returns both: the per-image scalar readouts and the per-image C x C confusion matrix (key "confusion_matrix" as a list-of-lists). The dashboard (or any offline reducer) sums the per-image matrices over rows and re-derives the metrics to get the benchmark-correct cumulative mean IoU: cumulative_mIoU = metrics_from_confusion_matrix(sum_of_per_image_cms)["mean_iou"].

numpy-only — no torch.

Module Contents¶

Functions¶

Function

Description

confusion_matrix

Build the C x C pixel confusion matrix for one image.

metrics_from_confusion_matrix

Derive the standard semantic-segmentation metrics from a C x C confusion matrix.

semantic_segmentation_metrics

Convenience: per-image semantic-segmentation readouts plus the per-image confusion matrix.

API¶

confusion_matrix(
pred_label_map: ndarray,
gt_label_map: ndarray,
class_ids: list[int],
*,
void_id: int | None = None,
) → ndarray¶

Build the C x C pixel confusion matrix for one image.

Rows are indexed by ground-truth class, columns by predicted class, both ordered by class_ids (so cell [i, j] counts pixels whose GT is class_ids[i] and whose prediction is class_ids[j]).

Pixels whose GT equals void_id are excluded entirely (void is GT-only and out of metrics). Predicted (or GT) labels not in class_ids are dropped from the tally rather than clamped into a real class — a prediction of an unlisted class simply does not land in any cell, so it counts as neither a true positive nor a confusion against a real class. (GT labels outside class_ids cannot occur for the real classes by construction; if present they are likewise dropped.)

Computed efficiently via a single np.bincount over gt_index * C + pred_index on the valid pixels.

Parameters:

  • pred_label_map – (H, W) integer array of predicted class ids.

  • gt_label_map – (H, W) integer array of ground-truth class ids.

  • class_ids – The real class ids, in the desired row/column order.

  • void_id – Class id whose GT pixels are excluded, or None for no void class.

Returns:

A (C, C) int64 array, C = len(class_ids); rows = GT, cols = pred.

metrics_from_confusion_matrix(
cm: ndarray,
) → dict[str, Any]¶

Derive the standard semantic-segmentation metrics from a C x C confusion matrix.

Works identically for a single per-image matrix and for a cumulative matrix summed over many images — it is pure array reductions over cm.

For a class i with TP = cm[i, i], FP = sum(cm[:, i]) - TP (predicted i but GT not i) and FN = sum(cm[i, :]) - TP (GT i but predicted otherwise):

  • per-class IoU = TP / (TP + FP + FN)

  • per-class recall = TP / (TP + FN) (a.k.a. per-class / GT-normalized accuracy)

  • per-class Dice / F1 = 2 TP / (2 TP + FP + FN)

Divide-by-zero policy: a class absent from both GT and prediction (TP + FP + FN == 0) is undefined; its per-class IoU / recall / Dice are reported as nan and are excluded from mean_iou (via np.nanmean) so empty classes neither inflate nor deflate the headline. mean_iou over an all-empty matrix is nan. pixel_accuracy and frequency_weighted_iou over an empty matrix are 0.0.

Parameters:

cm – A (C, C) confusion matrix; rows = GT, cols = pred.

Returns:

A dictionary with the following keys.

  • per_class_iou: list of length C (nan for empty classes).

  • mean_iou: float, np.nanmean of per_class_iou.

  • pixel_accuracy: float, trace(cm) / sum(cm).

  • per_class_recall: list of length C (nan where GT has no pixels).

  • dice: list of length C (nan for empty classes).

  • frequency_weighted_iou: float, GT-frequency-weighted mean of per-class IoU.

semantic_segmentation_metrics(
pred_label_map: ndarray,
gt_label_map: ndarray,
classes: str | Sequence[str] | Sequence[dict[str, str]] | Sequence[MapElement] | dict[float, str] | dict[int, str] | dict[float, MapElement] | dict[int, MapElement],
*,
background: int | None = None,
void: int | None = None,
include_background: bool = True,
) → dict[str, Any]¶

Convenience: per-image semantic-segmentation readouts plus the per-image confusion matrix.

Speaks the same classes + background + void vocabulary as SemanticSegmentationRleSchema and Table.from_semantic_segmentation(): pass a plain class map plus the role ids as keyword arguments. void may be omitted when classes is a value map already tagged with the void role (e.g. one read back off a column schema) — it is then recovered from the tag. Background is not a class in the value map (it is a column-level property), so pass its id via background to count it when include_background is set.

Returns both the per-image scalar readouts (for curation: sort / filter / cluster by IoU, accuracy, etc.) and the per-image C x C confusion matrix under key "confusion_matrix" (a list-of-lists). This duality is deliberate: the per-image scalars are for the per-sample story, while the cumulative, benchmark-correct headline is obtained by summing the per-image matrices across rows and re-deriving — i.e. cumulative_mIoU = metrics_from_confusion_matrix(sum_of_per_image_cms)["mean_iou"] — not by averaging the per-image mean_iou scalars.

Parameters:

  • pred_label_map – (H, W) integer array of predicted class ids.

  • gt_label_map – (H, W) integer array of ground-truth class ids.

  • classes – The class map for the column — a list of names or a dict of id -> name / MapElement.

  • background – Id of the background class, if any (not in the value map). Used only when include_background is set, to add background to the matrix and readouts.

  • void – Id of the void / ignore class, if any (excluded from the matrix and all readouts). Omit to recover it from a classes value map that is already void-tagged.

  • include_background – If True (default, VOC convention) and background is given, the background class is kept in the confusion matrix and the mean IoU. If False (Cityscapes convention) the background class is excluded from the matrix and all readouts.

Returns:

The same keys as metrics_from_confusion_matrix (per_class_iou, mean_iou, pixel_accuracy, per_class_recall, dice, frequency_weighted_iou), plus the following.

  • class_ids: the real class ids, in the matrix’s row/column order.

  • confusion_matrix: the per-image C x C matrix as a list-of-lists.