Self-Authenticated Media URLs¶
When the 3LC Dashboard displays media samples (images, videos, audio), it must load the underlying bytes into the browser. Where those bytes come from, and how the browser fetches them, affect performance and memory usage for the Dashboard.
The Dashboard prefers to render media samples through native browser tags (<img>, <video>, and <audio>) wherever
possible. Native media tags let the browser stream and decode media incrementally and let it discard decoded bytes once
they are no longer on screen.
For a media tag to be used, however, the browser must be able to fetch the underlying bytes from a plain URL, without
any custom Authorization headers. The Object Service makes that possible by handing the Dashboard a
self-authenticated URL for each media sample - a URL that carries its own short-lived credentials in the query string,
so the browser can fetch the bytes directly through a media tag.
Where Media Bytes Come From¶
Self-authenticated URLs handed to the Dashboard can take one of two forms:
Cloud-direct. When the media is stored in cloud storage (e.g. AWS S3, GCS, or Azure Blob Storage), the Object Service can issue a URL that points directly at the cloud object - an AWS presigned URL, a GCS V4 signed URL, or an Azure SAS URL. The Dashboard embeds that URL in a media tag and the bytes flow directly from cloud storage to the browser, without going through the Object Service.
Proxied through the Object Service. When a cloud-direct URL is not available or not appropriate (see below), the Object Service issues a URL that points back at itself. The browser fetches that URL and the Object Service reads the underlying bytes and streams them back.
The cloud-direct route avoids the latency and memory overhead of pulling every byte through the Object Service. The proxied route is a safe fallback that works for any URL the Object Service itself can read.
How It Works¶
When the Dashboard needs to display a media sample, it asks the Object Service to resolve the sample’s URL. The Object Service responds with either a cloud-direct URL or a proxied URL. Both URL kinds are short-lived, expiring after 30 minutes.
The Object Service chooses the proxied URL whenever a cloud-direct URL is not appropriate, which happens when:
The original sample URL points to a file on disk.
The original sample URL points to data that requires server-side transcoding before the browser can render it (e.g. TIFF images and NumPy
.npyfiles).The URL adapter for the storage backend does not implement self-authenticated URL generation.
Self-authenticated URL generation fails at runtime (e.g. transient cloud API failure or insufficient permissions on the credentials the Object Service is running with) - the error is logged and the proxy is used as a safe fallback.
Cloud-direct URLs have been explicitly disabled (see below).
Disabling Cloud-Direct URLs¶
Cloud-direct URLs assume that the machine running the Dashboard in the browser can reach the cloud storage backend. That is often true, but not always. In cases where it is not true, a cloud-direct URL would be unreachable by the browser, even though the Object Service itself can read the bytes. One particular example of such a case is when using the 3LC Enterprise On-Prem deployment with end-user machines that do not have internet access.
To handle such cases, the Object Service exposes a configuration option that disables cloud-direct URLs and forces every media URL to be served through the proxy endpoint instead:
Config key:
service.disable-external-media-urlsEnvironment variable:
TLC_SERVICE_DISABLE_EXTERNAL_MEDIA_URLSCLI argument:
--disable-external-media-urls(on the3lc servicecommand)Default:
False
When this option is set, the Object Service always hands the Dashboard a proxied URL, regardless of whether the underlying URL adapter supports cloud-direct URLs. Media samples will still display in the Dashboard, but their bytes will flow through the Object Service rather than directly from cloud storage.