tlc.core.objects.tables.system_tables.indexing

Helper classes for indexing Urls and their content

Module Contents

Functions

Function

Description

blacklist_message

resolve_message

blacklist_visitor

Create a blacklist visitor that automatically sets skip if the URL is blacklisted.

API

tlc.core.objects.tables.system_tables.indexing.blacklist_message(url: tlc.core.url.Url, exception: BaseException, backoff: datetime.timedelta | None) → str
tlc.core.objects.tables.system_tables.indexing.resolve_message(url: tlc.core.url.Url, exception: BaseException, force: bool) → str
tlc.core.objects.tables.system_tables.indexing.blacklist_visitor(blacklist: tlc.core.objects.tables.system_tables.indexing._UrlBlacklist, url: tlc.core.url.Url) → tlc.core.objects.tables.system_tables.indexing._BlacklistVisitor

Create a blacklist visitor that automatically sets skip if the URL is blacklisted.

This is a convenience function for the common pattern of creating a blacklist visitor and setting skip if the URL is blacklisted.

Parameters:

  • blacklist – The blacklist to check against

  • url – The URL to check

Returns:

A configured _BlacklistVisitor instance