tlc.core.objects.tables.system_tables.indexing

Helper classes for indexing Urls and their content

Module Contents

Functions

Function	Description
blacklist_message
resolve_message
blacklist_visitor	Create a blacklist visitor that automatically sets skip if the URL is blacklisted.

API

blacklist_message(

url: Url,

exception: BaseException,

backoff: timedelta | None,

) → str

resolve_message(

url: Url,

exception: BaseException,

force: bool,

) → str

blacklist_visitor(

blacklist: tlc.core.objects.tables.system_tables.indexing._UrlBlacklist,

url: Url,

) → tlc.core.objects.tables.system_tables.indexing._BlacklistVisitor

Create a blacklist visitor that automatically sets skip if the URL is blacklisted.

This is a convenience function for the common pattern of creating a blacklist visitor and setting skip if the URL is blacklisted.

Parameters:

• blacklist – The blacklist to check against

• url – The URL to check

Returns:

A configured _BlacklistVisitor instance