slipstream.checkpointing ======================== .. py:module:: slipstream.checkpointing .. autoapi-nested-parse:: Slipstream checkpointing. Attributes ---------- .. autoapisummary:: slipstream.checkpointing.STATE_NAME slipstream.checkpointing.STATE_MARKER_NAME slipstream.checkpointing.CHECKPOINT_STATE_NAME slipstream.checkpointing.CHECKPOINT_MARKER_NAME slipstream.checkpointing.CHECKPOINTS_NAME Classes ------- .. autoapisummary:: slipstream.checkpointing.Downtime slipstream.checkpointing.Dependency slipstream.checkpointing.Checkpoint Functions --------- .. autoapisummary:: slipstream.checkpointing.bind_checkpoint Module Contents --------------- .. py:data:: STATE_NAME :value: 'state' .. py:data:: STATE_MARKER_NAME :value: 'state_marker' .. py:data:: CHECKPOINT_STATE_NAME :value: 'checkpoint_state' .. py:data:: CHECKPOINT_MARKER_NAME :value: 'checkpoint_marker' .. py:data:: CHECKPOINTS_NAME :value: 'checkpoints' .. py:class:: Downtime Bases: :py:obj:`dict` Per-dependency pulse result. Falsy when empty. A single entry compares equal to that value, so ``downtime == timedelta(...)`` still holds for one leader. .. py:class:: Dependency(name: str, dependency: collections.abc.AsyncIterable[Any], downtime_threshold: Any = timedelta(minutes=10), downtime_check: slipstream.utils.AsyncCallable[[Checkpoint, Dependency], Any] | None = None, recovery_check: slipstream.utils.AsyncCallable[[Checkpoint, Dependency], bool] | None = None, marker: collections.abc.Callable[..., Any] | str | None = None, state: collections.abc.Callable[[Any, dict[str, Any]], dict[str, Any]] | None = None) Track the dependent stream state to recover from downtime. The dependency name should not be changed once created, it is used to persist the dependency in the cache. >>> async def emoji(): ... for emoji in '🏆📞🐟👌': ... yield emoji >>> Dependency('emoji', emoji()) {'checkpoint_state': None, 'checkpoint_marker': None} .. py:attribute:: name .. py:attribute:: dependency .. py:attribute:: checkpoint_state :type: Any :value: None .. py:attribute:: checkpoint_marker :type: Any :value: None .. py:attribute:: downtime_threshold .. py:attribute:: downtime_check .. py:attribute:: recovery_check .. py:attribute:: marker :value: None .. py:attribute:: state_extractor :value: None .. py:attribute:: is_down :value: False .. py:method:: uses_default_downtime_check() -> bool Return whether first-pulse event-time seeding applies. .. py:method:: save(cache: slipstream.interfaces.ICache, cache_key_prefix: str, checkpoint_state: Any, checkpoint_marker: datetime.datetime) -> None Save checkpoint state to cache. .. py:method:: load(cache: slipstream.interfaces.ICache, cache_key_prefix: str) -> None Load checkpoint state from cache. .. py:class:: Checkpoint(dependent: collections.abc.AsyncIterable[Any], *leaders: collections.abc.AsyncIterable[Any] | Dependency, dependencies: collections.abc.AsyncIterable[Any] | Dependency | list[collections.abc.AsyncIterable[Any] | Dependency] | None = None, name: str | None = None, on_downtime: collections.abc.Callable[[Checkpoint, Dependency], Any] | None = None, on_recovery: collections.abc.Callable[[Checkpoint, Dependency], Any] | None = None, cache: slipstream.interfaces.ICache | None = None, cache_key_prefix: str = '_', pause_dependent: bool = True, downtime_threshold: datetime.timedelta | None = None, marker: collections.abc.Callable[..., Any] | str | None = None, state: collections.abc.Callable[[Any, dict[str, Any]], dict[str, Any]] | None = None) Track one stream against the streams it depends on. >>> async def emoji(): ... for emoji in '🏆📞🐟👌': ... yield emoji >>> dependent, dependency = emoji(), emoji() >>> checkpoint = Checkpoint( ... dependent, ... Dependency('dependency', dependency), ... name='dependent', ... ) Pass a marker and bind the checkpoint to call ``heartbeat`` and ``check_pulse`` automatically: >>> checkpoint = Checkpoint( ... dependent, ... Dependency('dependency', dependency, marker='timestamp'), ... name='dependent', ... marker='timestamp', ... ) >>> from slipstream import handle >>> @handle(checkpoint) ... async def dependent_handler(msg, checkpoint=None): ... yield msg If no cache is provided, the checkpoint lasts only for this process. .. py:attribute:: name .. py:attribute:: dependent .. py:attribute:: dependencies .. py:attribute:: pause_dependent :value: True .. py:attribute:: marker :value: None .. py:attribute:: state_extractor :value: None .. py:attribute:: downtime :type: Any | None :value: None .. py:attribute:: state .. py:attribute:: state_marker :type: Any :value: None .. py:method:: heartbeat(marker: datetime.datetime | Any, dependency_name: str | None = None, checkpoint_state: Any = None) -> dict :async: Update checkpoint to latest state. Args: marker (datetime | Any): Typically the event timestamp that is compared to the event timestamp of a dependent stream. dependency_name (str, optional): Required when there are multiple dependencies to specify which one the heartbeat is for. checkpoint_state: Complete caller-selected dependency state. .. py:method:: check_pulse(marker: datetime.datetime | Any, checkpoint_state: Any = None, **kwargs: Any) -> Any | None :async: Update state that can be used as checkpoint. Args: marker (datetime | Any): Typically the event timestamp that is compared to the event timestamp of a dependency stream. checkpoint_state: Complete caller-selected dependent state. kwargs (Any): Any information that can be used for reprocessing any incorrect data that was sent out during downtime of a dependency stream, stored in `state`. Returns: None when every dependency is healthy. Otherwise a :class:`Downtime` map of name → check (or ``True`` if still down but not over threshold). One leader still compares equal to its timedelta / ``True``. .. py:function:: bind_checkpoint(f: collections.abc.Callable[..., Any], handler: collections.abc.Callable[..., collections.abc.Awaitable[Any]], checkpoint: Checkpoint) -> collections.abc.Callable[..., collections.abc.Awaitable[Any]] Heartbeat dependencies and pulse the dependent around a handler.