archivebox.progressmonitor.collection

Cached display totals for progress polling and the public snapshot list.

Missing/stale statistics are intentional: returning useful rows and progress quickly matters more than exact initial counts. Each viewer may poll once per second; repeating aggregates per request would consume the small server’s CPU and hold SQLite read locks needed by the archiving runner. Callers must accept None on a cold cache and must never wait for the refresh or count synchronously.

Background reads still take normal SQLite read locks. Existing indexes, short batches where needed, and throttled refreshes limit their cost; moving work to a thread does not make a scan free or require switching the database to WAL.

Module Contents

Functions

crawl_summary	Sample sealed snapshot metadata without adding table scans to polls.
collection_summary	Return cached totals immediately and, when due, start one refresh.

Data

_refresh_lock

API

archivebox.progressmonitor.collection._refresh_lock

‘Lock(…)’

archivebox.progressmonitor.collection.crawl_summary(crawl_id, *, snapshot_id=None)

Sample sealed snapshot metadata without adding table scans to polls.

archivebox.progressmonitor.collection.collection_summary(user, *, public=False)

Return cached totals immediately and, when due, start one refresh.

Public pages need no extra AJAX/polling for these totals: an ordinary later page load reads the populated cache. Keep this best-effort display cache separate from pagination/authorization decisions, which need actual rows.