archivebox.cli.archivebox_update

Module Contents

Functions

_apply_pattern_filters
_get_snapshot_crawl
_get_search_indexing_plugins
_build_filtered_snapshots_queryset
reindex_snapshots
update	Update snapshots: migrate old dirs, reconcile DB, and re-queue for archiving.
drain_old_archive_dirs	Drain old archive/ directories (0.8.x → 0.9.x migration).
process_all_db_snapshots	O(n) scan over entire DB from most recent to least recent.
process_filtered_snapshots	Process snapshots matching filters (DB query only).
print_stats	Print statistics for filtered mode.
print_combined_stats	Print statistics for full mode.
print_index_stats
main

Data

LINK_FILTERS

API

archivebox.cli.archivebox_update.LINK_FILTERS: dict[str, collections.abc.Callable[[str], django.db.models.Q]]

None

archivebox.cli.archivebox_update._apply_pattern_filters(snapshots: django.db.models.QuerySet[archivebox.core.models.Snapshot, archivebox.core.models.Snapshot], filter_patterns: list[str], filter_type: str) → django.db.models.QuerySet[archivebox.core.models.Snapshot, archivebox.core.models.Snapshot]

archivebox.cli.archivebox_update._get_snapshot_crawl(snapshot: archivebox.core.models.Snapshot) → Crawl | None

archivebox.cli.archivebox_update._get_search_indexing_plugins() → list[str]

archivebox.cli.archivebox_update._build_filtered_snapshots_queryset(*, filter_patterns: collections.abc.Iterable[str], filter_type: str, before: float | None, after: float | None, resume: str | None = None)

archivebox.cli.archivebox_update.reindex_snapshots(snapshots: django.db.models.QuerySet[archivebox.core.models.Snapshot, archivebox.core.models.Snapshot], *, search_plugins: list[str], batch_size: int) → dict[str, int]

archivebox.cli.archivebox_update.update(filter_patterns: collections.abc.Iterable[str] = (), filter_type: str = 'exact', before: float | None = None, after: float | None = None, resume: str | None = None, batch_size: int = 100, continuous: bool = False, index_only: bool = False) → None

Update snapshots: migrate old dirs, reconcile DB, and re-queue for archiving.

Three-phase operation (without filters):

• Phase 1: Drain old archive/ dirs by moving to new fs location (0.8.x → 0.9.x)

• Phase 2: O(n) scan over entire DB from most recent to least recent

• No orphan scans needed (trust 1:1 mapping between DB and filesystem after phase 1)

With filters: Only phase 2 (DB query), no filesystem operations. Without filters: All phases (full update).

archivebox.cli.archivebox_update.drain_old_archive_dirs(resume_from: str | None = None, batch_size: int = 100) → dict[str, int]

Drain old archive/ directories (0.8.x → 0.9.x migration).

Only processes real directories (skips symlinks - those are already migrated). For each old dir found in archive/:

1. Load or create DB snapshot

2. Trigger fs migration on save() to move to data/users/{user}/…

3. Leave symlink in archive/ pointing to new location

After this drains, archive/ should only contain symlinks and we can trust 1:1 mapping between DB and filesystem.

archivebox.cli.archivebox_update.process_all_db_snapshots(batch_size: int = 100, resume: str | None = None) → dict[str, int]

O(n) scan over entire DB from most recent to least recent.

For each snapshot:

1. Reconcile index.json with DB (merge titles, tags, archive results)

2. Queue for archiving (state machine will handle it)

No orphan detection needed - we trust 1:1 mapping between DB and filesystem after Phase 1 has drained all old archive/ directories.

archivebox.cli.archivebox_update.process_filtered_snapshots(filter_patterns: collections.abc.Iterable[str], filter_type: str, before: float | None, after: float | None, resume: str | None, batch_size: int) → dict[str, int]

Process snapshots matching filters (DB query only).

archivebox.cli.archivebox_update.print_stats(stats: dict)

Print statistics for filtered mode.

archivebox.cli.archivebox_update.print_combined_stats(stats_combined: dict)

Print statistics for full mode.

archivebox.cli.archivebox_update.print_index_stats(stats: dict[str, Any]) → None

archivebox.cli.archivebox_update.main(**kwargs)