ArchiveBox Logo
  • Contents
  • Overview
    • ArchiveBox Documentation
    • UI Screenshots
      • Login
      • Public snapshot list
      • Snapshot View (capture in progress)
      • Add URLs
      • Admin dashboard
      • AI agent
      • Snapshots table
      • Snapshots grid
      • Snapshot admin detail
      • Snapshot files
      • Archive results
      • Archive result detail
      • Tags
      • Tag detail
      • Users
      • User detail
      • Crawls
      • Crawl detail
      • Crawl schedules
      • Crawl schedule detail
      • Personas
      • Persona detail
      • Machines
      • Machine detail
      • Network interfaces
      • Network interface detail
      • Binaries
      • Binary detail
      • Processes
      • Process detail
      • API tokens
      • API token detail
      • Webhooks
      • Webhook detail
      • Environment
      • Configuration
      • Configuration detail
      • Dependencies
      • Dependency detail
      • Plugins
      • Workers
      • Worker detail
      • Logs
      • Log detail
      • Snapshot View (singlefile)
      • Snapshot View (screenshot)
      • Snapshot View (wget)
      • Snapshot View (dom)
      • Snapshot View (pdf)
      • Snapshot View (readability)
      • Snapshot View (ytdlp)
      • Snapshot View (responses)
      • Snapshot View (chrome_mhtml)
      • Snapshot View (defuddle)
      • Snapshot View (mercury)
      • Snapshot View (archivewebpage)
      • Snapshot View (chrome)
      • Snapshot View (consolelog)
      • Snapshot View (dns)
      • Snapshot View (sslcerts)
      • Snapshot View (redirects)
      • Snapshot View (headers)
      • Snapshot View (seo)
      • Snapshot View (accessibility)
      • Snapshot View (htmltotext)
      • Snapshot View (trafilatura)
      • Snapshot View (liteparse)
      • Snapshot View (parse_html_urls)
      • Snapshot View (parse_txt_urls)
      • Snapshot View (parse_dom_outlinks)
      • Snapshot View (hashes)
      • Snapshot View (header collapsed)
    • Key Features
    • Quickstart
      • ✳️  Easy Setup
      • 🛠  Package Manager Setup
      • 🎗  Other Options
      • ➡️  Next Steps
      • Usage
        • ⚡️  CLI Usage
        • ArchiveBox Subcommands
    • Overview
      • Input Formats: How to pass URLs into ArchiveBox for saving
      • Output Formats: What ArchiveBox saves for each URL
      • Configuration
      • Dependencies
      • Archive Layout
      • Static Archive Exporting
      • Caveats
        • Archiving Private Content
        • Security Risks of Viewing Archived JS
        • Working Around Sites that Block Archiving
        • Saving Multiple Snapshots of a Single URL
        • Storage Requirements
      • Screenshots
    • Background & Motivation
      • Comparison to Other Projects
      • Internet Archiving Ecosystem
    • Documentation
      • Getting Started
      • Advanced
      • Developers
      • More Info
    • ArchiveBox Development
      • Setup the dev environment
        • 1. Setup the monorepo
        • 2. Option A: Install the Python, JS, and system dependencies directly on your machine
        • 2. Option B: Build the docker container and use that for development instead
      • Common development tasks
        • Run in DEBUG mode
        • Install and run a specific GitHub branch
          • Use a Pre-Built Image
          • Build Branch from Source
        • Run the linters / tests
        • Make DB migrations, enter Django shell, other dev helper commands
        • Contributing a new extractor
        • Build the docs, pip package, and docker image
      • Further Reading
  • Getting Started
    • Quickstart
      • 1. Set up ArchiveBox
      • 2. Get your list of URLs to archive
      • 3. Add your URLs to the archive
      • ✅ Done!
    • Install
      • Supported Systems
      • Option A. Docker / Docker Compose Setup ⭐️
      • Option B. uv Install Shortcut
      • Option C. Bare Metal Setup
        • 1. Install uv or the ArchiveBox OS package
          • macOS
          • Ubuntu/Debian-based Systems
        • 2. Install ArchiveBox using uv
        • 3. Install runtime dependencies using archivebox install
        • Troubleshooting
        • Next Steps: Add some URLs to archive and try out CLI / Web UI
        • Next Steps: Upgrading Archivebox to a new version
        • Further Reading
    • Docker
      • Overview
      • Docker Compose
        • Setup
        • Upgrading
        • Usage
        • Accessing the data
          • Docker PUID / PGID
        • Configuration
        • Cloudflare Tunnel: production example
          • 1. Choose the domain and create the configuration
          • 2. Obtain the certificate and tunnel credentials
          • 3. Route DNS and start the stack
          • Notifications and private access
      • Docker
        • Setup
        • Upgrading
        • Usage
        • Accessing the data
        • Configuration
    • Configuration
      • General Settings
        • ONLY_NEW
        • TIMEOUT
        • RESOLUTION
        • CHECK_SSL_VALIDITY
        • USER_AGENT
        • COOKIES_FILE
        • DEFAULT_PERSONA
        • URL_DENYLIST
        • URL_ALLOWLIST
        • TAG_SEPARATOR_PATTERN
        • CRAWL_MAX_URLS
        • CRAWL_MAX_SIZE
        • CRAWL_TIMEOUT
        • CRAWL_MAX_CONCURRENT_SNAPSHOTS
        • SNAPSHOT_MAX_SIZE
        • DELETE_AFTER
        • PERMISSIONS
        • PLUGINS
      • Server Settings
        • ADMIN_USERNAME / ADMIN_PASSWORD
        • PUBLIC_INDEX / PUBLIC_ADD_VIEW
        • SECRET_KEY
        • BIND_ADDR
        • BASE_URL
        • SERVER_SECURITY_MODE
        • SNAPSHOTS_PER_PAGE
        • FOOTER_INFO
        • REVERSE_PROXY_USER_HEADER
        • REVERSE_PROXY_WHITELIST
        • LOGOUT_REDIRECT_URL
        • LDAP Settings
          • LDAP_ENABLED
          • LDAP_SERVER_URI
          • LDAP_BIND_DN
          • LDAP_BIND_PASSWORD
          • LDAP_USER_BASE
          • LDAP_USER_FILTER
          • LDAP_USERNAME_ATTR
          • LDAP_FIRSTNAME_ATTR
          • LDAP_LASTNAME_ATTR
          • LDAP_EMAIL_ATTR
          • LDAP_CREATE_SUPERUSER
      • Storage Settings
        • OUTPUT_PERMISSIONS
        • ENFORCE_ATOMIC_WRITES
        • TMP_DIR
        • ABXPKG_LIB_DIR
        • ALLOW_NO_UNIX_SOCKETS
      • Database Settings
        • DATABASE_ENGINE
        • DATABASE_HOST / DATABASE_PORT / DATABASE_USER / DATABASE_PASSWORD
        • DATABASE_NAME
        • SQLITE_JOURNAL_MODE
        • SQLITE_MMAP_SIZE
        • SQLITE_BUSY_TIMEOUT
        • SQLITE_LOCK_RETRY_TIMEOUT
        • SQLITE_LOCK_RETRY_INTERVAL
      • Search Settings
        • SEARCH_BACKEND_ENGINE
      • Shell Options
        • DEBUG
        • USE_COLOR
        • SHOW_PROGRESS
      • Plugin Configuration
        • Shared core options that plugins fall back to
        • Listing & setting plugin options
        • Why is plugin config documented separately?
    • Security Overview
      • Web UI Permissions
      • ArchiveBox Use-Cases
        • Archiving Public Content Only ⭐️ [Default, recommended for most people]
        • Archiving Content Behind Log-Ins 🚨 [Advanced users only]
        • ⚠️ Things to watch out for: ⚠️
        • Publishing
      • Run ArchiveBox as an unprivileged user
      • Output Folder
        • Database
        • Filesystem
          • Purging entries
          • Permissions
    • Usage
      • CLI Usage
        • Run ArchiveBox with configuration options
        • Import a single URL
        • Import a list of URLs from a text file
        • Import list of links from browser history
        • Import browser cookies into a persona
      • UI Usage
        • Explanation of buttons in the web UI - admin snapshots list
      • Browser Extension Usage
        • More Info
      • Disk Layout
        • Large Archives
      • SQL Shell Usage
      • Python Shell Usage
      • Python API Usage
  • Guides
    • Setting Up Storage
      • Supported Local Filesystems
        • EXT4 (default on Linux), APFS (default on macOS)
        • ZFS (recommended for experienced Linux/BSD operators) ⭐️
        • NTFS, HFS+, BTRFS
        • EXT2, EXT3, FAT32, exFAT
      • Supported Remote Filesystems
        • NFS (Docker Driver)
        • SMB / Ceph (Docker CIFS Driver)
        • Amazon S3 / Backblaze B2 / Google Drive / etc. (RClone)
          • Rclone Config Examples
          • Option A: Running Rclone on a bare-metal host
          • Option B: Running Rclone with the Docker storage plugin
        • More Docker Storage Plugins
    • Setting Up Authentication
      • Set Up Admin Web UI Permissions
      • Admin Web UI Authentication Methods
        • Username & Password (the default)
        • Reverse Proxy Authentication
        • LDAP Authentication
        • Not Yet Supported: SAML / OAuth2 / OpenID Authentication
      • REST API
        • API Bearer Token Authentication
        • API Request Header Authentication
        • API Query Parameter Authentication
          • Further Reading
    • Setting Up Search
      • How to Search in ArchiveBox
      • How Search Works
      • ArchiveBox Search Backends
        • ripgrep (fallback)
          • Pros
          • Cons
        • ripgrep-all (aka rga)
        • ugrep
          • Pros
          • Cons
        • sonic ⭐️ (the recommended upgrade path for most people)
          • Pros
          • Cons
        • SQLite FTS5
          • Pros
          • Cons
        • Further Reading
    • Publishing Your Archive
      • 1. Use the built-in web server
      • 2. Export and host it as static HTML
      • Security Concerns
      • Copyright Concerns
        • Further Reading: USA Copyright Law & Fair Use Exemptions
    • Scheduled Archiving
      • How It Works
      • CLI Usage
      • Docker Compose
      • Examples
    • Archive pages when they change
      • 1. Create an ArchiveBox API key
      • 2. Allow LAN connections
      • 3. Create an archiving group
      • 4. Choose the pages to archive
      • 5. Check your first capture
      • Containers on the same host
      • Cloudflare: choose the shortest connection
        • 1. Connect the public hostname
        • 2. Authorize the notification sender
        • 3. Paste one notification URL
        • 4. Test each authentication layer
    • OSINT preservation and verification
      • Private collection configuration
      • Administrator login
      • Repeated captures and upgrades
      • Verify the evidence
        • File integrity
        • Authenticated response
        • Independent timestamp
      • Logged-in sources and growing archives
    • Archiving URLs from Google Sheets
    • Chat Bots
      • ArchiveBox Bot
      • ArchiveBox AI Bot
    • Chrome / Chromium Setup
      • Install ArchiveBox’s browser
    • Setting Up a Chromium User Profile
      • Import an existing browser profile
        • 1. Export the profile on your computer
        • 2. Transfer the persona to your server
        • 3. Save and check the page
      • Setting up a new profile
        • Non-Docker Setup (Local Host)
        • Docker VNC Setup
          • One-off Compose setup
      • Troubleshooting Chromium Install
    • Upgrading Versions
      • Upgrading with Docker Compose ⭐️
      • Upgrading with plain Docker
      • Upgrading with a package manager
      • Merge two or more existing archives
      • Related Documents
    • Upgrading or Merging Archives
    • Merging Collections
      • Modify the ArchiveBox SQLite3 DB directly
        • Example: Modifying an existing user’s email
        • Example: Adding a new user with a hashed password
      • Database Troubleshooting
      • Related Documents
    • Troubleshooting
      • Installing
        • macOS
        • Python and uv
        • Chromium/Google Chrome
        • Wget & Curl
        • NPM Dependencies
      • Archiving
        • No links parsed from export file
        • Lots of skipped sites
        • Lots of errors
        • Lots of broken links from the index
        • Removing unwanted links from the index
      • Hosting the Archive
        • Other database or filesystem issues
          • Docker Permissions issues
      • Database
        • Filesystem doesn’t support FSYNC (e.g. network mounts)
        • Database and filesystem contention issues when running multiple ArchiveBox processes
        • Database migrations errors or upgrade issues
        • Repairing a corrupted SQLite3 database file
  • Architecture
    • ArchiveBox Architecture Diagrams
      • High-Level Execution Flow
      • Persistent Data
      • Crawl Queue Lifecycle
      • Snapshot Queue Lifecycle
      • ArchiveResult Projection
  • API Reference
    • Filesystem
    • SQL API
    • REST API
    • Python API
      • archivebox
        • Subpackages
          • archivebox.importers
          • archivebox.machine
          • archivebox.progressmonitor
          • archivebox.ldap
          • archivebox.misc
          • archivebox.cli
          • archivebox.api
          • archivebox.workers
          • archivebox.memento
          • archivebox.core
          • archivebox.search
          • archivebox.mcp
          • archivebox.personas
          • archivebox.crawls
          • archivebox.config
          • archivebox.base_models
          • archivebox.services
          • archivebox.plugins
        • Submodules
          • archivebox.uuid_compat
          • archivebox.__main__
          • archivebox.manage
        • Package Contents
          • Classes
          • Functions
          • Data
          • API
  • Meta
    • Roadmap
      • Planned Specification
        • v0.7: Schema improvements
        • v0.8:  Security
        • v0.9:  Performance
        • v1.0: Full headless browser control
        • v2.0 Federated or distributed archiving
        • Major long-term changes
        • Smaller planned features
      • Past Releases
      • UI / UX Improvements Planned
      • New Extractors Planned
        • Social Media
        • Video/Streams
        • Audio/Music
        • Photos/Images/Comics
        • Text/Forums
        • MOOC/Educational Content
        • Re-Archiving / WARC Creation
        • Other
    • Changelog
    • ArchiveBox v0.9.x: Safer, Faster, and Much More Powerful Web Archiving
      • ✨ What’s New in v0.9.x
      • ⬆️ Upgrading
        • Docker Compose
        • Native / uv
    • Supporting Development
    • Web Archiving Community
      • The Master Lists
      • Web Archiving Projects
        • Bookmarking Services
        • From the Archive.org & Archive-It teams
        • From Webrecorder
        • From Rhizome.org (Conifer)
        • From the Old Dominion University: Web Science Team
        • From the Archives Unleashed Team
        • From the IIPC team
        • Other Public Archiving Services
        • Other ArchiveBox Alternatives
        • Smaller Utilities
      • Reading List
        • Blogs Friends of ArchiveBox
        • Articles We Like About Internet Archiving
        • ArchiveBox-Specific Posts, Tutorials, and Guides
        • ArchiveBox Discussions in News & Social Media
      • Communities
        • Most Active Communities
        • Web Archiving Communities
        • General Archiving Foundations, Coalitions, Initiatives, and Institutes
      • ArchiveBox Community Resources
        • ArchiveBox Chat Rooms
        • ArchiveBox on Social Media
        • ArchiveBox on Package Distribution Platforms
ArchiveBox
  • ArchiveBox
  • Edit on GitHub

Welcome to ArchiveBox!

Just getting started?

Check out the Quickstart guide.

Need help with something?

Open an issue on Github or chat on Zulip.

Want to join the community?

See our Community Wiki page.

ArchiveBox Logo

ArchiveBox

“The open-source self-hosted internet archive.”

Website | Github | Source | Bug Tracker

mkdir my-archive; cd my-archive/
uv tool install --python 3.13 --prerelease explicit --upgrade 'archivebox>=0.9.0rc0,<0.10'

archivebox init
archivebox install
archivebox add https://example.com
archivebox status

Documentation

  • Contents
  • Overview
    • ArchiveBox Documentation
    • UI Screenshots
    • Key Features
    • Quickstart
    • Overview
    • Background & Motivation
    • Documentation
    • ArchiveBox Development
  • Getting Started
    • Quickstart
    • Install
    • Docker
    • Configuration
    • Security Overview
    • Usage
  • Guides
    • Setting Up Storage
    • Setting Up Authentication
    • Setting Up Search
    • Publishing Your Archive
    • Scheduled Archiving
    • Archive pages when they change
    • OSINT preservation and verification
    • Archiving URLs from Google Sheets
    • Chat Bots
    • Chrome / Chromium Setup
    • Setting Up a Chromium User Profile
    • Upgrading Versions
    • Upgrading or Merging Archives
    • Merging Collections
    • Troubleshooting
  • Architecture
    • ArchiveBox Architecture Diagrams
  • API Reference
    • Filesystem
    • SQL API
    • REST API
    • Python API
  • Meta
    • Roadmap
    • Changelog
    • ArchiveBox v0.9.x: Safer, Faster, and Much More Powerful Web Archiving
    • Supporting Development
    • Web Archiving Community
Next

© Copyright 2026 ArchiveBox.