ArchiveBox

ArchiveBox is an open-source, self-hosted application that preserves content from websites in multiple durable formats. It allows users to create local archives of public and private web content to prevent data loss from link rot or site deletions. The software is deployed as a server via Docker or installed as a Python package on Linux, macOS, and Windows, providing a centralized system for digital preservation.

Users can interact with the system through a web interface, a command line interface, a browser extension, or a Python API. Content is added by feeding URLs individually or by scheduling regular imports from RSS feeds, social media, and bookmarking services. The application saves snapshots in redundant formats to ensure data remains readable without requiring the original software to open them, ensuring that the information is preserved for decades.

Key features

• Saves content in HTML, PDF, PNG, TXT, JSON, and WARC formats
• Extracts media from YouTube and SoundCloud using yt-dlp
• Clones source code from GitHub and GitLab repositories
• Imports links from RSS feeds and bookmarking services
• Provides a REST API and webhooks for third party integrations
• Supports scheduled crawls and automated importing
• Syncs snapshots to archive.org for additional redundancy
• Stores data in ordinary files and folders for direct filesystem access

Built with Python, ArchiveBox utilizes standard tools such as Chrome and wget to capture pages. It is designed for a wide range of users, including journalists preserving cited sources, lawyers collecting evidence for legal cases, and researchers gathering training data for LLMs. The architecture emphasizes data ownership and transparency, ensuring that all archived materials are stored in open formats that are not locked into a proprietary database. It can also handle content requiring login credentials, cookies, or paywalls for advanced archival needs.

This tool serves as a personal or organizational knowledge hub for long-term web preservation and backup recovery.