AnythingLLM

AnythingLLM is an open-source and self-hosted Web App / Server. It functions as an all-in-one AI application that allows users to chat with local or cloud-based large language models while maintaining full control over their data and privacy. The software enables the creation of a private AI interface that integrates document ingestion and agent capabilities without requiring a complex setup or the creation of an account.

The software is available as a desktop application for Mac, Windows, and Linux, or as a server deployment via Docker. It is designed for users who need to build a private knowledge base or an AI-driven workspace. By allowing users to connect their favorite LLM providers, the application transforms static documents into an interactive chat experience through a local-first approach.

Key features

• Support for multiple document types including PDF, TXT, and DOCX
• Dynamic model routing based on user-defined rules
• Multi-user support and permissioning for Docker installations
• Built-in AI agents capable of web browsing and task automation
• Scheduled tasks and prompts via cron schedules
• Custom embeddable chat widget for websites
• Integration with various open-source and closed LLM providers
• Native embedding and vector database support via LanceDB

The system architecture is built as a monorepo consisting of a React frontend, a NodeJS Express server for LLM interactions and vector database management, and a dedicated collector server for parsing documents. It supports a wide range of technical integrations, including OpenAI-compatible embedding APIs and text-to-speech services. The platform is intended for individuals and organizations needing a private RAG (Retrieval-Augmented Generation) system that can handle large document sets with source citations and optimized response times. It also provides a full developer API for custom integrations and a Chrome browser extension for expanded utility.

AnythingLLM is a local-first, no-code framework for deploying private AI workspaces and agentic workflows.