Manifest

Manifest is an open-source, self-hosted model router for AI agents and applications. It aggregates multiple AI providers, API keys, and local models into a single OpenAI-compatible endpoint, allowing queries to be directed to specific models based on predefined rules. The software is deployed as a web application and server, available as a Docker image or via one-click deployment on platforms such as Railway, Render, DigitalOcean, AWS, and GCP.

Users connect their credentials to the gateway to avoid provider lock-in and centralize their AI infrastructure. The system handles the routing logic and provides a unified interface for interacting with a wide variety of large language models. It is designed for developers and teams who need to manage multiple AI subscriptions and API keys across different environments, ensuring that each query is sent to the most appropriate model for the task.

Key features

• Routing based on query complexity, specificity, and custom HTTP headers
• Support for API keys, subscriptions, and local models
• Cost tracking with notifications and spending limits
• Automatic fallback to alternative models upon query failure
• Integration with 300+ models via 31 built-in provider connections
• Support for local inference via Ollama, LM Studio, and llama.cpp
• Compatibility with any OpenAI or Anthropic compatible custom endpoints
• Unified access through a single /auto endpoint

Manifest utilizes a PostgreSQL database for managing configurations and tracking usage. It supports a broad range of providers, including major cloud AI services like OpenAI, Anthropic, and Google, as well as specialized coding subscriptions from GitHub Copilot and others. The architecture allows for dynamic discovery of provider catalogs once credentials are authenticated, making it suitable for low-code AI integration and API infrastructure management. It can be deployed in cloud environments using CloudFormation for AWS or Cloud Shell for GCP, providing flexibility in how the gateway is hosted and scaled.

This tool serves as an intermediary gateway for managing LLM observability and routing for AI-driven applications.