Open Source LLM Application Frameworks Software
LLM Application Frameworks provide the infrastructure needed to build, deploy, and monitor applications powered by large language models. These tools manage complex agentic workflows, RAG pipelines, and prompt versions to ensure consistent model responses. This type of software is for developers who need to track LLM calls and evaluate AI behavior. By self-hosting these apps, you keep prompt histories and interaction logs on your own hardware. This approach allows you to audit agent decisions and manage memory layers without sending sensitive conversation data to external providers.
This page lists 9 open source tools in the LLM Application Frameworks category. The most popular are Ollama, Dify and Mem0. Most use the Apache-2.0 or MIT license, and 8 offer an official Docker image.
Ollama
Run and manage open source large language models locally on macOS, Windows, and Linux desktops.
Dify
A visual development platform for building agentic workflows, RAG pipelines, and applications powered by large language models.
Mem0
Provides a persistent memory layer for AI agents to remember user preferences and interactions across sessions.
Langfuse
Track LLM calls, manage prompts, and run evaluations to debug and monitor large language model applications.
Opik
An observability and evaluation platform for monitoring, testing, and optimizing generative AI and agentic systems.
Parlant
A conversational AI server that manages agent behavior through context aware guidelines for consistent and auditable responses.
Botpress
Build and deploy AI agents and chatbots using large language models with a focus on integrations.
Agenta
An LLMOps platform for prompt management, evaluation, and observability to help teams build reliable AI applications.
AgentOS
A TypeScript framework for autonomous AI agents featuring cognitive memory, multi-agent orchestration, and runtime tool generation.
Join our newsletter to get shiny new open source software delivered to your inbox. Unsubscribe anytime.