Langfuse

Langfuse is an open-source LLM engineering platform. It provides tools for tracing, evaluating, and monitoring large language model applications to help teams debug and improve AI performance. The platform allows teams to collaboratively develop and monitor AI applications by ingesting traces to track LLM calls and other relevant logic such as retrieval, embedding, or agent actions.

The software is deployed as a self-hosted instance via Docker Compose, Kubernetes using Helm, or through cloud templates for AWS, Azure, and GCP. It integrates into applications through typed SDKs for Python and JS/TS, or via automated instrumentation for specific frameworks. This allows developers to inspect complex logs and user sessions to identify where a model may be failing or producing suboptimal results.

Key features

• Application observability for tracking LLM calls, retrieval, and agent actions
• Centralized prompt management with version control and server side caching
• Evaluation support for LLM-as-a-judge, code evaluators, and manual labeling
• Dataset creation for benchmarks and pre-deployment testing
• Interactive playground for testing prompts and model configurations
• Public API with OpenAPI specification for custom LLMOps workflows
• Automated instrumentation for various frameworks

Langfuse is built using the ClickHouse open source database for high performance data storage. It is designed for developers building AI agents and LLM applications, offering deep integrations with tools like AutoGen, Haystack, and the Vercel AI SDK. The platform allows users to jump from a traced error directly into the playground to iterate on the prompt and configuration, shortening the feedback loop during development. It also supports a wide range of model providers through LiteLLM, including Azure, Anthropic, and local models via Ollama.

It serves as a technical infrastructure layer for teams requiring observability and evaluation within their AI development lifecycle.