Arize Phoenix

Arize Phoenix is an open-source AI observability platform. It provides a specialized environment for the experimentation, evaluation, and troubleshooting of LLM applications through runtime tracing and performance benchmarking. The platform helps developers identify bottlenecks in AI workflows and refine the accuracy of model responses.

The software is deployed as a server that can run on local machines, within Jupyter notebooks, in containerized environments via Docker or Kubernetes, or in the cloud. It is available as a comprehensive platform package via pip or conda, while also offering lightweight Python and TypeScript sub-packages. These sub-packages allow developers to implement specific functionalities, such as OpenTelemetry instrumentation or API interactions, without deploying the full platform locally.

Key features

• OpenTelemetry-based instrumentation for tracing LLM application runtimes
• LLM-assisted benchmarking using response and retrieval evaluations
• Versioned datasets for experimentation, evaluation, and fine-tuning
• Tracking and evaluation of changes to prompts, models, and retrieval
• Prompt playground for comparing models and replaying traced calls
• Systematic prompt management with version control and tagging
• Integrated AI engineering agent for debugging traces and iterating on prompts

Phoenix is designed to be vendor and language agnostic, utilizing the OpenInference project for auto-instrumentation. It supports a broad array of LLM providers, including OpenAI, Anthropic, Google GenAI, and AWS Bedrock. The framework integrates with numerous popular AI libraries such as LlamaIndex, CrewAI, and DSPy. To facilitate developer workflows, the architecture includes specialized clients and a CLI for fetching traces and datasets for use with coding agents.

It serves as a technical infrastructure layer for AI engineers to monitor, evaluate, and optimize the performance of generative AI applications throughout the development lifecycle.