Arize-ai/phoenix Alternatives

Choose when you want a dedicated LLM evaluation and agent-testing platform with a TypeScript stack.

The platform for LLM evaluations and AI agent testing

Choose for agent-first observability/evals with a lightweight TypeScript platform.

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

Choose for a mature LLM observability/eval platform with prompt management and a large integration ecosystem.

🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

Choose for a Python-native LLM tracing/eval platform with RAG and agentic workflow dashboards.

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Choose for a Python evaluation/tracking library that plugs into existing LLM apps with MIT licensing.

Evaluation and Tracking for LLM Experiments and AI Agents

Choose for agent/LLM observability and evaluation in Python when you need self-hosted dashboards and analytics.

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

Choose for an end-to-end LLM/agent eval, tracing, dataset, and guardrail platform under Apache-2.0.

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

Choose when you need general OpenTelemetry APM/logs/metrics alongside AI agent tracing, accepting shallower LLM-specific evals.

SigNoz is an open-source, OpenTelemetry-native observability platform for your team and their AI agents. Get logs, metrics, and traces in one tool with features like APM, distributed tracing, log management, infra monitoring, etc. Combined with SigNoz MCP and a native AI teammate (in SigNoz Cloud) it helps you build more resilient apps.

Choose if you want an LLM gateway with built-in observability/evals in Rust rather than a Python eval-first library.

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

Choose for proxy-based one-line LLM monitoring/evals when a gateway layer fits better than SDK instrumentation.

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

Choose for full-lifecycle agent development/optimization in Go when you need agent debugging plus evals.

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

Choose when you want OpenTelemetry-native LLM/agent tracing and evals with broad framework auto-instrumentation.

Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.

Choose if you already standardize on MLflow for ML lifecycle and want LLM eval/tracking in the same system.

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

Choose when prompt design/version management is the priority and you need basic LLMOps observability.

🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

Choose for lightweight agent session monitoring/cost tracking rather than a full eval and dataset platform.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

Choose for OpenTelemetry-based LLM tracing/evals if an AGPL-3.0 platform is acceptable.

Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊

Choose for declarative CI prompt testing and red-teaming rather than production tracing dashboards.

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

Choose for automated LLM testing/vulnerability scanning in development rather than live observability.

🐢 Open-Source Evaluation & Testing library for LLM Agents

Choose only as a legacy eval platform; stale for over two years and lacks current maintenance.

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.