9 Best LLM Observability Tools in 2026

Dylan Cable September 9, 2026

Nine platforms compared on tracing depth, not feature counts. What each one sees, where it stops, and what it costs.

LLM observability is the practice of recording what your models and agents actually did in production, not just whether they responded. Traces of every step, scores on the outputs, and cost attached to both. The category exists because an LLM app can return a 200 and still be wrong, and nothing in a traditional monitoring stack is built to notice.

Choosing between the platforms that do this work is harder than it looks, because they do not all do the same job. Some capture a full agent run down to the tool arguments. Some see requests passing through a proxy and nothing between them. Some score output quality on live traffic, while others only run evals against datasets you assemble by hand.

Those differences are set by where the tool sits in your stack, and that position is fixed on the day you instrument, which makes it the decision worth getting right rather than the pricing page you compared first.

The 9 best LLM observability tools below are compared on how deep each one traces an agent run, what it can score, and what it costs at production volume.

Short answer: Respan is the pick for teams running agents in production. Tracing, evals, prompt versions, and routing sit on one platform, so a bad output traces to the step that caused it instead of being correlated across four tools.

What is LLM observability?

LLM observability is the ability to reconstruct what a model or agent did on any given request, from the input a user sent through every retrieval, tool call, and model call to the output that came back. It differs from traditional observability in what it has to capture, since the thing that failed is usually the content of a response rather than its status code.

What to look for in an LLM observability tool

Nearly every platform here traces, and most of them score. What separates them is how much of an agent run they can see, and that is decided by architecture rather than by feature lists.

• Tracing depth for agent runs - The unit that fails in an agent system is the run, not the request. A run that calls retrieval, reasons over the result, calls a second tool, and drafts a reply produces one visible outcome and a dozen internal steps. Request-level logging shows a successful transaction that produced a bad result, and the full tree shows which step introduced the error.
• Where the instrumentation sits - A proxy receives HTTP requests, so it reports tokens, latency, and cost per call and never sees the reasoning between two calls. An SDK or OpenTelemetry instrumentation runs inside the application, so it captures tool arguments and retrieved documents. This sets the ceiling on everything else, and it is fixed on the day you instrument.
• Evaluation on live traffic - An LLM call can return HTTP 200 in 400ms and invent a refund policy. Latency, error rate, and token count are all fine, and the output is wrong. What matters is whether scoring runs against production spans or only against datasets somebody assembled in advance, because the second kind catches what your test set anticipated and nothing else.
• Cost and token attribution - A monthly invoice from a provider gives you nothing actionable. Attribution by model, feature, customer, and prompt version identifies which change doubled spend and when, and it is the metric most likely to move without anyone noticing, since nothing breaks while it does.
• OpenTelemetry support and trace portability - OpenTelemetry has GenAI semantic conventions for LLM spans, so instrumentation written once can export to more than one backend. That is the difference between switching platforms in an afternoon and reinstrumenting a codebase, and it matters more than it feels like it should, because most teams choose before they know what their production traffic looks like.
• Alerting on quality, not only errors - Alerting inherited from infrastructure monitoring fires on HTTP status and latency thresholds, which are the failures LLM systems are least likely to produce. Monitors on cost, token volume, and evaluation scores matter more, and an alert that arrives with the failing runs attached starts a fix rather than an investigation.

Weight the first two above the rest. Instrumentation position determines how expensive it is to be wrong about this decision later, and tracing depth determines whether the platform can show you the failure that motivated the purchase.

The 9 best LLM observability tools

1. Respan

Respan runs tracing, evaluation, prompt management, and model routing on one platform. Route, observe, and evaluate every LLM call. A trace carries the prompt version that produced it and the routing decision that served it, which removes the correlation step that eats most of the time in a multi-tool setup.

What that gets you:

• See the whole agent run - Every LLM call, tool run, retrieval, and agent turn becomes a span in one nested trace, each with its own input, output, latency, and cost.
• Score quality on live traffic - LLM judges, deterministic code checks, and human review compose into one evaluator, running against sampled production spans filtered by status, customer, or thread.
• Know where the money goes - Requests, tokens, errors, latency, and cost broken down by model, user, and API key, with soft warnings and hard caps per key.
• Hear it from Respan, not a customer - Monitors watch error rate, cost, latency, or tokens and alert Slack, email, or a webhook on breach, scoped by model, project, environment, or user.
• Stay up when a provider fails - The AI gateway routes to 1,000+ models with failover, response caching, and per-key spend limits.
• Ship prompts without shipping code - Versioned templates deploy instantly, with the version that served each request attached to its trace.

Tracing: Full agent run as a nested span tree, with evals and routing decisions attached to the same record.

Pricing: Free covers 100k logs, 1k scores, 5 datasets, 2 evaluators, and 7-day retention with unlimited seats. Team is $199 per month billed yearly, lifting scores to 10k and making datasets, evaluators, and prompts unlimited. Enterprise adds custom retention, SAML, and a 99.99% uptime SLA.

Best fit: teams running agents in production who would rather debug one system than four.

2. Langfuse

Langfuse is an open-source LLM engineering platform covering tracing, prompt management, datasets, and evaluation, with an MIT-licensed core that runs the whole product on your own infrastructure rather than a limited community build.

The trace model handles deep multi-step runs without struggling, and the meter is unusually clean: a unit is one trace, one observation, or one score, so scoring more of production costs the same as adding a retrieval step.

Tracing: Full agent run, span-level, with OpenTelemetry ingest alongside native Python and TypeScript SDKs.

Pricing: Hobby is free with 50k units and 2 users. Core is $29 per month for 100k units and unlimited users. Pro is $199 and Enterprise $2,499, with a $300 Teams add-on carrying SSO and fine-grained RBAC. Overage is $8 per additional 100k units on every paid tier.

Best fit: teams with data residency requirements and the appetite to run the infrastructure.

3. LangSmith

LangSmith ships from the LangChain team and covers tracing, datasets, evaluation, annotation queues, and a prompt hub. For teams on LangGraph it adds a development environment where you visualize the graph, set breakpoints, modify state mid-run, and resume from a checkpoint.

Tracing: Full execution tree including tool selections, retrieved documents, and model parameters.

Pricing: Free Developer tier with 5k base traces a month and one seat. Plus is $39 per seat per month with 10k base traces, then $2.50 per 1k, rising to $5.00 per 1k on extended retention. Enterprise is custom.

Best fit: teams whose stack is LangChain today and will still be LangChain in two years.

4. Arize Phoenix

Phoenix is Arize's open-source tracing and evaluation project, built on OpenTelemetry and OpenInference rather than a proprietary trace format. It runs in a notebook, locally, or in Docker with no external dependencies, which makes it usable during development rather than only after deploy.

Tracing: Span-level over OpenInference, with agent trajectory views on AX. Covers LangChain, LlamaIndex, Haystack, DSPy, and others.

Pricing: Phoenix self-hosts free with no usage caps. AX Free covers 25k spans a month and 15-day retention. AX Pro is $50 per month for 50k spans and 10 GB, then $10 per million spans and $3 per GB. Enterprise is custom.

Best fit: teams for whom OpenTelemetry portability is the deciding constraint, or organizations running classical ML alongside LLM workloads.

5. Traceloop

Traceloop is built on OpenLLMetry, its own Apache 2.0 instrumentation layer for OpenTelemetry. Agent runs arrive as standard OTel span trees, which means the same instrumentation can export to another backend without rewriting anything.

Tracing: Standard OpenTelemetry span tree, portable to other backends by design.

Pricing: Free to 50K spans per month with 24-hour retention. Enterprise is quoted and includes SOC 2 and an on-premise option.

Best fit: teams committed to OpenTelemetry who want instrumentation they can point somewhere else later.

6. Opik

Opik is Comet's Apache 2.0 platform for tracing and evaluation, and the open-source build runs the same codebase as the hosted product rather than a stripped core with the useful parts held back.

Tracing: Span-level with agent execution graphs, 60-day retention on the free tier.

Pricing: Free Cloud covers 10 team members, 25,000 spans a month, and 60-day retention. Pro is $19 a month for up to 50 members and 100,000 spans, with additional spans at $5 per 100,000. Enterprise adds unlimited members, SSO, and RBAC.

Best fit: mid-sized teams that want a genuinely open core and cheap span volume.

7. Laminar

Laminar is a debugger and observability platform built around a storage problem. Agents re-send the whole conversation every turn, so it stores each unique message once and reconstructs the trace on read.

Tracing: Span-level with deduplicated message storage and SQL access to the underlying data.

Pricing: Free covers 1 GB with no overage, $5 in Signals credits, 7-day retention, one project and one seat. Starter is $30 per month for 3 GB then $2 per GB. Pro is $150 for 10 GB then $1.50 per GB with 6-month retention. Apache 2.0 and free to self-host via Helm.

Best fit: teams debugging long agent runs who want the storage cost to track unique content rather than turn count.

8. AgentOps

AgentOps organizes everything around the session rather than the request, which is the right shape for agent work and rarer than it should be.

Tracing: Session-level replay covering every call, tool invocation, and step in order.

Pricing: $40 per seat per month, plus $0.10 per 1K spans after the first 100K, plus $0.20 per 1M LLM tokens. Three meters accrue independently, so a small team at high volume and a large team at low volume land in very different places.

Best fit: framework-native teams who want session replay during development.

9. TruLens

TruLens is a library rather than a platform. It instruments an application with feedback functions that score outputs as they flow through it, then aggregates results into a leaderboard for comparing app versions.

Tracing: OpenTelemetry spans with selector-based targeting of any span attribute.

Pricing: The library is free, and no standalone pricing exists.

Best fit: teams scoring retrieval pipelines who want span-attribute targeting and already have somewhere to put the results.

How to choose an LLM observability tool

Most teams compare features first and discover the constraints later, which wastes an evaluation cycle. Reverse it, because constraints eliminate whole categories in one pass and what survives is usually a shortlist of two or three.

• Start with what disqualifies, not what appeals. Data residency, required certifications, and deployment model rule out options before any feature enters the conversation.
• Decide whether you need a platform or a library. TruLens produces scores and leaves storage, retention, and alerting to you.
• Check where the instrumentation sits before trusting the feature list. A proxy cannot report reasoning it never receives.
• Weight portability higher than it feels worth today. Most teams choose before they know what production traffic looks like.
• Decide whether routing belongs in scope. Running several means the record of which model served a request and how it performed will either live together or be correlated by hand during every incident.
• Model cost at the volume you expect, not the volume you have. Per-seat, per-span, per-GB, and per-trace pricing diverge sharply at scale.

Whichever way that lands, run a pilot on real production traffic before committing.

FAQ

What are LLM observability tools?

Platforms that capture, monitor, and evaluate what LLM applications and agents do in production.

How is LLM observability different from AI observability?

In practice the terms are used interchangeably. AI observability is the broader label and gets applied to model monitoring and data pipeline work too.

How is LLM observability different from APM?

APM monitors infrastructure: latency, error rates, throughput, resource use. LLM observability monitors what the system produced.

Which LLM observability tools are open source?

Langfuse has an MIT core, Opik and Laminar are Apache 2.0, Traceloop's OpenLLMetry instrumentation layer is Apache 2.0, Arize Phoenix is under Elastic License 2.0, and TruLens is a free open-source library.

What is the best free LLM observability tool?

Respan covers 100k logs, 1k scores, datasets, evaluators, and monitors at no cost with unlimited seats.

Which LLM observability tool is cheapest?

It depends on whether you are paying in money or in operations.

Do I need a gateway as well as an observability tool?

If you run more than one provider, yes. A gateway handles failover, caching, and spend limits.

Can I switch tools later?

Usually, and the cost depends on how much you built against proprietary interfaces.

What should I monitor first?

Cost per feature, latency at p95, and error rate give you a baseline in an afternoon.