R_REDDYX.XYZ
Infrastructure2026-06-23

LLM Monitoring in Production 2026: Langfuse, Helicone, and Other Observability Tools

LLM Monitoring in Production 2026: Langfuse, Helicone, and Other Observability Tools
TL;DR: LLM observability is not a luxury, but a mandatory infrastructure layer if you are running models in production. Langfuse (open-source, self-host) and Helicone (proxy-first, one-minute setup) cover 80% of tasks for tracing, token counting, and cost. Below — how to choose what to measure and a working connection code in 5 minutes.

WHY APM IS NOT ENOUGH FOR LLM

Classic monitoring (Datadog, Grafana, Sentry) answers the question 'is the service alive and what is the latency'. For an LLM application, this is catastrophically little. A request returned HTTP 200 and fit within 800 ms — but the model hallucinated, spent 12,000 tokens on a trivial matter, or got stuck in a tool-calling loop. Standard APM will not see this.

LLM observability answers other questions: what exactly went into the model, how much it cost, why the agent chain made that decision, and how good the answer was. In 2026, when a typical application is not a single call but a graph of a retriever, several LLM steps, and tool calls, without tracing you are debugging blind.

Three pains that observability solves:

  • Cost. One forgotten re-prompting loop can burn through the budget overnight. Without per-request token accounting, you will find out about this from the provider's bill.
  • Chain debugging. An 8-step agent failed at step 6. Without a trace, you won't understand which intermediate output broke the logic.
  • Quality. Prompt regression is not caught by unit tests. You need evaluations on real traffic — LLM-as-judge, user thumbs up/down, evals.

WHAT SHOULD YOU MEASURE AT ALL

Before installing the tool, decide on metrics. Minimum set for production:

  1. Tokens (prompt / completion / total) — the basis for cost and for finding bloated prompts.
  2. Cost per request and per trace — broken down by model, user, feature.
  3. Latency and TTFT (time to first token) — for streaming, TTFT is critical, not total time.
  4. Full-chain trace — nested spans: retriever → LLM → tool → LLM.
  5. Frequency of errors and retries — rate limits, timeouts, model refusals (refusals).
  6. Answer quality — scores from evals, user feedback, hallucination flags.

Keep PII in mind separately. Prompts often contain personal data, and sending them to a cloud dashboard without masking is a straight path to compliance problems. Langfuse and Helicone both can scrub sensitive fields, but you need to enable this manually.

LANGFUSE: OPEN-SOURCE DE FACTO STANDARD

Langfuse is the most popular open-source tool for LLM observability. It is deployed via Docker self-hosted, there is also a cloud offering. The core — hierarchical traces: trace contains nested span and generation, which fits perfectly onto agent chains.

Strong points

  • Full self-host under MIT-compatible core license — data never leaves your perimeter.
  • Prompt management with versioning: prompts live outside code, rollback in one click.
  • Built-in evals and datasets for regression testing of prompts.
  • SDK for Python and JS/TS + decorators, native integration with LangChain, LlamaIndex, OpenAI SDK.

Weak points

Self-host requires Postgres + ClickHouse + Redis — this is not 'spin up one container and forget'. For a team without DevOps, it's easier to take the cloud. The UI is sometimes overloaded, the learning curve is higher than Helicone's.

# pip install langfuse openai
from langfuse.openai import openai  # drop-in обёртка над OpenAI SDK

client = openai.OpenAI()

# Каждый вызов автоматически трейсится в Langfuse
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Объясни трейсинг LLM в двух фразах"}],
    # метаданные для группировки в дашборде
    metadata={"user_id": "u-42", "feature": "chat", "env": "prod"},
)
print(response.choices[0].message.content)

# Ручной трейс для агентной цепочки
from langfuse import Langfuse
lf = Langfuse()
trace = lf.trace(name="rag-pipeline", user_id="u-42")
retr = trace.span(name="retriever", input={"q": "что мерить"})
retr.end(output={"docs": 5})

HELICONE: PROXY-FIRST, ONE MINUTE TO START

Helicone is built on a different principle — it's a proxy. You change the base_url in your client to the Helicone endpoint, add the API key in the header, and all traffic is logged automatically. Zero changes to business logic.

Strengths

  • Integration literally in a minute — replace the URL and add a header.
  • Response caching at the proxy level — saves both money and latency on repeats.
  • Rate limiting and cost alerts out of the box.
  • Not tied to a specific framework: works with any HTTP client.

Weaknesses

Proxy in the critical path — this is an additional point of failure and an extra network hop (self-host eliminates latency concern but adds operational overhead). Hierarchical agent traces are weaker than Langfuse’s — Helicone excels at logging individual calls, not deep graph tracing.

from openai import OpenAI

# Весь фокус — в base_url и заголовке. Код приложения не меняется.
client = OpenAI(
    base_url="https://oai.helicone.ai/v1",
    default_headers={
        "Helicone-Auth": "Bearer sk-helicone-xxxx",
        "Helicone-Property-Feature": "chat",   # кастомные свойства
        "Helicone-Cache-Enabled": "true",       # кэш повторов
    },
)

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Привет из прода"}],
)

COMPARISON: LANGFUSE, HELICONE AND OTHERS

On the 2026 market, LangSmith (native to the LangChain ecosystem, but closed-source and paid), Phoenix from Arize (strong in evals and based on OpenTelemetry), and W&B Weave still hold. Key differences:

InstrumentModelSelf-hostStrengthFor
LangfuseSDK + decoratorsYes (OSS)Chain tracing, prompt managementTeams needing data in their own environment
HeliconeProxyYes (OSS)Connection speed, cache, rate-limitThose who want logs 'here and now'
LangSmithSDKEnterprise onlyDeep integration with LangChainEntire stack on LangChain
Arize PhoenixOpenTelemetryYes (OSS)Evals, drift, embeddings analysisML teams focused on quality

Practical rule: if unsure — start with Helicone for quick log collection, and when you hit the need to trace multi-step agents and version prompts, add Langfuse. They are not mutually exclusive. Fresh releases and alternatives of these tools are tracked in the REDDYX catalog.

OPENTELEMETRY: WHERE IS EVERYTHING HEADING

The main trend of 2026 is standardization via OpenTelemetry. A semantic conventions specification for GenAI has appeared (attributes such as gen_ai.usage.input_tokens, gen_ai.request.model), and most serious tools now can accept OTLP. Practical takeaway: instrument your application via an OpenTelemetry-compatible layer, and you will be able to change the backend dashboard without rewriting instrumentation.

Langfuse and Phoenix already accept OTLP traces. This protects against vendor lock-in: once you tag spans, you can send them either to Langfuse or to your own Grafana Tempo.

Practical production pitfalls

  • Do not block the main thread. Tracing sending should be asynchronous and batched. Synchronous flush on each request will add latency for no reason.
  • Sample at scale. Logging 100% at millions of requests per day is costly in storage and money. Full trace for errors and slow requests, sample 5-10% for the rest — a reasonable start (pick numbers according to your volume).
  • Calculate cost yourself, don’t rely only on the dashboard. Model pricing changes, and the dashboard may have outdated pricing baked in. Keep your own pricing table as the source of truth.
  • Mask PII before sending. Client-side scrubbing is more reliable than on the dashboard side.
  • Set budget alerts. A $/hour threshold catches a runaway retry loop earlier than the finance department notices.

Minimal deployment checklist

  1. Connect one tool (Helicone in a minute or Langfuse SDK) — start collecting anything.
  2. Add metadata to each call: user_id, feature, env.
  3. Set up chain tracing — nested spans for agents and RAG.
  4. Enable token and cost tracking, tie it to your pricing table.
  5. Set up budget and error-rate alerts.
  6. Create a dataset and evals for prompt regression before deployment.

Tools and their updates can be conveniently monitored via the REDDYX catalog — new OSS alternatives also appear there if Langfuse or Helicone don’t suit you.

Frequently asked questions

What is LLM observability in simple terms?

This is the collection and analysis of data about what happens inside an LLM application in production: which prompts go to the model, how many tokens and money are spent, how agent chains are executed and how good the answers are. Unlike regular monitoring, observability shows content and quality, not just service availability.

Langfuse or Helicone — which to choose?

Helicone — if you need quick connection speed (you change base_url and see logs in a minute) and caching. Langfuse — if you need deep tracing of multi-step agents, prompt versioning, and full self-hosted data. They can be used together.

Does observability significantly slow down the application?

With proper configuration — almost none. Traces are sent asynchronously and in batches, not blocking the main thread. Proxy tools like Helicone add one network hop, which is eliminated by self-hosting deployment next to the application.

How to control LLM costs in production?

Log tokens per request with breakdown by model, feature, and user, maintain your own up-to-date pricing table, and set alerts for a $/hour threshold. This catches runaway retry loops before they blow the budget.

If you catch fresh tools for the LLM stack earlier than others — check out the Telegram channel REDDYX AI — new repositories every 30-60 minutes.

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram

← All articles

LLM Monitoring in Production 2026: Langfuse, Helicone, and Other Observability Tools | REDDYX AI