Your AI agent works nice in testing. Then you definately ship it, and one thing kinda breaks. A device known as loops endlessly, prefer it by no means learns. A retrieval step returns rubbish and prices spike. You don’t have any thought why, in any respect.<\/p>\n
That\u2019s the agent observability downside. And for those who\u2019re constructing with LLMs, it is advisable clear up it earlier than manufacturing, not after. This publish kinda breaks down three of the most-used observability instruments: LangSmith, Langfuse and Arize. We\u2019ll set every one up, hint the identical agent and examine what you truly get.\u00a0<\/p>\n
What’s Agent Observability?<\/h2>\n
Conventional utility monitoring tracks requests, errors, and latency, however that’s not sufficient for Brokers.<\/p>\n
An Agent could name a number of instruments in sequence, with every LLM step having its personal immediate, token utilization, latency, and potential failure level. A single failed retrieval or device name can result in an incorrect remaining response.<\/p>\n
Agent observability<\/span> captures the complete execution graph: each step, resolution, LLM enter and output, device name, arguments, outcomes, token utilization, latency, and analysis rating. With out this visibility, debugging agent conduct turns into guesswork.<\/p>\n We are going to make the most of a quite simple LangChain agent to check them. The agent receives a query from the consumer, retrieves related context, and responds utilizing a number of instruments to supply a solution. \u00a0<\/p>\n First, it is advisable create the take a look at agent and for that set up all of the required libraries. \u00a0\u00a0<\/p>\n Let\u2019s have a look at the bottom agent with two strategies (search_docs and get_order_status). This may act as our foundational base for comparability with the three observability instruments.\u00a0<\/p>\n “”” Swap the OPENAI_API_KEY env var or name build_agent() from any demo file. import os<\/p>\n from dotenv import load_dotenv load_dotenv()<\/p>\n @device for key phrase, content material in docs.gadgets(): return f”Discovered common docs associated to: {question}”<\/p>\n @device return statuses.get( def build_agent() -> AgentExecutor: instruments = [search_docs, get_order_status]<\/p>\n immediate = ChatPromptTemplate.from_messages( agent = create_openai_tools_agent(llm, instruments, immediate)<\/p>\n return AgentExecutor( TEST_QUESTIONS = [ if __name__ == “__main__”: for query in TEST_QUESTIONS: outcome = executor.invoke({“enter”: query})<\/p>\n print(f”A: {outcome[‘output’]}”)<\/p>\n This creates a candidate agent that will also be used with every of the instruments. The primary device we are going to discover would be the one supplied by LangSmith.\u00a0<\/p>\n The LangChain staff has developed LangSmith. In case you are utilizing LangChain, then integration might be fast and straightforward.\u00a0<\/p>\n “”” Setup:<\/p>\n pip set up langsmith<\/p>\n Set LANGCHAIN_API_KEY in your .env file.<\/p>\n The way it works:<\/p>\n LangSmith hooks into LangChain’s callback system through env vars, so no code import os<\/p>\n from dotenv import load_dotenv<\/p>\n from agent_base import TEST_QUESTIONS, build_agent<\/p>\n load_dotenv()<\/p>\n # Allow LangSmith tracing. These two vars are all you want. # LANGCHAIN_API_KEY have to be set in your .env or setting.<\/p>\n def run_with_metadata( def primary(): executor = build_agent()<\/p>\n for query in TEST_QUESTIONS: outcome = run_with_metadata(executor, query)<\/p>\n print(f”A: {outcome[‘output’]}n”)<\/p>\n print(“Achieved. Open LangSmith to examine the complete hint tree for every run.”)<\/p>\n if __name__ == “__main__”: LangSmith routinely connects to LangChain\u2019s callback system with out the necessity for decorators or wrappers to see every run seem in your mission dashboard.\u00a0<\/p>\n LangSmith\u2019s hint view exhibits the complete agent execution tree, from the preliminary name to device use, LLM responses, and remaining output. Every node contains inputs, outputs, and latency.<\/p>\n You possibly can tag runs, add metadata, filter by final result, save runs as datasets, and run evaluations. That is helpful when bettering prompts or retrieval logic.<\/p>\n The immediate playground is one other sturdy function. You possibly can open any hint, edit the immediate inline, and rerun it to debug poor LLM efficiency.<\/p>\n LangSmith\u2019s limitations seem at scale. The free tier has caps, and integration takes extra effort if you’re not utilizing LangChain, although OpenTelemetry is supported.<\/p>\n Langfuse is the open-source various right here. You possibly can both host it in your server, or use their cloud service. It additionally integrates with all frameworks like LangChain, LlamaIndex, uncooked OpenAI APIs, and so forth.\u00a0<\/p>\n # Learn this Doc-string for putting in the dependencies and their setup Setup:<\/p>\n pip set up langfuse<\/p>\n Set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY in your .env file.<\/p>\n LANGFUSE_HOST defaults to https:\/\/cloud.langfuse.com; override for self-hosted.<\/p>\n Key variations from LangSmith:<\/p>\n – Callback handler is handed per-invoke for extra express management. import os<\/p>\n from dotenv import load_dotenv from agent_base import TEST_QUESTIONS, build_agent<\/p>\n load_dotenv()<\/p>\n def build_handler( def score_trace( lf.rating( lf.flush()<\/p>\n print(f”Scored hint {trace_id}: {rating}”)<\/p>\n def run_single_session( for query in TEST_QUESTIONS: outcome = executor.invoke( print(f”A: {outcome[‘output’]}n”)<\/p>\n # handler.get_trace_id() returns the hint ID for the final run. # Flush ensures traces are despatched earlier than the method exits. return trace_ids<\/p>\n def primary(): executor = build_agent() trace_ids = run_single_session(executor, session_id)<\/p>\n # Instance: programmatically rating the primary hint. print(f”nDone. Discover all runs below session ‘{session_id}’ in your Langfuse dashboard.”)<\/p>\n if __name__ == “__main__”: You possibly can go callback handlers each run, which is somewhat bit extra express than LangSmith is, however gives better flexibility since you’ll be able to assign consumer IDs, session IDs, and customized metadata while you invoke it.\u00a0<\/p>\n Langfuse has a extremely good analysis workflow as effectively; you’ll be able to add scores after the hint has been accomplished.\u00a0<\/p>\n from langfuse import Langfuse<\/p>\n lf = Langfuse()<\/p>\n # Rating a particular hint by ID. This works together with human evaluations of the responses your staff scores, permitting you to get aggregated analysis metrics over time.\u00a0<\/p>\n Customers can set up their classes by connecting them, so brokers can simply observe conversations throughout a number of turns. All of the traces in a person consumer session are linked within the utility, which lets you observe a complete dialog in a single place.\u00a0<\/p>\n Initially developed as a platform for monitoring standard machine studying fashions, Arize is now able to observing each language fashions and brokers. The truth that it was initially created to assist groups deploy fashions into manufacturing at scale has remained intact.\u00a0<\/p>\n Along with utilizing the OpenInference normal as its measurement scheme, Arize integrates with OpenTelemetry for instrumentation. Configuring Arize is extra difficult than it’s for many suppliers.\u00a0<\/p>\n # Learn this Doc-string for putting in the dependencies and their setup Setup:<\/p>\n pip set up arize-otel openinference-instrumentation-langchain<\/p>\n Set ARIZE_SPACE_ID and ARIZE_API_KEY in your .env file.<\/p>\n Key variations from the others:<\/p>\n – Makes use of OpenTelemetry below the hood, so it integrates with current OTel stacks. import os<\/p>\n from arize.otel import register from agent_base import TEST_QUESTIONS, build_agent<\/p>\n load_dotenv()<\/p>\n def setup_arize_tracing(): LangChainInstrumentor().instrument(tracer_provider=tracer_provider)<\/p>\n return tracer_provider<\/p>\n def run_with_attributes( tracer = hint.get_tracer(__name__)<\/p>\n with tracer.start_as_current_span(“agent_run”) as span: outcome = executor.invoke({“enter”: query})<\/p>\n span.set_attribute(“response.textual content”, outcome[“output”])<\/p>\n return outcome<\/p>\n def primary(): setup_arize_tracing()<\/p>\n executor = build_agent()<\/p>\n # Simulate two consumer segments to exhibit cohort evaluation in Arize. for query, phase in zip(TEST_QUESTIONS, segments): outcome = run_with_attributes( print(f”A: {outcome[‘output’]}n”)<\/p>\n print(“Achieved. In Arize, use the cohort filter to check premium vs normal responses.”) if __name__ == “__main__”: The instrumentation is international like that of LangSmith, but it surely turns into a part of OpenTelemetry\u2019s general measurement framework. Subsequently, Arize can make the most of the prevailing observability stack of your group whatever the precise framework you utilize (i.e., Jaeger, Grafana, and so forth.).\u00a0<\/p>\n To be fully open, there isn’t any single proper device for all use instances; all of it is determined by the place you might be within the improvement cycle and what your staff wants. \u00a0<\/p>\n Function Setup complexity Framework help Self-hosting Hint visualization Analysis \/ scoring Manufacturing monitoring Multi-turn \/ classes Open supply Free tier Finest for Use LangSmith<\/span> if you’re constructing with LangChain and need the quickest setup for immediate debugging and iteration.<\/p>\n Use Langfuse<\/span> for those who want self-hosting, stronger knowledge possession, multi-framework help, or session-level monitoring for conversational brokers.<\/p>\n Use Arize<\/span> when your agent is shifting into manufacturing and also you want monitoring, drift detection, cohorts, and alerts.<\/p>\n Agent observability is a kind of stuff you solely remorse skipping after one thing goes fallacious in manufacturing. Tracing an agent run after the very fact, with none instrumentation is like debugging a distributed system with print statements. \u00a0<\/p>\n All three instruments coated listed below are manufacturing prepared. They every have a free path in. And so they every take below half-hour to combine with a LangChain agent. There\u2019s no good purpose to ship an unobservable agent anymore.\u00a0<\/p>\n Choose the device that matches your present stage. Add scoring early, even informally. And when your agent begins doing one thing bizarre at 2am, you\u2019ll be glad you probably did.\u00a0<\/p>\nSetting Up the Check Agent<\/h2>\n
![\"Dependencies](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2026\/06\/image-2.webp\")
<\/figure>\n<\/div>\n
\nBase agent used throughout all three observability demos.<\/p>\n
\n“””<\/p>\n
\nfrom langchain.brokers import AgentExecutor, create_openai_tools_agent
\nfrom langchain.instruments import device
\nfrom langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
\nfrom langchain_openai import ChatOpenAI<\/p>\n
\ndef search_docs(question: str) -> str:
\n “””Search inner docs for related data.”””
\n # Simulated retrieval \u2014 swap along with your precise vector retailer
\n docs = {
\n “refund”: (
\n “Refunds are processed inside 5-7 enterprise days. ”
\n “Gadgets have to be returned inside 30 days.”
\n ),
\n “delivery”: (
\n “Customary delivery takes 3-5 enterprise days. ”
\n “Categorical is 1-2 days.”
\n ),
\n “account”: (
\n “You possibly can reset your password through the login web page. ”
\n “Contact help for account points.”
\n ),
\n }<\/p>\n
\n if key phrase in question.decrease():
\n return content material<\/p>\n
\ndef get_order_status(order_id: str) -> str:
\n “””Search for the standing of an order by ID.”””
\n # Simulated order lookup
\n statuses = {
\n “ORD-001”: “Shipped \u2014 anticipated supply 2026-05-30”,
\n “ORD-002”: “Processing \u2014 not but shipped”,
\n “ORD-003”: “Delivered on 2026-05-25″,
\n }<\/p>\n
\n order_id,
\n f”Order {order_id} not discovered within the system.”,
\n )<\/p>\n
\n llm = ChatOpenAI(
\n mannequin=”gpt-4o”,
\n temperature=0,
\n api_key=os.environ[“OPENAI_API_KEY”],
\n )<\/p>\n
\n [
\n (
\n “system”,
\n “You are a helpful customer support assistant. ”
\n “Use tools when needed.”,
\n ),
\n (“user”, “{input}”),
\n MessagesPlaceholder(variable_name=”agent_scratchpad”),
\n ]
\n )<\/p>\n
\n agent=agent,
\n instruments=instruments,
\n verbose=False,
\n )<\/p>\n
\n “What are the refund policies?”,
\n “What is the status of order ORD-002?”,
\n “How long does shipping take?”,
\n]<\/p>\n
\n executor = build_agent()<\/p>\n
\n print(f”nQ: {query}”)<\/p>\nLangSmith: Native Langchain Tracing<\/h2>\n
\nLangSmith observability demo.<\/p>\n
\nmodifications are wanted past the 2 os.environ strains under.
\n“””<\/p>\n
\nos.environ[“LANGCHAIN_TRACING_V2”] = “true”
\nos.environ[“LANGCHAIN_PROJECT”] = “agent-observability-demo”<\/p>\n
\n executor,
\n query: str,
\n user_id: str = “demo-user”,
\n):
\n “””Run the agent and fix per-run metadata through config.”””
\n return executor.invoke(
\n {“enter”: query},
\n config={
\n “metadata”: {
\n “user_id”: user_id,
\n “supply”: “langsmith_demo”,
\n },
\n # Non-compulsory: tag runs for filtering within the dashboard.
\n “tags”: [“observability-blog”, “demo”],
\n },
\n )<\/p>\n
\n print(“=== LangSmith Demo ===”)
\n print(“Traces will seem at: https:\/\/smith.langchain.com”)
\n print(f”Mission: {os.environ[‘LANGCHAIN_PROJECT’]}n”)<\/p>\n
\n print(f”Q: {query}”)<\/p>\n
\n primary()<\/p>\nWhat you\u2019ll see on the dashboard:\u00a0<\/h3>\n
Langfuse: Open Supply and Framework-Agnostic<\/h2>\n
\n“””
\nLangfuse observability demo.<\/p>\n
\n– Native session grouping for multi-turn conversations.
\n– You possibly can rating any hint after the very fact through the Langfuse consumer.
\n“””<\/p>\n
\nfrom langfuse import Langfuse
\nfrom langfuse.callback import CallbackHandler<\/p>\n
\n session_id: str,
\n user_id: str = “demo-user”,
\n) -> CallbackHandler:
\n return CallbackHandler(
\n public_key=os.environ[“LANGFUSE_PUBLIC_KEY”],
\n secret_key=os.environ[“LANGFUSE_SECRET_KEY”],
\n host=os.getenv(“LANGFUSE_HOST”, “https:\/\/cloud.langfuse.com”),
\n session_id=session_id,
\n user_id=user_id,
\n metadata={“supply”: “langfuse_demo”},
\n tags=[“observability-blog”, “demo”],
\n )<\/p>\n
\n trace_id: str,
\n rating: float,
\n remark: str = “”,
\n):
\n “””Add a correctness rating to a hint after reviewing the output.”””
\n lf = Langfuse(
\n public_key=os.environ[“LANGFUSE_PUBLIC_KEY”],
\n secret_key=os.environ[“LANGFUSE_SECRET_KEY”],
\n host=os.getenv(“LANGFUSE_HOST”, “https:\/\/cloud.langfuse.com”),
\n )<\/p>\n
\n trace_id=trace_id,
\n title=”correctness”,
\n worth=rating,
\n remark=remark,
\n )<\/p>\n
\n executor,
\n session_id: str,
\n):
\n “””Run all take a look at questions in a single session so that they’re linked within the UI.”””
\n handler = build_handler(session_id=session_id)
\n trace_ids = []<\/p>\n
\n print(f”Q: {query}”)<\/p>\n
\n {“enter”: query},
\n config={“callbacks”: [handler]},
\n )<\/p>\n
\n trace_ids.append(handler.get_trace_id())<\/p>\n
\n # That is essential in batch jobs.
\n handler.flush()<\/p>\n
\n print(“=== Langfuse Demo ===”)
\n print(f”Dashboard: {os.getenv(‘LANGFUSE_HOST’, ‘https:\/\/cloud.langfuse.com’)}n”)<\/p>\n
\n session_id = “demo-session-001”<\/p>\n
\n if trace_ids and trace_ids[0]:
\n print(“nScoring first hint for example:”)
\n score_trace(trace_ids[0], rating=0.9, remark=”Reply was correct”)<\/p>\n
\n primary()<\/p>\n\u00a0Analysis Workflow\u00a0<\/h4>\n
\nlf.rating(
\n trace_id=”trace-abc123″,
\n title=”correctness”,
\n worth=0.9,
\n remark=”Reply was correct however barely verbose”,
\n)<\/p>\nArize: Manufacturing-Grade ML Observability<\/h2>\n
Using OpenInference\u00a0<\/h3>\n
\n“””
\nArize observability demo.<\/p>\n
\n– Instrumentation is international like LangSmith, not per-invoke like Langfuse.
\n– Finest-in-class manufacturing monitoring: drift detection, cohort evaluation, alerting.
\n– Phoenix, arize-phoenix, is the free native sibling for improvement use.
\n“””<\/p>\n
\nfrom dotenv import load_dotenv
\nfrom openinference.instrumentation.langchain import LangChainInstrumentor<\/p>\n
\n “””Register Arize because the OTel tracer supplier and instrument LangChain globally.”””
\n tracer_provider = register(
\n space_id=os.environ[“ARIZE_SPACE_ID”],
\n api_key=os.environ[“ARIZE_API_KEY”],
\n project_name=”agent-observability-demo”,
\n )<\/p>\n
\n executor,
\n query: str,
\n user_segment: str = “normal”,
\n):
\n “””Run the agent and fix span attributes for cohort evaluation in Arize.”””
\n from opentelemetry import hint<\/p>\n
\n span.set_attribute(“consumer.phase”, user_segment)
\n span.set_attribute(“question.textual content”, query)
\n span.set_attribute(“demo.supply”, “arize_demo”)<\/p>\n
\n print(“=== Arize Demo ===”)
\n print(“Traces will seem at: https:\/\/app.arize.com”)
\n print(“Mission: agent-observability-demon”)<\/p>\n
\n segments = [“premium”, “standard”, “standard”]<\/p>\n
\n print(f”Q: {query} [segment={segment}]”)<\/p>\n
\n executor,
\n query,
\n user_segment=phase,
\n )<\/p>\n
\n print(“Arrange screens on the Arize dashboard to alert on response high quality drift.”)<\/p>\n
\n primary()<\/p>\nWhich Ought to You Choose for Agent Observability?<\/h2>\n
\n LangSmith
\n Langfuse
\n Arize<\/p>\n
\n Minimal (2 env vars)
\n Low (callback handler)
\n Most boilerplate<\/p>\n
\n LangChain-native; others through OTel
\n Any framework
\n Any framework through OTel<\/p>\n
\n Restricted
\n First-class (Docker Compose)
\n Phoenix solely (native dev)<\/p>\n
\n Wonderful tree view
\n Good, session-linked
\n Good, OTel-standard<\/p>\n
\n Dataset + playground
\n Session-level human scores
\n Rubric-based evals<\/p>\n
\n Primary
\n Primary
\n Drift, alerting, cohorts<\/p>\n
\n Thread-level
\n Native session grouping
\n Hint-level solely<\/p>\n
\n Proprietary
\n Absolutely open supply
\n Phoenix is OSS; platform isn\u2019t<\/p>\n
\n Restricted traces\/month
\n Beneficiant (self-host = limitless)
\n Restricted<\/p>\n
\n LangChain dev & iteration
\n Knowledge possession + any framework
\n Manufacturing-scale monitoring<\/p>\n<\/figure>\n<\/div>\nConclusion<\/h2>\n
![\"Riya](\"https:\/\/av-eks-lekhak.s3.amazonaws.com\/media\/lekhak-profile-images\/converted_image_6IQzGzn.webp\")
<\/p><\/div><\/div>\n