On this article, you’ll find out how the seven layers of a manufacturing AI agent stack match collectively, from the inspiration mannequin right down to deployment infrastructure.<\/p>\n
Matters we’ll cowl embrace:<\/p>\n
What every layer of the stack does, from the inspiration mannequin and orchestration framework by way of reminiscence, retrieval, instruments, observability, and deployment.
\nFind out how to implement every layer with working code, together with a stateful agent, a reminiscence system, a RAG pipeline, customized instruments, and tracing.
\nWhich mixture of applied sciences to make use of at every layer relying on whether or not you might be prototyping, scaling a startup, or working in an enterprise atmosphere.<\/p>\n
![\"The](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2026\/06\/MLM-Shittu-The-AI-Agent-Tech-Stack-scaled.png\")
<\/p>\n
Introduction<\/h2>\n
Image this: you ask an AI agent to analysis three opponents, pull the pricing knowledge from every of their web sites, summarize the findings right into a structured report, and drop it in a Slack channel by 9am. You hit enter. Thirty seconds later, the report is there.<\/p>\n
What simply occurred beneath the hood will not be magic, and it isn’t one factor. It’s seven distinct layers of expertise working in sequence, every one dealing with a selected job, every one able to breaking in its personal particular manner. The mannequin on the prime will get all the eye. The six layers beneath it are what decide whether or not the agent truly works.<\/p>\n
In accordance with Gartner, 40% of enterprise purposes will likely be built-in with task-specific AI brokers by the tip of 2026, up from lower than 5% in 2025. That isn’t a gradual curve. That may be a near-vertical adoption line, and the engineers and technical leads chargeable for these deployments want to know the complete stack, not simply the layer they occur to personal.<\/p>\n
This text goes by way of every layer so as, from the inspiration mannequin right down to deployment infrastructure. By the tip, you’ll know what every bit is, why it exists, how the layers join to one another, and what to really use at every degree.<\/p>\n
Layer 1: The Basis Mannequin<\/h2>\n
The inspiration mannequin is the cognitive core of an agent. It’s the place reasoning occurs, language is known, and choices about what to do subsequent are made. Every thing else within the stack is both feeding context into it or appearing on what it produces.<\/p>\n
In sensible phrases, your principal choices in 2026 are OpenAI\u2019s GPT-5.5, Anthropic\u2019s Claude Sonnet 4.6 (or Claude Opus 4.8 for tougher reasoning), Google\u2019s Gemini 3.1 Professional, and open-weight fashions like Meta\u2019s Llama 4 and Mistral Massive 3. Every has trade-offs value understanding earlier than you commit.<\/p>\n
GPT-5.5 is quick for on a regular basis calls and dependable at tool-calling, and it has probably the most mature ecosystem of integrations and the widest group of builders who’ve already run into and solved the sting circumstances you’ll encounter. Claude Sonnet 4.6 handles lengthy paperwork and nuanced instruction-following nicely at a lower cost level than Anthropic\u2019s Opus tier, which issues in document-heavy workflows; attain for Claude Opus 4.8 when a process wants deeper, longer-horizon reasoning. Gemini 3.1 Professional has a 1 million token context window, which is related in case your agent must course of massive codebases or prolonged data bases in a single cross. Open-weight fashions like Llama 4 offer you full management over deployment and knowledge residency, at the price of the infrastructure overhead of working them your self.<\/p>\n
There isn’t a longer a tough break up between \u201ccommonplace\u201d and \u201creasoning\u201d mannequin households, the best way there was in 2025; OpenAI, Anthropic, and Google have every folded reasoning right into a single mannequin that decides how lengthy to suppose. GPT-5.5 ships with adjustable reasoning effort ranges (from none as much as xhigh), and the identical applies to Claude\u2019s effort parameter and Gemini\u2019s pondering ranges. For many agent workflows, the default or low-effort setting is the precise selection: quick and low-cost. For duties that require cautious planning or mathematical reasoning, dialling the hassle degree up earns again its value in correctness.<\/p>\n
Layer 2: The Orchestration Framework<\/h2>\n
If the inspiration mannequin is the mind, the orchestration framework is the nervous system. It handles the management stream: deciding what the agent ought to do subsequent, when it ought to name a software, the way it ought to deal with the consequence, and the way the entire reasoning loop stays coherent throughout a number of steps.<\/p>\n
The sample that almost all frameworks implement is known as ReAct (Reasoning and Appearing). The agent produces a thought, decides on an motion, executes the motion by way of a software, observes the consequence, after which thinks once more. This loop repeats till the agent produces a closing reply. It sounds easy. In apply, it’s the place most manufacturing failures happen: the agent calls the flawed software, will get caught in a loop, or fails to recognise when it has sufficient info to cease.<\/p>\n
LangChain is probably the most extensively adopted framework. It provides a big ecosystem of integrations and good documentation. The criticism that it provides an excessive amount of abstraction is truthful on the prototype stage, however much less related when you want the options that abstraction gives. LangGraph, constructed by the identical crew, is best fitted to stateful multi-agent workflows the place you want fine-grained management over the execution graph. In case your agent entails a number of specialists coordinating on a process, LangGraph is the cleaner selection.
\nCrewAI is designed particularly for multi-agent coordination. It allows you to outline brokers with roles, assign them duties, and have them collaborate inside a structured workflow. It’s higher-level than LangGraph and quicker to get working, however provides you much less management over the execution particulars. AutoGen, from Microsoft, takes a conversational method to multi-agent techniques. Brokers work together with one another by way of a message-passing interface, which makes the interplay logic very readable.
\nSemantic Kernel is Microsoft\u2019s enterprise-focused choice, with production-ready help for C#, Python, and Java. If you’re working in an enterprise atmosphere already working on the Microsoft stack, it suits naturally. LlamaIndex began as a doc ingestion and retrieval framework and has since grown right into a full agent framework, with significantly robust help for RAG-heavy workflows.<\/p>\n
The precise selection relies on what your agent must do. For a single-agent process runner: LangGraph or LangChain. For a coordinated crew of specialised brokers: CrewAI or AutoGen. For enterprise environments: Semantic Kernel. For document-heavy retrieval workflows: LlamaIndex.<\/p>\n
Here’s a minimal working agent in LangGraph that handles software use and maintains state.<\/p>\n
Stipulations:<\/p>\n
\npip set up langgraph langchain-openai langchain-community python-dotenv<\/p>\n
pip <\/span>set up <\/span>langgraph <\/span>langchain<\/span>–<\/span>openai <\/span>langchain<\/span>–<\/span>group <\/span>python<\/span>–<\/span>dotenv<\/span><\/p>\n<\/div><\/div><\/div>\n Find out how to run: Save as agent.py, add your OPENAI_API_KEY to a .env file, then run python agent.py<\/p>\n \n# agent.py
1<\/p>\n 2<\/p>\n 3<\/p>\n 4<\/p>\n 5<\/p>\n 6<\/p>\n 7<\/p>\n 8<\/p>\n 9<\/p>\n 10<\/p>\n 11<\/p>\n 12<\/p>\n 13<\/p>\n 14<\/p>\n 15<\/p>\n 16<\/p>\n 17<\/p>\n 18<\/p>\n 19<\/p>\n 20<\/p>\n 21<\/p>\n 22<\/p>\n 23<\/p>\n 24<\/p>\n 25<\/p>\n 26<\/p>\n 27<\/p>\n 28<\/p>\n 29<\/p>\n 30<\/p>\n 31<\/p>\n 32<\/p>\n 33<\/p>\n 34<\/p>\n 35<\/p>\n 36<\/p>\n 37<\/p>\n 38<\/p>\n<\/div>\n # agent.py<\/span><\/p>\n # Minimal stateful agent with software use constructed on LangGraph<\/span><\/p>\n # Python 3.10+ | LangGraph 0.2+ | LangChain 0.3+<\/span><\/p>\n \u00a0<\/p>\n import <\/span>os<\/span><\/p>\n from <\/span>dotenv <\/span>import <\/span>load_dotenv<\/span><\/p>\n from <\/span>langchain_openai <\/span>import <\/span>ChatOpenAI<\/span><\/p>\n from <\/span>langchain_community<\/span>.<\/span>instruments <\/span>import <\/span>DuckDuckGoSearchRun<\/span><\/p>\n from <\/span>langchain_core<\/span>.<\/span>messages <\/span>import <\/span>HumanMessage<\/span><\/p>\n from <\/span>langgraph<\/span>.<\/span>prebuilt <\/span>import <\/span>create_react<\/span>_<\/span>agent<\/p>\n \u00a0<\/p>\n # Load API key from .env file<\/span><\/p>\n load_dotenv<\/span>(<\/span>)<\/span><\/p>\n \u00a0<\/p>\n # Initialize the language mannequin<\/span><\/p>\n # temperature=0 for deterministic, targeted responses in agentic duties<\/span><\/p>\n llm<\/span> <\/span>=<\/span> <\/span>ChatOpenAI<\/span>(<\/span><\/p>\n \u00a0\u00a0\u00a0\u00a0<\/span>mannequin<\/span>=<\/span>“gpt-5.5”<\/span>,<\/span><\/p>\n \u00a0\u00a0\u00a0\u00a0<\/span>temperature<\/span>=<\/span>0<\/span>,<\/span><\/p>\n \u00a0\u00a0\u00a0\u00a0<\/span>api_key<\/span>=<\/span>os<\/span>.<\/span>getenv<\/span>(<\/span>“OPENAI_API_KEY”<\/span>)<\/span><\/p>\n )<\/span><\/p>\n \u00a0<\/p>\n # Register the instruments the agent can use<\/span><\/p>\n # DuckDuckGoSearchRun requires no API key — good for improvement<\/span><\/p>\n instruments<\/span> <\/span>=<\/span> <\/span>[<\/span>DuckDuckGoSearchRun<\/span>(<\/span>)<\/span>]<\/span><\/p>\n \u00a0<\/p>\n # create_react_agent from LangGraph wires collectively the LLM,<\/span><\/p>\n # instruments, and a built-in ReAct loop — no boilerplate required<\/span><\/p>\n agent<\/span> <\/span>=<\/span> <\/span>create_react_agent<\/span>(<\/span>llm<\/span>,<\/span> <\/span>instruments<\/span>)<\/span><\/p>\n \u00a0<\/p>\n # Run the agent with a pattern question<\/span><\/p>\n # The agent will resolve whether or not to make use of a software primarily based on the query<\/span><\/p>\n consequence<\/span> <\/span>=<\/span> <\/span>agent<\/span>.<\/span>invoke<\/span>(<\/span>{<\/span><\/p>\n \u00a0\u00a0\u00a0\u00a0<\/span>“messages”<\/span>:<\/span> <\/span>[<\/span>HumanMessage<\/span>(<\/span>content<\/span>=<\/span>“What is the current market cap of Nvidia?”<\/span>)<\/span>]<\/span><\/p>\n }<\/span>)<\/span><\/p>\n \u00a0<\/p>\n # The ultimate response is the final message within the messages checklist<\/span><\/p>\n print<\/span>(<\/span>consequence<\/span>[<\/span>“messages”<\/span>]<\/span>[<\/span>–<\/span>1<\/span>]<\/span>.<\/span>content material<\/span>)<\/span><\/p>\n<\/div><\/div><\/div>\n What this does: create_react_agent handles the complete ReAct loop routinely. The agent receives the query, decides it wants present knowledge, calls the DuckDuckGo search software, reads the consequence, and synthesizes a closing reply. The messages checklist within the output accommodates the complete hint of that reasoning course of.<\/p>\n Statelessness is the default habits of any LLM. Each name begins from scratch, with no data of what got here earlier than except you explicitly cross that context in. For a one-shot query, that’s superb. For an agent that should observe a dialog, keep in mind a person\u2019s preferences, or construct on work it did yesterday, it’s a basic drawback.<\/p>\n In accordance with Atlan\u2019s analysis on AI agent reminiscence, 95% of enterprise generative AI pilots delivered zero measurable ROI in 2025, with failure attributed to context readiness fairly than mannequin high quality. Brokers are failing not as a result of the mannequin is flawed, however as a result of the reminiscence layer will not be there.<\/p>\n There are 4 varieties of reminiscence in a manufacturing agent, and every one handles a unique job:<\/p>\n Working reminiscence (in-context) is the energetic context window. It holds the present dialog, any paperwork you’ve handed in, and the outcomes of current software calls. It’s quick and requires no infrastructure, however it’s session-bound. When the session ends, it’s gone. Right here is implement working and episodic reminiscence collectively utilizing LangChain\u2019s really helpful sample for LangChain 0.3+:<\/p>\n Stipulations:<\/p>\n \npip set up langchain langchain-openai python-dotenv<\/p>\n pip <\/span>set up <\/span>langchain <\/span>langchain<\/span>–<\/span>openai <\/span>python<\/span>–<\/span>dotenv<\/span><\/p>\n<\/div><\/div><\/div>\n Find out how to run: Save as reminiscence.py, guarantee your .env has OPENAI_API_KEY, then run python reminiscence.py<\/p>\n \n# reminiscence.py
1<\/p>\n 2<\/p>\n 3<\/p>\n 4<\/p>\n 5<\/p>\n 6<\/p>\n 7<\/p>\n 8<\/p>\n 9<\/p>\n 10<\/p>\n 11<\/p>\n 12<\/p>\n 13<\/p>\n 14<\/p>\n 15<\/p>\n 16<\/p>\n 17<\/p>\n 18<\/p>\n 19<\/p>\n 20<\/p>\n 21<\/p>\n 22<\/p>\n 23<\/p>\n 24<\/p>\n 25<\/p>\n 26<\/p>\n 27<\/p>\n 28<\/p>\n 29<\/p>\n 30<\/p>\n 31<\/p>\n 32<\/p>\n 33<\/p>\n 34<\/p>\n 35<\/p>\n 36<\/p>\n 37<\/p>\n 38<\/p>\n 39<\/p>\n 40<\/p>\n 41<\/p>\n 42<\/p>\n 43<\/p>\n 44<\/p>\n 45<\/p>\n 46<\/p>\n 47<\/p>\n 48<\/p>\n 49<\/p>\n 50<\/p>\n 51<\/p>\n 52<\/p>\n 53<\/p>\n 54<\/p>\n 55<\/p>\n 56<\/p>\n 57<\/p>\n 58<\/p>\n 59<\/p>\n 60<\/p>\n 61<\/p>\n 62<\/p>\n 63<\/p>\n 64<\/p>\n 65<\/p>\n 66<\/p>\n 67<\/p>\n 68<\/p>\n 69<\/p>\n 70<\/p>\n 71<\/p>\n 72<\/p>\n 73<\/p>\n 74<\/p>\n 75<\/p>\n 76<\/p>\n 77<\/p>\n 78<\/p>\n 79<\/p>\n 80<\/p>\n 81<\/p>\n 82<\/p>\n 83<\/p>\n 84<\/p>\n 85<\/p>\n 86<\/p>\n 87<\/p>\n 88<\/p>\n 89<\/p>\n 90<\/p>\n 91<\/p>\n<\/div>\n # reminiscence.py<\/span><\/p>\n # Working reminiscence + episodic reminiscence for persistent agent context<\/span><\/p>\n # Makes use of the present LangChain 0.3+ sample (legacy ConversationBufferMemory is deprecated)<\/span><\/p>\n \u00a0<\/p>\n import <\/span>os<\/span><\/p>\n import <\/span>json<\/span><\/p>\n from <\/span>datetime <\/span>import <\/span>datetime<\/span><\/p>\n from <\/span>dotenv <\/span>import <\/span>load_dotenv<\/span><\/p>\n from <\/span>langchain_openai <\/span>import <\/span>ChatOpenAI<\/span><\/p>\n from <\/span>langchain_core<\/span>.<\/span>messages <\/span>import <\/span>HumanMessage<\/span>,<\/span> <\/span>AIMessage<\/span>,<\/span> <\/span>SystemMessage<\/span>,<\/span> <\/span>trim_messages<\/span><\/p>\n \u00a0<\/p>\n load_dotenv<\/span>(<\/span>)<\/span><\/p>\n \u00a0<\/p>\n llm<\/span> <\/span>=<\/span> <\/span>ChatOpenAI<\/span>(<\/span><\/p>\n \u00a0\u00a0\u00a0\u00a0<\/span>mannequin<\/span>=<\/span>
\n# Minimal stateful agent with software use constructed on LangGraph
\n# Python 3.10+ | LangGraph 0.2+ | LangChain 0.3+
\n
\nimport os
\nfrom dotenv import load_dotenv
\nfrom langchain_openai import ChatOpenAI
\nfrom langchain_community.instruments import DuckDuckGoSearchRun
\nfrom langchain_core.messages import HumanMessage
\nfrom langgraph.prebuilt import create_react_agent
\n
\n# Load API key from .env file
\nload_dotenv()
\n
\n# Initialize the language mannequin
\n# temperature=0 for deterministic, targeted responses in agentic duties
\nllm = ChatOpenAI(
\n mannequin=”gpt-5.5″,
\n temperature=0,
\n api_key=os.getenv(“OPENAI_API_KEY”)
\n)
\n
\n# Register the instruments the agent can use
\n# DuckDuckGoSearchRun requires no API key — good for improvement
\ninstruments = [DuckDuckGoSearchRun()]
\n
\n# create_react_agent from LangGraph wires collectively the LLM,
\n# instruments, and a built-in ReAct loop — no boilerplate required
\nagent = create_react_agent(llm, instruments)
\n
\n# Run the agent with a pattern question
\n# The agent will resolve whether or not to make use of a software primarily based on the query
\nconsequence = agent.invoke({
\n “messages”: [HumanMessage(content=”What is the current market cap of Nvidia?”)]
\n})
\n
\n# The ultimate response is the final message within the messages checklist
\nprint(consequence[“messages”][-1].content material)<\/p>\nLayer 3: Reminiscence Methods<\/h2>\n
\nEpisodic reminiscence is a log of prior interactions. As described within the analysis on reminiscence varieties, episodic reminiscence shops what occurred: timestamp, process, actions taken, final result. That is what permits an agent to reply \u201cWhat did we work on final Tuesday?\u201d or \u201cWhat did the person say about this undertaking three periods in the past?\u201c
\nSemantic reminiscence is factual data saved externally, together with definitions, entity relationships, and domain-specific information that the mannequin was not skilled on. That is the place your RAG pipeline feeds in (extra on that within the subsequent layer).
\nProcedural reminiscence encodes workflows and tool-use patterns, repeatable behaviors the agent ought to all the time comply with. This lives within the system immediate or a version-controlled instruction file, and it shapes each response the agent produces.<\/p>\n
\n# Working reminiscence + episodic reminiscence for persistent agent context
\n# Makes use of the present LangChain 0.3+ sample (legacy ConversationBufferMemory is deprecated)
\n
\nimport os
\nimport json
\nfrom datetime import datetime
\nfrom dotenv import load_dotenv
\nfrom langchain_openai import ChatOpenAI
\nfrom langchain_core.messages import HumanMessage, AIMessage, SystemMessage, trim_messages
\n
\nload_dotenv()
\n
\nllm = ChatOpenAI(
\n mannequin=”gpt-5.5″,
\n temperature=0.2,
\n api_key=os.getenv(“OPENAI_API_KEY”)
\n)
\n
\n# \u2500\u2500 EPISODIC MEMORY STORE \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
\n# In manufacturing, exchange this checklist with a database (SQLite, Postgres, Redis).
\n# The construction right here: every episode is a dict with timestamp, person enter, and agent response.
\nepisodic_store: checklist[dict] = []
\n
\ndef save_episode(user_input: str, agent_response: str) -> None:
\n “””Save a dialog flip to the episodic retailer.”””
\n episodic_store.append({
\n “timestamp”: datetime.now().isoformat(),
\n “person”: user_input,
\n “agent”: agent_response
\n })
\n
\ndef load_recent_episodes(n: int = 5) -> str:
\n “””Retrieve the final N episodes as a formatted string for injection into context.”””
\n if not episodic_store:
\n return “No prior dialog historical past.”
\n current = episodic_store[-n:]
\n return “n”.be part of(
\n f”[{ep[‘timestamp’]}] Person: {ep[‘user’]} | Agent: {ep[‘agent’]}”
\n for ep in current
\n )
\n
\n# \u2500\u2500 WORKING MEMORY (IN-CONTEXT) \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
\n# We handle the message checklist ourselves and cross it by way of trim_messages
\n# earlier than every LLM name to remain throughout the mannequin’s context restrict.
\n# max_tokens=4000 leaves headroom for the mannequin’s response.
\nworking_memory: checklist = []
\n
\ndef chat(user_input: str) -> str:
\n “””
\n Ship a message to the agent.
\n Episodic historical past is injected into the system immediate.
\n Working reminiscence is trimmed earlier than every name to forestall context overflow.
\n “””
\n # Inject episodic reminiscence into the system immediate so the mannequin has long-term context
\n system = SystemMessage(content material=(
\n “You’re a useful, context-aware assistant.nn”
\n “Latest dialog historical past:n”
\n f”{load_recent_episodes()}”
\n ))
\n
\n # Add the brand new person message to working reminiscence
\n working_memory.append(HumanMessage(content material=user_input))
\n
\n # Trim working reminiscence to remain throughout the context window
\n # This compresses older messages fairly than dropping them totally
\n trimmed = trim_messages(
\n working_memory,
\n max_tokens=4000,
\n technique=”final”, # Hold the latest messages
\n token_counter=llm, # Use the mannequin’s tokenizer for correct counts
\n include_system=True,
\n allow_partial=False
\n )
\n
\n # Name the mannequin with system context + trimmed working reminiscence
\n response = llm.invoke([system] + trimmed)
\n reply = response.content material
\n
\n # Save the trade to episodic reminiscence and add the reply to working reminiscence
\n save_episode(user_input, reply)
\n working_memory.append(AIMessage(content material=reply))
\n
\n return reply
\n
\n
\n# \u2500\u2500 DEMO \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
\nif __name__ == “__main__”:
\n print(chat(“My identify is Alex and I am constructing a RAG pipeline for authorized paperwork.”))
\n print(chat(“What’s one of the best vector database for my use case?”))
\n print(chat(“What did I let you know I used to be constructing?”)) # Assessments episodic recall<\/p>\n