AI brokers have modified rather a lot within the final two years. The primary might solely reply one query at a time. Then got here multi-turn chat, the place the mannequin might hold some context throughout a session. In the present day, we’ve got long-horizon brokers. Methods that plan many steps, cut up work between sub-agents, hold context throughout an extended process, and run instruments in a protected sandbox.<\/p>\n
The NVIDIA AI-Q Blueprint is an open supply reference for this sort of agent. It’s constructed on LangChain Deep Brokers and the NVIDIA NeMo Agent Toolkit. You need to use it for fast cited solutions, or for longer analysis reviews with sources.<\/p>\n
This submit reveals you find out how to deploy AI-Q 2.0 on Oracle Cloud Infrastructure (OCI) utilizing Terraform to create the OCI sources and Helm to put in the workloads on OKE. By the tip, you should have a working AI-Q endpoint in your individual OCI tenancy, and one command to take all of it down if you find yourself carried out.<\/p>\n
Who that is for: Builders and platform engineers snug with Kubernetes, Terraform, and the shell, and who wish to run AI-Q on OCI moderately than on a laptop computer.\u00a0<\/p>\n
What you\u2019ll study: How AI-Q\u2019s multi-agent structure maps to OCI companies, plus the precise instructions to provision, deploy, and open the blueprint from begin to end.<\/p>\n
Extra background on the multi-agent structure (similar to intent router, shallow analysis agent, deep agent, planning sub-agent, researcher sub-agent), is on the AI-Q product web page and the NeMo Agent Toolkit docs.<\/p>\n
Stipulations<\/p>\n
Be sure you have:<\/p>\n
OCI tenancy entry with a compartment you’ll be able to deploy into, and sufficient service limits for:<\/p>\n
OKE: One enhanced cluster and one node pool<\/p>\n
Block Quantity: At the least 10 GB (dynamically provisioned by the OKE CSI driver for the in-cluster PostgreSQL)<\/p>\n
Load Balancer: One versatile<\/p>\n
Vault: One vault plus secrets and techniques<\/p>\n
API keys:<\/p>\n
NGC API key from construct.nvidia.com, format nvapi-\u2026 used each because the NVIDIA inference key and to authenticate to the NGC container registry (nvcr.io).<\/p>\n
Tavily API key from tavily.com, format tvly-\u2026<\/p>\n
Native instruments: terraform 1.5 or later, kubectl 1.28 or later, helm 3.x or later, the oci CLI arrange along with your API signing key<\/p>\n
Some fundamental data of Kubernetes, Helm charts, Terraform, and the shell. LangChain or NeMo Agent Toolkit expertise is sweet to have, however not required.<\/p>\n
Structure overview<\/h2>\n
AI-Q makes use of a multi-agent design. An intent router reads every consumer question and sends it to the proper workflow.<\/p>\n The blueprint is constructed to be extensible. Each layer (fashions, instruments, RAG backends, sub-agents, evaluators) might be swapped by way of YAML config or by way of the NeMo Agent Toolkit plugin system. We’ll use that extensibility in Elements 2 and three of this sequence.<\/p>\n The deployment makes use of Terraform for the OCI sources and Helm for the Kubernetes workloads. This provides a clear cut up between infrastructure and utility, and one terraform destroy is sufficient to take away every part later.<\/p>\n The Helm chart installs three workloads on OKE:<\/p>\n Backend (aiq-backend): A FastAPI-based agent server that runs the AI-Q workflow.<\/p>\n Frontend (aiq-frontend): A subsequent.js internet UI uncovered over NodePort 30080.<\/p>\n PostgreSQL (aiq-postgres): An in-cluster database for the job retailer, checkpoints, and summaries.<\/p>\n Whole time: round 20 to 25 minutes. The complete reference is in aiq-2.0\/README.md.<\/p>\n Step 1. Configure Terraform variables<\/p>\n Copy the instance file and edit it along with your tenancy particulars:<\/p>\n At minimal, set these variables in terraform.tfvars:<\/p>\n tenancy_ocid, compartment_id, area (for instance us-chicago-1)<\/p>\n user_ocid, fingerprint, private_key_path (similar values as your ~\/.oci\/config)<\/p>\n db_admin_password, used to bootstrap the in-cluster PostgreSQL, saved in OCI Vault.<\/p>\n nvidia_api_key, your NVIDIA NGC key from construct.nvidia.com. Used for inference and to drag container photographs from nvcr.io.<\/p>\n tavily_api_key, your Tavily key from tavily.com, for internet search.<\/p>\n Step 2. Create the infrastructure<\/p>\n Initialize the suppliers, test the plan, and apply:<\/p>\n This takes about 10 to fifteen minutes. Terraform creates the VCN, OKE cluster, Load Balancer, and the Vault with the NGC and Tavily API keys encrypted at relaxation.<\/p>\n Examine: terraform output ought to present values for oke_cluster_id and lb_public_ip. If both is empty, run terraform apply once more \u2013 the apply is protected to repeat.<\/p>\n Seize the 2 values you\u2019ll want within the subsequent step:<\/p>\n Step 3. Set up AI-Q from the NGC Helm chart<\/p>\n The chart and container photographs are printed on NGC, so there\u2019s nothing to construct domestically. We level kubectl on the new OKE cluster, create the secrets and techniques the chart consumes, then helm pull and helm set up.<\/p>\n 3a. Configure kubectl for the OKE cluster<\/p>\n oci ce cluster create-kubeconfig # sanity test. nodes ought to be prepared<\/p>\n kubectl get nodes\n<\/p><\/div>\n 3b. Export the API keys<\/p>\n Reuse the identical NGC and Tavily keys you set in terraform.tfvars. The NGC key does double responsibility. It\u2019s each the inference key and the nvcr.io pull credential.<\/p>\n 3c. Create the namespace and secrets and techniques<\/p>\n # Software credentials (NVIDIA + Tavily inference, Postgres consumer) # Picture-pull secret for nvcr.io (NGC container registry) 3d. Pull and set up the chart from NGC<\/p>\n helm pull https:\/\/helm.ngc.nvidia.com\/nvidia\/blueprint\/charts\/aiq2-web-2.0.0.tgz helm improve –install aiq aiq2-web-2.0.0.tgz The OCI overlay (values-oci-ngc.yaml) is deliberately tiny \u2014 it solely pins the frontend service to NodePort 30080 (the port the OCI Load Balancer health-checks) and names the ngc-secret image-pull secret. Picture repositories, the Postgres init SQL, and the dynamically provisioned 10 Gi Block Quantity PVC all come from the chart\u2019s personal defaults.<\/p>\n Examine: kubectl get pods -n ns-aiq ought to present aiq-backend, aiq-frontend, and aiq-postgres pods in Operating state after 3 to five minutes.<\/p>\n Step 4. Open AI-Q<\/p>\n The LB IP is already in your shell from Step 2:<\/p>\n In the event you opened a brand new shell since then, re-export it from Terraform:<\/p>\n export LB_PUBLIC_IP=”$(terraform output -raw lb_public_ip)”<\/p>\n echo “http:\/\/$LB_PUBLIC_IP”\n<\/p><\/div>\n Open http:\/\/ in your browser. You must see the AI-Q frontend.<\/p>\n Attempt a easy query first, for instance, \u201cWhat’s the NeMo Agent Toolkit?\u201d, to verify the routing works. Then strive a deeper one, for instance, \u201cEvaluate the highest three open-source deep-research brokers by benchmark rating and value\u201d, to see the deep agent in motion.<\/p>\n terraform apply fails on OKE creation with a quota error. Examine the service limits in your compartment for \u201cCluster rely\u201d and \u201cNode rely\u201d, and ask for extra quota if wanted.<\/p>\n Pods caught in ImagePullBackOff. Examine that the image-pull secret was created (kubectl get secret -n ns-aiq) and that your NGC_API_KEY was right if you ran the kubectl create secret docker-registry ngc-secret command in Step 3c. To rotate, delete the key and re-create it, then kubectl rollout restart deployment -n ns-aiq aiq-backend aiq-frontend.<\/p>\n postgres pod stays in Pending for greater than 2 minutes. The Block Quantity PVC didn\u2019t get dynamically provisioned. Run kubectl describe pvc -n ns-aiq. Typical causes are the OKE CSI driver not working, the default StorageClass lacking, or inadequate Block Quantity quota. Examine the storage class with kubectl get sc and your compartment\u2019s Block Quantity service restrict.<\/p>\n Load Balancer IP comes again as null. OCI can take a minute or two after Terraform to complete the LB. Run terraform refresh after which terraform output lb_public_ip once more.<\/p>\n Frontend masses however queries return 500. Take a look at kubectl logs -n ns-aiq deploy\/aiq-backend. The most typical trigger is a mistaken or lacking NVIDIA_API_KEY or TAVILY_API_KEY within the aiq-credentials secret you created in Step 3c.<\/p>\n You now have a working AI-Q 2.0 deployment on OCI, and one command (terraform destroy) to take away it cleanly if you find yourself carried out. Just a few issues to bear in mind as you go additional:<\/p>\n Price: The OKE node pool and the Load Balancer hold costing you whereas they run. Destroy the stack between experiments, or scale the node pool all the way down to zero.<\/p>\n Secrets and techniques: Terraform shops the NGC and Tavily keys in OCI Vault at provision time (for audit and catastrophe restoration), however the working pods learn them from the aiq-credentials Kubernetes secret you created in Step 3c. To rotate, delete and re-create that secret with the brand new values, then kubectl rollout restart deployment -n ns-aiq aiq-backend. Modifying terraform.tfvars alone received\u2019t attain the pods.<\/p>\n Extensibility: The whole lot you simply deployed is pushed by YAML and by the NeMo Agent Toolkit plugin system. Swapping an LLM, including a sub-agent, or plugging a brand new RAG backend is a configuration change, not a rewrite.<\/p>\n Clone the AI-Q in OCI repo and share on the NVIDIA Developer Discussion board the answer you constructed and what downside you solved.<\/p>\n<\/div>\n![\"Diagram](\"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/05\/AI-Q-multi-agent-architecture.webp\")
OCI deployment structure<\/h2>\n
![\"Architecture](\"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/06\/oci-aiq-arch-diagram.webp\")
Deployment steps<\/h2>\n
\ncd ai-q\/oke-samples\/aiq-2.0\n<\/div>\n
\ncp terraform.tfvars.instance terraform.tfvars\n<\/div>\n
\nterraform plan
\nterraform apply\n<\/div>\n
\nexport LB_PUBLIC_IP=”$(terraform output -raw lb_public_ip)”\n<\/div>\n
\n –cluster-id “$OKE_CLUSTER_ID”
\n –file ~\/.kube\/config
\n –region us-ashburn-1
\n –token-version 2.0.0
\n –kube-endpoint PUBLIC_ENDPOINT<\/p>\n
\nexport TAVILY_API_KEY=”tvly-…” # from tavily.com
\nexport DB_USER_PASSWORD=””\n<\/div>\n
\nkubectl create secret generic aiq-credentials -n ns-aiq
\n –from-literal=NVIDIA_API_KEY=”$NGC_API_KEY”
\n –from-literal=TAVILY_API_KEY=”$TAVILY_API_KEY”
\n –from-literal=DB_USER_NAME=”aiq”
\n –from-literal=DB_USER_PASSWORD=”$DB_USER_PASSWORD”<\/p>\n
\nkubectl create secret docker-registry ngc-secret -n ns-aiq
\n –docker-server=nvcr.io
\n –docker-username=’$oauthtoken’
\n –docker-password=”$NGC_API_KEY”\n<\/p><\/div>\n
\n –username=’$oauthtoken’
\n –password=”$NGC_API_KEY”<\/p>\n
\n -n ns-aiq
\n –wait –timeout 10m
\n -f values-oci-ngc.yaml\n<\/p><\/div>\nTroubleshooting<\/h3>\n
Be taught extra<\/h3>\n