Shippy is a maritime AI agent constructed for high-stakes selections, the place the flawed reply has actual impacts. This is the structure behind it—and the teachings we’re carrying into Ai2’s different environmental platforms.
Shippy answering a stay question about Ghana’s EEZ. The response exhibits its work: the boundary supply, the info cutoff, the question timestamp, and a deep hyperlink again to the Skylight map so the analyst can confirm each quantity.
Constructing an AI agent for a high-stakes operational area like defending the ocean is, above all, an issue of reliability. For a maritime analyst, a flawed reply might ship a patrol vessel miles within the flawed route, costing vital sources which are already stretched skinny and doubtlessly placing personnel in hurt’s method.
So when the Skylight crew got down to construct Shippy, our AI for real-time maritime area consciousness, the actual work wasn’t the mannequin. It was constructing a system we might belief to be appropriate, to remain inside its limits, and to carry up throughout a variety of duties. And we needed to confirm all of it in opposition to Skylight’s stay knowledge, up to date repeatedly as new satellite tv for pc and vessel alerts arrive—not a static snapshot.
Agent anatomy: abilities, soul, and config
We consider an agent like Shippy as three issues: a soul, abilities, and config.
The soul is the system immediate that frames Shippy’s persona and units behavioral boundaries. Abilities inform Shippy the right way to deal with particular sorts of requests. Collectively, the soul and abilities are baked right into a Docker picture—a versioned, deployable artifact that defines what Shippy is. Config covers every thing else: which agent harness to run (in Shippy’s case, OpenClaw, an open-source agent framework), which LLM to make use of (at the moment, Shippy depends on Claude Opus 4.6), and runtime settings. Secrets and techniques like API keys are injected at runtime; swapping the mannequin or the harness is a config change, not a rebuild.
Shippy’s abilities observe the identical agent-skills spec utilized by coding instruments like Claude Code and Codex—plain markdown information with structured frontmatter. This retains every ability understandable, versioned, and straightforward to revise. Shippy at the moment contains abilities for:
Querying the Skylight API for Occasions (the vessel behaviors Skylight surfaces, like fishing or transshipment between two vessels) and vessel knowledge
Wanting up Unique Financial Zones (EEZ) and Marine Protected Space (MPA) boundaries
Decoding vessel observe knowledge, the place and motion alerts ships broadcast, constructing on the exercise classifications Skylight’s fashions, together with Atlantes, already produce
Producing interactive map hyperlinks that permit an analyst bounce from a Shippy in-chat reply to an actual location on the Skylight map
For instance, the Skylight API question ability encodes the total workflow for answering a query a couple of particular space. When an analyst or consumer asks, “present me fishing exercise in Panama’s EEZ final month,” the ability’s directions direct Shippy to first resolve “Panama EEZ” to a boundary polygon by means of Skylight’s areas API as a substitute of guessing or hard-coding coordinates, then question Fishing Occasions inside that geometry, format the outcomes with deep hyperlinks again to the Skylight map, and attribute any vessel metadata drawn from Skylight companions like International Fishing Watch or TMT.
A single query posed to Shippy can hook into a number of abilities directly. “Are there vessels working close to the Cordillera de Coiba MPA?” attracts on the Skylight ability for knowledge question, our accomplice ProtectedSeas’ database for MPA boundary context, and the vessel observe ability for decoding vessel conduct. All of this occurs in a single dialogue flip.
The soul defines what Shippy will and will not do. It will not make authorized determinations about whether or not a vessel is breaking the regulation—that may be a willpower for folks, not an agent. It additionally will not speculate past what the info helps. These boundaries are express within the system immediate, not implicit in fine-tuning, which makes them auditable and straightforward to revise.
Deterministic instruments for a nondeterministic agent
Brokers are nondeterministic. You may’t management what the mannequin decides to do, however you can also make the instruments it reaches for predictable. To that finish, Shippy ‘talks’ to Skylight by means of a purpose-built CLI that calls the API, reasonably than issuing uncooked calls itself.
Our API has dozens of enter sorts, nested filter objects, pagination cursors, and complicated geometry inputs. In early prototypes, we let Shippy assemble API calls from scratch. It produced a gentle stream of delicate bugs: malformed pagination that silently dropped outcomes, geometry encoding errors, and correct-looking queries that returned flawed knowledge due to a misunderstood filter sort.
The Skylight CLI collapses that complexity right into a predictable interface. Shippy points a single command – skylight occasions search with typed filter flags – and the CLI handles authentication, pagination, and structured output. The CLI can be self-documenting: in depth –help textual content and detailed error messages give the agent (and human builders) sufficient context to get well from errors with out guessing. Its output is at all times written to a neighborhood JSON file reasonably than piped by means of the shell. Early on, massive end result units would hit pipe buffer limits or break downstream instruments like jq. Writing to disk sidesteps each issues and lets the agent programmatically entry question outcomes throughout subsequent steps.
Beneath the CLI is a standardized API: a number of useful resource sorts – Skylight Occasions, vessels, areas, satellite tv for pc imagery, vessel tracks, and extra – accessible by means of a typical pair of operations, search and mixture. The APIs’ inputs and outputs are outlined as typed schemas with field-level descriptions.
This layering – typed API, deterministic CLI, and agent abilities that reference the CLI’s instructions – implies that every of Shippy’s elements may be examined independently. The API has its personal take a look at suite. The CLI may be exercised by a human or an agent. And the agent abilities reference CLI instructions that deal with the plumbing in order that Shippy does not must reinvent the wheel each time it hits the Skylight API. Every layer narrows what the following layer can get flawed.
Sandboxed internet hosting and isolation
Skylight serves tons of of presidency businesses and NGOs throughout over 70 international locations. A fisheries officer within the Philippines has Areas of Curiosity, vessel watchlists, and alert configurations which are scoped to their Skylight account. After they ask Shippy a query, the agent’s API calls have to return their knowledge, and their dialog historical past must not ever be seen to anybody else.
Each consumer talks to Shippy inside their very own ephemeral, remoted session, and making that work reliably at scale was one of the crucial vital engineering efforts behind the mission. We constructed Mothership, an agent internet hosting platform that provisions a devoted Kubernetes deployment for every consumer session. When a consumer opens a dialog, the system spins up a set of pods packaging the agent runtime, its abilities, and the Skylight CLI. The consumer’s Skylight JWT is injected at provision time so the agent’s API calls are scoped to that consumer’s knowledge.
Recordsdata the agent writes throughout a multi-step evaluation exist solely inside that session and are by no means shared throughout customers. Contained in the sandbox, the agent can write and run code, set up dependencies, pull in datasets, and work by means of multi-step analyses. On the community degree, the sandbox is restricted to solely the companies it wants.
Evaluating an agent, not a mannequin
Most benchmarks rank general-purpose AI on static questions. They do not seize how an agent behaves as soon as it is wired into an actual workflow: the way it selects instruments, queries stay knowledge, acts on outcomes, and is aware of the place to cease. So we constructed our personal eval system round how Shippy works, scoring the entire agent – mannequin, abilities, and sandbox collectively – in opposition to stay knowledge.
In our eval framework, subject-matter consultants write situations and rubrics, selecting which standards apply to every process and setting the weights, so each process is graded on what truly issues for it. A fishing-events question, as an illustration, weights knowledge accuracy most closely, with boundary decision and timeframe subsequent, and supply attribution and response type carrying much less. Additionally they annotate particular person responses as appropriate or incorrect, giving the choose floor fact to attain in opposition to. Topic-matter consultants moreover annotate particular person responses as appropriate or incorrect, giving the choose floor fact to attain in opposition to.
The pipeline is simple: a natural-language immediate runs by means of the sandbox, an LLM choose grades every criterion from 0 to 1 and explains in writing why the response did or did not meet it, and the weighted mixture is checked in opposition to a hard and fast go threshold, because the diagram beneath exhibits.
How a single process is scored in our pipeline. A natural-language immediate runs by means of the sandbox, an LLM choose grades every rubric criterion with written reasoning, and the weighted mixture resolves to a go or fail in opposition to a hard and fast threshold.
Duties are executed by means of Harbor, an open analysis framework. We wrote a Harbor plugin that spins up an actual Shippy session on the precise model being examined, in opposition to the identical actual knowledge a consumer would encounter. The suite runs in parallel in opposition to a particular versioned Shippy construct, producing a timestamped outcomes file and a report of rating modifications in opposition to the earlier run. We rerun the suite each time the talents, mannequin, or underlying knowledge change, and a model of Shippy that regresses on our eval standards does not attain finish customers.
Shippy scores constantly throughout knowledge retrieval and guardrail duties, accurately refusing army intelligence requests, sustaining consumer knowledge isolation, and attributing sources precisely. In our newest run, the clearest patterns had been patrol-planning duties the place Shippy overstepped into tactical suggestions reasonably than resolution assist, geometry-sensitive queries the place boundary simplification triggered missed Occasions, and one case the place the agent invented a CLI command that did not exist. Every of those immediately informs our subsequent spherical of ability enhancements.
Shippy’s eval suite working inside Skylight: every situation is scored on weighted standards, with the choose’s reasoning seen so a failure factors to a particular conduct to repair.
The place we’re headed
We’re opening Shippy to early adopters on a rolling foundation and welcoming them to stress-test it—to search out the questions the agent solutions poorly and guardrails that will want tightening. Right here’s what we’re constructing subsequent:
Agent-driven UI management. Shippy returns map hyperlinks at present; subsequent it would drive the Skylight map itself, shifting to a area, making use of filters, and adjusting time ranges.
Mannequin routing. Not each query wants a frontier mannequin, so we’re routing easy lookups to smaller, quicker ones and saving the full-weight mannequin for advanced investigations.
Cross-thread reminiscence. Dialog historical past persists inside a thread, however context does not carry throughout threads. We’re constructing reminiscence so Shippy carries persistent info (e.g., an analyst’s jurisdiction, most well-liked sources) and applies them routinely. So, “Present me fishing exercise this week” will not imply re-specifying an analyst’s EEZ every time.
Our work on Shippy is already shaping how we take into consideration brokers elsewhere at Ai2—most instantly EarthRanger, our wildlife-conservation platform, and OlmoEarth, our open suite of Earth statement instruments. Mothership was constructed to be normal and to host different brokers, so whereas maritime is the primary area we’re making use of it to, we do not anticipate it to be the final.
Shippy is constructed by the Skylight crew at Ai2. Skylight is a free maritime area consciousness platform utilized by 300+ companions throughout 70 international locations.
