As AI assistants evolve from easy textual content interfaces to multimodal companions, we’re seeing a shift towards extra proactive, located help. Current improvements like Mission Astra and Gemini 3.1 Flash Reside already permit customers to debate their bodily environment in actual time, typically using visible bounding field overlays to determine objects in a digital camera feed. Whereas these overlays are extremely efficient for 2D screens, the transition to immersive platforms like Android XR presents a novel problem: how can we transfer past flat UI to create a really embodied, spatially conscious dialogue?
To bridge this hole, we introduce AgentHands, printed at CHI 2026, a analysis prototype that brings the ability of co-speech gestures to the 3D world. In human communication, our palms do extra than simply level; they describe shapes, mimic actions, and emphasize factors, all synchronized with our voice. By leveraging the spatial understanding capabilities of Prolonged Actuality (XR), AgentHands replicates this pure synergy. Following up our prior analysis in Human I/O and Smart Agent, AgentHands additional equips AI brokers with expressive, synchronized hand gestures that remodel summary verbal directions into intuitive, bodily demonstrations, making conversations about your environment extra pure and fascinating.
