Hands-Free AI Help for Robotic Surgery

In robotic surgery, a surgeon’s hands and eyes are busy. That makes it hard to pull up labs, flip through CT slices, or inspect 3D anatomy without breaking focus.

This study introduces the voice-directed Surgical Agent Orchestrator Platform (SAOP): a coordinated team of AI agents that listens to spoken requests and turns them into precise on-screen actions.

• Find info: Retrieve patient history, labs, or notes on demand.
• Work with images: Scroll, zoom, and adjust CT/MRI views.
• Navigate 3D: Explore anatomical models aligned with surgical video.

Under the hood, Large Language Model–powered agents plan, refine, validate, and reason about commands. The authors also propose a new score—MOEM (Multi‑level Orchestration Evaluation Metric)—to measure accuracy and robustness across command types.

Tested on 240 voice commands, SAOP achieved high success and handled speech-recognition slips and ambiguous phrasing, showing promise as hands-free support during minimally invasive da Vinci procedures.

Note: Research prototype; assists clinicians and does not replace medical judgment.

Paper: http://arxiv.org/abs/2511.07392v1

Register: https://www.AiFeta.com

AI RoboticSurgery Healthcare VoiceAI MedicalImaging LLM HumanComputerInteraction SurgicalTech