[Summarized from “The Batch” newsletter: https://www.deeplearning.ai/the-batch/issue-313/]

An autonomous robot performed intricate surgical operations without human intervention.

What’s new: Ji Woong (Brian) Kim and colleagues at Johns Hopkins, Stanford, and the surgical technology company Optosurgical developed Hierarchical Surgical Robot Transformer (SRT-H), a system that performs surgery with only routine help from humans. The system, which uses a two-armed surgical robot (Intuitive’s da Vinci X) that ordinarily is operated by hand, successfully completed the key clipping-and-cutting steps to remove gallbladders.

Behind the news: Prior autonomous surgical systems often rely on custom hardware and setup. For instance, Smart Tissue Autonomous Robot (STAR), which combines model-based planning with a hand-crafted state machine, uses an enhanced endoscope. The instrument integrates near-infrared fluorescence (NIR) and 3D imaging, so the system can be guided by NIR markers on a patient’s tissue and plan sutures on 3D surfaces. By contrast, SRT-H uses the widely deployed da Vinci robot (over 10,000 units in hospitals globally) and learned from RGB video with annotations in natural language — no NIR markers, 3D scanners, or special fixtures.

Why it matters: SRT-H is a significant step toward surgeries that can be performed safely by an autonomous robot. There’s still a long way to go: The system performed only portions of gallbladder removals, and it did so on tissues that were outside the body. Nonetheless, it did its job nearly flawlessly. Its natural language interface makes its decisions interpretable and enables humans to override or correct the system using verbal commands, important steps toward safe autonomous surgeries. And since SRT-H relies on imitation learning, presumably it could learn to perform other procedures, given appropriate demonstrations.

We’re thinking: In an operating room, the ability to recover from unexpected events trumps perfect execution of predetermined plans. SRT-H’s correction system enables the system to recover from its own mistakes — an important advantage over rigid systems that may work well in the lab but struggle under real-world conditions.