
A South Korean research team has developed artificial intelligence (AI) technology capable of understanding the context of surgical scenes. Advancing beyond existing AI vision systems that merely identify instruments, the technology raises the prospect of equipping surgical robots with real-time visual perception once commercialized.
Samsung Medical Center announced on Nov. 17 that a research team from its Intelligent Medical Robotics Research Center and Department of Transplantation Surgery had developed the new AI model, called CurConMix+. The model combines spatial information and temporal context to interpret surgery hierarchically, from fine-grained surgical movements to the overall phases of an operation.
While camera-based AI systems for analyzing surgical scenes already exist, earlier models have largely focused on recognizing surgical instruments or isolated movements by medical staff in single frames. CurConMix+, by contrast, integrates spatial features from operating room footage with the progression of the procedure over time in real time—a process researchers say mirrors how humans perceive surgical situations.
AI Recognizes Operating Room Activity in Real Time
A robot’s ability to recognize what is happening inside an operating room in real time is widely viewed as an essential prerequisite for robotic surgical assistance systems. To work effectively in an operating room and carry out assigned tasks, a robot must understand what stage a surgery has reached and what is occurring at any given moment.
However, real-time perception—which comes naturally to humans—remains a major hurdle for AI. Unlike humans, who rely on context and clinical intuition, AI systems must convert visual data into numerical values and process them computationally, making it difficult to adapt to unexpected variables or atypical surgical situations.
To overcome this limitation, the researchers designed CurConMix+ to learn the entire surgical workflow, encompassing individual movements, specific tasks and broader procedural stages. By grasping the broader context of the operation, the AI can independently determine which stage is currently underway.
Benchmark evaluations reflected that design. In tests using a public surgical video benchmark that allows researchers worldwide to compare models under standardized criteria, CurConMix+ achieved an average precision of 42.2%, surpassing the previous best-performing model’s record of 41.5%.
Average precision is a standard metric used to evaluate AI performance in fields such as autonomous driving, surveillance video analysis and medical imaging. A higher score indicates that a model is more likely to identify a target accurately while minimizing false detections. In liver resection videos specifically, CurConMix+ reached an average precision of 58.4%, outperforming the previous record by more than 10 percentage points.
The model also demonstrated strong performance in detecting infrequent yet critical steps during an operation. Its average precision in recognizing maneuvers aimed at preventing bile leakage prior to gallbladder removal reached 29% in gallbladder surgery videos and 56% in liver surgery videos—exceeding previous records by 6 and 13 percentage points, respectively.
Potential Integration Into Humanoid Surgical Robots
Jung Kyu-Hwan, director of Samsung Medical Center’s Intelligent Medical Robotics Research Center, said the breakthrough establishes a core technological foundation enabling surgical robots to collaborate effectively with medical teams.
In September, the hospital staged the world’s first demonstration of a humanoid surgical assistant robot working alongside surgeons during a cholecystectomy. During that demonstration, the robot accurately understood the lead surgeon’s verbal instructions and handed over the requested instruments. Equipping such a robot with advanced visual perception would allow it to assist in procedures by observing and assessing surgical situations autonomously.

“For robots and medical staff to collaborate naturally, robots must go beyond simply identifying surgical instruments or organs and understand what procedure is underway and what the surgical team is doing,” Jung said. “This study is significant because it introduces AI technology that integrates recognition of everything from individual surgical movements to the overall surgical workflow. We expect it to pave the way for intelligent surgical robots that actively assist clinicians.”
The researchers also compiled a new training dataset from videos of 48 laparoscopic liver resections performed at Samsung Medical Center between February 2020 and March 2022, totaling approximately 52 hours of footage. Two hepatobiliary and pancreatic surgeons and three specialists with more than 10 years of experience reviewed every scene, annotating the footage into five surgical stages, 18 detailed tasks and 195 specific actions.
The dataset expands surgical AI research—which has largely centered on cholecystectomy—into the more complex domain of liver surgery. Both the dataset and source code have been released on an open-source platform so other researchers can independently validate the findings.
Trained on these granular, scene-by-scene annotations, CurConMix+ achieved more than 93% accuracy in predicting the surgical stages of cholecystectomy. “In effect, an AI system capable of reading the overall flow of surgery has now learned to recognize even the subtle movements of a surgeon’s fingertips,” said Oh Namkee, a professor in Samsung Medical Center’s Department of Transplantation Surgery. “Following applications in automated surgical documentation, postoperative review and resident training, the technology will ultimately be deployed in humanoid surgical assistant robots capable of real-time situational awareness during operations.”
The research was funded by the Korea ARPA-H Project, supported by the Korea Health Industry Development Institute under the Ministry of Health and Welfare; a National Research Foundation of Korea grant program under the Ministry of Science and ICT; and Samsung Medical Center’s Future Medicine 2030 Project. The findings were recently published in the international journal IEEE Transactions on Medical Imaging.
