The Centre for Artificial Intelligence and Robotics (CAIR), Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences has unveiled at Hong Kong Science Park a new large multimodality model for surgeons: CARES Copilot 1.0.

The launch of the new system follows internal testing and optimisation across various departments in multiple hospitals, proving its value as a comprehensive, powerful, and reliable AI surgery assistant.

Integrating intelligently with medical devices, CARES Copilot 1.0 comprehends multimodal surgical data, including images, text, voice, video, MRI, CT scans, and ultrasound. It understands and processes diverse medical scenarios and tasks, and supports functions like surgical phase identification, segmentation of instruments and anatomical structures, instrument detection and counting, and generation of MRI high-resolution images.

Equipped with a 100K context window for efficient understanding and analysis, CARES Copilot 1.0 can handle complex surgical materials of more than 3,000 pages, supporting the training and education of young medical professionals.

Its deep retrieval functions ensure that it can quickly and accurately retrieve information from surgical teaching materials, expert guidelines, and medical literature. Tests demonstrate that the system can retrieve information from datasets of over a million records within few seconds, with an accuracy rate of up to 95%.

Accorcding to Dr. Chan Tat Ming, Head of Division of Neurosurgery at the Department of Surgery, The Chinese University of Hong Kong (CUHK), LLMs such as CARES Copilot 1.0 can directly support frontline medical staff in emergencies by supervising, issuing warnings, and preventing risky procedures.