Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems

Yonghang Tai, Junsheng Shi, Junjun Pan, Aimin Hao, Victor Chang

Research output: Contribution to journalArticlepeer-review

44 Downloads (Pure)


Background: Compared with traditional thoracotomy, video-assisted thoracoscopic surgery (VATS) has less minor trauma, faster recovery, higher patient compliance, but higher requirements for surgeons. Virtual surgery training simulation systems are important and have been widely used in Europe and America. Augmented reality (AR) in surgical training simulation systems significantly improve the training effect of virtual surgical training, although AR technology is still in its initial stage. Mixed reality has gained increased attention in technology-driven modern medicine but has yet to be used in everyday practice. Methods: This study proposed an immersive AR lobectomy within a thoracoscope surgery training system, using visual and haptic modeling to study the potential benefits of this critical technology. The content included immersive AR visual rendering, based on the cluster-based extended position-based dynamics algorithm of soft tissue physical modeling. Furthermore, we designed an AR haptic rendering systems, whose model architecture consisted of multi-touch interaction points, including kinesthetic and pressure-sensitive points. Finally, based on the above theoretical research, we developed an AR interactive VATS surgical training platform. Results: Twenty-four volunteers were recruited from the First People's Hospital of Yunnan Province to evaluate the VATS training system. Face, content, and construct validation methods were used to assess the tactile sense, visual sense, scene authenticity, and simulator performance. Conclusions: The results of our construction validation demonstrate that the simulator is useful in improving novice and surgical skills that can be retained after a certain period of time. The video-assisted thoracoscopic system based on AR developed in this study is effective and can be used as a training device to assist in the development of thoracoscopic skills for novices.

Original languageEnglish
Pages (from-to)274-286
Number of pages13
JournalVirtual Reality and Intelligent Hardware
Issue number4
Publication statusPublished - 1 Aug 2021

Bibliographical note

Funding Information:
? Supported by the National Science Foundations of China (62062069, 62062070, and 62005235).

Publisher Copyright:
© 2021 Beijing Zhongke Journal Publishing Co. Ltd


Dive into the research topics of 'Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems'. Together they form a unique fingerprint.

Cite this