ISMRPhoto ISMRPhoto ISMRPhoto

Event Details

  • Date: April 22, 2026 (Half-Day Workshop, 9:00AM - 12:30PM)
  • Venue: UT Conference Center 4F, 600 Henley Street, Knoxville, TN (The University of Tennessee, Knoxville)

Table of Contents

Overview

Robot-assisted surgery and intervention have become integral to medicine over the past two decades. Recent rapid advances in machine learning (ML) have shifted the focus of medical robotics research toward automating parts of surgical tasks to enhance surgeons’ skills, reduce cognitive burden, and support decision-making, ultimately improving treatment safety and outcomes. This focus shift has also changed how medical robotics research is carried out, because development of autonomous systems generally requires (a) a large amount of training data for ML, which is often not readily available for preclinical research, and (b) a greater emphasis on system and acceptance testing of the integrated system rather than unit and integration testing of individual components. Those requirements promoted a growing demand for an in silico simulation environment that allows the observation of the dynamic behaviour of the robotic system without physical robotic hardware.

In recent years, several open-source physics simulation tools have become available to support in silico simulation across a wide range of medical and non-medical robotics research, including Gazebo, NVIDIA Isaac, AMBF, SOFA, and MuJoCo. While each tool offers different capabilities, their open-source development model enables roboticists to integrate their robots into the model for in silico simulation. There have also been efforts to integrate those simulation tools into a robotic platform, such as the Robot Operating System (ROS), enabling low-code integration of a given robotic system into the simulation environment. Meanwhile, rapid advances in AI-based segmentation tools have made it easier, faster, and more robust to construct patient-specific 3D anatomical models from images. 3D Slicer, a medical image computing software that offers a set of AI-based segmentation tools, has already integrated with ROS through SlicerROS2, enabling a streamlined in silico simulation workflow, including the generation of a patient-specific organ model, executing dynamic simulation, and visualizing the outcome.

This workshop will focus on in silico testing of a custom robot for medical robotics research using open-source tools. The workshop will consist of two parts: A presentation session will offer a series of lectures that provide an overview of open-source software platforms for the development and in silico testing of image-guided robotic surgical systems. The workshop participants will be introduced to the latest efforts to achieve interoperability among 3D Slicer, ROS2, AMBF, and Gazebo. We will also cover various open-source tools developed by the community for training and deploying AI models for surgical robotic systems.

The presentation session is followed by a tutorial in which an in silico simulation of the surgical robot system is demonstrated using 3D Slicer and ROS. Using a demo robot implemented in ROS and an example clinical image from a public database, participants will learn how to (a) set up a virtual robot on 3D Slicer and ROS2 to visualize and control it, (b) create a 3D geometric model of the subject on 3D Slicer using AI-based segmentation tools, and (c) run dynamic simulation to test the robot’s ability to navigate through the workspace inside the subject. All source code, data, and instructions will be provided so that participants can test the demo on their own computers, either on-site during the event or at home after the event.

Keywords: Image-guided interventions, Navigation, Open-source software, Software-hardware integration, In silico simulation

Intended Audience

Researchers, engineers, and students working in medical robotics and image-guided interventions are welcome to join. The tutorial sessions would be particularly useful for those already engaged in, or who will be engaged in, the design, implementation, and clinical translation of a system for image-guided robot-assisted interventions. For the hands-on session, we strongly recommend that participants bring their own laptops to follow the tutorial. While the tutorial will not involve coding, some experience in running commands on a UNIX-like system and compiling open-source software using Make or CMake would be helpful.

Time Table

Time Agenda Speaker
9:00 Introduction, Logistics, Tutorial Preparation  
  Session 1: Showcasing Medical Robotics Research with Open Source Software and In Silico Simulation Tools  
9:15 3D Slicer as a platform for image-guided robot-assisted needle insertion Junichi Tokuda
9:30 SlicerCR: Open-Source Continuum Robot Visualization and Control Architecture in 3D Slicer Yue Chen
  Session 2: Hands-On Tutorial on In Silico Medical Robot Testing (Part 1)  
9:45 In silico evaluation of a flexible catheter in a virtual patient using 3D Slicer, ROS2, and Gazebo Junichi Tokuda
10:30 Coffee Break  
  Session 3: Hands-On Tutorial on In Silico Medical Robot Testing (Part 2)  
11:00 ROS2 Motion Planning and Control with SlicerROS2 Laura Connolly
12:00 Closing  

Materials

The tutorial sessions will follow the workflow outlined in the following pages. Participants are expected to perform in silico testing using SlicerROS2 using a cloud-based Linux desktop environment provided by this workshop. The all materials of this hands-on tutorial, including source codes and instructions, are all open-sourced and provided free of charge.

Preparation

Session 2

Session 3

Links

Past Workshops

Organizers

  • Junichi Tokuda, Brigham and Women’s Hospital and Harvard Medical School, Boston, MA, USA
  • Laura Connolly, Johns Hopkins University, Baltimore, MD, USA
  • Simon Leonard, Johns Hopkins University, Baltimore, MD, USA
  • Lydia Al-Zogbi, Vanderbilt University, Nashville, TN, USA
  • Mariana Bernardes, Brigham and Women’s Hospital and Harvard Medical School, Boston, MA, USA
  • Anton Deguet, Johns Hopkins University, Baltimore, MD, USA
  • Axel Krieger, Johns Hopkins University, Baltimore, MD, USA
  • Pedro Moreira, Brigham and Women’s Hospital and Harvard Medical School, Boston, MA, USA

Contributors

  • Ruofeng Wei(Brigham and Women’s Hospital and Harvard Medical School, Boston, MA, USA) wrote a script to generate a flexible catheter model for in silico testing.
  • Kaito Hara-Lee(Queen’s University, Kingston, ON, Canada) - developed the ROS2 Motion Control module for 3D Slicer.

Acknowledgements

This work is supported in part by NIH grants (R01EB020667, R01EB034359, R01EB036015, R01CA235134) and NIH-funded National Center for Image Guided Therapy at Brigham and Women’s Hospital (P41EB028741).

Contact

Junichi Tokuda, Ph.D.

Associate Professor of Radiology Brigham and Women’s Hospital / Harvard Medical School