A joint publication between my old and new group has just appeared in the Robotics and Automation Magazine:
From Macro to Micro: Autonomous Multiscale Image Fusion for Robotic Surgery
Lin Zhang, Menglong Ye, Petros Giataganas, Michael Hughes, Adrian Bradu, Adrian Podoleanu, and Guang-Zhong Yang
It’s available open access from here: https://doi.org/10.1109/MRA.2017.2680543
This paper was the result of a project integrating two high-resolution, probe-based imaging systems, endoscopic microscopy and optical coherence tomography (OCT), with the da Vinci surgical robot, and then using these imaging systems to support automated scanning to generate large-area maps of tissue at high resolution.
The imaging systems and the robot
The two different imaging modalities support the robotic guidance in different ways. The endomicroscope provides an en face view of the tissue (as though you are looking down on it from the top), whereas the OCT gives a cross-sectional image through the tissue. The endomicroscope images are circular, and about a quarter of a millimetre in diameter, while the OCT gives cross-sectional images that are 5 mm in depth (although in practice it can only penetrate 1-2 mm into tissue before the signal becomes too small to be useful). So the endomicroscope images were used to guide the position along the surface of the tissue, and the OCT images were used to guide the robot’s distance from the tissue. Between the two, this provided three-dimensional control of the robot, with accuracy measured in the 10’s of microns. This made the robotic control far more accurate than simply relying on its own, internal position estimation.
This project was a great example of two Universities collaborating – the OCT system was built here at Kent, I developed the endomicroscope at Imperial College and put together an integrated software package for the two probes, while Imperial also developed the robotic control system using the da Vinci Research Kit (dVRK). A version of this system was entered into the Surgical Robotic Challenge Last year, and although we didn’t win the overall prize, we did win the Best Video category.