NAMIC Wiki - User contributions [en]

Project Week 25/Segmentation for improving image registration of preoperative MRI with intraoperative ultrasound images for neuro-navigation

2017-06-28T10:16:29Z

Julia-Rackerseder: /* Project Description */

__NOTOC__
Back to [[Project_Week_25#Projects|Projects List]]

==Key Investigators==
*[http://www.mic.uni-bremen.de/jennifer-nitsch/ Jennifer Nitsch] (University of Bremen, Germany)
*[http://www.mic.uni-bremen.de/cmt-management-team/scheherazade-kras/ Scheherazade Kraß] (University of Bremen, Germany)
*[http://campar.in.tum.de/Main/JuliaRackerseder Julia Rackerseder] (Technical University of Munich, Germany)

==Project Description==
{| class="wikitable"
! style="text-align: left; width:27%" | Objective
! style="text-align: left; width:27%" | Approach and Plan
! style="text-align: left; width:27%" | Progress and Next Steps
|- style="vertical-align:top;"
|

*Segmented multiple anatomical structures/landmarks in both MRI and Ultrasound (US) images, using machine learning algorithms (applicability of Deep Learning algorithms is currently tested, DL for US-images, data augmentation...).

*The next step would be to analyze the improvement of registration quality with different segmentations/generated landmarks in order to adapt the segmentation algorithms.

*Using advanced segmentation and registration features for iterative robot control. The respective status of the Kuka LWR iiwa (position, configuration) is simulated and visualized as 3-D model in MeVisLab. Medical data sets including target pose (position and path towards it) can be send and received via the OpenIGTLink network protocol as well.

|

*Starting from multi-modal image segmentation in preopertive MRI and intraoperative Ultrasound images,
it would be to discuss in which "form" one or multiple segmented structures should influence the registration result.
|

*Working with the public RESECT dataset [http://sintef.no/projectweb/usigt-en/data/]
*Using segmentations to register MRI with US
*Locally refine with image based registration
*Plastimatch B-spline deformable registration module works well for image based registration [https://www.slicer.org/wiki/Documentation/4.6/Modules/PlmBSplineDeformableRegistration]

TODO:
*Implement LC2 norm to improve image based registration between US and MRI (Wein, Wolfgang, et al. "Global registration of ultrasound to mri using the LC2 metric for enabling neurosurgical guidance." International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Berlin, Heidelberg, 2013.)
*Create module to load RESECT dataset including landmarks for ground truth
|}

==Illustrations==
[[image:RegistrationInNeuroNavigationSystem.png|500px]]

Using segmented structures as guiding frame for multi-modal image registration:

[[image:MultimodalImageSegmentation3.png|500px]]

LWR Robot simulation in MeVisLab:

[[image:Picture 2016-12-19 13 55 31.png|500px]]
[[image:Picture 2017-06-26 11 04 59.png|500px]]

==Background and References==

In glioma surgery neuronavigation systems assist in determining the tumor's location and estimating its extent. However, the intraoperative situation diverges seriously from the preoperative situation in the MRI scan
displayed on the navigation system. The movement of brain tissue during surgery, i.e., caused by brainshift
and tissue removal, must be considered mentally by the surgeon. A task that gets more challenging in later
phases of the tumor resection.

Besides, it is an exhaustive issue and the shift of cerebral structures must be
expected being non-uniform and that it implies a deformation of the image data. This makes it especially hard
to mentally predict and model.

Thus, intraoperative imaging modalities are used to visualize the current intraoperative situation. IUS,
for instance, is easy to use intraoperatively, offers real-time information, is widely available at low cost and
causes no radiation. These are important advantages when iUS is compared with iCT or iMRI. However, in
image-guided surgery precise image registration of iUS and preMRI and the thereon-based image fusion is still
an unsolved problem. The different representations of cerebral structures in both modalities as well as
artifacts within the iUS, hinder direct fusion of both modalities.

Project Week 25/Segmentation for improving image registration of preoperative MRI with intraoperative ultrasound images for neuro-navigation

2017-06-28T10:00:44Z

Julia-Rackerseder: /* Project Description */

__NOTOC__
Back to [[Project_Week_25#Projects|Projects List]]

==Key Investigators==
*[http://www.mic.uni-bremen.de/jennifer-nitsch/ Jennifer Nitsch] (University of Bremen, Germany)
*[http://www.mic.uni-bremen.de/cmt-management-team/scheherazade-kras/ Scheherazade Kraß] (University of Bremen, Germany)
*[http://campar.in.tum.de/Main/JuliaRackerseder Julia Rackerseder] (Technical University of Munich, Germany)

==Project Description==
{| class="wikitable"
! style="text-align: left; width:27%" | Objective
! style="text-align: left; width:27%" | Approach and Plan
! style="text-align: left; width:27%" | Progress and Next Steps
|- style="vertical-align:top;"
|

*Segmented multiple anatomical structures/landmarks in both MRI and Ultrasound (US) images, using machine learning algorithms (applicability of Deep Learning algorithms is currently tested, DL for US-images, data augmentation...).

*The next step would be to analyze the improvement of registration quality with different segmentations/generated landmarks in order to adapt the segmentation algorithms.

*Using advanced segmentation and registration features for iterative robot control. The respective status of the Kuka LWR iiwa (position, configuration) is simulated and visualized as 3-D model in MeVisLab. Medical data sets including target pose (position and path towards it) can be send and received via the OpenIGTLink network protocol as well.

|

*Starting from multi-modal image segmentation in preopertive MRI and intraoperative Ultrasound images,
it would be to discuss in which "form" one or multiple segmented structures should influence the registration result.
|

*Working with the public RESECT dataset [http://sintef.no/projectweb/usigt-en/data/]
*Using segmentations to register MRI with US
*Locally refine with image based registration

TODO:
*Implement LC2 norm to improve image based registration between US and MRI (Wein, Wolfgang, et al. "Global registration of ultrasound to mri using the LC2 metric for enabling neurosurgical guidance." International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Berlin, Heidelberg, 2013.)
*Create module to load RESECT dataset including landmarks for ground truth
|}

==Illustrations==
[[image:RegistrationInNeuroNavigationSystem.png|500px]]

Using segmented structures as guiding frame for multi-modal image registration:

[[image:MultimodalImageSegmentation3.png|500px]]

LWR Robot simulation in MeVisLab:

[[image:Picture 2016-12-19 13 55 31.png|500px]]
[[image:Picture 2017-06-26 11 04 59.png|500px]]

==Background and References==

In glioma surgery neuronavigation systems assist in determining the tumor's location and estimating its extent. However, the intraoperative situation diverges seriously from the preoperative situation in the MRI scan
displayed on the navigation system. The movement of brain tissue during surgery, i.e., caused by brainshift
and tissue removal, must be considered mentally by the surgeon. A task that gets more challenging in later
phases of the tumor resection.

Besides, it is an exhaustive issue and the shift of cerebral structures must be
expected being non-uniform and that it implies a deformation of the image data. This makes it especially hard
to mentally predict and model.

Thus, intraoperative imaging modalities are used to visualize the current intraoperative situation. IUS,
for instance, is easy to use intraoperatively, offers real-time information, is widely available at low cost and
causes no radiation. These are important advantages when iUS is compared with iCT or iMRI. However, in
image-guided surgery precise image registration of iUS and preMRI and the thereon-based image fusion is still
an unsolved problem. The different representations of cerebral structures in both modalities as well as
artifacts within the iUS, hinder direct fusion of both modalities.

Project Week 25/Segmentation for improving image registration of preoperative MRI with intraoperative ultrasound images for neuro-navigation

2017-06-26T13:24:30Z

Julia-Rackerseder: /* Key Investigators */

__NOTOC__
Back to [[Project_Week_25#Projects|Projects List]]

==Key Investigators==
*[http://www.mic.uni-bremen.de/jennifer-nitsch/ Jennifer Nitsch] (University of Bremen, Germany)
*[http://www.mic.uni-bremen.de/cmt-management-team/scheherazade-kras/ Scheherazade Kraß] (University of Bremen, Germany)
*[http://campar.in.tum.de/Main/JuliaRackerseder Rackerseder, Julia] (Technical University of Munich, Germany)

==Project Description==
{| class="wikitable"
! style="text-align: left; width:27%" | Objective
! style="text-align: left; width:27%" | Approach and Plan
! style="text-align: left; width:27%" | Progress and Next Steps
|- style="vertical-align:top;"
|

*Segmented multiple anatomical structures/landmarks in both MRI and Ultrasound (US) images, using machine learning algorithms (applicability of Deep Learning algorithms is currently tested, DL for US-images, data augmentation...).

*The next step would be to analyze the improvement of registration quality with different segmentations/generated landmarks in order to adapt the segmentation algorithms.

*Using advanced segmentation and registration features for iterative robot control. The respective status of the Kuka LWR iiwa (position, configuration) is simulated and visualized as 3-D model in MeVisLab. Medical data sets in-cluding target pose (position and path towards it) can be send and received via the OpenIGTLink network protocol as well.

|

*Starting from multi-modal image segmentation in preopertive MRI and intraoperative Ultrasound images,
it would be to discuss in which "form" one or multiple segmented structures should influence the registration result.
|

|}

==Illustrations==
[[image:RegistrationInNeuroNavigationSystem.png|500px]]

Using segmented structures as guiding frame for multi-modal image registration:

[[image:MultimodalImageSegmentation3.png|500px]]

LWR Robot simulation in MeVisLab:

[[image:Picture 2016-12-19 13 55 31.png|500px]]
[[image:Picture 2017-06-26 11 04 59.png|500px]]

==Background and References==

In glioma surgery neuronavigation systems assist in determining the tumor's location and estimating its extent. However, the intraoperative situation diverges seriously from the preoperative situation in the MRI scan
displayed on the navigation system. The movement of brain tissue during surgery, i.e., caused by brainshift
and tissue removal, must be considered mentally by the surgeon. A task that gets more challenging in later
phases of the tumor resection.

Besides, it is an exhaustive issue and the shift of cerebral structures must be
expected being non-uniform and that it implies a deformation of the image data. This makes it especially hard
to mentally predict and model.

Thus, intraoperative imaging modalities are used to visualize the current intraoperative situation. IUS,
for instance, is easy to use intraoperatively, offers real-time information, is widely available at low cost and
causes no radiation. These are important advantages when iUS is compared with iCT or iMRI. However, in
image-guided surgery precise image registration of iUS and preMRI and the thereon-based image fusion is still
an unsolved problem. The different representations of cerebral structures in both modalities as well as
artifacts within the iUS, hinder direct fusion of both modalities.