SlicerOpenCV

About This Page

This page provides information about integration of 3D Slicer and OpenCV. OpenCV is a library of programming functions for real time computer vision. The primary goal of 3D Slicer-OpenCV integration is to provide functionalities to import and process video images used in the medical field e.g. ultrasound and endoscopic video images in 3D Slicer.

How to build 3D Slicer with OpenCV library

The first step to use OpenCV functionalities in 3D Slicer is to build 3D Slicer with the OpenCV library. The current version in trunk repository (3.7 alpha) can be built with OpenCV 2.1. There are two ways to do it:

Using getbuildtests.tcl build script

getbuildtests.tcl build script has been used to build 3D Slicer and it has an option to build it with OpenCV. To activate this option, open Slicer3/slicer_variables.tcl and change the following line (around line 180)

set ::USE_OPENCV "OFF"

to

set ::USE_OPENCV "ON"

then, follow Build Instruction Page.

Using SuperBuild

SuperBuild is a build system for Slicer4, which is being developed during the second half of 2010. We are preparing configuration files to build 3D Slicer with OpenCV. If you are interested, please contact Junichi Tokuda.

Example 3D Slicer modules

A good example of application is image overlay system, which projects 3D organ model onto endoscopic video image. There are two experimental module with different approaches are available:

Secondary Window with OpenCV module (Nagoya Institute of Technology)

This module creates a new window with a VTK renderer independent from 3D Slicer's scene. The module copies all VTK models from the MRML scene to its own VTK scene and overlays them on to a video image imported by using OpenCV. The merit of this approach is that the developers can change any properties of models, cameras etc. without modifying any objects in MRML scene.

Screenshot of the Secondary Window with OpenCV module. A 3D model is overlaid on the laparoscopic video image in the right side of secondary window. The left side of the secondary window is 3D viewer which viewpoint is independent of the 3D viewer in 3D Slicer.

StereoCalib module (Nagoya Institute of Technology)

This module is for camera calibration not only single camera but also stereo camera. The output files are the camera intrinsic parameter matrix and the camera distortion compensation vector for Secondary Window with OpenCV module.

Screenshot of the StereoCalib module. You can obtain the intrinsic parameters matrix and the lens distortion compensation vector of your medical camera device by using a checker board.

Video Importer module (BWH)

This is a simple demo module of an image overlay. This module copies the video frame into the background of 3D viewer in 3D Slicer and displays 3D models in front of video image. Since the video image is directly imported into the 3D viewer in 3D Slicer, all properties of VTK models in the MRML scene are reflected in the overlaid image.

Screenshot of the VideoImporter module. A 3D model of prostate is overlaid on the laparoscopic video image in the left 3D viewer.

The code is available in:
http://svn.na-mic.org/NAMICSandBox/trunk/IGTLoadableModules/VideoImporter/

Support

This work is supported by

• Intelligent Surgical Instrument Project (Funded by New Energy and Industrial Technology Development Organization (NEDO), Ministry of Economy, Trade and Industry (METI), Japan)
• Image Guided Therapy Center (NIH P41RR019703)
• IMage Guided Therapy (NIH P01CA67165)

Contact

• About 3D Silcer-OpenCV integration
  • Junichi Tokuda, Ph.D. /Brigham and Women's Hospital, (tokuda at bwh.harvard.edu)
• About Secondary Window with OpenCV module and StereoCalib module
  • Atsushi Yamada, Ph.D. / Nagoya Institute of Technology, Japan, (yamada.atsushi at nitech.ac.jp)
• About the project
  • Nobuhiko Hata, Ph.D. / Brigham and Women's Hospital (hata at bwh.harvard.edu)
  • Kiyoyuki Chinzei, Ph.D. / AIST, Japan (chin at ni.aist.go.jp)