2012 Summer Project Week Breakout Session:Slicer in Networked Environment

Agenda

In this breakout session, we will discuss applications of 3D Slicer in networked environments. Specifically we will discuss network communications between 3D Slicer and external software/devices using OpenIGTLink and HTTP web service API.

Introduction: Network Interfaces in 3D Slicer 4.1

• Current status of OpenIGTLink IF (by Junichi Tokuda)
• HTTP Web service API in 3D Slicer (by Steve Pieper)
  • HTTP Web service API allows users to getting/setting data and invoke module functionality remotely from web browser. See the project page for more information and demo video.
• 3D Slicer + PLUS integration (by Tamas Ungi, Andras Lasso)

Future Directions

Applications

• CT/Ultrasound integration for needle guidance
  • How to control the device / data stream.
  • How to manage request?
  • In Web, long pole approach is used...
• Track catheter in the lung and show in 3D view of 3D Slicer
• Tracker for NDI,
• 4D Ultrasound
• Kinect
• Brain LAB neurosurgery
  • Ultrasound in brainlab
• VPN and other remote environment

• Import tracking data
  • OpenIGTLink for 3D tracking tools
• Import static images
  • DICOM
• Import real-time images from imaging scanners
• Import target information from external software
• Import/exporting processed image to external software
• Import/exporting commands
• Display images in remote computer
• Control Slicer from remote computer
• Share images to discuss
• Device monitoring
• Event Recording

Wish lists

• OpenIGTLink
  • 3D Slicer Interface
    • Offset function in RealTime Imaging module
    • Performance... move from timer driven mechanism to event driven mechanism
  • Profiles
    • • In many OpenIGTLink applications, multiple types of messages are streamed or the same type of messages are used in multiple purpose in a single connection. For example, needle insertion robot and guidance software exchanges TRANSFORM message for image-robot registration information, target position and orientation, current position etc. Another example is imaging scanner and visualization software that exchanges TRANSFORM messages and IMAGE messages through a single OpenIGTLink connection (see bellow). In such case, it is important to define high-level OpenIGTLink communication scheme (Profile) in the system design process and share it among the OpenIGTLink node.
      • Example in medical robotics: http://www.na-mic.org/Wiki/index.php/OpenIGTLink/Protocol/JHUBRP
      • Real-time imaging
  • Clock synchronization mechanism
    • Data exchanged among OpenIGTLink nodes have timestamps that record the times when they are generated. It is important to keep timestamps consistent among the OpenIGTLink nodes that share the same data. While there are several mechanism to synchronize system clocks among multiple computers such as Network Time Protocol (NTP) and Precision Time Protocol (PTP), it would also make sense to have a mechanism to detect time difference among the nodes and manage the time difference.
  • HTTP Web service API in 3D Slicer
    • Compression: PNG or JPEG?
      • PNG compression: loss-less, spends large amount of CPU time
      • JPEG: small data size, concern about image quality
    • Good for many different interfaces, e.g. iPad, etc.
    • Use multi-touch interface for registration. For example use one finger as a pivot point and use another finger to rotate images
    • Interface for commandline modules