Difference between revisions of "2016 Summer Project Week/Auditory Display"

From NAMIC Wiki
Jump to: navigation, search
m (Text replacement - "[http://www.na-mic.org/Wiki/images/" to "[https://na-mic.org/w/images/")
 
(24 intermediate revisions by 2 users not shown)
Line 5: Line 5:
  
 
==Key Investigators==
 
==Key Investigators==
*David Black, Bremen
+
*David Black, University of Bremen, Fraunhofer MEVIS
 
*Sarah Frisken, BWH/HMS
 
*Sarah Frisken, BWH/HMS
 
*Christian Hansen, Uni Magdeburg
 
*Christian Hansen, Uni Magdeburg
 +
*Longquan Chen, BWH/HMS
 +
* Tamas Ungi
 +
* Rocio Lopez
 +
* Elvis Chen
 +
*Julian Hettig
  
 
==Related Work==
 
==Related Work==
 
Existing methods for intraoperative navigation guidance already implemented in [https://puredata.info Puredata], see
 
Existing methods for intraoperative navigation guidance already implemented in [https://puredata.info Puredata], see
  
VIDEO: [https://www.dropbox.com/s/6wfrklm8by4fx29/Dual-frequency-feedback.mov?dl=0 for ureteroscopy, with BWH/HMS][http://wiki.na-mic.org/Wiki/index.php/File:Dual-frequency-feedback.mov download link]
+
*VIDEO: [https://www.dropbox.com/s/6wfrklm8by4fx29/Dual-frequency-feedback.mov?dl=0 for ureteroscopy, with BWH/HMS and with Slicer][http://wiki.na-mic.org/Wiki/index.php/File:Dual-frequency-feedback.mov download link]
VIDEO: [https://www.dropbox.com/s/35rvm7kx30hwl08/WirbelBeispielMitAudio.mov?dl=0 for ablation needle insertion, with Univeristy of Magdeburg]
+
*VIDEO: [https://www.dropbox.com/s/35rvm7kx30hwl08/WirbelBeispielMitAudio.mov?dl=0 for ablation needle insertion, with Univeristy of Magdeburg]
 +
*VIDEO: [https://www.youtube.com/watch?v=gCg5nJSI2pY Auditory Display for Liver Resection] [http://www.ncbi.nlm.nih.gov/pubmed/23192891 Paper]
 +
*[http://ioi.cs.uni-bremen.de/?page_id=780 Towards Uncertainty-Aware Auditory Display for Surgical Navigation (CARS 2016)]
 +
*D. Black, J. Al Issawi, C. Rieder, and H. Hahn. Enhancing medical needle placement with auditory display. In Mensch & Computer 2013: Interaktive Vielfalt, pages 289–292, 2013. [https://na-mic.org/w/images/7/7d/Curac2013_dblack_Auditory_support_for_Navigated_Radiofrequency_Ablation_FINAL.pdf  PDF ]
  
[https://www.youtube.com/watch?v=gCg5nJSI2pY Auditory Display for Liver Resection]
+
==Possible application areas / IDEAS==
[http://www.ncbi.nlm.nih.gov/pubmed/23192891 Paper]
+
*Acquiring 3D data sets for US, did we acquire all we need?
 
+
* Uncertainty in navigation information
[http://ioi.cs.uni-bremen.de/?page_id=780 Towards Uncertainty-Aware Auditory Display for Surgical Navigation (CARS 2016)]]
+
*Brain / Structure Shift
 
+
* reduce complexity of displays by offloading to audio
D. Black, J. Al Issawi, C. Rieder, and H. Hahn. Enhancing medical needle placement with auditory display. In Mensch & Computer 2013: Interaktive Vielfalt, pages 289–292, 2013. [http://www.na-mic.org/Wiki/images/7/7d/Curac2013_dblack_Auditory_support_for_Navigated_Radiofrequency_Ablation_FINAL.pdf  PDF ]
+
*Depth cues for 3D data (shown on 2D screens)
 +
*coverage of needle in US
  
 
==Project Description==
 
==Project Description==
Line 33: Line 42:
 
<div style="width: 27%; float: left; padding-right: 3%;">
 
<div style="width: 27%; float: left; padding-right: 3%;">
 
<h3>Approach, Plan</h3>
 
<h3>Approach, Plan</h3>
* Connect OpenIGTLink to OpenSoundcontrol for use in common sound synthesis software such as [https://puredata.info PureData], [https://cycling74.com/products/max/ Max], or [http://supercollider.github.io/ SuperCollider]
+
* common sound synthesis software: [https://puredata.info PureData], [https://cycling74.com/products/max/ Max], or [http://supercollider.github.io/ SuperCollider] DOWNLOAD PUREDATA EXTENDED>>> [https://puredata.info/downloads/pd-extended here]
* libraries for OpenSoundControl include [http://liblo.sourceforge.net/ liblo], [http://das.nasophon.de/pyliblo/ PyLiblo]
+
* Test program for Puredata to listen for incoming OSC messages: [http://wiki.na-mic.org/Wiki/images/8/81/Test_incoming_OSC.zip link]
 +
* libraries for OpenSoundControl include C++: [http://liblo.sourceforge.net/ liblo], Python: [http://das.nasophon.de/pyliblo/ PyLiblo]
 +
 
 
</div>
 
</div>
 
<div style="width: 27%; float: left; padding-right: 3%;">
 
<div style="width: 27%; float: left; padding-right: 3%;">
 
<h3>Progress</h3>
 
<h3>Progress</h3>
 
* Existing methods for intraoperative navigation guidance already implemented in [https://puredata.info Puredata], see [https://www.dropbox.com/s/6wfrklm8by4fx29/Dual-frequency-feedback.mov?dl=0 for ureteroscopy][http://wiki.na-mic.org/Wiki/index.php/File:Dual-frequency-feedback.mov download link], [https://www.dropbox.com/s/35rvm7kx30hwl08/WirbelBeispielMitAudio.mov?dl=0 for ablation needle insertion]
 
* Existing methods for intraoperative navigation guidance already implemented in [https://puredata.info Puredata], see [https://www.dropbox.com/s/6wfrklm8by4fx29/Dual-frequency-feedback.mov?dl=0 for ureteroscopy][http://wiki.na-mic.org/Wiki/index.php/File:Dual-frequency-feedback.mov download link], [https://www.dropbox.com/s/35rvm7kx30hwl08/WirbelBeispielMitAudio.mov?dl=0 for ablation needle insertion]
 +
* Tried out iplementing liblo library for OSC messages, this workson Mac but not well on Windows.
 +
* Switched to python-based library, which works well. Rocio tried this out but not enough time to integrate with their existing apps.
 +
* Friendly contact between groups who are interested in working together (Rocio, Tamas, Elvis, Longquan)
  
  
[[Code from Jay Jagadeesan for OgenIGTLink]], [http://www.na-mic.org/Wiki/images/3/3d/ReceiveClient.cxx.zip here as zip.]
+
*[[Code from Jay Jagadeesan for OgenIGTLink]], [https://na-mic.org/w/images/3/3d/ReceiveClient.cxx.zip here as zip.]
 
+
*Only for Linux and Mac system, as the "lo" library has some build issue with visual studio. For windows user, it is recommended to use the python version of lo [http://das.nasophon.de/pyliblo/  Python library for lo]
 +
*[http://www.na-mic.org/Wiki/index.php/File:AudioSimulator.zip A small lo simulator from Longquan Chen]
 +
** This simulator use the lo library, sends note to the local/remote server running PD-Extended.
 +
** Instructions about how to build and test the program is written in the readme.txt file
 
</div>
 
</div>
 
</div>
 
</div>

Latest revision as of 18:27, 10 July 2017

Home < 2016 Summer Project Week < Auditory Display

Key Investigators

  • David Black, University of Bremen, Fraunhofer MEVIS
  • Sarah Frisken, BWH/HMS
  • Christian Hansen, Uni Magdeburg
  • Longquan Chen, BWH/HMS
  • Tamas Ungi
  • Rocio Lopez
  • Elvis Chen
  • Julian Hettig

Related Work

Existing methods for intraoperative navigation guidance already implemented in Puredata, see

Possible application areas / IDEAS

  • Acquiring 3D data sets for US, did we acquire all we need?
  • Uncertainty in navigation information
  • Brain / Structure Shift
  • reduce complexity of displays by offloading to audio
  • Depth cues for 3D data (shown on 2D screens)
  • coverage of needle in US

Project Description

Objective

  • Explore possibilities of auditory display for intraoperative use
  • Find opportunities for extending existing projects with auditory display

Approach, Plan

  • common sound synthesis software: PureData, Max, or SuperCollider DOWNLOAD PUREDATA EXTENDED>>> here
  • Test program for Puredata to listen for incoming OSC messages: link
  • libraries for OpenSoundControl include C++: liblo, Python: PyLiblo

Progress

  • Existing methods for intraoperative navigation guidance already implemented in Puredata, see for ureteroscopydownload link, for ablation needle insertion
  • Tried out iplementing liblo library for OSC messages, this workson Mac but not well on Windows.
  • Switched to python-based library, which works well. Rocio tried this out but not enough time to integrate with their existing apps.
  • Friendly contact between groups who are interested in working together (Rocio, Tamas, Elvis, Longquan)