Difference between revisions of "Project Week 25/Human-Computer Interaction under sterile conditions"
From NAMIC Wiki
(14 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
− | |||
+ | __TOC__ | ||
Back to [[Project_Week_25#Projects|Projects List]] | Back to [[Project_Week_25#Projects|Projects List]] | ||
Line 6: | Line 6: | ||
*[http://isgwww.cs.uni-magdeburg.de/cas/ Christian Hansen] (University of Magdeburg, Germany) | *[http://isgwww.cs.uni-magdeburg.de/cas/ Christian Hansen] (University of Magdeburg, Germany) | ||
*[http://isgwww.cs.uni-magdeburg.de/cas/team.php Julian Hettig] (University of Magdeburg, Germany) | *[http://isgwww.cs.uni-magdeburg.de/cas/team.php Julian Hettig] (University of Magdeburg, Germany) | ||
+ | *[http://perk.cs.queensu.ca/users/lasso Andras Lasso] (PerkLab, Queen's) | ||
*[http://isgwww.cs.uni-magdeburg.de/cas/team.php Benjamin Hatscher] (University of Magdeburg, Germany) | *[http://isgwww.cs.uni-magdeburg.de/cas/team.php Benjamin Hatscher] (University of Magdeburg, Germany) | ||
*[http://www.researchgate.net/profile/David_Black11 David Black] (University of Bremen; Fraunhofer Institute for Medical Image Computing MEVIS, Bremen, Germany) | *[http://www.researchgate.net/profile/David_Black11 David Black] (University of Bremen; Fraunhofer Institute for Medical Image Computing MEVIS, Bremen, Germany) | ||
Line 30: | Line 31: | ||
## With OSC communication protocol, David will make quick, flexible sound methods | ## With OSC communication protocol, David will make quick, flexible sound methods | ||
## foot/eye interaction (?) | ## foot/eye interaction (?) | ||
− | |||
− | |||
| | | | ||
<!-- Progress and Next steps (fill out at the end of project week), please start each sentence in a new line. --> | <!-- Progress and Next steps (fill out at the end of project week), please start each sentence in a new line. --> | ||
+ | # Connected Slicer to OSC: added SlicerSoundControl extension (https://github.com/SlicerIGT/SlicerSoundControl), see below for necessary OSC message names | ||
+ | # Tested avoidance warning with experimental integration into breast lumpectomy navigation module (LumpNav): https://youtu.be/gSz8IHmogMo | ||
+ | # tried using gestures to zoom windows in and out | ||
+ | |||
+ | |} | ||
+ | |||
+ | ==Previous Work== | ||
+ | Demo Videos | ||
+ | * ablation needle or general tool placement, with U. Magdeburg and U. Hannover, [https://www.dropbox.com/s/35xzp51tk0rui6x/Singing.m4v?dl=0 “singing" method] and [https://www.dropbox.com/s/05et9abov10yvmk/AblationDemoVideo1.m4v?dl=0 synthetic sounds] | ||
+ | * [https://www.dropbox.com/s/ullaaab4gp9pucq/BWH_Dual-frequency-feedback.mp4?dl=0 ureteroscopy], with Harvard Surgical Planning Lab | ||
+ | * [https://www.dropbox.com/s/cdfc77ugjgg8o96/LIM%20Short.mp4?dl=0 sacral neuromodulation], with University Carlos III Madrid | ||
+ | * [https://www.dropbox.com/s/bn0u0g22yu14uvt/AD%20Resection%20short.m4v?dl=0 resection guidance], with Robert Bosch Hospital Stuttgart ([https://www.youtube.com/watch?v=gCg5nJSI2pY longer version with explanation]) | ||
+ | |||
+ | In addition, please see the following articles: | ||
+ | * [https://link.springer.com/article/10.1007/s11548-017-1537-1 Auditory feedback to support image-guided medical needle placement] | ||
+ | * [https://link.springer.com/article/10.1007/s11548-017-1547-z A Survey of auditory display in image-guided interventions] | ||
+ | * [https://www.researchgate.net/publication/233791033_Auditory_support_for_resection_guidance_in_navigated_liver_surgery Auditory support for resection guidance in navigated liver surgery.] | ||
+ | |||
+ | ==Illustrations== | ||
+ | |||
+ | {| class="wikitable" | ||
+ | ! style="text-align: left; width:50%" | [[image:OpenSoundControl.png|400px]] OpenSoundControl module for configuring and testing OpenSoundControl server communication | ||
+ | ! style="text-align: left; width:50%" | [[image:SoundNavigation.png|400px]] SoundNavigation module for automatically generating OpenSoundControl messages for tool navigation, from tool and reference transform nodes | ||
+ | |- style="vertical-align:top;" | ||
|} | |} | ||
+ | |||
+ | <embedvideo service="youtube">https://youtu.be/gSz8IHmogMo</embedvideo> | ||
+ | |||
+ | ==SlicerSoundControl== | ||
+ | see https://github.com/SlicerIGT/SlicerSoundControl | ||
+ | |||
+ | === Environment for Testing Slicer Sound Control === | ||
+ | |||
+ | OSC Message names: | ||
+ | |||
+ | Base name: /SoundNav/Instrument/ | ||
+ | |||
+ | Translation: each fro 0 to 100 mm | ||
+ | * /TranslationX | ||
+ | * /TranslationY | ||
+ | * /TranslationZ | ||
+ | * /Distance (absolute distance) | ||
+ | |||
+ | Translation: each from 0 to 180 degrees | ||
+ | * /OrientationX | ||
+ | * /OrientationY | ||
+ | * /OrientationZ | ||
+ | * /Orientation (overall orientation) | ||
− | + | GUI for test: | |
+ | [[File:OSCTest GUI.png|500px|frameless|OSCTest GUI]] | ||
− | + | === Environment for Sterile Gestures (planned, not yet implemented) === | |
− | * In/ out | + | * In/ out |
+ | ** --> "/InOut 0" out | ||
+ | ** --> "/InOut 1" in | ||
* Edge distance (0 to 3 cm?) | * Edge distance (0 to 3 cm?) | ||
+ | ** --> "/EdgeDist x" where x is float between 0 and 3 | ||
− | + | === Ambient === | |
* Overall acceleration | * Overall acceleration | ||
+ | ** --> "/OverallAccel x" where x is float between 0 and 1 | ||
* Background noise | * Background noise | ||
* Absolute position (x/y, not depth) to show current selected window | * Absolute position (x/y, not depth) to show current selected window | ||
+ | ** --> "/AbsPolX x" where x is float between 0 and 1 | ||
+ | ** --> "/AbsPolY x" where x is float between 0 and 1 | ||
− | + | === Gestures === | |
* select image | * select image | ||
+ | ** --> "/SelectGesture x" where x is gesture number | ||
* zoom to left window | * zoom to left window | ||
* zoom to right window | * zoom to right window |
Latest revision as of 13:23, 30 June 2017
Home < Project Week 25 < Human-Computer Interaction under sterile conditionsContents
Back to Projects List
Key Investigators
- Christian Hansen (University of Magdeburg, Germany)
- Julian Hettig (University of Magdeburg, Germany)
- Andras Lasso (PerkLab, Queen's)
- Benjamin Hatscher (University of Magdeburg, Germany)
- David Black (University of Bremen; Fraunhofer Institute for Medical Image Computing MEVIS, Bremen, Germany)
- Marco Nolden (German Cancer Research Center (DKFZ), Germany)
- Juan Ruiz Alzola (University of Las Palmas de Gran Canaria, Spain)
- Salvatore Scaramuzzino ("Magna Graecia" University - ASL Vercelli, Italy)
Project Description
Objective | Approach and Plan | Progress and Next Steps |
---|---|---|
Human-Computer Interaction under Sterile Conditions.
|
|
|
Previous Work
Demo Videos
- ablation needle or general tool placement, with U. Magdeburg and U. Hannover, “singing" method and synthetic sounds
- ureteroscopy, with Harvard Surgical Planning Lab
- sacral neuromodulation, with University Carlos III Madrid
- resection guidance, with Robert Bosch Hospital Stuttgart (longer version with explanation)
In addition, please see the following articles:
- Auditory feedback to support image-guided medical needle placement
- A Survey of auditory display in image-guided interventions
- Auditory support for resection guidance in navigated liver surgery.
Illustrations
SlicerSoundControl
see https://github.com/SlicerIGT/SlicerSoundControl
Environment for Testing Slicer Sound Control
OSC Message names:
Base name: /SoundNav/Instrument/
Translation: each fro 0 to 100 mm
- /TranslationX
- /TranslationY
- /TranslationZ
- /Distance (absolute distance)
Translation: each from 0 to 180 degrees
- /OrientationX
- /OrientationY
- /OrientationZ
- /Orientation (overall orientation)
Environment for Sterile Gestures (planned, not yet implemented)
- In/ out
- --> "/InOut 0" out
- --> "/InOut 1" in
- Edge distance (0 to 3 cm?)
- --> "/EdgeDist x" where x is float between 0 and 3
Ambient
- Overall acceleration
- --> "/OverallAccel x" where x is float between 0 and 1
- Background noise
- Absolute position (x/y, not depth) to show current selected window
- --> "/AbsPolX x" where x is float between 0 and 1
- --> "/AbsPolY x" where x is float between 0 and 1
Gestures
- select image
- --> "/SelectGesture x" where x is gesture number
- zoom to left window
- zoom to right window
- put back into small viewer
Background
Example of previous related work using standard surgical gloves and OR compatible plastic draping confirmed to be compatible with sterile requirements at BWH AMIGO:
]