Difference between revisions of "Project Week 25/Human-Computer Interaction under sterile conditions"

From NAMIC Wiki
Jump to: navigation, search
Line 43: Line 43:
 
|}
 
|}
  
==Necessary Parameters==
+
==SlicerSoundControl==
 
see https://github.com/SlicerIGT/SlicerSoundControl
 
see https://github.com/SlicerIGT/SlicerSoundControl
  
/BlackLegend
+
=== Environment for Testing Slicer Sound Control ===
  
=== Environment ===
+
OSC Message names:
 +
 
 +
Base name: /SoundNav/Instrument/
 +
 
 +
Translation: each fro 0 to 100 mm
 +
/TranslationX
 +
/TranslationY
 +
/TranslationZ
 +
/Distance  (absolute distance)
 +
 
 +
Translation: each from 0 to 180 degrees
 +
/OrientationX
 +
/OrientationY
 +
/OrientationZ
 +
/Orientation (overall orientation)
 +
 
 +
=== Environment for Sterile Gestures (planned, not yet implemented) ===
 
* In/ out  
 
* In/ out  
 
** --> "/InOut 0" out
 
** --> "/InOut 0" out

Revision as of 12:03, 30 June 2017

Home < Project Week 25 < Human-Computer Interaction under sterile conditions

Back to Projects List

Key Investigators

Project Description

Objective Approach and Plan Progress and Next Steps

Human-Computer Interaction under Sterile Conditions.

  • Review of state of the art (tochless interaction)
  • Development of new user interfaces to support surgical interterventions
  1. Long, intensive conceptualization (2 days) of possibilities for gesture interaction and audio feedback for it
    1. We will need moderation kit (paper, pencil, markers, etc)
  2. Prototypes of possible auditory/visual feedback based on conceptualization
    1. With OSC communication protocol, David will make quick, flexible sound methods
    2. foot/eye interaction (?)
  3. File:OSC Test Input.pd.zip
    1. Here is a test patch for incoming messages, also makes sound. Need to download PD-Extended to use this.
  1. Connected Slicer to OSC: added SlicerSoundControl extension (https://github.com/SlicerIGT/SlicerSoundControl)
  2. Tested avoidance warning with experimental integration into breast lumpectomy navigation module (LumpNav): https://youtu.be/gSz8IHmogMo
  3. tried using gestures to zoom windows in and out

SlicerSoundControl

see https://github.com/SlicerIGT/SlicerSoundControl

Environment for Testing Slicer Sound Control

OSC Message names:

Base name: /SoundNav/Instrument/

Translation: each fro 0 to 100 mm /TranslationX /TranslationY /TranslationZ /Distance (absolute distance)

Translation: each from 0 to 180 degrees /OrientationX /OrientationY /OrientationZ /Orientation (overall orientation)

Environment for Sterile Gestures (planned, not yet implemented)

  • In/ out
    • --> "/InOut 0" out
    • --> "/InOut 1" in
  • Edge distance (0 to 3 cm?)
    • --> "/EdgeDist x" where x is float between 0 and 3

Ambient

  • Overall acceleration
    • --> "/OverallAccel x" where x is float between 0 and 1
  • Background noise
  • Absolute position (x/y, not depth) to show current selected window
    • --> "/AbsPolX x" where x is float between 0 and 1
    • --> "/AbsPolY x" where x is float between 0 and 1

Gestures

  • select image
    • --> "/SelectGesture x" where x is gesture number
  • zoom to left window
  • zoom to right window
  • put back into small viewer

Background

example HCI with freehand gestures for sound


Example of previous related work using standard surgical gloves and OR compatible plastic draping confirmed to be compatible with sterile requirements at BWH AMIGO: ]

References

Mewes et al. (2017) Touchless Interaction with Software in Interventional Radiology and Surgery: A Systematic Literature Review

Hettig et al. (2017) Comparison of Gesture and Conventional Interaction Techniques for Interventional Neuroradiology