|
|
| (134 intermediate revisions by 4 users not shown) |
| Line 1: |
Line 1: |
| − | [[Slicer3:Interface_Design|Return to Slicer3 Interface Design and Usability ]]
| + | <big>'''Note:''' We are migrating this content to the slicer.org domain - <font color="orange">The newer page is [https://www.slicer.org/wiki/Slicer3:UIDesign here]</font></big> |
| − | | |
| − | = Slicer3 design & usability goals =
| |
| − | | |
| − | 1. Enable a user to understand and effectively use the content and tools being presented;
| |
| − | | |
| − | 2. Enable a user to accomplish a principle task by following an appropriate and satisfying workflow or a curiosity-driven exploration, at an appropriate pace;
| |
| − | | |
| − | 3. Enhance and support the developer’s experience with GUI infrastructure and guidelines that simplify and clarify their work.
| |
| − | | |
| − | = Core values =
| |
| − | | |
| − | These are the established core values of the 3D Slicer software and development effort:
| |
| − | | |
| − | Software associations:
| |
| − | | |
| − | * Clarity & usability
| |
| − | * Control & Precision
| |
| − | * Information richness
| |
| − | * Interactive & responsive
| |
| − | * Reliable & Trusted
| |
| − | * Easily extensible
| |
| − | * Open source & cross-platform
| |
| − | * Showcase for advanced research
| |
| − | | |
| − | Effort associations:
| |
| − | | |
| − | * Advancing scientific research
| |
| − | * Assisting treatment/therapy
| |
| − | * Established and long-term
| |
| − | | |
| − | = Embodiment =
| |
| − | | |
| − | * Current schematic of the main application GUI: [[Media:GUIMap.png ]]
| |
| − | * Current schematic of a Slice Viewer and Controller: [[ Media:SliceGUI.png ]]
| |
| − | * Design sketch for the 3D navigation & Slice magnifier widget [[ Media:NavZoomWidgetSketch.png]]
| |
| − | * Early look and feel draft [[Media:SlicerLookFeelDraftSmall.jpg]]
| |
| − | * First sketch of the GUI layout design [[Media:LayoutSketch.gif]]
| |
| − | * Link to a running [[Slicer3:VisualBlog|visual blog ]] highlighting progress in the Slicer3 embodiment.
| |
| − | | |
| − | == Current status and TO-DO ==
| |
| − | | |
| − | The existing base classes form a basic framework for the UI. For the Alpha/Beta releases, we provided a main application window with the general layout described in our current design. For the Release deadline, we plan to complete the overall application GUI. Things we are focusing on:
| |
| − | | |
| − | * make default font size a function of display size.
| |
| − | * implementing functionality for view control widgets,
| |
| − | * implementing lightbox viewer
| |
| − | * implement GUIs for specification of oblique views.
| |
| − | * design/implement mouse-mode interaction paradigm (using one mouse button, hot-keys, and view xform or action mouse-modes).
| |
| − | * design mouse-mode icons and mouse-mode cursors.
| |
| − | * build base classes for pop-up information.
| |
| − | * copyright Slicer base icons and logo (all versions)
| |
| − | | |
| − | == Working problem: controlling mouse mode in Slicer's Viewers ==
| |
| − | | |
| − | We are currently deciding a good paradigm for controlling mouse modes in Slicer's
| |
| − | 3D viewer and 2D slice windows. We rely on mouse interaction to transform the view, and to perform explicit actions like placing a fiducial, selecting and moving a fiducial, collecting voxel-value samples, and so on. The challenge is to make all actions possible with just the left mouse button, and to disambiguate what a left-mouse-button click in the 3D and Slice viewers means. We'd like the interaction paradigm to have the following features:
| |
| − | | |
| − | * Consistent in all slice windows and 3D viewer
| |
| − | * Intuitive and easy for beginners
| |
| − | * Fast for intermediate and expert users
| |
| − | * Rely on only one mouse button (since some mice have only two, and some laptops give us less to work with)
| |
| − | * Support direct manipulation
| |
| − | * Provide clear visual feedback
| |
| − | * Support both '''view transformation''' (rotate, pan, zoom) and '''mouse actions''' (pick, manipulate, place, etc...).
| |
| − | | |
| − | --- | |
| − | | |
| − | ===Current problems in existing Slicer3 implementation:===
| |
| − | | |
| − | * Mouse motion always in "view transform" mode
| |
| − | * Relies on three mouse buttons (for rotate, pan, zoom, slice scroll) which aren't always present
| |
| − | * Relies on hot-keys for non-transform mouse actions (tough for novice users, or forgetful intermediate users)
| |
| − | * 3D viewer mouse event bindings and Slice viewer mouse event bindings not entirely consistent
| |
| − | * Would like to free up right mouse button (and/or space bar) to invoke the display of a 'context menu'.
| |
| − | * Would like intuitive one-handed mouse-only interaction to be possible AND hot-key assisted shortcuts to be available.
| |
| − | | |
| − | In addition, in Slicer2, special keys were used in place of application-wide paradigms for actions. For instance, Fiducials used 'p' to place a fiducial, 'q' to select it. Other modules might redefine these key-bindings.
| |
| − | | |
| − | | |
| − | ====Current bindings (not entirely consistent across 3D Viewer and Slice Viewers):====
| |
| − | | |
| − | * 3D viewer: left mouse button >> rotate
| |
| − | * 3D viewer: middle mouse button >> pan
| |
| − | * 3D viewer: mouse wheel button >> zoom in/out
| |
| − | * 3D viewer: right mouse button, pull drag >> zoom in
| |
| − | * 3D viewer: right mouse button, push drag >> zoom out
| |
| − | | |
| − | * Slice viewer: left mouse button >> select/move a fiducial point
| |
| − | * Slice viewer: middle mouse button >> pan
| |
| − | * Slice viewer: mouse wheel button >> scroll slices
| |
| − | * Slice viewer: right mouse button, pull drag >> zoom in
| |
| − | * Slice viewer: right mouse button, push drag >> zoom out
| |
| − | * special key-bindings enable placing of fiducials
| |
| − | | |
| − | ---
| |
| − | | |
| − | ====How Actions and View Transformations are done in Maya====
| |
| − | | |
| − | Maya is a very heavily used professional 3D animation package. Basic mouse bindings are:
| |
| − | | |
| − | * Rotate: alt-left mouse button
| |
| − | * Pan: alt-middle mouse button
| |
| − | * Zoom: alt-right or alt-right-and-middle mouse button
| |
| − | * Actions: always left mouse button, specific action depends on current mode
| |
| − | * Context menu: space bar
| |
| − | | |
| − | The advantage of this approach is that the left mouse button is freed up for many possible tasks such as selection, grabbing widget handles, etc. Disadvantage is that it requires two hands to operate. (A free copy of an almost-fully functional version of Maya is [http://www.autodesk.com/maya-trial available] for testing ). | |
| − | | |
| − | | |
| − | | |
| − | ====How Actions and View Transformations are done in Blender====
| |
| − | | |
| − | [http://www.blender.org Blender] is an Open Source modeling and animation package similar in many ways to Maya (almost like GIMP is to Photoshop). Basic mouse bindings are:
| |
| − | | |
| − | * Rotate: alt-left mouse button OR middle mouse button
| |
| − | * Pan: shift-middle mouse button
| |
| − | * Zoom: control-middle mouse button
| |
| − | * Actions: mostly left mouse button
| |
| − | * Context menu: space bar
| |
| − | | |
| − | | |
| − | ====Note: Mac bindings====
| |
| − | | |
| − | On a one-button mac, the following translations work by default in Mac OSX:
| |
| − | * single button -> left mouse button
| |
| − | * alt (option) + button -> middle mouse button
| |
| − | * apple (command) + button -> right mouse button
| |
| − | | |
| − | ---
| |
| − | | |
| − | ===Proposed approach -- please comment!:===
| |
| − | | |
| − | We are looking for user feedback on a proposed approach to solving the problems listed above, in a manner that echos what other 3D modeling packages do: providing explicit '''mouse modes''' which clearly differentiate whether a user is transforming a view, or performing some "action" by clicking or dragging the mouse. Please have a look at the interaction paradigm we're proposing and the description of the workflow it would generate for novice users and for expert users. This way of interacting is a little different from Slicer2's paradigm. Also see the analogous interfaces from Maya (a popular 3D modeling and rendering software package) and also SketchUp (which does some simple but powerful 3D modeling). Please contribute any feedback you have in the bottom section. Thanks for your help!
| |
| − | | |
| − | | |
| − | '''Features of the interaction design:'''
| |
| − | | |
| − | * Mouse mode icons in "selected" (top) and "deselected" (bottom) states: [[image:MouseModeIcons.png | icons ]]
| |
| − | * Selected state indicates current mouse mode:
| |
| − | ** pick+manipulate [[image:ToolbarMouseManipulate.png]] pick and manipulate an object as appropriate, like moving a fiducial point, or grabbing and manipulating a 3D vtkWidget;
| |
| − | ** region-select [[image:ToolbarMouseSelectRegion.png]] rubber-band box to select multiple objects, selected state is persistent until deselected;
| |
| − | ** lasso-a-region [[image:ToolbarMouseLasso.png]] manually drawn outline to select multiple objects, persistent until deselected;
| |
| − | ** deselect all [[image:ToolbarMouseDeselectAll.png]] not really a *mode* per se. The Deselect-All icon becomes highlighted when anything has been "selected" with the region or lasso (persistent selectors), and it becomes lowlighted when nothing is "selected". Clicking the button when highlighted deselects all, and then lowlights the DeselectAll icon;
| |
| − | ** put [[image:ToolbarMousePlace.png]] for placing fiducials, seed points, etc.;
| |
| − | ** view-tumble [[image:ToolbarMouseRotate.png]];
| |
| − | ** view-track [[image:ToolbarMousePan.png]];
| |
| − | ** view-dolly [[image:ToolbarMouseZoom.png]].
| |
| − | * Each mouse-mode causes a special cursor to be displayed, as feedback to user.
| |
| − | * Hot-keys are contiguous 1 2 3 4 5 6 7 8 on the keyboard.
| |
| − | * All mouse modes apply consistently in 3D Viewer and Slice Viewers (except the view-tumble mode, which is not defined in the Slice Viewers. When Slicer is in view-tumble mode, either the view-tumble cursor would be displayed in the Slice Viewers but nothing would happen when the mouse is clicked *or* the pick-and-manipulate cursor will be displayed in the Slice Viewers -- undecided.)
| |
| − | * Novice user can select mouse modes by clicking icons above 3D viewer, and current mode is clearly displayed.
| |
| − | * Intermediate or Expert user selects mouse modes by using a combination of keyboard keys and mouse buttons.
| |
| − | * Works with one (left) mouse button only, but other mouse buttons are mapped (for tumble, track and zoom) for people who have and want to use them.
| |
| − | * Context menu can be displayed with space bar, or with quick click-and-release of right mouse button.
| |
| − | | |
| − | | |
| − | ---
| |
| − | | |
| − | Shown in the figure below:
| |
| − | | |
| − | * the icons for mouse-modes (their appearance when highlighted and lowlighted),
| |
| − | * suggested hot-keys and mouse-selection actions, and
| |
| − | * cursor appearance when any one mode is chosen.
| |
| − | * an example workflow for novice and expert users to perform the same simple task.
| |
| − | | |
| − | [[image:mouseModeMockup.png | 3D Slicer Mouse mode interaction design ]]
| |
| − | | |
| − | ---
| |
| − | | |
| − | ===Workflow(s)===
| |
| − | Below is how a user would use the mouse-mode interface to
| |
| − | tumble, track and dolly the camera to get the 3D View to look the way they like, and then put down three fiducial points and adjust their position.
| |
| − | | |
| − | A novice user's workflow is described and an expert user's workflow using for the same task is given for comparison.
| |
| − | | |
| − | ---
| |
| − | | |
| − | ====A '''novice''' user's workflow using these mouse-mode icons (top row: selected; bottom row: deselected):====
| |
| − | 1 '''tumble:''' click the view tumble mouse-mode icon [[image:ToolbarMouseRotate.png]]
| |
| − | * --> mouse-tumble icon highlights and all others deselect, cursor changes to indicate mouse-tumble mode.
| |
| − | * then left-Click and drag in the 3D Viewer to rotate the view,
| |
| − | 2 '''track:''' click the view track mouse-mode icon [[image:ToolbarMousePan.png]]
| |
| − | * --> mouse-track icon highlights and all others deselect, cursor changes to indicate mouse-track mode.
| |
| − | * then left-Click and drag in the 3D Viewer to translate the view left-right, and up-down,
| |
| − | 3 '''dolly:''' click the view dolly mouse-mode icon [[image:ToolbarMouseZoom.png]]
| |
| − | * --> mouse-dolly icon hightlights and all others deselect, cursor changes to i ndicate mouse-dolly mode.
| |
| − | * then left-Click and drag in the 3D Viewer to dolly the camera in and out.
| |
| − | 4 '''place fiducials:''' click the mouse-place icon [[image:ToolbarMousePlace.png]]
| |
| − | * --> mouse-place icon highlights and all others deselect, cursor changes to indicate mouse-place mode.
| |
| − | * then left-Click three times at x,y locations in a Slice Viewer to put down three fiducial points.
| |
| − | 5 '''zoom the slice view:''' click the mouse-dolly icon [[image:ToolbarMouseZoom.png]]
| |
| − | * --> mouse-dolly icon highlights and all others deselect, cursor changes to indicate mouse-dolly mode.
| |
| − | * then left-Click and drag in a Slice Viewer to zoom in.
| |
| − | 6 '''select and move fiducials:''' click the mouse-pick-and-manipulate icon [[image:ToolbarMouseManipulate.png]]
| |
| − | * --> mouse-pick-and-manipulate icon highlights, all others deselect, cursor changes to indicate mouse-pick-and-manipulate mode.
| |
| − | * move mouse to location of fiducial in a Slice Viewer and see the fiducial highlight; left-Click and drag in a Slice Viewer to reposition the fiducial point.
| |
| − | * move the mouse away from the fiducial and see it lowlight.
| |
| − | 7 '''pan the slice view:''' click the mouse-track icon [[image:ToolbarMousePan.png]]
| |
| − | * --> mouse-pan icon highlights and all others deselect, cursor changes to indicate mouse-pan mode.
| |
| − | * left-Click and drag in a Slice View to pan.
| |
| − | 8 '''select multiple fiducials:''' click the mouse select-region icon [[image:ToolbarMouseSelectRegion.png]]
| |
| − | * --> mouse-select-region icon highlights and all others deselect, cursor changes to indicate mouse-select-region mode.
| |
| − | * sweep out a rubber band to enclose two fiducials and see them highlight.
| |
| − | 9 '''move fiducials:''' click the mouse-pick-and-manipulate icon [[image:ToolbarMouseManipulate.png]]
| |
| − | * --> mouse-pick-and-manipulate icon highlights, all others deselect, cursor changes to indicate mouse-pick-and-manipulate mode.
| |
| − | * left-Click and drag in a Slice Viewer to reposition the fiducial points.
| |
| − | 10 '''de-select fiducial:''' click the mouse-deselect-all icon [[image:ToolbarMouseDeselectAll.png]]
| |
| − | * --> mouse-deselect icon hightlights, all other icons deselect, and the fiducials are deselected.
| |
| − | | |
| − | ---
| |
| − | | |
| − | ====An '''expert''' user's workflow using hot-keys (icon state provides visual feedback of mode):====
| |
| − | 1 '''tumble:''' ctrl+left-Click and drag in the 3D Viewer,
| |
| − | * --> (mouse-tumble icon [[image:ToolbarMouseRotate.png]] highlights and all others deselect; cursor changes to indicate mouse-tumble mode; view tumbles)
| |
| − | 2 '''track:''' alt+left-Click and drag in the 3D Viewer,
| |
| − | * --> mouse-track icon [[image:ToolbarMousePan.png]] highlights and all others deselect; cursor changes to indicate mouse-track mode; view tracks
| |
| − | 3 '''dolly:''' shift+left-Click and drag in the 3D Viewer
| |
| − | * --> mouse-dolly icon [[image:ToolbarMouseZoom.png]] highlights and all others deselect; cursor changes to indicate mouse-dolly mode; view zooms
| |
| − | 4 '''place fiducials:''' press 5-key to switch into mouse-put (or place) mode
| |
| − | * --> mouse put icon [[image:ToolbarMousePlace.png]] highlights, and all others deselect; cursor changes to indicate mouse-place mode,
| |
| − | * then left-Click on three x,y locations in a Slice Viewer to put down three new fiducial points.
| |
| − | 5 '''dolly:''' press 8-key and drag in the 3D Viewer
| |
| − | * --> mouse-dolly icon [[image:ToolbarMouseZoom.png]] highlights and all others deselect; cursor changes to indicate mouse-dolly mode; view zooms
| |
| − | 6 '''pick-and-manipulate fiducials:''' press 1-key to switch into mouse-pick-and-manipulate mode.
| |
| − | * --> mouse pick-and-manipulate icon [[image:ToolbarMouseManipulate.png]] highlights, and all others deselect; cursor changes to indicate mouse-pick-and-manipulate mode,
| |
| − | * then mouse-over a fiducial point, see it highlight, left-Click and drag to reposition it. Release mouse button and move away, and see the fiducial lowlight.
| |
| − | 7 '''track the view:''' press 7-key to switch into mouse-track mode.
| |
| − | * --> mouse-track icon [[image:ToolbarMousePan.png]] highlights and all others deselect; cursor changes to indicate mouse-track mode; left-Click and drag in the Slice Viewer to pan the view.
| |
| − | 8 '''select multiple fiducials:''' press 2-key to switch into region-select mode.
| |
| − | * --> mouse select-region icon [[image:ToolbarMouseSelectRegion.png]] highlights, all other icons deselect; cursor changes to indicate mouse-region-select mode.
| |
| − | * sweep out a rubber-band box to enclose two fiducial points, see them highlight.
| |
| − | 9 '''manipulated selected fiducials:''' press 1-key to switch into mouse-pick-and-manipulate mode.
| |
| − | * --> mouse pick-and-manipulate icon [[image:ToolbarMouseManipulate.png]] highlights, and all others deselect; cursor changes to indicate mouse-pick-and-manipulate mode,
| |
| − | * left-Click and drag in the Slice Viewer to reposition multiple fiducials.
| |
| − | 10 '''de-select fiducials:''' press 4-key again to deselect the fiducial points
| |
| − | * --> the deselect-all icon [[image:ToolbarMouseDeselectAll.png]] highlights and all other icons are deselected, the fiducial is no longer highlighted.
| |
| − | | |
| − | | |
| − | | |
| − | ---
| |
| − | | |
| − | ===Please contribute comments here -- Thank you!: ===
| |
| − | | |
| − | | |
| − | ---
| |
| − | | |
| − | === Slicer mockup and GUI screenshots from other packages: ===
| |
| − | Below is how the proposed interface using these icons would appear; Slicer would start up in mouse-tumble mode, showing the tumble cursor over the 3D Viewer. Beneath that image are SketchUp's interface, and Maya Interface, and the proposed Slicer mouse-mode icons (selected and deselected) as proposed. Maya's mouse modes switch between object transforms (translate, rotate and scale), object or region select, and an extrude function. SketchUp's mouse-mode icons switch between view transforms (translate, rotate, zoom) and an object extrude function.
| |
| − | | |
| − | [[image:mouseModeMockup2.png | 3D Slicer Mouse mode ]]
| |
| − | | |
| − | [[image:sketchupInterface.png | SketchUp Example ]]
| |
| − | | |
| − | [[image:mayaInterface.png | Maya Example ]]
| |
| − | | |
| − | | |
| − | ---
| |
| − | | |
| − | | |
| − | | |
| − | <br />''' Return to TOC ''' | |
| − | | |
| − | [[Slicer3:Interface_Design|Return to Slicer3 Interface Design and Usability ]]
| |
