Projects:RegistrationDocumentation:Developer

Back to ARRA main page
Back to Registration main page
Registration Use-case Inventory

Registration Modules

Separate those functions that use different algorithms. Main distinctions are affine vs. b-spline' and intensity vs. fiducials. Combinations are done by the user as workflows/pipelines. This provides clean modular design where changes in one do not easily spill over to the other.

1. Intensity Affine (exists, under construction)

creates Affine Xform, Input: image pair.

Parameters: DOF (3,6,7,9,12), multi-res level, sampling %, cost function, stop criterion fail (iteration), mask image or ROI

2. Intensity BSpline (exists, scheduled to become a separate registration submodule)

creates BSpline Xform, Input: Affine aligned image pair.

Parameters: grid size/spacing, multi-res level, sampling %, cost function, stop criterion fail (iteration), mask image or ROI

3. Manual Affine (exists, but not as registration module, needs scale and shear added)

interactive visual alignment with 6-12 DOF using Transforms module. This is a good tool for cases in need of an initialization.

4. Fiducials Affine (exists, needs to become a separate registration submodule)

solves Procrustes registration problem with 6-12 DOF using ICP or similar Input: fiducial pairs.

Parameters: none

5. Fiducials BSpline (does not exist)

creates an interpolation grid from fiducials (Krigging or similar): Input: fiducial pairs

Parameters: grid size/spacing, continuity constraints

Registration - Related Modules

6. Xform Concatenation

combines multiple Xforms

Status: exists, needs to be expanded and obtain more prominent position as separate Registration Submodule.

7. Interpolator

applies Affine and/or BSpline Xform and resamples image data

Status: exists, integrated into indiv. registration methods, isolated on ITK level, needs more uniform representation in GUI

8. Evaluation

registration quality assessment.

This may be better integrated with each of the above modules instead of a separate module, esp. when using measurements such as final cost function. Visualization also may depend on DOF.

Parameter GUI

• DOF: radio buttons (3,6,7,9,12)
• Multi-resolution: check boxes (2x, 4x, 8x, 16x)
• Combinations of the two above as a single list of checkboxes for increasing DOF. Media:DOFHierarchy_GUI.jpg‎
• Cost function: radio buttons (MI, NC, MSq, IR)
• Mask Image/ROI: pulldown menu for MRML nodes (labelmap or ROI from ROI module)

Masking / ROI

• High on the priority list for GUI implementation is a link to the ROI Module. This will enable the user to quickly focus on the area of interest and restrict active registration to this region. The registration module will therefore accept 2 kinds of masking input: 1) a box ROI as defined with the ROI module, 2) a masking image. While possible to convert the box ROI into a binary mask image, a direct link will greatly facilitate processing pipeline and also bring a performance boost within the algorithm.

Presets

• Parameter Presets: To navigate the jungle of parameter settings and avoid complex GUI design efforts, all parameters are listed and controllable via a preset dropdown menu (see image). The menu reflects a subset of the full hierarchy accessible via a Wizard/Organizer. Users can save and add their own presets. Adding a shared or downloaded preset consists of dropping the preset MRML file into a dedicated preset/settings folder and restarting slicer. Each preset is associated with a MRML file like this one. . Adding all possible combinations would make the menu too long. Consider hierarchical menu or an organizer to add/remove individual items.
• Parameter Settings Wizard: the wizard is basically the hierarchical form of the Parameter Presets. As the presets become comprehensive, the list evolves into a tree (which corresponds to our Use Case Hierarchy Tree). Traversing this tree can be done with a Wizard that asks questions to branch. At the end is a set of presets. So there's 3 ways to pick the parameters: 1) directly from the dropdown Custom Presets menu, 2) via the Wizard, 3) by setting each manually

Algorithm/Strategic

• Transform Concatenation: (partially available: harden transforms) Multiple spatial transforms applied to the same volume appear as nested MRML entries, which can be collapsed. Important to apply before a volume resampling is executed to avoid accumulation of interpolation effects. Affine is simple matrix multiplication, Bspline would have to send the gridpoints through all Xforms and reparameterize. A complete system of arranging multiple registrations also needs an easy copy function for all MRML entry data (images and Xforms). More Details.
• Recursive DOF hierarchy: Both multi-resolution and the sequence of calls with increasing DOF could be implemented as a single recursive scheme. The caller provides an array of DOF and a matching array of resolution levels, which is processed by recursive calls with the first/last element taken from the list. A simulation Matlab program and example is here: Projects:RegistrationImprovement:RecursiveScheme

• Offline WIKI: can the documentation WIKI be downloaded and used offline, e.g. for laptop users?
• Affine SVD: (in development) decomposition of affine into the 4 3DOF components: translation, rotation, scaling, shearing. Can an SVD do this for 12DOF vs. 3 DOF? Having values that relate directly to geometry can help greatly in evaluating and adjusting results. E.g. user can tell visually which of the 3 rotations is most out of alignment. Checking the result rotation angles already provides a quick check and valuable info about where things went awry if solution is suboptimal. Also enables to decouple individual elements, i.e. extract the shear component etc. The current "Transforms" module already does this for translation & rotation, but the scale and shear portions are missing.
• Distortion Correction: e.g. spherical harmonics Gradient Nonlinearity correction (BIRN) is a form of spatial Xform to be housed in the Registration or Transformations module.
• Volumes/Display' would it be possible to have a switch in the selected volume here automatically trigger a switch in the FG/BG toggle if all FG/BG views have uniform images assigned (i.e. all FG panels show same image)

Result Evaluation Tools

• Registration Metrics: Can we use the Label Map Hausdorff Distance Feature ?
• Methods design: We can categorize 2 ways at the top level: Affine/BSpline or Fiducial/Intensity. All 4 combinations will likely use different algorithms: Fiducial+Affine = Procrustes, Fiducial+Bspline = direct fitting/krigging , Intensity+Affine = 12 DOF optimizer, Intensity+BSpline = non-rigid. Fiducial/Intensity obviously use different cost function, whereas Affine/BSpline is a distinction in spatial DOF. So Fiducial/Intensity seems to be the more intuitive choice as top level distinction.
• Registration Evaluation/Quality Assessment: tools for the user to tell : "how good is this?" "can I get this better still?" "is this about as good as it gets?"
  • Compare 2 transforms: how diffferent are 2 transforms: run fiducial set through one and the invert of the other and report distance. Allow cross-fading between 2 transforms w/o creating duplicate
  • Checkerboard Display, as shown in Vervet Registration Example.
  • Fiducial RMS Error fiducial markers set by the user on both images. Tool returns RMS distance error. Pick points inside and at the margins of your analysis ROI, i.e. if taking metrics from entire brain, use fiducials posterior and anterior, superior and inferior, because alignment quality may differ. The tool should also return the variance of RMS, because that gives an idea of possible improvement: if the error is very low in some but large in other areas, chances for improvement may be less than if a steady error bias is seen across all fiducials. Hausdorff Distance.
  • ROI Similarity Metric: quick similarity metric report on a manually chosen ROI, e.g. draw a box and see the image similarity metric in that box.
  • Cost Function Visualizer: visualize the cost function optimization curve. and possibly a history of the tried movement: e.g. a movie of a grid/box moving thru the optimization. That will tell the user if a particular position was ever explored or not, and if/which parameters should be adjusted based on that.
    • extract cost function value as registr. quality metric (-> currently via ErrorLog )
    • graphic representation of registr. progress (both cartesian and cost function), maybe realtime?
    • stepable movie of concatenated functions (e.g. rigd, affine, B-spline etc. visualized on simple grid/cube)
  • Fiducial Weights: this is tricky, but consider a set of manually set fiducials to be integrated with the intensity metric 1) make sure those points are among the point set and have a large (custom) weight, 2) have fiducial RMS distance contribute to the cost function: this is difficult because we're mixing metrics and a good balance thru normalization may be hard to achieve. But 1) would be interesting, in the sense that it tells the registration to emphasize that region in sampling. In the most extreme case this could be done via the mask image, where the mask value determines the weight, i.e. 0= ignore, 1=count 100%. The user could then draw and edit the mask image manually.
• Procrustes/Manual Fiducial Registration: As a final destination/last resort for all tough cases it would be great to offer this. User selects fiducial pairs and Slicer calculates rigid/affine even BSpline registration. This is currently offered in the Register Images module in the form of Landmarks as initialization and None as Registration. Implementation Pending? Preferable for modular design to have a separate tab/module because it will be using another algorithm. Esp. in conjunction with Xfrom concatenation this could be a great asset, i.e. the user can nudge things little by little and get closer to an ideal case and also undo,i.e. delete, a Xform that made things worse, and in the end merge/flatten them all. See also 'Adjustment Layers' below.

Processing & Bug Notes

• to run slicer3.5 on fat node: on X11: ssh -Y george; cd /workspace/meier
• NRRD (.nhdr) files seem to load only when raw file is compressed (raw.gz). Should be fixed (i.e. specific error message suggesting zipping) or documented
• data loading method needs to cleanup: it leaves "X11" active objects behind: progress bars. Not a refresh issue. Objects close when X11 closes.
• MRML tree applies to all module and is important interactive element: have always visible and provide functionality (e.g. save) via context/right click menu
• save dialog not intuitive: allow to choose/change filename. The combination of auto-selecting what to save and the same filename can easily cause unwanted data deletion. Waiting on QT?