Summer2009:Registration reproducibility in Slicer
Transform for test data/GUI registration
Transform for test data/CLI registration
Key Investigators
- BWH: Andriy Fedorov, Steve Pieper, Tina Kapur
- GE: Jim Miller
- Kitware: Luis Ibanez
- EAB: Bill Lorensen
Objective
Rigid registration in general, and RigidRegistration module of Slicer3 in particular, are important workflow components for a number of applications. Two issues are of our interest:
- We found gross inconsistency between the result of using rigid registration module on volumetric brain MRI in GUI vs command line invocation.
- There is also significant difference between command line invocations of the module on different platforms.
The issue has been confirmed independently by Andriy, Bill, Jim and Steve. It was also confirmed by Kilian. The problem is thoroughly documented in Slicer3 bug 416.
We would like to understand the source of this inconsistency, together with the understanding what is reasonable to expect expect in terms of reproducibility from such complex numerical codes like rigid registration.
Approach, Plan
- use clinical data to demonstrate the problem, once again
- collect feedback from registration experts
- create a test that demonstrates the problem, and add it to the ITK dashboard
- Questions to consider:
- is reproducibility a function of the architecture? is it dependent on multi-threading? can it be quantified and estimated?
- what is the correct way to test reproducibility for this kind of applications?
Progress
- added testing mode to RigidRegistration to measure the maximum difference
- abused Slicer3 dashboard to collect the magnitude of difference on various platforms
Test setup
1. Get images and transform from Slicer3 trunk:
2. To get the transform in GUI, go to RigidRegistration module, use the default parameters to initialize (see figure), EXCEPT reduce iteration cout from the default to "100,100,50,20" 3. To get the transform with CLI, here's the command line you need to run: Slicer3 --launch RigidRegistration --fixedsmoothingfactor 0 --movingsmoothingfactor 0 --histogrambins 30 --spatialsamples 10000 --iterations 100,100,50,20 --learningrate 0.01,0.005,0.0005,0.0002 --translationscale 100 --outputtransform ./RigidRegistrationTransform-CLI.txt MRMeningioma0.nrrd MRMeningioma1.nrrd 4. To get the actual difference in the corner of your image using one transform vs another, you can run RigidRegistration in the testing mode. Testing feature is not available in stable release -- you need to download nightly, or use your build from scratch!. In testing mode, initial transform is NOT used as the initial tranform, but is simply a "ground truth" transform to test reproducibility. Slicer3 --launch RigidRegistration --testingmode --initialtransform RigidRegistrationTransform-GUI.tfm --fixedsmoothingfactor 0 --movingsmoothingfactor 0 --histogrambins 30 --spatialsamples 10000 --iterations 100,100,50,20 --learningrate 0.01,0.005,0.0005,0.0002 --translationscale 100 --outputtransform ./RigidRegistrationTransform-CLI.txt MRMeningioma0.nrrd MRMeningioma1.nrrd |
Participate!
Here are the results of running RigidRegistration module with the parameters set up as above, on the provided data. Please feel free to run the test on your platform, as described in Step 4 above, and contribute the transform, details of your system, and the error value here:
- Baseline: george.bwh.harvard.edu (linux)
MRMeningioma01.tfm: #Insight Transform File V1.0 # Transform 0 Transform: QuaternionRigidTransform_double_3_3 Parameters: 0.00993255 0.00724952 0.00477429 0.999913 7.40702 4.83973 -0.0941685 FixedParameters: 0.00025 0.00025 -1.42109e-14 --- Magnitude of error vector: 4.7826e-06
- cluster.bwh.harvard.edu (linux)
#Insight Transform File V1.0 # Transform 0 Transform: QuaternionRigidTransform_double_3_3 Parameters: 0.00687841 0.00580804 0.0113849 0.999895 7.65038 5.16471 0.0120382 FixedParameters: 0.00025 0.00025 -1.42109e-14 --- Magnitude of error vector: 2.72849
