Difference between revisions of "Projects:LocalStatisticalAnalysisViaPermutationTests"

From NAMIC Wiki
Jump to: navigation, search
 
(29 intermediate revisions by 4 users not shown)
Line 1: Line 1:
  Back to [[NA-MIC_Collaborations|NA-MIC_Collaborations]]
+
  Back to [[NA-MIC_Internal_Collaborations:StructuralImageAnalysis|NA-MIC Collaborations]], [[Algorithm:UNC|UNC Algorithms]], [[Algorithm:Utah|Utah Algorithms]]
 
__NOTOC__
 
__NOTOC__
 
= Local Statistical Analysis via Permutation Tests =
 
= Local Statistical Analysis via Permutation Tests =
Line 5: Line 5:
 
[[Image:UNCShape_CaudatePval_MICCAI06.png|thumb|right|300px|]]
 
[[Image:UNCShape_CaudatePval_MICCAI06.png|thumb|right|300px|]]
  
In our statistical framework, group differences between groups of surfaces are computed locally for every surface point using the standard robust Hotelling T^2 two sample metric. Statistical p-values, both raw and corrected for multiple comparisons, result in significance maps. Additional visualization of the group tests are provided via mean difference magnitude and vector maps, as well as maps of the group covariance information.
+
As part of NA-MIC, we have developed two statistical frameworks for computing shape analysis of 3D surfaces, both based on permutation tests. The first framework is useful for straightforward group difference testing and computes differences locally for every surface point via the standard robust Hotelling T^2 two sample metric. The second one compute locally first a Generalized Linear Model (GLM) followed by a MANCOVA based metric computation for group difference testing. The second framework also allows the computation of correlation statistics with the GLM fitted shape coordinates. In both cases, out tools will proveide statistical p-value maps, both raw and corrected for multiple comparisons, as well as mean difference magnitude and vector maps, group covariance maps, correlation maps and z-score maps.
  
 
= Description =
 
= Description =
  
The local '''shape''' '''analysis''' involves testing from a few to many thousands of hypothesis (one per surface ele- ment) for statistically significant effects. It is thus important to control for the multiple testing problem, and the most common measure of multiple false positives is the familywise error rate (FWER). The multiple testing problem has been an active area of research in the functional neuroimaging community. One of the widely used methods in the '''analysis''' of neuroimaging data makes inferences based on the maximum distribution. In our work, we are employing non-parametric permutation tests to estimate uniformly sensitive extrema distributions. The correction method is based on computing first the local p-values using permutation tests. The minimum of these p-values across the surface is then computed for every permutation. The appropriate corrected p-value at level α can then be obtained by the computing the value at the α-quantile in the histogram of these minimum values. Using the minimum statistic of the p-values, this method correctly controls for the FWER, or the false positives, but no control of the false negatives is provided. The resulting corrected local significance values can thus be regarded as pessimistic estimates akin to a simple Bonferroni correctection.
+
== Shape Statistics: Permutation tests ==
 +
 
 +
The local '''shape''' '''analysis''' involves testing from a few to many thousands of hypothesis (one per surface point) for statistically significant effects. At each surface point, we propose to perform a separate statistical test that analyzes either the local coordinate (in a general group difference shape study) or the change difference vector (in a longitudinal study or twin study). Here we present a set of methods for computing a) the local shape metric as well as b) the global correction for doing multiple (local) tests. For the shape metric, two separate methods were developed for different use scenarios depending on whether correction for / correlation with patient variables such as gender, age or clinical scores is needed.
 +
 
 +
The most common measure of multiple false positives is the familywise error rate (FWER). The multiple testing problem has been an active area of research in the functional neuroimaging community. One of the widely used methods in the '''analysis''' of neuroimaging data makes inferences based on the maximum distribution. Common to both testing framework is the use of non-parametric permutation tests to estimate uniformly sensitive extrema distributions. The correction method is based on computing first the local p-values using permutation tests. The minimum of these p-values across the surface is then computed for every permutation. The appropriate corrected p-value at level α can then be obtained by the computing the value at the α-quantile in the histogram of these minimum values. Using the minimum statistic of the p-values, this method correctly controls for the FWER, or the false positives, but no control of the false negatives is provided. The resulting corrected local significance values can thus be regarded as pessimistic estimates akin to a simple Bonferroni correctection.
  
 
Additionally to the non-parametric permutation correction, we have also implemented and applied a False Discovery Rate Estimation (FDR) method. The innovation of this procedure is that it controls the expected proportion of false positives only among those tests for which a local significance has been detected. The FDR method thus allows an expected proportion (usually 5%) of the FDR corrected significance values to be falsely positive. The correction using FDR provides an interpretable and adaptive criterion with higher power than the non-parametric permutation tests. FDR thus results in a less conservative estimate of the false-negatives.
 
Additionally to the non-parametric permutation correction, we have also implemented and applied a False Discovery Rate Estimation (FDR) method. The innovation of this procedure is that it controls the expected proportion of false positives only among those tests for which a local significance has been detected. The FDR method thus allows an expected proportion (usually 5%) of the FDR corrected significance values to be falsely positive. The correction using FDR provides an interpretable and adaptive criterion with higher power than the non-parametric permutation tests. FDR thus results in a less conservative estimate of the false-negatives.
  
Current development is extending the framework to include subject variables, such as gender, age and clinical scores into the current pure group difference computation.
+
== Hotelling T^2 metric ==
 +
 
 +
For the case group mean difference testing of balanced groups that are well controlled, our original testing framework is of best use. Here we compute locally the standard robust Hotelling T^2 two sample metric, then estimate p-values via standard permutation tests and correct for multiple comparison via FDR and permutation tests of the p-value extrema distributions.
 +
 
 +
== GLM & MANCOVA ==
 +
 
 +
 
 +
[[Image:MANCOVA_intercation_test_mrml.jpg|thumb|right|400px|shapeAnalysisMANCOVA: quality control for an interaction test]]
 +
 
 +
In case that correlation statistics are needed, or for the case of unbalanced or not well controlled groups, the novel GLM & MANCOVA statistical framework should be employed. This method first computes a Generalized Linear Model (GLM), followed by a MANCOVA analysis and p-value computation via the same permutation approach mentioned above. Correction for multiple-comparisons is performed via False Discovery Rate.
 +
The main MANCOVA metric is the Hotelling Trace, but other standard MANCOVA metrics can be used (such as Ray's Max Lambda). In addition to the permutation based p-values, F-test based significance maps are generated. The MANCOVA based testing framework also contains facilities for z-score testing and correlation testing.
 +
 
 +
The MANCOVA based testing has been implemented in a Slicer compatible module, as well as a QT4 based Wizard ([http://www.nitrc.org/projects/shape_mancova both available on NITRC]). The Wizard allows the user to visualize the group object describing all subjects, their surface data as well as any subject variables (such as gender, age, diagnosis, scores etc), intuitively select variables for testing and covarying and finally run the statistical analysis.
  
 
= Key Investigators =
 
= Key Investigators =
  
* UNC: Martin Styner, Ipek Oguz, Christine Xu
+
* UNC Algorithms: Martin Styner, Ipek Oguz, Marc Niethammer, Marc Macenko, Beatriz Paniagua, Christine Shun Xu, Hongtu Zhu, Corentin Hamel, Lucile Bompard
* Utah: Guido Gerig
+
* Utah Algorithms: Guido Gerig
  
 
= Publications =
 
= Publications =
 
+
* [http://www.na-mic.org/publications/pages/display?search=LocalStatisticalAnalysisViaPermutationTests NA-MIC Publications Database on Local Statistical Analysis via Permutation Tests]
''In Print''
 
* [http://www.na-mic.org/Special:Publications?text=Projects:LocalStatisticalAnalysisViaPermutationTests&submit=Search&words=all&title=checked&keywords=checked&authors=checked&abstract=checked&sponsors=checked&searchbytag=checked| NA-MIC Publications Database]
 
* S. Olmos, M. Bossa, M. Styner: PCA-based filtering for hypothesis testing on hippocampus shape differences between schizophrenics and normals. Medical Image Understanding and Analysis 2005, Bristol, pp. 167-170.
 
* M Styner, JA Lieberman, RK McClure, DR Weingberger, DW Jones, G Gerig: Morphometric analysis of lateral ventricles in schizophrenia and healthy controls regarding genetic and disease-specific factors, Proceedings of the National Academy of Science, 2005, Vol 102, No. 12, March 29, p 4872-4877.
 
* D. Pantazis, R.M. Leahy, T.E.Nichols, M. Styner: Statistical Surface Based Morphometry Using a Non-Parametric Approach, IEEE Symposium on Biomedical Imaging ISBI 2004,1283-1286.
 
* S.Vetsa, M. Styner, S. Pizer, J. Lieberman, G. Gerig: Caudate shape discrimination in schizophrenia using template-free non-parametric tests, Medical Image Computing and Computer-Assisted Intervention MICCAI 2003, pp.661-669
 
* M. Styner, J. Lieberman, G. Gerig: Boundary and Medial Shape Analysis of the Hippocampus in Schizophrenia, Medical Image Computing and Computer-Assisted Intervention MICCAI 2003, II, pp. 464-471.
 
  
 
= Links =
 
= Links =
 
+
* [[Projects:ShapeAnalysisFrameworkUsingSPHARMPDM | Main SPHARM shape analysis framework ]]
 
 
 
* [[NA-MIC/Projects/Structural/Shape_Analysis/ShapeStatisticsWithPermTestCorrectionAndFDR|ITK statistical analysis using non-parametric permutation analysis and false discovery rate]]
 
* [[NA-MIC/Projects/Structural/Shape_Analysis/ShapeStatisticsWithPermTestCorrectionAndFDR|ITK statistical analysis using non-parametric permutation analysis and false discovery rate]]
 +
* [http://www.nitrc.org/projects/spharm-pdm  NITRC page for SPHARM PDM toolbox]
 +
* [http://www.nitrc.org/projects/shape_mancova  NITRC page for Shape analysis via GLM & MANCOVA]
  
 
  Project Week Results: [[media:2006_06_PW_StatAnal.ppt|Jun 2006]]
 
  Project Week Results: [[media:2006_06_PW_StatAnal.ppt|Jun 2006]]
 +
 +
[[Category: Statistics]] [[Category:Shape Analysis]]

Latest revision as of 20:33, 1 April 2011

Home < Projects:LocalStatisticalAnalysisViaPermutationTests
Back to NA-MIC Collaborations, UNC Algorithms, Utah Algorithms

Local Statistical Analysis via Permutation Tests

UNCShape CaudatePval MICCAI06.png

As part of NA-MIC, we have developed two statistical frameworks for computing shape analysis of 3D surfaces, both based on permutation tests. The first framework is useful for straightforward group difference testing and computes differences locally for every surface point via the standard robust Hotelling T^2 two sample metric. The second one compute locally first a Generalized Linear Model (GLM) followed by a MANCOVA based metric computation for group difference testing. The second framework also allows the computation of correlation statistics with the GLM fitted shape coordinates. In both cases, out tools will proveide statistical p-value maps, both raw and corrected for multiple comparisons, as well as mean difference magnitude and vector maps, group covariance maps, correlation maps and z-score maps.

Description

Shape Statistics: Permutation tests

The local shape analysis involves testing from a few to many thousands of hypothesis (one per surface point) for statistically significant effects. At each surface point, we propose to perform a separate statistical test that analyzes either the local coordinate (in a general group difference shape study) or the change difference vector (in a longitudinal study or twin study). Here we present a set of methods for computing a) the local shape metric as well as b) the global correction for doing multiple (local) tests. For the shape metric, two separate methods were developed for different use scenarios depending on whether correction for / correlation with patient variables such as gender, age or clinical scores is needed.

The most common measure of multiple false positives is the familywise error rate (FWER). The multiple testing problem has been an active area of research in the functional neuroimaging community. One of the widely used methods in the analysis of neuroimaging data makes inferences based on the maximum distribution. Common to both testing framework is the use of non-parametric permutation tests to estimate uniformly sensitive extrema distributions. The correction method is based on computing first the local p-values using permutation tests. The minimum of these p-values across the surface is then computed for every permutation. The appropriate corrected p-value at level α can then be obtained by the computing the value at the α-quantile in the histogram of these minimum values. Using the minimum statistic of the p-values, this method correctly controls for the FWER, or the false positives, but no control of the false negatives is provided. The resulting corrected local significance values can thus be regarded as pessimistic estimates akin to a simple Bonferroni correctection.

Additionally to the non-parametric permutation correction, we have also implemented and applied a False Discovery Rate Estimation (FDR) method. The innovation of this procedure is that it controls the expected proportion of false positives only among those tests for which a local significance has been detected. The FDR method thus allows an expected proportion (usually 5%) of the FDR corrected significance values to be falsely positive. The correction using FDR provides an interpretable and adaptive criterion with higher power than the non-parametric permutation tests. FDR thus results in a less conservative estimate of the false-negatives.

Hotelling T^2 metric

For the case group mean difference testing of balanced groups that are well controlled, our original testing framework is of best use. Here we compute locally the standard robust Hotelling T^2 two sample metric, then estimate p-values via standard permutation tests and correct for multiple comparison via FDR and permutation tests of the p-value extrema distributions.

GLM & MANCOVA

shapeAnalysisMANCOVA: quality control for an interaction test

In case that correlation statistics are needed, or for the case of unbalanced or not well controlled groups, the novel GLM & MANCOVA statistical framework should be employed. This method first computes a Generalized Linear Model (GLM), followed by a MANCOVA analysis and p-value computation via the same permutation approach mentioned above. Correction for multiple-comparisons is performed via False Discovery Rate. The main MANCOVA metric is the Hotelling Trace, but other standard MANCOVA metrics can be used (such as Ray's Max Lambda). In addition to the permutation based p-values, F-test based significance maps are generated. The MANCOVA based testing framework also contains facilities for z-score testing and correlation testing.

The MANCOVA based testing has been implemented in a Slicer compatible module, as well as a QT4 based Wizard (both available on NITRC). The Wizard allows the user to visualize the group object describing all subjects, their surface data as well as any subject variables (such as gender, age, diagnosis, scores etc), intuitively select variables for testing and covarying and finally run the statistical analysis.

Key Investigators

  • UNC Algorithms: Martin Styner, Ipek Oguz, Marc Niethammer, Marc Macenko, Beatriz Paniagua, Christine Shun Xu, Hongtu Zhu, Corentin Hamel, Lucile Bompard
  • Utah Algorithms: Guido Gerig

Publications

Links

Project Week Results: Jun 2006