Difference between revisions of "2009 prostate segmentation challenge MICCAI"

From NAMIC Wiki
Jump to: navigation, search
m (Text replacement - "http://www.slicer.org/slicerWiki/index.php/" to "https://www.slicer.org/wiki/")
 
(40 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 
=Goal=
 
=Goal=
To discuss the state-of-art segmentation of Prostate MRI in the context of MRI-guided prostate therapy, through comparison of the segmentation methods using sample data.
+
To discuss the state-of-art for prostate segmentation of MR images in the context of MRI-guided prostate therapy, through comparison of the segmentation methods using sample data.
  
 
=Workshop=
 
=Workshop=
Line 18: Line 18:
 
*Theo van Walsum (Organizer of the umbrella MICCAI workshop "3D segmentation in Clinic").
 
*Theo van Walsum (Organizer of the umbrella MICCAI workshop "3D segmentation in Clinic").
  
=Rule=
+
=Rules=
#Download 3D Slicer for viewing and manipulating the dataset we provide below.
+
#Download 3D Slicer for viewing and manipulating the dataset we provide below. All files to be downloaded are available in the Download section below.
#(Optional) Download the tutorial and its associated images regarding prostate image processing from Slicer's website.
+
#(Optional) Download the prostate image processing tutorial and its associated images.
#Download the Prostate MRI and the manual segmentation (as training data) from the website of National Center for Image Guided Therapy (PI: Jolesz and Tempany).
+
#Download the MR images of the prostate and the manually segmented gland (as training data). The segmentations have been performed by expert radiologists.
#Submit segmentation results with the paper describing the method developed and applied. The segmentation result should be [http://teem.sourceforge.net/nrrd/index.html NRRD format ] and loadable to Slicer. The segmentation result should aligned to original MRI when loaded to Slicer.
+
#Perform the segmentation of the prostate gland with your algorithm. The segmentation result should be [http://teem.sourceforge.net/nrrd/index.html NRRD format ] and loadable to Slicer. The segmentation result should be aligned to the original MR image when loaded to Slicer. In addition the segmentation result should be a label map image maintaining the same pixel intensity value assigned to each segmentation image in the training dataset. A [https://www.slicer.org/wiki/Modules:RegistrationMetrics-Documentation-3.5 module] has been created in Slicer3 which can calculate the 95% Hausdorff distance and Dice Similarity Coefficient between your segmentation and the manual segmentations of the prostate provided.
#segmentation result should maintain the same grayscale value assigned to each anatomy in the training dataset. See the description of the training dataset,  
+
#Create an account at this NA-MIC wiki: [http://wiki.na-mic.org/Wiki/index.php/Special:RequestAccount Account Request]. Once your account has been accepted, upload the NRRD file with your segmentation results in the Contestants section of this page, along with the paper describing the method developed and applied.
#the organizer will score the segmentation result by comparing it with manual segmentation of the expert using the announced comparison scheme.
+
#The organizer will score the segmentation result by comparing it with the manual segmentation of the expert using the announced comparison scheme: 95% Hausdorff distance and Dice Similarity Coeeficient.
#Repeat step above at the workshop.
+
#Repeat step above at the workshop with the competition dataset available on the day.
  Those who submit paper and make their codes available at Insight Journal (http://www.insight-journal.org/)  will get extra points.  
+
  Those who submit paper and make their code available at Insight Journal (http://www.insight-journal.org/)  will get extra points.  
  Those who submit paper and make their codes available as Slicer module will get extra recognition at the contest.
+
  Those who submit paper and make their code available as a Slicer module will get extra recognition at the contest.
  
 
=Dates=
 
=Dates=
#Training data available: May, 2009
+
#Training data available: June 2009
 
+
#Submission of results of segmented training data: starts June 25th 2009 and continues until the paper submission deadline.
#Submission of results: starts June 15 29, 2009 and continues till the paper submission deadline.
 
 
 
 
#Submission of papers (with results) (no page limit, Springer/MICCAI format/PDF file): July 20, 2009
 
#Submission of papers (with results) (no page limit, Springer/MICCAI format/PDF file): July 20, 2009
 
 
#Workshop: September 2009
 
#Workshop: September 2009
  
=Download=
+
=Downloads=
 
#[http://www.slicer.org/pages/Special:SlicerDownloads 3D Slicer]
 
#[http://www.slicer.org/pages/Special:SlicerDownloads 3D Slicer]
 
#[http://wiki.na-mic.org/Wiki/index.php/IGT:ToolKit/Prostate-Planning Prostate MRI tutorial for 3D Slicer]
 
#[http://wiki.na-mic.org/Wiki/index.php/IGT:ToolKit/Prostate-Planning Prostate MRI tutorial for 3D Slicer]
#[http://prostatemrimagedatabase.com/index.html |Training MRI data with expert's segmentation]
+
#[http://wiki.na-mic.org/Wiki/index.php/Training_Data_Prostate_Segmentation_Challenge_MICCAI09 Training MRI data with expert's segmentation]
#Subject data (for paper submission)
+
#[http://prostatemrimagedatabase.com/Database/000132/00003/001/index.html Subject data (for paper submission)]
#Subject data (for contest on-site)
+
#[http://wiki.na-mic.org/Wiki/index.php/ContestData Contest Data on site]
 +
 
 +
=Contestants Results Submission=
 +
Use the following template to upload results:
 +
 
 +
#Albert Gubern-Merida and Robert Marti, University of Girona, [[User:Marly | Link to Page]]
 +
#Jason Dowling (AEHRC CSIRO, Australia), Jurgen Fripp (AEHRC CSIRO, Australia), Peter Greer (Newcastle Mater Hospital, Australia), Sebastien Ourselin (UCL, UK), Olivier Salvado (AEHRC CSIRO, Australia),  [[User:Jason.dowling | Link to Page]]
 +
 
 +
=Contestants On-site Results Submission=
  
=Evaluation scheme=
+
#Albert Gubern-Merida and Robert Marti, University of Girona, [http://wiki.na-mic.org/Wiki/index.php/MarlyResults Link to Page]
*95% Hausdorff distance (HD)  
+
#Jason Dowling (AEHRC CSIRO, Australia), Jurgen Fripp (AEHRC CSIRO, Australia), Peter Greer (Newcastle Mater Hospital, Australia), Sebastien Ourselin (UCL, UK), Olivier Salvado (AEHRC CSIRO, Australia),  [http://wiki.na-mic.org/Wiki/index.php/DowlingResults Link to Page]
  
 +
=Results=
 +
[[Image:MICCAI2009SegmentationChallenge.ppt]]
  
The HD between two point datasets formed by the edges of the segmented prostate sub-anatomy from the MRI and manual segmentation. We will measured the accuracy of alignment between these two segmentations, by extracting the edges from the subsections. The HD is the maximum distance of a set to the nearest point in the other set. More formal description of the HD can be found at:
+
=Evaluation Scheme=
 +
Two evaluation methods are used to determine the accuracy of the segmentation: the 95% Hausdorff distance and the Dice Similarity Coefficient.
 +
 
 +
*95% Hausdorff distance (HD)
 +
The HD is calculated between the contours of the segmented prostates which result from the manual segmentation and the proposed contestant algorithm. The HD is the maximum distance of a set to the nearest point in another set. A more formal description of the HD can be found in the following publication:
  
 
**Archip N, Clatz O, Whalen S, et al. Non-rigid alignment of pre-operative MRI, fMRI, and DT-MRI with intra-operative MRI for enhanced visualization and navigation in image-guided neurosurgery. Neuroimage 2007; 35:609-624.
 
**Archip N, Clatz O, Whalen S, et al. Non-rigid alignment of pre-operative MRI, fMRI, and DT-MRI with intra-operative MRI for enhanced visualization and navigation in image-guided neurosurgery. Neuroimage 2007; 35:609-624.
  
 
*Dice Similarity Coefficient (DSC)
 
*Dice Similarity Coefficient (DSC)
 
+
The DSC, which has been extensively used in the evaluation of segmentation, gives a measure of the volumetric overlap between the two segmented prostates. The DSC indicates twice the number of voxels which are shared by or are common to both segmentations divided by the total number of voxels in both datasets. The DSC can range from zero to one, where zero is no alignment between segmented glands and one is perfect alignment. An example of the DSC used in prostate segmetation can be found at:
The DSC, which has been used in evaluation of segmentation, gives a measure of the volumetric overlap between the segmented livers from the registered images. The DSC indicates twice the number of voxels which are shared by or are common to both registered datasets. The denominator is the total number of voxels in both datasets. The DSC can range from zero to one, where zero is no alignment between registered livers and one is perfect alignment. An example of DSC used in prostate segmetation can be found at
 
  
 
**A. Bharatha, M. Hirose, N. Hata, S. K. Warfield, M. Ferrant, K. H. Zou, E. Suarez-Santana, J. Ruiz-Alzola, A. D'Amico, R. A. Cormack, R. Kikinis, F. A. Jolesz, and C. M. Tempany, "Evaluation of three-dimensional finite element-based deformable registration of pre- and intraoperative prostate imaging," Med Phys, vol. 28(12), pp. 2551-60, 2001.
 
**A. Bharatha, M. Hirose, N. Hata, S. K. Warfield, M. Ferrant, K. H. Zou, E. Suarez-Santana, J. Ruiz-Alzola, A. D'Amico, R. A. Cormack, R. Kikinis, F. A. Jolesz, and C. M. Tempany, "Evaluation of three-dimensional finite element-based deformable registration of pre- and intraoperative prostate imaging," Med Phys, vol. 28(12), pp. 2551-60, 2001.

Latest revision as of 17:15, 10 July 2017

Home < 2009 prostate segmentation challenge MICCAI

Goal

To discuss the state-of-art for prostate segmentation of MR images in the context of MRI-guided prostate therapy, through comparison of the segmentation methods using sample data.

Workshop

Thursday - 24th September at MICCAI 2009 (London, UK) http://www.miccai2009.org/

  • National Center for Image Guided Therapy (NCIGT, PI: Jolesz, Tempany)
  • National Alliance for Medical Image Computing (NA-MIC, PI: Kikinis)

Organizer

  • Nobuhiko Hata (co-lead organizer, BWH, Boston, MA, Email hata at-mark bwh.harvard.edu)
  • Gabor Fichtinger (co-lead organizer, Queens Univ, Kingston, ON)
  • Sota Oguro (Data managing, BWH, Boston, MA)
  • Haytham Elhawary (Slicer module for validation, BWH, Boston, MA)
  • Theo van Walsum (Organizer of the umbrella MICCAI workshop "3D segmentation in Clinic").

Rules

  1. Download 3D Slicer for viewing and manipulating the dataset we provide below. All files to be downloaded are available in the Download section below.
  2. (Optional) Download the prostate image processing tutorial and its associated images.
  3. Download the MR images of the prostate and the manually segmented gland (as training data). The segmentations have been performed by expert radiologists.
  4. Perform the segmentation of the prostate gland with your algorithm. The segmentation result should be NRRD format and loadable to Slicer. The segmentation result should be aligned to the original MR image when loaded to Slicer. In addition the segmentation result should be a label map image maintaining the same pixel intensity value assigned to each segmentation image in the training dataset. A module has been created in Slicer3 which can calculate the 95% Hausdorff distance and Dice Similarity Coefficient between your segmentation and the manual segmentations of the prostate provided.
  5. Create an account at this NA-MIC wiki: Account Request. Once your account has been accepted, upload the NRRD file with your segmentation results in the Contestants section of this page, along with the paper describing the method developed and applied.
  6. The organizer will score the segmentation result by comparing it with the manual segmentation of the expert using the announced comparison scheme: 95% Hausdorff distance and Dice Similarity Coeeficient.
  7. Repeat step above at the workshop with the competition dataset available on the day.
Those who submit paper and make their code available at Insight Journal (http://www.insight-journal.org/)  will get extra points. 
Those who submit paper and make their code available as a Slicer module will get extra recognition at the contest.

Dates

  1. Training data available: June 2009
  2. Submission of results of segmented training data: starts June 25th 2009 and continues until the paper submission deadline.
  3. Submission of papers (with results) (no page limit, Springer/MICCAI format/PDF file): July 20, 2009
  4. Workshop: September 2009

Downloads

  1. 3D Slicer
  2. Prostate MRI tutorial for 3D Slicer
  3. Training MRI data with expert's segmentation
  4. Subject data (for paper submission)
  5. Contest Data on site

Contestants Results Submission

Use the following template to upload results:

  1. Albert Gubern-Merida and Robert Marti, University of Girona, Link to Page
  2. Jason Dowling (AEHRC CSIRO, Australia), Jurgen Fripp (AEHRC CSIRO, Australia), Peter Greer (Newcastle Mater Hospital, Australia), Sebastien Ourselin (UCL, UK), Olivier Salvado (AEHRC CSIRO, Australia), Link to Page

Contestants On-site Results Submission

  1. Albert Gubern-Merida and Robert Marti, University of Girona, Link to Page
  2. Jason Dowling (AEHRC CSIRO, Australia), Jurgen Fripp (AEHRC CSIRO, Australia), Peter Greer (Newcastle Mater Hospital, Australia), Sebastien Ourselin (UCL, UK), Olivier Salvado (AEHRC CSIRO, Australia), Link to Page

Results

File:MICCAI2009SegmentationChallenge.ppt

Evaluation Scheme

Two evaluation methods are used to determine the accuracy of the segmentation: the 95% Hausdorff distance and the Dice Similarity Coefficient.

  • 95% Hausdorff distance (HD)

The HD is calculated between the contours of the segmented prostates which result from the manual segmentation and the proposed contestant algorithm. The HD is the maximum distance of a set to the nearest point in another set. A more formal description of the HD can be found in the following publication:

    • Archip N, Clatz O, Whalen S, et al. Non-rigid alignment of pre-operative MRI, fMRI, and DT-MRI with intra-operative MRI for enhanced visualization and navigation in image-guided neurosurgery. Neuroimage 2007; 35:609-624.
  • Dice Similarity Coefficient (DSC)

The DSC, which has been extensively used in the evaluation of segmentation, gives a measure of the volumetric overlap between the two segmented prostates. The DSC indicates twice the number of voxels which are shared by or are common to both segmentations divided by the total number of voxels in both datasets. The DSC can range from zero to one, where zero is no alignment between segmented glands and one is perfect alignment. An example of the DSC used in prostate segmetation can be found at:

    • A. Bharatha, M. Hirose, N. Hata, S. K. Warfield, M. Ferrant, K. H. Zou, E. Suarez-Santana, J. Ruiz-Alzola, A. D'Amico, R. A. Cormack, R. Kikinis, F. A. Jolesz, and C. M. Tempany, "Evaluation of three-dimensional finite element-based deformable registration of pre- and intraoperative prostate imaging," Med Phys, vol. 28(12), pp. 2551-60, 2001.