Back to [[NA-MIC_External_Collaborations|NA-MIC External Collaborations]]
__NOTOC__
==Grant#==
U54EB005149-05S2
==Key Personnel==
*Stanford: Scott Delp, PI, Harris Doddi, Saikat Pal
*NA-MIC: Ron Kikinis, Steve Pieper
*Kitware: Luis Ibanez
==Grant Duration==
09/19/2008-07/31/2010

<gallery widths="300px">
Image:58.jpg
Image:All_Three.JPG
Image:Femur_Patella_Tibia.jpg
</gallery>

==Aim==
To develop a generic semi-automatic segmentation toolkit to convert images of musculoskeletal structures to 3D models.

==Process Flowchart==
[[Image:FlowChart.jpg|left| Process Diagram.]]

==Progress==
==Atlas Generation from Input MR Images==
===Model Generation from Input MR Images===
Pre-Segmented models for femur, patella and tibia are obtained for a patient (in .stl format).
[[Image:All_Three.JPG|left|thumb| Pre-Segmented Femur/Patella/Tibia Model]]

===Create a filled label map using PolyDataToFilledLabelMap module in slicer===
The models generated in the above step are closed but hollow. But EM Segmentation requires the atlas given to be in closed filled format. Hence we converted the hollow closed models into filled label volumes using the module PolyDataToFilledLabelMap module in Slicer. The output label map of this module is of datatype unsigned char and is converted to short datatype.
<gallery widths="400px" perrow="6">
Image:LabelMap_Femur.jpg
Image:LabelMap_Tibia.jpg
Image:LabelMap_Patella.jpg
</gallery>

===EM Segmentation based on the atlas===
The output label map in the above is given as input atlas in the EM Segmentation step. As an initial step we ran EM Segmentation on the same patient. The EM Segmented output is given in the below figure.
[[Image:Femur_Patella_Tibia.jpg|left|thumb| EM Segmented Output]]
==Register Images==
===Register Images Module in Slicer===
We are now in the process of trying to register new patient image to the existing patient image. If we register the images, we can get the transform which can be used to register the existing atlas (label map) to the new patient.

We think that Pipelined Bspline Registration may give promising results for our dataset. Below are some of the results. Here we consider 64_MRI as the fixed image and 58_MRI as the moving image and used '''Pipelined BSpline Image registration method''' for registration. Also we gave one point as landmark point.
<gallery widths="400px" perrow="6">
Image:58_MRI.jpg
Image:64_MRI.jpg
Image:64_Registered.jpg
</gallery>

We also tried '''affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''one landmark point'''. The zipped file which contains scene along with screenshots is
[[Image:64 58 Affine Registration.zip |left|64_Affine_Registered_Scene]]
<gallery widths="400px" perrow="6">
Image:64 Affine Registration.JPG
Image:64 Registerd SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and one landmark point. The zipped file which contains scene along with screenshots is
[[Image:64 58 BSpline Registration.zip |left|64_BSpline_Registered_Scene]]. Here we used '''4 points''' as landmarks.
<gallery widths="400px" perrow="6">
Image:64 BSpline Registration.JPG
Image:64 BSpline Registered SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and four landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. Here we used '''4 points''' as landmarks. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 13, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 14, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''10''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 Affine 50 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 100 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 150 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 250 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 350 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 450 Iterations 10 Landmark Points.jpg
</gallery>

==='''Update May 20, 2009'''===
We tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 BSpline 10 Iterations.JPG
Image:64 58 BSpline 110 Iterations.JPG
Image:64 58 BSpline 160 Iterations.JPG
Image:64 58 BSpline 210 Iterations.JPG
Image:64 58 BSpline 260 Iterations.JPG
Image:64 58 BSpline 360 Iterations.JPG
</gallery>

==='''Update May 21, 2009'''===
We tried '''Pipelined Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 PipeLinedAffine Registration 50 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 100 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 150 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 200 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 300 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 400 Iterations.JPG
</gallery>

==='''Update May 25, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 5000.
<gallery widths="200px" perrow="6">
Image:AffineRegistration InitialTransform 200Iterations.JPG
Image:AffineRegistration InitialTransform 300Iterations.JPG
Image:AffineRegistration InitialTransform 400Iterations.JPG
</gallery>

===Multi Image Registration===
We have also tried the approach of Non-rigid Groupwise Registration using Bspline Deformation Model by Serdar K. Balci, Polina Golland and William M. Wells. In this approach, we try to give one slice each of two different patients we need to register as input. Here are some of the results of their approach.

<gallery widths="200px" perrow="2">
Slice 1:
Image:1_Slicez0.JPG‎
Slice 2:
Image:1_Slicez1.JPG
Mean Slice of above two :
Image:1_MeanSliceZ.JPG
</gallery>

<gallery widths="200px" perrow="2">
Slice 1:
Image:2_Z0.JPG
Slice 2:
Image:2_Z1.JPG
Mean Slice of above two :
Image:2_MeanSliceZ.JPG
</gallery>
==Image Segmentation Methods (Oct-Dec) 2009==
==='''Shape Dectection Level Set Approach'''===
We started in October with Shape Dectection Level Set algorithm. In this algorithm, the governing differential equation has an additional curvature-based term. This term acts as a smoothing term where areas of high curvature, assumed to be due to noise,
are smoothed out. Scaling parameters are used to control the tradeoff between the expansion term and the smoothing term. One consequence of this additional curvature term is that the fast marching algorithm is no longer applicable, because the contour is no longer guaranteed to always be expanding. Instead, the level set function is updated iteratively. The ShapeDetectionLevelSetImageFilter expects two inputs, the first being an initial Level Set in the form of an itk::Image, and the second being a feature image. For this algorithm, the feature image is an edge potential image that basically follows the same rules applicable to the speed image used for the FastMarchingImageFilter. The pipeline of the algorithm is as follows

<gallery widths="200px" perrow="2">
Image:Algo.JPG
</gallery>

The output of this algorithm is as follows.
<gallery widths="200px" perrow="3">
Image:Femur_Tibia_Segmented.JPG‎
Image:Tibia_Boundary.JPG
Image:Femurr Boundary.JPG
</gallery>

The models extracted from the segmented label maps are as follows.
<gallery widths="200px" perrow="2">
Image:ShapeDetectionModel.JPG
Image:ShapeDetectionModelSmoothing.JPG
</gallery>

==='''Multi-Contrast MR Approach'''===
In this algorithm, we try to collect MR images of Knee using different MR scans like IDEAL-SPGR-FAT, IDEAL-SPGR-WATER, IDEAL-GRE-FAT etc. The main assumption in this algorithm is that different structures of interest ( eg: bones/cartilage) have different intensity values in different scans. For eg: Bones are colored with lower pixel intensity values in IDEAL-SPGR-WATER dataset whereas they are colored with higher pixel intensity values in IDEAL-SPGR-WATER dataset.

Firstly, we collect the seed points of the regions of interest using an insight application "ImageViewer". So each file with the seed points is considered as one cluster. For eg: we can have clusters for background, bones, cartilage etc. Next, we calculate the cluster centers (µ) and standard deviation (ρ) for each seed file. We try to define user defined parameter called numberOfSd(α) which represents the extent of how far a cluster can go. The numberofSd(α) has been split to be bi-directional in the x-axis representing one contrast image and y-axis representing another contrast image.

Below is the dataset of Knee MR image we used
<gallery widths="200px" perrow="2">
Image:IdealSPGRWater.JPG | IDEAL-SPGR-WATER
Image:IdealSPGRFat.JPG | IDEAL-SPGR-FAT
</gallery>

Scatter Plot of the above two images
<gallery widths="200px" perrow="2">
Image:ScatterPlot.JPG | Scatter Plot
</gallery>

Segmentation Output we obtained :
<gallery widths="200px" perrow="1">
Image:SegmentationResult.JPG | White: Bones | Green: Cartilage | Red: Background
</gallery>

We have also proved that our algorithm works fairly well for hip dataset. Below is the segmented output of Hip dataset.
<gallery widths="200px" perrow="2">
Image:Hip.JPG
</gallery>

The segmented output we got using our approach is
<gallery widths="200px" perrow="2">
Image:HipOutput.JPG
</gallery>
==Datasets==
1. IDEAL-SPGR-FAT dataset [[File:idealSPGRFat.zip]]

2. IDEAL-SPGR-WATER dataset [[File:idealSPGRWater.zip]]

==Future Work==
Build Models from the segmented label maps that we get from Multi MR contrast images.

Extend our existing Multi Contrast MR solution to other body datasets.

Improve segmentation results from our current dataset by using "more" multi contrast images.

==References==
1. ITK Software Guide ( http://www.itk.org)

2. Slicer Software (http://www.slicer.org)

3. Multi-Contrast MR for Enhanced Bone Imaging and Segmentation Ruphin Dalvi, Rafeef Abugharbieh, Derek C. Wilson, David R. Wilson.

4. HE Cline. WE Lorensen, R Kikinis, F Jolesz 3D Segmentation of the Head Using Probability and Connectivity Journal of Computer Assisted Tomography 14:1037-1045 (1990)

Stanford Simbios group

2009-12-18T22:17:00Z

Harishd: /* Future Work */

Back to [[NA-MIC_External_Collaborations|NA-MIC External Collaborations]]
__NOTOC__
==Grant#==
U54EB005149-05S2
==Key Personnel==
*Stanford: Scott Delp, PI, Harris Doddi, Saikat Pal
*NA-MIC: Ron Kikinis, Steve Pieper
*Kitware: Luis Ibanez
==Grant Duration==
09/19/2008-07/31/2010

<gallery widths="300px">
Image:58.jpg
Image:All_Three.JPG
Image:Femur_Patella_Tibia.jpg
</gallery>

==Aim==
To develop a generic semi-automatic segmentation toolkit to convert images of musculoskeletal structures to 3D models.

==Process Flowchart==
[[Image:FlowChart.jpg|left| Process Diagram.]]

==Progress==
==Atlas Generation from Input MR Images==
===Model Generation from Input MR Images===
Pre-Segmented models for femur, patella and tibia are obtained for a patient (in .stl format).
[[Image:All_Three.JPG|left|thumb| Pre-Segmented Femur/Patella/Tibia Model]]

===Create a filled label map using PolyDataToFilledLabelMap module in slicer===
The models generated in the above step are closed but hollow. But EM Segmentation requires the atlas given to be in closed filled format. Hence we converted the hollow closed models into filled label volumes using the module PolyDataToFilledLabelMap module in Slicer. The output label map of this module is of datatype unsigned char and is converted to short datatype.
<gallery widths="400px" perrow="6">
Image:LabelMap_Femur.jpg
Image:LabelMap_Tibia.jpg
Image:LabelMap_Patella.jpg
</gallery>

===EM Segmentation based on the atlas===
The output label map in the above is given as input atlas in the EM Segmentation step. As an initial step we ran EM Segmentation on the same patient. The EM Segmented output is given in the below figure.
[[Image:Femur_Patella_Tibia.jpg|left|thumb| EM Segmented Output]]
==Register Images==
===Register Images Module in Slicer===
We are now in the process of trying to register new patient image to the existing patient image. If we register the images, we can get the transform which can be used to register the existing atlas (label map) to the new patient.

We think that Pipelined Bspline Registration may give promising results for our dataset. Below are some of the results. Here we consider 64_MRI as the fixed image and 58_MRI as the moving image and used '''Pipelined BSpline Image registration method''' for registration. Also we gave one point as landmark point.
<gallery widths="400px" perrow="6">
Image:58_MRI.jpg
Image:64_MRI.jpg
Image:64_Registered.jpg
</gallery>

We also tried '''affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''one landmark point'''. The zipped file which contains scene along with screenshots is
[[Image:64 58 Affine Registration.zip |left|64_Affine_Registered_Scene]]
<gallery widths="400px" perrow="6">
Image:64 Affine Registration.JPG
Image:64 Registerd SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and one landmark point. The zipped file which contains scene along with screenshots is
[[Image:64 58 BSpline Registration.zip |left|64_BSpline_Registered_Scene]]. Here we used '''4 points''' as landmarks.
<gallery widths="400px" perrow="6">
Image:64 BSpline Registration.JPG
Image:64 BSpline Registered SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and four landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. Here we used '''4 points''' as landmarks. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 13, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 14, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''10''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 Affine 50 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 100 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 150 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 250 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 350 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 450 Iterations 10 Landmark Points.jpg
</gallery>

==='''Update May 20, 2009'''===
We tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 BSpline 10 Iterations.JPG
Image:64 58 BSpline 110 Iterations.JPG
Image:64 58 BSpline 160 Iterations.JPG
Image:64 58 BSpline 210 Iterations.JPG
Image:64 58 BSpline 260 Iterations.JPG
Image:64 58 BSpline 360 Iterations.JPG
</gallery>

==='''Update May 21, 2009'''===
We tried '''Pipelined Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 PipeLinedAffine Registration 50 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 100 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 150 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 200 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 300 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 400 Iterations.JPG
</gallery>

==='''Update May 25, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 5000.
<gallery widths="200px" perrow="6">
Image:AffineRegistration InitialTransform 200Iterations.JPG
Image:AffineRegistration InitialTransform 300Iterations.JPG
Image:AffineRegistration InitialTransform 400Iterations.JPG
</gallery>

===Multi Image Registration===
We have also tried the approach of Non-rigid Groupwise Registration using Bspline Deformation Model by Serdar K. Balci, Polina Golland and William M. Wells. In this approach, we try to give one slice each of two different patients we need to register as input. Here are some of the results of their approach.

<gallery widths="200px" perrow="2">
Slice 1:
Image:1_Slicez0.JPG‎
Slice 2:
Image:1_Slicez1.JPG
Mean Slice of above two :
Image:1_MeanSliceZ.JPG
</gallery>

<gallery widths="200px" perrow="2">
Slice 1:
Image:2_Z0.JPG
Slice 2:
Image:2_Z1.JPG
Mean Slice of above two :
Image:2_MeanSliceZ.JPG
</gallery>
==Image Segmentation Methods (Oct-Dec) 2009==
==='''Shape Dectection Level Set Approach'''===
We started in October with Shape Dectection Level Set algorithm. In this algorithm, the governing differential equation has an additional curvature-based term. This term acts as a smoothing term where areas of high curvature, assumed to be due to noise,
are smoothed out. Scaling parameters are used to control the tradeoff between the expansion term and the smoothing term. One consequence of this additional curvature term is that the fast marching algorithm is no longer applicable, because the contour is no longer guaranteed to always be expanding. Instead, the level set function is updated iteratively. The ShapeDetectionLevelSetImageFilter expects two inputs, the first being an initial Level Set in the form of an itk::Image, and the second being a feature image. For this algorithm, the feature image is an edge potential image that basically follows the same rules applicable to the speed image used for the FastMarchingImageFilter. The pipeline of the algorithm is as follows

<gallery widths="200px" perrow="2">
Image:Algo.JPG
</gallery>

The output of this algorithm is as follows.
<gallery widths="200px" perrow="3">
Image:Femur_Tibia_Segmented.JPG‎
Image:Tibia_Boundary.JPG
Image:Femurr Boundary.JPG
</gallery>

The models extracted from the segmented label maps are as follows.
<gallery widths="200px" perrow="2">
Image:ShapeDetectionModel.JPG
Image:ShapeDetectionModelSmoothing.JPG
</gallery>

==='''Multi-Contrast MR Approach'''===
In this algorithm, we try to collect MR images of Knee using different MR scans like IDEAL-SPGR-FAT, IDEAL-SPGR-WATER, IDEAL-GRE-FAT etc. The main assumption in this algorithm is that different structures of interest ( eg: bones/cartilage) have different intensity values in different scans. For eg: Bones are colored with lower pixel intensity values in IDEAL-SPGR-WATER dataset whereas they are colored with higher pixel intensity values in IDEAL-SPGR-WATER dataset.

Firstly, we collect the seed points of the regions of interest using an insight application "ImageViewer". So each file with the seed points is considered as one cluster. For eg: we can have clusters for background, bones, cartilage etc. Next, we calculate the cluster centers (µ) and standard deviation (ρ) for each seed file. We try to define user defined parameter called numberOfSd(α) which represents the extent of how far a cluster can go. The numberofSd(α) has been split to be bi-directional in the x-axis representing one contrast image and y-axis representing another contrast image.

Below is the dataset of Knee MR image we used
<gallery widths="200px" perrow="2">
Image:IdealSPGRWater.JPG | IDEAL-SPGR-WATER
Image:IdealSPGRFat.JPG | IDEAL-SPGR-FAT
</gallery>

Scatter Plot of the above two images
<gallery widths="200px" perrow="2">
Image:ScatterPlot.JPG | Scatter Plot
</gallery>

Segmentation Output we obtained :
<gallery widths="200px" perrow="1">
Image:SegmentationResult.JPG | White: Bones | Green: Cartilage | Red: Background
</gallery>

We have also proved that our algorithm works fairly well for hip dataset. Below is the segmented output of Hip dataset.
<gallery widths="200px" perrow="2">
Image:Hip.JPG
</gallery>

The segmented output we got using our approach is
<gallery widths="200px" perrow="2">
Image:HipOutput.JPG
</gallery>
==Future Work==
Build Models from the segmented label maps that we get from Multi MR contrast images.

Extend our existing Multi Contrast MR solution to other body datasets.

Improve segmentation results from our current dataset by using "more" multi contrast images.

==References==
1. ITK Software Guide ( http://www.itk.org)

2. Slicer Software (http://www.slicer.org)

3. Multi-Contrast MR for Enhanced Bone Imaging and Segmentation Ruphin Dalvi, Rafeef Abugharbieh, Derek C. Wilson, David R. Wilson.

4. HE Cline. WE Lorensen, R Kikinis, F Jolesz 3D Segmentation of the Head Using Probability and Connectivity Journal of Computer Assisted Tomography 14:1037-1045 (1990)

Stanford Simbios group

2009-12-18T22:16:45Z

Harishd: /* References */

Back to [[NA-MIC_External_Collaborations|NA-MIC External Collaborations]]
__NOTOC__
==Grant#==
U54EB005149-05S2
==Key Personnel==
*Stanford: Scott Delp, PI, Harris Doddi, Saikat Pal
*NA-MIC: Ron Kikinis, Steve Pieper
*Kitware: Luis Ibanez
==Grant Duration==
09/19/2008-07/31/2010

<gallery widths="300px">
Image:58.jpg
Image:All_Three.JPG
Image:Femur_Patella_Tibia.jpg
</gallery>

==Aim==
To develop a generic semi-automatic segmentation toolkit to convert images of musculoskeletal structures to 3D models.

==Process Flowchart==
[[Image:FlowChart.jpg|left| Process Diagram.]]

==Progress==
==Atlas Generation from Input MR Images==
===Model Generation from Input MR Images===
Pre-Segmented models for femur, patella and tibia are obtained for a patient (in .stl format).
[[Image:All_Three.JPG|left|thumb| Pre-Segmented Femur/Patella/Tibia Model]]

===Create a filled label map using PolyDataToFilledLabelMap module in slicer===
The models generated in the above step are closed but hollow. But EM Segmentation requires the atlas given to be in closed filled format. Hence we converted the hollow closed models into filled label volumes using the module PolyDataToFilledLabelMap module in Slicer. The output label map of this module is of datatype unsigned char and is converted to short datatype.
<gallery widths="400px" perrow="6">
Image:LabelMap_Femur.jpg
Image:LabelMap_Tibia.jpg
Image:LabelMap_Patella.jpg
</gallery>

===EM Segmentation based on the atlas===
The output label map in the above is given as input atlas in the EM Segmentation step. As an initial step we ran EM Segmentation on the same patient. The EM Segmented output is given in the below figure.
[[Image:Femur_Patella_Tibia.jpg|left|thumb| EM Segmented Output]]
==Register Images==
===Register Images Module in Slicer===
We are now in the process of trying to register new patient image to the existing patient image. If we register the images, we can get the transform which can be used to register the existing atlas (label map) to the new patient.

We think that Pipelined Bspline Registration may give promising results for our dataset. Below are some of the results. Here we consider 64_MRI as the fixed image and 58_MRI as the moving image and used '''Pipelined BSpline Image registration method''' for registration. Also we gave one point as landmark point.
<gallery widths="400px" perrow="6">
Image:58_MRI.jpg
Image:64_MRI.jpg
Image:64_Registered.jpg
</gallery>

We also tried '''affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''one landmark point'''. The zipped file which contains scene along with screenshots is
[[Image:64 58 Affine Registration.zip |left|64_Affine_Registered_Scene]]
<gallery widths="400px" perrow="6">
Image:64 Affine Registration.JPG
Image:64 Registerd SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and one landmark point. The zipped file which contains scene along with screenshots is
[[Image:64 58 BSpline Registration.zip |left|64_BSpline_Registered_Scene]]. Here we used '''4 points''' as landmarks.
<gallery widths="400px" perrow="6">
Image:64 BSpline Registration.JPG
Image:64 BSpline Registered SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and four landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. Here we used '''4 points''' as landmarks. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 13, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 14, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''10''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 Affine 50 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 100 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 150 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 250 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 350 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 450 Iterations 10 Landmark Points.jpg
</gallery>

==='''Update May 20, 2009'''===
We tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 BSpline 10 Iterations.JPG
Image:64 58 BSpline 110 Iterations.JPG
Image:64 58 BSpline 160 Iterations.JPG
Image:64 58 BSpline 210 Iterations.JPG
Image:64 58 BSpline 260 Iterations.JPG
Image:64 58 BSpline 360 Iterations.JPG
</gallery>

==='''Update May 21, 2009'''===
We tried '''Pipelined Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 PipeLinedAffine Registration 50 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 100 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 150 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 200 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 300 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 400 Iterations.JPG
</gallery>

==='''Update May 25, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 5000.
<gallery widths="200px" perrow="6">
Image:AffineRegistration InitialTransform 200Iterations.JPG
Image:AffineRegistration InitialTransform 300Iterations.JPG
Image:AffineRegistration InitialTransform 400Iterations.JPG
</gallery>

===Multi Image Registration===
We have also tried the approach of Non-rigid Groupwise Registration using Bspline Deformation Model by Serdar K. Balci, Polina Golland and William M. Wells. In this approach, we try to give one slice each of two different patients we need to register as input. Here are some of the results of their approach.

<gallery widths="200px" perrow="2">
Slice 1:
Image:1_Slicez0.JPG‎
Slice 2:
Image:1_Slicez1.JPG
Mean Slice of above two :
Image:1_MeanSliceZ.JPG
</gallery>

<gallery widths="200px" perrow="2">
Slice 1:
Image:2_Z0.JPG
Slice 2:
Image:2_Z1.JPG
Mean Slice of above two :
Image:2_MeanSliceZ.JPG
</gallery>
==Image Segmentation Methods (Oct-Dec) 2009==
==='''Shape Dectection Level Set Approach'''===
We started in October with Shape Dectection Level Set algorithm. In this algorithm, the governing differential equation has an additional curvature-based term. This term acts as a smoothing term where areas of high curvature, assumed to be due to noise,
are smoothed out. Scaling parameters are used to control the tradeoff between the expansion term and the smoothing term. One consequence of this additional curvature term is that the fast marching algorithm is no longer applicable, because the contour is no longer guaranteed to always be expanding. Instead, the level set function is updated iteratively. The ShapeDetectionLevelSetImageFilter expects two inputs, the first being an initial Level Set in the form of an itk::Image, and the second being a feature image. For this algorithm, the feature image is an edge potential image that basically follows the same rules applicable to the speed image used for the FastMarchingImageFilter. The pipeline of the algorithm is as follows

<gallery widths="200px" perrow="2">
Image:Algo.JPG
</gallery>

The output of this algorithm is as follows.
<gallery widths="200px" perrow="3">
Image:Femur_Tibia_Segmented.JPG‎
Image:Tibia_Boundary.JPG
Image:Femurr Boundary.JPG
</gallery>

The models extracted from the segmented label maps are as follows.
<gallery widths="200px" perrow="2">
Image:ShapeDetectionModel.JPG
Image:ShapeDetectionModelSmoothing.JPG
</gallery>

==='''Multi-Contrast MR Approach'''===
In this algorithm, we try to collect MR images of Knee using different MR scans like IDEAL-SPGR-FAT, IDEAL-SPGR-WATER, IDEAL-GRE-FAT etc. The main assumption in this algorithm is that different structures of interest ( eg: bones/cartilage) have different intensity values in different scans. For eg: Bones are colored with lower pixel intensity values in IDEAL-SPGR-WATER dataset whereas they are colored with higher pixel intensity values in IDEAL-SPGR-WATER dataset.

Firstly, we collect the seed points of the regions of interest using an insight application "ImageViewer". So each file with the seed points is considered as one cluster. For eg: we can have clusters for background, bones, cartilage etc. Next, we calculate the cluster centers (µ) and standard deviation (ρ) for each seed file. We try to define user defined parameter called numberOfSd(α) which represents the extent of how far a cluster can go. The numberofSd(α) has been split to be bi-directional in the x-axis representing one contrast image and y-axis representing another contrast image.

Below is the dataset of Knee MR image we used
<gallery widths="200px" perrow="2">
Image:IdealSPGRWater.JPG | IDEAL-SPGR-WATER
Image:IdealSPGRFat.JPG | IDEAL-SPGR-FAT
</gallery>

Scatter Plot of the above two images
<gallery widths="200px" perrow="2">
Image:ScatterPlot.JPG | Scatter Plot
</gallery>

Segmentation Output we obtained :
<gallery widths="200px" perrow="1">
Image:SegmentationResult.JPG | White: Bones | Green: Cartilage | Red: Background
</gallery>

We have also proved that our algorithm works fairly well for hip dataset. Below is the segmented output of Hip dataset.
<gallery widths="200px" perrow="2">
Image:Hip.JPG
</gallery>

The segmented output we got using our approach is
<gallery widths="200px" perrow="2">
Image:HipOutput.JPG
</gallery>
==Future Work==
Build Models from the segmented label maps that we get from Multi MR contrast images.
Extend our existing Multi Contrast MR solution to other body datasets.
Improve segmentation results from our current dataset by using "more" multi contrast images.

==References==
1. ITK Software Guide ( http://www.itk.org)

2. Slicer Software (http://www.slicer.org)

3. Multi-Contrast MR for Enhanced Bone Imaging and Segmentation Ruphin Dalvi, Rafeef Abugharbieh, Derek C. Wilson, David R. Wilson.

4. HE Cline. WE Lorensen, R Kikinis, F Jolesz 3D Segmentation of the Head Using Probability and Connectivity Journal of Computer Assisted Tomography 14:1037-1045 (1990)

Stanford Simbios group

2009-12-18T22:16:19Z

Harishd:

Back to [[NA-MIC_External_Collaborations|NA-MIC External Collaborations]]
__NOTOC__
==Grant#==
U54EB005149-05S2
==Key Personnel==
*Stanford: Scott Delp, PI, Harris Doddi, Saikat Pal
*NA-MIC: Ron Kikinis, Steve Pieper
*Kitware: Luis Ibanez
==Grant Duration==
09/19/2008-07/31/2010

<gallery widths="300px">
Image:58.jpg
Image:All_Three.JPG
Image:Femur_Patella_Tibia.jpg
</gallery>

==Aim==
To develop a generic semi-automatic segmentation toolkit to convert images of musculoskeletal structures to 3D models.

==Process Flowchart==
[[Image:FlowChart.jpg|left| Process Diagram.]]

==Progress==
==Atlas Generation from Input MR Images==
===Model Generation from Input MR Images===
Pre-Segmented models for femur, patella and tibia are obtained for a patient (in .stl format).
[[Image:All_Three.JPG|left|thumb| Pre-Segmented Femur/Patella/Tibia Model]]

===Create a filled label map using PolyDataToFilledLabelMap module in slicer===
The models generated in the above step are closed but hollow. But EM Segmentation requires the atlas given to be in closed filled format. Hence we converted the hollow closed models into filled label volumes using the module PolyDataToFilledLabelMap module in Slicer. The output label map of this module is of datatype unsigned char and is converted to short datatype.
<gallery widths="400px" perrow="6">
Image:LabelMap_Femur.jpg
Image:LabelMap_Tibia.jpg
Image:LabelMap_Patella.jpg
</gallery>

===EM Segmentation based on the atlas===
The output label map in the above is given as input atlas in the EM Segmentation step. As an initial step we ran EM Segmentation on the same patient. The EM Segmented output is given in the below figure.
[[Image:Femur_Patella_Tibia.jpg|left|thumb| EM Segmented Output]]
==Register Images==
===Register Images Module in Slicer===
We are now in the process of trying to register new patient image to the existing patient image. If we register the images, we can get the transform which can be used to register the existing atlas (label map) to the new patient.

We think that Pipelined Bspline Registration may give promising results for our dataset. Below are some of the results. Here we consider 64_MRI as the fixed image and 58_MRI as the moving image and used '''Pipelined BSpline Image registration method''' for registration. Also we gave one point as landmark point.
<gallery widths="400px" perrow="6">
Image:58_MRI.jpg
Image:64_MRI.jpg
Image:64_Registered.jpg
</gallery>

We also tried '''affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''one landmark point'''. The zipped file which contains scene along with screenshots is
[[Image:64 58 Affine Registration.zip |left|64_Affine_Registered_Scene]]
<gallery widths="400px" perrow="6">
Image:64 Affine Registration.JPG
Image:64 Registerd SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and one landmark point. The zipped file which contains scene along with screenshots is
[[Image:64 58 BSpline Registration.zip |left|64_BSpline_Registered_Scene]]. Here we used '''4 points''' as landmarks.
<gallery widths="400px" perrow="6">
Image:64 BSpline Registration.JPG
Image:64 BSpline Registered SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and four landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. Here we used '''4 points''' as landmarks. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 13, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 14, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''10''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 Affine 50 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 100 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 150 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 250 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 350 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 450 Iterations 10 Landmark Points.jpg
</gallery>

==='''Update May 20, 2009'''===
We tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 BSpline 10 Iterations.JPG
Image:64 58 BSpline 110 Iterations.JPG
Image:64 58 BSpline 160 Iterations.JPG
Image:64 58 BSpline 210 Iterations.JPG
Image:64 58 BSpline 260 Iterations.JPG
Image:64 58 BSpline 360 Iterations.JPG
</gallery>

==='''Update May 21, 2009'''===
We tried '''Pipelined Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 PipeLinedAffine Registration 50 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 100 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 150 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 200 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 300 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 400 Iterations.JPG
</gallery>

==='''Update May 25, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 5000.
<gallery widths="200px" perrow="6">
Image:AffineRegistration InitialTransform 200Iterations.JPG
Image:AffineRegistration InitialTransform 300Iterations.JPG
Image:AffineRegistration InitialTransform 400Iterations.JPG
</gallery>

===Multi Image Registration===
We have also tried the approach of Non-rigid Groupwise Registration using Bspline Deformation Model by Serdar K. Balci, Polina Golland and William M. Wells. In this approach, we try to give one slice each of two different patients we need to register as input. Here are some of the results of their approach.

<gallery widths="200px" perrow="2">
Slice 1:
Image:1_Slicez0.JPG‎
Slice 2:
Image:1_Slicez1.JPG
Mean Slice of above two :
Image:1_MeanSliceZ.JPG
</gallery>

<gallery widths="200px" perrow="2">
Slice 1:
Image:2_Z0.JPG
Slice 2:
Image:2_Z1.JPG
Mean Slice of above two :
Image:2_MeanSliceZ.JPG
</gallery>
==Image Segmentation Methods (Oct-Dec) 2009==
==='''Shape Dectection Level Set Approach'''===
We started in October with Shape Dectection Level Set algorithm. In this algorithm, the governing differential equation has an additional curvature-based term. This term acts as a smoothing term where areas of high curvature, assumed to be due to noise,
are smoothed out. Scaling parameters are used to control the tradeoff between the expansion term and the smoothing term. One consequence of this additional curvature term is that the fast marching algorithm is no longer applicable, because the contour is no longer guaranteed to always be expanding. Instead, the level set function is updated iteratively. The ShapeDetectionLevelSetImageFilter expects two inputs, the first being an initial Level Set in the form of an itk::Image, and the second being a feature image. For this algorithm, the feature image is an edge potential image that basically follows the same rules applicable to the speed image used for the FastMarchingImageFilter. The pipeline of the algorithm is as follows

<gallery widths="200px" perrow="2">
Image:Algo.JPG
</gallery>

The output of this algorithm is as follows.
<gallery widths="200px" perrow="3">
Image:Femur_Tibia_Segmented.JPG‎
Image:Tibia_Boundary.JPG
Image:Femurr Boundary.JPG
</gallery>

The models extracted from the segmented label maps are as follows.
<gallery widths="200px" perrow="2">
Image:ShapeDetectionModel.JPG
Image:ShapeDetectionModelSmoothing.JPG
</gallery>

==='''Multi-Contrast MR Approach'''===
In this algorithm, we try to collect MR images of Knee using different MR scans like IDEAL-SPGR-FAT, IDEAL-SPGR-WATER, IDEAL-GRE-FAT etc. The main assumption in this algorithm is that different structures of interest ( eg: bones/cartilage) have different intensity values in different scans. For eg: Bones are colored with lower pixel intensity values in IDEAL-SPGR-WATER dataset whereas they are colored with higher pixel intensity values in IDEAL-SPGR-WATER dataset.

Firstly, we collect the seed points of the regions of interest using an insight application "ImageViewer". So each file with the seed points is considered as one cluster. For eg: we can have clusters for background, bones, cartilage etc. Next, we calculate the cluster centers (µ) and standard deviation (ρ) for each seed file. We try to define user defined parameter called numberOfSd(α) which represents the extent of how far a cluster can go. The numberofSd(α) has been split to be bi-directional in the x-axis representing one contrast image and y-axis representing another contrast image.

Below is the dataset of Knee MR image we used
<gallery widths="200px" perrow="2">
Image:IdealSPGRWater.JPG | IDEAL-SPGR-WATER
Image:IdealSPGRFat.JPG | IDEAL-SPGR-FAT
</gallery>

Scatter Plot of the above two images
<gallery widths="200px" perrow="2">
Image:ScatterPlot.JPG | Scatter Plot
</gallery>

Segmentation Output we obtained :
<gallery widths="200px" perrow="1">
Image:SegmentationResult.JPG | White: Bones | Green: Cartilage | Red: Background
</gallery>

We have also proved that our algorithm works fairly well for hip dataset. Below is the segmented output of Hip dataset.
<gallery widths="200px" perrow="2">
Image:Hip.JPG
</gallery>

The segmented output we got using our approach is
<gallery widths="200px" perrow="2">
Image:HipOutput.JPG
</gallery>
==Future Work==
Build Models from the segmented label maps that we get from Multi MR contrast images.
Extend our existing Multi Contrast MR solution to other body datasets.
Improve segmentation results from our current dataset by using "more" multi contrast images.

==References==
1. ITK Software Guide ( http://www.itk.org)
2. Slicer Software (http://www.slicer.org)
3. Multi-Contrast MR for Enhanced Bone Imaging and Segmentation Ruphin Dalvi, Rafeef Abugharbieh, Derek C. Wilson, David R. Wilson.
4. HE Cline. WE Lorensen, R Kikinis, F Jolesz 3D Segmentation of the Head Using Probability and Connectivity Journal of Computer Assisted Tomography 14:1037-1045 (1990)

File:HipOutput.JPG

2009-12-18T22:14:29Z

Harishd:

Stanford Simbios group

2009-12-18T22:13:26Z

2009-12-18T21:49:31Z

Harishd:

Back to [[NA-MIC_External_Collaborations|NA-MIC External Collaborations]]
__NOTOC__
==Grant#==
U54EB005149-05S2
==Key Personnel==
*Stanford: Scott Delp, PI, Harris Doddi, Saikat Pal
*NA-MIC: Ron Kikinis, Steve Pieper
*Kitware: Luis Ibanez
==Grant Duration==
09/19/2008-07/31/2010

<gallery widths="300px">
Image:58.jpg
Image:All_Three.JPG
Image:Femur_Patella_Tibia.jpg
</gallery>

==Aim==
To develop a generic semi-automatic segmentation toolkit to convert images of musculoskeletal structures to 3D models.

==Process Flowchart==
[[Image:FlowChart.jpg|left| Process Diagram.]]

==Progress==
==Atlas Generation from Input MR Images==
===Model Generation from Input MR Images===
Pre-Segmented models for femur, patella and tibia are obtained for a patient (in .stl format).
[[Image:All_Three.JPG|left|thumb| Pre-Segmented Femur/Patella/Tibia Model]]

===Create a filled label map using PolyDataToFilledLabelMap module in slicer===
The models generated in the above step are closed but hollow. But EM Segmentation requires the atlas given to be in closed filled format. Hence we converted the hollow closed models into filled label volumes using the module PolyDataToFilledLabelMap module in Slicer. The output label map of this module is of datatype unsigned char and is converted to short datatype.
<gallery widths="400px" perrow="6">
Image:LabelMap_Femur.jpg
Image:LabelMap_Tibia.jpg
Image:LabelMap_Patella.jpg
</gallery>

===EM Segmentation based on the atlas===
The output label map in the above is given as input atlas in the EM Segmentation step. As an initial step we ran EM Segmentation on the same patient. The EM Segmented output is given in the below figure.
[[Image:Femur_Patella_Tibia.jpg|left|thumb| EM Segmented Output]]
==Register Images==
===Register Images Module in Slicer===
We are now in the process of trying to register new patient image to the existing patient image. If we register the images, we can get the transform which can be used to register the existing atlas (label map) to the new patient.

We think that Pipelined Bspline Registration may give promising results for our dataset. Below are some of the results. Here we consider 64_MRI as the fixed image and 58_MRI as the moving image and used '''Pipelined BSpline Image registration method''' for registration. Also we gave one point as landmark point.
<gallery widths="400px" perrow="6">
Image:58_MRI.jpg
Image:64_MRI.jpg
Image:64_Registered.jpg
</gallery>

We also tried '''affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''one landmark point'''. The zipped file which contains scene along with screenshots is
[[Image:64 58 Affine Registration.zip |left|64_Affine_Registered_Scene]]
<gallery widths="400px" perrow="6">
Image:64 Affine Registration.JPG
Image:64 Registerd SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and one landmark point. The zipped file which contains scene along with screenshots is
[[Image:64 58 BSpline Registration.zip |left|64_BSpline_Registered_Scene]]. Here we used '''4 points''' as landmarks.
<gallery widths="400px" perrow="6">
Image:64 BSpline Registration.JPG
Image:64 BSpline Registered SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and four landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. Here we used '''4 points''' as landmarks. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 13, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 14, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''10''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 Affine 50 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 100 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 150 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 250 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 350 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 450 Iterations 10 Landmark Points.jpg
</gallery>

==='''Update May 20, 2009'''===
We tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 BSpline 10 Iterations.JPG
Image:64 58 BSpline 110 Iterations.JPG
Image:64 58 BSpline 160 Iterations.JPG
Image:64 58 BSpline 210 Iterations.JPG
Image:64 58 BSpline 260 Iterations.JPG
Image:64 58 BSpline 360 Iterations.JPG
</gallery>

==='''Update May 21, 2009'''===
We tried '''Pipelined Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 PipeLinedAffine Registration 50 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 100 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 150 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 200 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 300 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 400 Iterations.JPG
</gallery>

==='''Update May 25, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 5000.
<gallery widths="200px" perrow="6">
Image:AffineRegistration InitialTransform 200Iterations.JPG
Image:AffineRegistration InitialTransform 300Iterations.JPG
Image:AffineRegistration InitialTransform 400Iterations.JPG
</gallery>

===Multi Image Registration===
We have also tried the approach of Non-rigid Groupwise Registration using Bspline Deformation Model by Serdar K. Balci, Polina Golland and William M. Wells. In this approach, we try to give one slice each of two different patients we need to register as input. Here are some of the results of their approach.

<gallery widths="200px" perrow="2">
Slice 1:
Image:1_Slicez0.JPG‎
Slice 2:
Image:1_Slicez1.JPG
Mean Slice of above two :
Image:1_MeanSliceZ.JPG
</gallery>

<gallery widths="200px" perrow="2">
Slice 1:
Image:2_Z0.JPG
Slice 2:
Image:2_Z1.JPG
Mean Slice of above two :
Image:2_MeanSliceZ.JPG
</gallery>

==='''Updates Oct-Dec, 2009'''===
We started in October with Shape Dectection Level Set algorithm. In this algorithm, the governing differential equation has an additional curvature-based term. This term acts as a smoothing term where areas of high curvature, assumed to be due to noise,
are smoothed out. Scaling parameters are used to control the tradeoff between the expansion term and the smoothing term. One consequence of this additional curvature term is that the fast marching algorithm is no longer applicable, because the contour is no longer guaranteed to always be expanding. Instead, the level set function is updated iteratively. The ShapeDetectionLevelSetImageFilter expects two inputs, the first being an initial Level Set in the form of an itk::Image, and the second being a feature image. For this algorithm, the feature image is an edge potential image that basically follows the same rules applicable to the speed image used for the FastMarchingImageFilter. The pipeline of the algorithm is as follows

<gallery widths="200px" perrow="2">
Image:Algo.JPG
</gallery>

The output of this algorithm is as follows.
<gallery widths="200px" perrow="3">
Image:Femur_Tibia_Segmented.JPG‎
Image:Tibia_Boundary.JPG
Image:Femurr Boundary.JPG
</gallery>

The models extracted from the segmented label maps are as follows.
<gallery widths="200px" perrow="3">
Image:ShapeDetectionModel.JPG
</gallery>

File:ShapeDetectionModel.JPG

2009-12-18T21:48:35Z

Harishd:

Stanford Simbios group

2009-12-18T21:46:54Z

Harishd:

Back to [[NA-MIC_External_Collaborations|NA-MIC External Collaborations]]
__NOTOC__
==Grant#==
U54EB005149-05S2
==Key Personnel==
*Stanford: Scott Delp, PI, Harris Doddi, Saikat Pal
*NA-MIC: Ron Kikinis, Steve Pieper
*Kitware: Luis Ibanez
==Grant Duration==
09/19/2008-07/31/2010

<gallery widths="300px">
Image:58.jpg
Image:All_Three.JPG
Image:Femur_Patella_Tibia.jpg
</gallery>

==Aim==
To develop a generic semi-automatic segmentation toolkit to convert images of musculoskeletal structures to 3D models.

==Process Flowchart==
[[Image:FlowChart.jpg|left| Process Diagram.]]

==Progress==
==Atlas Generation from Input MR Images==
===Model Generation from Input MR Images===
Pre-Segmented models for femur, patella and tibia are obtained for a patient (in .stl format).
[[Image:All_Three.JPG|left|thumb| Pre-Segmented Femur/Patella/Tibia Model]]

===Create a filled label map using PolyDataToFilledLabelMap module in slicer===
The models generated in the above step are closed but hollow. But EM Segmentation requires the atlas given to be in closed filled format. Hence we converted the hollow closed models into filled label volumes using the module PolyDataToFilledLabelMap module in Slicer. The output label map of this module is of datatype unsigned char and is converted to short datatype.
<gallery widths="400px" perrow="6">
Image:LabelMap_Femur.jpg
Image:LabelMap_Tibia.jpg
Image:LabelMap_Patella.jpg
</gallery>

===EM Segmentation based on the atlas===
The output label map in the above is given as input atlas in the EM Segmentation step. As an initial step we ran EM Segmentation on the same patient. The EM Segmented output is given in the below figure.
[[Image:Femur_Patella_Tibia.jpg|left|thumb| EM Segmented Output]]
==Register Images==
===Register Images Module in Slicer===
We are now in the process of trying to register new patient image to the existing patient image. If we register the images, we can get the transform which can be used to register the existing atlas (label map) to the new patient.

We think that Pipelined Bspline Registration may give promising results for our dataset. Below are some of the results. Here we consider 64_MRI as the fixed image and 58_MRI as the moving image and used '''Pipelined BSpline Image registration method''' for registration. Also we gave one point as landmark point.
<gallery widths="400px" perrow="6">
Image:58_MRI.jpg
Image:64_MRI.jpg
Image:64_Registered.jpg
</gallery>

We also tried '''affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''one landmark point'''. The zipped file which contains scene along with screenshots is
[[Image:64 58 Affine Registration.zip |left|64_Affine_Registered_Scene]]
<gallery widths="400px" perrow="6">
Image:64 Affine Registration.JPG
Image:64 Registerd SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and one landmark point. The zipped file which contains scene along with screenshots is
[[Image:64 58 BSpline Registration.zip |left|64_BSpline_Registered_Scene]]. Here we used '''4 points''' as landmarks.
<gallery widths="400px" perrow="6">
Image:64 BSpline Registration.JPG
Image:64 BSpline Registered SuperImposed On 64 Actual.JPG
</gallery>

We also tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and four landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. Here we used '''4 points''' as landmarks. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 13, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:64_58_Affine_Various_Iterations.zip |left|64_Affine_Registered_Various_Iterations_Scene]]. We varied the number of iterations from 200 to 400.
<gallery widths="400px" perrow="6">
Image:64 58 Affine 200 Iterations.JPG
Image:64 58 Affine 300 Iterations.JPG
Image:64 58 Affine 400 Iterations.JPG
</gallery>

==='''Update May 14, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''10''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 Affine 50 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 100 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 150 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 250 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 350 Iterations 10 Landmark Points.jpg
Image:64 58 Affine 450 Iterations 10 Landmark Points.jpg
</gallery>

==='''Update May 20, 2009'''===
We tried '''BSpline registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:Affine Registration 10 Landmark Points Varying Iterations.zip |left|64_Affine_Registered_10_Landmarks_Various_Iterations_Scene]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 BSpline 10 Iterations.JPG
Image:64 58 BSpline 110 Iterations.JPG
Image:64 58 BSpline 160 Iterations.JPG
Image:64 58 BSpline 210 Iterations.JPG
Image:64 58 BSpline 260 Iterations.JPG
Image:64 58 BSpline 360 Iterations.JPG
</gallery>

==='''Update May 21, 2009'''===
We tried '''Pipelined Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 450.
<gallery widths="200px" perrow="6">
Image:64 58 PipeLinedAffine Registration 50 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 100 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 150 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 200 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 300 Iterations.JPG
Image:64 58 PipeLinedAffine Registration 400 Iterations.JPG
</gallery>

==='''Update May 25, 2009'''===
We tried '''Affine registration''' for the above mentioned patients with 64 as fixed MRI and 58 as moving MRI and '''no''' landmark point. The zipped file which contains scene along with screenshots is
[[Image:BSpline_Registration_Varying_Iterations.zip‎ |left|BSpline_Registration_Varying_Iterations]]. We varied the number of iterations from 50 to 5000.
<gallery widths="200px" perrow="6">
Image:AffineRegistration InitialTransform 200Iterations.JPG
Image:AffineRegistration InitialTransform 300Iterations.JPG
Image:AffineRegistration InitialTransform 400Iterations.JPG
</gallery>

===Multi Image Registration===
We have also tried the approach of Non-rigid Groupwise Registration using Bspline Deformation Model by Serdar K. Balci, Polina Golland and William M. Wells. In this approach, we try to give one slice each of two different patients we need to register as input. Here are some of the results of their approach.

<gallery widths="200px" perrow="2">
Slice 1:
Image:1_Slicez0.JPG‎
Slice 2:
Image:1_Slicez1.JPG
Mean Slice of above two :
Image:1_MeanSliceZ.JPG
</gallery>

<gallery widths="200px" perrow="2">
Slice 1:
Image:2_Z0.JPG
Slice 2:
Image:2_Z1.JPG
Mean Slice of above two :
Image:2_MeanSliceZ.JPG
</gallery>

==='''Updates Oct-Dec, 2009'''===
We started in October with Shape Dectection Level Set algorithm. In this algorithm, the governing differential equation has an additional curvature-based term. This term acts as a smoothing term where areas of high curvature, assumed to be due to noise,
are smoothed out. Scaling parameters are used to control the tradeoff between the expansion term and the smoothing term. One consequence of this additional curvature term is that the fast marching algorithm is no longer applicable, because the contour is no longer guaranteed to always be expanding. Instead, the level set function is updated iteratively. The ShapeDetectionLevelSetImageFilter expects two inputs, the first being an initial Level Set in the form of an itk::Image, and the second being a feature image. For this algorithm, the feature image is an edge potential image that basically follows the same rules applicable to the speed image used for the FastMarchingImageFilter. The pipeline of the algorithm is as follows

<gallery widths="200px" perrow="2">
Image:Algo.JPG
</gallery>

The output of this algorithm is as follows.
<gallery widths="200px" perrow="2">
Image:Femur_Tibia_Segmented.JPG‎
Image:Tibia_Boundary.JPG
Image:Femurr Boundary.JPG
</gallery>

File:Femurr Boundary.JPG

2009-12-18T21:46:36Z

Harishd:

File:Tibia Boundary.JPG

2009-12-18T21:45:05Z

Harishd:

2009-12-18T21:38:02Z

Harishd: