Difference between revisions of "2008 Summer Project Week:Batch Processing"

From NAMIC Wiki
Jump to: navigation, search
(New page: {| |thumb|320px|Return to [[2008_Summer_Project_Week|Project Week Main Page ]] |} __NOTOC__ ===Key Investigators=== * Julien Jomier * Steve Pieper * Marco...)
 
 
(3 intermediate revisions by 2 users not shown)
Line 11: Line 11:
 
* Marco Ruiz
 
* Marco Ruiz
 
* Jim Miller
 
* Jim Miller
 +
* Chris Churas
 +
* Hans Johnson
 +
* Killian Pohl
 +
* Jeff Grethe
 +
* Mark Anderson
  
 
<div style="margin: 20px;">
 
<div style="margin: 20px;">
Line 33: Line 38:
  
 
<h1>Progress</h1>
 
<h1>Progress</h1>
 +
* Identified separation between GWE and BatchMake
 +
** GWE: Running Slicer modules on parameter space from within slicer, console utility or scripts.
 +
** BatchMake: Scripting using Slicer3 modules (outside of Slicer3 GUI) and reporting
  
 +
* Cases
 +
** 175 cases processed with BSplineDeformableRegistration on the SPL cluster using GWE.
 +
** 2275 cases processed with BSplineDeformableRegistration on the BIRN cluster using GWE.
 +
** 16 cases processed with FreeSurfer on the BIRN cluster using GWE.
 +
** Working out some Condor issues on the SPL cluster.
 +
** Prepared commands to process 50 cases with Killian's pipeline on the SPL cluster using GWE.
 +
** Reviewed Sylvain's cases requirements to verify they could be processed in a cluster using GWE.
 +
 +
* Future Work:
 +
** BatchMake
 +
*** Implement support to run through GWE
 +
*** Provide reporting tool as slicer execution modules
 +
** GWE
 +
*** Detailed on the "future work" section of [[Project_Week_2008_Special_topic_breakout:_GWE]]
 +
** GSlicer3
 +
*** Auto-deploy Slicer3 in clusters where module invocations will be parallelize.
 +
*** Real time monitor interface to track status of requests submitted to clusters.
 +
*** Integrate GWE with the "Slicer3 Chain Module Engine": [[Slicer3:JavaBasedChainCLMsEngine]]
  
 
</div>
 
</div>
Line 44: Line 70:
  
 
*http://www.batchmake.org
 
*http://www.batchmake.org
−
*http://www.gridwizard.org
 
 
*http://www.gridwizardenterprise.org
 
*http://www.gridwizardenterprise.org

Latest revision as of 23:13, 26 June 2008

Home < 2008 Summer Project Week:Batch Processing
Return to Project Week Main Page



Key Investigators

  • Julien Jomier
  • Steve Pieper
  • Marco Ruiz
  • Jim Miller
  • Chris Churas
  • Hans Johnson
  • Killian Pohl
  • Jeff Grethe
  • Mark Anderson

Objective

Implement tests of batch processing techniques using slicer3 command line modules (EMSegmenter, Registration).

Approach, Plan

BatchMake and GWE are both ready to apply to sample datasets using distributed grid/cluster computing. This week we will flesh out the use cases and run actual jobs.

Also, review the concepts and issues documented on the Large Scale Experiment Brainstorming page.

Progress

  • Identified separation between GWE and BatchMake
    • GWE: Running Slicer modules on parameter space from within slicer, console utility or scripts.
    • BatchMake: Scripting using Slicer3 modules (outside of Slicer3 GUI) and reporting
  • Cases
    • 175 cases processed with BSplineDeformableRegistration on the SPL cluster using GWE.
    • 2275 cases processed with BSplineDeformableRegistration on the BIRN cluster using GWE.
    • 16 cases processed with FreeSurfer on the BIRN cluster using GWE.
    • Working out some Condor issues on the SPL cluster.
    • Prepared commands to process 50 cases with Killian's pipeline on the SPL cluster using GWE.
    • Reviewed Sylvain's cases requirements to verify they could be processed in a cluster using GWE.
  • Future Work:
    • BatchMake
      • Implement support to run through GWE
      • Provide reporting tool as slicer execution modules
    • GWE
    • GSlicer3
      • Auto-deploy Slicer3 in clusters where module invocations will be parallelize.
      • Real time monitor interface to track status of requests submitted to clusters.
      • Integrate GWE with the "Slicer3 Chain Module Engine": Slicer3:JavaBasedChainCLMsEngine


References

Retrieved from "https://www.na-mic.org/w/index.php?title=2008_Summer_Project_Week:Batch_Processing&oldid=27640"