Slicer3:Pipeline Integration - Revision history

Grundlett: Text replacement - "http://www.slicer.org/slicerWiki/index.php/" to "https://www.slicer.org/wiki/"

2017-07-10T18:02:39Z

Text replacement - "http://www.slicer.org/slicerWiki/index.php/" to "https://www.slicer.org/wiki/"

Marianna: Unprotected "Slicer3:Pipeline Integration"

2008-05-19T22:20:35Z

Unprotected "Slicer3:Pipeline Integration"

Marianna: Replacing page with ''''Note:''' We are migrating this content to the slicer.org domain - The newer page is [http://www.slicer.org/slicerWiki/index.php/Slicer3:Pipeline_In...'

2008-05-19T22:20:22Z

Replacing page with '<big>'''Note:''' We are migrating this content to the slicer.org domain - <font color="orange">The newer page is [http://www.slicer.org/slicerWiki/index.php/Slicer3:Pipeline_In...'

Apayan at 20:42, 4 February 2008

2008-02-04T20:42:54Z

Jags at 04:00, 26 January 2007

2007-01-26T04:00:55Z

Jags at 04:00, 26 January 2007

2007-01-26T04:00:36Z

Jags at 04:00, 26 January 2007

2007-01-26T04:00:10Z

Jags at 03:59, 26 January 2007

2007-01-26T03:59:51Z

Andy: Update from Wiki

2006-12-18T13:42:12Z

Update from Wiki

Show changes

Andy: Update from Wiki

2006-12-18T13:42:11Z

Update from Wiki

New page

# The main features of the project
## http://pipeline.loni.ucla.edu
# The main outstanding questions that must be answered before you can complete the project (or even start)
## Pipeline -> Slicer3
### Once the API for submission to Slicer3 is released, we can integrate it into the Pipeline.
## Slicer3 -> Pipeline
### Current Status
#### We have some code that allows any external application to start an instance of Pipeline and submit workflows.
#### Implement a JSON Interface to the pipeline that allows external applications to get a JSON parametric-description of submitted pipelines
#### Example usage: java -jar pipeline.jar -json MultiModalityRigidRegistration.module - The list of parameters in JSON format -

{"parameters":[
{
"phantom":"false",
"optional":"false",
"connection-type":"",
"class-type":"Command Line Option",
"name":"ResolutionLevel"
},
{
"phantom":"false",
"optional":"false",
"connection-type":"",
"class-type":"Command Line Option",
"name":"QualityLevel"
},
{
"phantom":"false",
"optional":"false",
"connection-type":"input",
"class-type":"File",
"name":"FixedImage"
},
{
"phantom":"false",
"optional":"false",
"connection-type":"output",
"class-type":"File",
"name":"OutputTransform"
},
{
"phantom":"false",
"optional":"false",
"connection-type":"input",
"class-type":"File",
"name":"MovingImage"
},
{
"phantom":"false",
"optional":"false",
"connection-type":"input",
"class-type":"File",
"name":"InputTransform"
}
]
}

# This JSON description can be then used to bind values to parameters and then submit the pipeline for

execution -

# The XML representation is given below:

<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" xmlns:owl="http://www.w3.org/2002/07/owl#">
<owl:Ontology>
<owl:versionInfo><![CDATA[20050412]]></owl:versionInfo>
<rdfs:label locale="en" />
<rdfs:comment locale="en" />
</owl:Ontology>
<FunctionPipeline rdf:ID="1118265959469">
<rdfs:label locale="en"><![CDATA[MultiModalityRigidRegistration pipeline function]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[MultiModalityRigidRegistration (Programmers Week)]]></rdfs:comment>
<Assign rdf:ID="11182659594690">
<owl:allValuesFrom rdf:resource="1118265959620" />
<rdfs:label locale="en"><![CDATA[11182659594690]]></rdfs:label>
</Assign>
<GUI>
<rdfs:label locale="en" />
<rdfs:comment locale="en" />
</GUI>
</FunctionPipeline>
<Module rdf:ID="1118265959469:1118265959586" rdf:resource="1118265959469">
<rdfs:label locale="en"><![CDATA[MultiModalityRigidRegistration pipeline]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[MultiModalityRigidRegistration (Programmers Week)]]></rdfs:comment>
<Module rdf:ID="1118265959620:1118265959625" rdf:resource="11182659594690">
<rdfs:label locale="en"><![CDATA[MultiModalityRigidRegistration]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[MultiModalityRigidRegistration (Programmers Week)]]></rdfs:comment>
<GUI x="196" y="171">
<rdfs:label locale="en" />
<rdfs:comment locale="en" />
</GUI>
<Assign rdf:resource="Parameter1118266141102">
<owl:allValuesFrom rdf:resource="File:1118266118686" />
<rdfs:label locale="en" />
</Assign>
<Assign rdf:resource="Parameter1118266282720">
<owl:allValuesFrom rdf:resource="File:1118266266201" />
<rdfs:label locale="en" />
</Assign>
<Assign rdf:resource="Parameter1118266397117">
<owl:allValuesFrom rdf:resource="File:1118266372629" />
<rdfs:label locale="en" />
</Assign>
<Assign rdf:resource="Parameter1118266521616">
<owl:allValuesFrom rdf:resource="CommandLineOption:1118266435955" />
<rdfs:label locale="en" />
</Assign>
<Assign rdf:resource="Parameter1118266541544">
<owl:allValuesFrom rdf:resource="CommandLineOption:1118266461834" />
<rdfs:label locale="en" />
</Assign>
<Assign rdf:resource="Parameter1118266740529">
<owl:allValuesFrom rdf:resource="File:1118266713458" />
<rdfs:label locale="en" />
</Assign>
</Module>
</Module>
<ExecutableFile rdf:ID="1118265959620" url="pipeline://localhost//home/jrajendi/ProgrammersWeek/ItkGridPipeline/testdir/grid-reg/MultiModalityRigidRegistration">
<rdfs:label locale="en"><![CDATA[MultiModalityRigidRegistration function]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[MultiModalityRigidRegistration (Programmers Week)]]></rdfs:comment>
<GUI>
<rdfs:label locale="en" />
<rdfs:comment locale="en" />
</GUI>
<Parameter rdf:ID="Parameter1118266141102" connectionType="input">
<rdfs:label locale="en"><![CDATA[FixedImage]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[Fixed Image parameter]]></rdfs:comment>
<rdf:type rdf:resource="File" />
</Parameter>
<Parameter rdf:ID="Parameter1118266282720" connectionType="input">
<rdfs:label locale="en"><![CDATA[MovingImage]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[moving image parameter]]></rdfs:comment>
<rdf:type rdf:resource="File" />
</Parameter>
<Parameter rdf:ID="Parameter1118266397117" connectionType="input">
<rdfs:label locale="en"><![CDATA[InputTransform]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[input transform parameter]]></rdfs:comment>
<rdf:type rdf:resource="File" />
</Parameter>
<Parameter rdf:ID="Parameter1118266521616" connectionType="">
<rdfs:label locale="en"><![CDATA[ResolutionLevel]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[resolutionlevel parameter]]></rdfs:comment>
<rdf:type rdf:resource="CommandLineOption" />
</Parameter>
<Parameter rdf:ID="Parameter1118266541544" connectionType="">
<rdfs:label locale="en"><![CDATA[QualityLevel]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[Quality level]]></rdfs:comment>
<rdf:type rdf:resource="CommandLineOption" />
</Parameter>
<Parameter rdf:ID="Parameter1118266740529" connectionType="output">
<rdfs:label locale="en"><![CDATA[OutputTransform]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[output transform file]]></rdfs:comment>
<rdf:type rdf:resource="File" />
</Parameter>
</ExecutableFile>
<CommandLineOption rdf:ID="CommandLineOption:1118266044377">
<Values>
<Value>
<string><![CDATA[pipeline://localhost//home/jrajendi/ProgrammersWeek/ItkGridPipeline/testdir/grid-reg/]]></string>
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[Input Directory]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[input directory for images and input configuration files]]></rdfs:comment>
</CommandLineOption>
<CommandLineOption rdf:ID="CommandLineOption:1118266073584">
<Values>
<Value>
<string><![CDATA[reg.nhdr]]></string>
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[Fixed Image Filename]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[reg.nhdr filename]]></rdfs:comment>
</CommandLineOption>
<File rdf:ID="File:1118266118686">
<Values>
<Value>
<string rdf:resource="CommandLineOption:1118266044377" />
<string rdf:resource="CommandLineOption:1118266073584" />
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[FixedImage]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[fixed image variable]]></rdfs:comment>
</File>
<CommandLineOption rdf:ID="CommandLineOption:1118266234482">
<Values>
<Value>
<string><![CDATA[mrt.nhdr]]></string>
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[Moving Image Filename]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[filename for moving image input]]></rdfs:comment>
</CommandLineOption>
<File rdf:ID="File:1118266266201">
<Values>
<Value>
<string rdf:resource="CommandLineOption:1118266044377" />
<string rdf:resource="CommandLineOption:1118266234482" />
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[MovingImage]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[Moving image variable]]></rdfs:comment>
</File>
<CommandLineOption rdf:ID="CommandLineOption:1118266341295">
<Values>
<Value>
<string><![CDATA[xformIN]]></string>
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[InputTransform Filename]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[Input Transformation Filename]]></rdfs:comment>
</CommandLineOption>
<File rdf:ID="File:1118266372629">
<Values>
<Value>
<string rdf:resource="CommandLineOption:1118266044377" />
<string rdf:resource="CommandLineOption:1118266341295" />
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[InputTransform]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[input transformation variable]]></rdfs:comment>
</File>
<CommandLineOption rdf:ID="CommandLineOption:1118266435955">
<Values>
<Value>
<string><![CDATA[2]]></string>
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[ResolutionLevel]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[resolution levels {1, 2, 3}]]></rdfs:comment>
</CommandLineOption>
<CommandLineOption rdf:ID="CommandLineOption:1118266461834">
<Values>
<Value>
<string><![CDATA[1]]></string>
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[QualityLevel]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[quality levels {1,2}]]></rdfs:comment>
</CommandLineOption>
<CommandLineOption rdf:ID="CommandLineOption:1118266576527">
<Values>
<Value>
<string><![CDATA[xformOUT]]></string>
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[OuputTransform Filename]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[output transform filename]]></rdfs:comment>
</CommandLineOption>
<CommandLineOption rdf:ID="CommandLineOption:1118266669340">
<Values>
<Value>
<string><![CDATA[pipeline://localhost//home/jrajendi/ProgrammersWeek/ItkGridPipeline/testdir/grid-reg/refinedmodule/]]></string>
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[Output Directory]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[output directory for outputTransformfile]]></rdfs:comment>
</CommandLineOption>
<File rdf:ID="File:1118266713458">
<Values>
<Value>
<string rdf:resource="CommandLineOption:1118266669340" />
<string rdf:resource="CommandLineOption:1118266576527" />
</Value>
</Values>
<rdfs:label locale="en"><![CDATA[OutputTransform]]></rdfs:label>
<rdfs:comment locale="en"><![CDATA[output transform file]]></rdfs:comment>
</File>
</rdf:RDF>

<br />

# To start a pipeline on the commandline with parameters:

java -jar pipeline.client.jar -batch -module=MultiModalityRigidRegistration.module -parameter:Paramter_1=Value_1 -parameter:Parameter_2=Value_2

'''Team Members'''<nowiki>: </nowiki>

* Michael Pan - UCLA
* Jagadeeswaran Rajendiran - UCLA

← Older revision		Revision as of 18:02, 10 July 2017
Line 1:		Line 1:
−	<big>'''Note:''' We are migrating this content to the slicer.org domain - <font color="orange">The newer page is [~~http~~://www.slicer.org/~~slicerWiki/index.php~~/Slicer3:Pipeline_Integration here]</font></big>	+	<big>'''Note:''' We are migrating this content to the slicer.org domain - <font color="orange">The newer page is [https://www.slicer.org/wiki/Slicer3:Pipeline_Integration here]</font></big>

← Older revision		Revision as of 22:20, 19 May 2008
Line 1:		Line 1:
−	~~LONI Pipeline website: http://pipeline.loni.ucla.edu~~	+	<big>'''Note:''' We are migrating this content to the slicer.org domain - <font color="orange">The newer page is [http://www.slicer.org/slicerWiki/index.php/Slicer3:Pipeline_Integration here]</font></big>
−
−	~~With the release of Pipeline v4.0 in December~~ '~~07, we~~'~~ve developed a stable and robust framework for continuing development into the future. With regards to the Slicer 3 (currently in Beta) integration, the LONI Pipeline has a complete [http~~:~~//cms.loni.ucla.edu/pipeline/default.aspx?id=2134#Running%20from%20the%20command%20line command line interface] that can be used to run workflows with dynamically bound values in addition to it~~'~~s GUI interface.~~
−	~~<code>~~
−	~~Usage: pipeline [OPTIONS] PIPEFILE [paramID1=value ... paramIDN=valueN]~~
−	~~-execute Validates then executes the specified workflow~~
−	~~-validate Just validates the specified workflow then exits~~
−	~~-display Displays all the required/enabled parameters in the~~
−	~~workflow~~
−	~~-sync Program will not exit until workflow finishes executing~~
−	~~-credentials=FILE Specify an XML file containing credentials to a Pipeline~~
−	~~server that will be needed for execution or validation.~~
−	~~This is not necessary if you~~'~~re username and password~~
−	are ~~saved in your list of connections in~~ the ~~graphical~~
−	~~client~~. ~~Format is the following:~~
−	~~<?xml version="1.0" encoding="UTF~~-~~8"?>~~
−	~~<connections>~~
−	<~~conn address~~="~~cranium.loni.ucla.edu"~~
−	~~user="jane"~~
−	~~password=~~"~~secret" /~~>
−	~~</connections>~~
−	~~-help Prints out LONI Pipeline help. (this text)~~
−	~~-version Prints out the LONI Pipeline version~~
−	~~</code>~~
−
−	~~Using information from the~~ [http://~~wiki~~.~~na-mic~~.org/~~Wiki~~/index.php/Slicer3:~~Execution_Model_Documentation#Executable_plugins_with_command_line_options Slicer3 execution model~~] ~~we will add support for the --xml flag to the Pipeline command line so Slicer can use the Pipeline from it's GUI. This will occur after the release of the final version of Slicer3.~~
−
−	~~'''Developers'''~~<~~nowiki~~>: </~~nowiki~~>
−
−	* Celia Cheung
−	* Arash Payan
−	* Jia Wei Tam

@@ Line 1: / Line 1: @@
-# The main features of the project
+LONI Pipeline website: http://pipeline.loni.ucla.edu
- - Completely new data model/xml
+With the release of Pipeline v4.0 in December '07, we've developed a stable and robust framework for continuing development into the future. With regards to the Slicer 3 (currently in Beta) integration, the LONI Pipeline has a complete [http://cms.loni.ucla.edu/pipeline/default.aspx?id=2134#Running%20from%20the%20command%20line command line interface] that can be used to run workflows with dynamically bound values in addition to it's GUI interface.
- - Significantly easier parameter editing
+<code>
- - Cleaner drag and drop of modules
+  Usage: pipeline [OPTIONS] PIPEFILE [paramID1=value ... paramIDN=valueN]
- - Easier list processing
+    -execute            Validates then executes the specified workflow
- - Stronger type checking (with feedback to help the user)
+    -validate           Just validates the specified workflow then exits
- - Variable can be used on parameter bindings
+    -display            Displays all the required/enabled parameters in the
- - Personal Library of workflows and moduels
+                           workflow
- - Copy/Paste of single modules and groups of modules (and still retain connections)
+    -sync               Program will not exit until workflow finishes executing
- - Undo/Redo on most operations
+    -credentials=FILE   Specify an XML file containing credentials to a Pipeline
- - Workflows can be layed out horizontally now
+                           server that will be needed for execution or validation.
- - Faster and easier zooming
+                           This is not necessary if you're username and password
- - Scalable GUI functionality - is able to handle hundreds of modules in a single workflow
+                           are saved in your list of connections in the graphical
-# ## ### #### DataModel : The data model, associated XML descriptions have been simplified greatly. The data model has been decoupled from the execution and the GUI, this component has been completed
+Using information from the  [http://wiki.na-mic.org/Wiki/index.php/Slicer3:Execution_Model_Documentation#Executable_plugins_with_command_line_options Slicer3 execution model] we will add support for the --xml flag to the Pipeline command line so Slicer can use the Pipeline from it's GUI. This will occur after the release of the final version of Slicer3.
- - Agnostic of the data model
+'''Developers'''<nowiki>: </nowiki>
-# To start a pipeline on the commandline with parameters:
+* Celia Cheung
+* Arash Payan
-java -classpath pipeline.client.jar ui.cli.Main -execute prebound_workflow
+* Jia Wei Tam

@@ Line 7: / Line 7: @@
 ### We have made considerable progress in making Pipeline more stable, robust and scalable. Detailed progress is listed below ::
 #### GUI : A new and improved graphical interface is now available. The gui is now much simpler in design and easier to use. The list of improvements/ new features is as follows :
   - Completely new data model/xml
@@ Line 22: / Line 20: @@
   - Faster and easier zooming
   - Scalable GUI functionality - is able to handle hundreds of modules in a single workflow
 # ## ### #### DataModel : The data model, associated XML descriptions have been simplified greatly. The data model has been decoupled from the execution and the GUI, this component has been completed

@@ Line 7: / Line 7: @@
 ### We have made considerable progress in making Pipeline more stable, robust and scalable. Detailed progress is listed below ::
 #### GUI : A new and improved graphical interface is now available. The gui is now much simpler in design and easier to use. The list of improvements/ new features is as follows :
   - Completely new data model/xml
@@ Line 20: / Line 22: @@
   - Faster and easier zooming
   - Scalable GUI functionality - is able to handle hundreds of modules in a single workflow
 # ## ### #### DataModel : The data model, associated XML descriptions have been simplified greatly. The data model has been decoupled from the execution and the GUI, this component has been completed

← Older revision	Revision as of 22:20, 19 May 2008
(No difference)

@@ Line 1: / Line 1: @@
 # The main features of the project
 ## http://pipeline.loni.ucla.edu
-# The main outstanding questions that must be answered before you can complete the project ::
+# The main outstanding questions that must be answered before you can complete the project ::
 ## Pipeline -> Slicer3
 ### Once the API for submission to Slicer3 is released, we can integrate it into the Pipeline.
 ## Slicer3 -> Pipeline
-### We have made considerable progress in making Pipeline more stable, robust and scalable. Detailed progress is listed below ::
+### We have made considerable progress in making Pipeline more stable, robust and scalable. Detailed progress is listed below ::
-#### GUI : A new and improved graphical interface is now available. The gui is now much simpler in design and easier to use. The list of improvements/ new features is as follows :
+#### GUI : A new and improved graphical interface is now available. The gui is now much simpler in design and easier to use. The list of improvements/ new features is as follows :
   - Completely new data model/xml
@@ Line 21: / Line 23: @@
   - Scalable GUI functionality - is able to handle hundreds of modules in a single workflow
-# ## ### #### DataModel : The data model, associated XML descriptions have been simplified greatly. The data model has been decoupled from the execution and the GUI, this component has been completed
+</verbatim?
-#### Translation : The translation portion of the software is responsible for accomodating logic that separates GUI/Execution vs Execution. This component has been completed, it takes the workflow and converts it into a flattened graph (it loses the property of module groups at this stage) and constructs the commands using the data model
-#### Validation : The validation portion conducts sanity checks on the workflow. It allows the user to fix errors prior to "actual" execution and informs the user of all steps needed to make the workflow run
+# ## ### #### DataModel : The data model, associated XML descriptions have been simplified greatly. The data model has been decoupled from the execution and the GUI, this component has been completed
-#### Execution : The execution portion now is an independent component that is agnostic of the data model and can be invoked on the command line directly. Self-sufficient workflows (with previously bound parameters), can be now invoked and executed from the commandline, bypassing the GUI entirely. Workflows or pipes can now be executed both locally and on the LONI grid. The scalability has dramatically improved from 30 inputs to thousands of inputs per workflow. The changes in the execution model are ::
+#### Translation : The translation portion of the software is responsible for accomodating logic that separates GUI/Execution vs Execution. This component has been completed, it takes the workflow and converts it into a flattened graph (it loses the property of module groups at this stage) and constructs the commands using the data model
   - Agnostic of the data model