Slicer3:Remote Data Handling - Revision history

Grundlett: Text replacement - "http://www.slicer.org/slicerWiki/index.php/" to "https://www.slicer.org/wiki/"

2017-07-10T18:07:32Z

Text replacement - "http://www.slicer.org/slicerWiki/index.php/" to "https://www.slicer.org/wiki/"

Marianna: Unprotected "Slicer3:Remote Data Handling"

2008-05-19T22:46:16Z

Unprotected "Slicer3:Remote Data Handling"

Marianna: Replacing page with ''''Note:''' We are migrating this content to the slicer.org domain - The newer page is [http://www.slicer.org/slicerWiki/index.php/Slicer3:Remote_Data...'

2008-05-19T22:46:05Z

Replacing page with '<big>'''Note:''' We are migrating this content to the slicer.org domain - <font color="orange">The newer page is [http://www.slicer.org/slicerWiki/index.php/Slicer3:Remote_Data...'

Show changes

Wjp: /* Slicer's Data I/O Manager */

2008-05-08T14:43:33Z

Slicer's Data I/O Manager

Wjp: /* XNAT questions */

2008-05-08T14:42:42Z

XNAT questions

Wjp: /* XNAT questions */

2008-05-08T14:42:25Z

XNAT questions

Wjp: /* vtkDataIOManagerLogic (Base/Logic) */

2008-05-08T14:39:41Z

vtkDataIOManagerLogic (Base/Logic)

Wjp: /* vtkDataIOManager (MRML) */

2008-05-08T14:39:19Z

vtkDataIOManager (MRML)

Wjp: /* vtkCacheManager class */

2008-05-08T14:38:07Z

vtkCacheManager class

Wjp: /* pseudocode & notes worked out in 2/14/08 meeting */

2008-05-08T14:37:41Z

pseudocode & notes worked out in 2/14/08 meeting

← Older revision		Revision as of 18:07, 10 July 2017
Line 1:		Line 1:
−	<big>'''Note:''' We are migrating this content to the slicer.org domain - <font color="orange">The newer page is [~~http~~://www.slicer.org/~~slicerWiki/index.php~~/Slicer3:Remote_Data_Handling here]</font></big>	+	<big>'''Note:''' We are migrating this content to the slicer.org domain - <font color="orange">The newer page is [https://www.slicer.org/wiki/Slicer3:Remote_Data_Handling here]</font></big>

← Older revision		Revision as of 14:43, 8 May 2008
Line 183:		Line 183:

	[[ Slicer3:XCEDE_use_cases \| For more info, see this page for more discussion of QueryAtlas's current use of Xcede catalogs, and assumptions... ]]		[[ Slicer3:XCEDE_use_cases \| For more info, see this page for more discussion of QueryAtlas's current use of Xcede catalogs, and assumptions... ]]
−
−	~~= Slicer's Data I/O Manager =~~
−
−	~~[[image:DataIOManager.png]]~~

← Older revision		Revision as of 14:42, 8 May 2008
Line 58:		Line 58:

	= XNAT questions =		= XNAT questions =
−
−	~~[[image:XNATSlicer.png]]~~

	* when we upload a file, can we get back a URI that includes ticket information? (we can use this to write a MRML file)		* when we upload a file, can we get back a URI that includes ticket information? (we can use this to write a MRML file)

@@ Line 61: / Line 61: @@
 [[image:XNATSlicer.png]]
-==Questions:==
+* when we upload a file, can we get back a URI that includes ticket information? (we can use this to write a MRML file)
+* can XNAT-generated uris end with the filename.extension, so that firefox will know to save them to disk, rather than to interpret them as html?
 = Implementation: new classes =

← Older revision		Revision as of 14:39, 8 May 2008
Line 158:		Line 158:


−	~~== vtkDataIOManagerLogic (Base/Logic) ==~~

−	~~In '''vtkDataIOManagerLogic''' class:~~
−
−	This class either does the work of queueing data transfers for asynchronous download/upload, or performs download/upload in a blocking fashion depending on the state of the DataIOManager's EnableAsynchronousIO flag. It uses the ApplicationLogic's scheduling mechanisms, and spawns a new thread for each requested asynchronous data transfer.
−
−	~~vtkDataIOManagerLogic::ProcessMRMLEvents ( vtkObject* caller, unsigned long event, void *calldata)~~
−	{
−	~~if ( this->DataIOManager == vtkDataIOManager::SafeDownCast ( caller ) )~~
−	{
−	~~//--- unpack the calldata into a vtkMRMLNode *node...;~~
−	~~if ( event == vtkDataIOManager::RemoteReadEvent )~~
−	{
−	~~this->QueueRead ( node );~~
−	}
−	~~else if ( event == vtkDataIOManager::RemoteWriteEvent )~~
−	{
−	~~this->QueueWrite ( node );~~
−	}
−	}
−	}
−
−	~~vtkDataIOManagerLogic::QueueRead ( vtkMRMLNode *node)~~
−	{
−	~~//--- do some null checking on node,~~
−	~~...~~
−	~~//--- safedowncast the node into a vtkMRMLDisplayableNode *dnode~~
−	~~...~~
−	~~vtkURIHandler *handler = dnode->GetStorageNode()->GetURIHandler();~~
−	~~//--- null check~~
−	~~...~~
−	~~const char *source = dnode->GetStorageNode()->GetURI();~~
−	~~const char *dest = this->GetDataIOManager()->GetCacheManager()->GetFilenameFromURI(source);~~
−	~~//---~~
−	~~//--- create a new data transfer and stuff it full of all the information associated~~
−	~~vtkDataTransfer *transfer = this->GetDataIOManager()->AddNewDataTransfer ( node );~~
−	~~//--- maybe move the above method into the logic class?~~
−	~~if ( transfer == NULL )~~
−	{
−	~~return 0;~~
−	}
−	~~transfer->SetSourceURI(source);~~
−	~~transfer-SetDestinationURI (dest );~~
−	~~transfer->SetHandler(handler);~~
−	~~transfer->SetTransferType ( vtkDataTransfer::RemoteDownload);~~
−	~~transfer->SetStatus ( vtkDataTransfer::Unspecified );~~
−	~~//---~~
−	~~//--- if the transfer is ASYNCHRONOUS, Schedule it into~~
−	~~//--- the ApplicationLogic's task processing pipeline...~~
−	~~//---~~
−	~~if ( this->GetDataIOManager()->GetEnableAsynchronousIO() )~~
−	{
−	~~vtkSlicertask *task = vtkSlicerTask::New();~~
−	~~task->SetTaskFunction( this, (vtkSlicerTask::TaskFunctionPointer)~~
−	~~&vtkDataIOManagerLogic::ApplyTransfer, transfer);~~
−	~~bool ret=0;~~
−	~~if ( ret = this->GetApplicationLogic()->ScheduleTask(task))~~
−	{
−	~~transfer->SetTransferStatus ( vtkDataTransfer::Scheduled );~~
−	}
−	~~task->Delete();~~
−	~~if (!ret )~~
−	{
−	~~return 0;~~
−	}
−	}
−	~~//---~~
−	~~//--- if the transfer is SYNCHRONOUS... just execute it~~
−	~~//---~~
−	~~else~~
−	{
−	~~this->ApplyTransfer ( transfer );~~
−	}
−	~~return 1;~~
−	}
−	~~...~~
−
−	}
−
−	~~And the '''ApplyTransfer()''' method that gets either scheduled or called synchronously, which contains all the URI handling:~~
−
−	~~vtkDataIOManagerLogic::ApplyTransfer ( void *clientdata )~~
−	{
−	~~//--- cast the clientdata into a vtkDataTransfer~~
−	~~vtkDataTransfer dt = reinterpret_cast <vtkDataTransfer> (clientdata);~~
−	~~if ( dt == NULL ) return;~~
−	~~//---~~
−	~~//--- get the node~~
−	~~//---~~
−	~~vtkMRMLNode *node = this->GetMRMLScene()->GetNodeByID((dt->GetTransferNodeID()));~~
−	~~if ( node == NULL ) return;~~
−	~~//---~~
−	~~//--- download data~~
−	~~//---~~
−	~~const char *source = dt->GetSourceURI();~~
−	~~const char *dest = dt->GetDestinationURI();~~
−	~~if ( dt->GetTransferType() == vtkDataTransfer::RemoteDownload )~~
−	{
−	~~vtkURIHandler *handler = dt->GetHandler();~~
−	~~if ( this->GetDataIOManager()->GetEnableAsynchronousIO())~~
−	{
−	~~//---~~
−	~~//--- THIS IS BEING EXECUTED IN A SEPARATE THREAD.~~
−	~~//---~~
−	~~handler->StageFileRead(source, dest);~~
−	~~this->GetApplicationLogic()->RequestReadData(node->GetID(), dest, 0, 0);~~
−	}
−	~~else~~
−	{
−	~~//---~~
−	~~//--- THIS IS BEING EXECUTED IN THE MAIN THREAD.~~
−	~~//---~~
−	~~handler->StageFileRead (source, dest);~~
−	}
−	}
−	~~...~~
−	}
−	~~...~~
−	}

	= ITK-based mechanism handling remote data (for command line modules, batch processing, and grid processing) (Nicole) =		= ITK-based mechanism handling remote data (for command line modules, batch processing, and grid processing) (Nicole) =

← Older revision	Revision as of 22:46, 19 May 2008
(No difference)

← Older revision		Revision as of 14:39, 8 May 2008
Line 156:		Line 156:


−	~~== vtkDataIOManager (MRML) ==~~

−	~~In '''vtkDataIOManager''' class:~~
−
−	~~This class is basically a container for all data transfers and provides~~
−	* some convenience methods for accessing data transfers and their info
−	* and Get/Set access to manage the Synchronous versus Asynchronous data IO state.
−	* Invokes a RemoteReadEvent, or RemoteWriteEvent with the node as calldata, which the vtkDataIOManagerLogic is listening for and will process.
−
−	~~vtkDataIOManager::QueueRead ( vtkMRMLNode *node )~~
−	{
−	~~//--- creates a vtkDataTransferObject, sets values, adds to collection~~
−	~~this->AddDataTransfer( node );~~
−	~~//--- if ForceReUpload flag is set:~~
−	~~this->DeleteFileInCache ( );~~
−	~~handler = node->GetStorageNode()->GetURIHandler();~~
−	~~//--- Invokes a vtkDataIOManager::RemoteReadEvent and sends the node as calldata~~
−	~~this->InvokeEvent ( vtkDataIOManager::RemoteReadEvent, node );~~
−	}

	== vtkDataIOManagerLogic (Base/Logic) ==		== vtkDataIOManagerLogic (Base/Logic) ==

← Older revision		Revision as of 14:38, 8 May 2008
Line 154:		Line 154:


−	~~== vtkCacheManager class ==~~

−	~~vtkCachedFileCollection = vtkCollection::New();~~

	== vtkDataIOManager (MRML) ==		== vtkDataIOManager (MRML) ==

@@ Line 152: / Line 152: @@
 * Read/write of individual datasets referenced in the scene should work with or without the asynchronous read/write turned on, and with or without the dataIOManager GUI interface.
 == vtkCacheManager class ==
@@ Line 465: / Line 297: @@
    ...
+  }
 = ITK-based mechanism handling remote data (for command line modules, batch processing, and grid processing) (Nicole) =