Data:DownloadInstructions - Revision history

Melonakos at 20:10, 23 June 2008

2008-06-23T20:10:07Z

Melonakos at 20:09, 23 June 2008

2008-06-23T20:09:55Z

Melonakos: New page: == Search for NAMIC Data == Users earch for data available on the BIRN data grid through metadata applied to the data itself. Each data object and collection can be assigned any number o...

2008-06-23T18:41:51Z

New page: == Search for NAMIC Data == Users earch for data available on the BIRN data grid through metadata applied to the data itself. Each data object and collection can be assigned any number o...

New page

== Search for NAMIC Data ==

Users earch for data available on the BIRN data grid through metadata applied to the data itself. Each data object and collection can be assigned any number of metadata tags. Currently the metadata tags consist of items such as: Modality (DTI, T1, T2, etc.), Institution, etc.

To see an example of metadata on a data file, take a look at the following UCI file: First
* Ensure you aare logged into the [https://portal.nbirn.net Portal]
* Then view the [https://portal.nbirn.net:443/gridsphere/gridsphere?cid=srbfilebrowser&gs_action=gotoDirectory&up=21I&21I_dirpath=%2Fhome%2FProjects%2FStudy5000__0005%2FFiles%2FArchive&JavaScript=enabled contents of that archive]
* Then to search for data, click the Query Metadata button [[image:Tango_search_metadata_large.png]]

== Replication of Data ==

Currently, the NAMIC data sets are stored in BIRN at:

/home/Projects/NAMIC__0003/Files (physicall located at NCMIR) <br /> /home/Projects/Study5000__0005/Files (physically located at UCI)

For redundancy in case a Birn Rack were to go offline we have replicated the data to the BWH rack in preparation for the Programmer's Week.

This can be seen when using 'Sls -a'

[jgerk@ncmir-gpop scratch]$ Sls -a UCI07110773.tgz
jgerk 0 z-ucsd-ncmir-nas1 451315092 2005-02-18-14.00 % /home/Projects/Study5000__0005/Files/Archive/UCI07110773.tgz
jgerk 1 z-harvard-bwh-nas1 451315092 2005-06-24-17.02 % /home/Projects/Study5000__0005/Files/Archive/UCI07110773.tgz

During regular use it is not important that the file resides in two places, but it should be understood that removing the file will result in removing both copies.

As more data is added to NAMIC we can periodically run the following command to update the BWH resource (provided you have the correct permissions).

Sbkupsrb -r -S z-harvard-bwh-nas1 Files

== Notes ==

''Instructions on creating Data Repository Pages are [[DataRepositoryNotes|here]].''

* Data Access Uses the BIRN Controlled Infrastructure (requires account)
** [https://portal.nbirn.net:443/gridsphere/gridsphere?cid=navigatorPortletMyProjects&gs_action=selectProject&up=39w&39w_projectID=3&JavaScript=enabled NA-MIC Project Page]

<br /> Namic Domain Creation

'''Please note''' that users who previously belonged to the external domain have been added to the namic domain. If you use the SRB S-commands, this mandates a change to your .MdasEnv file if you wish for your default directory to be /home/<your_user_name>.namic or if your default directory was previously /home/<your_user_name>.external. Please update your .MdasEnv file to point to your new default directory.

= MIDAS =

MIDAS is a public repository of freely available data. New contributions from NAMIC collaborators are welcome.

MIDAS is a system for collecting, processing, and distributing massive collections of data. It is particularly well suited for large images and their meta-data.

MIDAS provides the following services:
* Indexing: MIDAS automatically indexes text in any of over 20 different files types including XML, Microsoft Word(TM) and Adobe PDF(TM) files. Also, it automatically indexes header information from over 20 different image types including MetaImage and DICOM header information.
* Searching: MIDAS supports the OAI data harvesting standard, so that public data is readily discovered by Google and other search engines. Private data is readily searched using a local search engine.
* Storing: MIDAS provides batch upload methods and supports arbitrary file types and meta-data fields so that it can manage all of your raw and processed data as well as your companion reports and summary statistics.
* Processing: Local, server-side, and distributed processing can be automatically initiated on upload of data to MIDAS or initiated using thin or thick clients. Server-side and distributed processing is provided using the BatchMake scripting language.
* Accessing: You can browse large images on MIDAS, without downloading the data; you can download MIDAS's data over the web or directly into your legacy software using SSHFS. SSHFS presents the hierarchical organization of the data on MIDAS as a file system that can be mounted on Windows, Linux, and MacIntosh platforms.

MIDAS has been used to host publications for e-journals, to host and process data for multi-center research projects, and to offer public access to data in compliance with NIH and NSF data sharing policies.

== Slicer Tutorial Data ==

* http://www.insight-journal.org/midas/view_community.php?communityid=17

== Slicer Brain and Abdomen Data ==

* http://www.insight-journal.org/midas/view_collection.php?collectionid=34

@@ Line 1: / Line 1: @@
   Back to [[Data:DBP1|DBP 1 NA-MIC Data]]
 __NOTOC__
 == Search for NAMIC Data ==

← Older revision		Revision as of 20:09, 23 June 2008
Line 1:		Line 1:
		+	Back to [[Data:DBP1\|DBP 1 NA-MIC Data]]
		+	__NOTOC__

	== Search for NAMIC Data ==		== Search for NAMIC Data ==
Line 39:		Line 41:

	'''Please note''' that users who previously belonged to the external domain have been added to the namic domain. If you use the SRB S-commands, this mandates a change to your .MdasEnv file if you wish for your default directory to be /home/<your_user_name>.namic or if your default directory was previously /home/<your_user_name>.external. Please update your .MdasEnv file to point to your new default directory.		'''Please note''' that users who previously belonged to the external domain have been added to the namic domain. If you use the SRB S-commands, this mandates a change to your .MdasEnv file if you wish for your default directory to be /home/<your_user_name>.namic or if your default directory was previously /home/<your_user_name>.external. Please update your .MdasEnv file to point to your new default directory.
−
−	~~= MIDAS =~~
−
−	~~MIDAS is a public repository of freely available data. New contributions from NAMIC collaborators are welcome.~~
−
−	~~MIDAS is a system for collecting, processing, and distributing massive collections of data. It is particularly well suited for large images and their meta-data.~~
−
−	~~MIDAS provides the following services:~~
−	* Indexing: MIDAS automatically indexes text in any of over 20 different files types including XML, Microsoft Word(TM) and Adobe PDF(TM) files. Also, it automatically indexes header information from over 20 different image types including MetaImage and DICOM header information.
−	* Searching: MIDAS supports the OAI data harvesting standard, so that public data is readily discovered by Google and other search engines. Private data is readily searched using a local search engine.
−	* Storing: MIDAS provides batch upload methods and supports arbitrary file types and meta-data fields so that it can manage all of your raw and processed data as well as your companion reports and summary statistics.
−	* Processing: Local, server-side, and distributed processing can be automatically initiated on upload of data to MIDAS or initiated using thin or thick clients. Server-side and distributed processing is provided using the BatchMake scripting language.
−	* Accessing: You can browse large images on MIDAS, without downloading the data; you can download MIDAS's data over the web or directly into your legacy software using SSHFS. SSHFS presents the hierarchical organization of the data on MIDAS as a file system that can be mounted on Windows, Linux, and MacIntosh platforms.
−
−	MIDAS has been used to host publications for e-journals, to host and process data for multi-center research projects, and to offer public access to data in compliance with NIH and NSF data sharing policies.
−
−	~~== Slicer Tutorial Data ==~~
−
−	* http://www.insight-journal.org/midas/view_community.php?communityid=17
−
−	~~== Slicer Brain and Abdomen Data ==~~
−
−	* http://www.insight-journal.org/midas/view_collection.php?collectionid=34