Projects:QIN:3D Slicer Annotation Image Markup

Scope of Work

Annotation and Image Markup (AIM) project provides the foundation for enabling quantitative analysis of the results produced by the software tools by establishing the methodology to organize and describe the various representation of anatomical entities together with the semantic content and the image data. Unfortunately, the support of AIM in the key medical imaging research tools is currently lacking.

3D Slicer is a multi-platform free and open source software for visualization and medical image computing. NIH and NCI are major sponsors. 3D Slicer is currently central to the QIN grant activity at several of the QIN network sites and in the broader community. 3D Slicer currently includes support for rich set of annotations that can be created using 3D Slicer to support quantitative image analysis. However, these annotations are currently stored in a non-AIM format.

This activity will implement support of AIM in 3D Slicer, including storage of annotations produced by 3D Slicer in AIM format and importing AIM annotations into 3D Slicer. As a result, we will enable standardized storage and access to the results of quantitative analysis produced by the networked QIN grantees for improved analysis and biomarker validation based on the specific requirements and priorities determined by the QIN community. The driving set of use annotation/markup cases from QIN community is available here: Projects:QIN:3D_Slicer_Annotation_Image_Markup:Use cases.

Research Plan

Our implementation plan will be driven by the use-cases provided by the QIN community.

First, we will collect a collection of detailed use cases that utilize annotations and/or AIM. These use cases will provide specific examples to drive and test our implementation.

Second, based on the defined use cases, we will develop AIM import capability in 3D Slicer so that the annotations created using other tools (e.g., ClearCanvas and EPAD) can be loaded and displayed in 3D Slicer.

Third, we will implement functionality to save the annotations created in 3D Slicer into AIM format.

The compatibility of the implementation will be tested using the QIN-defined use cases and the existing tools that support AIM functionality.

Funding

Supplement to U01CA151261 (NCI, PI Fiona Fennessy)

Key Personnel

• 0% Fiona Fennessy (PI)
• 0% Steve Pieper (NAC Collaboration consultant)
• 0% Ron Kikinis (NAC Collaboration consultant)
• 15% Andrey Fedorov (12/01/2011-07/31/2012)
• 50% Nicole Aucoin (12/01/2011-07/31/2012)
• 0% David Clunie (DICOM expertise)
• 0% Neha Agrawal (assistance in SNOMED Standard terminology mapping for Slicer GeneralAnatomyLabels)

Progress

• Jan - now: Implementation work, see details Projects:QIN:3D_Slicer_Annotation_Image_Markup:Design_and_Implementation
• Jan 9-13: NA-MIC Project week discussion with NA-MIC, SparKit, QIN community (represented by BWH and MGH) on AIM support architecture and implementation in 3D Slicer (see meeting notes and presentation slides here: 2012_Winter_Project_Week_DICOM_RT_Breakout)
• Jan 5: experiments with syngo.via, see Projects:QIN:3D Slicer Annotation Image Markup:DICOM-based annotations
• Dec 22: planning meeting (Nicole, Andrey): requirements, design, implementation strategy discussed Projects:QIN:3D Slicer Annotation Image Markup:Design and Implementation
• Dec 15: planning meeting (Steve, Nicole, Andrey): specific task formulated: add support for linking DICOM image UIDs to the Slicer image volumes
• Dec 8: Planning meeting with Steve Pieper.
  • Discussed relation bw DICOM SR and AIM
  • demo of annotation capabilities of ClearCanvas, reporting template
  • discussed currently available QIN use cases (NCI TCGA, Stanford, MGH, Iowa).
  • Tentative implementation plan: support DICOM SR import into Slicer (the limited subset of DICOM SR that covers the QIN use cases: measurement, polyline (?)). Add functionality to establish correspondence between slice as it is presented in Slicer and the DICOM image UID. Advantages of DICOM SR over AIM: this is a standard, libraries to interface are available (DCMTK), converter between DICOM SR and AIM objects is provided by Pat Mongkolwat team (AIMConverter).
  • finalized personnel and effort for the project duration
• Dec 2: RSNA2011: meeting with Pat Mongolwat, Vlad Kleper, Larry Tarbox. Discussed C++ API for AIM v.3, currently available on Windows. ClearCanvas can save annotations in either DICOM SR or AIM. AIM can be converted into DICOM SR using a standalone tool distributed with AIM API Windows libraries. DICOM SR can be loaded from file. Discussed ideas for implementation:

1. use C++ AIM API (this is Win only for now)
2. write XML directly (may not be compatible with other AIM versions)
3. convert MRML into AIM XML (there is no MRML schema right now)

• Nov 25: ClearCanvas workstation v 3.0.3 was installed and tested. Justin Kirby (NCI) provided first complete use case (breast MRI bi-dimensional tumor measurement). ClearCanvas can save annotations locally on disk, and can push to the AIM Data Service server, but cannot load them from file. See Projects:QIN:3D Slicer Annotation Image Markup:Existing AIM-compatible tools

• Nov 7: AIM API TCON with the QIN participants and BWH team.

• Dec 3 2015: DICOM Standards Committee (DSC) approves work item to develop mapping to convert AIM 4.0 annotations to equivalent DICOM TID 1500 SR documents (rather than literal transcoding of AIM UML classes) ... allows convergence with QIICR extention of Slicer to create/use TID 1500 SR objects

Project-related links

• Source code on github: https://github.com/fedorov/Reporting
• Design and implementation
• DICOM SR related materials
• AIM related materials
• Motivating QIN use-cases
• Standard Terminology

Resources

Web

• NCI QIN Wiki https://wiki.nci.nih.gov/display/CIP/QIN
• 3D Slicer Annotations Projects:ARRA:miAnnotation
• http://qibawiki.rsna.org/index.php?title=Segmentation_and_Markup_Formats

DICOM related

• DICOM for quantitative imaging biomarker development: A standards based approach to sharing of clinical data and structured PET/CT analysis results in head and neck cancer research
• Proposal for New Work Item from WG-08 Transcoding of NCI Annotation and Image Markup (AIM) and DICOM-SR Measurement Templates
• DICOM Structured Reporting by David Clunie at Google Books
• Frontiers in PACS: DICOM Structured Reporting (slides by D.Clunie) URL
• DICOM Supplement 23: Structured Reporting Storage SOP Class - Final text PDF - DO NOT USE - SEE CURRENT STANDARD)
• DICOM Supplement 111: Segmentation Storage SOP Class URL - DO NOT USE - SEE CURRENT STANDARD
• The DICOM Standard all docs
• Index of DICOM Standard, Supplements and Corrections

• DICOM PS3.3 Information Object Definitions - Structured Report Document Information Object Definitions HTML PDF
• DICOM PS3.16 Content Mapping Resource - Structured Reporting Templates HTML PDF
• DICOM PS3.17 Explanatory Information HTML PDF
• RSNA Radiology Reporting initiative
• dicom-sr-qi project (Google Code)
• DICOM standard in HTML form
• UMLS terminology services (SNOMED CT browser)
• Slicer3 GenericAnatomyColors LUT
• DICOM Lookup (HTML DICOM, search, tables)

XML related

• XSD http://searchsoa.techtarget.com/definition/XSD
• CodeSynthesis wiki http://wiki.codesynthesis.com/Main_Page (commercial, free license for code under 10k lines)
• Qt 4.7 QtXml module
• Qt 4.7 QXmlPatterns module
• XSD: XML Data binding for Qt (discussion thread)
• Python and XML: summary of existing solutions and tools
• Komodo Edit: free cross-platform XML/XSD editor http://www.activestate.com/komodo-edit

AIM related

• caBIG AIM portal https://cabig.nci.nih.gov/tools/AIM
• NCI Wiki AIM Documentation https://wiki.nci.nih.gov/display/AIM/AIM+Documentation
• caBIG Imaging Knowledge Center discussion boards
• https://cdebrowser.nci.nih.gov/CDEBrowser/ -- definitions of attributes used in AIM (see caBIG->Classifications->AIM 3.0 in the left-side tree hierarchy)
• https://wiki.nci.nih.gov/display/AIM/AIM+on+ClearCanvas+Workstation+3.0.4+and+3.0.4.x+Release+Notes

Bibliography

• Rubin, D. L., Mongkolwat, P., Kleper, V., Supekar, K., & Channin, D. S. (2009). Annotation and Image Markup: Accessing and Interoperating with the Semantic Content in Medical Imaging. IEEE Intelligent Systems, 24(1), 57-65. doi:10.1109/MIS.2009.3 IEEE Explore
• Channin, D. S., Mongkolwat, P., Kleper, V., Sepukar, K., & Rubin, D. L. (2010). The caBIG annotation and image Markup project. Journal of digital imaging : the official journal of the Society for Computer Applications in Radiology, 23(2), 217-25. doi:10.1007/s10278-009-9193-9 Pubmed
• Clunie, D. A. (2007). DICOM Structured Reporting and Cancer Clinical Trials Results. Cancer Informatics, 4, 33-56. Libertas Academica. Retrieved from http://www.la-press.com/dicom-structured-reporting-and-cancer-clinical-trials-results-article-a240
• Hussein, R., Engelmann, U., Schroeter, A., & Meinzer, H.-P. (2004). DICOM structured reporting: Part 1. Overview and characteristics. Radiographics : a review publication of the Radiological Society of North America, Inc, 24(3), 891-6. doi:10.1148/rg.243035710 http://www.ncbi.nlm.nih.gov/pubmed/15143238
• Hussein, R., Engelmann, U., Schroeter, A., & Meinzer, H.-P. (2004). DICOM structured reporting: Part 2. Problems and challenges in implementation for PACS workstations. Radiographics : a review publication of the Radiological Society of North America, Inc, 24(3), 897-909. doi:10.1148/rg.243035722 http://www.ncbi.nlm.nih.gov/pubmed/15143239