Projects:QIN:3D Slicer Annotation Image Markup

Scope of Work

Annotation and Image Markup (AIM) project provides the foundation for enabling quantitative analysis of the results produced by the software tools by establishing the methodology to organize and describe the various representation of anatomical entities together with the semantic content and the image data. Unfortunately, the support of AIM in the key medical imaging research tools is currently lacking.

3D Slicer is a multi-platform free and open source software for visualization and medical image computing. NIH and NCI are major sponsors. 3D Slicer is currently central to the QIN grant activity at several of the QIN network sites and in the broader community. 3D Slicer currently includes support for rich set of annotations that can be created using 3D Slicer to support quantitative image analysis. However, these annotations are currently stored in a non-AIM format.

This activity will implement support of AIM in 3D Slicer, including storage of annotations produced by 3D Slicer in AIM format and importing AIM annotations into 3D Slicer. As a result, we will enable standardized storage and access to the results of quantitative analysis produced by the networked QIN grantees for improved analysis and biomarker validation based on the specific requirements and priorities determined by the QIN community. The driving set of use annotation/markup cases from QIN community is available here: Projects:QIN:3D_Slicer_Annotation_Image_Markup:Use cases.

Research Plan

Our implementation plan will be driven by the use-cases provided by the QIN community.

First, we will collect a collection of detailed use cases that utilize annotations and/or AIM. These use cases will provide specific examples to drive and test our implementation.

Second, based on the defined use cases, we will develop AIM import capability in 3D Slicer so that the annotations created using other tools (e.g., ClearCanvas and EPAD) can be loaded and displayed in 3D Slicer.

Third, we will implement functionality to save the annotations created in 3D Slicer into AIM format.

The compatibility of the implementation will be tested using the QIN-defined use cases and the existing tools that support AIM functionality.

Funding

Supplement to U01CA151261 (NCI, PI Fiona Fennessy)

Key Personnel

• 0% Fiona Fennessy (PI)
• 0% Steve Pieper (NAC Collaboration consultant)
• 0% Ron Kikinis (NAC Collaboration consultant)
• 15% Andrey Fedorov (12/01/2011-07/31/2012)
• 50% Nicole Aucoin (12/01/2011-07/31/2012)

Progress

• Jan 5: experiments with syngo.via, see Projects:QIN:3D Slicer Annotation Image Markup:DICOM-based annotations
• Dec 22: planning meeting (Nicole, Andrey): requirements, design, implementation strategy discussed Projects:QIN:3D Slicer Annotation Image Markup:Design and Implementation
• Dec 15: planning meeting (Steve, Nicole, Andrey): specific task formulated: add support for linking DICOM image UIDs to the Slicer image volumes
• Dec 8: Planning meeting with Steve Pieper.
  • Discussed relation bw DICOM SR and AIM
  • demo of annotation capabilities of ClearCanvas, reporting template
  • discussed currently available QIN use cases (NCI TCGA, Stanford, MGH, Iowa).
  • Tentative implementation plan: support DICOM SR import into Slicer (the limited subset of DICOM SR that covers the QIN use cases: measurement, polyline (?)). Add functionality to establish correspondence between slice as it is presented in Slicer and the DICOM image UID. Advantages of DICOM SR over AIM: this is a standard, libraries to interface are available (DCMTK), converter between DICOM SR and AIM objects is provided by Pat Mongkolwat team (AIMConverter).
  • finalized personnel and effort for the project duration
• Dec 2: RSNA2011: meeting with Pat Mongolwat, Vlad Kleper, Larry Tarbox. Discussed C++ API for AIM v.3, currently available on Windows. ClearCanvas can save annotations in either DICOM SR or AIM. AIM can be converted into DICOM SR using a standalone tool distributed with AIM API Windows libraries. DICOM SR can be loaded from file. Discussed ideas for implementation:

1. use C++ AIM API (this is Win only for now)
2. write XML directly (may not be compatible with other AIM versions)
3. convert MRML into AIM XML (there is no MRML schema right now)

• Nov 25: ClearCanvas workstation v 3.0.3 was installed and tested. Justin Kirby (NCI) provided first complete use case (breast MRI bi-dimensional tumor measurement). ClearCanvas can save annotations locally on disk, and can push to the AIM Data Service server, but cannot load them from file. See Projects:QIN:3D Slicer Annotation Image Markup:Existing AIM-compatible tools

• Nov 7: AIM API TCON with the QIN participants and BWH team.

Project-related wiki pages

• Design and implementation discussion
• Summary/evaluation of DICOM-based annotation tools
• Summary/evaluation of AIM-compatible tools
• Motivating QIN use-cases

Resources

Web

• NCI QIN Wiki https://wiki.nci.nih.gov/display/CIP/QIN
• caBIG AIM portal https://cabig.nci.nih.gov/tools/AIM
• 3D Slicer Annotations Projects:ARRA:miAnnotation
• DICOM Structured Reporting by David Clunie at Google Books
• Frontiers in PACS: DICOM Structured Reporting (slides by D.Clunie) URL
• DICOM Supplement 23: Structured Reporting Storage SOP Class URL
• DICOM Supplement 111: Segmentation Storage SOP Class URL
• NCI Wiki AIM Documentation https://wiki.nci.nih.gov/display/AIM/AIM+Documentation
• http://qibawiki.rsna.org/index.php?title=Segmentation_and_Markup_Formats
• caBIG Imaging Knowledge Center discussion boards
• The DICOM Standard all docs
• DICOM part 17: Explanatory information
• RSNA Radiology Reporting initiative

Bibliography

• Rubin, D. L., Mongkolwat, P., Kleper, V., Supekar, K., & Channin, D. S. (2009). Annotation and Image Markup: Accessing and Interoperating with the Semantic Content in Medical Imaging. IEEE Intelligent Systems, 24(1), 57-65. doi:10.1109/MIS.2009.3 IEEE Explore
• Channin, D. S., Mongkolwat, P., Kleper, V., Sepukar, K., & Rubin, D. L. (2010). The caBIG annotation and image Markup project. Journal of digital imaging : the official journal of the Society for Computer Applications in Radiology, 23(2), 217-25. doi:10.1007/s10278-009-9193-9 Pubmed
• Clunie, D. A. (2007). DICOM Structured Reporting and Cancer Clinical Trials Results. Cancer Informatics, 4, 33-56. Libertas Academica. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/21469002
• Hussein, R., Engelmann, U., Schroeter, A., & Meinzer, H.-P. (2004). DICOM structured reporting: Part 1. Overview and characteristics. Radiographics : a review publication of the Radiological Society of North America, Inc, 24(3), 891-6. doi:10.1148/rg.243035710 http://www.ncbi.nlm.nih.gov/pubmed/15143238
• Hussein, R., Engelmann, U., Schroeter, A., & Meinzer, H.-P. (2004). DICOM structured reporting: Part 2. Problems and challenges in implementation for PACS workstations. Radiographics : a review publication of the Radiological Society of North America, Inc, 24(3), 897-909. doi:10.1148/rg.243035722 http://www.ncbi.nlm.nih.gov/pubmed/15143239

AIM Q&A

What are the advantages of AIM over DICOM SR?

• Pat Mongkolwat:

AIM is an information model. Information that AIM represents can be stored as AIM DICOM SR objects, AIM XML documents and HL7 CDA (available in AIM 1.0 only for now). The information in the three formats are virtually the same.

AIM DICOM SR is suitable for an environment whereby the usage of DICOM technology is primary source of image management e.g. PACS. The ClearCanvas implementation works well in the PACS environment. If you retrieve a study from a PACS or DICOM store SCP (with DICOM query/retrieve capability) and you create AIM for the study, the workstation will store AIM DICOM SR you crated to the PACS or DICOM store SCP as well as to the local DICOM storage on the workstation. The workstation also creates and stores AIM XML documents. The workstation cannot import AIM XML currently. We are going to add this capability in the future.

ClearCanvas user's manual can be found at https://wiki.nci.nih.gov/pages/viewpage.action?pageId=40599934

AIM XML document is good for people who are not familiar with DICOM technology. HL7 CDA is used in a larger context of medical information systems. These systems are mostly not aware of DICOM technology. So, if you want to pass AIM information to other medical systems, HL7 CDA is the format.

See the below web sites for further information.

https://wiki.nci.nih.gov/display/ImagingKC/Imaging+Knowledge+Center https://wiki.nci.nih.gov/display/AIM/Annotation+Imaging+Markup+%28AIM%29

• Daniel Rubin:

DICOM SR is a larger spec than AIM, focused generically on the world of non-image data. A variety of templates for SR have been created for a variety of purposes (it's use case driven). A template has even been created for serializing AIM..

AIM is a semantic model of the "results" of images (ROIs, observations, calculations, anatomy, etc). It includes an XML serialization, but as mentioned, an SR serialization is also provided in the AIM toolkit (in a tool called ANIVATR). However, extracting info from SR is an exercise for the SR user.. no tools for that are yet available. All that said, the main advantage of AIM is ease of use for developers to create/use image metadata--a variety of tools and some vendors support it, and recently we have created AIM API to make it easier for developers to create AIM XML. SR is part of the DICOM standard, so that's an advantage, and SR can be stored in a dicom database if vendors support SR (few do at present)