Andy: Update from Wiki

2006-12-18T13:23:52Z

Update from Wiki

New page

[[IGT:October_2006_IGT_Workshop#Agenda|Back to workshop agenda]]

'''Panel Chair: Jim Duncan'''

Panelists: Russ Taylor, Steve Pizer, and Presenters.

== Guidelines ==

Guidelines for panel discussion:

# Questions from the audience on the morning session
# Challenges proposed by presenters in the morning session
# Are there any low-hanging fruit? anything that NCIGT and NA-MIC can help with?
# Are there any specific issues that should be added to the breakout discussions agenda?
# Any opportunities for problems that could be solved with industry-academia partnerships?

== Jim's Introduction ==

Some Common Themes in Challenges proposed by Speakers

# we need to do a better job at defining IGT systems requirements
# we need to develop useful data for evaluation and validation based on models, simulations, actual patient data; also need performance standards/gold standards to compare to (breakout session 2)
# we need to develop better/additional hardware and software standards for IGT (breakout 1 ?)
# we need to develop algorithm repositories for open source IGT software/hardware solutions– within this want to be able to leverage existing toolkits and fill toolkit gaps furthermore, need to coordinate research in tracking technologies (breakout 4)
# we need to provide a strong rationale/motivation to manufacturers to provide API’s/ research interfaces (part of breakout 3 ?)
# need to think about system design approaches that indicate algorithm confidence in produced result;
# need to develop systems that capitalize on incorporating robust training data into system decision making (robust across normal/abnormal)

<br />

== Discussion ==

(from Jim's slide) Common issue

* System requirements is needed
* Needs to have "flight recording"

----

<br />

* How to specify requirements?
** Is there something inbetween highly rigorous industrial practices and back of the envelope specifications?
** synchronization of code and requirements
** different scenarios, could be iterative process = no single requirement specification
*** often need refinement
*** ISO9000 lite
** see FDA requirements
*** need method for reporting problems

* Should we collect and disseminate case studies?
** Workshop?
** record processes = best practices manual?
** sell the development process...
* How do we develop guidelines for validation?
** need mechanism to pool our results
** testing: gathing information, evaluation metrics
** case data costs money
** uniform methods/tools for gathering data (e.g., OpenTracker)
** see Nafis study for methodology for tracking validation - example of how to share methods?
** NEED consensus on how to evaluate "correctness"
** "Open Data", just like "Open Source" used to be?
** suggestion: when designing IRBs, try to broaden a little bit so that the results can be used more widely or more effectively.
** patient advocacy groups - mechanisms to make data available for patients to share if they wish,

* Do we need to develop better/additional hardware and software standards for IGT?
** let the software be the driver by using "open source", at least initially
** standards problem = no standards and too many standards simultaneously (standards, but not enough agreement) - this is probably not a research problem - where should it come from?
** e.g., motivation for APIs exist for trackers (due to demand of customer), but for others (e.g., interventional imaging systems) there is less motivation for the manufacturer.
** NIH play a role in funding industry to develop these standard interfaces (APIs?)
** portability of standards is important (e.g., explicit support of legacy systems)
** what is the value proposition for industry to invest into opening interfaces or developing standards?
*** there will always be competing standards - it is up to the marketplace which will win - where is the marketplace located in our community?
*** therapy is smaller niche than diagnostic, how to convince manufacturers of new methods
*** work with imaging people who also need to open up interfaces for new diagnostic methods
*** Chris Hasser: incentive to undustry - take advantage of resources, brain power that are being brought to bear by the research community - low cost and accelerated prototyping
*** shouldn't underestimate amount of engineering required to produce stable API
*** existing hooks exist in most vendor's equipment - until vendors agree on standards, community can contribute work to interface with these interfaces under research agreements,
*** Guy Schechter (Philips): these things are understood
**** need to take the lead and chrystallize the needs (leverage the power of the group)
**** but real power is buying power? research community is often weak!
**** critical mass (need to convince a few companies)
**** standards may need to be narrow in their respective areas in order to be feasible (broad standards will take too much time, or not happen at all)

* Develop algorithm repositories?
** awareness - disseminate links to information about things like OpenTracker
** participants of this workshop should add these links on the workshop wiki
** perhaps recast: ask companies to permit us to provide a uniform interface, with having them be forced to comply with the interface.
** hardware knowledge repositories should also be considered,
** Open Source (1. freely available use) (2. distributed community of contributors)
*** should we choose prototype applications to drive an open interaction?
*** heirarchy of need - simple lab-based tests, versus complex assembly of many modules (IGT-lite and IGT-serious)
*** evolutionary process of developing open collaborations
** PeterK - how about a forum for user evaluation and comments? (e.g., just like Amazon customer reviews)
*** Insight Journal could host such discussions. Forum for discussion or review.
**** what is the incentive for the review? danger in review bias.
**** won't this just end up evolving into peer review?
** This community could come together to write requirements to give to industry - agree on common elements of functionality.
*** Larry Clarke: there is dialog between NIST and FDA on this topic
** FDA/legal/regulatory impact on opening access to machine

<br />

* Database of simulators (see work in the MRI neuro-imaging world)

<br />

* Is there some need for focused study section in this area?
** absence of reference standards for judging IGT research proposals
** the community is its own worst enemy - study section
** what role can NIH play?
** new study section?
** how many IGT proposals are actually going in.
*** there is a coding for diseases, but not for technologies, so this is difficult to track. keywords? this group could help define these keywords.
*** e.g., what should we call this community?
*** goal for breakouts: identify keywords

2006 IGT Workshop Panel Discussion - Revision history

Andy: Update from Wiki