SDIWG:Ongoing Discussion 2011
Meeting notes for NIH Roadmap National Centers for Biomedical Computing (NCBC) Centers to discuss plans for the last four years (2011-2015) of the NCBC effort
- 1 Discussion Points: General
- 2 Tcon on February 2 with Kikinis NA-MIC team
- 3 Tcon on March 4 with Califano MAGNet team
- 4 Tcon on March 15 with Ohno-Machado iDASH team
- 5 Written response on March 16 from Mark Musen NCBO team
- 6 Written response on March 27 from Russ Altman Simbios team
- 7 Overall comments on action items by Peter Lyster:
- 8 Appendix A: Spreadsheet of funded Collaborating R01/R21
- 9 Appendix B: Original charter of SDI (software and data integration): Charter of SDI:
- 10 Appendix C: Original Working group was composed of LSOs from each Center: Software and Data Integration Working Group
- 11 Appendix D: The current list of NIH NCBC Project Team members and other information on the centers
- 12 Appendix E: Spreasheet of awarded Collaborative NCBC R01 and R21
Discussion Points: General
On behalf of the NCBC Project Team Peter Lyster is calling the PIs to discuss their thoughts on future joint activities of the NCBCs. After the output has been collected and published on the wiki the plan is to have a joint-PIs tcon with NIH staff to plan the next four years of joint activities.
- What are we going to do to plan a legacy so that program and the community can know the progress after 10 years, so that future leaders in government and research community plan future programs can plan and scientists can plan future efforts in computational science (BICB) at different scales from R01s up to Centers?
- Five years ago I had similar phone calls with PIs. Original minutes on discussion with PIs is on the NCBC wiki at http://na-mic.org/Wiki/index.php/SDIWG:Software_and_Data_Integration_Working_Group_(SDIWG)_and_Centers_Discussion, more particularly http://na-mic.org/Wiki/index.php/SDIWG:Ongoing_Discussion
- There are a number of thoughts on the table, e.g., joint outreach efforts of the investigator teams such as: session at significant meeting; contribution to a joint issue of BCR; a significant review paper; keep improving the outreach through the portal http://www.ncbcs.org/ … We want to hear from you.
- In addition, the Project team would like for ideas/input from PIs for the next AHM/NCBCmeeting
Tcon on February 2 with Kikinis NA-MIC team
Topic: Plans for joint activities among NCBCS for the next 4 years.
20110201: Tcon with NA-MIC Center
Attendees: Ron Kikinis, Will Schroeder, Zohara Cohen, Mike Ackerman, Peter Lyster (notes)
Peter: Introduction: for the last five years we have run three-monthly tcon called “NCBC Working Groups” (was originally “Software and Data Integration Working Groups”). This is been successful in informal information exchange between Centers, initiating relationships that have often led to progress.
Peter Suggests: Claim victory, dissolve old working groups; we can continue with the efforts, especially the portal (although need to get word out about that: is it linked to all PIs pages?). I suggest that in the next four years we address (i) outreach and (ii) impact.
Zohara: What is status press release? Peter: Probably not from NIH. New Center had an announcement.
Ron: Big P. Issue, we were selected to cover large area, and with very little overlap. NA-MIC has some obvious interactions with other Centers (Simbios/SimTK, i2b2,…).
Peter: Original SDIWG/Working Groups had larger vision, but there was no reason to force interactions (e.g., standards for software engineering). Thus we did not do interoperability tests (which can be ‘toy problems’), but is there still room for this now that the Centers have matured?
Zohara: Perhaps we shouldn't focus on interoperability between Centers, but on adoption of the tools by the broader biomedical computing community. We need to get word out about all range of our activities.
Peter: Possible article in BCR (Stanford Biomedical Computational Review http://biomedicalcomputationreview.org/, which is funded under Simbios). Russ Altman (PI Simbios) has indicated he is enthusiastic to take on a role here.
Ron: Biositemaps is 50 foot view. But need 5000 foot view of Centers.
Peter: Previous applications for funding haven’t done good job of convincing reviewers as a whole; it is difficult to write interesting by sufficiently detailed applications for funding in scientific computing. A related issue is then outreach, e.g., can we get the word out about accomplishments through editorials, command performances at large meetings, … how about a large review article spanning detailed content of Biomedical Informatics and Computational Biology (BICB). The first cut of that is perhaps the article of Morris et al 2002 after the first BISTI symposium. [ref: Morris et al. “Digital Biology: an emerging and promising discipline”, Trends in Biotechnology, 23(3), 113-117 (2005).]
Ron: Detail is in the content of the applications. Integrating the content into coherent picture is a lot of work and difficult.
Peter: Approach at first is to have ‘volunteer’ e.g., to run tcons, the portal, run tcons to develop impact document. Under the current system of running tcons ever couple of months there is a sense that ‘Peter will handle things’. If we can parse out responsibility for organizing the efforts then there will be more shared responsibility.
Ron: It depends on interest from NIH. Peter: the NIH is interested but the content is likely too much for Program Staff to gather, edit, coordinate on their own. Needs more ‘volunteer’ effort from NCBCs to step up to leadership, e.g., edit a joint review journal article.
Ron: No program officer knows that content of all Centers, but then no PI does either.
Peter: Yes, a volunteer would need to lead a collective effort.
Zohara: Can we hire a contractor to help with, say, a large review article? Peter: Let’s talk with Karin.
Peter: Also, don’t forget about nuts and bolts: how often tcon to share news of day.
Action Items: Work to develop method for 5000" view of Centers, e.g., and extension of Biositemaps. How about a review article which would need specific effort from each Center.
Tcon on March 4 with Califano MAGNet team
Topic: Plans for joint activities among NCBCS for the next 4 years.
20110304: Tcon with MAGNet Center
Attendees: Ais Floratos, Andrea Califano, Dan Gallahan, Peter Lyster
Andrea: Lot of PIs will be working to find other sources of funding. Teams will be difficult to work in that environment. So how do we create legacy: many Centers will have different approach.
Dan: Similar sentiments. Unique situation. One of legacies is this thing about Roadmap -> IC which will cause Centers to align more with IC. If we are doing something that is good and useful to the community then why would we stop it? ICBP (Cancer sysbio) and NCBC program have science and infrastructure component (enabling technologies and enabling sciences).
Dan: Need to say something as an NCBC community. Could be beneficial to individual Centers as well. Need AHM as worthwhile endeavor.
Aris: Legacy. Is this about big P or small p? What does Peter think this is about?
Peter: I think Big P is primarily about getting the word out about all the achievements of small p. But here is how it spills over to big P: we know that our computing is magic, but our profile is low and reviewers tend to be negative about the way we write proposals.
Aris: There is a lot of effort on sequencing and handling the next gen data. Andrea is doing a lot of traveling and (proselytizing)
Peter: I want to propose that the PIs can do it as a team and have an impact.
Andrea: Knows computing is important, and knows that it will be a success. Centers actually don’t have a lot of common ground. Maybe can do a good job of proselytizing by liaising with own communities, for Andrea, it is systems biology at NCI, NIGMS, etc.
Dan: The computational efforts that are successful are seen as strong biological papers, but behind the scenes there is a lot of infrastructure.
Peter: It is more difficult to do both computing/biology. Always, biology has to be the driver.
Peter: Agrees that NCBCs proselytizing with own communities is important. Peter asks, when this is done do you say “Here I am, PI X funded by NCBC U54, and here is my work, and I’m here to show why the NCBC Program created this”. The key is can this be done by the NCBC PIs as a community? If PIs and senior leaders in NCBCs focus, e.g., through a series of regular phone calls, and at the AHM, then they can do separately?
Andrea: PIs have double legacy: one in their own area and community and institution, the other is the program.
Peter: To some extent we are in together. Think of this is program staff working with you all (cooperative agreement) to help promote impact/outreach.
Dan: Review paper will help the overall community. Not too onerous. Beneficial.
Andrea: Got TTD2 white paper on Biotechnology, all PIs signed, new approach for patient-centric therapeutics. We can do something like that here. Give program ammunition when we try to create new programs.
Action Items: Proscelitize in own communities. Maybe joint review paper.
Tcon on March 15 with Ohno-Machado iDASH team
Topic: Plans for joint activities among NCBCS for the next 4 years.
20110315: Tcon with iDASH Center
Attendees: Lucila Ohno-Machado, Heidi Sofia, Peter Good, Peter Lyster
Peter Lyster: Introduction to the past 5 years. Have regular phone calls. PIs often attended. Michigan runs Portal at http://www.ncbcs.org and still maintains the content. A lot of the early communication on the phone calls was helpful in exchanging information about software development approaches and centers appreciated the opportunity to get help from each other. Had three working groups which were quite active: (i) biositemaps-resourceome-Resource Discovery System (RDS) http://biositemaps.org/. A lot of help from Pittsburgh. (ii) Ontologies; (iii) impact as shown in the work on the http://www.ncbcs.org/
Lucila: Do we have stats on how many people access ncbcs.org? Peter doesn’t know. Needs iDASH on bottom of biositemaps.org page. Is already actively working with Pitt on CTSA and on the Resource Discovery System (RDS) that was developed with biositemaps; talking with Stanford (Musen yes, Altman to do) to talk about collaboration; familiar with i2b2. iTools is UCLA. How does one generate and publish a biositemaps file? Peter, here is Cheat Sheet for getting started in biositemps(customized for NCBCs). Please use that to create and maintain your biositemaps.
Lucila: What are expectations? For example, October 23-26 AMIA in DC. Deadline is March 17. Lucila is organizing panel: may be Kikinis, Zac, Califano, Altman, Musen, Athey. Need four on Panel. Editor of AMIA Journal (Journal of American Medical Informatics Association JAMIA), is interested in this.
Peter: Working groups as formulated for first round is over. My thought is: what can we do to maximize the impact of our legacy at the end of the program (11 years from the start). Most of the impetus from this will come from PIs and their staff; in the past some have provided deep software development, or someone to run a web site… a lot of effort.
Peter Good: What should we do in an All Hands Meeting?
Lucila: It is difficult to tease out what PIs have? What has worked and what not. Can we have smaller meeting and do it without formality of presentations.
Peter Good: We realized seven years ago that we are not covering the waterfront. How do we maximize without overreaching, acknowledge gaps… say what we have.
Lucila: Make it clear what gaps are. Possibly have doors closed session, acknowledge (i) gaps and (ii) what didn’t work. Of course a lot did work so add that in. We should know that, the hard part is finding out what did not work. What is our position on important issues, e.g., shutting down of SRA (short read archive)?
Peter: We can ask, how do we (ncbcs) express our opinion? Get the word out on all media, not just ncbcs.org
Peter Good: how about ncbcs blog? Why not?
Lucila: There are a lot of media.
Peter Lyster: Any work needs regular attention. Need regular phone calls with senior people who focus on a regular basis on, say, media. Needs to be (i) regular, and (ii) proactive. Should be monthly or better.
Lucila: Has dissemination core codirector (Wendy Chapman). Maybe identify point person in each Centers.
Peter: Says it is difficult to describe large informatics efforts, can be pages of gray text. Yes we know that the output is ‘magic’. How do we do a better job of making the case with the community of biomedical researchers. Better ‘use cases’.
Lucila: Needs to explain what we do for audiences ourselves.
Action Items: Panel at AMIA. Wendy Chapman is dissemination person, may engage others. Review article on past 6 years, can start and might take a year. Will consider her journal. JAMIA or PLOS (Bourne journal). Maybe scientific video (SciVee) youtube for scientists. Pacific symposium? What is our opinion on SRA, next gen analysis, databases. NCBCs may formulate an opinion on this and get the word out (show a presence).
Written response on March 16 from Mark Musen NCBO team
Topic: Plans for joint activities among NCBCS for the next 4 years.
Thanks for restarting this conversation. These are very appropriate issues for the NCBC PIs to be discussing. I believe the NCBCs will develop linkages between their Core 1 activities on a point-to-point basis—and this there is not going to be a need for much top–down management on this front—although it will be helpful for the NCBCs to be able to fill in one another about what their new Core 1 directions are.
I view the issue of legacy to be highly tied to our success at outreach. I believe that most of the NCBCs are still struggling with how best to optimize their outreach work, and yet our success in the long term will be measured in terms of who uses our technologies and what they do with them. As you know, we in the NCBO have been trying to identify mechanisms to reach out to communities beyond the early adopters, seeking ways to educate people who would not otherwise know what our technologies can offer them. I would imagine that the other NCBC PIs continue to face similar challenges, and opportunities to discuss strategies for converting the unwashed masses (as Barry Smith likes to call them) would be very valuable.
Similarly, all the NCBC PIs are confronted with the task of maximizing the value to their centers of DBPs and collaborating R01s. As you know, at the NCBO, these relationships historically have ranged from intense collaboration that clearly has been mutually beneficial to "parallel play" that actually causes more overhead for the Center than anything that we get back in return. It would be very helpful for the PIs to be able to discuss strategies for keeping collaborations on track and mutually advantageous. I would imagine that discussion of mechanisms for selecting DBPs and collaborating projects would also be welcome.
Another topic worth discussing across the NCBCs is training. As the budgets for the NCBCs have been whittled away, all of is have taken resources away from our training activities. And yet preparing the next generation of investigators is closely tied to the legacy issue. We may end up simply wringing our hands about this issue, but I think it's worth putting on the table.
When we had the first NCBC AHM way back when, an extremely successful component of that meeting involved outreach talks that the NCBCs gave to the NIH community. I imagine that we would like to pursue something along these lines again.
Action Items: Get the word out better to our user communities. Deal with training. Improve collabortive R01/R21, get the word out etc. Discuss selection of DBPs
Written response on March 27 from Russ Altman Simbios team
Topic: Plans for joint activities among NCBCs for the next 4 years.
First, I agree with Mark's comments from March 16: (1) the Core 1 activities are very peculiar to each center (we are focusing on computing performance, NCBO is focusing on web tools, etc...) and so linkages there should be entirely opportunistic, (2) the Core 2 activities in the DBPs should impact the chosen fields, and create a certain momentum that will create markets, users, and dependencies. Mark's point about outreach is correct. A key question is how is NIH going to continue to serve those constituencies when the NCBC program goes away? Ideally, the categorical institutes might realize that these are key technologies for their supported science, and will ramp up support. But this is a very bad economic time, and leadership of this type will be difficult to find as ICs "hunker down" to fund core activities. Most importantly, however, the technologies created by the NCBCs are cross-cutting and no single institute (for the most part) depends on one particular software, and so there needs to be a follow on program (perhaps the "continued maintenance of software" RFA needs to be expanded and maintained itself) to allow the most valuable fruits of the NCBC program to be perpetuated. (3) The collaborating R01 program also needs to be carefully considered because these are the grants that are most directly dependent on NCBC tools, and should form the nucleus of support for the NCBC program. I'm not sure we have achieved this, but we should know whether we have or not. (4) Training. I agree that this will be an important legacy of the NCBC program. Simbios tracks students and post-docs, where they wind up, and how they pursue science. I think we should create a report of the "scientific lives affected" by the NCBC program to document this very clearly.
In terms of legacy of NCBC program, I would propose that the PIs consider the following ideas:
1. One or more special issues of the Biomedical Computation Review focusing on application areas that were revolutionized (? or at least affected!) by the activities of the NCBC. Ideally this would be a cross-cutting issue with multiple contributions from multiple NCBCs but all thematically linked. Neuroscience, Clinical Research, whatever. As the publisher of this magazine, that is now getting out to 5000+ people/issue, I can guarantee that we could reach lots of opinion leaders and colleagues, and also create an archival record that could be referenced in the future. It would also be great to be able to wave these around in 2014 when we are clamoring for a follow on program.
2. I would propose that all major software, data and models be submitted as projects on Simtk.org, so that there is a single repository for the full fruits of the NCBC program. We at Stanford are committed to making sure that Simtk.org continues to exist even after Simbios, and I don't think anyone else should have to build this. I would be happy to provide a demo, but I can assure you that all my students and post-docs have been putting their code, data, software, models on this site for the last 7 years, and it provides an amazing set of functionalities. We have more than 10,000 registered users, and we could easily accommodate all the products of the NCBCs. NOTE: I don't think we need to be the only source of this material, and any effective dissemination infrastructures that individual NCBCs have created should be continued, but I think there is a strong positive in having at least one place where all the artifacts lie in archival form. We can provide links back to the "home" location of these, and we can make sure that credit assignment is done appropriately. But we have a huge investment in this resource, and I am fairly sure that most people don't appreciate how good and general purpose it is. So I am happy to talk more. We could potentially fork the code and rename it NCBCtk.org or whatever, and we are willing to do this, but I really don't think this is necessary. Instead, we could redo the front page to make it more general. One of the things that simtk.org does is automatically populate biositemaps fields.
3. Just to state the obvious. There needs to be an NIH follow on program that takes the next evolutionary step from the NCBCs. At a minimum we need a way to continue supporting the important software we have created that people now depend on. We also should add an ability to identify new challenges and address them similarly. We need to continue to train the cadre of young scientists who do this work, specifically devoted to biomedical application areas, and we need centers for collaboration for investigators for whom computation is becoming mandatory but do not have the background to just jump in. This is a tough one, but I bet we all think it is critical. There also should probably be a follow-on training program (e.g. an extension of the NIGMS biocomputation training program) to continue momentum at centers of excellence around the country.
Overall comments on action items by Peter Lyster:
1. Joint activities of NCBC PIs to promote impact and outreach which will give NIH Program (etc) ammo to use to justify future programmatic efforts. As well as other efforts. A thread on this is on the wiki (with action items) at http://na-mic.org/Wiki/index.php/SDIWG:Ongoing_Discussion_2011.
2. Working group to address item 1 needs to engage in continuous activity (e.g., monthly brainstorming tcons of relevant staff). One shot activity will not do much.
3. What can be done with existing opportunities http://www.bisti.nih.gov/funding/index.asp. NIH need to try to grow this, and it is cyclic activity with dynamic input from NCBCs, e.g., joint outreach to get more PIs to respond to Collaboration R01/R21 etc. Program Projects. BISTI R01s PAR 09 218. Unsolicited R01s etc.
Appendix A: Spreadsheet of funded Collaborating R01/R21
The initiatives are/were:
• par05-063, par07-249, par08-184 (R01) • par06-223, par07-250, par08-183 (R21)
Appendix B: Original charter of SDI (software and data integration): Charter of SDI:
The RFA states the goal of creating 'the networked national effort to build the computational infrastructure for biomedical computing for the nation'. Here is a link to the wording in the original RFA-RM-04-002 http://grants.nih.gov/grants/guide/rfa-files/RFA-RM-09-002.html The recent RFA was http://grants.nih.gov/grants/guide/rfa-files/RFA-RM-09-002.html In furthering this, the goals of the SDIWG in concert with the Project Team and Centers staff are: 1. To advance the domain sciences, and promote software interoperability and data exchange. 2. To capture the collective knowledge of software engineering and practices among the Centers and publish this knowledge widely.
Appendix C: Original Working group was composed of LSOs from each Center: Software and Data Integration Working Group
Peter Lyster (NIGMS, Chair) Stephan Bour (NIAID) Carol Bean (NCRR) Arthur Castle (NIDDK) German Cavelier (NIMH) Larry Clarke (NCI) Elaine Collier (NCRR) Peter Covitz (NCI) Jennifer Couch (NCI) Valentina Di Francesco (NIAID) Peter Good (NHGRI) John Haller (NIBIB) Donald Harrington (NIBIB) Peter Highnam (NCRR) Michael Huerta (NIMH) Jennie Larkin (NHLBI) Yuan Liu (NINDS) Michael Marron (NCRR) Richard Morris (NIAID) Bret Peterson (NCRR) Karen Skinner (NIDA) Michael Twery (NHLBI) Terry Yoo (NLM)
Appendix D: The current list of NIH NCBC Project Team members and other information on the centers
Appendix E: Spreasheet of awarded Collaborative NCBC R01 and R21
Can be found at http://www.ncbcs.org/summary.html