Funding Agencies and Open Science

I've been invited to participate in a panel discussion on "New tools in research, teaching, and publishing" on May 24, 2010 at the annual PI meeting for the Integrative Graduate Education and Research Traineeship (IGERT) program at NSF. After speaking with program manager Vikram Jaswal, I feel encouraged that funding agencies are interested in exploring the emerging role of Open Science and related novel communication channels for facilitating scientific progress.


The role that funding agencies can play in Open Science has been the subject of some discussion in the blogosphere. One view is that they can require more openness as a condition of funding. The NIH's requirement to make papers resulting from funding Open Access after 12 months of publication is a step in that direction. There is a debate about whether this should be extended to Open Data - even to the point of Open Notebook Science, where even failed experiments would be shared for the scientific community to learn from.

I tend to prefer the carrot to the stick. I think that funding agencies could value plans for "sharing beyond the norms" in proposals without imposing strict requirements. In the long run OS will succeed because each stakeholder (researcher, funder, publisher, etc.) acts out of selfish motives. I believe that the most effective way to stimulate this selfishness is to show concrete examples of practice and benefits.

Funding agencies should see the benefits of OS as a higher ROI - in terms of knowledge gained and shared with the scientific community - as well as the wider population ultimately footing the bill. A perceived downside of higher transparency might be the greater difficulty in fueling hype cycles. Most things aren't as pretty up close and science is no exception. If you measure success as the absence of failure and ambiguity then increased transparency is going to be a problem. Most experiments are failures of some sort (as the saying goes - if you're not failing you're not trying hard enough). But failed or successful - both categories of results can be useful to others if they are made available in a way that they can be discovered easily. Funding agencies can help transparency by making it clear that the whole truth is more valuable than a subset of the truth presented in a way that might be conveniently misleading.

This doesn't mean that you can't put your best foot forward and give a slick PowerPoint presentation to guide your audience. It is ok to construct an easily digestible narrative of your research. It is ok to distill your work down to key conclusions. It isn't necessary to confuse your audience with every ambiguous result and unanswered question.

But - in addition to the streamlined version of your work - if you provide all the details of the failures and ambiguities for those who can benefit from further exploration of what you have done - there is a great potential for accelerating the scientific process. For a funding agency OS can mean a bigger bang for the buck.

Happy Accidents: A Must-Read for Open Scientists

I usually limit my book reviews to Goodreads or Shelfari but this one deserves much more attention.

In Happy Accidents: Serendipity in Modern Medical Breakthroughs; When Scientists Find What They're NOT Looking for, Morton Meyers reviews examples of the unpredictability of scientific progress.

This could just be a collection of interesting anecdotes - and some of the stories are truly fascinating. My favorite is probably the discovery of platinum compounds for the treatment of cancer. It came about from the accidental electro-dissolution of a platinum electrode during an experiment studying the effect of electricity on cell cultures!

But Meyers goes further and uses these examples to make larger observations about the way science operates today in both academia and industry. A quote from the preface foreshadows the tone of the book:

The dominant convention of all scientific writing is to present discoveries as rationally driven and to let the facts speak for themselves. This humble ideal has succeeded in making scientists look as if they never make errors, that they straightforwardly answer every question they investigate. It banishes any hint of blunders and surprises along the way. Consequently, not only the general public but the scientific community itself is unaware of the vast role of serendipity in medical research. Typically, a discoverer may finally admit this only towards the end of his or her career, after the awards have been received.

And starting on page 304:

An applicant for a research grant is expected to have a clearly defined program for a period of three to five years. Implicit is the assumption that nothing unforeseen will be discovered during that time and, even if something were, it would not cause distraction from the approved line of research. Yet the reality is that many medical discoveries were made by researchers working on the basis of a fallacious hypothesis that led them down an unexpected fortuitous path.
....
The peer review system forces investigators to work on problems others think are important and to describe the work in a way that convinces the reviewers that results will be obtained. This is precisely what prevents funded work from being highly preliminary, speculative or radical. How can a venture into the unknown offer predictability of results?(my emphasis)
....
Indeed the basic process of peer review demands conformity of thinking and disdains a maverick's approach.
....
What it comes down to is this: Who on a review committee is the peer of a maverick? (my emphasis)

The fact that some of us in the Open Science community are discussing this does not mean that we are advocating for the abolition of peer review or the NIH. We are not that naive. We still submit proposals and manuscripts for publication in peer-reviewed journals (although given a choice we probably would pick an Open Access journal over one running on a paid subscription model).

The point is what we do in addition to all those traditional processes.

We can share our failed experiments. We can share our research plans. We can discuss science freely admitting what we don't know. We can record our talks at closed meetings and make them public. We can initiate and participate in serious scientific conversations going on in the blogosphere without worrying about everyone's title and rank.

Basically, we can collaborate in ways that are most conducive to serendipitous discoveries. The free social software, databases and other infrastructure now available make this information exchange easier than ever.

The key question for a researcher today: to hoard or not to hoard?

To me, it seems likely that data hoarders will find it more and more difficult to claim priority for a contribution when competing against loose associations of open collaborators motivated by insatiable curiosity.

Some of the folks from the funding side are getting it. Take a look at SubMeta.

Crowdsourcing Drug Development

Yesterday I had the privilege of attending a workshop at the NIH on the National Cancer Institute Clinical Development of Small Molecules:

This one-day workshop will provide specialized training and information to NCI-supported investigators who plan to undertake clinical development of novel concepts and who are directly involved with implementing translational clinical research. Individuals will benefit from the opportunity for direct interaction with FDA’s Center for Drug Evaluation and Research and NCI’s Developmental Therapeutics Program senior staff.

I am grateful to Dan Zaharevitz for the invitation to the workshop yesterday and to a visit of his screening labs later today.

Dan has a vision that the drug discovery and development process could benefit tremendously through openness at all stages by facilitating communication between all parties involved from discovery through clinical testing.

For example, if discussions about formulation, ease of scale-up and toxicity occurred early on, perhaps a more efficient overall process would result. Our selection of Ugi products that crystallize from the reaction mixture and that are obtainable from cheap commercially available starting materials is an attempt at anticipating the scale-up and cost factors down the road.

Of course, this is what Open Notebook Science is all about - getting feedback from those with expertise at the earliest possible moment. The scale-up consideration I mentioned is an obvious factor from the perspective of an organic chemist. But there are other parameters that could benefit immensely from input from the drug development community.

Formulation is a big one. So far I haven't placed any restriction or requirements on functional groups in our Ugi virtual libraries. Most are not going to be very soluble in water and I don't have a good feeling about how important that is. At the workshop, it was repeated several times that a water soluble compound is preferable but it is certainly not a deal breaker. For IV delivery liposomal formulations are an option, as are some other FDA approved co-solvent systems. Right now we're shipping powders for our assays. It is my understanding that these are usually taken up in DMSO. But we don't have to do that - I have some experience with the preparation of liposomes and it would not be too much of a hassle to ship our compounds formulated in that way. There are issues of stability that have to be considered but these are manageable.

Now another approach would be creating compounds that are water soluble from the start. We could do that by introducing tertiary amines as a second functionality on our starting materials but that would severely reduce the size of our virtual libraries. Another strategy would involve using boc-protected amino acids, which we know work well in the Ugi reaction. The problem there is that they would have to be deprotected and that would reduce the convenience of the preparation.

Then there is the issue of optimizing by metabolites. Perhaps we should not be docking only the Ugi products but also their likely metabolic products. An expert in pharmacodynamics could certainly provide valuable input here. Anticipating toxicity is another consideration.

These are just examples of the types of conversations that we could be having well before any compounds get to in vivo or clinical trials.

And that is what Open Notebook Science is all about. Instead of waiting for our paper on the synthesis of new inhibitors of a certain enzyme to appear in print, give us some feedback while the experiment is being done.

There are definitely some obstacles to overcome to achieving this type of transparency and collaboration but technology is not the bottleneck. Even finding collaborators is no longer the key issue, with people like Dan Zaharevitz, Rajarshi Guha, Egon Willighagen, Gus Rosania, Philip Rosenthal, Tsu-Soo Tan, Cameron Neylon, Antony Williams, Kevin Owens, Peter Murray-Rust, and others stepping up to contribute what they can.

I think the main issue now is convincing a funding organization that this is a model worthy of support. Maybe NSF will be one of them.