Yesterday I met with folks at Lawrence Berkeley labs. The PI entered the room, full of energy and clearly ducking briefly out of the fray to speak with us. Part of the discussion revolved around microarray experiments. We’ve all heard about how notoriously difficult it is to reproduce microarray experiments. People have proposed minimum information standards (really they’re guidelines) to combat this problem, and we’ve also all heard that often these standards aren’t enough. Even if people are following the guidelines, inevitably a crucial piece of information isn’t obviously critical and therefore isn’t communicated.
The PI noted that he has seen it to be helpful when more than one lab conducts an experiment, so that each can help the other avoid finicky and/or tacit experimental conditions that would prevent others from reproducing their results. I have wondered for some time (and for the case of microarrays in particular) whether the practice of “pair programming” that we use in software development would be more helpful than minimum information standards to increase the reproducibility of complex experiments. The problem with this, as the PI pointed out, is that duplicating every experiment can get expensive, and in the world of soft money (especially today’s world), people are always looking for ways to make the research dollar go farther. The possible long term efficiency of duplicating some efforts to increase data value and reduce a tendency to go down blind alleys might not be easy to quantify, and thus not easy to weigh quantitatively against the immediate penalty of “getting half as much work done”. (That’s certainly true in software.)
The PI pointed out that even if direct duplication was too expensive, he still advocated some kind of collaboration on experiments. In particular he advocated getting people together in the same room to look at the experiment together as it was being performed, so that the collaborator might catch important things that weren’t immediately apparent to the person performing the experiment. This, at most, only costs a small amount of travel funds.
I asked the PI if others shared his views, and he said that most of the larger microarray efforts had some sort of distributed work going on, but he wasn’t sure that this idea had been formalized anywhere.
I’m interested in this not only because of its parallel with software work, but also because I work for a company focused on facilitating collaborative science. I’m very interested in the different forms that scientific collaboration can take, and how best to help them along.
First, a few questions about microarray experiments:
What do people hope to gain from microarray experiments in 2008? What relevance does transcriptional profiling on a genome/transcriptome wide scale have when thousands of these experiments have already been done and made publicly available through GEO and MGED? And what about the more recent realizations that short, non-coding RNAs (that were thrown away during sample preparation) are orchestrating another level of regulation – you can’t go back to the original samples to measure the microRNAs? What techniques could make microarray data more usable and meaningful? The GEO interface is god-awful for the occasional user, but look at what Atul Butte can do with these data. Is there still a meaningful discovery aspect of transcriptional profiling studies? How many of the people doing microarrays have the infrastructure to do the kind of followup studies that really demonstrate relevance of their discoveries? I’d wager not many.
Now, a few thoughts on collaboration:
Collaboration is certainly the key to moving knowledge forward in an increasingly interdisciplinary world, but the paradigm we have been brought up in (at least on the science side) is that to be successful academically you have to the one with the idea, the persistence and the ability to tell a story. It doesn’t help to be the third guy from the left when you’re getting credit. Unless everyone feels that their interests are protected, no one (except the young and foolish) will move forward with a collaboration.
So is there a unique role for a small company in this process? I think there are several. But that’s a topic for a future post.
Pingback: Bookmarks about Microarray
recently there was a publication about how it is difficult to reproduce microarray experiments, mostly because the experimental conditions are not annotated properly, but I guess when more eyes will work together it can be improved. between nice blog, one more in my list http://www.abhishek-tiwari.com/2009/02/30-blogs-about-bioinformatics-and.html