Scientists Often Skip A Simple Test That Could Verify Their Work
There's a simple test that scientists could use to make sure the cells they're studying in the lab are what they think they are. But most of the time, academic scientists don't bother.
That omission is a problem. One study found that between 18 percent and 36 percent of all cell lines have been misidentified. And this kind of mistaken identity is one reason that many results from experiments run in scientific labs can't be reproduced elsewhere.
"These are really not very easy discussions to have," says Leonard Freedman, founding president of the Global Biological Standards Institute. "Irreproducibility is kind of an uncomfortable subject to talk about, particularly among researchers."
But Freedman is trying to kindle that conversation. Last month he hosted a discussion in Washington, D.C., on the subject that I was invited to moderate and allowed to record.
"We are fully convinced that this is a significant enough problem that we have to take steps to address it," Jon Lorsch, director of the NIH's National Institute of General Medical Sciences, said during the panel discussion.
One obvious step would be to require scientists who get federal funding to test their cells. Howard Soule, chief science officer at the Prostate Cancer Foundation, said that's what his charity requires of the scientists it funds.
There's a commercial lab that will run this test for about $140, so "this is not going to break the bank," Soule said.
But Lorsch at the NIH argued that it's not so simple on the scale at which his institute hands out funding. "We really can't go and police 10,000 grants," Lorsch said.
"Sure you can," Soule shot back. "How can you not?"
Lorsch said if they do police this issue, "there are dozens and dozens of other issues" that the NIH should logically police as well. "It becomes a Hydra," Lorsch said. "You know, you chop off one head and others grow."
Biomedical research gets more expensive all the time, and the NIH is reluctant to pile on a whole bunch of new rules. It's a balancing act.
"If we become too draconian we're going to end up squashing creativity and slowing down research, which is not good for the taxpayers because they aren't going to get as much for their money," Lorsch said.
"On the other hand, if it's the wild, wild West, we're going to end up using taxpayer money in nonproductive ways ... which is absolutely not acceptable."
The scientific journals that publish this research could also require that all cell lines be authenticated. But at the moment, that would mean rejecting the vast majority of submissions.
Veronique Kiermer, executive editor at the Nature journals, said editors there have tried to coax scientists to test their cells by adding it to a checklist they have to provide when they submit a paper. But only about 10 percent of the scientists submitting papers check that box.
"The thing that keeps me up is that we've tried to address this issue and we've failed," she said.
This may all come down to money. Scientists can avoid most of these problems by purchasing cells from a company that routinely tests them. But most scientists would rather walk down the hall and borrow cells from another lab.
"Academics share their cell lines like candy because they don't want to go back and spend another $300," said Richard Neve from Genentech. "It is economics. And they don't want to spend another $100 to [verify] that's still the same cell line."
For companies like Genentech, though, the economic driver is to make sure that scientists there know exactly which cells they're using.
Neve runs a massive cell bank at the Bay Area biotech giant, and like other biotech and pharmaceutical companies, it maintains strict quality controls. When scientists at his company find an intriguing result from an academic lab, the first thing they do is try to replicate the result.
Neve said often they can't, and misidentified cells are a common reason.
This is a problem. But "the exact implications of it, though, I don't think we'll ever understand," he said. "And I think what we need to do at this point is to set these standards and move forward. And then we can say, from this point on, we can believe the science that was generated."
In an ideal world, scientists would toss all of their unauthenticated cell lines into the trash and start from scratch. Freedman's nonprofit is trying to nudge scientists in that direction. But so far, there's not enough incentive for academic scientists to take that step.
9(MDE1MTIxMDg0MDE0MDQ3NTY3MzkzMzY1NA001))
Transcript
RENEE MONTAGNE, HOST:
I, for one, was shocked yesterday to learn from our science correspondent Richard Harris that researchers studying cells might be looking at the wrong cells - accidental mix-ups in the lab, for example, using melanoma cells when they thought they were working on breast cancer. It turns out, this is a big problem in university labs but there is a simple test that scientists can use to prevent this confusion. Most academic scientists don't bother to take that extra step, and this morning Richard Harris asks, why not?
RICHARD HARRIS, BYLINE: This is not a minor issue. Leonard Freedman says up to a third of all cell lines that scientists use in their experiments are actually not what the scientists think they are. Over the years, scientists have identified over 400 cell lines that are misidentified. And Freedman says this mistaken identity is one reason that scientists often have trouble reproducing results from other labs.
LEONARD FREEDMAN: These are really not very easy discussions to have. Your reproducibility is kind of an uncomfortable subject to talk about, especially among researchers.
HARRIS: But Freedman has amped up that conversation with a small nonprofit called GBSI. He hosted a discussion last month here in Washington, D.C. Among the participants was John Lorsch, a senior official at the National Institutes of Health.
JOHN LORSCH: We are fully convinced that this is a significant enough problem that we have to take steps to address it.
HARRIS: One obvious step would be to require everyone who gets a federal grant to test their cells. Howard Soule, from the Prostate Cancer Foundation, says that's what his charity requires of the scientists he funds. There's a commercial lab that will run this test for about $140, Soule says.
HOWARD SOULE: This is not going to break the bank.
HARRIS: But Lorsch at the NIH argued that it's not so simple on the scale at which his institute hands out funding.
LORSCH: We can't really go and police 10,000 grants.
HARRIS: Sure you can.
LORSCH: You think?
HARRIS: You have to. How can you not?
LORSCH: Well...
HARRIS: How can you not?
LORSCH: You know, we could decide we're going to police this one issue, but there are dozens and dozens of other issues. And it becomes a hydra; you know, you chop off one head and others grow.
HARRIS: Lorsch says biomedical research gets more expensive all the time. And he's reluctant to pile on a whole bunch of new rules. It's a balancing act.
LORSCH: If we become too draconian, we're going to end up squashing creativity and slowing down research, which is not good for the taxpayers because they're not going to get as much for their money. On the other hand, if it's the wild, wild West, we're going to end up using taxpayers' money in nonproductive ways as we heard, which is absolutely not acceptable.
HARRIS: The scientific journals that publish this research could also require that all cell lines be authenticated. But at the moment, that would mean rejecting the vast majority of submissions. Veronique Kiermer, who is executive editor at the Nature journal, says they've tried to coax scientists to test their cells by adding it to a checklist they have to provide when they submit a paper. But only about 10 percent of the scientists submitting papers check that box.
VERONIQUE KIERMER: The thing that keeps me up is that we've tried to address this issue, and we failed.
HARRIS: This may all come down to money. Richard Neve at Genentech says scientists can avoid most of these problems by purchasing cells from a company that routinely tests them. But most scientists would rather walk down the hall and borrow cells from another lab.
RICHARD NEVE: Academics share their cell lines like candy because they don't want to go back and spend another $300. It is economics. And they don't want to spend another $100 to say that's still the same cell line. It's economics.
HARRIS: For companies like Genentech, though, the economic driver is to make sure that they know exactly what cells they're using. Neve runs a massive cell bank at Genentech, and like other pharmaceutical companies, maintains strict quality controls. When scientists at his company find an intriguing result from an academic lab, the first thing they do is try to replicate the result. Neve says often they can't, and misidentified cells are a common reason. This is a problem.
NEVE: The exact implications of it, though, I don't think we'll ever understand. And I think what we need to do at this point is to set these standards and move forward. And that way we can say, well, from this point on, we can believe the science was regenerated.
HARRIS: In an ideal world, scientists would toss all their unauthenticated cells into the trash and start from scratch. Len Freedman's nonprofit is trying to nudge scientists in that direction. But so far, there's not enough incentive for academic scientists to take that step. Richard Harris, NPR News. Transcript provided by NPR, Copyright NPR.
300x250 Ad
300x250 Ad