Should I Trust Wikipedia With My Health?

Dr. James Heilman isn't an easy man to get hold of — he kept offering us odd, off-hour windows of availability to do a phone interview. When we finally connected, he explained: He works the night shift as an emergency room physician in British Columbia. He also puts in time as a clinical assistant professor in emergency medicine.

And then there's the 60 — count 'em, 60! — hours a week he toils away editing Wikipedia, the massive online encyclopedia written and edited by, well, anyone who wants to give it a whirl.

"My wife likes to joke that Wikipedia's my mistress," Heilman says. "I prefer to call it our first child." He chuckles with a tinge of recognition that he may have gone off the rails a bit, in his zeal to make the profuse number of health-related pages on the site more accurate.

So how does an emergency doc become an obsessive Wikipedian?

About 10 years ago, he says, during a quiet night shift, he got to browsing one of the medical articles.

"I thought, 'Oh, my God this is really bad,' " he says. "And then I saw the edit button, and I realized, 'Oh, my God, I could fix the internet!' I sort of got madly hooked. I've been trying to fix the Internet ever since."

Accuracy questioned

Part of that quest was creating the WikiProject Medicine group, where 320 like-minded editors work on the site's health content. Heilman estimates about half are medical professionals.

Reworking Wikipedia health entries is not a trivial task. A 2014 study found about 25,000 pages of health-related articles in the English language. That number is now up to 32,000, Heilman says.

The health pages worldwide attracted almost 4.9 billion page views in 2013. A 2012 survey of several hundred medical students found 94 percent use the site for health information.

But, despite its popularity, the reliability of Wikipedia's medical content has often been questioned. A 2011 review on the accuracy and thoroughness of the site's medical entries found mixed results. Other studies show that the site fell short in gastroenterology and hepatology, as well as in general drug information, and in information on the cholesterol-lowering drugs statins (which was later improved). Pages having to do with ear, nose and throat problems were also found wanting.

Other studies found the website provided better information on conditions related to nephrology and on the pages having to do with depression and schizophrenia.

But just recently, in the journal The Lancet Global Health, researchers complained that Wikipedia entries on stillbirths were missing critical information.

And in a 2014 study that compared published, peer-reviewed research from scientific journals to Wikipedia articles on the same 10 diseases and conditions, the authors found a significant number of assertions in the Wikipedia material that were unsupported by the evidence.

Heilman doesn't think much of that particular review; he and several other medical specialists who contribute to Wikipedia co-wrote a rebuttal that questioned the study's rigor. Still, he acknowledges "there's a lot of work to do" to bring health entries on Wikipedia up to snuff.

Calls to action

An editorial last month in the online media outlet The Conversation, Thomas Shafee, a biochemist at La Trobe University, in Melbourne, Australia, (and frequent contribute to Wikipedia) called on other researchers and doctors to make Wikipedia better.

"Health professionals have a duty to improve the accuracy of medical entries in Wikipedia, because it's the first port of call for people all over the world seeking medical information," Shafee wrote. "The accuracy of the site is vital, because every medical entry... has the potential for immediate real-world health consequences."

Some other academics agree. Dr. Amin Azzam, a research psychiatrist at the University of California, San Francisco, recently co-authored a report published in the journal Academic Medicine titled "Why Medical Schools Should Embrace Wikipedia."

Improving Wikipedia

Efforts to improve the entire site, including the health section, are already underway. Hundreds of students in both the U.S. and Canada, for example, are working as editors through the Wiki Education Foundation, a nonprofit organization based in San Francisco that acts as a bridge between Wikipedia and academia. The organization encourages university faculty to assign students Wikipedia entries as a part of coursework.

"Rather than write a biography of a woman scientist for their class that their instructor would read and throw away and never engage in again, these students instead have the opportunity to create content for Wikipedia where thousands of people can gain from their work," says the foundation's director of programs, LiAnna Davis.

Pharmacology students at the University of California, San Francisco, for example, are working to improve entries on popular drugs like the HIV antiviral drug Abacavir. Students in a voice disorders class at McGill University in Montreal are adding information about laryngitis. As students fill in missing information, they improve gross inadequacies, Davis explains, and rewrite articles in a way that uses fewer technical terms, so they can be better understood by the general public.

Since the start of 2016, about 280 science classes at universities in the U.S. and Canada have added content to 3,650 articles and generated 300 new entries receiving more than 102 million views, Davis says.

Heilman cites other improvements to the crowd-sourced compendium, including a collaboration with the group Translators Without Borders to rewrite Wikipedia medical content in as many languages as possible — including Odia, a language spoken by tens of millions of people in India that is not included in Google Translate. Heilman is also proud of the Wikipedia entry for dengue fever, which was published in the peer-reviewed, open-access journal Open Medicine.

Some federal agencies are also partnering with Wikipedia. The National Institutes of Health is creating an online biomedical dictionary for PubMed Health, which provides consumer-friendly summaries of research information. Usually, the NIH creates the definitions by drawing on material in its own collection, but sometimes uses Wikipedia as a source, according to NIH staff. Both organizations benefit, as the NIH will flag Heilman if it comes across something inaccurate in the Wikipedia material.

"I spend a fair amount of time trying to convince my colleagues to come work on Wikipedia," Heilman says. "As a physician I believe all people deserve access to quality health care information."

How to find good health information online

So, after all is said and done, how should the average person approach a web search for good consumer health information?

Azzam, the UCSF psychiatrist who called for medical schools to embrace Wikipedia, says he would, nonetheless, send his patients first to government sites — like the National Institutes of Health, or the Centers for Disease Control and Prevention, or the National Library of Medicine for reliable health information.

"Many of those sites have information targeting 'lay' people," he says. "Additionally there are national advocacy organizations for specific disease and/or health conditions. For example, as a psychiatrist I will refer my patients and their families to the National Alliance on Mental Illness (NAMI)."

And for anyone turning to Wikipedia as a health resource, Azzam advises going one more step, and clicking the "Talk" tab near the top left of every page. That will take you to a rating of how thorough and accurate Wikipedia editors think that particular page is. He also says it's a good idea to check the references that each assertion of fact in Wikipedia entries is required to link to. (You can do this by clicking on the superscript numbers at the end of a sentence.)

"That gives me some idea of how much to trust the content," Azzam says.

And get information from multiple sources, he recommends, to ferret out any outliers. That's "effectively a proxy measure for validity," he says.

Finally, if you want to consult material that health professionals use themselves, says Azzam, try UpToDate, or PubMed, or studies in peer-reviewed journals.

Health information is on a "continuum," he believes, with subscription-only, top tier scholarly journals on one end and the free, user-created Wikipedia on the other.

"We believe [journal] content to be the scientific truth as we know it because content experts have peer-reviewed the manuscripts," Azzam says. "But the challenge at that extreme is gaining access to that knowledge — especially with high-cost subscription charges."

That is why he's working with students to improve Wikipedia, "pulling that pole toward the other end of the spectrum."

"Since so many consumers of health information online are reading that content, I want to be a part of a movement to make it as high quality as possible!"

Heilman, the avid health editor, also recommends checking multiple, high-quality sources when looking for health information, paying closest attention to areas where the sources agree.

Meanwhile, he'll keep carving out hours to improve Wikipedia.

"As a physician, I can help one person at a time," he says. "But as a Wikipedian, I can make a difference in millions and millions of people's lives."

This story was produced by KQED's health and technology blog, Future of You.