Adding evidence of the effects of treatments into relevant wikipedia pages: a randomised trial

Abstract

Background Wikipedia is among the most popular sites on the internet, and around a fifth of all healthcare searches online directs to a Wikipedia page. In recent years steps have been taken to enhance the quality of their healthcare pages, such as the creation of the WikiProject Medicine initiative. In 2014 a partnership was formalised between Wikipedia and Cochrane Collaboration with similar aims. Three years ago, a group of medical students helped to formulate a protocol for the first ever randomised trial, whose aim was to test the value of Wikipedia in informing users about healthcare. Objectives To investigate the effects of adding high-grade quantitative evidence of outcomes of treatments into relevant Wikipedia pages on further information-seeking behaviour by the end-user. Methods: We randomised 70 Wikipedia pages, identified as highly relevant to up-to-date Cochrane Schizophrenia systematic reviews that contained a Summary of Findings table. Eligible Wikipedia pages in the intervention group (35) were seeded with tables listing best evidence of the effects of treatment, and hyperlinks to the source Cochrane review. Eligible pages in the control group (35) remained unchanged. Results The main outcome measures were routinely collected data on access to the full text and summary web page of the relevant Cochrane reviews (after 12 months). There was 100% follow-up of the 70 randomised pages. Six of the 35 in the intervention group had the tabular format deleted by other Wikipedia users during the course of the study, but all pages continued to report the same data within the text. The study found no evidence of significant effect on either of the co-primary outcomes: full text access adjusted ratio of geometric means 1.30, 95% CI: 0.71 to 2.38; page views 1.14, 95% CI: 0.6 to 2.13. Results were similar for all other outcomes, with the exception of Altmetric score for which there was evidence of clear effect (1.36, 95% CI: 1.05 to 1.78). Conclusions The pursuit of fair balance within Wikipedia healthcare pages is impressive and its reach unsurpassed. Enriching Wikipedia content is, potentially, a powerful way to improve health literacy among the public, and it is possible to test the effects of seeding pages with evidence. Though increased traffic to Cochrane reviews in the intervention group lacked statistical significance, there was nonetheless consistently an increase in all outcome measures. For every person that sought and clicked the reference on the 'intervention' Wikipedia page to seek more information (the primary outcome), many more are likely to have been informed by the page alone; the inclusion of high-quality data may have lessened the desire to click out to the original source. If demonstrated that the end-user becomes more informed as a result of such intervention, the potential as a tool for dissemination could be invaluable. Future studies should take such factors into consideration, and aim to replicate, expand and further develop our trial. Trial registration number IRCT2017070330407N2