Measuring update ability of evidence-based practice online point-of-care information services

Background: Point-of-care information services (PoCISs) provide to physicians comprehensive and condense evidence into easily digestible formats. Publishers encourage physicians to use them in their clinical decision-making and tag these services as regularly updated. Objective: This study measures the updating ability of relevant evidence for medical practice by international PoCISs. Methods: Out of 18 PoCISs available in 2008, we selected the top-five ranking for coverage, editorial quality and evidence-based methodology: Clinical Evidence, Dynamed, EBM Guidelines, EMedicine, UptoDate. Updating ability was defined as the incidence proportion of newsworthy systematic reviews (SRs) inserted in PoCIS content out of newsworthy SRs potentially eligible. These were all the SRs signalled by two literature surveillance journals (ACP Journal Club and Evidence-Based Medicine Primary Care and Internal Medicine) and the Cochrane SRs labelled as Conclusion changed in the Cochrane Library from April to December 2009. For each SR we defined: birth as the publication date in the two literature surveillance journals or in the Cochrane Library; death as the occurrence of its citation in PoCIS. SRs were censored when two independent investigators agreed on the inclusion of that evidence. The monitoring of PoCIS started in June 2009 and will last one year. We report the planned interim analysis at December 2009. We assessed the updating cumulative rate using Kaplan-Meier survival analyses. Cox model was used to estimate hazard ratio (HR) between the first PoCIS and the top second (p = 0.025 two-sided for statistical significance). We intentionally avoid reference to the service name. Anonymity will be discharged with the final analyses. Results: The interim analysis included 125 SRs. One service has an updating process which strongly dominates the others (HR First vs two top seconds: 5.43, IC 95% 3.63 8.12 and 5.14, IC 95% 3.38 7.83, both p & 0.0001). The fourth and fifth-ranked PoCIS delineated survival curves close to the bottom. Conclusion: These differences in updating ability are possibly justified by different approaches to content development. Nevertheless our results identify a clear update lead.