• Core Outcome Set-STAndards for Reporting: The COS-STAR statement

    15 December 2017

    BACKGROUND: Core outcome sets (COS) can enhance the relevance of research by ensuring that outcomes of importance to health service users and other people making choices about health care in a particular topic area are measured routinely. Over 200 COS to date have been developed, but the clarity of these reports is suboptimal. COS studies will not achieve their goal if reports of COS are not complete and transparent. METHODS AND FINDINGS: In recognition of these issues, an international group that included experienced COS developers, methodologists, journal editors, potential users of COS (clinical trialists, systematic reviewers, and clinical guideline developers), and patient representatives developed the Core Outcome Set-STAndards for Reporting (COS-STAR) Statement as a reporting guideline for COS studies. The developmental process consisted of an initial reporting item generation stage and a two-round Delphi survey involving nearly 200 participants representing key stakeholder groups, followed by a consensus meeting. The COS-STAR Statement consists of a checklist of 18 items considered essential for transparent and complete reporting in all COS studies. The checklist items focus on the introduction, methods, results, and discussion section of a manuscript describing the development of a particular COS. A limitation of the COS-STAR Statement is that it was developed without representative views of low- and middle-income countries. COS have equal relevance to studies conducted in these areas, and, subsequently, this guideline may need to evolve over time to encompass any additional challenges from developing COS in these areas. CONCLUSIONS: With many ongoing COS studies underway, the COS-STAR Statement should be a helpful resource to improve the reporting of COS studies for the benefit of all COS users.

  • The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.

    15 December 2017

    Systematic reviews and meta-analyses are essential to summarize evidence relating to efficacy and safety of health care interventions accurately and reliably. The clarity and transparency of these reports, however, is not optimal. Poor reporting of systematic reviews diminishes their value to clinicians, policy makers, and other users. Since the development of the QUOROM (QUality Of Reporting Of Meta-analysis) Statement-a reporting guideline published in 1999-there have been several conceptual, methodological, and practical advances regarding the conduct and reporting of systematic reviews and meta-analyses. Also, reviews of published systematic reviews have found that key information about these studies is often poorly reported. Realizing these issues, an international group that included experienced authors and methodologists developed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) as an evolution of the original QUOROM guideline for systematic reviews and meta-analyses of evaluations of health care interventions. The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram. The checklist includes items deemed essential for transparent reporting of a systematic review. In this Explanation and Elaboration document, we explain the meaning and rationale for each checklist item. For each item, we include an example of good reporting and, where possible, references to relevant empirical studies and methodological literature. The PRISMA Statement, this document, and the associated Web site (www.prisma-statement.org) should be helpful resources to improve reporting of systematic reviews and meta-analyses.

  • Increasing response rates to postal questionnaires: systematic review.

    12 December 2017

    OBJECTIVE: To identify methods to increase response to postal questionnaires. DESIGN: Systematic review of randomised controlled trials of any method to influence response to postal questionnaires. STUDIES REVIEWED: 292 randomised controlled trials including 258 315 participants INTERVENTION REVIEWED: 75 strategies for influencing response to postal questionnaires. MAIN OUTCOME MEASURE: The proportion of completed or partially completed questionnaires returned. RESULTS: The odds of response were more than doubled when a monetary incentive was used (odds ratio 2.02; 95% confidence interval 1.79 to 2.27) and almost doubled when incentives were not conditional on response (1.71; 1.29 to 2.26). Response was more likely when short questionnaires were used (1.86; 1.55 to 2.24). Personalised questionnaires and letters increased response (1.16; 1.06 to 1.28), as did the use of coloured ink (1.39; 1.16 to 1.67). The odds of response were more than doubled when the questionnaires were sent by recorded delivery (2.21; 1.51 to 3.25) and increased when stamped return envelopes were used (1.26; 1.13 to 1.41) and questionnaires were sent by first class post (1.12; 1.02 to 1.23). Contacting participants before sending questionnaires increased response (1.54; 1.24 to 1.92), as did follow up contact (1.44; 1.22 to 1.70) and providing non-respondents with a second copy of the questionnaire (1.41; 1.02 to 1.94). Questionnaires designed to be of more interest to participants were more likely to be returned (2.44; 1.99 to 3.01), but questionnaires containing questions of a sensitive nature were less likely to be returned (0.92; 0.87 to 0.98). Questionnaires originating from universities were more likely to be returned than were questionnaires from other sources, such as commercial organisations (1.31; 1.11 to 1.54). CONCLUSIONS: Health researchers using postal questionnaires can improve the quality of their research by using the strategies shown to be effective in this systematic review.

  • Grey literature in meta-analyses of randomized trials of health care interventions.

    12 December 2017

    BACKGROUND: The inclusion of grey literature (i.e. literature that has not been formally published) in systematic reviews may help to overcome some of the problems of publication bias, which can arise due to the selective availability of data. OBJECTIVES: To review systematically research studies, which have investigated the impact of grey literature in meta-analyses of randomized trials of health care interventions. SEARCH STRATEGY: We searched the Cochrane Methodology Register (The Cochrane Library Issue 3, 2005), MEDLINE (1966 to 20 May 2005), the Science Citation Index (June 2005) and contacted researchers who may have carried out relevant studies. SELECTION CRITERIA: A study was considered eligible for this review if it compared the effect of the inclusion and exclusion of grey literature on the results of a cohort of meta-analyses of randomized trials. DATA COLLECTION AND ANALYSIS: Data were extracted from each report independently by two reviewers. The main outcome measure was an estimate of the impact of trials from the grey literature on the pooled effect estimates of the meta-analyses. Information was also collected on the area of health care, the number of meta-analyses, the number of trials, the number of trial participants, the year of publication of the trials, the language and country of publication of the trials, the number and type of grey and published literature, and methodological quality. MAIN RESULTS: Five studies met the inclusion criteria. All five studies showed that published trials showed an overall greater treatment effect than grey trials. This difference was statistically significant in one of the five studies. Data could be combined for three of the five studies. This showed that, on average, published trials showed a 9% greater treatment effect than grey trials (ratio of odds ratios for grey versus published trials 1.09; 95% CI 1.03-1.16). Overall there were more published trials included in the meta-analyses than grey trials (median 224 (IQR 108-365) versus 45(IQR 40-102)). Published trials had more participants on average. The most common types of grey literature were abstracts (55%) and unpublished data (30%). There is limited evidence to show whether grey trials are of poorer methodological quality than published trials. AUTHORS' CONCLUSIONS: This review shows that published trials tend to be larger and show an overall greater treatment effect than grey trials. This has important implications for reviewers who need to ensure they identify grey trials, in order to minimise the risk of introducing bias into their review.

  • Handsearching versus electronic searching to identify reports of randomized trials.

    8 December 2017

    BACKGROUND: Systematic reviewers need to decide how best to reduce bias in identifying studies for their review. Even when journals are indexed in electronic databases, it can still be difficult to identify all relevant studies reported in these journals. Over 1700 journals have been or are being handsearched within The Cochrane Collaboration to identify reports of controlled trials in order to help address these problems. OBJECTIVES: To review systematically empirical studies, which have compared the results of handsearching with the results of searching one or more electronic databases to identify reports of randomized trials. SEARCH STRATEGY: Studies were sought from The Cochrane Methodology Register (The Cochrane Library, Issue 2, 2002), MEDLINE (1966 to Week 1 July 2002), EMBASE (1980 to Week 25 2002), AMED (1985 to June 2002), BIOSIS (1985 to June 2002), CINAHL (1982 to June 2002), LISA (1969 to July 2002) and PsycINFO (1972 to May 2002). Researchers who may have carried out relevant studies were contacted. SELECTION CRITERIA: A research study was considered eligible for this review if it compared handsearching with searching one or more electronic databases to identify reports of randomized trials. DATA COLLECTION AND ANALYSIS: The main outcome measure was the number of reports of randomized trials identified by handsearching as compared to electronic searching. Data were extracted on the electronic database searched, the complexity of electronic search strategy used, the characteristics of the journal reports identified, and the type of trial report identified. MAIN RESULTS: Thirty-four studies were included. Handsearching identified between 92% to 100% of the total number of reports of randomized trials found in the various comparisons in this review. Searching MEDLINE retrieved 55%, EMBASE 49% and PyscINFO 67%. The retrieval rate of the electronic database varied depending on the complexity of the search. The Cochrane Highly Sensitive Search Strategy (HSSS) identified 80% of the total number of reports of randomized trials found, searches categorised as 'complex' (including the Cochrane HSSS) found 65% and 'simple' found 42%. The retrieval rate for an electronic search was higher when the search was restricted to English language journals; 62% versus 39% for journals published in languages other than English. When the search was restricted to full reports of randomized trials, the retrieval rate for an electronic search improved: a complex search strategy (including the Cochrane HSSS) retrieved 82% of the total number of such reports of randomized trials. AUTHORS' CONCLUSIONS: Handsearching still has a valuable role to play in identifying reports of randomized trials for inclusion in systematic reviews of health care interventions, particularly in identifying trials reported as abstracts, letters and those published in languages other than English, together with all reports published in journals not indexed in electronic databases. However, where time and resources are limited, searching an electronic database using a complex search (or the Cochrane HSSS) will identify the majority of trials published as full reports in English language journals, provided, of course, that the relevant journals have been indexed in the database.

  • Time to publication for results of clinical trials.

    8 December 2017

    BACKGROUND: It has been suggested that a time-lag bias exists whereby research studies with striking results are more likely to be stopped earlier than originally planned, published quicker, or both. If time-lag bias exists, new interventions might be mistakenly assumed to be effective. OBJECTIVES: To study the extent to which time to publication of a clinical trial is influenced by the significance of its result. SEARCH STRATEGY: Studies were identified by searching the Cochrane Methodology Register (The Cochrane Library, Issue 3, 2005), MEDLINE (1966 to May 2005), EMBASE (1980 to May 2005), Science Citation Index (June 2005) and by handsearching journals and conference abstracts. SELECTION CRITERIA: Studies were eligible if they contained analyses of any aspect of the time to publication of clinical trials and tracked the publication of a cohort of clinical trials. DATA COLLECTION AND ANALYSIS: Data extraction was performed independently by two authors. Data were extracted on the median time from the date the trial started to the date of publication. Data were also extracted on source of trials under investigation; source of funding; area of health care; means by which the publication status of these trials were sought; and methodological quality of the empirical study. MAIN RESULTS: Two studies with a total of 196 trials met the inclusion criteria. In both studies just over half of all trials had been published in full. Trials with positive results (i.e. statistically significant in favour of the experimental arm) were published in approximately 4 to 5 years. Trials with null or negative results (i.e. not statistically significant or statistically significant in favour of the control arm) were published after about 6 to 8 years. One study suggested that this difference could, in part, be attributed to the length of time taken to publish the results of a trial once follow up has been completed. This study showed that trials with null or negative findings took, on average, just over a year longer to be published than those with positive results. AUTHORS' CONCLUSIONS: Our review shows that trials with positive results are published sooner than other trials. This has important implications for the timing of the initiation and updating of a review, especially if there is an association between the inclusion of a trial in a review and its publication status. It is of particular concern when one considers reviews containing only a small number of studies.