• How useful are Cochrane reviews in identifying research needs?

    30 March 2018

    OBJECTIVES: To determine the extent to which reports of Cochrane reviews recommend the need for further research and, if so, the extent to which they make suggestions regarding that research. METHODS: We examined all 2535 reviews in Issue 4, 2005 of The Cochrane Library. Each review was categorized on the basis of whether a suggestion was included about specific interventions, participants, or outcome measures that should be included in future research. We also identified the frequency with which reviews conclude that no more research is needed or feasible, noted the need for further systematic reviewing, and refered to a relevant ongoing or planned study. We also report the number of studies listed in the 'Ongoing Studies' section in each review. RESULTS: Only 3.2% of reviews suggested explicitly that no more research is needed or feasible. In 82.0% of reviews, suggestions were made as to the specific interventions that need evaluating, in 30.2% the appropriate participants were suggested, and in 51.9% outcome measures were suggested. Suggestions for all three domains were made in 16.9% of the reviews. While 11.6% did not include a specific suggestion about any of these domains, 21.2% of reviews mention a relevant ongoing or planned study in one or both of the 'Implications for Research' and the 'Ongoing Studies' sections. CONCLUSIONS: Most Cochrane reviews identify residual uncertainty and are a rich source of suggestions for further health-care research.

  • Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report.

    30 April 2018

    OBJECTIVE: Scientific and ethical justification for new clinical trials requires them to have been designed in the light of scientifically defensible assessments of relevant previous research. Reliable interpretation of the results of new clinical trials entails setting them in the context of updates of the reviews upon which they were deemed scientifically and ethically justifiable. We have shown previously that most reports of randomized trials published in five general medical journals in May 1997 and in May 2001 failed to set their results in the context of the findings from similar research. In the current study, we assess whether there had been progress in this respect in 2005 and also investigate the extent to which reports begin by referring to systematic reviews providing the justification for the new research reported. DESIGN: Assessment of the Introduction and Discussion sections in all reports of randomized trials published during May 2005 in five general medical journals. SETTING: Reports of randomized trials in five general medical journals. PARTICIPANTS: Annals of Internal Medicine, BMJ, JAMA, Lancet and New England Journal of Medicine. INTERVENTIONS: None. MAIN OUTCOME MEASURES: The inclusion or mention of one or more systematic reviews in the Introduction or Discussion section of each report assessed. RESULTS: We found 18 reports of randomized trials. The Introduction sections referred to systematic reviews in five (27%) of these reports. None of the discussion sections of the 15 reports of trials that were not the first published trials to address the question studied placed the results of the new trial in the context of an updated systematic review of other research. Although reference was made to relevant systematic reviews in five of these 15 reports, there was no integration - quantitative or qualitative - of the results of the new trials in an update of these reviews. In the remaining ten reports there was no evidence that any systematic attempt had been made to set the new results in the context of previous trials. CONCLUSIONS: There is no evidence of progress between 1997 and 2005 in the proportion of reports of trials published in general medical journals which discussed new results within the context of up-to-date systematic reviews of relevant evidence from other controlled trials. Although the proportion of trials referring to systematic reviews in Discussion sections has increased, the majority of reports continued to fail even to do this. Similarly, most researchers appear not to have considered a systematic review when designing their trial. Researchers and journal editors do a disservice to the interests of the public and others involved in healthcare decision-making by acquiescing in this situation.

  • Commentary: the 1944 patulin trial: the first properly controlled multicentre trial conducted under the aegis of the British Medical Research Council.

    23 May 2018

    The 1948 report of the British Medical Research Council's randomized trial of streptomycin for pulmonary tuberculosis is widely regarded as marking the beginning of the modern history of controlled clinical trials. Four years earlier, however, a methodologically sophisticated multicentre trial conducted under the aegis of the Medical Research Council was reported, which assessed the effects of the antibiotic patulin on the course of common colds. Philip D'Arcy Hart and Joan Faulkner (later Joan Doll) were the secretary and assistant secretary, respectively, to the committee overseeing the trial, and they clearly recognized the importance of preventing foreknowledge of allocations from those admitting patients to the study. To do this and to 'muddle people up', they and Ruth D'Arcy Hart devised a scheme involving the use of two patulin groups and two placebo groups, allocating patients to one of these four groups using strict rotation. Philip D'Arcy Hart believes that this study has been overshadowed by the celebrated streptomycin trial (for which he was also secretary to the oversight committee) because no beneficial effect of patulin was detected, and because the report of the streptomycin trial referred to the use of random sampling numbers to generate the allocation schedule. This article makes clear why we agree with Philip D'Arcy Hart that the 1944 patulin trial deserves wider recognition as the first well controlled, multicentre clinical trial to have been conducted under the aegis of the British Medical Research Council. This status is reflected in the International Journal of Epidemiology's reproduction of the full text of the trial report in this issue of the journal.

  • Dicing with chance, life and death in systematic reviews and meta-analyses: D.I.C.E. 3, a simulation study.

    30 January 2018

    OBJECTIVES: To show the effects of chance on meta-analyses, and the potential dangers of being prompted to do a meta-analysis by one favourable trial. DESIGN: In total, 100,000 trials were simulated and combined into 10,000 meta-analyses, using data from the control group of a cancer trial. Each participant record was randomly coded to simulate allocation to 'treatment' or 'control'. SETTING: Simulated study. PARTICIPANTS: De-identified records for 578 patients from the control group of a cancer trial, of whom 147 had died. MAIN OUTCOME MEASURE: Time to death from any cause. RESULTS: Of the 100,000 trials, 4897 (4.9%) were statistically significant at 2p < 0.05 and 123 (1.2%) of the 10,000 meta-analyses were significant at 2p < 0.01. The most extreme result was a 20% reduction (99% CI: 0.70-0.91; 2p = 0.00002) in the annual odds of dying in the 'treatment' group. If a meta-analysis contained at least one trial with a statistically significant result (at 2p < 0.05), the likelihood of the meta-analysis being significant (at 2p < 0.01) increased strikingly. For example, among the 473 meta-analyses in which the first trial in a batch of 10 was statistically significant (at 2p < 0.05), 18 (3.8%) favoured treatment at 2p < 0.01. CONCLUSIONS: Chance can influence the results of meta-analyses regardless of how well they are conducted. Researchers should not ignore this when they plan a meta-analysis and when they report their results. People reading their reports should also be wary. Caution is particularly important when the results of one or more included studies influenced the decision to do the meta-analysis.

  • Systematic review of purine analog treatment for chronic lymphocytic leukemia: lessons for future trials.

    15 June 2018

    A systematic review of purine analogs revealed heterogeneity between trials in treatment effects on response and progression free survival, but not survival, perhaps partly due to variations in analytical methods. In addition, combination treatments required evaluation. Therefore, individual patient data were sought for all randomized trials in untreated chronic lymphocytic leukemia which involved a purine analog, but which did not include antibody therapies. Sixteen trials were found, addressing seven comparisons. Eight trials, with 2,753 patients, showed that single agent purine analog improved progression free survival (odds ratio=0.71; 95% confidence interval=0.63-0.79). Heterogeneity remained substantial. Three trials, with 1,403 patients, showed that progression free survival was further improved by the addition of cyclophosphamide (odds ratio=0.54; 0.47-0.62). Fewer data were available on the addition of other drugs to purine analog, and none showed clear benefit. Two trials, with 544 patients, suggested cladribine improved progression free survival compared to fludarabine (odds ratio=0.77; 0.63-0.95). No differences were seen in overall survival for any comparisons. In conclusion, purine analogs, particularly combined with cyclophosphamide, significantly improve progression free survival but not survival. Some groups, such as the elderly, may not see the same benefits and maximizing doses may be important for all treatments, including chlorambucil. Longer follow up, consistent definitions and detailed reporting of trials should be encouraged.

  • Sixteen-year single-surgeon experience with coil embolization for ruptured intracranial aneurysms: recurrence rates and incidence of late rebleeding. Clinical article.

    31 May 2018

    OBJECT: Over a 16-year period, 570 patients presenting with acute aneurysmal subarachnoid hemorrhage were successfully treated using endosaccular coil embolization within 30 days of hemorrhage by a single surgeon. Patients were followed to assess the stability of aneurysm occlusion and its longer-term efficacy in protecting against rebleeding. METHODS: Patients were followed for 6 to 191 months (mean 73.7 months, median 67 months) by clinical review, angiography performed at 6 and 24 months posttreatment, and questionnaires sent via the postal service every 5 years. Late rebleeding was defined as > 30 days after treatment. RESULTS: Stable angiographic occlusion was evident in 74.5% of small, 72.2% of large, and 60% of giant aneurysms. Recurrent filling was found in 119 (26.3%) of 452 aneurysms. Rebleeding was diagnosed in 9 patients (6 treated aneurysms) and occurred between 2 and 114 months posttreatment. It was due to aneurysm recurrence in 6 patients, rupture of a coincidental untreated aneurysm in 2 patients, and rupture of a de novo aneurysm in 1 patient. Rebleeding occurred in 3 (2.5%) of 119 unstable aneurysms and in 3 (0.9%) of 333 stable aneurysms, as seen on initial follow-up angiography studies. Annual rebleeding rates ranged from 0.2% to 0.6% for all causes and from 0.2% to 0.4% for rebleeding of treated aneurysms. No rebleeding was recorded after the first decade, with 138 patients having more than 10 years of follow-up. CONCLUSIONS: Periodic follow-up with angiographic studies after coil embolization is recommended to identify aneurysm recurrence and patients at a high risk of late rebleeding in the medium term. More frequent follow-up is recommended for patients harboring coincidental unruptured aneurysms.

  • [CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration].

    14 June 2018

    BACKGROUND: Clear, transparent, and sufficiently detailed abstracts of conferences and journal articles related to randomized controlled trials (RCTs) are important, because readers often base their assessment of a trial solely on information in the abstract. Here, we extend the CONSORT (Consolidated Standards of Reporting Trials) Statement to develop a minimum list of essential items, which authors should consider when reporting the results of a RCT in any journal or conference abstract. METHODS AND FINDINGS: We generated a list of items from existing quality assessment tools and empirical evidence. A three-round, modified-Delphi process was used to select items. In all, 109 participants were invited to participate in an electronic survey; the response rate was 61%. Survey results were presented at a meeting of the CONSORT Group in Montebello, Canada, January 2007, involving 26 participants, including clinical trialists, statisticians, epidemiologists, and biomedical editors. Checklist items were discussed for eligibility into the final checklist. The checklist was then revised to ensure that it reflected discussions held during and subsequent to the meeting. CONSORT for Abstracts recommends that abstracts relating to RCTs have a structured format. Items should include details of trial objectives; trial design (e.g., method of allocation, blinding/masking); trial participants (i.e., description, numbers randomized, and number analyzed); interventions intended for each randomized group and their impact on primary efficacy outcomes and harms; trial conclusions; trial registration name and number; and source of funding. We recommend the checklist be used in conjunction with this explanatory document, which includes examples of good reporting, rationale, and evidence, when available, for the inclusion of each item. CONCLUSIONS: CONSORT for Abstracts aims to improve reporting of abstracts of RCTs published in journal articles and conference proceedings. It will help authors of abstracts of these trials provide the detail and clarity needed by readers wishing to assess a trial's validity and the applicability of its results.