• Choosing important health outcomes for comparative effectiveness research: a systematic review.

    16 March 2018

    BACKGROUND: A core outcome set (COS) is a standardised set of outcomes which should be measured and reported, as a minimum, in all effectiveness trials for a specific health area. This will allow results of studies to be compared, contrasted and combined as appropriate, as well as ensuring that all trials contribute usable information. The COMET (Core Outcome Measures for Effectiveness Trials) Initiative aims to support the development, reporting and adoption of COS. Central to this is a publically accessible online resource, populated with all available COS. The aim of the review we report here was to identify studies that sought to determine which outcomes or domains to measure in all clinical trials in a specific condition and to describe the methodological techniques used in these studies. METHODS: We developed a multi-faceted search strategy for electronic databases (MEDLINE, SCOPUS, and Cochrane Methodology Register). We included studies that sought to determine which outcomes/domains to measure in all clinical trials in a specific condition. RESULTS: A total of 250 reports relating to 198 studies were judged eligible for inclusion in the review. Studies covered various areas of health, most commonly cancer, rheumatology, neurology, heart and circulation, and dentistry and oral health. A variety of methods have been used to develop COS, including semi-structured discussion, unstructured group discussion, the Delphi Technique, Consensus Development Conference, surveys and Nominal Group Technique. The most common groups involved were clinical experts and non-clinical research experts. Thirty-one (16%) studies reported that the public had been involved in the process. The geographic locations of participants were predominantly North America (n = 164; 83%) and Europe (n = 150; 76%). CONCLUSIONS: This systematic review identified many health areas where a COS has been developed, but also highlights important gaps. It is a further step towards a comprehensive, up-to-date database of COS. In addition, it shows the need for methodological guidance, including how to engage key stakeholder groups, particularly members of the public.

  • Sharing individual participant data from clinical trials: an opinion survey regarding the establishment of a central repository.

    15 December 2017

    BACKGROUND: Calls have been made for increased access to individual participant data (IPD) from clinical trials, to ensure that complete evidence is available. However, despite the obvious benefits, progress towards this is frustratingly slow. In the meantime, many systematic reviews have already collected IPD from clinical trials. We propose that a central repository for these IPD should be established to ensure that these datasets are safeguarded and made available for use by others, building on the strengths and advantages of the collaborative groups that have been brought together in developing the datasets. OBJECTIVE: Evaluate the level of support, and identify major issues, for establishing a central repository of IPD. DESIGN: On-line survey with email reminders. PARTICIPANTS: 71 reviewers affiliated with the Cochrane Collaboration's IPD Meta-analysis Methods Group were invited to participate. RESULTS: 30 (42%) invitees responded: 28 (93%) had been involved in an IPD review and 24 (80%) had been involved in a randomised trial. 25 (83%) agreed that a central repository was a good idea and 25 (83%) agreed that they would provide their IPD for central storage. Several benefits of a central repository were noted: safeguarding and standardisation of data, increased efficiency of IPD meta-analyses, knowledge advancement, and facilitating future clinical, and methodological research. The main concerns were gaining permission from trial data owners, uncertainty about the purpose of the repository, potential resource implications, and increased workload for IPD reviewers. Restricted access requiring approval, data security, anonymisation of data, and oversight committees were highlighted as issues under governance of the repository. CONCLUSION: There is support in this community of IPD reviewers, many of whom are also involved in clinical trials, for storing IPD in a central repository. Results from this survey are informing further work on developing a repository of IPD which is currently underway by our group.

  • Barriers to the uptake of evidence from systematic reviews and meta-analyses: a systematic review of decision makers' perceptions.

    6 March 2018

    OBJECTIVE: To review the barriers to the uptake of research evidence from systematic reviews by decision makers. SEARCH STRATEGY: We searched 19 databases covering the full range of publication years, utilised three search engines and also personally contacted investigators. Reference lists of primary studies and related reviews were also consulted. SELECTION CRITERIA: Studies were included if they reported on the views and perceptions of decision makers on the uptake of evidence from systematic reviews, meta-analyses and the databases associated with them. All study designs, settings and decision makers were included. One investigator screened titles to identify candidate articles then two reviewers independently assessed the quality and the relevance of retrieved reports. DATA EXTRACTION: Two reviewers described the methods of included studies and extracted data that were summarised in tables and then analysed. Using a pre-established taxonomy, the barriers were organised into a framework according to their effect on knowledge, attitudes or behaviour. RESULTS: Of 1726 articles initially identified, we selected 27 unique published studies describing at least one barrier to the uptake of evidence from systematic reviews. These studies included a total of 25 surveys and 2 qualitative studies. Overall, the majority of participants (n=10 218) were physicians (64%). The most commonly investigated barriers were lack of use (14/25), lack of awareness (12/25), lack of access (11/25), lack of familiarity (7/25), lack of usefulness (7/25), lack of motivation (4/25) and external barriers (5/25). CONCLUSIONS: This systematic review reveals that strategies to improve the uptake of evidence from reviews and meta-analyses will need to overcome a wide variety of obstacles. Our review describes the reasons why knowledge users, especially physicians, do not call on systematic reviews. This study can inform future approaches to enhancing systematic review uptake and also suggests potential avenues for future investigation.

  • Long-term effects of autologous bone marrow stem cell treatment in acute myocardial infarction: factors that may influence outcomes.

    6 March 2018

    AIMS: To investigate whether there are important sources of heterogeneity between the findings of different clinical trials which administer autologous stem cell treatment for acute myocardial infarction (AMI) and to evaluate what factors may influence the long-term effects of this treatment. METHODS AND RESULTS: MEDLINE (1950-January 2011), EMBASE (1974-January 2011), CENTRAL (The Cochrane Library 2011, Issue 1), CINAHL (1982-January 2011), and ongoing trials registers were searched for randomised trials of bone marrow stem cells as treatment for AMI. Hand-searching was used to screen recent, relevant conference proceedings (2005-2010/11). Meta-analyses were conducted using random-effects models and heterogeneity between subgroups was assessed using chi-squared tests. Planned analyses included length of follow-up, timing of cell infusion and dose, patient selection, small trial size effect, methodological quality, loss of follow-up and date of publication. Thirty-three trials with a total of 1,765 participants were included. There was no evidence of bias due to publication or time-lag, methodological quality of included studies, participant drop-out, duration of follow-up or date of the first disclosure of results. However, in long-term follow-ups the treatment seemed more effective when administered at doses greater than 10(8) cells and to patients with more severe heart dysfunction. CONCLUSIONS: Evaluation of heterogeneity between trials has not identified significant sources of bias in this study. However, clinical differences between trials are likely to exist which should be considered when undertaking future trials.

  • African HIV/AIDS trials are more likely to report adequate allocation concealment and random generation than North American trials.

    1 March 2018

    BACKGROUND: Adherence to good methodological quality is necessary to minimise bias in randomised conrolled trials (RCTs). Specific trial characteristics are associated with better trial quality, but no studies to date are specific to HIV/AIDS or African trials. We postulated that location may negatively impact on trial quality in regions where resources are scarce. METHODS: 1) To compare the methodological quality of all HIV/AIDS RCTs conducted in Africa with a random sample of similar trials conducted in North America; 2) To assess whether location is predictive of trial quality. We searched MEDLINE, EMBASE, CENTRAL and LILACS. Eligible trials were 1) randomized, 2) evaluations of preventive or treatment interventions for HIV/AIDS, 3) reported before 2004, and 4) conducted wholly or partly (if multi-centred) in Africa or North America. We assessed adequacy of random generation, allocation concealment and masking of assessors. Using univariate and multivariate logistic regression analyses we evaluated the association between location (Africa versus North America) and these domains. FINDINGS: The African search yielded 12,815 records, from which 80 trials were identified. The North American search yielded 13,158 records from which 785 trials were identified and a random sample of 114 selected for analysis. African trials were three times more likely than North American trials to report adequate allocation concealment (OR = 3.24; 95%CI: 1.59 to 6.59; p<0.01) and twice as likely to report adequate generation of the sequence (OR = 2.36; 95%CI: 1.20 to 4.67; p = 0.01), after adjusting for other confounding factors. Additional significant factors positively associated with quality were an a priori sample size power calculation, restricted randomization and inclusion of a flow diagram detailing attrition. We did not detect an association between location and outcome assessor masking. CONCLUSIONS: The higher quality of reporting of methodology in African trials is noteworthy. Most African trials are externally funded, and it is possible that stricter agency requirements when leading trials in other countries and greater experience and training of principal investigators of an international stature, may account for this difference.

  • Cost-effectiveness of using prognostic information to select women with breast cancer for adjuvant systemic therapy

    21 February 2018

    OBJECTIVES: To investigate the cost-effectiveness of using prognostic information to identify patients with breast cancer who should receive adjuvant therapy. DATA SOURCES: Electronic databases from 1980 through to February 2002. A survey of clinical practice in UK cancer centres and units. Large retrospective dataset containing data on prognostic factors, treatments and outcomes for women with early breast cancer treated in Oxford. REVIEW METHODS: Between six and nine databases were searched by an information expert. Evidence-based methods were used to review and select those studies and the quality of each included paper was assessed using standard assessment tools reported in the literature or piloted and developed for this study. A survey of clinical practice in UK cancer centres and units was carried out to ensure that conclusions drawn from the report could be implemented. These data, along with the information gathered in the systematic reviews, informed the methodological approach adopted for the health economic modelling. An illustrative framework was developed for incorporating patient-level prediction within a health economic decision model. This framework was applied to a large retrospective dataset containing data on prognostic factors, treatments and outcomes for women with early breast cancer treated in Oxford. The data were used to estimate directly a parametric regression-based risk equation, from which a prognostic index was developed, and prognosis-specific estimates of the baseline breast cancer hazard could be observed. Published estimates of treatment effects, health service treatment costs and utilities were used to construct a decision analytic framework around this risk equation, thus enabling simulation of the effectiveness and cost-effectiveness of adjuvant therapy for all possible combinations of prognostic factors included in the model. RESULTS: The lack of good-quality systematic reviews and well-conducted studies of prognostic factors in breast cancer is a striking finding. There are no registers of studies of prognostic factors or of reviews of prognostic studies. Many of the reviews used weak methods, primary studies are similar with poor methodology and reporting of results. In addition, there is much variation in patient populations, assay methods, analysis of results, definitions used and reporting of results. Most studies appear to be retrospective and some use inappropriate methods likely to inflate outcomes such as optimising cut points and failing to test the results in an independent population. Very few reviews used meta-analysis to conduct a pooled analysis and to provide an estimate of the average size of any association. Instead, most reviews relied on vote counting. Although many prognostic models for breast cancer have been published, remarkably few have been re-examined by independent groups in independent settings. The few validation studies have been carried out on ill-defined samples, sometimes of smaller size and short follow-up, and sometimes using different patient outcomes when validating a model. The evidence from the validation studies shows support for the prognostic value of the Nottingham Prognostic Index (NPI). No new prognostic factors have been shown to add substantially to those identified in the 1980s. Improvement of this index depends on finding factors that are as important as, but independent of, lymph node, stage and pathological grade. The NPI remains a useful clinical tool, although additional factors may enhance its use. We accepted that hormone receptor status (ER) for hormonal therapy such as tamoxifen and prediction of response to trastuzumab by HER2 did not require systematic review, as the mechanism of action of these drugs requires intact receptors. There was no clear evidence that other factors were useful predictors of response and survival. The survey confirmed pathological nodal status, tumour grade, tumour size and ER status as the most clinically important factors for consideration when selecting women with early breast cancer for adjuvant systemic therapy in the UK. The protocols revealed that although UK cancer centres appear to be using the same prognostic and predictive factors when selecting women to receive adjuvant therapy, much variation in clinical practice exists. Some centres use protocols based upon the NPI whereas others do not use a single index score. Within NPI and non-NPI users, between-centre variability exists in guidelines for women for whom the benefits are uncertain. Consensus amongst units appears to be greatest when selecting women for adjuvant hormone therapy with the decision based primarily upon ER or progesterone receptor status rather than combinations of a number of factors. Guidelines as to who should receive adjuvant chemotherapy, however, were found to be much less uniform. Searches of the literature revealed only five published papers that had previously examined the cost-effectiveness of using prognostic information for clinical decision-making. These studies were of varying quality and highlight the fact that economic evaluation in this area appears still to be in its infancy. By combining methodologies used in determining prognosis with those used in health economic evaluation, it was possible to illustrate an approach for simulating the effectiveness (survival and quality-adjusted survival) and the cost-effectiveness associated with the decision to treat individual women or groups of women with different prognostic characteristics. The model showed that effectiveness and cost-effectiveness of adjuvant systemic therapy have the potential to vary substantially depending upon prognosis. For some women therapy may prove very effective and cost-effective, whereas for others it may actually prove detrimental (i.e. the reductions in health-related quality of life outweigh any survival benefit). CONCLUSIONS: Outputs from the framework constructed using the methods described here have the potential to be useful for clinicians, attempting to determine whether net benefits can be obtained from administering adjuvant therapy for any presenting woman; and also for policy makers, who must be able to determine the total costs and outcomes associated with different prognosis based treatment protocols as compared with more conventional treat all or treat none policies. A risk table format enabling clinicians to look up a patient's prognostic factors to determine the likely benefits (survival and quality-adjusted survival) from administering therapy may be helpful. For policy makers, it was demonstrated that the model's output could be used to evaluate the cost-effectiveness of different treatment protocols based upon prognostic information. The framework should also be valuable in evaluating the likely impact and cost-effectiveness of new potential prognostic factors and adjuvant therapies.

  • Abstracts presented at the American Society of Clinical Oncology conference: how completely are trials reported?

    9 March 2018

    PURPOSE: To assess how completely trials published in conference proceedings are reported and whether this has changed over time. METHODS: Conference abstracts published at the American Society of Clinical Oncology (ASCO) conference (1992 and 2002) were read to identify reports of randomized trials. A checklist was devised (based on CONSORT) to assess the completeness of reporting. RESULTS: Four-hundred and ninety-four abstracts reporting randomized trials were identified; 209 in 1992 and 285 in 2002. More trials included "randomized" in the title in 2002 compared to 1992 (54% versus 36%). Almost no trials stated the method of allocation concealment, 12% stated the method of blinding, 95% described eligible participants and 98% described the interventions. Ninety-five per cent reported the number of participants in each trial. The median number of participants per trial increased over time; 120 in 1992 and 209 in 2002 (P < 0.01). In 1992, 67% of trials reported the number of participants analysed, compared to only 49% in 2002 (P < 0.01), 28% reported or suggested intention to treat analysis dropping to 15% in 2002. Twenty-nine abstracts in 2002 and five in 1992 reported no results, with a promise of presentation at the meeting. CONCLUSIONS: The reporting of conference abstracts for trials should be improved to further facilitate understanding of their conduct and validity.

  • How important is the size of a reprint order?

    8 December 2017

    OBJECTIVES: This study aims to assess the impact of articles with very high reprint orders ("high-reprint articles") by measuring their citation in the subsequent literature as compared with a control group of articles. METHODS: The twenty-one articles (published in the Lancet in 1998) with reprint orders of over 10,000 were matched with a control set of twenty-one articles with smaller reprint orders. The Science Citation Index was used to obtain the number of citations for each of the forty-two articles. RESULTS: The twenty-one high-reprint articles were cited 2,548 times; the mean number of citations was 121 (range, 3 to 499 citations per article). Five of the twenty-one high-reprint articles had more than 200 citations, but seven (33%) were cited twenty-five times or fewer. The twenty-one control articles were cited 986 times; the mean number of citations was forty-seven (range, 1 to 165). Fifteen (71%) of the twenty-one control articles were cited twenty-five times or fewer. Thirteen of the high-reprint articles were reports of randomized trials with a mean of 163 citations. In the control articles, six were reports of randomized trials with a mean of eighty-eight citations. CONCLUSIONS: Articles with a high-reprint order were cited more frequently than other articles. However, some high-reprint articles were cited infrequently. If the size of a reprint order is related to the importance of an article, those articles with very high reprint orders may, therefore, be perceived as more important. Further research is needed to explore other aspects of the relative importance and impact of high-reprint articles.