Data Analysis Plan (revised February 2005)

7.1 Background

This section describes the strategy, rationale and statistical methods which will guide assessment of the clinical efficacy and safety of the study treatments in SEARCH. All analyses and reports will be prepared by the coordinating centre in the Clinical Trial Service Unit, University of Oxford.

During the two-month pre-randomisation Run-in period, potentially eligible patients received 20 mg simvastatin and placebo-vitamins daily. At the end of the Run-in period, approximately 12,000 compliant and eligible patients were to be randomly allocated to receive 80 mg or 20 mg simvastatin daily, and 2 mg folic acid plus 1 mg vitamin B12 daily or matching placebo, in a 2x2 factorial design. Study treatment was scheduled to continue for a minimum of 4 years median follow-up (ie at least 4 years after randomisation of 6000 patients) until a total of at least 1900 patients had confirmed 'major coronary events' (MCE), defined in this study as fatal CHD (ICD 410-414 in the 9th International Classification of Diseases [ICD9]), non-fatal myocardial infarction or coronary revascularisation procedure (ie coronary artery grafts [CABG] or angioplasty [PTCA]). It was pre-specified that the Steering Committee could decide to stop the study early in the light of recommendations from the independent Data Monitoring Committee (see Section 7.7.1) or to modify the schedule based on review of the blinded MCE rate during follow-up and differences in blood cholesterol and homocysteine observed between the treatment groups.

A total of 12,064 patients were randomised during September 1998 and October 2001 (with median follow-up from November 2000). But, during a median of 3-4 years follow-up, both the overall vascular event rate and the LDL cholesterol difference between the treatment groups were smaller than originally anticipated (see Section 7.6). Consequently, based on discussions at the March 2004 meeting of the Steering Committee, it was decided (blind to the interim results for clinical outcomes) to change the primary outcome to 'major vascular events' (MVE), defined as major coronary event, non-fatal or fatal stroke, or peripheral revascularisation (ie peripheral artery angioplasty or arterial surgery, including amputations), and to continue until a total of at least 2800 patients had confirmed MVEs.

7.2 Comparisons of 80 mg versus 20 mg daily simvastatin

For the cholesterol-lowering comparison with the different simvastatin doses, it is hypothesised that the more substantial reduction in LDL cholesterol produced by 80 mg simvastatin daily than by 20 mg simvastatin daily will reduce the incidence of non-fatal and fatal occlusive vascular events without adversely affecting the incidence of other non-fatal or fatal serious adverse events (in particular, haemorrhagic strokes and cancers), and that the same absolute reduction in LDL cholesterol will be associated with similar proportional reductions in vascular risk throughout the blood cholesterol range studied. All simvastatin dose comparisons will involve comparing outcome among all those patients allocated at randomisation to receive 80 mg daily simvastatin versus all those allocated to receive 20 mg daily simvastatin (ie 'intention-to-treat' analyses¹).

7.2.1 Primary comparison

The primary comparison for the simvastatin dose allocation will be of 'major vascular events' (MVE) during the scheduled study treatment period.

7.2.2 Secondary comparisons

The secondary comparisons for the simvastatin dose allocation will be of:

1. MVEs separately in the first year after randomisation (when little difference is anticipated) and in the later years of the scheduled treatment period;
2. MVEs among patients subdivided into 3 similar-sized groups with respect to blood LDL cholesterol levels at the end of the pre-randomisation Run-in period on 20 mg daily simvastatin (with the hypothesis that the same absolute reduction in LDL cholesterol will be associated with similar proportional reductions in vascular risk in each of these groups: see Section 7.4.3)
3. MVEs in the presence and in the absence of the allocated study folic acid plus vitamin B12 (with the hypothesis that the effects will be similar: see Section 7.4.3);
4. MCEs; and
5. total strokes.

7.2.3 Tertiary comparisons

The tertiary comparisons for the simvastatin dose allocation will be of the effects during the scheduled treatment period on:

1. total mortality;
2. cause-specific mortality (ie considering separately deaths from vascular causes [ICD9 410-459] and from non-vascular causes);
3. vascular mortality excluding the first year after randomisation (when little difference is anticipated);
4. coronary and non-coronary revascularisations;
5. confirmed haemorrhagic and other strokes considered separately;
6. pulmonary embolus;
7. total and site-specific cancers;
8. hospitalisations for various causes; and
9. possible adverse effects of treatment, including, in particular, evidence of liver function abnormalities (defined as two or more consecutive elevations of ALT >4 x upper limit of laboratory normal [ULN]) and evidence of muscle abnormalities (defined as any elevation of CK >10x ULN).

7.3 Comparisons of folic acid plus vitamin B12 versus placebo

For the assessment of folic acid plus vitamin B12, it is hypothesised that this treatment will reduce the incidence of non-fatal and fatal occlusive vascular events without adversely affecting the incidence of other non-fatal or fatal serious adverse events, and that the same absolute reduction in homocysteine will be associated with similar proportional reductions in vascular risk throughout the blood homocysteine range studied. All folate-based therapy comparisons will involve comparing outcome among all those patients allocated at randomisation to receive 2 mg folic acid plus 1 mg vitamin B12 versus all those allocated to receive placebo (ie 'intention-to-treat' analyses¹).

7.3.1 Primary comparison

The primary comparison for the folate-based therapy allocation will be of 'major vascular events' (MVE) during the scheduled study treatment period.

7.3.2 Secondary comparisons

The secondary comparisons for the folate-based therapy allocation will be of:

1. MVEs separately in the first year after randomisation (when little difference is anticipated) and in the later years of the scheduled treatment period;
2. MVEs among patients subdivided into 3 similar-sized groups with respect to (a) plasma folate levels and (b) blood homocysteine levels at the end of the pre-randomisation Run-in period on placebo-vitamins (with the hypothesis that the same absolute reduction in homocysteine will be associated with similar proportional reductions in CHD risk in each of these groups: see Section 7.4.3);
3. MVEs in the presence and in the absence of each of the simvastatin dose regimens (with the hypothesis that the effects will be similar: see Section 7.4.3);
4. MCEs; and (v) total strokes.

7.3.3 Tertiary comparisons

The tertiary comparisons for the folate-based therapy allocation will be of the effects during the scheduled treatment period on:

1. total mortality;
2. cause-specific mortality (ie considering separately deaths from vascular causes and from non-vascular causes);
3. vascular mortality excluding the first year after randomisation (when little difference is anticipated);
4. coronary and non-coronary revascularisations;
5. confirmed haemorrhagic and other strokes considered separately;
6. pulmonary embolus;
7. total and site-specific cancers;
8. fractures of any kind, and 'osteoporotic' fractures (ie hip, wrist or spine combined), excluding, in both cases, those due to road traffic accidents;
9. cognitive impairment (ie <22 for TICS-m score) at final follow-up;
10. hospitalisations for various causes; and (ix) possible adverse effects of treatment.

7.4 Details of analyses

7.4.1 Methods of analysis

The fundamental assessments of efficacy will involve comparisons among all randomised patients in their originally allocated treatment group, irrespective of compliance, during the scheduled treatment period (ie 'intention to treat' analyses1 ). All time-to-event analyses will be based on the first relevant event, and will use log-rank methods1 to calculate P-values and Cox regression analyses2,3 to calculate odds ratios and confidence intervals. Comparisons of the overall proportions of affected individuals, irrespective of time, will involve standard Mantel-Haenszel methods4 for the analysis of contingency tables.

The main assessment of the effects of different doses of simvastatin will involve comparing outcome among all patients allocated 80 mg daily simvastatin versus outcome among all those allocated 20 mg daily simvastatin, without stratification for the folate-based therapy allocation or other factors. Similarly, the main assessment of the effects of folic acid plus vitamin B12 will involve unstratified comparison of outcome among all patients allocated folic acid plus vitamin B12 versus outcome among all those allocated matching placebo. Use of a factorial design instead of a simple 2-way design is anticipated to have little or no effect on the statistical sensitivity with which the overall benefits of different simvastatin doses or of folic acid plus vitamin B12 can be assessed, or on the size of the study¹. (The approach to any cost-effectiveness analyses will be based on that developed previously for HPS^5,6.)

7.4.2 Allowance for multiplicity of comparisons

No allowance will be made for multiple hypothesis testing in the primary comparison of each of the two separate treatment modalities being assessed (different cholesterol-lowering regimens and folate-based therapy) in this 2x2 factorial study. For secondary and, particularly, tertiary comparisons, allowance in their interpretation will be made for multiple hypothesis testing1 , taking into account the nature of events (including timing, duration and severity) and evidence from other studies. In addition to the prespecified comparisons, many other analyses will be performed, with due allowance for their exploratory and, perhaps, data-dependent nature. Conventionally, two-sided P-values (2P) <0.05 are often described as 'significant'. But, the larger the number of events on which a comparison is based and the more extreme the P-value (or, analogously, the further the lower limit of the confidence interval is from zero) after any allowance has been made for the nature of the particular comparison (ie primary, secondary or tertiary; pre-specified or exploratory), the more reliable the comparison and, hence, the more definite any finding will be considered.

7.4.3 Tests for heterogeneity of effects

The large number of vascular events expected in this study may allow reasonably reliable direct assessment of the effects of treatment in some subcategories of patient (eg baseline levels of plasma cholesterol or homocysteine) or of vascular events (eg fatal versus non-fatal). But, when a number of different subgroups are considered, chance alone may lead to there being no apparent effect in several small subgroups in which the effect of treatment really is about the same as is observed overall. In such circumstances, 'lack of direct evidence of benefit' is not good 'evidence of lack of benefit', and clearly significant overall results would provide strong indirect evidence of benefit in some small subgroups where the results, considered in isolation, are not conventionally significant (or, even, perhaps, slightly adverse)^1,7. Hence, unless the proportional effect in some specific subcategory is clearly different from that observed overall, the effect in that subcategory is likely to be best estimated indirectly by applying the proportional effect observed among all patients in the trial to the absolute risk of the event observed among control patients in that category⁷. Tests for heterogeneity of the proportional effect on particular outcomes in specific subgroups will be used (with allowance for multiple comparisons and for other differences between the subgroups) to determine whether the effects in those subgroups are clearly different from the overall effect⁸. If, however, such subgroups can be arranged in some meaningful order (eg baseline cholesterol subdivided into 3 similar sized groups of low, medium and high) then assessment of any trend in the proportional effects on outcome will also be made.

7.5 Analyses of adverse events or biochemical abnormalities

7.5.1 Adverse events

Only those adverse events that are serious (as defined in Section 3.5.6), that lead to discontinuation of study treatment, or that are believed with a reasonable probability to be due to study treatment are to be recorded systematically during follow-up. Comparison of the incidence of these adverse events between the randomly allocated treatment groups will be made using the Mantel-Haenszel method⁴. Statistical hypothesis testing of differences in adverse events must be interpreted cautiously as this is, essentially, a screening exercise. Hence, in interpreting these results, substantial allowance will be made for multiple hypothesis testing, the data-derived nature of the exercise, the nature of the events (including timing, duration and severity) and evidence from other studies.

7.5.2 Biochemical abnormalities

Blood samples are scheduled to be taken from all randomised patients at 2, 4, 8 and 12 months after randomisation and then 6-monthly, as well as at additional clinic visits if any problems (including biochemical abnormalities) are thought to have arisen. On each occasion ALT and CK will be measured. For ALT, elevations of more than twice the upper limit of normal [ULN] will result in an Early Recall visit for repeat sampling (Appendix 13D), and two or more consecutive elevations of >4x ULN will be defined as an adverse event. For CK, elevations of >5x ULN will result in an Early Recall visit for repeat sampling (Appendix 13E), and a single elevation of >10x ULN will be defined as an adverse event. Comparisons of the incidence of such elevations of ALT and of CK between patients allocated 80 mg simvastatin and those allocated 20 mg simvastatin will be made using the Mantel-Haenszel method without stratification, and estimates made of the absolute differences and their standard deviations. Proportional and absolute differences between the randomly allocated treatment groups in mean ALT, CK, vitamin B12 and various aspects of the full blood count measured during follow-up (see Section 5.4) will also be calculated with their standard deviations.

7.6 Sample size and predicted number of events

Compared with 20 mg simvastatin daily, it was originally anticipated that 80 mg simvastatin daily would produce an average reduction in blood total and LDL cholesterol levels of at least 0.5 mmol/l. If this translated into a 15-20% further reduction in MCEs, then a study involving at least 1900 MCEs would have a good chance of demonstrating an effect on MCEs at a convincing level of statistical significance. But, although such differences in cholesterol levels were observed during the first year of follow-up, the average differences during a median of 4 years were only about 0.4 mmol/l. Consequently, the Steering Committee decided (blind to the interim results for clinical outcomes) that the trial should aim to be able to detect differences in risk of 10% reliably, which requires about 2800 events (see Table). Similarly, if the 3-4 µmol/l reduction in plasma homocysteine with the folate-based therapy being studied produces a 10-15% reduction in vascular events (as might be expected from the association of homocysteine with risk in a recent meta-analysis of observational studies⁹), then this too could be detected reliably. Based on experience in previous studies among patients with a history of MI, including HPS⁵ (which used a very similar recruitment strategy), it was estimated prior to the start of SEARCH that the annual rate of MCEs on 20 mg simvastatin daily would be about 4%. But, despite similar baseline characteristics among the MI patients in SEARCH and HPS, the annual MCE rate during the first 4 years of follow-up in both treatment groups of SEARCH combined is only about 2.7%. Consequently, at a median follow-up of 4 years (ie November 2004), there were only about 1200 – instead of the anticipated 1900 – confirmed MCEs. Given this lower than anticipated rate of MCEs and the evidence from other trials10 that statin therapy also reduces the risk of stroke, the Steering Committee decided to change the primary outcome to confirmed MVEs. At the current annual MVE rate of 3.5%, it is not anticipated that 2800 MVEs will have occurred until there is about 7 years median follow-up (ie end 2007).

Table: Predicted power to detect 10-15% reductions in vascular events

7.7 Data and safety monitoring

7.7.1 Interim analyses by the Data Monitoring Committee

During the period of the study, unblinded interim analyses of mortality, of vascular events and of any other information that is available on major events (including serious adverse events), along with any other analyses requested, will be supplied at least annually, in strict confidence, to the chairman of the independent Data Monitoring Committee. In the light of these analyses and the results of any other relevant trials, the Data Monitoring Committee will advise the Steering Committee if, in their view, the randomised comparisons in SEARCH have provided both (i) 'proof beyond reasonable doubt'* that for all patients, or for some specific types, really prolonged use of higher-dose simvastatin or of folate-based therapy is clearly indicated or clearly contraindicated in terms of a net difference in mortality or major morbidity, and (ii) evidence that might reasonably be expected to influence materially the patient management of many clinicians who are already aware of any other main trial results. The Steering Committee can then decide whether to modify the study (or to seek extra data).

Unless this happens, the Steering Committee, the collaborators, MSD and the coordinating centre staff (except those who supply the confidential analyses) will remain ignorant of the interim unblinded results on mortality and major morbidity until the study is terminated. Collaborators, and all others associated with the study, may write (preferably through the Oxford coordinating centre) to the chairman of the Data Monitoring Committee, drawing attention to any worries they may have about the possibility of particular side-effects, or about particular categories of patient requiring special consideration, or about any other matters that may be relevant. (Minutes of all Data Monitoring and Steering Committee meetings will be kept and these will be available for consideration at the end of the study.)

*Appropriate criteria of proof beyond reasonable doubt cannot be specified precisely, but in general a difference of at least 3 standard deviations in an interim analysis of a major endpoint would be needed to justify halting, or modifying, such a study prematurely, especially if the comparison was based on relatively few events (eg less than 100). If this criterion were to be adopted, it would have the practical advantage that the exact number of interim analyses would be of little importance, and so no fixed schedule is proposed¹.

7.7.2 Monitoring of any serious adverse events believed to be due to study treatment

Throughout the trial, all serious adverse events believed with a reasonable probability to be due to study treatment are to be reported immediately by telephoning the 24-hour Freefone service (see Section 3.5.6). These reports will be reviewed promptly, blind to treatment allocation, by one of the clinical coordinators, and any further information required sought urgently. Confirmed reports will be promptly forwarded 'unblinded' to the chairman of the Data Monitoring Committee, and 'blinded' to the chairman of the South Thames Multicentre Research Ethics Committee, to medical collaborators and, by facsimile, to MSD (UK). The company will notify the coordinating centre if any further information is needed (including unblinding) and will then forward relevant reports to drug regulatory agencies (see Section 1.2.1).

References

1. Peto R, Pike MC, Armitage P, Breslow NE et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. Part I: Introduction and design. Br J Cancer 1976; 34: 585-612; & Part II: Analysis and examples. Br J Cancer 1977; 35: 1-39
2. Cox DR. Regression models and life tables. J R Stat Soc (Series B) 1972; 34: 187-220
3. Cox DR. Partial likelihood. Biometrika 1975; 62: 269-276
4. Mantel N, Haenszel W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst 1959; 22: 719-748
5. Heart Protection Study Collaborative Group. MRC/BHF Heart Protection Study of cholesterol-lowering with simvastatin in 20 536 high risk individuals: a randomised placebo-controlled trial. Lancet 2002; 306: 7-22
6. Heart Protection Study Collaborative Group. Cost-effectiveness of simvastatin in people at different levels of vascular disease risk: a randomised trial in 20,536 individuals. Lancet 2005 (in press)
7. Collins R, MacMahon S. Reliable assessment of the effects of treatment on mortality and major morbidity, I: clinical trials. Lancet 2001; 357: 373-80
8. Gail M, Simon R. Testing for qualitative interactions between treatment effects and patient subsets. Biometrics 1985; 41: 361-372
9. Homocysteine Studies Collaboration (writing committee: Clarke R, Lewington S, Donald A, Collins R). Homocysteine and risk of ischemic heart disease and stroke: a meta-analysis. JAMA 2002; 288: 2015-2022
10. Heart Protection Study Collaborative Group. Effects of cholesterol-lowering with simvastatin on stroke and other major vascular events in 20,536 people with cerebrovascular disease or other high-risk conditions. Lancet 2004; 363: 757-67

Data Analysis Plan (revised February 2005) [PDF]