Effect Size Meta-analysis

Menu location: Analysis_Meta-Analysis_Effect Size.

Case-control studies of continuous outcomes (e.g. serum creatinine) may be investigated with respect to overall size of effect of an intervention. Meta-analysis may be used to investigate the combination or interaction of a group of independent studies, for example a series of effect sizes from similar studies conducted at different centres. This StatsDirect function examines the effect size within each stratum and across all of the studies/strata.

There are a number of statistical methods for estimating effect size, StatsDirect uses g (modified Glass statistic with pooled sample standard deviation) and the unbiased estimator d (Hedges and Olkin, 1985):

- where n_e is the number in the experimental group, n_c is the number in the control group, μ_e is the sample mean of the experimental group, μ_c is the sample mean of the control group, σ_e is the sample standard deviation for the experimental group, σ_c is the sample standard deviation for the control group, N = n_e + n_c, J(m) is the correction factor given m and Γ is the gamma function.

For each study StatsDirect gives g with an exact confidence interval and d with an approximate confidence interval. An iterative method based on the non-central t distribution is used to construct the confidence interval for g (Hedges and Olkin, 1985).

The pooled mean effect size estimate (d+) is calculated using direct weights defined as the inverse of the variance of d for each study/stratum. An approximate confidence interval for d+ is given with a chi-square statistic and probability of this pooled effect size being equal to zero (Hedges and Olkin, 1985).

StatsDirect also gives the option to base effect size calculations on weighted mean difference (a non-standardized estimate unlike g and d) as described in the Cochrane Collaboration Handbook (Mulrow and Oxman, 1996).

The inconsistency of results across studies is summarised in the I² statistic, which is the percentage of variation across studies that is due to heterogeneity rather than chance – see the heterogeneity section for more information.

Please note that the results from StatsDirect may be slightly different from the results you obtain using other packages or from those quoted in papers; this is due to the use of exact bias correction calculated from the gamma distribution in StatsDirect.

DATA INPUT:

You may enter number, mean and standard deviation for control and experimental groups of each study. Alternatively you may just enter numbers in experimental groups, numbers in control groups and effect size g (nb. please make sure you use g only as defined above!).

Example

From personal communication from Dr N. Freemantle.

The following data represent test outcomes for six studies in which an educational intervention was investigated:

	Experimental			Control
Trial	N	Mean	SD	N	Mean	SD
Kottke	27	18.50	14.90	17	5.40	17.30
Levinson	16	60.00	29.20	15	55.00	29.20
oliver intensive	25	10.72	6.46	66	6.92	6.83
oliver standard	62	9.20	6.16	66	6.92	6.83
sulmasy	9	3.75	2.55	22	1.05	2.12
white	63	60.00	13.30	40	46.30	18.60
Wilson	23	9.24	5.35	23	5.33	4.48

To analyse these data in StatsDirect first prepare them in four workbook columns and label these columns appropriately. Alternatively, open the test workbook using the file open function of the file menu. Then select effect size from the meta-analysis section of the analysis menu, select the option to use mean, n and sd, and then select the columns ’Exptal. number’, ’Exptal. mean’, Exptal. SD’, ’Control number’, ’Control mean’, ’Control SD’ and ’Trial’ as prompted.

For this example:

Study	J(N-2)	g	Exact 95% CI
1	0.982	0.8261	0.19	1.453	kottke
2	0.9739	0.1712	-0.536	0.8759	levinson
3	0.9915	0.5644	0.0954	1.0303	oliver intensive
4	0.994	0.35	2.945E-17	0.6986	oliver standard
5	0.9739	1.2017	0.3581	2.0267	sulmasy
6	0.9925	0.8804	0.4639	1.2928	white
7	0.9828	0.7924	0.187	1.3893	wilson

Study	N (exptal.)	N (control)	d	Approximate 95% CI
1	27	17	0.8113	0.1812	1.4413	kottke
2	16	15	0.1668	-0.5389	0.8724	levinson
3	25	66	0.5597	0.0923	1.0271	oliver intensive
4	62	66	0.3479	-0.0013	0.6972	oliver standard
5	9	22	1.1703	0.3419	1.9987	sulmasy
6	63	40	0.8738	0.46	1.2876	white
7	23	23	0.7788	0.1794	1.3783	wilson

Fixed effects (Hedges-Olkin)

Pooled effect size d+ = 0.612354 (95% CI = 0.421251 to 0.803457)

Z (test d+ differs from 0) = 6.280333 P < 0.0001

Non-combinability of studies

Cochran Q = 7.737692 (df = 6) P = 0.258

Moment-based estimate of between studies variance = 0.020397

I² (inconsistency) = 22.5% (95% CI = 0% to 67.1%)

Random effects (DerSimonian-Laird)

Pooled d+ = 0.627768 (95% CI = 0.403026 to 0.85251)

Z (test d+ differs from 0) = 5.474734 P < 0.0001

Bias indicators

Begg-Mazumdar: Kendall's tau = 0.238095 P = 0.5619 (low power)

Egger: bias = 1.439076 (95% CI = -2.5809 to 5.459051) P = 0.3997

Here we can say with 95% confidence, assuming a random effects model, that the true size of the effect was at least 0.4 greater for the group who received the educational intervention compared with those who did not. Assuming a fixed effects model a slightly stronger inference could be made about an effect size of 0.42 (the lower confidence limit) but the high inter-study variation makes the fixed effects model less appropriate.

P values

confidence intervals