Cochrane review of bicycle helmets

Cochrane Review: Helmets for preventing head and facial injuries in bicyclists

Thompson RS, Rivara FP & Thompson DC. Cochrane Database of Systematic Reviews, The Cochrane Library.

Full source document (Note: Access to the Cochrane database may be denied from some countries)
Authors' abstract

This page gives information about this paper and summarises the key criticisms.

For a more detailed critique please see here

Introduction

The Cochrane database has established a reputation as a key resource for medical research. This particular review, however, has attracted considerable criticism. Some of this criticism is included with the reviewers' paper on the Cochrane database and thus forms an integral part of the review. However, submissions were sometimes shortened and summarised in a way that omits key concerns. See here for the full version of one detailed critique (BHRF, 1243).

Published peer criticism

Elvik, 2011 has criticised the criteria used for the inclusion of studies in this review, noting that most of the included studies were the work of the reviewers themselves. Citing criteria estalished by Littell, Corcoran and Pillai, 2008, Elvik suggests that the authors had a conflict of interest in carrying out the review which they have not declared. In considering the assessment of quality of the same papers used by Towner et al, 2002, Elvik, 2011 finds that it is not at all clear that the omitted studies were of lower quality than the studies included. A more transparent process would have included all the papers together with the publication of quality scores and a sensitivity analysis.

As part of his re-analysis of Attewell, Glase and McFadden, 2001, Elvik, 2011 considered the same studies used in the Cochrane Review and also more recently published studies. Later studies show no net benefit from helmets with regard to injuries to the head, face and neck.

Curnow, 2005 examines the Cochrane Review in detail. He concludes that the review takes no account of scientific knowledge of the types and mechanisms of brain injury. It provides, at best, evidence that hard-shell helmets, now rarely used, protect the brain from injury consequent upon damage to the skull. The review therefore is not a reliable guide to the efficacy of helmets and to interventions concerning their use.

Studies on which the review is based

Injury patterns in cyclists attending an accident and emergency department: a comparison of helmet wearers and non-wearers.
Maimaris C, Summers CL, Browning C, Palmer CR. BMJ, 1994;308(6943):1537-1540.
See: Commentary

The effectiveness of bicyclist helmets: a study of 1701 casualties.
McDermott FT, Lane JC, Brazenore GA, Debney EA. Journal of Trauma, 1993;34(6):834-845.
See: Commentary

Effectiveness of bicycle helmets in preventing head injury in children.
Thomas S, Acton C, Nixon J, Battistutta D, Pitt WR, Clark R. BMJ, 1994;308:173-176.
See: Brief Notes

Effectiveness of bicycle safety helmets in preventing head injuries: a case-control study.
Thompson DC, Rivara FP, Thompson RS. JAMA, 1996;276(24):1968-1973.
See: Commentary

A case-control study on the effectiveness of bicycle safety helmets.
Thompson RS, Rivara FP, Thompson DC. New England Journal of Medicine, 1989;320:1361-1367.
See: Commentary

A case-control study on the effectiveness of bicycle safety helmets in preventing facial injury.
Thompson DC, Thompson RS, Rivara FP, Wolf ME. American Journal of Public Health, 1990;80(12):1471-1474.

Effectiveness of bicycle safety helmets in preventing serious facial injury.
Thompson DC, Nunn ME, Thompson RS, Rivara FP. JAMA 1996a;276(24):1974-1975.

Summary of criticisms

Principal criticisms of the review are:

The review is not independent. Four of the seven papers selected for inclusion were the work of the reviewers themselves and their data dominate the analysis, comprising 77% of the cyclists on whom the review is based. Furthermore, these four papers are based on only two data sets and have themselves been much criticised for fundamental methodological shortcomings (BHRF, 1068).
Only case-control studies were considered for inclusion, although non-randomised studies of this type are acknowledged to be prone to bias because of the difficulty in controlling for the many independent variables (BHRF, 1052).
The paradox presented by the failure of other types of studies (e.g. whole population and time-series data) to show any benefit from large increases in helmet use - let alone the substantial benefits predicted by the studies reviewed - is left unstated and unaccounted for (BHRF, 1096).
The authors are dismissive of the possibility of risk compensation. However, it has subsequently been demonstrated that child cyclists often ride more riskily and suffer more crashes when wearing a cycle helmet (Mok et al, 2004) and that adults are more likely to ride on busier roads if helmeted (Gregory, Inwood and Sexton, 2003).
Similarly no consideration is given to rotational injuries, which dominate the most serious injuries. Helmets have not been shown to mitigate rotational brain injury and there is evidence they may increase the risk and/or severity of rotational injury.(BHRF, 1039).
Claims are accepted of efficacy for which no plausible mechanism exists (e.g. the prevention of mid-face injuries), where the reviewed data set is too small to make reliable inference (e.g. the result of collisions with motor vehicles), and which would not be possible even if helmets prevented all head injuries (e.g. an increase of 35% in cyclists wearing helmets leading to 66% fewer head injuries).
There is misleading interpretation of 'odds ratio', which is used interchangeably in the review with 'percentage reduction in head injuries'. This exaggerates the predicted benefit of helmets and masks the fact that studies of this type are not truly predictive, being essentially the authors' estimate of what proportion of the observed differences between two groups can be assigned to a single factor. Furthermore, the reviewed paper showing the least benefit from helmets is omitted from computation of odds ratio, thus again exaggerating benefit.

References

Attewell, Glase and McFadden, 2001

Attewell RG, Glase K, McFadden M, 2001. Bicycle helmet efficacy: a meta-analysis. Accident Analysis & Prevention 2001-05 v33 n3 p345-52.

BHRF, 1039

Cycle helmets and rotational injuries. .

BHRF, 1052

Contradictory evidence about the effectiveness of cycle helmets. .

BHRF, 1068

A case study of the effectiveness of bicycle helmets. .

BHRF, 1096

Helmet laws: what has been their effect?. .

BHRF, 1243

Helmets for preventing head and facial injuries in bicycles - critique. .

Curnow, 2005

Curnow WJ, 2005. The Cochrane Collaboration and bicycle helmets. Accident Analysis & Prevention 2005;37(3):569-573.

Elvik, 2011

Elvik R, 2011. Publication bias and time-trend bias in meta-analysis of bicycle helmet efficacy: A re-analysis of Attewell, Glase and McFadden, 2001. Accident Analysis & Prevention 2011;43(3):1245-1251.

Gregory, Inwood and Sexton, 2003

Gregory K, Inwood C, Sexton B, 2003. Cycle helmet wearing in 2002. Transport Research Laboratory Report 578.

Littell, Corcoran and Pillai, 2008

Littell JH, Corcoran J, Pillai V, 2008. Systematic reviews and meta-analysis. Oxford University Press ISBN13: 9780195326543.

Mok et al, 2004

Mok D, Gore G, Hagel B, Mok E, Magdalinos H, Pless IB, 2004. Risk compensation in children's activities: A pilot study. Paediatr Child Health 2004;9(5):327-330.

Towner et al, 2002

Towner E, Dowswell T, Burkes M, Dickinson H, Towner J, Hayes M, 2002. Bicycle helmets - a review of their effectiveness: a critical review of the literature. Department for Transport Road Safety Research Report 30.