BICYCLE HELMET
RESEARCH
FOUNDATION

cyclehelmets.org


Home page

Main topics
News Headlines

Frequently asked Questions
For Policy Makers

Research evidence
Misleading claims
Helmet laws
Analysis

Search Engine

Australia
Canada
New Zealand
UK
USA
Other countries

Full index
Links


BHRF
Policy statement
Register as a supporter
Feedback

Download this page

Publication bias and time-trend bias in meta-analysis of bicycle helmet efficacy

Elvik R. Accident Analysis & Prevention, 2011;43(3):1245-1251.

A re-analysis of
Attewell RG, Glase K, McFadden M. Bicycle helmet efficacy: a meta-analysis. Accident Analysis & Prevention, 2001v33 n3 p345-52. (Attewell, Glase and McFadden, 2001)

and comments on
Thompson DC, Rivara FP, Thompson RS.. Helmets for preventing head and facial injuries in bicyclists (Cochrane Review). Cochrane Database Syst Rev, issue 4, 2002-9 (Thompson, Rivara and Thompson, 2002-9)

Author's abstract

This paper shows that the meta-analysis of bicycle helmet efficacy reported by Attewell, Glase and McFadden, 2001 was influenced by publication bias and time-trend bias that was not controlled for. As a result, the analysis reported inflated estimates of the effects of bicycle helmets. This paper presents a re-analysis of the study. The re-analysis included: (1) detecting and adjusting for publication bias by means of the trim-and-fill method; (2) ensuring the inclusion of all published studies by means of continuity corrections of estimates of effect that rely on zero counts; (3) detecting and trying to account for a time-trend bias in estimates of the effects of bicycle helmets; (4) updating the study by including recently published studies evaluating the effects of bicycle helmets. The re-analysis shows smaller safety benefits associated with the use of bicycle helmets than the original study.

BHRF Commentary

Attewell, Glase and McFadden, 2001 and Thompson, Rivara and Thompson, 2002-9 are meta-analyses of the effects of bicycle helmets. Both have been criticised for bias, such as by Curnow, 2005.

Since publication of Attewell, Glase and McFadden, 2001 new techniques for detecting and adjusting for bias have been established (e.g. Rothstein, Sutton & Borenstein, 2005).

In this paper, Rune Elvik from the Norwegian Institute of Transport Economics uses those techniques to re-analyse Attewell, Glase and McFadden, 2001 and also comments on a conflict of interest by the authors of Thompson, Rivara and Thompson, 2002-9.

Much of Elvik's paper is detailed statistical analysis. This page summarises the outcomes.

DEFINITIONS

For explanations of the following terms used in this Commentary

Publication bias
Time-trend bias
Zero-count bias
Fixed effect model
Random effects model

see: Statistical research: some concepts and definitions

 

Although Attewell, Glase and McFadden, 2001 acknowledged the possibility of publication bias, they thought it unlikely to have influenced their results. In re-analysing the meta-analysis, Elvik found evidence of publication bias with regard to the assessment of the effect of bicycle helmets in reducing head injury, brain injury, facial injury and fatal injury. It was not possible to test for publication bias in estimates of neck injury as the data was too limited.

Elvik's readjustment for publication bias showed a large effect in particular aspects of facial injury but only a very small effect on summary estimates. He therefore concludes that Attewell, Glase and McFadden, 2001 were correct in their conjecture that publication bias did not greatly influence their conclusions.

Time-trend bias

Due to technological innovation, safety measures usually become more effective over time. However, this does not appear to have been the case with bicycle helmets. Recent studies show significantly smaller benefits of cycle helmets than older studies. Attewell, Glase and McFadden, 2001 do not acknowledge this effect and therefore give a too optimistic impression of new helmets.

Zero-count bias

Attewell, Glase and McFadden, 2001 included studies in which one of the four numbers used to calculate the odds ratio was zero. This suggests a larger effect for bicycle helmets than is justified.

Addition of more recent papers

To update the meta-analysis, Elvik added four more recent studies to those used by Attewell, Glase and McFadden, 2001:

Hausotter, 2000; Hansen, Engesaeter and Viste, 2003; Heng et al, 2006; Amoros, Chiron, Ndiaye and Laumon, 2009.

Estimates based on the recently published studies show much smaller effects of bicycle helmets on head injury and facial injury than the original study. In fact, in a random-effects model, there is a statistically non-significant tendency for the wearing of bicycle helmets to be associated with an increase in the risk of injury. As far as neck injury is concerned, the tendency found in the original study for the risk of injury to increase when a helmet is worn is confirmed.

One reason for the reduction in protective effect shown in recent papers is that helmet design has changed. Early studies included mostly hard-shell helmets but now more helmets have soft shells, which have been shown to be less effective.

In general, the reanalysis of Attewell, Glase and McFadden, 2001 shows a modest benefit from bicycle helmets. However, based on the more recent papers only, the benefit vanishes entirely. Taking all the papers together, the best estimate is for helmets to reduce injury to the head, face or neck by 15%.

A conflict of interest in Cochrane Review

Elvik notes that Thompson, Rivara and Thompson, 2002-9 used more restrictive inclusion criteria than Attewell, Glase and McFadden, 2001. 4 of the 7 studies included in their review had been undertaken by the authors themselves, thus judging their own work to be worthy of inclusion but not 8 other studies in which they had not been involved. Elvik contrasts the assessment of quality by Thompson, Rivara and Thompson, 2002-9 with that of Towner et al, 2002 for the same studies, finding that it is not at all clear that the omitted studies were of lower quality than the studies included.

Elvik cites Littell, Corcoran and Pillai, 2008 to suggest that the authors of Thompson, Rivara and Thompson, 2002-9 had a conflict of interest in carrying out the Cochrane Review (which has not been declared). He criticises the omission of studies from the Review instead of developing a quality score for each study and using that in a sensitivity analysis, which would have been a more transparent process.

Disparity between research and helmet law outcomes

Elvik notes that while, on the one hand, studies have predicted large benefits from the use of cycle helmets, large increases in helmet use brought about by helmet laws have not always shown a clear decline in head injuries to cyclists. This could be due to selective recruitment – that the most cautious and safety-minded cyclists with a lower rate of accident involvement are the first to start wearing helmets – or because of behavioural adaptation (or risk compensation), whereby helmeted cyclists feel safer and thereby ride less safely.

Conclusion

This is a powerful paper that discredits some of the most widely cited pro-helmet research by applying rigorous tests for bias and conflict of interest. It also notes that more recent research does not show any clear benefit from helmet use and that the failure of post-law research to find the predicted benefits from helmet use may be explained by behaviour patterns that could be an artefact of promoting and mandating helmet use.

References

Amoros, Chiron, Ndiaye and Laumon, 2009

Amoros E, Chiron M, Ndiaye A, Laumon B, 2009. Effet du casque sur les blessures à la tête, à la face et au cou. INRETS, Lyon Rapport UMRESTTE No0912.

Attewell, Glase and McFadden, 2001

Attewell RG, Glase K, McFadden M, 2001. Bicycle helmet efficacy: a meta-analysis. Accident Analysis & Prevention 2001-05 v33 n3 p345-52.

Curnow, 2005

Curnow WJ, 2005. The Cochrane Collaboration and bicycle helmets. Accident Analysis & Prevention 2005;37(3):569-573.

Hansen, Engesaeter and Viste, 2003

Hansen KS, Engesaeter LB, Viste A, 2003. Protective effect of different types of bicycle helmets. Traffic Inj Prev 2003;4(4):285-290.

Hausotter, 2000

Hausotter W, 2000. Fahrradunfälle mit und ohne Fahrradhelm. Versicherungsmedizin 52:28-32.

Heng et al, 2006

Heng KWJ, Lee AH, Zhu S, Tham KY, Seow E, 2006. Helmet use and bicycle-related trauma in patients presenting to an acute hospital in Singapore. Singapore Medical Journal 2006;47(5):367-372.

Littell, Corcoran and Pillai, 2008

Littell JH, Corcoran J, Pillai V, 2008. Systematic reviews and meta-analysis. Oxford University Press ISBN13: 9780195326543.

Rothstein, Sutton & Borenstein, 2005

Rothstein H, Sutton AJ, Borenstein M, 2005. Publication bias in meta-analysis. Prevention, assessment and adjustments. John Wiley & Sons .

Thompson, Rivara and Thompson, 2002-9

Thompson DC, Rivara FP, Thompson RS., 2002. Helmets for preventing head and facial injuries in bicyclists (Cochrane Review). Cochrane Database Syst Rev issue 4, 2002.

Towner et al, 2002

Towner E, Dowswell T, Burkes M, Dickinson H, Towner J, Hayes M, 2002. Bicycle helmets - a review of their effectiveness: a critical review of the literature. Department for Transport Road Safety Research Report 30.