Download the original PDF (5 pages, 115 KB)

Contents

In 2009, The Cochrane Collaboration published a review of “Social norms interventions to reduce alcohol misuse in University or College students.” The review included 22 “randomized control trials with individual or cluster designs” published between 2000 and 2008. In their “Plain Language Summary”, the authors state:

Two large studies showed contradictory results for a social marketing campaign. Only a small number of good quality studies were available for many of the outcomes and analyses, and most of the studies were from the USA.1 (emphasis added)

This balanced statement about one specific segment of the literature on social norms marketing interventions has unfortunately been summarized in an NIAAA statement as:

Social norms approaches, which focus on correcting student misperceptions about how much their peers drink, have been widely implemented. However, new research shows that these approaches … are much less effective as part of campuswide campaigns.2 (emphasis added)

The actual wording in the body of the Cochrane article is “Results were inconclusive for the effectiveness of a social norms intervention delivered in the context of a social marketing campaign.”1 (p. 38; emphasis added) The authors further explain that only two studies of social norms marketing campaigns met the criteria for inclusion: both studies were by deJong and colleagues3,4 and the results appeared to be contradictory. The authors of the Cochrane Review conclude, “… further research is needed to test definitively the effectiveness of these delivery modes.”1 (p. 39)

In the years since that statement was written, further research has been done, including by deJong and colleagues to investigate the apparently contradictory results of their two randomized control trials.5 In an Impact Essay written for Health Communications, deJong offered this perspective on the apparent replication failure, on other available evidence and on the current challenges to the field:

…looking at all 32 institutions, we calculated the density of on-premise alcohol outlets (bars, taverns, restaurants) within a three-mile radius of campus. We found that the SNM campaigns worked in campus communities with low outlet density, but failed in those with high outlet density. This explained the contradictory results for the two studies. In the first, with effective SNM campaigns, 13 of the 18 colleges were located in low-density communities. In the second, a replication failure, 11 of the 14 colleges were located in high-density communities.

In both studies, the colleges received only modest funding for their campaigns (only $2,000 to $3,650 per year), leaving open the possibility that better resourced campaigns may have succeeded in the high-density communities. In fact, a later study at the University of Virginia did show that a large, highly visible SNM campaign can overcome an intense drinking culture and reduce alcohol-related problems (Turner, Perkins, & Bauerle, 2008).6

Despite the early controversy, the case for social norms marketing seems pretty solid at this point. The job now is to train college officials on how to conduct a proper campaign that relies on well-executed student surveys, builds on extensive message testing, employs a mix of media channels, and provides varied communications to avoid message fatigue. A difficulty these campaigns will always face is student incredulity. After all, students have been told—by alcohol advertising, movies, and campus lore—to believe that frequent heavy drinking is the norm and not the exception. Innovative strategies, such as using real-time data collection in classroom settings, are being developed to meet that challenge.7

Further, it is necessary to consider the difference between evaluation of efficacy and evaluation of effectiveness. The differentiation provided in a Technical Review prepared for the Agency for Healthcare Research and Quality (AHRQ) in the U.S. Department of Health & Human Services is instructive:

Randomized controlled trials (RCTs) are the gold standard in evaluating the effects of treatments. … Clinicians and policymakers often distinguish between the efficacy and the effectiveness of an intervention. Efficacy trials (explanatory trials) determine whether an intervention produces the expected result under ideal circumstances. Effectiveness trials (pragmatic trials) measure the degree of beneficial effect under “real world” clinical settings.8

The Cochrane review found the evidence of efficacy -- that is, RCTs conducted under “ideal circumstances” -- to be lacking for social norms marketing campaigns. This is more a consequence of the difficulty of obtaining funding for and conducting RCTs of population-based interventions, than an indictment of the intervention being assessed.

Additionally, as noted in the AHRQ Technical Review, RCTs are the gold standard for evaluating “treatments.” Social norms marketing interventions are most definitively NOT treatments but rather are population based preventive interventions, classically characterized in epidemiology as primary prevention and characterized by NIAAA as universal prevention.9 Most social norms marketing interventions, by far, are conducted in “real world” circumstances, funded either internally or externally as preventive interventions within an environment of synergistic and competing efforts, rather than as research studies, much less as randomized controlled trials.

We question the wisdom of total reliance on randomized controlled trials to evaluate the potential impact of social norms marketing. Many grading systems are being adopted by AHRQ, CDC, and WHO that weigh the total body of evidence, including observational studies, when studying interventions. Though randomized trials begin as high-quality evidence, they can be rated down by limitations in methodology. Similarly, observational studies begin as low-quality evidence and may be rated up by a large magnitude of effect and/or a dose-response relationship.10 Evidence of large magnitude effects and dose-response relationships as well as evidence from similar populations and issues should not be discounted when judging any method of health education. There is an accumulation of such evidence related to the relevance of perceived social norms to a wide variety of health risk behaviors and pro-social behaviors, including many studies that evaluate intervention effectiveness.6,11-23

Not only is further research needed, as suggested by the authors of the Cochrane review article, but the research paradigm for assessing effectiveness needs to be expanded beyond randomized controlled trials. Although such an expansion presents multiple challenges, it is the only route to obtaining a true evaluation of effectiveness in the multivariate context in which most social norms marketing interventions are delivered. Perhaps nothing serves better to emphasize the challenges to evaluating effectiveness in a multivariate “real world” context than the lessons learned through deJong and colleagues’ resolution of the apparent replication failure in their RCTs.

As suggested in a number of government publications on program evaluation,24,25 at NSNI we have focused on assisting sites using social norms interventions in designing and implementing evaluation strategies that will provide information on how well their particular implementation adheres to the principles of social norms marketing and is associated with desired outcomes in their setting.26,27 We have found this pragmatic approach to be very valuable to those responsible for such interventions and their stakeholders.

References

  1. Moreira MT, Smith LA, Foxcroft D. Social norms interventions to reduce alcohol misuse in University or College students. Cochrane Database of Systematic Reviews 2009, Issue 3. Art. No.: CD006748. DOI: 10.1002/14651858.CD006748.pub2
  2. http://pubs.niaaa.nih.gov/publications/CollegeFactSheet/CollegeFactSheet.pdf
  3. DeJong W, Schneider SK, Towvim LG, Murphy MJ, Doerr EE, Simonsen NR, et al. A multisite randomized trial of social norms marketing campaigns to reduce college student drinking. J Stud Alcohol Drugs 2006;67(6):868–79.
  4. DeJong W, Schneider SK, Towvim LG, Murphy MJ, Doerr EE, Simonsen NR, et al. A multisite randomized trial of social norms marketing campaigns to reduce college student drinking: A replication failure. Substance Abuse 2009;30:127-140.
  5. Scribner RA, Theall KP, Mason K, Simonsen N, Schneider SK, Towvim LG, DeJong W. Alcohol prevention on college campuses: the moderating effect of the alcohol environment on the effectiveness of social norms marketing campaigns. J Stud Alcohol Drugs 2011 Mar; 72(2):232-9.
  6. Turner J, Perkins HW, Bauerle J. Declining negative consequences related to alcohol misuse among students exposed to a social norms marketing intervention on a college campus. J Am Coll Health 2008; 57(1):85-93.
  7. DeJong W. Social norms marketing campaigns to reduce campus alcohol problems. Health Commun. 2010 Sep; 25(6-7):615-6.
  8. http://www.ahrq.gov/downloads/pub/evidence/pdf/efftrials/efftrials.pdf
  9. http://pubs.niaaa.nih.gov/publications/Social/Module3Prevention/module3.html
  10. Falck-Ytter Y, Schunemann H, Guyatt G. AHRQ Series Commentary1: Rating the evidence in comparative effectiveness reviews. J. Clinical Epidemiology 2010; 63:474-475.
  11. Keller A. Multi-site, multi-year evaluation of social norms marketing campaigns. Listening for Change: 2011 National Conference on Health Communication, Marketing, and Media. U.S. Department of Health and Human Services, Center for Disease Control and Prevention, August 2011, Atlanta, GA.
  12. Flemming B & Holmberg L (2011) The ripple effect: a randomized trial of a social norms intervention in a Danish middle school setting. Journal of Scandinavian Studies in Criminology and Crime Prevention, 13 April 2011; Online publication, DOI: 0.1080/14043858.2011.561619.
  13. Neighbors C, Jensen M, Tidwell J, Walter T, Fossos N., & Lewis MA. Social-norms interventions for light and nondrinking students. Group Processes & Intergroup Relations 2011. 1-19.
  14. Perkins HW, Craig DW & Perkins JM. Using social norms to reduce bullying: A research intervention among adolescents in five middle schools. Group Processes & Intergroup Relations, 7 April 2011 Online publication.
  15. Perkins HW, Linkenbach JW, Lewis MA, & Neighbors C. (2010). Effectiveness of social norms media marketing in reducing drinking and driving: A statewide campaign. Addictive Behaviors 2010; 35(10):866-874.
  16. Linkenbach J & Perkins HW. Montana's MOST of Us Don't Drink and Drive campaign: A social norms strategy to reduce impaired driving among 21- to 34-year-olds. National Highway Traffic Safety Administration (NHTSA) Publication 2006: DOT HS 809 869.
  17. Perkins, H.W. & Craig, D.W. (2006). A successful social norms campaign to reduce alcohol misuse among college student-athletes. J Stud Alcohol 2006;67:880-889.
  18. Fugas, C. S., Meliá, J. L., & Silva, S. A. (2011). The “is” and the “ought”: How do perceived social norms influence safety behaviors at work? Journal of Occupational Health Psychology, 16(1), 67-79. doi:10.1037/a0021731.
  19. Gidycz, C. A., Orchowski, L. M., & Berkowitz, A. D. (2011). Preventing sexual aggression among college men: An evaluation of a social norms and bystander intervention program. Violence Against Women, doi:10.1177/1077801211409727.
  20. Perkins, J.M.; Perkins, H. W.; and Craig, D.W. (2010). Misperceptions of peer norms as a risk factor for sugar-sweetened beverage consumption among secondary-school students. Journal of the American Dietetic Association, 110 (12), 1916-1921.
  21. Primmer, E. and Karpinnen, H. (2010). Professional judgment in non-industrial private forestry: Forester attitudes and social norms influencing biodiversity conservation. Forest Policy and Economics, 12 (2), 136-146.
  22. Traxler, C. (2010). Social norms and conditional cooperative taxpayers. European Journal of Political Economy, 26 (1), 89-103.
  23. Zhang, X., Cowling, D. W., & Tang, H. (2010). The impact of social norm change strategies on smokers’ quitting behaviors. Tobacco Control, 19 (Supplement 1), 51-55
  24. http://www.gao.gov/assets/590/588146.pdf
  25. http://www.cdc.gov/eval/resources/index.htm
  26. http://www.socialnorms.org/Resources/snmodelevaluationtool.php
  27. Keller A, Bauerle JA. Using a logic model to relate the strategic to the tactical in program planning and evaluation: An illustration based on social norms interventions. American Journal of Health Promotion 2008;24(2):89-92.

Download the original PDF (5 pages, 115 KB)