Should you disable comments?

A few months ago, the aptly named science magazine Popular Science decided to shut off comments on their new articles. Their rationale for doing so was to prevent a ‘fractious minority’ from altering the focus or perceived validity of accepted scientific topics. The folks at Popular Science may have been on to something. It turns out that  even positive comments can do more harm than good.

University of Pennsylvania researchers Rui Shi, Paul Messaris and Joseph Cappella tested the effect comments have on anti-smoking public service announcements (PSAs). The researchers selected eight PSAs, four of which had previously been rated as highly effective (strong) and four of which had proven to be ineffective (weak). They then combined each PSA with comments that were positive (pro-PSA and anti-smoking), or negative (anti-ad, anti-quitting) and civil (no insults or profanity) or uncivil. Thus, a PSA could be strong, with positive but uncivil comments, or any other combination of the three variables. There were also control PSAs with no comments.

The authors then recruited close to 600 current smokers to watch the anti-smoking PSAs. Before and after watching the ads, the participants were asked about their readiness to quit smoking. 

As expected, strong ads were more effective than weak ads. People who were more highly motivated to quit smoking found the positive comments more convincing than the negative comments, but people who were not ready to quit were not swayed by either set of comments. 

What was more interesting was that the PSAs that lacked comments of any kind were more effective than PSAs with positive comments. And if comments were present, the fewer of them people read the more effective they found the PSA.

One reason for this may be that comments, even ones that agree with the message of the PSA, are distracting. This was confirmed by testing participants’ memories of details of the PSA they had watched. People didn’t remember a PSA as well when it was followed by comments.

This suggests that if you really want to get your message out, you should make everyone else shut up. 

Rui Shi, Paul Messaris, & Joseph N. Cappella (2014). Effects of Online Comments on Smokers' Perception of Antismoking Public Service Announcements Journal of Computer-Mediated Communication DOI: 10.1111/jcc4.12057.