After I concluded my qualitative content analysis for the poster I presented in CSCW this past October, I still had some hypotheses and impressions bouncing around in my head that were difficult to gauge or inappropriate to gauge from the interpretive method I used. One of these, which I did evaluate in a definite way, made it to the title of my paper: headlines rarely soothe nerves, or, headlines of articles about social media mental health research are pretty negative and alarmist. My coding found that 61% of headlines came to negative conclusions about social media, and an additional 20% were gave no hint of a conclusion (e.g., “How Does Habitual Social Media Use Impact Teens’ Mental Health?”). This probably didn’t surprise anyone who wandered by my poster in the conference hall: most of our general impressions as news consumers corroborate this, as well as prior work about clickbait.

There’s an implied nuance to this conclusion that I didn’t really test: if headlines are (unjustly? unfairly? inaccurately?) negative, are the articles themselves not so negative? This question could be answered by my qualitative analysis, but it’s actually a pretty testable hypothesis by quantitative methods. Take the text of headlines, take the text of articles, and compare them. I took a course this semester titled Computation for Research whose final project gave me the perfect excuse to learn how to conduct a sentiment analysis that could answer this question.

Here is a link to my final writeup, which includes a very in-depth description of my method (including thorough references of the resources I used), as well as the exact findings I came up with. Here is a link to a Google Drive folder with the R file containing my code, and the CSV files of the datasets I used, so you (reader, who in theory knows a lot more or a lot less than me about how to conduct a sentiment analysis) can do it for yourself. I addressed other questions than the sentiment of headlines vs articles: I also attempted to validate my extremely subjective impression that the news articles got more positive when the pandemic started (sensibly, right? if we are trapped by quarantine on social media, we would like to believe it’s not a wholly harmful place to be.) If you read my initial paper and thought, “quantitative methods might be useful for this” at any point, this addendum is for you!

The TLDR of my findings was: yes, headlines are more negative than articles. The margin by which they are more negative varies from inconsequential to significant, depending on what exact function you use. No, there are no sentiment trends overtime – the graphs I generated of sentiment vs time are decidedly chaotic.

If you’re someone who is experienced in sentiment analysis and you think I did something right or wrong, I’m very happy to chat. If you’re not bothered much about methods, but you’re interested in the findings and general topics of this research, I’m equally happy to chat. I’m chronically on my email at faye.kollig@colorado.edu, and the comments of this post are open, too!