Mathew Kiang (.com)Yes, with one "t".

· 5 min read

Student’s Tay Distribution

Taylor Swift has recorded 9 albums, each of them (except the most recent) has gone multi-platinum. In total, she has sold over 200 million records, won 10 Grammy’s, an Emmy, 32 AMA’s, and 23 Billboard Music Awards. Not bad for somebody who just turned 31.

This year, she’s managed to release two albums — they’re both very good. However, I noticed there seemed to be more profanity than I had remembered on her older albums. Here, I’ll use tidytext to see if she has actually increased her rate of profanity or if I’m simply misremembering things.

Descriptives

We’ll begin with some simple descriptives. How many words (y-axis) does each album (x-axis) have? Note, the x-axis uses albums as references but ticks are are spaced temporally. Below, we show all words (green) as well as after removing stop words (orange) and counting only distinct words (purple). It appears like the number of distinct words has more or less stayed the same throughout the discography while the number of total words increased a bit and then decreased.

A line chart titled Total words by type. The x-axis names nine albums with their release dates, spaced by date rather than evenly: Taylor Swift (Oct 2006), Fearless (Nov 2008), Speak Now (Oct 2010), Red (Oct 2012), 1989 (Oct 2014), Reputation (Nov 2017), Lover (Aug 2019), Folklore (Jul 2020) and Evermore (Dec 2020). The y-axis is word frequency, 0 to about 7,000. Three lines: all words in green, rising from about 4,300 to a peak near 7,200 on Reputation then dropping to about 4,900 on Folklore; excluding stop words in orange, tracking the same shape roughly 1,500 lower; and distinct words in purple, nearly flat the whole way at about 800 to 1,100.

Perhaps this increase in the middle is explained to having more tracks on each album? Below we plot the average number of words per track (green) and the average number of distinct words per track (purple). Between the two plots, I think a likely explanation for the increase in words in the middle albums is due to more repetition and not due to more tracks.

A line chart titled Average words per track, over the same nine albums from Taylor Swift (Oct 2006) to Evermore (Dec 2020), y-axis word frequency 0 to about 350. Words per track in green climbs from about 215 to a peak near 350 on Reputation, then falls to about 220 on Folklore and 240 on Evermore. Distinct words per track in purple is nearly flat throughout at roughly 45 to 70, so the green line's rise and fall is not matched by any change in vocabulary.

Sentiment

How has sentiment changed over time? That is, have albums gotten more or less sad or happy? Is there a pattern such that the albums get more (or less) happy (or sad) as the album progresses? Below, I plot the average sentiment (y-axis) for each track (x-axis) by album (facets). The lines are fitted generalized additive models.

A three-by-three grid of scatter plots titled Average track sentiment by album, one panel per album in release order: Taylor Swift, Fearless and Speak Now on the top row, Red, 1989 and Reputation next, Lover, Folklore and Evermore last. Each plots one grey dot per track against track position, with average track sentiment on the y-axis from -0.50 to 0.50, a dotted line at zero, and a fitted smooth curve. Nearly every panel's curve is flat and close to zero. The two exceptions are 1989, where the first track sits at about 0.47 and two mid-album tracks near -0.30 and -0.47 bend the curve into a deep U, and Fearless, which has a shallower version of the same shape.

The sentiment across albums appears more or less flat (i.e., tracks do not get more or less positive as the album progresses). The exception is 1989, where the first track is “Welcome to New York” and “welcome” is classified as “positive”. In addition, the sixth track is “Shake it off” and “hater” (repeated in the chorus) is classified as “negative”. Excluding these two tracks results in a flat line.

Given there appears to be no relationship with average sentiment and track position, maybe albums themselves (across all lyrics and tracks) have different distributions of sentiment? Below is the density of sentiment for individual lyrics (i.e., lines) by album.

A three-by-three grid of density curves titled Distribution of lyric sentiment by album, one panel per album in release order: Taylor Swift, Fearless and Speak Now on the top row, Red, 1989 and Reputation next, Lover, Folklore and Evermore last. Each x-axis runs from -1 marked Negative through 0 marked Neutral to +1 marked Positive, with a dotted line at zero. Every album is a tall narrow spike at zero — most lyrics are neutral — with low bumps to either side. Reputation's spike is by far the narrowest and tallest and its shoulders the flattest. Lover, Folklore and Evermore have the broadest shoulders on both sides.

Most lyrics (in all albums) are fairly neutral. Reputation is especially pointy (yes, that’s the statistical term), but the last three albums have a bit more mass on both sides of 0 than previous albums.

Profanity

Ok. So now the main question. Has profanity increased over time? Below is the rate of profanity per 1,000 words (y-axis) over time/album (x-axis) by word (colors).

A line chart titled Taylor Swift's Profanity Rate, 2006 to 2020, over the same nine albums from Taylor Swift (Oct 2006) to Evermore (Dec 2020). The y-axis is profanity rate as curse words per 1,000 words after removing stop words, 0 to about 7. Five lines, one per word, censored as they appear on the chart: bitch in red, [god]damn in blue, f*ck[ed] in green, hell in purple and sh*t in orange. All five sit at or near zero through the first three albums, apart from a single blue point at about 1.9 on the 2006 debut. From Red onward small rises appear, none above about 1.4 until Folklore. On the last two albums every line jumps: blue reaches about 6.8, orange about 3.0 and green about 1.3 — far above anything earlier. A footnote reads: after removing stop words.

Yes, there’s more profanity in the recent albums than in the older albums. Maybe 2020 is getting to TSwift too? Check out Folklore and Evermore if you haven’t already.

(Code is here.)