Hunting the Friendship Theme: Three Methods, One Album

music

nlp

python

embeddings

The LLM labelers found friendship in only 2 of 107 Carly Rae Jepsen songs. A fan knew better — and topic models, embedding probes, and a keyword audit back the fan.

Published

July 13, 2026

A note on how this was made

Built by directing an AI research assistant.
I set the hypothesis, the chorus-control requirement, and the demand for placebo baselines. Claude wrote and ran the models.
I approved the probe texts and sent results back for rework.

The previous post had two Claude models tag every Carly Rae Jepsen song against a 13-theme vocabulary, keeping only tags both agreed on.
It worked: the models agreed, and most themes landed where a listener would put them.
Friendship — the theme that got me interested in the first place — survived consensus in only 2 of 107 songs.
The word “friend” alone appears in 19 songs. Boundary songs (The One, Fever, Body Language) got zero tags.
This post re-attacks the catalog with methods that don’t share the labelers’ assumptions: topic models, then embedding probes.

The corpus

113 catalog songs (Kiss onward), 107 unique after remix dedup.
Repeated lines removed within each song (~46% of all lines), so a chorus sung five times doesn’t get five times the weight.

Method 1: topic models

Both models run at k=8 on the chorus-deduplicated text.
Each has its own way to keep shared pop vocabulary (love, know, baby, heart) from swamping every topic:
- LDA on word counts, with document-frequency pruning — words in more than a quarter of songs are dropped.¹
- NMF on TF-IDF, where IDF weighting does the same job softly.
Topics paired by matching each LDA topic to its nearest NMF topic; labels drawn from both word lists.

Label	LDA topic (top words)	NMF topic (top words)
Wanting a friend	friend, real, feels, thinking, getting, things, good, alright, end, used, change, life	want, way, need, lost, cut, friend, real, just, feeling, hold, think, fall
Run away together	away, come, run, stay, body, little, thinkin, hands, open, head, close, hold	away, run, stay, thinkin, body, stuck, hands, baby, party, sleepin, touch, bout
Really liking a boy	touch, really, boy, sound, try, tell, mind, afraid, look, words, room, getting	like, say, really, know, feel, got, boy, just, did, tell, close, touch
Deciding on the beat	come, tell, making, lost, believe, good, day, thinking, boy, heartbeat, pick, speeding	thinking, come, making, know, sure, beat, dance, believe, heart, knees, regrets, bout
Big crazy love	hold, gimme, cut, warm, boy, falling, stand, bad, crazy, really, hard, hurt	love, know, baby, crazy, open, feel, girl, little, gimme, hold, heart, head
Turning it around tonight	turn, good, kiss, wish, talk, higher, somebody, high, alright, real, did, breaking	turn, alright, new, tonight, hold, play, wrong, tight, just, melt, outside, comes
Goodbyes & moving on	goodbye, body, scared, lost, kiss, city, heartbeat, devotion, feels, come, care, words	time, good, goodbye, try, like, matter, night, free, needed, mind, talk, live
The leftovers (no clean pair)	believe, girl, rest, open, said, told, door, sleep, turn, calling, wrong, hold	afraid, getting, right, feels, baby, close, happy, life, kinda, real, hypnotized, words

Both models produce a friend topic unprompted. Songs loading on LDA’s version: Your Type, Body Language, Real Love, Let’s Be Friends, After Last Night, Weekend Love.
The friend pair only overlaps on two words — friend and real. LDA’s topic is about friendship; NMF’s is about wanting, with friend inside it.
First sign of why a codebook defining friendship as “platonic bonds, companionship outside romance” missed it: friendship vocabulary co-occurs with wanting vocabulary in this catalog.

Method 2: embedding probes, with placebos

Partly a hedge against the topic models not working. They worked — but probes can do something topic models can’t: ask directly about a theme instead of waiting for it to emerge.
Songs split into sections (chorus kept once): 760 chunks, embedded with a local MiniLM encoder.
Short probe texts describing each theme embedded in the same space; chunks ranked by cosine similarity.
Three friendship probes, because “friendship” isn’t one thing:
- platonic — friendship as its own kind of love, loyalty that isn’t romantic
- friend-lover boundary — falling for a friend, insisting “we’re just friends” when it’s clearly more
- refuge — friends as comfort when romance fails, dancing with friends instead of a lover
Plus the 13 original codes and three placebo probes (war, religion, money) as the noise floor.
Every score z-scored within its own probe — raw cosines aren’t comparable across probes.

Chunk-similarity distributions per probe. The placebos (red) define the noise floor; all three friendship probes (blue) sit above it.

What the friendship probes found

Top 25 songs by best friendship-probe z-score. Red = the 2 LLM-consensus songs; orange = lyrics contain “friend”; blue = neither.

Both LLM-consensus songs are recovered: Let’s Be Friends (#4), Boy Problems (#11).
Keyword songs the labelers skipped rank at the top: Tonight I’m Getting Over You (“we’re not lovers, but more than friends”), Beautiful (“just friends, the beginning or the end”), The One, Your Type — the contested-boundary songs.
New candidates (no “friend” keyword, no LLM tag): Party for One (friendship 3.17 vs best placebo 1.27), Surrender My Heart (2.78 vs 2.58), After Last Night (3.10 vs 3.17). All three score via the refuge probe.
Of the three probe variants, refuge edges the others — best-scoring for 38 of 107 songs and driving 10 of the top 25 — though the three are close (friend-lover boundary 36 and 9, platonic 33 and 6).

How friendship is discussed: mostly, it hurts

Two contrast probes (friendship-as-joy vs. friendship-as-pain), scored on each song’s most friendship-like section.

The top 25 friendship songs, colored by sentiment contrast. Red = painful, blue = positive.

Friendship writing leans painful.
Boy Problems is the extreme (−3.15 — “what’s worse, losing a lover or losing your best friend?”).
Most of the top ten sit on the red (painful) side.
Warm friendship songs — Party for One (+1.09), Real Love, Shy Boy, Weekend Love — rank lower, skew late-catalog.

Song	Album	Friendship z	Best placebo z	Sentiment
Tonight I’m Getting Over You	Kiss	4.10	1.54	−0.14
Beautiful	Kiss	3.92	1.15	−1.22
The One	E·MO·TION: Side B	3.36	2.91	+0.34
Let’s Be Friends	Dedicated Side B	3.22	2.48	−0.46
Party for One	Dedicated	3.17	1.27	+1.09
Anything to Be With You	The Loveliest Time	3.14	3.55	−0.90
After Last Night	The Loveliest Time	3.10	3.17	−0.81
Lost in Devotion	E·MO·TION	3.10	2.14	+0.40
Your Type	E·MO·TION	3.08	0.94	+0.21
Happy Not Knowing	Dedicated	2.94	3.99	+0.26
Boy Problems	E·MO·TION	2.92	1.63	−3.15
Surrender My Heart	The Loneliest Time	2.78	2.58	−0.03

Six of the top 25 — Real Love, Happy Not Knowing, Keep Away, Anything to Be With You, Weekend Love and After Last Night — score higher on a placebo probe than on any friendship probe. Bold placebo values above mark those cases.
Scores are z-scored within each probe, so this compares how unusual a song is for each probe, not raw similarity.

Theme by album

The 8 unified themes from Method 1 (paired LDA/NMF topics), tracked by album instead of pooled across the catalog.
Cell = fraction of an album’s songs carrying that topic (≥25% of the song’s topic mass); numbers = song counts.

Topic prevalence by album, LDA and NMF side by side. Rows follow the Method 1 pairing table.

What I make of it

The labelers found friendship in 2 songs. The word “friend” appears in 19, and both topic models produced a friend topic without being asked for one.
The probes recover both labeled songs and rank the keyword songs the labelers skipped at the top — Tonight I’m Getting Over You and Beautiful have the widest margins over their placebos in the whole set.
Where the probes go beyond the keyword, the evidence thins. Of the three songs with neither a keyword nor a label, one has a clear margin (Party for One), one is narrow (Surrender My Heart), and one is beaten by a placebo (After Last Night).
The codebook was the constraint, not the models. Defining friendship as “platonic bonds, companionship outside romance” excludes the boundary songs, which is where this catalog actually puts it.

Next: an updated theme map in the style of the previous post — built on the NMF topics and their LLM-generated labels, so the vocabulary comes out of the songs themselves instead of a hand-written codebook.

Footnotes

Pruning must be by document frequency, not TF-IDF weight: ubiquitous words like love carry high TF-IDF here simply from repetition, so a TF-IDF cutoff wouldn’t remove them.↩︎