Sentiment and Affect Lexicons

Naive Bayes classified a whole document by pooling every word it contained: the review is positive because the product of per-word likelihoods came out that way. That model has no notion of whether any single word means something positive; it only learns, from labeled documents, that delicious tends to co-occur with the positive class. This lesson zooms in one level, to the word itself. The claim is that words carry affective meaning — a fixed emotional coloring, independent of the document they land in — and that we can write this meaning down in a lexicon: a list of words annotated with the sentiment or emotion each one evokes.¹

A sentiment lexicon is the smallest possible sentiment resource. Instead of using every word as a feature, we keep only the words that carry a strong cue to affect, and we attach a label or a score to each. The word excellent is positive; horrible is negative; vacation scores high on pleasantness and torture scores low. The rest of the lesson is about what those annotations should be (which requires a theory of emotion), which lexicons already exist, and — the real work — the three ways to build one.

Defining emotion

Before we can annotate a word with its emotion, we have to decide what the set of emotions is. Computational models in NLP draw on two families of theories from affective science.²

The first family treats emotions as basic emotions: a small fixed set of atomic units, present in all cultures, out of which everything else is built. The best-known list is Ekman's six — surprise, happiness, anger, fear, disgust, sadness — proposed as universal. Plutchik's wheel is a richer variant: eight basic emotions arranged in four opposing pairs (joy–sadness, anger–fear, trust–disgust, anticipation–surprise), with blends filling the space between neighbors.

The second family treats emotion not as a set of atoms but as a point in a continuous space of two or three dimensions. Almost every dimensional model includes valence and arousal, and many add a third, dominance:

Sentiment falls out of this second view as a special case: the valence axis, measuring how pleasant or unpleasant a word is, is what we usually mean by sentiment. Positive versus negative is just the sign of valence. The valence–arousal plane is the standard way to picture it: pleasant-and-calm (serene) sits apart from pleasant-and-intense (ecstatic), and the two negative quadrants split the same way.

The two families differ in resolution rather than substance. Basic-emotion lexicons give a word a discrete tag (this word is anger); dimensional lexicons give it real-valued coordinates. Both share a strong simplifying assumption: the affective meaning is fixed, the same regardless of the sentence, dialect, or culture the word appears in. Richer appraisal-theory models — where an emotion is a process that weighs an event against a person's goals — drop that assumption, but the lexicons in this lesson keep it, and it is what makes them lookup tables.

The lexicons that already exist

A great many affect lexicons have been released; you rarely have to start from nothing. The simplest label words along a single dimension — call it sentiment or valence — as a binary split into a positive wordlist and a negative wordlist.³

The oldest, the General Inquirer, is just two hand-built wordlists; MPQA Subjectivity adds a strong/weak reliability tag, Bing Liu's was bootstrapped from product reviews, and AFINN attaches a small integer valence. All are one-dimensional.

The richer lexicons go multi-dimensional. The NRC VAD lexicon scores 20,000 words on all three affective dimensions at once; a word is now a triple, not a tag.

The NRC Word-Emotion Association Lexicon, or EmoLex, takes the basic-emotion route instead: for each of ~14,000 words it records a binary 0/1 for each of Plutchik's eight emotions plus positive/negative. A word like reward lights up anticipation, joy, surprise, trust, and positive; garbage lights up disgust and negative.

LIWC (Linguistic Inquiry and Word Count) is different in kind: 73 curated category lists — negative-emotion, positive-emotion, anger, sadness, cognitive mechanisms, tentativeness, negation — built for social-psychology research rather than sentiment per se, but widely reused as features. Entries can be word prefixes (happy* matches happy, happiness, happily).

For many tasks the right first move is to use one of these off the shelf. But when the genre is unusual (financial text, a historical corpus, a new language) the pre-built lexicons miss, and you build your own. There are three ways.

Building a lexicon (1): human labeling

The oldest method, still standard, is to have humans label each word, now almost always via crowdsourcing — split the job into tiny questions and farm them to many annotators.⁴ The direct question — how positive is sublime, on a scale of 1 to 9? — gives noisy answers: people use the scale differently, anchor inconsistently, and struggle to be precise about an absolute number.

Best-worst scaling sidesteps this. Show an annotator four words at a time and ask only for the two extremes — which is most positive and which is least. Relative judgments among a few items are far more reliable than absolute ratings on a long scale.

Across many overlapping four-word tuples, a word's final score is the proportion of tuples in which it was chosen best minus the proportion in which it was chosen worst — a real number in $[-1,1]$. The NRC VAD lexicon was built by this procedure, run once per dimension.⁵ EmoLex used a related two-step scheme: a priming multiple-choice question fixed the intended word sense first (startle is closest to shake, not automobile), then annotators rated association with each of the eight emotions on a not/weak/moderate/strong scale, collapsed to binary. Annotation quality is checked by split-half reliability: split the annotators in two, and see whether the two halves' scores correlate.

Building a lexicon (2): semi-supervised induction

Human labeling is accurate but expensive. Semi-supervised induction produces a large lexicon from a few dozen labeled words. Hand-pick a small set of seed words at each pole of the axis — good, excellent, love for positive; bad, horrible, hate for negative — and then score every other word by how similar it is to the positive seeds and how dissimilar to the negative ones.⁶ Similar is where the methods differ.

Seeds are chosen by hand, and the right seeds depend on genre. General sentiment, Twitter, and finance each need their own poles:

The semantic-axis method

The simplest similarity metric is embedding cosine. Embed every word, then turn the seed sets into a single axis vector pointing from negative to positive. Take the centroid (mean vector) of the positive-seed embeddings, $V^{+}=\frac{1}{n}{\sum }_{i}E(w_i^{+})$, and of the negative seeds, $V^{-}=\frac{1}{m}{\sum }_{i}E(w_i^{-})$, and subtract:

$V_{\text{axis}}$ points in the direction of increasing positivity. Score any word $w$ by the cosine between its embedding and the axis:

A word whose embedding aligns with the axis is positive; one pointing the other way is negative; one orthogonal is neutral.⁷

Label propagation on a graph

An alternative, SentProp, propagates polarity over a graph instead of projecting onto an axis.⁸ Build a lexical graph: each word is a node, connected to its $k$ nearest neighbors by cosine, edge weights set from the angle between embeddings. Drop the seed labels onto their nodes and run a random walk from each pole: a word's positive score is proportional to how often a walk started from the positive seeds lands on it. Words tightly clustered around love and adore accumulate positive walks; those near hate and loathe accumulate negative ones; and a word in between, like find or notice, stays neutral because both walks reach it about equally.

Both raw scores (positive and negative walk frequencies) are combined into a single polarity, ${\mathrm{score}}^{+}(w)=\frac{r^{+}(w)}{r^{+}(w)+r^{-}(w)}$, and confidence is estimated by re-running on random subsets of the seeds (bootstrap): a word whose score wobbles as the seeds change is one to trust less.

PMI from co-occurrence: Turney's SO-PMI

The seed idea predates embeddings. Turney's SO-PMI (semantic orientation from pointwise mutual information) needs only a search engine and two seed words: excellent and poor. A word is positive to the degree it co-occurs with excellent more than with poor.⁹ Pointwise mutual information measures that association,

and the semantic orientation of a phrase is its PMI with the positive seed minus its PMI with the negative seed:

Estimate each probability from hit counts: $P(w_1,w_2)$ from how often the two appear near each other, $P(w)$ from how often $w$ appears at all. Positive SO means positive sentiment.

For a worked example: Turney estimated the counts by issuing queries to a web search engine and reading off the number of returned pages, using the NEAR operator for co-occurrence. Suppose a corpus (or index) of $N=10^9$ documents, and for the target word romantic these hit counts:

Divide each count by $N$ to get a probability, then compute each PMI. For the positive seed,

and for the negative seed,

The semantic orientation is the difference, $\mathrm{SO}\text{-}\mathrm{PMI}(\text{romantic})=1.91-(-0.42)=2.33$: strongly positive, because romantic co-occurs with excellent far more than chance and with poor less than chance. Run the same three queries for unpredictable and the sign flips — it appears near poor more than near excellent, so its SO comes out negative — recovering the polarity of both words from nothing but a search engine and two seed words. The $N$ cancels out of the difference, which is why Turney could estimate SO-PMI without ever knowing the true corpus size.

Other similarity cues work in place of cosine or co-occurrence. Two adjectives conjoined by and (fair and legitimate) usually share polarity, while but (fair but brutal) flips it; a morphological negative (adequate / inadequate) flips it; a thesaurus lets you add the synonyms of positive seeds and the antonyms of negative seeds. All follow the same pattern: a similarity metric plus seed poles.

Building a lexicon (3): supervised learning from reviews

Sometimes supervision comes free. Online reviews carry a star rating — 1 to 5, or 1 to 10 — and the rating is a label for the whole review's sentiment. Positive words cluster in 5-star reviews; negative words in 1-star reviews. We can learn word sentiment directly from the counts, and get something richer than a binary tag: a distribution over ratings.¹⁰

Concretely, count how often word $w$ appears in reviews of each class $c$, and turn counts into a likelihood $P(w\mid c)$. Then normalize across classes into the Potts score, which reads as the word's sentiment profile:

Work the normalization on one word. Suppose across a review corpus the word disappointing occurs with these raw counts by star rating, and each rating class has the shown total word count:

The per-class likelihoods $P(w\mid c)$ sum to ${\sum }_{c^{\prime }}P(w\mid c^{\prime })=(5.0+4.0+2.5+1.5+1.0)\times 10^{-4}=1.4\times 10^{-3}$. The Potts score for each rating is that class's share of the total:

with the intermediate ratings at $0.286$, $0.179$, and $0.107$. The profile $[0.357,0.286,0.179,0.107,0.071]$ falls monotonically from 1-star to 5-star — the reverse-J of a negative word — and it sums to $1$ by construction, so it reads as a distribution over ratings. Normalizing by class size is what makes this fair: without dividing by the tokens-per-class, a rating level that simply has more reviews would inflate every word's count there.

Plot the profile across ratings and the shape tells you the word's sentiment. Strongly positive scalars (excellent) trace a rising J; strongly negative scalars (terrible) trace a reverse J; weakly polarized words (good, disappointing) make a hump, peaked just above or just below the middle.¹¹

When the goal is instead to find the words that most distinguish two classes (1-star versus 5-star, Democrat versus Republican), raw frequency differences mislead: every difference looks big for frequent words and small for rare ones. The log-odds-ratio with an informative Dirichlet prior fixes this by shrinking counts toward a large background corpus and reporting a $z$-score, so the words it surfaces are genuinely over-represented, not just common. Applied to a Yelp corpus, it recovers the obvious sentiment words (1-star: worst, awful; 5-star: amazing, delicious) but also subtler tells — 1-star reviews use logical negation (no, not) and first-person plural (we, us), 5-star reviews use emphatics (very, highly, always).

Using a lexicon

Two uses, matching the two data regimes.¹² With no training data, run a rule-based classifier: count positive-lexicon words and negative-lexicon words in the document and pick the majority. If the lexicon carries weights ${\theta }_w^{+}$ and ${\theta }_w^{-}$, sum those instead, and classify by the ratio against a threshold $\lambda$:

With training data, the lexicon becomes a small set of features feeding a logistic-regression or SVM classifier, alongside the raw words. The simplest lexicon feature is an indicator — 1 if the document contains any word from lexicon $L$ — or a count, $f_L={\sum }_{w\in L}\mathrm{count}(w)$, optionally weighted by ${\theta }_w^L$. This is the affect-recognition recipe too: label a training set for whatever affective category you want (emotion, personality), extract word, bigram, and lexicon-count features, and train a standard classifier. When the training set is large and matched to the test set, using all the words as features is very hard to beat; lexicon features help when data is sparse or the domains differ.

Connotation frames

Every lexicon so far pins a word to a point in affect space — one score, or one triple. A connotation frame records something structurally richer: the sentiment a predicate implies about each of its arguments.¹³ It combines the affect lexicon with the frame-semantic idea that a verb has slots (agent, theme) each playing a role.

Consider Country A violated the sovereignty of Country B. The verb violate does more than express negativity; it takes a stance. It casts the object (Country B) as a sympathetic victim, the subject (Country A) as the antagonist, and signals the writer's sympathy with B and antagonism toward A — all before you know anything about the countries. Contrast the teenager survived the bombing: survive makes its subject the sympathetic party and marks the event as a hardship. The connotation is part of the verb's meaning.

The frame lexicons of Rashkin and Sap record several such relations per verb: the writer's sentiment toward each role, the effect on each role (something bad happened to it), its value, its mental state, the power differential (implore implies the agent has less power than the theme; demand implies more), and the agency of each argument (waited is low-agency, determined is high). Trained on such a lexicon, an entity-centric analysis of a novel or a plot summary can chart each character: run it on The Dark Knight and Batman comes out high-power, the Joker high-agency but low-sentiment, the love interest low-power but high-sentiment — the affective structure of the story recovered from the connotations of its verbs.

From lexicons to contextual sentiment

A lexicon assigns a word one fixed score regardless of context, and that assumption is both its strength and its main limitation. Two lines of work past Jurafsky & Martin's core push against it while keeping the lexicon useful.

The first refines rule-based counting so it works on real text. VADER (Hutto & Gilbert, 2014, ICWSM) is a hand-tuned, valence-aware lexicon-and-rules system built for social media.¹⁴ It starts from a crowd-scored lexicon but adds rules that a plain count ignores: punctuation and capitalization amplify intensity (good versus GOOD!!!), degree modifiers scale it (extremely up, marginally down), a contrastive but shifts the weight to the second clause, and — the case pure counting gets backwards — negation flips polarity, so not good is scored negative rather than as a positive word plus a negative one. Each is a failure mode of the threshold classifier this lesson opened with, here handled by rules rather than learning, and VADER remains a strong zero-training-data baseline for short informal text.

The second line drops fixed scores entirely. A transformer sentiment classifier — a pretrained encoder such as BERT (Devlin et al., 2019, NAACL) fine-tuned on labeled reviews — represents each word in context, so the same word takes different values in different sentences.¹⁵ This resolves the case a lexicon cannot: sick is negative in I feel sick and positive in that trick was sick, and unpredictable praises a plot but pans a car. A fixed lexicon must pick one sign; a contextual model reads the sign off the sentence. On benchmark sentiment tasks with adequate labeled data, these models are the accurate choice.

Lexicons remain useful for three reasons, each a limit of the neural approach. They need no training data, which matters for a new language or genre; they are interpretable, so you can read why a document scored as it did, which matters in the social-science settings LIWC and connotation frames were built for; and they still supply useful features to a supervised model when labeled data is sparse. The methods in this lesson complement representation learning rather than compete with it — and the semi-supervised induction methods, which run on the same embedding geometry the neural models are built from, are the bridge between the two.

Where this sits

Word-level affect is the layer beneath document sentiment. Naive Bayes and logistic regression pool words into a document label; a lexicon says what those words mean one at a time, and can feed that meaning back as features or, with no training data at all, classify by counting. The semi-supervised induction methods are the reason the next chapter matters: they run on embeddings, turning the geometry of vector space into the poles and neighborhoods that carry sentiment from a handful of seeds to a lexicon of twenty thousand words.