# Natural Language Processing

Every sentence of human language is ambiguous, context-dependent, and
full of knowledge the words never state — and we want machines to read
it, translate it, answer from it, and write it back.


_Figure 001 — Self-attention: each word draws on every other, weighted by relevance.
_

The field begins with the basics: **regular expressions** to
match patterns, **tokenization** to split text into units, and
**edit distance** to measure how far apart two strings are.
Then the first real model of language — the **n-gram**, which
predicts the next word from the last few.


_Figure 002 — An n-gram model: a window slides across the text, three tokens at a time.
_

The modern turn is _representation_: instead of treating words as
atomic symbols, embed them as vectors so that meaning becomes geometry —
king minus man plus woman lands near queen — and let a neural network
compose those vectors into the meaning of a whole sentence.


_Figure 003 — Word embeddings: meaning as geometry — king − man + woman ≈ queen.
_

The **transformer** lets every word attend to every other,
and scaled up on enough text it became the large language model — one
architecture that translates, summarizes, answers, and converses.


_Figure 004 — A parse tree: words group into phrases and phrases into a sentence.
_

From ELIZA's pattern-matching in 1966 to models that pass the bar exam,
the problem is the same: turn the fluid, ambiguous signal of language
into something a computer can compute with.


_Figure 005 — Sequence tagging: a part-of-speech tag drops in under each word.
_

---

## Contents

### 1. Foundations

1. [What Is Natural Language Processing?](/natural-language-processing/foundations/what-is-nlp)
2. [Regular Expressions and Text Normalization](/natural-language-processing/foundations/regex-and-text-normalization)
3. [Minimum Edit Distance](/natural-language-processing/foundations/minimum-edit-distance)
4. [N-Gram Language Models](/natural-language-processing/foundations/n-gram-language-models)
5. [Smoothing and Backoff](/natural-language-processing/foundations/smoothing-and-backoff)

### 2. Text Classification

1. [Naive Bayes and Sentiment Classification](/natural-language-processing/classification/naive-bayes-and-sentiment)
2. [Evaluating Classifiers](/natural-language-processing/classification/evaluating-classifiers)
3. [Logistic Regression](/natural-language-processing/classification/logistic-regression)
4. [Sentiment and Affect Lexicons](/natural-language-processing/classification/sentiment-and-affect-lexicons)

### 3. Semantics

1. [Vector Semantics and Embeddings](/natural-language-processing/semantics/vector-semantics-and-embeddings)
2. [Static Word Embeddings: word2vec and After](/natural-language-processing/semantics/static-word-embeddings)
3. [Neural Networks and Neural Language Models](/natural-language-processing/semantics/neural-language-models)

### 4. Sequences

1. [Sequence Labeling: POS and NER](/natural-language-processing/sequences/sequence-labeling)
2. [Viterbi Decoding, CRFs, and Neural Taggers](/natural-language-processing/sequences/crfs-and-neural-taggers)
3. [RNNs and LSTMs](/natural-language-processing/sequences/rnns-and-lstms)

### 5. Transformers

1. [Transformers and Self-Attention](/natural-language-processing/transformers/transformers-and-attention)
2. [The Transformer Architecture](/natural-language-processing/transformers/the-transformer-architecture)
3. [Large Language Models](/natural-language-processing/transformers/large-language-models)
4. [Large Language Models: Pretraining and Scaling](/natural-language-processing/transformers/llm-pretraining-and-scaling)
5. [Fine-Tuning and Prompting](/natural-language-processing/transformers/fine-tuning-and-prompting)
6. [Prompting and Alignment](/natural-language-processing/transformers/prompting-and-alignment)

### 6. Linguistic Structure

1. [Constituency Parsing](/natural-language-processing/linguistic-structure/constituency-parsing)
2. [CKY Scoring, Evaluation, and Shallow Parsing](/natural-language-processing/linguistic-structure/cky-scoring-and-evaluation)
3. [Dependency Parsing](/natural-language-processing/linguistic-structure/dependency-parsing)
4. [Graph-Based and Neural Dependency Parsing](/natural-language-processing/linguistic-structure/graph-based-and-neural-dependency-parsing)
5. [Word Senses and Disambiguation](/natural-language-processing/linguistic-structure/word-senses-and-wsd)
6. [WSD in Practice and Word Sense Induction](/natural-language-processing/linguistic-structure/wsd-in-practice-and-induction)
7. [Semantic Roles and Information Extraction](/natural-language-processing/linguistic-structure/semantic-roles-and-information-extraction)
8. [Relations, Events, and Templates](/natural-language-processing/linguistic-structure/relations-events-and-templates)
9. [Coreference and Discourse](/natural-language-processing/linguistic-structure/coreference-and-discourse)
10. [Coherence and Discourse Structure](/natural-language-processing/linguistic-structure/coherence-and-discourse-structure)
11. [Logical Representations of Meaning](/natural-language-processing/linguistic-structure/logical-semantics)
12. [Compositional Semantics and Description Logics](/natural-language-processing/linguistic-structure/compositional-semantics-and-description-logics)
13. [Semantic Parsing](/natural-language-processing/linguistic-structure/semantic-parsing)
14. [Learned and Neural Semantic Parsing](/natural-language-processing/linguistic-structure/neural-semantic-parsing)
15. [Information Extraction](/natural-language-processing/linguistic-structure/information-extraction)
16. [Extracting Times, Events, and Templates](/natural-language-processing/linguistic-structure/times-events-and-templates)
17. [Discourse Coherence](/natural-language-processing/linguistic-structure/discourse-coherence)
18. [Entity-Based and Global Coherence](/natural-language-processing/linguistic-structure/entity-based-and-global-coherence)
19. [Constituency Grammars](/natural-language-processing/linguistic-structure/constituency-grammars)
20. [Treebanks and Lexicalized Grammars](/natural-language-processing/linguistic-structure/treebanks-and-lexicalized-grammars)

### 7. Applications

1. [Machine Translation](/natural-language-processing/applications/machine-translation)
2. [Machine Translation: Decoding, Evaluation, and Scale](/natural-language-processing/applications/machine-translation-decoding-and-evaluation)
3. [Question Answering](/natural-language-processing/applications/question-answering)
4. [Question Answering: Knowledge Bases and Language Models](/natural-language-processing/applications/question-answering-knowledge-and-llms)
5. [Dialogue and Chatbots](/natural-language-processing/applications/dialogue-and-chatbots)
6. [Dialogue Systems: LLM Assistants, Evaluation, and Design](/natural-language-processing/applications/dialogue-systems-and-assistants)
7. [Text Summarization](/natural-language-processing/applications/text-summarization)
8. [Abstractive Summarization and Evaluation](/natural-language-processing/applications/abstractive-summarization-and-evaluation)

### 8. Speech

1. [Phonetics](/natural-language-processing/speech/phonetics)
2. [Acoustic Phonetics](/natural-language-processing/speech/acoustic-phonetics)
3. [Automatic Speech Recognition](/natural-language-processing/speech/automatic-speech-recognition)
4. [ASR Evaluation and Speech Applications](/natural-language-processing/speech/asr-evaluation-and-applications)