Images, sound, and text
Each modality needs its own encoding. For text, bag-of-words and n-grams are classic starting points.
Images become pixel grids (or embeddings). Audio becomes sampled waveforms or spectrograms. Text is a sequence of characters or words.
Bag-of-words (BoW)
Count how often each word appears; ignore order. Often drop stopwords (“the”, “and”) and stem related forms (realise / realized → same stem).
n-grams
Count length-$n$ sequences. Bigrams keep some local order: “not good” ≠ treating “not” and “good” separately.
“To be or not to be”
BoW (bag): be, is, not, or, question, that, the, to (counts; order gone).
Bigrams: “to be”, “be or”, “or not”, … — pairs preserve neighbouring words.
Key takeaways
- BoW is a strong simple baseline.
- n-grams add local order.
- Normalize counts by document length for retrieval.