15
Lesson 15 of 20 · Smart Helpers
The attention mechanism
Attention lets the model weigh which words matter most for predicting the next token. This allows understanding long-range dependencies in text.
- Attention weighs word importance for predictions.
- It captures relationships between distant words.
Think about it
What is retrieval-augmented generation (RAG)?