Attention
The mechanism that lets a model weigh the relevance of different tokens to each other.
Attention is a learned mechanism inside transformer models that assigns a weight to every token pair, letting the model decide how much one token should influence another’s representation. In self-attention, each token attends to all others in the same sequence, producing context-aware embeddings rather than fixed-meaning word vectors. On the AIF-C01 exam, distinguish self-attention from cross-attention: self-attention operates within one sequence, while cross-attention connects two, such as an encoder output and a decoder input in translation. Models accessed through Amazon Bedrock rely on attention-based transformers, which is why they handle long-range dependencies that earlier recurrent networks struggled with.
PlayPrepHQ study notes are written and reviewed against primary exam sources. How we create & review content →