Attention Mechanisms Quiz

Quiz covering Transformers and Attention

Questions in this quiz

A preview of the 5 questions covered. Start the quiz above to answer them, check your score, and read the explanations.

1. 1

   What is the primary purpose of the 'Attention' mechanism in a Transformer model?

2. 2

   In the context of Self-Attention, what do the Query (Q), Key (K), and Value (V) vectors represent?

3. 3

   Why is 'Multi-Head Attention' preferred over a single 'Attention' head in Transformer architectures?

4. 4

   What is the role of the 'Scaled Dot-Product' in the attention calculation?

5. 5

   In a Transformer's decoder, why is 'Masked Multi-Head Attention' used instead of standard multi-head attention?