Transformer Architecture Quiz

Quiz covering Transformers and Attention

Questions in this quiz

A preview of the 5 questions covered. Start the quiz above to answer them, check your score, and read the explanations.

1. 1

   What is the primary purpose of the 'Self-Attention' mechanism in a Transformer model?

2. 2

   In the original Transformer architecture, why are 'Positional Encodings' added to the input embeddings?

3. 3

   What is the function of the 'Feed-Forward Network' (FFN) that follows the Multi-Head Attention layer in each Transformer block?

4. 4

   When training a Transformer, what is the role of the 'Mask' in the decoder's masked self-attention layer?

5. 5

   How does increasing the number of 'Attention Heads' in Multi-Head Attention affect the model's learning capacity?