Transformer Architecture Features Quiz

Quiz covering Machine Learning Techniques

Questions in this quiz

A preview of the 5 questions covered. Start the quiz above to answer them, check your score, and read the explanations.

1. 1

   What is the primary role of the self-attention mechanism in a Transformer model?

2. 2

   In a Transformer encoder, what is the purpose of the feed-forward network applied after the multi-head attention layer?

3. 3

   Consider a Transformer model being used for machine translation. During the decoding phase, why is masked multi-head attention used in the decoder's self-attention layer?

4. 4

   What is the key advantage of using multi-head attention over single-head attention in a Transformer?

5. 5

   Imagine you are fine-tuning a pre-trained Transformer model (like BERT) for a specific downstream task, such as sentiment analysis. You have a relatively small labeled dataset. Which of the following strategies would be MOST effective for preventing catastrophic forgetting of the pre-trained knowledge while adapting to the new task?