Tokens and Tokenization Quiz

Quiz covering Transformers and Attention

Questions in this quiz

A preview of the 5 questions covered. Start the quiz above to answer them, check your score, and read the explanations.

1. 1

   What is a 'token' in the context of Large Language Models?

2. 2

   Why do modern LLMs use sub-word tokenization methods like Byte Pair Encoding (BPE) instead of whole-word tokenization?

3. 3

   If you are using a model with a context window of 8,000 tokens, what happens if your input prompt plus the expected output exceeds 8,000 tokens?

4. 4

   How does the choice of tokenizer affect the cost and performance of a language model?

5. 5

   When analyzing how a BPE tokenizer handles the word 'unbelievable', which of the following best describes the process?