Benchmarks Quiz

Quiz covering Model Evaluation

Questions in this quiz

A preview of the 5 questions covered. Start the quiz above to answer them, check your score, and read the explanations.

1. 1

   Which of the following is a commonly used benchmark for evaluating the general knowledge and reasoning capabilities of Large Language Models across various subjects?

2. 2

   When evaluating a model for code generation tasks, why is it considered best practice to use functional correctness (executing the code) rather than just string similarity metrics like BLEU?

3. 3

   A team is testing a model for a customer support chatbot. They notice the model performs well on the benchmark dataset but fails when real users provide messy, informal input. What is the most likely cause?

4. 4

   Why is 'contamination' a major concern when using public benchmarks to evaluate the latest Foundation Models?

5. 5

   You are evaluating a model for a multi-step reasoning task. You observe that the model's accuracy drops significantly when you change the order of the reasoning steps provided in the prompt. What does this suggest about the evaluation?