Human Evaluation Quiz

Quiz covering Model Evaluation

Questions in this quiz

A preview of the 5 questions covered. Start the quiz above to answer them, check your score, and read the explanations.

1. 1

   What is the primary purpose of using human evaluation for foundation models?

2. 2

   When setting up an A/B test for two different model responses, what is the best practice to minimize rater bias?

3. 3

   Which of the following is a significant drawback of using Likert scales (e.g., 1-5 rating) in human evaluation?

4. 4

   What is the primary benefit of using a 'pairwise comparison' approach over an absolute rating scale?

5. 5

   In a human evaluation study, you notice 'annotator drift' occurring over time. What is the most effective way to address this?