Model Evaluation & RLHFIntermediateTier 2 · $99
A/B Model Comparison
Blind pairwise comparison with order, length and formatting bias controlled.
3 modules110 minutes of material10 exam questions · 80% to pass
Syllabus
01Blind pairwise review
40 min- Judge without knowing the model
- Apply the pairwise rubric
02Biases that distort
40 min- Control for order and length
- Ignore confident tone
03Decision thresholds
30 min- Decide when a difference is material
- Record the call
Assessed concepts
The exam tests these directly. Definitions are summarised here; the exact answer key stays on the server.
Blind comparison
Pairwise rubric
Order bias
Length bias
Decision threshold