A/B Model Comparison Certification
Blind pairwise comparison with order, length and formatting bias controlled.
What this certification covers
A/B Model Comparison sits in the Model Evaluation & RLHF track (Intermediate). You read 3 written modules in order — each one has to be completed before the next opens — then sit a ten-question exam marked on our servers against a fixed answer key. Pass at 80% and the certificate is issued.
01Blind pairwise review40 min
- Judge without knowing the model
- Apply the pairwise rubric
02Biases that distort40 min
- Control for order and length
- Ignore confident tone
03Decision thresholds30 min
- Decide when a difference is material
- Record the call
Concepts tested
Blind comparison
Judging outputs with model identity hidden.
Pairwise rubric
The fixed criteria applied to both responses before choosing.
Order bias
Favouring whichever response was shown first.
Length bias
Rewarding verbosity that adds no information.
Decision threshold
The gap size that counts as a real difference between models.
How to get certified
- 1. Create a free account — no card, no payment at signup.
- 2. Read the 3 modules in order and mark each complete.
- 3. Sit the ten-question exam. Retakes are free.
- 4. If you pass, pay $99 once to unlock the signed certificate.