Free Engineering Workbench calculator
Pairwise win rate
Compare two models or prompts from head-to-head judgements, with confidence.
AI evaluationHow it's worked out
- Win rate = (wins + ties ÷ 2) ÷ judgements; Wilson interval. Sign test ignores ties
- Judgements independent; same prompt set for both
NoteRandomise which side each answer appears on; judges, human or model, favour position and length.
Source: Zheng et al., Judging LLM-as-a-judge (MT-Bench, Chatbot Arena, 2023)
Need help acting on this number?
VVnT SeQuor's AI Testing & Evaluations practice helps teams turn this metric into a plan.