Vote aggregation

VoteAggregation

Bases: Enum

VoteAggregation Enum

How the individual annotator responses on a single matchup (one comparison of two models on one prompt) are aggregated into that matchup's result.

Attributes:

Name	Type	Description
MAJORITY_VOTE	VoteAggregation	Collapses each matchup to a single win for the side the majority of responses picked, splitting ties 0.5/0.5.
ALL_VOTES	VoteAggregation	Counts every individual response as its own matchup.