The agent watches a full college volleyball match and lists every point that ended as a block: the set, the score after the point, the player or players credited with the block, and the opposing hitter who was blocked. The reference holds 23 block points.
What a good answer has to get right, and how the verifier checks it
- Every block point is found, and none is invented
- An entry is tied to a reference block by the set and the exact score after the point, which is unique within a set. The score is an F1, so missed blocks and entries for points that were not blocks both cost.
- The blockers are credited
- The set of credited blockers must be exact: a solo block and a shared block are different answers.
- The blocked hitter is named
- The hitter whose attack was stopped must be right. Blockers and hitter are on opposite sides of the net and have to be read within the same fraction of a second.
- One wrong name earns half
- An entry whose names are off by exactly one (a blocker missing, extra or wrong, or the hitter wrong) earns 0.5 instead of 1. With two or more errors it earns nothing. Exact matches are assigned first.
How the checks become one score
Score = F1 over credit: precision = credit / entries listed, recall = credit / 23, where credit is 1 per fully right block and 0.5 per block with one name wrong. A last name alone is accepted where it is unambiguous.
What scores zero
No solution file or one that cannot be read, an empty list, or a list in which no entry has the right set and score with at most one name wrong.
Outside the score: points won in other ways, such as service aces. The verifier also reports how many block points were found regardless of attribution, as a diagnostic.