The agent watches the first 50 minutes of a professional Dota 2 game and reports every hero death that belongs to a major teamfight (three or more deaths less than 15 s apart): the clock time, the victim, the player credited with the kill, and the victim’s minimap cell 10 s before the death, 5 s before, and at the death, on a 14 by 14 grid. The reference holds 39 deaths.
What a good answer has to get right, and how the verifier checks it
- Every teamfight death is found, and none is invented
- The score is an F1 over deaths with one-to-one matching. Isolated deaths and two-death exchanges are not part of the reference, so listing them lowers precision.
- The death is timed
- The clock time must be within 2 s.
- Victim and killer are named
- Both must be exact, out of the ten players in the game.
- The path before the death is traced
- All three minimap cells must be exactly right. A cell is one fourteenth of the map’s width, and the positions at 10 s and 5 s before have to be read from the minimap before the death happens.
How the checks become one score
Score = F1 = 2 × precision × recall / (precision + recall), with recall over the 39 deaths and precision over every entry in the file. Every death counts the same, and a death counts only when time, victim, killer and all three cells are right. One cell off by one square makes the entry wrong.
What scores zero
No solution file or one that cannot be read, more than 120 entries, an empty list, or a list in which no entry is right on every field.
Outside the score: deaths outside major teamfights and after 50:30. The verifier also reports F1 for partial matches (time only; time and victim; each cell; all cells within one square) as diagnostics that do not enter the score.