The agent watches a 261-minute reel of boxing training and lists every stretch where sound and picture are out of sync: its start and end, its kind, and either the offset at both ends or the moment the sound was taken from. The reference answer holds 35 stretches, 31 with an offset and 4 with foreign sound.
What a good answer has to get right, and how the verifier checks it
- Every faulty stretch is found, and nothing in sync is listed
- The score is an F1 over stretches with one-to-one matching. A missed stretch lowers recall. A listed stretch that matches no reference stretch lowers precision, and so does an entry the verifier cannot read.
- The kind of fault is right
- A stretch counts only when its kind, offset or foreign, is the same as the reference stretch’s.
- The stretch is placed in time
- Its start and end must each be within 1.0 s of the reference. The verifier’s default is 0.5 s; the ground-truth file sets 1.0 s for this reel, because the audit that re-measured every stretch from the media alone was off by up to 0.78 s.
- The offset is measured at both ends
- For an offset stretch, the offset at the start and the offset at the end must each be within 40 ms. A drifting offset therefore has to be reported as drifting, not as one average value.
- The source of foreign sound is identified
- For a foreign stretch, the moment of the reel the sound was taken from must be within 1.0 s of the true one.
How the checks become one score
Score = F1 = 2 × precision × recall / (precision + recall). Precision is matched entries over listed entries; recall is matched stretches over 35. Every stretch counts the same. A stretch that is right on some fields and wrong on one counts as not matched: there is no credit for a stretch found at the right place with the wrong offset.
What scores zero
No sync_ledger.json, a file that is not valid JSON or whose “stretches” is not a list, an empty list, or a list in which no entry matches a reference stretch on every field.
Outside the score: the jump cuts, audio glitches, jitter and level changes that the task says are not faults, and 5 stretches of a decoy class that the ground truth marks as unscored. An entry that matches one of those 5 on every field is left out of the count; any other entry placed on one counts as wrong.


