Aviary caller ledger
You are given 21 one-minute recordings of a group of eight zebra finches living together
in an indoor aviary. The birds often call at the same time. For each recording, find
every vocal bout of every bird: which bird it was, when the bout started and when it
ended.
Materials (all in the working directory)
clips/clip01.mp4 … clips/clip21.mp4 — video, 60 s each, 2976×2016 pixels,
390625/8192 ≈ 47.68 frames per second, no audio track.
clips/clip01.flac … clips/clip21.flac — 7-channel microphone audio for the same
60 s, 16-bit, 24414 Hz. Audio and video come from one hardware clock: audio sample 0 is
at video frame 0, and one video frame spans 512 audio samples.
roster.json — for each clip, the birds present. Each bird wears a small coloured
backpack on its back and is named by that colour: purple, red, peach, blue,
green, white, yellow, brown. A colour always means the same bird.
backpacks.png — one example of each backpack colour, cropped from the top camera.
The clips are in random order and come from several days.
The aviary
The birds live in a box with a 90 cm × 55 cm floor. The lower 40 cm (the "stage") holds
perches, food, water, a sand bath and two nest boxes (front-left and rear-right corners,
30–40 cm up); above it a tent-shaped upper part narrows towards the ceiling. Birds can be in the upper part or hidden behind one another.
Each video frame is a composite of camera views. Rectangles are given as left x, top y,
width × height, in pixels:
| region |
view |
| 1000, 726, 1976×1290 |
top camera, looking straight down on the stage |
| 1000, 0, 1976×726 |
back camera, through the rear wall (mirror image) |
| 220, 726, 780×1290 |
left camera, through the left wall (mirror image, rotated 90°) |
| 0, 0, 1000×726 |
two cameras looking into nest boxes (greyscale) |
The pixels at 0, 726, 220×1290 are unused.
Audio channels (1-based):
| channel |
microphone position |
| 1 |
centred above the rear wall of the stage |
| 2 |
centred above the left wall of the stage |
| 3 |
centred above the front wall of the stage |
| 4 |
centred above the right wall of the stage |
| 5 |
ceiling of the enclosure, next to the top camera |
| 6 |
inside the front-left nest box |
| 7 |
inside the rear-right nest box |
Definitions
- Vocalization: any sound a bird makes with its voice — song syllables and
introductory notes, every kind of call (short soft calls, loud distance calls, begging
calls), and young birds' unfinished, babbling song. Not vocalizations: wing beats,
take-offs and landings, pecking, seed husking, bill wiping, scratching, and noises from
perches, feeders or walls.
- Bout: take one bird's vocalizations in the clip in time order. Two consecutive
vocalizations of that bird belong to the same bout when the silence from the end of
the first to the start of the next is shorter than 0.5 s; otherwise the second one
starts a new bout. A single isolated call is a bout on its own. A bout starts at the
start of its first vocalization and ends at the end of its last. Only the bird's own
sounds count: another bird's vocalization never joins or splits this bird's bouts, and
when two birds vocalize at the same time each of them has its own bout.
- Times are seconds from the start of the clip (a number such as
12.43).
- A listed bout is correct when the clip and the bird are right and its start and its
end are each within 0.1 s of the true bout's start and end. A bout attributed to
the wrong bird, one bout split into several, and several bouts merged into one are all
wrong. Listing a bout that did not happen is wrong; missing one is also wrong.
- One bird per sound. Do not cover several possibilities by listing the same sound
under more than one bird. If two bouts you list for different birds in the same clip
have starts within 0.6 s of each other and ends within 0.6 s of each other, then unless
both of them are correct, both count as wrong. Two birds really vocalizing together is
fine when you name both correctly.
How much the birds vocalize differs a lot between clips: some clips have only a few
bouts, others more than a hundred.
What to submit
Write output/bouts.json in exactly this shape:
{
"bouts": [
{"clip": "clip01", "bird": "red", "start": 12.40, "end": 12.48},
{"clip": "clip01", "bird": "white", "start": 12.45, "end": 14.02}
]
}
- One entry per bout, in any order;
bird must be a colour listed for that clip in
roster.json.
- Write this file early and keep overwriting it as you complete clips. There is a time
limit, and whatever is in the file when the session ends is what gets graded.
Rules
- Work only inside the working directory. Do not read, write or search outside it.
- There is no internet access and none is needed;
pip install will not work. Do not
rely on memory of this or any other recording. Every entry must come from what you
observe in these files.
- Available tools are what is installed:
ffmpeg/ffprobe, sox, and Python 3 with
numpy, scipy, pandas, pillow, opencv-python-headless, soundfile and
matplotlib.