House Floor Plan and Inventory from a Walkthrough
walkthrough.mp4 in your working directory is one continuous, roughly 55-minute
first-person walkthrough of a single-storey house, filmed by a hand-held camera carried
through it at walking pace. The house is computer-rendered, but treat it as a real
building with real metric dimensions. The walk enters every room several times, but a
single visit usually shows only part of a room. Closed doors stay closed and are never
passed through. The video has no audio.
From this video, reconstruct the floor plan of the whole house (every room's type and
outline, and every doorway with its position) and the inventory of the objects listed
below, with each object's room and position.
Camera
- 1280×720 pixels, 15 frames per second, ideal pinhole camera with no lens distortion.
- Focal length 937.83 px horizontally and vertically (vertical field of view 42°);
principal point at pixel (640, 360), the image centre. These never change.
- The optical centre is always 1.576 m above the floor. The floor is flat and level
throughout the house.
- The camera never rolls: the image's horizontal axis always stays parallel to the
floor. Its heading and its downward tilt change smoothly as it is carried around; the
tilt stays between 0° (looking horizontally) and 30° below horizontal. Neither is given
per frame.
Coordinate frame
Report every coordinate in metres on the floor plane, viewed from above:
- Origin: the floor point directly below the camera in the first frame.
- +y: the camera's viewing direction in the first frame, projected onto the floor.
- +x: horizontal, 90° clockwise from +y seen from above, i.e. toward the right-hand
side of the first frame.
What to report
Rooms. Every room of the house, each exactly once. A room is a floor area enclosed
by walls; doorways join rooms but do not split or merge them. Each room is exactly one
of: kitchen, living_room, bedroom, bathroom. Its outline is the polygon traced
by the room's walls along the floor (the line where floor meets wall), continuing
straight across doorways; vertices in order around the room (either direction), without
repeating the first vertex. The number of rooms you report must equal the true number
of rooms in the house; a plan with the wrong room count is not accepted. A room counts
as correct when its type is right and the area overlap between your outline and the
true outline is at least 0.85 of their combined area (intersection over union ≥ 0.85).
Doors. Every doorway in a wall through which a person could walk — with or without
a door leaf, open or closed — including doors in outer walls. Report each doorway once,
with the pair of rooms on its two sides (for a door in an outer wall, the second side is
"outside") and its position: the centre of the opening, i.e. the midpoint of the gap
in the wall, on the floor–wall line. Windows are not doors. A door counts as correct
when it joins the right pair of rooms and its position is within 0.25 m of the true
opening centre.
Objects. Every instance, anywhere in the house, of the classes below — furniture,
fixtures, and the listed smaller items wherever they are (on the floor, on furniture, or
on a wall). Use exactly these labels and report nothing else. Not every class occurs in
this house.
| label |
what it is |
bed |
any bed, including bunk and loft beds |
sofa |
upholstered seat for two or more people |
armchair |
upholstered chair for one person, with arms |
chair_or_stool |
any other single seat: dining chair, desk chair, stool |
table_or_cabinet |
any table (dining table, desk, side table, nightstand, coffee table), kitchen counter unit, or free-standing storage furniture (dresser, chest of drawers, cabinet, TV stand, shelving unit) |
television |
flat-screen TV, on furniture or mounted on a wall |
floor_lamp |
tall lamp standing on the floor |
desk_lamp |
small lamp standing on furniture |
fridge |
refrigerator |
microwave |
microwave oven |
toaster |
toaster |
coffee_machine |
coffee machine |
sink |
washbasin, including its pedestal or cabinet |
toilet |
toilet |
garbage_can |
waste bin (not a loose trash bag) |
laundry_hamper |
lidded basket for laundry |
laundry_machine |
washing machine or clothes dryer |
safe |
small safe (lockable strongbox) |
dog_bed |
pet bed on the floor |
houseplant |
potted plant, any size |
painting |
framed picture or poster hanging on a wall |
window |
window in a wall |
pillow |
pillow or cushion |
teddy_bear |
stuffed toy bear |
basketball |
basketball |
baseball_bat |
baseball bat |
tennis_racket |
tennis racket |
laptop |
laptop computer |
book |
book |
newspaper |
newspaper |
box |
cardboard box |
statue |
decorative figurine or sculpture |
candle |
candle, with or without a holder |
alarm_clock |
alarm clock |
watch |
wristwatch |
cell_phone |
mobile phone |
remote_control |
remote control |
credit_card |
credit card |
cd |
compact disc |
key_chain |
keys on a ring |
pen_or_pencil |
pen or pencil |
bowl_or_vase |
bowl, dish or vase, open or lidded |
plate |
plate |
mug_or_cup |
mug or cup |
bottle |
bottle (not a spray or soap bottle) |
salt_or_pepper_shaker |
salt or pepper shaker |
food |
loose food item: apple, bread, egg, lettuce, potato, tomato |
utensil |
fork, spoon, knife, ladle or spatula |
cookware |
pan, pot or kettle |
plunger |
toilet plunger |
toilet_paper |
roll of toilet paper |
paper_towel_roll |
roll of paper towels |
tissue_box |
box of tissues |
spray_bottle |
spray bottle |
soap_bottle |
soap dispenser bottle |
soap_bar |
bar of soap |
dish_sponge |
dish sponge |
For each object give its room (the room whose outline contains it) and its position:
the centre of its footprint, i.e. the centre of the rectangle it covers when seen from
directly above (for an object on a wall, the rectangle directly below it on the floor).
An object counts as correct when its label and room are right and its position is within
0.25 m of the true footprint centre.
Output
Write output/floorplan.json:
{
"rooms": [
{"id": "r1", "type": "kitchen", "outline": [[20.0, 20.0], [20.0, 23.5], [23.5, 23.5], [23.5, 20.0]]},
{"id": "r2", "type": "bedroom", "outline": [[23.5, 20.0], [23.5, 23.5], [26.5, 23.5], [26.5, 20.0]]}
],
"doors": [
{"rooms": ["r1", "r2"], "position": [23.5, 21.2]},
{"rooms": ["r1", "outside"], "position": [21.0, 20.0]}
],
"objects": [
{"class": "bed", "room": "r2", "position": [25.0, 22.0]},
{"class": "fridge", "room": "r1", "position": [20.5, 23.0]}
]
}
Room ids are any unique strings you choose (not "outside"); doors and objects
refer to them. The values above only illustrate the format.
The session has a time limit and whatever is in output/floorplan.json when it ends is
what is graded, so write a complete file as soon as you have a first full plan and keep
overwriting it as you improve it.
Rules
- Work only inside your working directory. There is no internet access; do not try to
install packages.
- Use only what you observe in
walkthrough.mp4. Do not rely on memory of any dataset,
simulator, or house layout.
- Available tools are what is installed:
ffmpeg/ffprobe, and Python 3 with numpy,
scipy, pandas, pillow, opencv-python-headless, soundfile and matplotlib.