AgenticVBench-Omni

AgenticVBench-Omni / Asset reconstruction

Slide Deck

slide-deck-session-reimplementation

session.mp4 (1920x1080, with narration) is one continuous screen recording of a professional slide deck being built from scratch in a presentation editor. It starts from a blank, new presentation and ends with a final pass through every finished slide in order. The deck uses one consistent visual theme throughout (a fixed colour palette and a small set of slide layouts).

Family
Asset reconstruction
Inputs
video
Media
1 video, 14 min, with sound
Reference answer
29 slides with 536 shapes
Deliverable
/workspace/output
Scoring
per-componentone-to-one matching

Task package · verifier code

Instruction

Full instruction given to the agent

Rebuild a slide deck from a screen recording

session.mp4 (1920x1080, with narration) is one continuous screen recording of a professional slide deck being built from scratch in a presentation editor. It starts from a blank, new presentation and ends with a final pass through every finished slide in order. The deck uses one consistent visual theme throughout (a fixed colour palette and a small set of slide layouts).

Rebuild that deck: produce the .pptx it had at the end of the recording.

Deliverable

output/deck.pptx, a valid PowerPoint Open XML file (e.g. written with python-pptx). Only this file is read. The canvas is 13.333 x 7.5 inches (widescreen 16:9; in EMU, 12192000 x 6858000, 914400 EMU per inch). Your deck must have the same number of slides, in the same order, as the recording; slide 1 of your file is compared with the first slide shown, and so on.

What is not scored

  • All text. Wording, fonts, exact character count, and any placeholder prompt text — never scored. Do not spend effort matching phrasing; you can leave text boxes empty or put in anything.
  • Picture content. Where the recording shows a photo, icon or logo, put any image (or a plain filled rectangle) in that spot — only its position, size, rotation and stacking matter, never its pixel content.
  • A few things are said in the narration but never actually built, or built and then explicitly undone before the end ("actually, scratch that"). Nothing about them is scored — do not add anything for a change that was only discussed or that was reversed.

What must match

Everything below is compared only in its final, end-of-recording state; some elements are placed, then visibly adjusted or corrected later in the recording; only where they end up counts. Positions/sizes are in EMU. Rotation in degrees (0-360, clockwise, as PowerPoint stores it).

  1. Slide structure. For each slide, which of these four kinds of layout it uses, judged by the title text box alone (there is no other placeholder in this deck): - title_slide: a large, centered title (with or without a subtitle under it), and nothing else on the slide. - title_band: a title box spanning most of the slide's width, positioned near the top. - caption_title: a title box confined to roughly the left third of the slide width or less. - blank: no distinct title text box at all.

  2. Every shape shown, including shapes nested inside a group. A shape is scored as one unit; every part below must match for it to count: - Kind, from this closed list: rectangle, rounded rectangle, oval, teardrop (a rounded shape with one pointed corner), line, freeform/irregular shape, text box, title or subtitle placeholder, picture, table, chart, group. - Position (top-left corner) within 0.06 inch (54864 EMU). - Size (width and height) within 0.12 inch (109728 EMU). - Rotation within 5 degrees. - Fill colour: either an exact RGB colour (read from the recording; matched within a small colour-distance tolerance) or one of the theme's colour roles — background 1/2, text 1/2, accent 1-6, hyperlink, followed hyperlink — with a tint/shade (brightness) applied. If the recording shows a shape's fill picked from the deck's small, repeated set of swatches, using the matching theme colour role (shape.fill.fore_color.theme_color in python-pptx) is the reliable way to match it; an RGB value close enough to that swatch also counts. - Line (border) colour, the same way, only for shapes that visibly have one.

  3. Stacking order (front-to-back). For every pair of shapes on the same slide (or in the same group) whose final positions visibly overlap, whichever one is drawn on top in the recording must also be the one on top (later, i.e. added later to the slide's shape list / a group's shape list) in your file. Shapes that never overlap anything are not scored on this.

  4. Grouping. Shapes the recording visibly groups together (so they move as one) must be a single GroupShape in your output, containing exactly those shapes.

  5. Tables. Row and column count. For every cell that is visibly shaded a distinct colour (e.g. a header row or banding), that cell's fill in your table must match, the same way shape fills are matched above. Cell text is not scored.

  6. Charts. The chart's type (bar, line or pie, closed list — a donut is a pie) and its number of categories. Where the recording shows the chart's own numbers actually being set or changed from whatever a brand-new chart starts with, each series' values (read from the shape sizes/proportions on screen) must match within 15% of the true value; a chart whose numbers are never shown changing from a fresh chart's own starting data is only checked on type and category count.

Master/layout inheritance, chart data, table shading, grouping and stacking order are the reasons this task is hard from watching alone: none of them are stated out loud with an exact number, and none of them can be read as text. The narration tells you what is being done and in what order, and occasionally which of several similar-looking shapes is which; the picture is where every exact position, size and colour comes from.

Environment

  • Work only inside your current working directory. There is no internet access. Do not use any memory you may have of this deck or its authors — build only what the recording shows.
  • Python 3 with python-pptx, lxml, XlsxWriter, numpy, scipy, pandas, pillow, opencv-python-headless is installed; pip install will not work. ffmpeg/ffprobe and faster-whisper (model small.en) are available if you want a transcript of the narration.
  • Nothing else about the session is provided; there is no starting file.

Media

Media files are distributed separately from the repository; this task’s hash manifest lists 1 files (environment/media/SHA256SUMS.txt). Licences named for this task: see the provenance notes in the task package.

Score

AgentScore (0 to 1)
GPT-6-Astra0.218
Claude Opus 5.50.298
Claude Opus 50.157
Gemini 3.8 Flash0.054
DeepSeek 4.1 Flash0.086

One run per agent.

The agent watches a narrated screen recording of a slide deck being built from a blank presentation and delivers the finished deck as a .pptx. The verifier reads both files as XML, without opening an editor. The expert’s deck has 29 slides and 536 shapes.

What a good answer has to get right, and how the verifier checks it

Each slide uses the right layout
One item per slide: its layout, from a fixed list of kinds.
Each shape is reproduced
One item per shape of the expert’s deck, including shapes inside groups. Shapes are paired per slide by geometry, and a shape counts when its kind, fill and line match, its position is within 0.06 inch, its size within 0.12 inch and its rotation within 5°.
Front-to-back order is kept
One item per pair of overlapping shapes: which is in front.
Groups are built as groups
One item per group: every member must sit in the same group of the submission, and that group must have exactly as many members.
Tables and charts carry the right content
For a table: the row and column count, and each shaded cell. For a chart: its type, the number of categories, and each series value within 15% where the expert changed it from the editor’s default.
An equivalent construction is accepted
A picture in the expert’s deck may be matched by a plain filled rectangle in the same place. The instruction says so: only a picture’s position, size, rotation and stacking are compared.

How the checks become one score

Score = items right / items in total, counted over all items together and not averaged per kind, so the one layout item per slide cannot outweigh the hundreds of shape items. Each item is right or wrong as a whole.

What scores zero

No deck.pptx, or a file that cannot be read as a presentation.

Outside the score: all text (wording and fonts) and the content of pictures, which the instruction lists as never scored, and chart values the expert left at the editor’s default.

The score line and constants in the verifier code

Reward formula in the verifier code

Two sampled runs

Two of the five runs on this task in the paper’s diagnostic sample, which has one run per system: the highest-scoring run, and the run in the middle of the other systems by score. Only runs that ended normally and whose record could be exported are eligible. Each is shown step by step: what the agent said, which tools it called, and what it got back (long outputs and images are clipped). The verifier's output for the run comes first, then the steps.

Agent
Claude Opus 5.5 · Claude Code 2.1.280
Score
0.30
Wall time
30.4 min
Steps shown
136

Loading the run…