shot-reader-8b-2 / schema.md
multimodalart's picture
multimodalart HF Staff
Upload folder using huggingface_hub
f86d1b1 verified
|
Raw History Blame Contribute Delete
2.76 kB

Fact sheet schema (v1)

One JSON object per shot. Written by a sheet writer (frontier model or the shot reader) from 8–16 frames of one shot plus the subject/prop ids from the brief.

{
  "shot": {"start": 0.0, "end": 4.2},
  "camera": {"height_m": 0.4, "angle": "low", "pitch_deg": 12, "fov_deg": 40,
             "move": "orbit", "framing": "medium", "subject": "P01"},
  "subjects": [
    {"id": "P01", "class": "person", "size_m": [0.5, 0.5, 1.8], "posture": "stand",
     "path": [{"t": 0.0, "pos_m": [1.2, 3.4], "facing_deg": 90},
              {"t": 3.1, "pos_m": [2.6, 5.0], "facing_deg": 45}]},
    {"id": "chair_1", "class": "chair", "size_m": [0.5, 0.5, 0.9],
     "path": [{"t": 0.0, "pos_m": [2.0, 1.1]}]}
  ],
  "events": [{"t": 3.1, "what": "P01 reaches door"}]
}

Coordinates

  • Ground plane, metres. Origin = the point on the ground directly below the camera in the shot's first frame.
  • +x = to the camera's right, +y = forward along the camera's initial heading (its view direction projected onto the ground) in the first frame.
  • pos_m is the centre of the subject's ground footprint.
  • facing_deg: the direction the subject faces, degrees counter-clockwise from +x. So 90 = facing away from the camera, 270 (or -90) = facing the camera.
  • size_m = [width, depth, height] of the subject's bounding box.
  • t in seconds from the shot start. Paths are piecewise linear between keys and hold their end values outside them.

Camera

  • height_m: lens height above the ground in the first frame.
  • pitch_deg: positive = looking up, negative = looking down, first frame.
  • fov_deg: horizontal field of view, first frame.
  • angle (from pitch): low (pitch > +5), eye (βˆ’5..+5), high (βˆ’5..βˆ’50), overhead (< βˆ’50).
  • Height bands (scored by the gate, from height_m): ground < 0.6 m ≀ low < 1.2 m ≀ eye < 1.8 m ≀ high < 4.0 m ≀ aerial.
  • move: one of
    • static β€” no translation or rotation (handheld jitter allowed)
    • pan β€” rotates left/right in place
    • tilt β€” rotates up/down in place
    • push_in β€” translates forward along the view direction
    • pull_out β€” translates backward
    • truck β€” translates sideways, orientation fixed
    • crane β€” translates up or down (pedestal/crane), may re-aim
    • orbit β€” circles around a subject or point while aiming at it
    • follow β€” translates with a moving subject, keeping it framed
  • framing of camera.subject in the first frame, by visible subject height / frame height r: wide r < 0.35, full 0.35–0.8, medium 0.8–1.6, close β‰₯ 1.6.

Classes (grey-box generator)

person (posture stand | sit), chair, table, sofa, car, crate, lamp, bench, pillar.