Capture catalogue
Robot-ready capture, specified before it is collected
Every entry below is a capture specification we can scope and deliver: domain, sensor stack, annotation tier and delivery format fixed up front, so you know exactly what arrives. Read the FH-Ego v1 schema on any entry before committing to a pilot.
Every brief states where it stands:
Pipeline validated the annotation pipeline has been run end-to-end on a real recording of this task family
Scoped on demand quoted per engagement
Research phase under evaluation; not open for commission
Capture briefs you can commission across 6 domains. Each specification below states its availability.
8 of 8 specifications
| Specification | Domain | Availability | Capture device | Camera | Spatial tracks | Annotation |
|---|---|---|---|---|---|---|
| Precision Factory Assembly | Manufacturing | Pipeline validated | Stereo Global-Shutter Head Rig | 2560x960 @ 90 FPS (Stereo) | 6-DoF Camera Trajectory (SLAM) & Sync IMU (400Hz) | Tier 1 · Tier 2 · Tier 3 |
| Bimanual Household Tasks | Kitchen | Scoped on demand | Ray-Ban Meta | 1920x1080 @ 30 FPS | Camera-relative wrist position (x, y, z), no physical unit | Tier 1 · Tier 2 · Tier 3 |
| Specialised Caregiving & Nursing | Healthcare | Scoped on demand | GoPro Hero 12 | 3840x2160 @ 60 FPS | IMU Synchronized Telemetry (200Hz) | Tier 1 · Tier 2 |
| Warehouse & Logistics | Logistics | Scoped on demand | GoPro Hero 12 | 1920x1080 @ 30 FPS | Wrist-mounted IMU sensor synchronization | Tier 1 · Tier 2 |
| Cleaning & Maintenance | Household | Scoped on demand | Ray-Ban Meta | 1920x1080 @ 30 FPS | Wide-angle camera lens intrinsics | Tier 1 · Tier 2 |
| Agriculture & Outdoor Tasks | Outdoor | Scoped on demand | iPhone LiDAR | 1920x1080 @ 30 FPS | LiDAR depth map (Apple's fused 256×192) | Tier 1 · Tier 2 · Tier 3 |
| Video RLHF Policy Annotations | RLHF | Scoped on demand | Human Feedback Portal | Set by the Commissioning lab's rollouts | Reward score, preference rankings, and failure rationales | Tier 1 · Tier 2 |
| Game Environment & Simulator Data | Simulation | Research phase | Simulator Engine | Synthetic RGB-D @ 60 FPS | Ground-truth joint angles, SLAM trajectory, and depth matrices | Tier 1 · Tier 2 |
Common to all eight
What every brief includes, whatever you commission
The specifications above differ in domain, device and tier. Everything below is the same for all eight.
How a brief is scoped, and what "on demand" means
A brief marked scoped on demand is scoped, recruited for and quoted as one engagement when a Commissioning lab asks for it. Pipeline validated means a recording from that task family has been through the pipeline end to end and is published in full with its checksum; the badge describes the pipeline, not a rig. Research phase means under evaluation and not open for commission.
Three annotation tiers, one schema
Every brief is annotated against FH-Ego v1 and against nothing else. Tier 1 is the task and session record, Tier 2 is the action segment, and Tier 3 is the timeline tracks that carry the kinematics. The Annotation column above says which tiers a brief includes.
The exporter offers no custom field set. It emits one schema, and every field of it is published with its shape, its unit, its sampling rate and the tier it belongs to: FH-Ego v1, field by field.
Every FH-Ego v1 field is read from the video. The other streams in the Spatial tracks column, such as SLAM, IMU, depth maps and lens intrinsics, are streams of the rig itself, not FH-Ego v1 fields; how they travel with a delivery is set in the engagement.
Five delivery containers, and two we decline
The same annotated record is written into any of five containers, each read back off disk by its own verifier before it ships. Two more are declined.
Which five, which two, and the reasoning for each: what arrives, and in what container.
A consent receipt, written before the first frame is uploaded
Every recording is consented on the Capture Partner's own phone before capture, and the receipt is written before the upload begins. A receipt is eleven fields hashed into a chain, each entry carrying the hash of the one before it, so a receipt cannot be edited after the fact without breaking every entry that follows.
Consent is granted over the recording, not over a clip, and it can be withdrawn at any time. What a withdrawal does to a delivery is set out below.
You can test a receipt yourself, in your own browser, against the published chain: check a consent receipt. How the chain is built and what its digest proves: the consent ledger.
Faces blurred at ingestion
Faces the detector finds are blurred when footage arrives, before any person here opens it. If redaction cannot run, the clip stops there.
Audio is removed rather than blurred: what a Commissioning lab receives carries no audio track, and that is checked rather than assumed — the encoder writes video only, and the stage reads its own written file back and raises if it finds sound.
How redaction runs, and what happens when it cannot: how faces are blurred.
What has to happen before a sample appears
A sample appears on a brief page once the brief is commissioned and Capture Partners have recorded against it. The pipeline has run end to end on 1 recording, from precision factory assembly, published in full with its annotation and its checksum: the reference implementation.
The stages a recording passes
Five stages are implemented. The last 3, from TRACK on, run only on work a person has released.
| Stage | What it does | Today |
|---|---|---|
BLUR | Faces detected and blurred, before any person here opens the footage. | Runs today |
SEGMENT | The continuous upload cut into task-bounded clips, and each clip judged. | Runs today |
TRACK | Hand landmarks read per sampled frame. | Parked |
CAPTION | A caption written for each action segment. | Parked |
EXPORT | The annotated record written into the delivery container. | Parked |
The refusals
What a delivery does not include
These checks run when a package is built: a failed check stops the package and names the cause. They describe what the system does, not a warranty; warranties are agreed in writing for each engagement.
Footage nobody consented to
Every clip's consent receipt is re-checked at the moment a package is assembled, not at the moment the release was cut. A Capture Partner who withdraws after a release is frozen still stops that delivery — byte-identical footage becomes undeliverable, and the package refuses rather than quietly shipping without it.
You can test any receipt against the published chain in your own browser: check a consent receipt.
A package that does not verify as the package we named
A release that does not verify intact refuses. So does a clip with no exported annotation — a package with silently missing annotations is refused by name rather than shipped light. The delivery's own identifier is a hash over the sorted listing of every file inside it, each path paired with its own checksum, so rebuilding that listing is a test you can run on what arrives.
A verdict you have to take on trust
Every clip in a delivery carries its QA verdict and any rejection reasons in the compliance log, alongside its content hash, its consent receipt identifier and the status that receipt held at packaging time. The judgement travels with the data rather than staying with us. How a clip is judged.
A figure without its provenance
Every figure we publish carries its provenance and how it was computed, and the record is annotated field by field: what each one is, its shape, its unit, how often it is sampled and which layer produced it.