Egocentric video datasets
Egocentric video of hands at work, captured to a brief: the task, the environment and the device are fixed before anyone records.
The public egocentric datasets
Twelve public egocentric datasets, as their papers or dataset cards describe them, with what each licence says about training a commercial model.
| Dataset | Size, as published | Built for | Licence, and what it says about commercial training |
|---|---|---|---|
| Ego4D | 3,670 hours from 931 camera wearers (paper) | Daily-life perception benchmarks | Ego4D licence agreement: allowed for listed uses; the footage may not be redistributed |
| Ego-Exo4D | 1,286 hours, first- and third-person views together (paper) | Skilled activity, seen from both views | Ego-Exo4D licence agreement, signed on request: its documentation says it covers commercial use |
| EPIC-KITCHENS-100 | 100 hours in 700 videos (paper) | Kitchen action recognition | CC BY-NC 4.0: commercial licences arranged by email |
| EgoDex | 829 hours (paper) | Dexterous manipulation, with 3D hand tracking | CC BY-NC-ND: no commercial use |
| HOI4D | 2.4M RGB-D frames in 4,000 sequences (paper) | Category-level hand-object interaction | CC BY-NC 4.0: no commercial use |
| HoloAssist | 166 hours from 350 instructor-performer pairs (paper) | Assistants that guide physical tasks | CDLA-Permissive-2.0: commercial training allowed |
| Assembly101 | 4,321 videos; 4 of its 12 camera views are egocentric (paper) | Multi-view assembly procedures | CC BY-NC 4.0: no commercial use |
| Aria Everyday Activities | 143 activity sequences (paper) | Multimodal scene and activity research | AEA licence agreement: non-commercial research only |
| Nymeria | 300 hours from 264 participants (paper) | Full-body motion with language descriptions | CC BY-NC 4.0: no commercial use |
| HOT3D | Over 833 minutes (paper) | 3D hand and object tracking | HOT3D licence agreement: the footage, under CC BY-SA, allows it; the hand labels, under CC BY-NC-SA, do not |
| Ego-1K | 956 clips, 2.3 hours in total (paper) | Multiview 3D synthesis of hand-object scenes | FAIR Noncommercial Research License: no commercial use |
| Egocentric-10K | 10,000 hours (dataset card) | Factory-floor manipulation footage | Apache 2.0, after accepting access conditions on Hugging Face |
Most of these licences rule out commercial training or need a separate agreement, and Ego4D's also bars redistributing its footage. The licence column is our reading of each licence, not legal advice: read the licence before you train on a dataset. For a commissioned brief, the terms a delivery may be used under are agreed in the engagement.
What an egocentric brief pins down
Each row is agreed before the first Recording exists.
| What the brief pins down | How it is set |
|---|---|
| Task and environment | Named in the brief. The catalogue lists each brief with its availability |
| Device | Named in the brief, from Ray-Ban Meta glasses to a stereo global-shutter head rig |
| Consent | Given by the Capture Partner before recording starts, as a receipt in the hash-chained consent ledger. A withdrawal is appended to the same ledger |
| Capture Partner pay | From ₹229/hr of accepted capture, paid directly by UPI. Pay terms |
| Delivery | A directory named by a hash of its contents, with a manifest of every file's SHA-256 and a compliance log covering every Clip. How a delivery is checked |
Questions about egocentric datasets
What is an egocentric video dataset?
Video recorded from a camera worn at the head or eyes, so it shows what the wearer sees and what their hands do. Robotics teams use it to learn how people handle objects.
Can a company train a commercial model on a public egocentric dataset?
Sometimes: HoloAssist's licence allows it and Ego4D's allows listed uses, but most datasets in the table are non-commercial or need a separate agreement. Read the licence before training; a commissioned brief agrees the terms before capture.
Which devices record egocentric video?
The datasets above used head-mounted cameras, Project Aria glasses, HoloLens 2, Meta Quest 3 and Apple Vision Pro. A Capture Brief names the device before the first Recording.
How is a commissioned dataset different from a public one?
You choose the task, the environment, the device and the annotation, and the terms of use are agreed before capture. Every Recording comes with a consent receipt you can check on our ledger.
Scope an egocentric brief
Send the task and the environment your policy needs footage of. We scope the brief with you, from the device to the delivery format.