CASTLE

CASTLE is a dataset of egocentric video with synchronised metadata, designed to support the structured, entity-centric analysis and retrieval in such collections. More information and direct access to the dataset can be found on its webpage. It is the basis of an associated series of challenges that evaluate such approaches in a setting that does not require live interaction.

The CASTLE 2024 dataset was released at ACM Multimedia 2025, where the first CASTLE Grand Challenge also took place. A follow-up asynchronous Challenge edition ran in conjunction with the Joint Egocentric Vision Workshop at CVPR 2026.

Publications

• The CASTLE 2024 Dataset: Advancing the Art of Multimodal Understanding — Luca Rossetto, Werner Bailer, Duc-Tien Dang-Nguyen, Graham Healy, Björn Þór Jónsson, Onanong Kongmeesub, Hoang-Bao Le, Stevan Rudinac, Klaus Schoeffmann, Florian Spiess, Allie Tran, Minh-Triet Tran, Quang-Linh Tran, Cathal Gurrin — Proceedings of the 33rd ACM International Conference on Multimedia (MM 2025), Dublin, Ireland (pp. 12629–12635). PDF · doi:10.1145/3746027.3758199
• Overview of the First CASTLE Grand Challenge at ACM Multimedia 2025 — Luca Rossetto, Werner Bailer, Cathal Gurrin, Duc-Tien Dang-Nguyen, Klaus Schoeffmann, Allie Tran — Proceedings of the 33rd ACM International Conference on Multimedia (MM 2025), Dublin, Ireland (pp. 14271–14272). PDF · doi:10.1145/3746027.3760242
• Results of the 1st Asynchronous CASTLE Challenge at the Joint Egocentric Vision Workshop in Conjunction with CVPR 2026 — Luca Rossetto, Werner Bailer, Cathal Gurrin, Graham Healy, Omar Shahbaz Khan, Stevan Rudinac, Klaus Schoeffmann, Allie Tran — CoRR (arXiv:2608.22914). doi:10.48550/arXiv.2608.22914