Abstract:Users often take notes for instructional videos to access key knowledge later without revisiting long videos. Automated note generation tools enable users to obtain informative notes efficiently. However, notes generated by existing research or off-the-shelf tools fail to preserve the information conveyed in the original videos comprehensively, nor can they satisfy users' expectations for diverse presentation formats and interactive features when using notes digitally. In this work, we present NoteIt, a system, which automatically converts instructional videos to interactable notes using a novel pipeline that faithfully extracts hierarchical structure and multimodal key information from videos. With NoteIt's interface, users can interact with the system to further customize the content and presentation formats of the notes according to their preferences. We conducted both a technical evaluation and a comparison user study (N=36). The solid performance in objective metrics and the positive user feedback demonstrated the effectiveness of the pipeline and the overall usability of NoteIt. Project website: this https URL
| Comments: | Accepted to UIST 2025. Project website: this https URL |
| Subjects: | Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2508.14395 [cs.HC] |
| (or arXiv:2508.14395v1 [cs.HC] for this version) | |
| https://doi.org/10.48550/arXiv.2508.14395 arXiv-issued DOI via DataCite |
|
| Related DOI: | https://doi.org/10.1145/3746059.3747626
DOI(s) linking to related resources |
Submission history
From: Running Zhao [view email]
[v1]
Wed, 20 Aug 2025 03:45:18 UTC (5,797 KB)