Case-by-case browsers for multimodal / video-understanding benchmarks. Each viewer ships with a representative subset so it stays hostable on GitHub Pages; links to the official source are in the footer of every viewer.
Three-stage conversation mining pipeline demonstrated on two YouTube videos (a tennis serve lesson and an IELTS speaking test). Stage 1: timestamped transcripts, YouTube captions vs forced ASR (faster-whisper large-v3). Stage 2: speaker attribution and turn reconstruction (pyannote community-1 diarization, conservative word-level assignment, filler cleanup). Stage 3: semantic interaction grouping — Gemini 2.5 Pro proposes contiguous episode ranges under a JSON schema and the program validates the partition (candidate / review / skip, visual-context flags).
Fixed-camera surveillance anomaly benchmark (KISA2016): 100 videos, 6 event classes (Abandonment / Arson / Falldown / Intrusion / Loitering / Violence) with XML alarm intervals and DetectArea polygons. The viewer pairs the dataset preview with a pilot VLM evaluation — streaming per-second detection (JoyAI-VL-8B) vs offline whole-video QA (Qwen2.5-VL-3B/7B, Cosmos-Reason1-7B) — including per-clip case studies with unified alarm/response timelines and hallucination / region / attribute probes.
A stratified 46-video seed set mixing all four target benchmarks (Wearable-AI, CaptainCook4D, HowTo100M, EgoSchema), picked for diversity — deliberately not cooking-only, not manipulation-only. Balanced across event / scenario type (procedural, instructional, excursion, sport-performance, social-event, shopping, daily-routine), task-interaction form, temporal scale, viewpoint, and hand-object interaction intensity. Includes a video-plans page: each video's whole-procedure timeline (Gemini 2.5 Pro, grounded steps + timestamps) that feeds synthetic QA.
The successor to the Persona MVP, now on real CaptainCook4D egocentric cooking video. It fixes the MVP's core limitation — synthetic errors that contradict what the frames show — with an anti-poison gate: every error class is tagged visually indetectable (measurement / timing / temperature — frames are silent on them) or detectable (technique / order / missing-steps — frames show them); a detectable error may only be synthesized when anchored to a real labeled mistake at that step, otherwise it is downgraded — guaranteeing 0 poison samples. Personas are built Sparse-to-Rich (seed → expand → NLI self-consistency repair via DeBERTa-v3); a seeded code-side gate (never the LLM) decides clean vs which feasible error. Browse every sample with its gate branch, real anchor, persona, and the exact Gemini prompts, inline with the source video.
Two egocentric instructional benchmarks for wearable / streaming assistants. Meta Wearable-AI (ECCV 2026): EgoLongQA long-video multiple-choice, EgoConv multi-turn conversational QA, and EgoProactive interrupt-vs-silent decisions over head-mounted video (2,100 videos, 3 tasks × 700). Qualcomm Interactive Cooking (NeurIPS 2025, built on CaptainCook4D): live timed instruction + feedback streams with a 5-type mistake taxonomy (measurement / preparation / technique / timing / temperature) timestamped to the video.
Qualcomm AI Research benchmark from "Streaming Interventions: Can Video LLMs Correct Mistakes as They Occur?". Egocentric GoPro cooking videos in which the cook deliberately makes mistakes (wrong quantity, wrong tool, skipped step); expert annotations give the full assistant script — step Instructions, Success confirmations, and corrective Feedback timestamped to the exact moment each mistake occurs, plus user questions and clarifications. The viewer plays each video next to its clickable annotated transcript with a type-colored intervention timeline synced to playback.
Meta's proactive procedural assistance benchmark (ECCV 2026 Wearable AI Workshop), dedicated full-schema viewer. Three tasks on egocentric H.265 1080p video: EgoLongQA long-video multiple-choice QA, EgoConv multi-turn conversational QA with timestamped P1/P2/Assistant dialogs and typed questions (multimodal / unimodal / multi-turn), and EgoProactive streaming interrupt-vs-silent decisions per video chunk. Each case shows its task-specific rendering — MCQ options, seekable dialog with visible-interval timeline, or per-chunk decision table — plus the complete raw JSON record.
Situated question + answer benchmark for a smart-glasses AI assistant. 11 task types in 3 families: Proactive monitoring (event detection with repeating or first-occurrence trigger, interval tracking, threshold alerts), Reactive Q&A (localization with both past-placement & forward search, recall including temporal "when?", diagnosis & guidance, verification, and cross-time comparison), and Interaction dynamics (concurrent tracking, instruction update, causal chains). Gemini 2.5 Pro first role-plays the person to write the questions they would ask, then role-plays the AI assistant to emit the spoken responses — `(response_time, response_text)` lists for proactive instructions, single answers for reactive questions, with an honest can't-answer escape hatch.
Persona-driven proactive-intervention generator on EgoSchema procedural clips. Gemini 2.5 Pro role-plays 8 explicit user personas (impatient novice, distracted, misreader, wrong-tool, careful-slow, overconfident, temperature-blind, expert-clean control); for each (clip, step, eligible persona) it watches the video and writes the deviation that persona would naturally make at that moment, together with a `(onset_ts, deviation_description, proactive_query, gold_guidance)` tuple that can be serialized into SPOT-Bench or Ego-MC-Bench training rows. The v2 prompt adds explicit contrast across peers to widen per-step deviation diversity. The wearer in the real video did NOT actually make this error — visual realism is out of scope for this MVP and is the next milestone on CaptainCook4D / HoloAssist.
MLLM streaming video understanding benchmark. Four tasks: Real-Time Visual Understanding, Omni-Source Understanding, Sequential QA, and Proactive Output.
Omni-modal interactive benchmark for streaming video dialog. Six tasks covering action prediction, state grounding, proactive alerting, and speaker identification.
Proactive video question-answering with time-aligned streaming replies. Four subsets: WEB (open-domain), EGO (first-person), TV (dialogue), and VAD (anomaly detection).
Online video understanding across three temporal modes: backward tracing, real-time perception, and forward active responding. 12 fine-grained tasks.
Anomalous Events in Transportation (AI City Challenge 2026). Unified video VLM benchmark with 10 task types per video — binary verification, MCQ, temporal localization, causal linkage, and chain-of-thought reasoning.
Side-by-side outputs of three Qwen3-VL variants (zero-shot 2B, zero-shot 4B, and our Stage 0 LoRA r=16 fine-tune) on the same 50-item smoke-eval set. Per-task aggregate charts plus per-sample browser showing video, prompt, GT reasoning, and each model's full chain-of-thought + answer with auto-scored correctness pills.
Per-event sub-clips of the EgoSchema and Action100M candidates, segmented by Gemini 2.5 Flash. For each 3-minute source video, Gemini emits a list of coherent sub-events (e.g. wash → chop → fry); ffmpeg cuts each event into its own mp4. The viewer shows the source video alongside an interactive event timeline and a grid of playable sub-clips.
Benchmark-candidate clips auto-filtered from EgoSchema (5,031 × 3-minute egocentric clips, sourced from Ego4D). Each clip kept by GPT-4o judging the clip's Ego4D narration timeline against domain + multi-event criteria — kitchen, manipulation, or navigation, with > 3 distinct procedural events.
Benchmark-candidate videos auto-filtered from Action100M-preview (HowTo100M source). Each video was kept by GPT-4o judging Action100M's hierarchical Tree-of-Captions against domain + multi-event criteria — kitchen, manipulation, or navigation, with > 3 distinct procedural events.