One take, no edits
Each segment is a stretch of video with no cuts, jump cuts or zooms. Pauses, overlaps and the moment before someone starts to speak keep their real timing.
31 hours of real, unscripted conversation between two or three people. Each segment is one continuous take from a single static camera, and every word is transcribed, timed and tied to the face of the person who said it.
Test video diiJlowDHbA, segment 29. The boxes, captions, timeline and turn events are all drawn from the released annotation files.
A turn-taking model predicts who speaks next, and when. To learn that from video it needs conversations whose timing is untouched, where everyone stays in view and every voice belongs to a known face. AVCC was collected for exactly that.
Each segment is a stretch of video with no cuts, jump cuts or zooms. Pauses, overlaps and the moment before someone starts to speak keep their real timing.
A single static camera shows all participants, as an outside observer would see them. A model sees the listeners as well as the speaker.
Speakers 0, 1 and 2 keep the same label in the words, utterances, RTTM and per-frame face boxes for the whole segment, so every voice is matched to a face.
Word-level transcripts were force-aligned, then corrected by hand. The test set adds balanced hold/shift events and next-speaker guesses from 154 people.
Test video sL3jHOX826s, segment 4. Turn the sound on.
Real conversation is not tidy. Here two speakers talk at the same time for several seconds, back-channels like "yeah" and "mhm" land in the middle of other people's sentences, and the floor still changes hands. Every word keeps its own speaker and timing, so overlaps are annotated, not smoothed away.
Test video yGGsg7_xxsI, segment 10. Turn the sound on.
Podcasts, livestreams and get-togethers, recorded in studios, living rooms, kitchens and garages. People eat, drink, stand up, walk around and talk over each other. Each loop is five seconds of a released segment with its annotations drawn on.
Because nothing is cut, the timing between speakers is exactly as it happened. The next speaker often starts before the previous one has finished, and gaps are short.
Each speaker change between two utterances, in 0.1 s bins. Negative values are overlaps: the next speaker started early. Two- and three-person segments are shown separately, as share of all speaker changes in that group.
AVA-ActiveSpeaker and MSDWild are the standard audio-visual benchmarks for active speaker detection and diarization. Both are valuable, but neither lets you study multiparty turn-taking from video, for two different reasons.
| AVA-ActiveSpeakerRoth et al., ICASSP 2020 | MSDWildLiu et al., Interspeech 2022 | AVCCQi & Skantze, Interspeech 2026 | |
|---|---|---|---|
| Source | 15-minute clips of feature films | Public videos, mostly vlogs | Podcasts and livestreams from YouTube and Twitch |
| Speech | Scripted film dialogue | Spontaneous, many languages | Spontaneous, English |
| Size | 38.5 h of face tracks | 80 h, 3,143 clips | 31.0 h, 408 segments from 29 conversations |
| Camera | Edited film with frequent shot changes | Edited: jump cuts and zooms inside clips | One static camera, no edits |
| Unit of labelling | Face tracks, speaking or not. Not linked to people | Speakers (who spoke when), plus face boxes | Speakers, with one label shared by words, RTTM and face boxes |
| People in view | Mostly one face at a time | 2 to 10 speakers per clip | 2 or 3, all on screen 96.9% of the time |
| Transcripts | None | None | Word level, hand-checked |
| Turn-taking | None | None | 1,730 hold/shift events and a human baseline |
Swipe the table sideways to see MSDWild and AVCC.
AVA and MSDWild sizes are from their papers. The AVA figures below are measured on its labels for the 153 train and val movies, the MSDWild speaker counts on its RTTM files, and the MSDWild frames come from its released videos.
AVA marks each face track as speaking or not. A track ends at every shot change and is never linked to the person, so the same actor gets a new ID each time the film cuts back to them. You can tell whether a face is talking, but not who holds the turn or who takes it next.
MSDWild is spontaneous conversation, but its clips come from edited videos. Jump cuts skip time and zooms re-frame the scene inside a single clip, so a pause on screen is not the pause that happened. Each clip below is four seconds around one edit; the badge appears on the frame where it happens.
AVCC is on Hugging Face. Request access with your name, institution and what you will use it for. It is free for non-commercial research.
hf auth login
hf download Haotian-Qi/AVCC --repo-type dataset \
--local-dir AVCC
python3 AVCC/script/unzip_videos.py --delete-zips
python3 AVCC/script/viewer.py # browse any segment
The release includes tools to rebuild utterances and RTTM from the words, extract face crops, audio and model windows the way MuVAP does, browse any segment and correct the annotations. Formats are described on the dataset page.
| Split | Videos | Segments | Hours | Words |
|---|---|---|---|---|
| Train | 17 | 258 | 20.6 | 230,071 |
| Val | 5 | 54 | 2.7 | 30,820 |
| Test | 7 | 96 | 7.7 | 95,697 |
| Total | 29 | 408 | 31.0 | 356,588 |
Splits are by video, so no conversation appears in two splits. Videos are 25 fps, mostly 1920×1080, with 44.1 kHz audio.
If you use AVCC, please cite the AVCC paper. AVCC was first introduced with MuVAP, a multimodal model that predicts multiparty turn-taking from one camera and one microphone; if you also use MuVAP or compare against it, please cite the MuVAP paper as well.
@misc{qi2026avcc,
title = {{AVCC: Audio-Visual Conversation Corpus for Turn-Taking Prediction with a Human Benchmark}},
author = {Haotian Qi and Gabriel Skantze},
year = {2026},
eprint = {2611.13957},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2611.13957},
}
@inproceedings{qi26_interspeech,
title = {{MuVAP: Multimodal Multiparty Voice Activity Projection for Turn-taking Prediction in the wild}},
author = {Haotian Qi and Gabriel Skantze},
year = {2026},
booktitle = {{Interspeech 2026 [Long Track]}},
pages = {5952--5961},
doi = {10.21437/Interspeech.2026-1381},
issn = {2958-1796},
}