DATASET 08 · Human-computer interaction
Real-World Voice Assistant Interaction Corpus
760 hours of voice interaction from 3,200 users across 5 real-world settings, with intent, slot and error-recovery annotation.
- Scale
- 760 hours
- Participants
- 3,200 users
- Languages
- Mandarin and 6 common regional accents
- Annotation
- Wake-word and command transcription, Intent and slot annotation, Dialogue state tracking, Error-recovery behaviour coding
- Formats
- WAV / JSONL / CSV / TextGrid
- Sync
- Device log and audio sync error ≤ 2 ms
- License
- Commercial license / Research license
- Updated
- August 5, 2026
01 / Composition
Scale, key metrics and data streams
Observing people and voice assistants in real devices and real environments
- 760 h
- Total duration
- 3,200
- Users
- 5
- Scenario classes
- ~420,000
- Commands
| Stream | Specification | Notes |
|---|---|---|
| Audio | 16 kHz / 24-bit · near and far field | Device microphone and reference array on two channels |
| Text | JSONL / UTF-8 | ASR output, human transcription and interaction logs |
| Logs | JSONL | Wake, response, barge-in and retry events |
| Video | 1280 × 720 · 30 fps | Optional scenario video capturing interaction posture |
The table lists capture parameters for a sample batch; final specifications follow the documentation delivered with the data.
02 / Dataset notes
Collection, annotation and quality control
The notes below explain how the data was collected, annotated and verified, and can be used as a basis for methodological citation and proposal assessment.
Overview
The Real-World Voice Assistant Interaction Corpus records 3,200 users interacting with voice assistants across five real-world settings, totaling 760 hours and roughly 420,000 commands recorded between August 2024 and May 2026. The settings are the home living room, a car cabin, an office, a public space and the outdoors, covering quiet, moderate-noise and high-noise acoustic conditions.
The corpus is stratified by device form factor (smart speaker, in-car voice, phone assistant, earbuds and wearables) and accent type, with six common regional accents annotated separately. Each user completes 20 to 45 minutes of free use and a task list; the task list covers four interaction types: queries, control, small talk and multi-turn correction.
Collection protocol
Audio is recorded on two paths: the device’s own microphone and a reference array deployed in the scene. The first reflects real recognition conditions, the second supports acoustic analysis and alignment; both are sampled at 16 kHz / 24-bit. Interaction logs are exported from the device side and record wake-word hits, response latency, barge-in and retry events, with log timestamps aligned to audio after device-clock correction.
Collection happens with users’ informed consent and does not record identifiable conversations of bystanders. Video is recorded only with separate user authorization and retains only interaction-related segments. For home settings, a research assistant visits to deploy the equipment; public-space and in-car settings follow the relevant venue permission processes.
Annotation
Annotation revolves around three questions: what the user said, what the system understood and how the user corrected it.
| Layer | Content | Scale | Review |
|---|---|---|---|
| Transcript | Wake words, commands, ambient speech, overlapping speech | About 420,000 commands | 20% double-blind review |
| Semantics | Intent, slot, reference and ellipsis | 38 intent classes, 64 slot classes | Dual annotation, arbitration |
| Dialogue | Dialogue state, system confirmation, user correction | All multi-turn sessions | Rule validation + 15% spot-check |
| Behaviour | Retry, abandonment, modality switch, volume increase | 6 behaviour classes | Cross-validation of logs and audio |
Quality & verification
The median sync error between device logs and audio is 1.1 ms, with the 95th percentile at or below 2 ms. Human transcription character accuracy is 98.4%, intent annotation agreement is κ = 0.89 and slot annotation F1 is 0.92. Twelve percent of sessions in every batch are double-blind reviewed, and low-SNR segments recorded under high-noise conditions are flagged separately. Deliverables include device configuration notes, acoustic condition grading, the intent-slot taxonomy and an error analysis report.
Research questions
- How wake-word false triggers and misses differ across noise conditions.
- How regional accents affect intent recognition accuracy.
- Users’ correction strategies and abandonment thresholds after repeated failures.
- Comparison of interaction duration and barge-in patterns between in-car and home settings.
- How voice coordinates with other modalities such as screens and gesture.
Access & licensing
The commercial license serves speech technology, smart device and in-car voice teams; the research license serves human-computer interaction and speech research laboratories. Data is delivered as WAV, JSONL, CSV and TextGrid, filterable by device form factor, setting and accent; raw device logs and human annotation are delivered in separate layers. Applicants must state their purpose and sign a data processing agreement. You can inspect the session-level field structure through the sample data page, or arrange an evaluation scoping conversation on the contact page.
03 / Typical use cases
Questions this dataset can answer
The directions below are typical research and product applications; a specific analysis plan depends on the study design and statistical method.
- 01
Far-field speech recognition evaluation
- 02
Dialogue system intent understanding
- 03
Wake-word false-trigger analysis
- 04
Multi-turn interaction error-recovery research
04 / Related datasets
Datasets that combine well
A single research question often calls for a cross-modal combination; the entries below share scenarios, speakers or annotation schemes with this dataset.
NEXT STEP
Need a different collection setting or annotation scheme?
An existing dataset can be licensed as it stands, or extended with additional speakers, scenarios and annotation layers around your study design.
Reply within one business day · NDA and data licensing supported · Dedicated academic track