Skip to main content

DATASET 08 · Human-computer interaction

Real-World Voice Assistant Interaction Corpus

760 hours of voice interaction from 3,200 users across 5 real-world settings, with intent, slot and error-recovery annotation.

SpeechTextMultimodalHuman-computer interactionVoice products
Real-World Voice Assistant Interaction Corpus · cover artwork
Scale
760 hours
Participants
3,200 users
Languages
Mandarin and 6 common regional accents
Annotation
Wake-word and command transcription, Intent and slot annotation, Dialogue state tracking, Error-recovery behaviour coding
Formats
WAV / JSONL / CSV / TextGrid
Sync
Device log and audio sync error ≤ 2 ms
License
Commercial license / Research license
Updated
August 5, 2026

01 / Composition

Scale, key metrics and data streams

Observing people and voice assistants in real devices and real environments

760 h
Total duration
3,200
Users
5
Scenario classes
~420,000
Commands
Data composition and specifications
StreamSpecificationNotes
Audio16 kHz / 24-bit · near and far fieldDevice microphone and reference array on two channels
TextJSONL / UTF-8ASR output, human transcription and interaction logs
LogsJSONLWake, response, barge-in and retry events
Video1280 × 720 · 30 fpsOptional scenario video capturing interaction posture

The table lists capture parameters for a sample batch; final specifications follow the documentation delivered with the data.

02 / Dataset notes

Collection, annotation and quality control

The notes below explain how the data was collected, annotated and verified, and can be used as a basis for methodological citation and proposal assessment.

Overview

The Real-World Voice Assistant Interaction Corpus records 3,200 users interacting with voice assistants across five real-world settings, totaling 760 hours and roughly 420,000 commands recorded between August 2024 and May 2026. The settings are the home living room, a car cabin, an office, a public space and the outdoors, covering quiet, moderate-noise and high-noise acoustic conditions.

The corpus is stratified by device form factor (smart speaker, in-car voice, phone assistant, earbuds and wearables) and accent type, with six common regional accents annotated separately. Each user completes 20 to 45 minutes of free use and a task list; the task list covers four interaction types: queries, control, small talk and multi-turn correction.

Collection protocol

Audio is recorded on two paths: the device’s own microphone and a reference array deployed in the scene. The first reflects real recognition conditions, the second supports acoustic analysis and alignment; both are sampled at 16 kHz / 24-bit. Interaction logs are exported from the device side and record wake-word hits, response latency, barge-in and retry events, with log timestamps aligned to audio after device-clock correction.

Collection happens with users’ informed consent and does not record identifiable conversations of bystanders. Video is recorded only with separate user authorization and retains only interaction-related segments. For home settings, a research assistant visits to deploy the equipment; public-space and in-car settings follow the relevant venue permission processes.

Annotation

Annotation revolves around three questions: what the user said, what the system understood and how the user corrected it.

LayerContentScaleReview
TranscriptWake words, commands, ambient speech, overlapping speechAbout 420,000 commands20% double-blind review
SemanticsIntent, slot, reference and ellipsis38 intent classes, 64 slot classesDual annotation, arbitration
DialogueDialogue state, system confirmation, user correctionAll multi-turn sessionsRule validation + 15% spot-check
BehaviourRetry, abandonment, modality switch, volume increase6 behaviour classesCross-validation of logs and audio

Quality & verification

The median sync error between device logs and audio is 1.1 ms, with the 95th percentile at or below 2 ms. Human transcription character accuracy is 98.4%, intent annotation agreement is κ = 0.89 and slot annotation F1 is 0.92. Twelve percent of sessions in every batch are double-blind reviewed, and low-SNR segments recorded under high-noise conditions are flagged separately. Deliverables include device configuration notes, acoustic condition grading, the intent-slot taxonomy and an error analysis report.

Research questions

  • How wake-word false triggers and misses differ across noise conditions.
  • How regional accents affect intent recognition accuracy.
  • Users’ correction strategies and abandonment thresholds after repeated failures.
  • Comparison of interaction duration and barge-in patterns between in-car and home settings.
  • How voice coordinates with other modalities such as screens and gesture.

Access & licensing

The commercial license serves speech technology, smart device and in-car voice teams; the research license serves human-computer interaction and speech research laboratories. Data is delivered as WAV, JSONL, CSV and TextGrid, filterable by device form factor, setting and accent; raw device logs and human annotation are delivered in separate layers. Applicants must state their purpose and sign a data processing agreement. You can inspect the session-level field structure through the sample data page, or arrange an evaluation scoping conversation on the contact page.

03 / Typical use cases

Questions this dataset can answer

The directions below are typical research and product applications; a specific analysis plan depends on the study design and statistical method.

  • 01

    Far-field speech recognition evaluation

  • 02

    Dialogue system intent understanding

  • 03

    Wake-word false-trigger analysis

  • 04

    Multi-turn interaction error-recovery research

NEXT STEP

Need a different collection setting or annotation scheme?

An existing dataset can be licensed as it stands, or extended with additional speakers, scenarios and annotation layers around your study design.

Reply within one business day · NDA and data licensing supported · Dedicated academic track