DATASET 02 · Educational linguistics
Classroom Multimodal Teaching Corpus
860 hours of classroom audio, video, teacher motion and board text from 312 lessons, supporting research on classroom discourse and teaching behaviour.
- Scale
- 860 hours
- Participants
- 1,120 students and 146 teachers
- Languages
- Mandarin (6 regional accents)
- Annotation
- Teacher and student discourse transcription, Classroom event coding, Teacher gesture and posture, Board and slide OCR
- Formats
- WAV / MP4 / ELAN / JSON / CSV
- Sync
- Audio-video and motion capture sync error ≤ 4 ms
- License
- Research license / Commercial license
- Updated
- August 20, 2026
01 / Composition
Scale, key metrics and data streams
Turning a lesson into an analyzable multimodal time series
- 860 h
- Total duration
- 312
- Lessons
- 1,266
- Participants
- 4
- School levels
| Stream | Specification | Notes |
|---|---|---|
| Audio | 16 kHz / 16-bit · 6 channels | Classroom array microphones, separating teacher and student zones |
| Video | 1920 × 1080 · 25 fps | Teacher, student-wide and board camera angles |
| Motion | 120 Hz · 17 joints | Motion capture of the teacher's upper body and torso |
| Text | UTF-8 / JSON | Slides, board OCR and sentence-level transcription |
The table lists capture parameters for a sample batch; final specifications follow the documentation delivered with the data.
02 / Dataset notes
Collection, annotation and quality control
The notes below explain how the data was collected, annotated and verified, and can be used as a basis for methodological citation and proposal assessment.
Overview
The Classroom Multimodal Teaching Corpus contains 312 complete lessons totaling 860 hours. It spans four school levels — primary, lower secondary, upper secondary and higher education — and subjects including Chinese, mathematics, English, science and the humanities. In total, 1,120 students and 146 teachers took part, recorded between March 2024 and May 2026. Every lesson is preserved in full, from the opening to the closing summary, with no clipping of segments, so teaching rhythm and interaction structure can be analyzed.
The corpus is stratified by school level, subject and lesson type (new content, review, experiment, discussion). A single teacher contributes at least three lessons, which makes it possible to examine the stability of that teacher’s behaviour.
Collection protocol
Each classroom is equipped with six-channel array microphones that record the teacher channel and the student-zone channel separately at 16 kHz. Video is captured by three 1920 × 1080 · 25 fps cameras: a teacher angle, a student wide shot and a board angle. The teacher wears an inertial motion-capture suit that records 17 upper-body and torso joints at 120 Hz, enabling analysis of how gesture couples with spoken explanation.
Site surveys and privacy notices precede every recording. Parents and teachers sign consent forms separately, and areas of the frame that contain non-participants are masked. All devices are synchronized and calibrated before the lesson, and board content and slides are aligned to audio and video by timestamp.
Annotation
Annotation is organized around who did what and when, using a unified coding manual.
| Layer | Content | Scale | Review |
|---|---|---|---|
| Discourse | Sentence-level teacher and student transcription, turn boundaries | About 920,000 sentences | 15% double-blind review |
| Events | Questioning, explanation, instruction, feedback, group work | 14 event classes | Dual annotation, arbitration |
| Nonverbal | Gesture type, gaze direction, posture | 9 gesture classes | Dual annotation of key segments |
| Text | Slide and board OCR, layout regions | About 46,000 pages | 10% manual spot-check |
Quality & verification
The median sync error across audio, video and motion capture is 2.1 ms, with the 95th percentile at or below 4 ms. Transcription character accuracy is 98.6%, classroom event coding agreement is κ = 0.87, and board OCR character accuracy is 97.3%. Ten percent of lessons in every batch go through full-process review, with particular attention to event boundaries and turn segmentation. Deliverables include a sync report, the coding manual, a lesson metadata table (school level, subject, lesson type, class size) and the review records.
Research questions
- How teacher question types relate to the length of student responses.
- The temporal coupling between gesture and the explanation of abstract concepts.
- How classroom discourse structure differs across lesson types.
- Whether multimodal signals can automatically identify a “student confusion” state.
- How the immediacy of teacher feedback relates to student engagement.
Access & licensing
The research license serves education, linguistics and psychology research teams; the commercial license serves education-technology and teacher-training products. Data is delivered as MP4, WAV, ELAN, JSON and CSV, and subsets can be filtered by school level, subject or lesson type. Because recordings include minors, applicants must provide ethics approval and a data management plan. You can inspect the field structure through the sample data page, or arrange a scoping conversation on the contact page.
03 / Typical use cases
Questions this dataset can answer
The directions below are typical research and product applications; a specific analysis plan depends on the study design and statistical method.
- 01
Classroom discourse interaction analysis
- 02
Teacher questioning and feedback research
- 03
Automatic recognition of multimodal teaching behaviour
- 04
Pre-service teacher classroom observation training
04 / Related datasets
Datasets that combine well
A single research question often calls for a cross-modal combination; the entries below share scenarios, speakers or annotation schemes with this dataset.
NEXT STEP
Need a different collection setting or annotation scheme?
An existing dataset can be licensed as it stands, or extended with additional speakers, scenarios and annotation layers around your study design.
Reply within one business day · NDA and data licensing supported · Dedicated academic track