Skip to main content

DATASET 02 · Educational linguistics

Classroom Multimodal Teaching Corpus

860 hours of classroom audio, video, teacher motion and board text from 312 lessons, supporting research on classroom discourse and teaching behaviour.

SpeechVideoTextMotion captureEducational linguisticsClassroom research
Classroom Multimodal Teaching Corpus · cover artwork
Scale
860 hours
Participants
1,120 students and 146 teachers
Languages
Mandarin (6 regional accents)
Annotation
Teacher and student discourse transcription, Classroom event coding, Teacher gesture and posture, Board and slide OCR
Formats
WAV / MP4 / ELAN / JSON / CSV
Sync
Audio-video and motion capture sync error ≤ 4 ms
License
Research license / Commercial license
Updated
August 20, 2026

01 / Composition

Scale, key metrics and data streams

Turning a lesson into an analyzable multimodal time series

860 h
Total duration
312
Lessons
1,266
Participants
4
School levels
Data composition and specifications
StreamSpecificationNotes
Audio16 kHz / 16-bit · 6 channelsClassroom array microphones, separating teacher and student zones
Video1920 × 1080 · 25 fpsTeacher, student-wide and board camera angles
Motion120 Hz · 17 jointsMotion capture of the teacher's upper body and torso
TextUTF-8 / JSONSlides, board OCR and sentence-level transcription

The table lists capture parameters for a sample batch; final specifications follow the documentation delivered with the data.

02 / Dataset notes

Collection, annotation and quality control

The notes below explain how the data was collected, annotated and verified, and can be used as a basis for methodological citation and proposal assessment.

Overview

The Classroom Multimodal Teaching Corpus contains 312 complete lessons totaling 860 hours. It spans four school levels — primary, lower secondary, upper secondary and higher education — and subjects including Chinese, mathematics, English, science and the humanities. In total, 1,120 students and 146 teachers took part, recorded between March 2024 and May 2026. Every lesson is preserved in full, from the opening to the closing summary, with no clipping of segments, so teaching rhythm and interaction structure can be analyzed.

The corpus is stratified by school level, subject and lesson type (new content, review, experiment, discussion). A single teacher contributes at least three lessons, which makes it possible to examine the stability of that teacher’s behaviour.

Collection protocol

Each classroom is equipped with six-channel array microphones that record the teacher channel and the student-zone channel separately at 16 kHz. Video is captured by three 1920 × 1080 · 25 fps cameras: a teacher angle, a student wide shot and a board angle. The teacher wears an inertial motion-capture suit that records 17 upper-body and torso joints at 120 Hz, enabling analysis of how gesture couples with spoken explanation.

Site surveys and privacy notices precede every recording. Parents and teachers sign consent forms separately, and areas of the frame that contain non-participants are masked. All devices are synchronized and calibrated before the lesson, and board content and slides are aligned to audio and video by timestamp.

Annotation

Annotation is organized around who did what and when, using a unified coding manual.

LayerContentScaleReview
DiscourseSentence-level teacher and student transcription, turn boundariesAbout 920,000 sentences15% double-blind review
EventsQuestioning, explanation, instruction, feedback, group work14 event classesDual annotation, arbitration
NonverbalGesture type, gaze direction, posture9 gesture classesDual annotation of key segments
TextSlide and board OCR, layout regionsAbout 46,000 pages10% manual spot-check

Quality & verification

The median sync error across audio, video and motion capture is 2.1 ms, with the 95th percentile at or below 4 ms. Transcription character accuracy is 98.6%, classroom event coding agreement is κ = 0.87, and board OCR character accuracy is 97.3%. Ten percent of lessons in every batch go through full-process review, with particular attention to event boundaries and turn segmentation. Deliverables include a sync report, the coding manual, a lesson metadata table (school level, subject, lesson type, class size) and the review records.

Research questions

  • How teacher question types relate to the length of student responses.
  • The temporal coupling between gesture and the explanation of abstract concepts.
  • How classroom discourse structure differs across lesson types.
  • Whether multimodal signals can automatically identify a “student confusion” state.
  • How the immediacy of teacher feedback relates to student engagement.

Access & licensing

The research license serves education, linguistics and psychology research teams; the commercial license serves education-technology and teacher-training products. Data is delivered as MP4, WAV, ELAN, JSON and CSV, and subsets can be filtered by school level, subject or lesson type. Because recordings include minors, applicants must provide ethics approval and a data management plan. You can inspect the field structure through the sample data page, or arrange a scoping conversation on the contact page.

03 / Typical use cases

Questions this dataset can answer

The directions below are typical research and product applications; a specific analysis plan depends on the study design and statistical method.

  • 01

    Classroom discourse interaction analysis

  • 02

    Teacher questioning and feedback research

  • 03

    Automatic recognition of multimodal teaching behaviour

  • 04

    Pre-service teacher classroom observation training

NEXT STEP

Need a different collection setting or annotation scheme?

An existing dataset can be licensed as it stands, or extended with additional speakers, scenarios and annotation layers around your study design.

Reply within one business day · NDA and data licensing supported · Dedicated academic track