SYNC · CONSISTENCY · AUDIT
Validation and QA
Independent verification of data and annotations: sync measurement, content sampling, agreement assessment and reproducibility audits, delivered as a traceable quality report.
Deliverables
- Sync and signal quality report
- Content verification and sampling records
- Agreement assessment results (κ / α)
- Reproducible audit package
- Service module
- 04 / 05
- Deliverables
- 4 items
- Process stages
- 4 steps
- Quality verification
- Batch-level
01 / Service scope
What we actually do
Each capability can be switched on as a standalone module or combined into a full delivery chain.
Sync verification
Measure inter-channel error with pulses and event markers, and document the measurement method.
Content verification
Sample transcripts and labels at a fixed ratio, with a second person verifying independently.
Agreement assessment
Compute κ or α per label dimension and report confidence intervals.
Reproducibility audit
Re-run processing scripts and compare checksums, then issue an audit package.
Issue closure
Failed batches are flagged for rework and only enter delivery after a re-check.
02 / Workflow
How it moves forward
Every stage has defined outputs and checkpoints, so progress and risk surface during the work rather than at acceptance.
- 01W1
Acceptance criteria
Agree sync tolerance, agreement thresholds and sampling ratios.
- 02W2
Sampling and checks
Run stratified sync, content and agreement checks.
- 03W3–W4
Diagnosis and re-check
Trace failures to their source, fix them, then re-run the same checks.
- 04W5
Report and audit package
Issue passed items, exceptions and reproducible materials.
03 / Service notes
Who it is for, and how to begin
What we do
Validation is where TOMCUBE differs from a conventional data vendor. We do not save quality checks for acceptance; we embed checkpoints in every stage. During collection we verify sync and signal quality; after processing we verify alignment and completeness; after annotation we verify agreement and content correctness; before delivery we run a reproducibility audit over the whole batch.
The instruments are concrete: sync pulses and event markers to measure inter-channel error; stratified sampling of transcripts and labels verified by a second person; κ and α to assess annotation agreement; re-running the processing chain to compare checksums. Any batch that fails a checkpoint is flagged and reworked, and the rework record ships with the data. A validation conclusion has to survive someone else re-running it: same data, same scripts, a different operator, the same result.
We report what we measured and where it came from. Each finding in the report traces back to a specific file and record, including the sample size, the tolerance used and the exceptions we could not resolve. If a batch cannot be brought within tolerance, we say so plainly rather than rounding it into a pass.
The audit is designed to be repeated by a client’s own team. We hand over the scripts, the environment description and a short runbook, so a second operator can reproduce the checks without asking us what a parameter meant. That is also how we test our own work: if a fresh run disagrees with a stored result, the discrepancy is investigated before delivery.
Who it’s for
Three kinds of need: independent third-party quality evidence before a grant close-out or a journal submission; due diligence before an organization purchases a corpus from a supplier; and existing data that produces unstable training results, where the question is whether the problem sits in the data, the annotation or the alignment. We also validate data from other vendors without commenting on the vendor itself. That independence extends to internal data teams preparing for external review, and to suppliers who want a corpus checked before offering it to customers. In both cases the report says what the data supports, not who produced it.
How it works
- Acceptance criteria. Agree sync tolerance, agreement thresholds and sampling ratios.
- Sampling and checks. Run stratified sync, content and agreement checks.
- Diagnosis and re-check. Trace failures to their source, fix them, then re-run the same checks.
- Report and audit package. Issue passed items, exceptions and reproducible materials.
Delivery and verification
Deliverables include the sync and signal quality report, content verification and sampling records, agreement results, and the reproducible audit pack. We publish the measurement method and sample size, and every conclusion in the report points to the file and record it came from. Batch pass rates and median sync error are reported alongside, so results can be compared across projects. Where a batch contains several conditions, results are broken out by condition so a strong overall pass rate does not hide a weak stratum.
How to get started
Send the data volume, existing documentation and acceptance requirements, and we will scope the validation and propose a sampling plan. You can also commission a single dimension, for example sync verification only, or an annotation agreement assessment. Most engagements start with a one-week scoping pass that ends in a written acceptance plan, which can then be attached to the project agreement.
04 / Deliverables
What you receive
Deliverables are organized to be citable and reproducible: naming, metadata and verification records are complete, ready for analysis or archiving.
Deliverable 01
Sync and signal quality report
Deliverable 02
Content verification and sampling records
Deliverable 03
Agreement assessment results (κ / α)
Deliverable 04
Reproducible audit package
Reproducible audit package
Data dictionary, processing scripts, parameter configuration and verification report ship with the delivery. Re-run the documented steps to check the result, or cite the package in a paper appendix, grant close-out or internal audit.
05 / FAQ
Common questions
If your question is not covered, contact us directly.
Can you validate data from a third party?
Yes. We apply the same standard for independent verification. The report states measurement results and exceptions only; it does not evaluate the supplier.
Can the validation report support a paper or grant close-out?
The report covers method, sample size, measurements and limitations, and can serve as methodological material describing data quality. Citation format can be adjusted to journal requirements.
How do you set the sampling ratio?
It is stratified by data volume, risk level and agreement performance, typically 5%–20%. Batches with lower agreement get a higher ratio.
What happens if validation fails?
We provide an issue list and remediation advice, then re-check after rework. Parts of the raw data or annotation that cannot be fixed are flagged explicitly.
06 / Related datasets
Datasets delivered with this service
Public entries in the data catalog, useful as references for specifications, annotation and delivery formats.
Start a project
Commission an independent verification
Whether it is pre-delivery batch QA or an independent assessment of third-party data, we apply the same standard and return a traceable quality report.
Reply within one business day · NDA and data licensing supported · Dedicated academic track