QuantumLearningMachines/qlm-nto-classifier

qlm-nto-classifier

A DeBERTa-v3 classifier that labels an utterance from K-12 mathematics classroom dialogue with the pedagogical talk move it performs.

Read the Limitations section before using this model.

Model details

Label mapping (config.id2label)

Training data

Trained on the TalkMoves dataset — authentic K-12 mathematics classroom transcripts, human-transcribed and human-annotated.

Unlike some other models in this collection, this corpus is authentic classroom dialogue with human annotations rather than synthetic or model-generated data. That is a genuine strength and the reason the limitations below concern documentation rather than data provenance.

Each utterance carries both a human and an AI annotation, which means human-versus-model agreement is computable directly from the corpus. It has not been computed or reported.

Split sizes and methodology are undocumented. Whether the split is by session, by speaker, or random matters: an utterance-level random split would leak session context between train and test and inflate any performance figure computed later. Establish a session-held-out split before generating metrics.

Attribution

The TalkMoves dataset is the work of Suresh, Jacobs, Harty, Perkoff, Martin, and Sumner (University of Colorado Boulder), published at LREC 2022. Cite it in any work using this model:

@inproceedings{suresh-etal-2022-talkmoves,
  title     = {The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts
               Annotated for Teacher and Student Discursive Moves},
  author    = {Suresh, Abhijit and Jacobs, Jennifer and Harty, Charis and
               Perkoff, Margaret and Martin, James H. and Sumner, Tamara},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation
               Conference},
  pages     = {4654--4662},
  year      = {2022},
  address   = {Marseille, France},
  publisher = {European Language Resources Association}
}

Evaluation

This accuracy was measured across all 6 teacher talk-move classes on n=500 utterances. The model performs roughly 1.9 times better than uniform random guessing (16.7%), but 31% accuracy remains low in absolute terms.

Important caveats about these numbers:

• —The majority-class baseline is unknown and likely higher than 16.7%, given typical class imbalance in talk-move distributions. The model's advantage over the actual baseline may be smaller than the uniform-chance comparison suggests.
• —Per-class precision, recall, and F1 are not reported.
• —The evaluation split methodology is undocumented — the 500-example set may not be session-held-out, which would make this figure optimistic.
• —Human-versus-model agreement on the TalkMoves corpus has not been computed.

Source: classifier_results.json, field nto.

Limitations

1. 1.Accuracy is low (31%). While above uniform chance, this is insufficient for high-stakes individual-level decisions.
2. 2.Training corpus is CC BY-NC-SA 4.0 — the TalkMoves dataset's NonCommercial and ShareAlike terms apply to this model. This model is free and not commercially deployed.
3. 3.No inter-annotator or human-model agreement is reported, despite the corpus containing paired human and AI annotations that permit it.
4. 4.Split composition and methodology are undocumented. A random utterance-level split would leak session context and make the accuracy figure optimistic.
5. 5.Class distribution is undocumented. Talk-move distributions in classroom dialogue are typically severely imbalanced, so the majority-class baseline is likely high and is currently unknown.
6. 6.No disaggregation across grade bands, teacher experience, student population, or classroom composition.
7. 7.Transcript-domain dependence. The source corpus is human-transcribed with particular conventions; behaviour on ASR output or differently transcribed dialogue is unmeasured.
8. 8.Mathematics classrooms, English only. No evaluation in other subjects or languages.
9. 9.Teacher moves only. The model classifies only teacher discursive moves and cannot characterise student discourse.

Licence

This model is released under CC BY-NC-SA 4.0 (Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International), matching the licence of its training data (the TalkMoves dataset). This model is provided free of charge and is not commercially deployed.

If you use or adapt this model, you must:

• —Attribute the TalkMoves dataset (Suresh et al., LREC 2022) and this model
• —Not use commercially without separate permission from the dataset authors
• —Share alike — distribute adaptations under the same or compatible licence

Out-of-scope uses

• —Any evaluation, rating, or ranking of a teacher or tutor. Talk-move classification invites exactly this use, and with 31% accuracy it would be indefensible. This is the primary prohibited use, and it is prohibited regardless of future performance figures.
• —Coaching feedback delivered to an educator without human review.
• —Any determination about an individual student.
• —Unsupervised deployment of any kind, given the low accuracy.
• —Non-English dialogue, or contexts outside K-12 mathematics classrooms.

Intended uses

Research on automated talk-move classification; a baseline for classroom-discourse benchmarks; and aggregate, corpus-level analysis where no individual prediction is acted upon.

Citation

@misc{qlm-nto-classifier-2026,
  title  = {qlm-nto-classifier: a DeBERTa-v3 pedagogical talk-move classifier},
  author = {Srivastava, Kumar Sumbhav},
  year   = {2026},
  note   = {Quantum Learning Machines, Inc. Trained on the TalkMoves corpus
            (Suresh et al., LREC 2022).},
  url    = {https://huggingface.co/QuantumLearningMachines/qlm-nto-classifier}
}