nikhilsingh254/live-confusion-tracker

Live Confusion Tracker (synthetic)

Problem: professors often don't find out students are lost until the exam.

Idea: during a lecture, students tap "I'm lost" (optionally typing a question). Pings are grouped per minute or per slide/segment. When the share of confused students goes above the lecture's threshold, the professor gets a quiet alert on their watch or tablet, and what they do about it is logged.

This repo contains the data model plus a synthetic dataset of 120 lectures across 5 courses, generated by a small simulation. No real students are included. Every ID is random.

Source code: github.com/nikhilsingh-254/live-confusion-tracker (data model, generator and the early-warning model)

Data model

erDiagram
    LECTURE ||--|{ SEGMENT : "is split into"
    LECTURE ||--o{ CONFUSION_PING : receives
    SEGMENT ||--o{ CONFUSION_PING : "is about"
    LECTURE ||--o{ CONFUSION_WINDOW : "aggregated into"
    CONFUSION_WINDOW ||--o| ALERT : "may trigger"
    ALERT ||--o| CLARIFICATION_ACTION : "may lead to"
    SEGMENT ||--o{ CLARIFICATION_ACTION : "is addressed by"

All times are stored twice: *_at is ISO-8601 UTC, and offset_s is seconds since the lecture started.

Fields

lectures: lecture_id, course_code, course_title, professor_id, starts_at, duration_s, students_present, alert_threshold (fraction of students present, 0.08–0.12), alert_granularity (minute | segment), alert_cooldown_s, alert_device (watch | tablet)

segments: segment_id, lecture_id, ordinal, topic, slide_number, start_offset_s, end_offset_s, difficulty (0–1, the simulation's hidden true value)

confusion_pings: ping_id, lecture_id, segment_id, student_id (pseudonymous), created_at, offset_s, question (null for about 70% of pings)

confusion_windows: lecture_id, granularity, window_start_s, window_end_s, segment_id, ping_count, unique_students, question_count, students_present, confusion_rate (= uniquestudents / studentspresent), threshold, above_threshold

alerts: alert_id, lecture_id, segment_id, window_start_s, window_end_s, confusion_rate, threshold, device, sent_at, offset_s, top_questions (up to 3)

clarification_actions: action_id, lecture_id, segment_id, alert_id (null = unprompted), action_type (re_explain, new_example, check_poll, answer_question, slow_down, defer_to_office_hours, dismiss), created_at, offset_s, confusion_rate_before, confusion_rate_after (rate in the first full minute after the action)

A PostgreSQL version of the schema is in schema.sql, and Python dataclasses are in models.py.

Alert rule

For each closed window: confusion_rate = unique students who pinged / students present. An alert fires when confusion_rate >= alert_threshold and at least alert_cooldown_s has passed since the last alert. The cooldown keeps alerts quiet. Each student counts at most once per window. See ConfusionMonitor in aggregate.py.

Usage

from datasets import load_dataset

repo = "nikhilsingh254/live-confusion-tracker"
pings   = load_dataset(repo, "confusion_pings", split="train")
windows = load_dataset(repo, "confusion_windows", split="train")
actions = load_dataset(repo, "clarification_actions", split="train")

Ideas for what to do with it:

• —predict whether the next minute will cross the threshold (early warning)
• —find the topics students struggle with most across a course
• —group the question text to build a "top questions" summary for the professor

To regenerate the data or make more: python3 generate.py --lectures 500 --seed 1

How the simulation works

• —The chance a student pings in a given minute grows with the segment's difficulty² and with how far into the segment the lecture is. Each student also has their own ping tendency.
• —About 85% of alerts get a response 20–90 seconds later. Each action type lowers confusion for the rest of that segment by a fixed amount (re_explain ×0.35, dismiss ×1.0).
• —About 30% of pings include a question built from a template.

Limitations

• —Synthetic. Real ping rates, class sizes and professor behavior will differ. Question text comes from templates, so it is repetitive.
• —Before/after is not a causal effect. Alerts fire at noisy peaks, so confusion usually drops afterward even with no real fix. dismiss still goes from 0.16 to 0.06. Compare each action against dismiss, not against zero.
• —In segment-granularity lectures, alerts fire only when a segment ends, so the response lands in the next topic.