paoo4511/asd-screening-tool

AI-Assisted Clinical Assessment of Autism (Term Paper)

End-to-end pipeline for extracting speech-language features from raw audio (via Whisper) or CHAT (.cha) transcripts and building:

1. 1.Screening classifier (ASD / DD / TD) from cross-sectional corpora — 13-feature LogReg reaches AUC 0.931 on binary ASD vs non-ASD with sensitivity/specificity/PPV/NPV, calibration, threshold, and uncertainty metrics exported for audit.
2. 2.Longitudinal progress tracker — detects improvement patterns in 9/12 children across multiple therapy sessions.
3. 3.Audio-to-assessment pipeline — upload .wav → Whisper ASR → diarization → CHAT transcript → features → prediction, all in the interactive dashboard.
4. 4.Per-prediction explainability (XAI) — every screening result is accompanied by a SHAP-equivalent decomposition showing how each feature pushed the log-odds toward ASD or non-ASD, so clinicians can review the model rationale as decision support.
5. 5.Uncertainty band (40–60%) — predictions with P(ASD) inside the indeterminate zone are reported as UNCERTAIN, recommend further assessment instead of forcing a binary verdict, mirroring the FDA-cleared device by Megerian et al. (2022).
6. 6.Graded severity scoring (0–10) — beyond the binary verdict the Screening Tool reports three clinically meaningful sub-scores: risk marker burden, communication strength, and ASD-marker burden, inspired by the ASDSpeech work of Eni et al. (2025).
7. 7.Parent public demo — a Thai-first, no-data-retention Streamlit flow for parents that summarizes concern level and next steps without making diagnostic claims. Audio upload is optional and gated by privacy consent.
8. 8.Model Trust Dashboard — threshold playground, confusion matrix, calibration bins, Brier score, decision curve, uncertainty zone, subgroup robustness, fairness/calibration audit exports, leave-one-corpus-out stress test, and model card.
9. 9.Pastel unified dashboard — the main Streamlit experience for the full project story, data inventory, corpus map, feature dictionary, EDA workspace, screening controls, audio/CHAT workflow, model trust, progress tracking, research evidence, safety, limitations, and presentation flow.
10. 10.AI Transcript Reviewer — a rule-based .cha quality reviewer for CHAT structure, speaker tiers, utterance quality, clinical/linguistic marker counts, Thai language-tag readiness, optional ASR confidence checks, and optional pylangacq parse checks before feature extraction.
11. 11.Therapist progress reports — Thai-safe Markdown/PDF reports from longitudinal_features.csv that summarize first-vs-last therapy-session trends for progress tracking only.
12. 12.Thai Validation Readiness Pack — documentation, model-card fields, and Streamlit wording that make clear the model is not yet validated for Thai children and requires expert assessment.
13. 13.AI Speech Therapist Assistant — a rule-based/template-based decision support layer that summarizes transcript quality, speech-language patterns, screening risk estimates, and progress trends for therapist review in Thai or English.
14. 14.Clinician Workflow Simulator — a compact Streamlit workflow that combines transcript QA, screening pattern interpretation, and progress case briefs while keeping a human-in-the-loop safety boundary.

Clinical safety boundary

This project is a research prototype and educational demo. It supports screening support, risk estimates, decision support, and progress tracking; it is not a diagnostic tool and does not replace clinician assessment. The current model was trained/evaluated on English-speaking public corpora and is not validated for Thai children. External Thai validation, calibration, subgroup audit, IRB/consent, privacy workflow, and clinician workflow testing are required before any real Thai clinical use.

Public access

Use these links when you want to show the project to parents, advisors, or anyone who does not have the repo locally:

• —Parent / clinician public app: <https://paoo4511-asd-screening-tool.hf.space>
• —Pastel unified dashboard: <https://paoo4511-asd-screening-tool.hf.space>
• —Short presenter guide: docs/PRESENTER_GUIDE_TH.md

Recommended demo flow:

1. 1.Open the public Pastel app for a safe parent-friendly screening demo.
2. 2.Use the same Pastel dashboard to explain the full structure, data, metrics, workflow, and limitations.
3. 3.Use the presenter guide as a 3-5 minute narrative when explaining the project to someone new.

Data sources (TalkBank / ASDBank)

Cross-sectional (classifier, 122 children)

Longitudinal (progress tracker, 87 sessions / 12 children)

Setup

pip install -r requirements.txt

Pipeline

Run in order:

python src/data_loader.py        # build combined_features.csv + longitudinal_features.csv
python src/eda.py                # summary stats + plots -> reports/figures/
python src/classifier.py         # sklearn models + trust metrics + model bundle
python scripts/compute_fairness_metrics.py  # fairness + calibration audit CSVs
python src/deep_learning.py      # PyTorch MLP + Bi-LSTM
python src/progress_tracking.py  # longitudinal analysis (Rollins + Flusberg)
python src/evaluate_asr.py       # (optional) WER evaluation of the audio pipeline
streamlit run app/dashboard_unified.py  # Pastel unified dashboard
streamlit run app/dashboard.py          # legacy dashboard, kept for fallback

Pastel unified dashboard

For the main polished project experience, use the Pastel Streamlit dashboard:

streamlit run app/dashboard_unified.py

The Hugging Face/Streamlit entry point app.py now launches app/dashboard_unified.py, so the public app shows the Pastel dashboard by default. The static project_dashboard/ files remain in the repository only as a legacy reference and are no longer the recommended deployment surface.

Audio pipeline (Whisper → CHAT)

The project includes an end-to-end module that turns raw audio into a .cha transcript the rest of the pipeline can consume:

python -m src.audio_pipeline.pipeline recording.wav \
    --model base --age-months 48 --sex male --group ASD
# -> writes recording.cha next to recording.wav

From the dashboard, pick the 🎤 Audio assessment page to do the same thing interactively: upload a .wav/.mp3, pick a Whisper size, and get features + ASD probability + a downloadable .cha.

The module has two diarization backends:

• —PitchHeuristicDiarizer (default) — uses librosa's pyin F0 estimate to separate child (high F0) from adult. Zero external dependencies beyond the ones in requirements.txt.
• —PyannoteDiarizer (optional) — uses pyannote/speaker-diarization-3.1 for SOTA results. Requires a HuggingFace token in HF_TOKEN and pip install pyannote.audio.

Transcript QA and therapist reports

Use the Streamlit Transcript QA & Reports page to upload a .cha file, run the rule-based reviewer, inspect marker counts and issues, then generate a Thai-safe therapist progress report as Markdown or PDF for a child in data/longitudinal_features.csv.

Programmatic usage:

python -c "from src.transcript_reviewer import review_cha_file; print(review_cha_file('data/Rollins/Carl/020800.cha'))"
python -c "from src.therapist_report import save_progress_report; save_progress_report('Roger'); save_progress_report('Mark')"
python scripts/compute_fairness_metrics.py

Generated sample reports live in reports/progress_reports/. These reports are progress tracking and decision support artifacts only, not ASD diagnosis.

The Streamlit AI Speech Therapist Assistant page adds an interpretation layer on top of those outputs. It can summarize transcript QA, explain speech-language patterns from the 13-feature schema, interpret screening risk estimates, and generate therapist-facing Markdown case briefs. It cannot replace a speech therapist, cannot establish Thai clinical accuracy without Thai validation data, and must be used with human expert review.

The Streamlit 🩺 Clinician Workflow Simulator page shows the same pieces in one compact flow: transcript QA, screening/pattern interpretation, and progress/case brief generation. It is designed for demonstration and workflow review only; no uploaded transcript is stored by the page.

Tests

python tests/test_audio_pipeline_smoke.py    # CHAT formatter round-trip via pylangacq
python tests/test_audio_pipeline_v015.py     # deterministic audio pipeline unit tests
python tests/test_feature_schema.py          # shared 13-feature schema alignment
python -m pytest tests/test_transcript_reviewer.py tests/test_therapist_report.py -q
python -m pytest tests/test_fairness_metrics.py -q
python -m pytest tests/test_speech_therapist_assistant.py -q
python -m py_compile src/feature_schema.py src/classifier.py src/transcript_reviewer.py src/fairness_metrics.py src/therapist_report.py src/speech_therapist_assistant.py app/dashboard.py scripts/compute_fairness_metrics.py

The classifier also writes dashboard-ready validation assets:

• —reports/metrics/threshold_metrics.csv
• —reports/metrics/calibration_bins.csv
• —reports/metrics/calibration_summary.csv
• —reports/metrics/decision_curve.csv
• —reports/metrics/fairness_metrics.csv
• —reports/metrics/subgroup_performance.csv
• —reports/metrics/leave_one_corpus_out.csv
• —artifacts/screening_model.joblib
• —artifacts/model_card.json

Current deep-learning baselines on the same 13-feature schema: TabularMLP reaches ROC-AUC 0.9320; UtteranceLSTM reaches ROC-AUC 0.7193. This supports the current project interpretation that compact clinical language features remain stronger than sequence deep learning on this small dataset.

Deployment

See `docs/DEPLOYMENT.md` for Streamlit Community Cloud, Hugging Face Spaces, and self-host Docker instructions. The public deployment uses the Pastel Streamlit dashboard through app.py.

Quick local Docker run:

docker build -t asd-dashboard .
docker run -p 8501:8501 asd-dashboard   # open http://localhost:8501

Project structure

asd-project/
├── data/                                 # raw .cha files + generated CSVs
│   ├── Eigsti/ Nadig/ NYU-Emerson/
│   ├── Flusberg/ Rollins/ QuigleyMcNally/
│   ├── combined_features.csv             # 122 rows (classification)
│   └── longitudinal_features.csv         # 87 rows (progress tracking)
├── src/
│   ├── audio_pipeline/                   # .wav -> .cha
│   │   ├── whisper_transcribe.py         #   faster-whisper wrapper
│   │   ├── diarization.py                #   pyannote + pitch heuristic
│   │   ├── chat_formatter.py             #   write valid CHAT transcripts
│   │   └── pipeline.py                   #   orchestrator (audio_to_cha)
│   ├── data_loader.py                    # CHAT -> features CSV
│   ├── feature_schema.py                  # shared 13-feature model schema
│   ├── eda.py                            # exploratory data analysis
│   ├── classifier.py                     # sklearn classifiers + trust metrics
│   ├── fairness_metrics.py               # ECE, Brier, and group fairness helpers
│   ├── deep_learning.py                  # PyTorch MLP + Bi-LSTM
│   ├── progress_tracking.py              # longitudinal trends + composite
│   ├── transcript_reviewer.py            # rule-based CHAT transcript QA
│   ├── therapist_report.py               # Thai-safe progress report generator
│   ├── speech_therapist_assistant.py     # safe therapist-facing interpretation layer
│   └── evaluate_asr.py                   # WER of Whisper vs gold .cha
├── app/
│   ├── dashboard.py                      # legacy Streamlit dashboard fallback
│   └── dashboard_unified.py              # Pastel unified dashboard
├── project_dashboard/                    # legacy static dashboard reference
│   ├── index.html
│   ├── styles.css
│   └── app.js
├── scripts/
│   ├── build_public_atlas.sh              # legacy local static bundle helper
│   └── compute_fairness_metrics.py        # fairness + calibration CSV export
├── tests/
│   └── test_audio_pipeline_smoke.py
├── reports/
│   ├── figures/                          # saved plots
│   ├── metrics/                          # saved metrics CSVs
│   └── progress_reports/                 # generated therapist Markdown/PDF reports
├── artifacts/
│   ├── screening_model.joblib             # versioned screening model bundle
│   ├── model_card.json                    # intended use + caveats
│   └── feature_schema.json                # dashboard/app schema contract
├── docs/                                 # documentation
│   ├── DEPLOYMENT.md                     # deployment guide
│   ├── DEVELOPMENT.md                    # workflow + version tracking
│   ├── PROJECT_SUMMARY_TH.md             # project summary (Thai)
│   ├── DISCUSSION_TH.md                  # discussion points for advisor
│   ├── NEXT_STEPS_TH.md                  # roadmap for next development
│   ├── THAI_VALIDATION_READINESS_TH.md   # Thai validation readiness and safe claims
│   ├── REFERENCES.md                     # bibliography
│   ├── SUMMARY_TH.md                     # original Thai summary
│   ├── VERSION_UPDATE_CHECKLIST.md       # version update checklist
│   └── literature/                       # raw bibliography exports
│       └── consensus_papers_2026-04-26.csv
├── .agents/
│   └── skills/                            # project-level AI agent skills
│       ├── project-update-workflow/       # docs/version/GitHub workflow
│       ├── asd-clinical-ml-reviewer/      # clinical ML validity + safety review
│       ├── asd-audio-pipeline-qa/         # Whisper/diarization/CHAT QA
│       ├── asd-advisor-report-writer/     # Thai advisor/report workflow
│       ├── personal-data-analyst/         # CSV/EDA/metrics/report analysis
│       ├── personal-code-quality/         # code review, tests, refactors
│       ├── personal-security-auditor/     # privacy/security review
│       ├── personal-researcher/           # literature and source-backed research
│       └── personal-devops-deployer/      # Streamlit/Docker/deploy workflow
├── .windsurf/
│   └── rules/
│       └── project-update-workflow.md     # Windsurf bridge rule
├── .streamlit/
│   └── config.toml                       # theme + upload size
├── Dockerfile                            # production container
├── packages.txt                          # Streamlit Cloud apt deps
├── CHANGELOG.md                          # version history
├── requirements.txt
└── README.md

Features extracted per .cha

• —Demographics: age_months, sex, group, corpus
• —Productivity: total_utterances, total_words
• —Complexity: mlu (morphemes), mluw (words)
• —Lexical diversity: ttr (type-token ratio)
• —ASD-relevant markers: unintelligible_count/ratio (xxx/yyy), zero_vocalization_count (0 .), nonverbal_vocalization_count (&=gasp, &=laugh, ...), echolalia_count/ratio (verbatim repetition of recent utterances)
• —Pragmatic: question_ratio