projectthinkings123/nih-chestxray14-swin-clinical-report

NIH ChestX-ray14 Swin Transformer — Clinical Report System

A medical AI project that uses a Swin Transformer to perform multi-label chest X-ray disease classification on the NIH ChestX-ray14 dataset.

The current repository contains the trained image-diagnosis model and the inference pipeline. The planned complete system will additionally accept clinical information and use an LLM/reasoning layer to generate a doctor-friendly clinical report.

Project Architecture

                         Doctor / User
                              |
                 +------------+------------+
                 |                         |
            Chest X-ray                Clinical Data
          PNG / JPG / DICOM          Text / PDF / Typed
                 |                         |
                 v                         v
       Swin Transformer              Clinical Data
       Image Diagnosis                Extraction
                 |                         |
                 |                         |
                 +-----------+-------------+
                             |
                             v
                    Fusion & Reasoning
                             |
                             v
                    LLM / Report Layer
                             |
              +--------------+--------------+
              |              |              |
              v              v              v
       Predicted Diseases  Confidence    Clinical
                           Scores         Reasoning
                             |
                             v
                    Doctor-friendly Report

Current status: The Swin Transformer image branch is trained and evaluated. The clinical-data and LLM report-generation branch is the next development stage.

Image Diagnosis Model

Model

• —Architecture: Swin Tiny Patch4 Window7 224
• —timm model: swin_tiny_patch4_window7_224
• —Framework: PyTorch
• —Pretraining: ImageNet pretrained weights were used during training
• —Input image size: 224 × 224
• —Number of classes: 15
• —Task: Multi-label classification
• —Output activation: Sigmoid
• —Loss: BCEWithLogitsLoss

During inference, the architecture is recreated without downloading pretrained weights and the trained best_model.pth checkpoint is loaded.

Dataset

The image model was trained on NIH ChestX-ray14.

The model predicts the following 15 classes:

Atelectasis
Cardiomegaly
Consolidation
Edema
Effusion
Emphysema
Fibrosis
Hernia
Infiltration
Mass
No Finding
Nodule
Pleural_Thickening
Pneumonia
Pneumothorax

The dataset itself is not included in this repository.

Model Performance

The trained model achieved:

Macro AUROC: 0.8386

This value was obtained from the test-set evaluation performed in the project Colab notebook.

Repository Structure

nih-chestxray14-swin-clinical-report/
│
├── best_model.pth
├── config.json
├── labels.json
├── model.py
├── inference.py
├── swin_chestxray14_training.ipynb
├── requirements.txt
└── README.md

File descriptions

Installation

Install the required Python packages:

pip install -r requirements.txt

The main dependencies are:

torch
torchvision
timm
Pillow
numpy

Running Inference

Place a chest X-ray image in the working directory and run:

python inference.py path/to/xray.png

Or:

python inference.py path/to/xray.jpg

The default classification threshold is:

0.5

You can specify another threshold:

python inference.py path/to/xray.png --threshold 0.5

The model outputs:

1. 1.Predicted disease labels
2. 2.Probability for each of the 15 classes

Example output format:

Predicted Labels:
- Infiltration

Probabilities:
Atelectasis: 0.4172
Cardiomegaly: 0.0345
Consolidation: 0.0173
...
Infiltration: 0.1427
...

How Inference Works

Input Chest X-ray
       |
       v
Resize to 224 × 224
       |
       v
Image normalization
       |
       v
Swin Tiny Transformer
       |
       v
15 logits
       |
       v
Sigmoid
       |
       v
15 disease probabilities
       |
       v
Threshold = 0.5
       |
       v
Predicted labels

Because this is a multi-label classification problem, multiple diseases can be predicted for a single X-ray.

Clinical Data Integration — Planned System

The final project is intended to extend the image model with clinical information.

The planned workflow is:

Chest X-ray
     |
     v
Swin Transformer
     |
     v
Disease Predictions + Confidence Scores
                         \
                          \
                           +--> Fusion & Reasoning --> LLM
                          /
Clinical Data -----------/
     |
     v
Age / Symptoms / Vitals / Laboratory Information / Other

The clinical branch is intended to provide information such as:

• —Age
• —Symptoms
• —Vital signs
• —Laboratory results
• —Relevant clinical history
• —Other information supplied by the user/doctor

The LLM/report layer will use the model's predicted findings together with the available clinical information to produce a structured, doctor-friendly report.

Important: The clinical-data/LLM component is a planned extension and is not represented as being part of the current trained Swin model.

Important Notes

Model output is not a medical diagnosis

This project is a research/educational prototype. The model's predictions and generated reports should not be treated as a definitive medical diagnosis or as a replacement for a qualified medical professional.

NIH ChestX-ray14

The dataset is not distributed with this repository. Users must obtain and use the dataset according to its applicable terms and permissions.

Checkpoint

best_model.pth contains the trained model parameters. The model architecture is reconstructed by model.py before the checkpoint is loaded.

Development Roadmap

Completed

• —[x] NIH ChestX-ray14 dataset preparation
• —[x] Swin Tiny Transformer model
• —[x] ImageNet-pretrained initialization during training
• —[x] 15-class multi-label classification
• —[x] Model training
• —[x] Checkpoint saving
• —[x] Test-set evaluation
• —[x] Macro AUROC evaluation
• —[x] Inference pipeline
• —[x] Hugging Face model repository

Next

• —[ ] Clinical-data input pipeline
• —[ ] Clinical information extraction
• —[ ] Image + clinical-data fusion
• —[ ] LLM/reasoning layer
• —[ ] Structured clinical report generation
• —[ ] Explainability/visualization such as Grad-CAM
• —[ ] End-to-end application interface

Project Goal

The goal is to build a multimodal clinical-assistance prototype that combines:

Medical Image
      +
Clinical Information
      +
AI Reasoning
      |
      v
Structured, doctor-friendly report

The Swin Transformer serves as the image-diagnosis component, while the planned clinical and LLM components provide additional contextual reasoning and report generation.