CoolFace
Apppublic

singhpuranpal/citation-intent-api

sourceHugging Faceupdated 5mo agoView on Hugging Face
0likes
App README

Citation Intent Classifier API

FastAPI backend for citation intent classification using SciBERT + Focal Loss. Classifies academic citations into 7 intent categories with per-class optimized thresholds.

Features

  • —🎯 Multi-label Classification: Predict multiple citation intents per text
  • —🔬 SciBERT Model: Fine-tuned on academic citation data
  • —⚖️ Focal Loss + Oversampling: Handles class imbalance (T1 technique)
  • —🎲 Per-class Thresholds: Optimized confidence thresholds (T2 technique)
  • —📊 Batch Citation Extraction: Extract and classify all citations from documents
  • —🚀 GPU Support: CUDA-accelerated inference
  • —🔄 CORS Enabled: Cross-origin requests supported

Intent Categories

IntentDescription
BackgroundProvides foundational context or knowledge
MotivationInspired or motivated this research
Future WorkSuggested for future exploration
SimilaritiesHas similar approaches or findings
DifferencesDiffers from this research
UsesMethod or tool directly adopted
ExtendsThis work builds upon the cited work

Requirements

  • —Python 3.8+
  • —PyTorch with CUDA support (optional but recommended)
  • —FastAPI
  • —Transformers (HuggingFace)
  • —See requirements.txt for full list

Installation

bash
# Install dependencies
pip install -r requirements.txt

# Ensure m3_results.json is in the backend folder
# (Contains optimized thresholds from model training)

Configuration

Pre-trained Model

  • —Base Model: allenai/scibert_scivocab_uncased
  • —Fine-tuned Weights: Downloaded from HuggingFace Hub (singhpuranpal/citation-intent-scibert)
  • —Technique: T1 (Focal Loss + Oversampling)

Key Parameters

python
MAX_LEN    = 64      # Maximum token length
N_CLS      = 7       # Number of intent classes
MODEL_NAME = "allenai/scibert_scivocab_uncased"

Thresholds

Per-class confidence thresholds are loaded from m3_results.json:

json
{
  "thresholds": {
    "Background": 0.45,
    "Motivation": 0.50,
    ...
  },
  "best_hp": {"lr": 3e-5, "gamma": 2.0},
  "metrics": {...}
}

API Endpoints

Health Check

http
GET /health

Returns device status and model info.

Response:

json
{
  "status": "ok",
  "device": "cuda",
  "gpu": "NVIDIA GeForce RTX 3090"
}

Get Available Intents

http
GET /intents

Lists all intent classes with thresholds and descriptions.

Get Model Info

http
GET /model

Returns full model metadata and M3 metrics.

Predict Citation Intent

http
POST /predict
Content-Type: application/json

{
  "text": "We use the BERT model introduced by Devlin et al. (2019)."
}

Response:

json
{
  "text": "We use the BERT model introduced by Devlin et al. (2019).",
  "predicted_intents": ["Uses"],
  "all_scores": [
    {
      "intent": "Uses",
      "confidence": 0.8432,
      "predicted": true,
      "description": "Method or tool directly adopted"
    },
    ...
  ],
  "model_info": {...}
}

Constraints:

  • —Maximum 5000 characters
  • —Empty text rejected (400 Bad Request)

Extract and Classify Citations

http
POST /extract-citations
Content-Type: application/json

{
  "text": "Full academic paper text with multiple citations..."
}

Response:

json
{
  "total_citations": 12,
  "citations": [
    {
      "citation_text": "Smith et al. (2021) proposed...",
      "predicted_intents": ["Background", "Motivation"],
      "all_scores": [...]
    },
    ...
  ],
  "model_info": {...}
}

Constraints:

  • —Maximum 50000 characters
  • —Extracts sentences containing citation patterns:
  • —(Author et al., YYYY)
  • —(Author, YYYY)
  • —Author et al. (YYYY)

Usage Examples

Using curl

bash
# Single prediction
curl -X POST http://localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"text": "We adopt the transformer architecture from Vaswani et al. (2017)."}'

# Extract citations - with real academic text
curl -X POST http://localhost:8000/extract-citations \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Attention mechanisms have revolutionized deep learning (Bahdanau et al., 2015). The transformer architecture introduced by Vaswani et al. (2017) further improved upon these mechanisms. Recent work by Devlin et al. (2019) showed that BERT could achieve state-of-the-art results. Similar approaches are explored in RoBERTa (Liu et al., 2019). Extensions of this work include ALBERT (Lan et al., 2019) and ELECTRA (Clark et al., 2020)."
  }'

# Check health
curl http://localhost:8000/health

Using Python

python
import requests

url = "http://localhost:8000/predict"
payload = {"text": "The attention mechanism (Bahdanau et al., 2015) improved NMT."}
response = requests.post(url, json=payload)
print(response.json())

Using Postman

Step 1: Create a New Request
  1. 1.Open Postman
  2. 2.Click + (New) or Create
  3. 3.Select HTTP
Step 2: Configure Request for /predict
  1. 1.Change method dropdown to POST
  2. 2.Enter URL: http://localhost:8000/predict
  3. 3.Click Body tab
  4. 4.Select raw radio button
  5. 5.Change format dropdown to JSON
  6. 6.Paste test data:
json
{
  "text": "We use the BERT model introduced by Devlin et al. (2019)."
}
  1. 1.Click Send

Expected Response:

json
{
  "text": "We use the BERT model introduced by Devlin et al. (2019).",
  "predicted_intents": ["Uses"],
  "all_scores": [
    {
      "intent": "Uses",
      "confidence": 0.85,
      "predicted": true,
      "description": "Method or tool directly adopted"
    },
    ...
  ],
  "model_info": {...}
}
Step 3: Configure Request for /extract-citations
  1. 1.Change URL to: http://localhost:8000/extract-citations
  2. 2.Keep method as POST
  3. 3.Click Body tab → Select raw → Select JSON
  4. 4.Paste test data:
json
{
  "text": "Attention mechanisms revolutionized neural networks (Bahdanau et al., 2015). The transformer architecture introduced by Vaswani et al. (2017) built upon these ideas. BERT (Devlin et al., 2019) leveraged this architecture for pre-training. Similar work includes RoBERTa (Liu et al., 2019). Future work could explore more efficient variants."
}
  1. 1.Click Send

Expected Response:

json
{
  "total_citations": 4,
  "citations": [
    {
      "citation_text": "Attention mechanisms revolutionized neural networks (Bahdanau et al., 2015).",
      "predicted_intents": ["Background"],
      "all_scores": [...]
    },
    {
      "citation_text": "The transformer architecture introduced by Vaswani et al. (2017) built upon these ideas.",
      "predicted_intents": ["Motivation", "Extends"],
      "all_scores": [...]
    },
    ...
  ],
  "model_info": {...}
}
Postman Headers (Auto-set)

When you select JSON format, Postman automatically sets:

Content-Type: application/json

If not, manually add it:

  1. 1.Click Headers tab
  2. 2.Add key: Content-Type | value: application/json
Quick Postman Tips
  • —Save requests: Click Save after creating a request to reuse later
  • —Create a Collection: Organize all API endpoints in one collection
  • —Click New → Collection
  • —Name it "Citation Intent API"
  • —Add requests to the collection for reuse
  • —Environment Variables: Store {{base_url}} for easy switching between localhost/production
  • —Click Environments (gear icon)
  • —Add variable base_url = http://localhost:8000
  • —Use in URL: {{base_url}}/predict
  • —Test Scripts: Add tests to validate responses automatically
  • —Click Tests tab after request
  • —Add assertions like:
javascript
  pm.test("Status is 200", function() {
    pm.response.to.have.status(200);
  });
  pm.test("Response has predicted_intents", function() {
    pm.expect(pm.response.json().predicted_intents).to.be.an('array');
  });

Using FastAPI Docs

Navigate to http://localhost:8000/docs for interactive Swagger UI.

Test Cases

Test 1: Single Citation Prediction

Request:

bash
curl -X POST http://localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"text": "We use the BERT model introduced by Devlin et al. (2019)."}'

Expected Response:

json
{
  "text": "We use the BERT model introduced by Devlin et al. (2019).",
  "predicted_intents": ["Uses"],
  "all_scores": [
    {
      "intent": "Uses",
      "confidence": 0.85,
      "predicted": true,
      "description": "Method or tool directly adopted"
    },
    {
      "intent": "Background",
      "confidence": 0.42,
      "predicted": false,
      "description": "Provides foundational context or knowledge"
    },
    ...
  ],
  "model_info": {...}
}

Test 2: Citation Extraction from Academic Text

Request:

bash
curl -X POST http://localhost:8000/extract-citations \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Attention mechanisms revolutionized neural networks (Bahdanau et al., 2015). The transformer architecture introduced by Vaswani et al. (2017) built upon these ideas. BERT (Devlin et al., 2019) leveraged this architecture for pre-training. Similar work includes RoBERTa (Liu et al., 2019). Future work could explore more efficient variants."
  }'

Expected Response:

json
{
  "total_citations": 4,
  "citations": [
    {
      "citation_text": "Attention mechanisms revolutionized neural networks (Bahdanau et al., 2015).",
      "predicted_intents": ["Background"],
      "all_scores": [
        {
          "intent": "Background",
          "confidence": 0.78,
          "predicted": true,
          "description": "Provides foundational context or knowledge"
        },
        ...
      ]
    },
    {
      "citation_text": "The transformer architecture introduced by Vaswani et al. (2017) built upon these ideas.",
      "predicted_intents": ["Motivation", "Extends"],
      "all_scores": [...]
    },
    {
      "citation_text": "BERT (Devlin et al., 2019) leveraged this architecture for pre-training.",
      "predicted_intents": ["Uses"],
      "all_scores": [...]
    },
    {
      "citation_text": "Similar work includes RoBERTa (Liu et al., 2019).",
      "predicted_intents": ["Similarities"],
      "all_scores": [...]
    }
  ],
  "model_info": {...}
}

Test 3: Error Handling - Empty Text

Request:

bash
curl -X POST http://localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"text": ""}'

Expected Response (400):

json
{
  "detail": "Text cannot be empty"
}

Test 4: Error Handling - Text Too Long

Request:

bash
curl -X POST http://localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"text": "' + python -c "print('word ' * 2500)" + '"}'

Expected Response (400):

json
{
  "detail": "Text too long. Max 5000 characters."
}

Test 5: Health Check

Request:

bash
curl http://localhost:8000/health

Expected Response:

json
{
  "status": "ok",
  "device": "cuda",
  "model": "/home/user/.cache/huggingface/hub/models--singhpuranpal--citation-intent-scibert/...",
  "gpu": "NVIDIA GeForce RTX 3090"
}

Test 6: Get Available Intents and Thresholds

Request:

bash
curl http://localhost:8000/intents

Expected Response:

json
{
  "intents": ["Background", "Motivation", "Future Work", "Similarities", "Differences", "Uses", "Extends"],
  "thresholds": {
    "Background": 0.45,
    "Motivation": 0.50,
    "Future Work": 0.42,
    "Similarities": 0.48,
    "Differences": 0.47,
    "Uses": 0.51,
    "Extends": 0.44
  },
  "descriptions": {...},
  "count": 7
}

Test 7: Get Full Model Info

Request:

bash
curl http://localhost:8000/model

Expected Response:

json
{
  "model_name": "allenai/scibert_scivocab_uncased",
  "checkpoint": "...",
  "technique": "T1 — Focal Loss + Oversampling",
  "thresholds": {...},
  "best_hp": {"lr": 3e-5, "gamma": 2.0},
  "max_len": 64,
  "device": "cuda",
  "num_classes": 7,
  "classes": ["Background", "Motivation", "Future Work", "Similarities", "Differences", "Uses", "Extends"],
  "m3_metrics": {
    "m2_baseline_macro_f1": 0.72,
    "t1_focal_macro_f1": 0.78,
    "t2_ensemble_macro_f1": 0.81,
    "t3_final_macro_f1": 0.83
  }
}

Using FastAPI Docs

Navigate to http://localhost:8000/docs for interactive Swagger UI.

Running the Service

Development

bash
python microservice.py
# Server runs on http://0.0.0.0:8000

With Uvicorn

bash
uvicorn microservice:app --host 0.0.0.0 --port 8000

Docker

bash
docker build -t citation-intent-api .
docker run -p 8000:8000 citation-intent-api

Performance & Metrics

Model performance from M3 (three-stage training):

  • —M2 Baseline: Macro F1 from baseline model
  • —T1 (Focal Loss): Improved with focal loss + oversampling
  • —T2 (Ensemble): Ensemble-based confidence tuning
  • —T3 (Final): Final optimized thresholds

See /model endpoint for current metrics.

Deployment Checklist

  • —[ ] m3_results.json is present in backend folder
  • —[ ] CUDA drivers installed (if using GPU)
  • —[ ] All dependencies installed: pip install -r requirements.txt
  • —[ ] Model weights cached or HuggingFace token configured
  • —[ ] Port 8000 is available
  • —[ ] /health endpoint returns 200 OK
  • —[ ] CORS headers configured for your frontend domain

Troubleshooting

IssueSolution
FileNotFoundError: m3_results.json not foundCopy m3_results.json to backend folder
Model download timeoutCheck internet connection, retry with huggingface_hub token
Out of memory (OOM)Reduce batch size or use CPU: set device = "cpu"
405 Method Not AllowedEnsure using POST for /predict and /extract-citations
CORS errorsConfigure allow_origins in middleware for your frontend

Development

  • —Model training code: See Jupyter notebooks (T1, T2, T3 stages)
  • —Fine-tuning details: Focal Loss + class oversampling for imbalanced data
  • —Citation extraction: Regex patterns for common academic citation formats

License

Project License - See LICENSE file for details