Apppublic

slamos/bc-test

sourceHugging Faceupdated 6mo agoView on Hugging Face

0likes

App README

Bottlecap Newline Fixer

An ML service for fixing newline placement in English text. It takes text with broken newlines (e.g. copied from PDFs) and reconstructs proper paragraph and line breaks using fine-tuned transformer models.

Live app: https://slamos-bc-test.hf.space/

Report: report.pdf

Setup

Install dependencies

bash

make install

Environment variables

Create a .env file in the project root:

env

HF_TOKEN=<your-huggingface-token>        # Required for downloading Wikipedia dataset
WB_TOKEN=<your-wandb-token>              # Required for training (W&B logging)
BETTERSTACK_SOURCE_TOKEN=<your-token>    # Optional, for API request logging

Data

Download pre-built data (recommended)

The full processed dataset is available as a zip on Google Drive. This skips all download/preprocess/split steps:

bash

make download_data

This downloads and extracts data.zip into the data/ folder.

Google Drive link: data.zip

Download datasets from source

Alternatively, you can build the data from scratch:

bash

make download_tedseg                    # TED segmentation dataset
make download_pubmed k=100              # PubMed samples (default 100)
make download_wikipedia k=100           # Wikipedia samples (default 100, requires HF_TOKEN)

Preprocess

bash

make preprocess_all                     # All datasets at once

# Or individually:
make preprocess_pubmed
make preprocess_wikipedia
make preprocess_gutenberg
make preprocess_ted

Sentence splitting

bash

make sentence_split_all                 # All datasets

# Or individually:
make sentence_split_gutenberg
make sentence_split_pubmed
make sentence_split_wikipedia
make sentence_split_recipes
make sentence_split_ted

Build pairs

bash

make create_recipes                     # Create recipes dataset
make build_recipes_pairs                # Build sentence pairs

Training

Training arguments

Argument	Default	Description
`--model`	`distilbert`	Model to train (`distilbert`, `bert`, `deberta`)
`--out`	-	Output directory for checkpoints
`--data_root`	`data`	Root directory for datasets
`--epochs`	`5`	Number of training epochs
`--batch_size`	`16`	Batch size
`--lr`	model-dependent	Learning rate (distilbert: 5e-6, bert/deberta: 1e-5)
`--weight_decay`	`0.05`	Weight decay
`--warmup_ratio`	model-dependent	LR warmup ratio (distilbert: 0.15, bert/deberta: 0.10)
`--max_length`	`512`	Max token sequence length
`--gutenberg_cap`	`45000`	Cap on Gutenberg training pairs
`--seed`	`42`	Random seed
`--bf16`	`false`	Enable bfloat16 training
`--patience`	`3`	Early stopping patience

Run training

bash

make train_distilbert
make train_bert
make train_deberta
make train_all                          # Train all three models

Or directly:

bash

python -m src.models.train --model bert --epochs 5 --batch_size 16 --lr 1e-5

Export and upload models

bash

make export_distilbert                  # Export checkpoint to ONNX
make export_all                         # Export all models

make upload_distilbert                  # Push to HuggingFace Hub
make upload_all                         # Upload all models

Download pre-trained models

bash

make download_distilbert
make download_bert
make download_deberta
make download_all                       # Download all models from HF Hub

Inference

CLI inference

bash

make inference_distilbert               # Interactive CLI (from HF Hub)
make inference_bert
make inference_deberta

make inference_local_distilbert         # From local checkpoints
make inference_local_bert
make inference_local_deberta

Inference arguments

Argument	Default	Description
`--model`	`distilbert`	Model to use (`distilbert`, `bert`, `deberta`)
`--max_length`	`512`	Max token sequence length
`--local`	`false`	Load from local `checkpoints/<model>/best` instead of HF Hub

API

Running the service locally

bash

make run-be                             # FastAPI backend on port 8000
make run-fe                             # Streamlit frontend

Endpoints

When deployed on HF Spaces, the API is available under the /api prefix:

Method	Endpoint	Description
`GET`	`/api/health`	Health check, lists available models
`POST`	`/api/fix-newlines`	Fix newlines using the default model (bert)
`POST`	`/api/fix-newlines-all-models`	Fix newlines using all available models
`GET`	`/api/docs`	Interactive Swagger UI

When running locally, endpoints are available at http://localhost:8000/api/ (same /api prefix).

Example requests

Health check:

bash

curl https://slamos-bc-test.hf.space/api/health

Response:

json

{
  "status": "ok",
  "available_models": ["bert", "distilbert", "deberta"]
}

Fix newlines (single model):

bash

curl -X POST https://slamos-bc-test.hf.space/api/fix-newlines \
  -H "Content-Type: application/json" \
  -d '{"text": "This is a sentence that\nw\nas broken acr\noss lines."}'

Response:

json

{
  "fixed_text": "This is a sentence that was broken across lines.",
  "model_used": "bert"
}

Fix newlines (all models):

bash

curl -X POST https://slamos-bc-test.hf.space/api/fix-newlines-all-models \
  -H "Content-Type: application/json" \
  -d '{"text": "This is a sentence that\nw\nas broken acr\noss lines."}'

Response:

json

{
  "results": [
    {
      "model_name": "bert",
      "fixed_text": "This is a sentence that was broken across lines."
    },
    {
      "model_name": "distilbert",
      "fixed_text": "This is a sentence that was broken across lines."
    },
    {
      "model_name": "deberta",
      "fixed_text": "This is a sentence that was broken across lines.."
    }
  ]
}

Testing

bash

make test-api                           # Test API endpoints
make test-dataset                       # Test dataset utilities
make test-pipelines                     # Test inference pipelines
make test-all                           # Run all tests

Cleanup

bash

make clean                              # Remove checkpoint and plot directories

Models

The three fine-tuned models are hosted on HuggingFace:

Each model classifies sentence pairs into three boundary types:

—SAME_PARAGRAPH — sentences belong to the same paragraph (join with space)
—NEW_PARAGRAPH — new paragraph boundary (join with \n\n)
—NEWLINE — line break within a paragraph (join with \n)