nevernever69/dit-doclaynet-segmentation

1---2library_name: transformers3license: mit4datasets:5- nevernever69/small-DocLayNet-v1.16pipeline_tag: image-segmentation7---8 9# 🧾 Model Card: `nevernever69/dit-doclaynet-segmentation`10 11## 🧠 Model Overview12 13This model is a fine-tuned version of [microsoft/dit-base](https://huggingface.co/microsoft/dit-base) for **document layout semantic segmentation** on the [DocLayNet](https://huggingface.co/datasets/ibm/DocLayNet) dataset (small subset: `nevernever69/small-DocLayNet-v1.1`). It segments scanned document images into 11 layout categories such as title, paragraph, table, and footer.14 15## 📚 Intended Uses16 17- Segment document images into structured layout elements18- Assist in downstream tasks like document OCR, archiving, and automatic annotation19- Useful for researchers and developers working in document AI or digital humanities20 21## 🏷️ Labels (11 Classes)22 23| ID | Label        | Color        |24|----|--------------|--------------|25| 0  | Background   | Black        |26| 1  | Title        | Red          |27| 2  | Paragraph    | Green        |28| 3  | Figure       | Blue         |29| 4  | Table        | Yellow       |30| 5  | List         | Magenta      |31| 6  | Header       | Cyan         |32| 7  | Footer       | Dark Red     |33| 8  | Page Number  | Dark Green   |34| 9  | Footnote     | Dark Blue    |35| 10 | Caption      | Olive        |36 37## 🧪 Training Details38 39- **Base model**: `microsoft/dit-base`40- **Dataset**: [`nevernever69/small-DocLayNet-v1.1`](https://huggingface.co/datasets/nevernever69/small-DocLayNet-v1.1)41- **Input size**: 1025×1025 (resized to 56×56 masks during training)42- **Batch size**: 843- **Epochs**: 244- **Learning rate**: 5e-545- **Loss function**: Cross-entropy46- **Hardware**: Trained with mixed precision (`fp16`) on GPU47 48## 📊 Evaluation49 50The model shows promising results on a validation subset, capturing distinct document elements with clear boundaries. Overlay visualizations confirm precise semantic segmentation of dense and sparse regions in historical and modern documents.51 52## 🚀 How to Use53 54```python55from transformers import AutoImageProcessor, BeitForSemanticSegmentation56from PIL import Image57import torch58 59# Load model60model = BeitForSemanticSegmentation.from_pretrained("nevernever69/dit-doclaynet-segmentation")61image_processor = AutoImageProcessor.from_pretrained("nevernever69/dit-doclaynet-segmentation")62 63# Load and preprocess image64image = Image.open("your-image.png").convert("RGB")65inputs = image_processor(images=image, return_tensors="pt").to("cuda")66 67# Inference68model.to("cuda").eval()69with torch.no_grad():70    outputs = model(**inputs)71    logits = outputs.logits72    upsampled = torch.nn.functional.interpolate(logits, size=image.size[::-1], mode="bilinear", align_corners=False)73    mask = upsampled.argmax(dim=1).squeeze().cpu().numpy()74```75 76## 🧑‍🎓 Author77 78Created by **Never** [`@nevernever69`](https://huggingface.co/nevernever69).  79Feel free to open issues or discuss improvements on the Hugging Face hub.80 81## 📝 Citation82 83If you use this model in your work, please consider citing:84 85```bibtex86@misc{never2025doclaynetseg,87  author = {Never},88  title = {Document Layout Segmentation using DiT-base fine-tuned on DocLayNet},89  year = {2025},90  howpublished = {\url{https://huggingface.co/nevernever69/dit-doclaynet-segmentation}}91}92```