CoolFace
Datasetpublic

mkenfenheuer/rvl-cdip-restoration

RVL-CDIP Restoration A document image restoration dataset based on the original RVL-CDIP dataset. Original dataset: - https://huggingface.co/datasets/ChainYo/rvl-cdip This version converts the classification dataset into a supervised image-to-image restoration dataset. Each sample contains: - lr → degraded document (input) - hr → original clean document (target) Images are stored as: - raw numpy bytes - dtype: uint8 - shape: (512, 512, 3) Dataset Overview This… See the full description on the dataset page: https://huggingface.co/datasets/mkenfenheuer/rvl-cdip-restoration.

sourceHugging Faceupdated 7mo agoView on Hugging Face
0likes56downloads
Dataset Card

RVL-CDIP Restoration

A document image restoration dataset based on the original RVL-CDIP dataset.

Original dataset: - https://huggingface.co/datasets/ChainYo/rvl-cdip

This version converts the classification dataset into a supervised image-to-image restoration dataset.

Each sample contains: - lr → degraded document (input) - hr → original clean document (target)

Images are stored as: - raw numpy bytes - dtype: uint8 - shape: (512, 512, 3)

Dataset Overview

This dataset is derived from the RVL-CDIP document classification dataset and reformulated for supervised document restoration tasks.

Instead of classification labels, each sample now contains a paired image:

ColumnDescription
lrDegraded document image
hrClean original document image

Both are stored as raw NumPy byte arrays to minimize storage overhead and maximize loading speed.

[image]

Image Format

Each image:

  • —Resolution: 512×512
  • —Channels: 3 (RGB)
  • —Datatype: uint8
  • —Storage: raw bytes

To convert into tensors:

python
from datasets import load_dataset
import numpy as np
import torch

ds = load_dataset("mkenfenheuer/rvl-cdip-restoration", split="train")

PATCH_SIZE = 512
H, W, C = PATCH_SIZE, PATCH_SIZE, 3

sample = ds[0]

# Convert bytes → numpy
lr_np = np.frombuffer(sample["lr"], dtype=np.uint8).reshape(H, W, C)
hr_np = np.frombuffer(sample["hr"], dtype=np.uint8).reshape(H, W, C)

# Convert numpy → torch tensor [C,H,W] in [0,1]
lr_tensor = torch.from_numpy(lr_np).permute(2,0,1).float() / 255.0
hr_tensor = torch.from_numpy(hr_np).permute(2,0,1).float() / 255.0

After conversion:

  • —Shape: [C, H, W]
  • —Range: [0,1]
  • —dtype: float32

Task Definition

Supervised image restoration:

model(lr) → hr

Where: - lr = degraded input document - hr = clean ground truth

Recommended Loss Functions

  • —L1 Loss
  • —SSIM / MS-SSIM
  • —Perceptual Loss (VGG-based)
  • —Combined L1 + SSIM

Example:

python
loss = 0.8 * L1(pred, hr) + 0.2 * (1 - SSIM(pred, hr))

Example Training Script

python
import torch
import torch.nn as nn
from torch.utils.data import DataLoader
from datasets import load_dataset
import numpy as np

DEVICE = "cuda"
PATCH_SIZE = 512
H, W, C = PATCH_SIZE, PATCH_SIZE, 3
BATCH_SIZE = 4

ds = load_dataset("mkenfenheuer/rvl-cdip-restoration", split="train")

def collate_fn(batch):
    lr_list = []
    hr_list = []

    for sample in batch:
        lr_np = np.frombuffer(sample["lr"], dtype=np.uint8).reshape(H,W,C)
        hr_np = np.frombuffer(sample["hr"], dtype=np.uint8).reshape(H,W,C)

        lr_tensor = torch.from_numpy(lr_np).permute(2,0,1).float() / 255.0
        hr_tensor = torch.from_numpy(hr_np).permute(2,0,1).float() / 255.0

        lr_list.append(lr_tensor)
        hr_list.append(hr_tensor)

    return {
        "lr": torch.stack(lr_list),
        "hr": torch.stack(hr_list)
    }

loader = DataLoader(ds, batch_size=BATCH_SIZE, shuffle=True, collate_fn=collate_fn)

model = torch.nn.Conv2d(3,3,3,padding=1).to(DEVICE)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)
criterion = nn.L1Loss()

for epoch in range(10):
    model.train()
    for batch in loader:
        lr = batch["lr"].to(DEVICE)
        hr = batch["hr"].to(DEVICE)

        pred = model(lr)
        loss = criterion(pred, hr)

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

    print(f"Epoch {epoch} | Loss: {loss.item():.4f}")

Example Inference Script

python
import torch
import matplotlib.pyplot as plt
import numpy as np
from datasets import load_dataset

DEVICE = "cuda"
PATCH_SIZE = 512
H, W, C = PATCH_SIZE, PATCH_SIZE, 3

ds = load_dataset("mkenfenheuer/rvl-cdip-restoration", split="val")

model = torch.load("model.pt").to(DEVICE)
model.eval()

sample = ds[0]

lr_np = np.frombuffer(sample["lr"], dtype=np.uint8).reshape(H, W, C)
hr_np = np.frombuffer(sample["hr"], dtype=np.uint8).reshape(H, W, C)

lr_tensor = torch.from_numpy(lr_np).permute(2,0,1).float() / 255.0
hr_tensor = torch.from_numpy(hr_np).permute(2,0,1).float() / 255.0

lr_tensor = lr_tensor.unsqueeze(0).to(DEVICE)

with torch.no_grad(), torch.amp.autocast(device_type="cuda"):
    restored_tensor = model(lr_tensor)

def tensor_to_rgb_np(t):
    if t.dim() == 4:
        t = t.squeeze(0)
    np_img = t.permute(1,2,0).cpu().numpy()
    np_img = np.clip(np_img, 0.0, 1.0)
    return np_img.astype(np.float32)

lr_np       = tensor_to_rgb_np(lr_tensor)
restored_np = tensor_to_rgb_np(restored_tensor)

fig, axes = plt.subplots(1, 2, figsize=(8,4))
axes[0].imshow(lr_np)
axes[0].set_title("LR")
axes[1].imshow(restored_np)
axes[1].set_title("Restored")

for ax in axes:
    ax.axis("off")

plt.tight_layout()
plt.show()

[image]

Citation

@inproceedings{harley2015evaluation, title={Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval}, author={Harley, Adam W. and Ufkes, Alex and Derpanis, Konstantinos G.}, booktitle={ICDAR}, year={2015} }

License

RVL-CDIP is a subset of IIT-CDIP, which came from the Legacy Tobacco Document Library, for which license information can be found here.