Datasetpublic

mimir-lcm/fineweb-2-sentence-split

Fineweb 2 split into sentences. Instances per languages were sampled by us to balance the data w.r.t. Fineweb-edu. To split the text into sentences we used the sat3-l model from the wtpsplit library. We fix a sentence threshold of 0.02 and a maximum sentence length of 256. If you use this dataset, you should cite: @misc{penedo2025fineweb2pipelinescale, title={FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language}, author={Guilherme Penedo and… See the full description on the dataset page: https://huggingface.co/datasets/mimir-lcm/fineweb-2-sentence-split.

sourceHugging Faceodc-byupdated 4mo agoView on Hugging Face

0likes5.8kdownloads

50 commits on main

c31bf204mo ago

Update README.md

m-elio

fe197dd4mo ago

Update README.md

m-elio

19698de4mo ago

Create README.md

m-elio

36ba3114mo ago

Upload folder using huggingface_hub

m-elio

fca61334mo ago

Upload folder using huggingface_hub

m-elio

98d0e1c4mo ago

Upload folder using huggingface_hub

m-elio

d1756674mo ago

Upload folder using huggingface_hub

m-elio

577cf544mo ago

Upload folder using huggingface_hub

m-elio

ade76894mo ago

Upload folder using huggingface_hub

m-elio

d70363a4mo ago

Upload folder using huggingface_hub

m-elio

c2b832f4mo ago

Upload folder using huggingface_hub

m-elio

7c2331b4mo ago

Upload folder using huggingface_hub

m-elio

91658704mo ago

Upload folder using huggingface_hub

m-elio

78991e54mo ago

Upload folder using huggingface_hub

m-elio

babbb864mo ago

Upload folder using huggingface_hub

m-elio

a2b3d044mo ago

Upload folder using huggingface_hub

m-elio

66b97d74mo ago

Upload folder using huggingface_hub

m-elio

bd50a344mo ago

Upload folder using huggingface_hub

m-elio

7684bc74mo ago

Upload folder using huggingface_hub

m-elio

df3e1a74mo ago

Upload folder using huggingface_hub

m-elio

fd9ceea4mo ago

Upload folder using huggingface_hub

m-elio

e3cb2214mo ago

Upload folder using huggingface_hub

m-elio

db185d24mo ago

Upload folder using huggingface_hub

m-elio

d513c704mo ago

Upload folder using huggingface_hub

m-elio

bc3b38d4mo ago

Upload folder using huggingface_hub

m-elio

8374b904mo ago

Upload folder using huggingface_hub

m-elio

77702134mo ago

Upload folder using huggingface_hub

m-elio

d9821b84mo ago

Upload folder using huggingface_hub

m-elio

ea319654mo ago

Upload folder using huggingface_hub

m-elio

d3a65fa4mo ago

Upload folder using huggingface_hub

m-elio

6865e814mo ago

Upload folder using huggingface_hub

m-elio

1bcf3d94mo ago

Upload folder using huggingface_hub

m-elio

f07293b4mo ago

Upload folder using huggingface_hub

m-elio

7a4376c4mo ago

Upload folder using huggingface_hub

m-elio

af4a43d4mo ago

Upload folder using huggingface_hub

m-elio

6ba6cb14mo ago

Upload folder using huggingface_hub

m-elio

f40138a4mo ago

Upload folder using huggingface_hub

m-elio

868f1d54mo ago

Upload folder using huggingface_hub

m-elio

7b8e9ec4mo ago

Upload folder using huggingface_hub

m-elio

cc2175b4mo ago

Upload folder using huggingface_hub

m-elio

fa14c9e4mo ago

Upload folder using huggingface_hub

m-elio

0fc58e84mo ago

Upload folder using huggingface_hub

m-elio

45a40874mo ago

Upload folder using huggingface_hub

m-elio

155f5174mo ago

Upload folder using huggingface_hub

m-elio

aab61f04mo ago

Upload folder using huggingface_hub

m-elio

305a1424mo ago

Upload folder using huggingface_hub

m-elio

0cce77b4mo ago

Upload folder using huggingface_hub

m-elio

3c5b8564mo ago

Upload folder using huggingface_hub

m-elio

53901184mo ago

Upload folder using huggingface_hub

m-elio

ffbc2c44mo ago

Upload folder using huggingface_hub

m-elio