Datasetpublic

saarus72/ficbook_text_norm

Texts inverse normalized obtained from ficbook dataset. Normalized using these notebooks for a personal russian normalization model (avaliable on HF Space as well). All put into single jsonl file with lines like (beautified): { "replaces": [ { "text_from": "Боль во всем теле...Боже...я так и знала.... ", "text_to": "Боль во всем теле...Боже...я так и знала.... " }, { "text_from": "5", "text_to": "Пятая" }, { "text_from": " точка буквально… See the full description on the dataset page: https://huggingface.co/datasets/saarus72/ficbook_text_norm.

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face

0likes9downloads

Dataset Card

Texts inverse normalized obtained from ficbook dataset. Normalized using these notebooks for a personal russian normalization model (avaliable on HF Space as well).

All put into single jsonl file with lines like (beautified):

json

{
  "replaces": [
    {
      "text_from": "Боль во всем теле...Боже...я так и знала.... ",
      "text_to": "Боль во всем теле...Боже...я так и знала.... "
    },
    {
      "text_from": "5",
      "text_to": "Пятая"
    },
    {
      "text_from": " точка буквально ныла от боли, ушиб будет тот еще...садиться не смогу дня ",
      "text_to": " точка буквально ныла от боли, ушиб будет тот еще...садиться не смогу дня "
    },
    {
      "text_from": "3",
      "text_to": "три"
    },
    {
      "text_from": " так, точно.",
      "text_to": " так, точно."
    }
  ]
}