datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
visual-novels-jsonvisual-novel-summary
Dataset Card for Visual Novel Summary
This dataset contains excerpts from visual novels, derived from the winglian/visual-novels-json dataset. Each excerpt is annotated with respect to elements of fiction, including setting, characters, plot, conflict, themes, point of view, and tone.
Dataset Details
Dataset Description
This dataset was created by splitting rows from the winglian/visual-novels-json dataset into chunks of 32 consecutive lines. Each chunk was… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/visual-novel-summary.alpindale_visual-novelsimport json
import os
import ftfy
dataset = []
for filename in sorted(os.listdir("./visual-novels-parsed/TXT")):
txt_path = os.path.join(TXT_DIR, filename)
with open(txt_path, "r") as f:
txt_data = ftfy.fix_text(f.read()).strip()
dataset.append(
{
"name": ftfy.fix_text(filename.replace(".txt", "")).strip(),
"text": txt_data,
}
)
with open("./visual-novels-parsed/train.json", "w") as f:
json.dump(dataset, f, indent=2)
winglian_visual-novels-json-axo