harsharajkumar273/ProphetNet-Large-Story-Generation

1---2language: en3license: apache-2.04base_model: harsharajkumar273/ProphetNet-Large-Summarization5tags:6  - text-generation7  - story-generation8  - research-paper9  - seq2seq10  - prophetnet11  - lora12  - peft13datasets:14  - custom15metrics:16  - bertscore17  - sbert18---19 20# ProphetNet-Large-Story-Generation21 22A fine-tuned model for transforming research paper summaries into engaging short stories. This is the second stage of a two-step **Research Paper Simplifier** pipeline, built on top of [harsharajkumar273/ProphetNet-Large-Summarization](https://huggingface.co/harsharajkumar273/ProphetNet-Large-Summarization).23 24## Model Description25 26This model takes a summary of a research paper and generates an immersive, narrative-style short story. Fine-tuned using LoRA (PEFT) with 4-bit quantization.27 28## Pipeline29 30```31Research Paper ──► [ProphetNet-Large-Summarization] ──► Summary ──► [ProphetNet-Large-Story-Generation] ──► Story32```33 34## Training Details35 36| Parameter | Value |37|-----------|-------|38| Base model | harsharajkumar273/ProphetNet-Large-Summarization |39| Task | Story Generation |40| Max input length | 1024 tokens |41| Max target length | 512 tokens |42| Learning rate | 5e-5 |43| Batch size | 2 |44| Gradient accumulation steps | 4 |45| Warmup steps | 1000 |46| Weight decay | 0.01 |47| Fine-tuning method | LoRA (r=16, alpha=64, targets: query_proj, value_proj) |48| Quantization | 4-bit NF4 (bitsandbytes) |49 50## Usage51 52```python53from transformers import AutoTokenizer, AutoModelForSeq2SeqLM54 55# Stage 1: Summarize the paper56sum_tokenizer = AutoTokenizer.from_pretrained("harsharajkumar273/ProphetNet-Large-Summarization")57sum_model = AutoModelForSeq2SeqLM.from_pretrained("harsharajkumar273/ProphetNet-Large-Summarization")58 59paper_text = "Your research paper text here..."60word_count = len(paper_text.split())61sum_prompt = f"Summarize this part of the research paper to less than {word_count // 10} words:\n{paper_text}"62sum_inputs = sum_tokenizer(sum_prompt, return_tensors="pt", max_length=2048, truncation=True)63sum_outputs = sum_model.generate(**sum_inputs, max_length=256, num_beams=4)64summary = sum_tokenizer.decode(sum_outputs[0], skip_special_tokens=True)65 66# Stage 2: Generate a story from the summary67story_tokenizer = AutoTokenizer.from_pretrained("harsharajkumar273/ProphetNet-Large-Story-Generation")68story_model = AutoModelForSeq2SeqLM.from_pretrained("harsharajkumar273/ProphetNet-Large-Story-Generation")69 70story_inputs = story_tokenizer(summary, return_tensors="pt", max_length=1024, truncation=True)71story_outputs = story_model.generate(**story_inputs, max_length=512, num_beams=4)72story = story_tokenizer.decode(story_outputs[0], skip_special_tokens=True)73print(story)74```75 76## Evaluation Metrics77 78Evaluated using BERTScore and SBERTScore on a held-out 10% test split.79 80## Related Models81 82- [harsharajkumar273/ProphetNet-Large-Summarization](https://huggingface.co/harsharajkumar273/ProphetNet-Large-Summarization) — previous stage83- [harsharajkumar273/Bart-Base-Story-Generation](https://huggingface.co/harsharajkumar273/Bart-Base-Story-Generation)84- [harsharajkumar273/T5-Base-Story-Generation](https://huggingface.co/harsharajkumar273/T5-Base-Story-Generation)85