ffurfaro/keep-it-simple

1---2language:3- en4- fr5license: mit6task_categories:7- text-generation8- translation9pretty_name: Keep it simple !10size_categories:11- 100K<n<1M12---13 14 15# keep-it-simple16 17**Objective:** An ultra-minimalist dataset for pre-training tiny language models. The logic relies on **bidirectional symmetry** (**A is B and B is A]**) to foster deep semantic understanding. By training the model to predict the "prompt" from the "text" and vice versa, we maximize the utility of every pair.18 19## Data Sources20 21* **Simple English Wikipedia**: Simplified encyclopedic articles.22* **Vikidia (FR)**: Educational content for younger audiences.23* **[OPUS Books (en-fr)](https://aclanthology.org/L12-1246/)**: Aligned English-French literary translations.24* **[Cosmopedia-100k](https://huggingface.co/blog/cosmopedia)**: Synthetic educational content.25 26## Structure27 28| Column | Description |29| --- | --- |30| `prompt` | Input (concept, title, or English translation). |31| `text` | Output (explanation, summary, or French translation). |32| `seed_data` | Origin identifier (traceability). |33 34## Context & Usage35 36* **Bidirectional Training**: Each source item yields two training entries (`prompt` $\rightarrow$ `text` and `text` $\rightarrow$ `prompt`). This enforces semantic symmetry, [reversal curve](https://arxiv.org/abs/2309.12288) and limitate span corruption.37* **Minimalism**: More compact than the **[BabyLM](https://arxiv.org/abs/2602.20092)** challenge; focused on density and the purity of pairs to maximize efficiency on tiny, resource-constrained architectures.38* **Goal**: Rapid testing of alignment theories and training "pocket" models for fundamental, bidirectional interactions.39 40*This dataset is a minimalist research tool.*