Haster1137/nora-g3-hei-reasoning-logic

Dataset Card for nora-g3-hei-reasoning-logic

This dataset functions as a highly dense, cross-domain knowledge matrix and logical reasoning repository. It is explicitly structured to support local, high-speed Retrieval-Augmented Generation (RAG) workflows for smaller, agile parameter models (such as 7B parameter local architectures) running on consumer-tier hardware environments. By coupling dense mathematical, algorithmic, and scholastic token structures with multimedia speech text alignments, it serves as an open-book library to maximize local intelligence without expanding model weight complexity.

Dataset Details

Dataset Description

The repository serves as a centralized, aggregated mirror repository comprising five primary core data branches:

• —Core Asset Base: A master foundation featuring over 4,200,000 rows of aggregated structural instructional tuning data.
• —Google WaxalNLP Sub-Corpus: An extensive 1.7 Terabyte dataset featuring automated speech recognition (ASR) text matrices, contextual image descriptions, and conversational phonetic speech scripts covering 24+ Sub-Saharan African regional languages.
• —UIUC Magicoder Evol Matrix: An industrial-grade software engineering dataset featuring 110,000 synthetically evolved instruction trees, code optimization prompts, and multi-turn debugging data.
• —Academic Chains Corpus: A high-level competitive reasoning benchmark dataset mapping intricate scientific, mathematical, and logical problem-solving branches structured within explicit <think> deliberation tags.
• —FineWeb-Edu Filtered Repository: A massive, high-density textbook and scholastic dataset extracted via an advanced educational classifier to isolate pure learning sequences, university lectures, and logic blocks.

• —Curated by: Independent AI Research & Development Workspace (Aiden Wade / Haster1137)
• —Shared by: Haster1137
• —Language(s) (NLP): Multilingual (English, Kiswahili, Luganda, Shona, Akan, and 20+ additional regional language profiles)
• —License: Open Academic / Research Use Permitted (Inherited via composite source distribution terms)

Dataset Sources

• —Repository: https://huggingface.co/datasets/Haster1137/nora-g3-hei-reasoning-logic
• —Primary Upstream Elements: * google/WaxalNLP
• —ise-uiuc/Magicoder-Evol-Instruct-110K
• —marcodsn/academic-chains
• —HuggingFaceFW/fineweb-edu

Uses

Direct Use

This dataset is designed for:

• —Local context injection via Vector Search and Retrieval-Augmented Generation (RAG) pipelines.
• —Providing lightweight local 7B models with instant access to advanced programming troubleshooting guides, deep logical thinking layouts, and textbook resources.
• —Cross-lingual language modeling and contextual automated speech transcription parsing.

Out-of-Scope Use

This repository is optimized for retrieval and reference lookup. Attempting to force-train or fine-tune the entirety of the 1.7+ Terabyte raw multi-media and time-series files directly into a small model's weights on standard local hardware is out of scope. Such an approach runs counter to the architectural intent of maintaining high-speed, lightweight local inference profiles.

Dataset Structure

The repository utilizes a modular, folder-based storage directory split across parquet data layers, serialized json instruction pairs, and historical simulation tables:

• —data/ASR/dag/: Contains dense, multi-partitioned parquet datasets carrying text alignments and corresponding audio asset chains.
• —magicoder/: Contains logical coding instructions, python validation structures, and software engineering instruction graphs.
• —academic-chains/: Contains explicit chain-of-thought scientific problem solutions wrapped in reasoning labels.
• —fineweb-edu/: Contains filtered textbook tokens, historical data records, and scholastic breakdowns.

All major sub-partitions employ uniform, uncorrupted sequential tracking indexes (e.g., index_level_0) to guarantee unbroken continuity when loading chunks through automated in-memory binary network streaming routines.

Dataset Creation

Curation Rationale

Traditional modern AI training paradigms prioritize massive parameter memorization, which demands severe enterprise hardware cluster resources and intense electrical utility overheads. This repository was constructed under an economic and computational constraint framework: storing high-intelligence logic chains externally within a structured repository so that a fast, local model can query data dynamically on demand.

Source Data

Data Collection and Processing

Data is aggregated utilizing an optimized, zero-local-storage memory streaming Python API. The streaming connector establishes server-to-server HTTP multi-part transfer channels between source repositories and the target profile. This approach streams binary chunks through transient RAM buffers (io.BytesIO) without touching local drive partitions or storage storage sectors.

Who are the source data producers?

• —Google Engineering Teams: Creators of the WaxalNLP multilingual audio-text transcription infrastructure.
• —iSE Laboratory (University of Illinois Urbana-Champaign): Developers of the Magicoder synthetic instruction evolution framework.
• —Academic & Community Open-Source Machine Learning Laboratories: Contributors behind specialized training matrices, educational web filters, and benchmark logic chains.

Bias, Risks, and Limitations

• —Multimedia Overhead: The dataset contains heavy raw audio streams inside the speech recognition blocks; text parsers must explicitly target text column variables to maintain search alignment efficiency.
• —Synthetic Alignment: Synthetically expanded instruction data should be verified against standard execution runtimes prior to safety-critical code deployments.

Recommendations

Users integrating this database into a local Flask-backed server interface should employ strict embedding models to chunk text elements efficiently. This ensures prompt queries yield high-density logical returns without saturating the model's active attention window.

Citation

If utilizing these constituent components in academic research, please cite the original upstream laboratories:

@misc{google2024waxalnlp,
  title={WaxalNLP: A Multilingual Speech Corpus for Sub-Saharan African Languages},
  author={Google Research},
  year={2024}
}

@misc{wei2023magicoder,
  title={Magicoder: Source Code Is All You Need},
  author={Yuyan Wei and Zhe Wang and Jiawei Liu and Yuxiang Gao and Tianqi Zhang and Shaohua Zhou},
  journal={arXiv preprint arXiv:2312.02120},
  year={2023}
}