Datasetpublic

Lambent/quantum-computing-2048-fineweb-sample

Test dataset for testing a data sampling and filtering script. Configuration for filtering: dataset: "HuggingFaceFW/fineweb-edu" output_file: "quantum_computing_entries.jsonl" state_file: "quantum_computing_dataset_state.json" # Processing parameters keywords: - "quantum computing" - "qubit" - "quantum entanglement" - "quantum supremacy" - "quantum algorithm" - "quantum error correction" max_entries: 1024 min_tokens: 1024 max_tokens: 2048 min_int_score: 4 # Shuffling… See the full description on the dataset page: https://huggingface.co/datasets/Lambent/quantum-computing-2048-fineweb-sample.

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes22downloads

Dataset Card

Test dataset for testing a data sampling and filtering script.

Configuration for filtering:

dataset: "HuggingFaceFW/fineweb-edu"
output_file: "quantum_computing_entries.jsonl"
state_file: "quantum_computing_dataset_state.json"

# Processing parameters
keywords:
  - "quantum computing"
  - "qubit"
  - "quantum entanglement"
  - "quantum supremacy"
  - "quantum algorithm"
  - "quantum error correction"

max_entries: 1024
min_tokens: 1024
max_tokens: 2048
min_int_score: 4

# Shuffling configuration
shuffle: true
seed: 42