Datasetpublic

solanaclawd/solana-clawd-repo-corpus

Solana Clawd Core AI Instruct Instruction-tuning dataset derived from the local core-ai source tree and the existing Solana Clawd AI training corpus. Contents Total examples: 441 Existing ai-training SFT examples: 0 Core AI source chunk examples: 0 Core AI knowledge JSONL examples: 0 Format Each row is a chat conversation in OpenAI/Hugging Face messages schema: {"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}… See the full description on the dataset page: https://huggingface.co/datasets/solanaclawd/solana-clawd-repo-corpus.

sourceHugging Facecc-by-4.0updated 21d agoView on Hugging Face

1likes121downloads

Dataset Card

Solana Clawd Core AI Instruct

Instruction-tuning dataset derived from the local core-ai source tree and the existing Solana Clawd AI training corpus.

—Total examples: 441
—Existing ai-training SFT examples: 0
—Core AI source chunk examples: 0
—Core AI knowledge JSONL examples: 0

Format

Each row is a chat conversation in OpenAI/Hugging Face messages schema:

json

{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

Reproduce

bash

cd ai-training
python3 scripts/build_core_ai_dataset.py
python3 scripts/prepare_dataset.py \
  --input data/core_ai_clawd_sft.jsonl \
  --output data/core_ai_processed \
  --train-ratio 0.9 --eval-ratio 0.05 --seed 42

Publish

bash

hf repos create solanaclawd/solana-clawd-repo-corpus --type dataset --exist-ok
hf upload solanaclawd/solana-clawd-repo-corpus data/core_ai_processed . --repo-type dataset --commit-message "Add processed Core AI splits"
hf upload solanaclawd/solana-clawd-repo-corpus data/core_ai_dataset_card.md README.md --repo-type dataset --commit-message "Add dataset card"
hf upload solanaclawd/solana-clawd-repo-corpus data/core_ai_clawd_sft.jsonl raw/core_ai_clawd_sft.jsonl --repo-type dataset --commit-message "Add raw JSONL"
hf upload solanaclawd/solana-clawd-repo-corpus data/core_ai_dataset_manifest.json metadata/core_ai_dataset_manifest.json --repo-type dataset --commit-message "Add build manifest"

Training

bash

python3 scripts/train_lora.py --config configs/core_ai_lora_config.yaml --no-push --num-epochs 1

For a remote Hugging Face Job, upload the dataset first and then launch with the same config or override --dataset-repo solanaclawd/solana-clawd-repo-corpus.

Safety Notes

The builder runs in public-safe mode by default. It excludes common secret filenames, private key/token patterns, binary artifacts, dependency folders, lockfiles, and high-risk security records that are not suitable for public dataset release.