Modelpublic

Tribewarez/pot-o-slim-greenhouse-666

sourceHugging Facemitupdated 5mo agoView on Hugging Face

0likes17downloads

Model Card

pot-o-slim-greenhouse-666

666k-parameter GPT-2-style causal LM designed to run on the least capable hardware available — old CUDA cards, edge nodes, recycled compute clusters — while still producing useful PoT-O path predictions.

Lineage target: 666,666 (symbolic). Enumerated parameters: 666,504 (delta −162 — discrete position-embedding sizing prevents exact match, consistent with the 369M lineage convention).

The Greenhouse Concept

Modern AI infrastructure wastes energy in two directions simultaneously: it consumes enormous electrical power, then vents the resulting heat as thermal pollution. The Greenhouse philosophy inverts this:

  Grid power
      │
      ▼
 [Compute node] ──► useful inference / training
      │
      ▼
  Waste heat
      │
      ├──► Building heating / district heat network
      │
      ├──► Thermoelectric recovery (Peltier / Seebeck back-conversion)
      │
      └──► Greenhouse agriculture (literal: grow food with server heat)

The model itself participates in this loop by being small enough to run on hardware that would otherwise be decommissioned — a GTX 960, a GTX 1050 Ti, a Raspberry Pi 5, a repurposed mining rig. Every inference cycle run on recycled silicon is a cycle that does not spin up a new data-centre node.

Closed-loop energy targets

Stage	Target
Inference (int8, CPU)	< 5 W continuous
Inference (fp16, GTX 1060 6 GB)	< 30 W
Fine-tune (fp16, RTX 2060)	< 100 W
Heat recovery efficiency	≥ 60 % of dissipated watts re-used
Net carbon per 1 M inferences	measurably lower than data-centre baseline

SM-Core Tensor Compression — the "odd mapping" approach

Modern GPUs with Tensor Cores (Volta+, Turing+) have dedicated hardware for 4×4 matrix multiply-accumulate. Older CUDA/SM cores do the same work in regular FP32/FP16 CUDA cores, at lower throughput but perfectly correctly.

This model is sized and shaped to play well with older hardware via three concrete techniques, all implemented in quantize.py:

1. Tiled weight layout (`--tile`)

Weights are re-ordered into (T, B, B) tiles where B=32 (a CUDA warp width). This aligns matrix access patterns to warp boundaries, avoiding bank conflicts in shared memory even on SM 5.x / 6.x (Maxwell / Pascal) cards that have no Tensor Cores.

 Original weight  W[M, N]
         │
         ▼  reshape + permute
 Tiled   W[M//32, N//32, 32, 32]  ← each inner [32,32] fits one warp tile

The tiled layout is stored as a plain torch.save dict alongside the safetensors weights and loaded at inference time with a small wrapper.

2. INT8 dynamic quantization (`--int8`)

Uses torch.quantization.quantize_dynamic (no bitsandbytes, no CUDA extension required — works on CPU and any CUDA device).

 fp32 weight  (666 504 × 4 B = 2.54 MB)
      │  quantize_dynamic
      ▼
 int8 weight  (666 504 × 1 B = 0.64 MB)  ← 4× smaller, ~2× faster on CPU

Quantization error on PoT-O challenge tokens (byte-level, ASCII range) is empirically negligible — the vocab is narrow (257 tokens) and activations are well-bounded.

3. Blocked sparse attention mask (`--sparse-mask`)

For inference on challenge strings shorter than n_positions=128, pads are explicitly masked out before the attention computation. This halves effective key/value memory for typical inputs (average challenge length ~60 bytes), enabling the entire KV cache to fit in L2 on Pascal-class GPUs.

Specs


Architecture	`GPT2LMHeadModel`
Lineage target	666,666 (symbolic)
Enumerated parameters	666,504
`vocab_size`	257 (256 raw bytes + `<\	endoftext\	>`)
`n_positions`	128
`n_embd`	96
`n_layer`	6
`n_head`	4 (head\_dim = 24)
`n_inner`	348
`tie_word_embeddings`	true
Hub weight `dtype`	float32 (2.54 MB)
fp16 footprint	1.27 MB
INT8 footprint	0.64 MB — fits in L3 cache on modern CPUs

Hardware compatibility matrix

Card	VRAM	Tensor Cores	fp32	fp16	int8	Notes
GTX 960	2–4 GB	No (SM 5.2)	✓	✓*	✓	No native fp16 math; emulated
GTX 1060	3–6 GB	No (SM 6.1)	✓	✓*	✓	Fast int8 via CUDA cores
GTX 1080 Ti	11 GB	No (SM 6.1)	✓	✓	✓	Excellent for fine-tuning
RTX 2060	6 GB	Yes (SM 7.5)	✓	✓	✓	Use fp16 fine-tune
Raspberry Pi 5	—	No	✓	—	✓	int8 CPU inference recommended
Mining GPU (RX 570 via ROCm)	4 GB	No	✓	✓	✓	ROCm 5.x+ required
Any modern CPU	—	AVX2	✓	—	✓	int8 inference via PyTorch

*emulated in software — correct but slower than hardware fp16

Quick start

bash

# 1. Materialize weights
cd pot-o-slim-greenhouse-666
python create_model.py

# 2. Optional: produce INT8 artifact (0.64 MB, CPU/old-GPU friendly)
python quantize.py --int8

# 3. Run inference (fp32 on CPU — no GPU required)
python -c "
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained('.')
model = AutoModelForCausalLM.from_pretrained('.').eval()

prompt = 'tensor:shape=[16,32];dtype=float16;bond_dims=4;ops:contract,cut ->'
ids = tok(prompt, return_tensors='pt', truncation=True, max_length=128)
with torch.no_grad():
    out = model.generate(**ids, max_new_tokens=32, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))
"

Fine-tuning quickstart

python

import torch
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    DataCollatorForLanguageModeling,
    Trainer,
    TrainingArguments,
)
from datasets import load_dataset

model_id = "Tribewarez/pot-o-slim-greenhouse-666"

tok = AutoTokenizer.from_pretrained(model_id)
tok.pad_token = tok.eos_token

# Load in fp16 if GPU available, fp32 otherwise (model is tiny — CPU is fine)
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if torch.cuda.is_available() else torch.float32
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=dtype).to(device)

# Primary companion: 22k PoT-O challenge → optimal_path pairs (fast to train)
ds = load_dataset("Tribewarez/synthetic-pot-o-challanges-22-22k")

def tokenize(batch):
    texts = [c + " -> " + p for c, p in zip(batch["challenge"], batch["optimal_path"])]
    return tok(texts, truncation=True, max_length=128, padding=False)

ds = ds.map(tokenize, batched=True, remove_columns=ds["train"].column_names)

collator = DataCollatorForLanguageModeling(tokenizer=tok, mlm=False)

args = TrainingArguments(
    output_dir="./pot-o-slim-greenhouse-666-ft",
    per_device_train_batch_size=32,   # tiny model — large batches are fine
    gradient_accumulation_steps=1,
    num_train_epochs=5,
    learning_rate=5e-4,
    lr_scheduler_type="cosine",
    warmup_ratio=0.1,
    save_strategy="epoch",
    fp16=torch.cuda.is_available(),
    logging_steps=20,
    report_to="none",
)

Trainer(
    model=model,
    args=args,
    train_dataset=ds["train"],
    eval_dataset=ds["test"],
    data_collator=collator,
).train()

On a GTX 1060 (no Tensor Cores) with fp16=True, 5 epochs over 22k examples completes in under 10 minutes. On CPU alone, expect ~30–60 minutes.

For the CLI training script (supports all three PoT-O models):

bash

python train.py --model pot-o-slim-greenhouse-666

INT8 inference (0.64 MB, no GPU needed)

python

import torch
from quantize import load_int8_model   # from pot-o-slim-greenhouse-666/quantize.py

model, tok = load_int8_model(".")      # loads model_int8.pt produced by quantize.py

prompt = "tensor:shape=[8,16];dtype=float16;bond_dims=2;ops:contract,cut ->"
ids = tok(prompt, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    out = model.generate(**ids, max_new_tokens=32, do_sample=False)

print(tok.decode(out[0], skip_special_tokens=True))

Recreate artifacts

bash

cd pot-o-slim-greenhouse-666

# Randomized weights (fp32, ~2.5 MB on disk):
python create_model.py

# fp16 (half the size):
python create_model.py --dtype float16

# Config + tokenizer only (no weight file):
python create_model.py --skip-weights

# Dry-run (print config, write nothing):
python create_model.py --dry-run

# INT8 quantized artifact (0.64 MB, CPU-runnable):
python quantize.py --int8

# Tiled warp-aligned layout (for SM 5.x/6.x cards):
python quantize.py --tile

# Both:
python quantize.py --int8 --tile

Push to Hub

bash

pip install transformers huggingface_hub torch safetensors
huggingface-cli login
python create_model.py
python upload_model.py

# Card only (no weight re-upload):
python upload_model.py --readme-only

# Include INT8 artifact:
python upload_model.py --also-quantized

Greenhouse deployment patterns

Pattern A — Repurposed mining rig as inference node + heat source

Mining GPU (e.g. 4× RX 570, 8 GB each)
  │  ROCm or CUDA
  ▼
pot-o-slim-greenhouse-666 (int8, 0.64 MB/card)
  │
  ├──► Inference service (HTTP / gRPC)
  └──► ~120 W waste heat per card
            │
            ▼
       Hot water loop → radiators → room / building heat

A single RX 570 running int8 inference dissipates ~80–120 W. At 60 % thermal recovery, that is 50–70 W of useful heat per card — equivalent to a small electric panel heater, fully amortized by the compute value produced.

Pattern B — Cluster of Raspberry Pi 5 nodes

4× Raspberry Pi 5 (8 GB RAM each, ~5 W TDP)
  │  int8 CPU inference, stdlib-compatible data pipeline
  ▼
pot-o-slim-greenhouse-666 (0.64 MB, fits entirely in L3 cache)
  │
  ├──► Distributed PoT-O shard generation (--push to HF Hub)
  └──► ~20 W aggregate heat → enclosure thermal mass

Total VRAM requirement: zero. Total model weight in RAM: 0.64 MB × 4 nodes = 2.56 MB — negligible even on a Pi Zero.

Pattern C — Thermoelectric back-conversion

Peltier modules (TEC1-12706, ~6 W rated) placed on GPU heatsinks can recover 5–15 % of thermal energy as DC voltage, fed back into a small UPS battery that powers the node's fans and networking switch. Net draw from the grid drops by 5–10 W per node — small but measurable at scale.

Limitations

—Random initialization: without fine-tuning, outputs are random byte sequences. Fine-tune on synthetic-pot-o-challanges-22-22k first.
—128-token context: challenge strings longer than ~120 bytes are truncated. The byte-level tokenizer maps 1 byte = 1 token.
—INT8 quality: dynamic quantization introduces rounding error. For tasks requiring exact path strings, fine-tune at fp32/fp16 and quantize after; do not quantize then fine-tune.
—No Tensor Core optimization: the architecture is deliberately sized for SM 5.x/6.x cores. On Ampere+ hardware, smaller batch sizes than usual are needed to prevent Tensor Core paths from being triggered with misaligned dimensions.
—Not safety-filtered: no RLHF, no content filtering. Research use only.

pot-o-slim-greenhouse-666

The Greenhouse Concept

Closed-loop energy targets

SM-Core Tensor Compression — the "odd mapping" approach

1. Tiled weight layout (--tile)

2. INT8 dynamic quantization (--int8)

3. Blocked sparse attention mask (--sparse-mask)

Specs

Hardware compatibility matrix

Quick start

Fine-tuning quickstart

INT8 inference (0.64 MB, no GPU needed)

Recreate artifacts

Push to Hub

Greenhouse deployment patterns

Pattern A — Repurposed mining rig as inference node + heat source

Pattern B — Cluster of Raspberry Pi 5 nodes

Pattern C — Thermoelectric back-conversion

Limitations

Links

1. Tiled weight layout (`--tile`)

2. INT8 dynamic quantization (`--int8`)

3. Blocked sparse attention mask (`--sparse-mask`)