CoolFace
Apppublic

LisaMegaWatts/JuliaFluxGPT-fused-v2

sourceHugging Facemitupdated 7mo agoView on Hugging Face
0likes
App README

JuliaFluxGPT-fused-v2

Cross-species symbiogenesis: Pythia-14m-deduped (300B tokens) + JuliaSLM (5M) fused into JuliaFluxGPT architecture (23M, d=512) and fine-tuned for 1500 steps on classical philosophy texts. Serves an OpenAI-compatible API with streaming support.

Endpoints

  • —GET / — Health check and model info
  • —GET /v1/models — List available models
  • —POST /v1/chat/completions — Generate text (supports streaming, top-k, top-p)

Usage

bash
# Non-streaming
curl -X POST https://your-space.hf.space/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "the nature of"}], "max_tokens": 200}'

# Streaming
curl -X POST https://your-space.hf.space/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "the nature of"}], "stream": true, "temperature": 0.7, "top_k": 40}'

Architecture

  • —Model: ~23M params, 512d embed, 8 layers, 8Q/2KV heads (GQA)
  • —Origin: Cross-species symbiogenesis — Pythia-14m (300B tokens, d=128) + JuliaSLM (5M, d=256) projected into JuliaFluxGPT (23M, d=512)
  • —Sequence mixing: Grouped Query Attention + RoPE
  • —Tokenizer: BPE (2000 tokens)
  • —Framework: PyTorch (inference)
  • —Normalization: RMSNorm (pre-norm)
  • —Feed-forward: SwiGLU activation
  • —Weight tying: Shared embedding/output projection
  • —Val loss: 3.873 (after 1500 fine-tuning steps)

Links