vikramlingam/PhysicsGIF-135M
019
PhysicsGIF-135M
� Natural Language to Scene Parser: A fine-tuned 135M parameter model that converts text descriptions into structured JSON scene specifications.
⚠️ Note: This model is the text parsing component of a larger physics-based GIF generation pipeline. It does NOT generate GIFs directly, it outputs structured JSON that is then processed by a separate physics engine and renderer.
What This Model Does
"a red ball bouncing to the right"
│
▼
┌─────────────────────┐
│ PhysicsGIF-135M │ ← THIS MODEL
│ (Text → JSON) │
└──────────┬──────────┘
│
▼
{
"objects": [{"type": "ball", "color": "#FF0000"}],
"motion": {"velocity": [3, 0], "gravity": 0.3, "bounce": 0.9},
"canvas": {"size": 128, "frames": 40}
}The JSON output is then processed by separate Python code (physics engine + renderer) to create the actual GIF.
🎬 Example Outputs
📊 Training Results
<details> <summary>📈 Training Visualizations</summary>
Training Loss Curve
Learning Rate Schedule
Gradient Norms
Per-Epoch Loss
Dataset Distribution
Convergence Analysis
</details>
🚀 Usage
With the Full Pipeline (Recommended)
To generate actual GIFs, you need the complete pipeline code:
git clone https://github.com/vikramlingam/PhysicsGIF-135M
cd PhysicsGIF-135M
pip install torch transformers peft pillow numpy tqdm
# Interactive mode - generates real GIFs
python generate.py🎬 PhysicsGIF Text-to-GIF Generator
Enter prompt: a red ball bouncing
Generating...
✓ Generated: output_1.gifUsing This Model Directly (Text → JSON only)
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("vikramlingam/PhysicsGIF-135M")
tokenizer = AutoTokenizer.from_pretrained("vikramlingam/PhysicsGIF-135M")
prompt = '''<|im_start|>system
You are a scene description parser. Convert text to JSON scene specification.<|im_end|>
<|im_start|>user
Convert to scene JSON: a red ball bouncing to the right<|im_end|>
<|im_start|>assistant
'''
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200, do_sample=False)
result = tokenizer.decode(outputs[0])
# Output: JSON scene specification
# You need physics.py and renderer.py to convert this to a GIF🏗️ Full Pipeline Architecture
┌───────────────────────────────────────────────────────────┐
│ Complete GIF Generation Pipeline │
├───────────────────────────────────────────────────────────┤
│ │
│ User Input: "a red ball bouncing" │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────┐ │
│ │ PhysicsGIF-135M (THIS MODEL) │ │
│ │ Fine-tuned LLM │ │
│ │ Converts text → JSON DSL │ │
│ └──────────────────┬──────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────┐ │
│ │ physics.py (Python code) │ │
│ │ Newtonian physics simulation │ │
│ │ Calculates positions per frame │ │
│ └──────────────────┬──────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────┐ │
│ │ renderer.py (Python code) │ │
│ │ PIL-based frame rendering │ │
│ │ Saves as animated GIF │ │
│ └──────────────────┬──────────────────┘ │
│ │ │
│ ▼ │
│ output.gif │
│ │
└───────────────────────────────────────────────────────────┘🎯 What This Model Understands
Objects
ball, square, triangle
Colors
red, blue, green, yellow, orange, purple, pink, cyan, white
Motion Patterns
bouncing— Gravity + elastic bouncefalling/dropping— Falls from topfloating— No gravitycolliding— Objects collideexploding— Triggers particle effects
Multi-Object
two balls, three triangles
📁 Required Files for GIF Generation
This model alone cannot generate GIFs. You need:
🔬 Training Details
- Method: LoRA fine-tuning
- Target Modules: qproj, vproj, kproj, oproj, gateproj, upproj, down_proj
- Hardware: CPU only (MacBook Pro)
- Dataset: 500 text-to-JSON examples
📜 License
Apache 2.0