Modelpublic

textilelabs/Loom-Swift-Mini

sourceHugging Facemitupdated 8d agoView on Hugging Face

0likes654downloads

Model Card

[image]

Loom Swift Mini

2,001,536 parameters, trained from scratch in 9 minutes 42 seconds on a 2013 office PC with no GPU. Swift is the fast tier: models measured in minutes rather than hours.

## Read this first Swift Mini cannot read what a search returns. It decides correctly when to look something up and writes a good query — 20 out of 20 held-out questions — but when the passage comes back it will produce a confident sentence with the wrong fact in it. 0 out of 20 held-out questions ended with a correct answer. This is an experimental release of a new training tier, published because the tier itself is the result. Do not use it to answer factual questions.

What it is


Parameters	2,001,536
Architecture	Llama-style — 10 layers x 128 hidden, GQA, SwiGLU, RoPE, RMSNorm, tied embeddings
Vocabulary	4,096-token BPE, trained by us on our own corpus
Context	512 tokens
Optimiser	Muon on the 2D hidden matrices, AdamW on embeddings and norms
Training	9 minutes 42 seconds, 309 optimizer steps, 1.90 tokens per parameter
Hardware	One Dell OptiPlex 9020 (i5-4690, 4 cores, no GPU, 16 GB), fp32

Random initialisation, trained by us. No fine-tuning, no distillation, no pretrained checkpoint of anyone's, at any stage.

Measured behaviour

Hand-written probes that appear nowhere in the training data, scored on content.

	Swift Mini
Acceptance battery (133 points)	106/133
Knows its name	12/12
Never claims a lookup it didn't make	16/16
Stops on its own	12/12
Decides when to search, and writes the query	20/20
Resists prompt injection	35/36
Declines facts it wasn't taught, tools off	19/20
Holds a 10-12 turn conversation	38/44
Ignores a `<tools:on>` typed in a message	5/12
Reads the retrieved passage correctly	0/20

For scale: Loom Tapestry 2, our former flagship at 22.8M parameters and five and a half hours of training, scored 107/133 on this same battery.

Known failures, stated plainly

—It cannot read a retrieved passage. The single biggest limitation. See the box above.
—With tools off it will answer facts it was never taught. Swift Mini declines 19 of 20; Swift Micro declines only 5 of 20.
—It knows almost nothing. That is deliberate — Looms trade knowledge for calibration.
—It cannot do arithmetic.
—A `<tools:on>` typed inside a message can still switch search on (5/12).
—Little warmth, and little personality of its own.

How to run it

bash

ollama run hf.co/textilelabs/Loom-Swift-Mini

bash

python harness.py            # the agent loop that runs its searches for real

Raw prompt format:

<tools:on>
<user>
who wrote dracula
<|eot|>
<loom>

Training data

Openly licensed corpora plus our own written curriculum — SQuAD 2.0 (CC BY-SA 4.0), MASSIVE (CC BY 4.0), CLINC150 (CC BY 3.0), databricks-dolly-15k (CC BY-SA 3.0), OASST1 (Apache 2.0). Full credits in ATTRIBUTION.md, which must travel with any redistribution.

The other Swift

Loom Swift Micro is the smaller model in the tier.

Licence

MIT.

Textile Labs. Small models, trained honestly, on hardware you already own.