Modelpublic

ryanscottbarrett/braille256-v6

sourceHugging Facemitupdated 9mo agoView on Hugging Face

0likes16downloads

Model Card

braille256-v6: Lattice-Aware Multimodal Braille Model

The first LLM with explicit dot-lattice structure in its architecture.

Model Description

braille256-v6 builds on the multimodal foundation of v5, integrating formal lattice theory into the training pipeline. This is not just a Braille-native model—it's a lattice-native model that understands the mathematical structure of Braille at the architectural level.

Key Innovations

Feature	Description
Lattice Attention	Attention scores incorporate Hamming-based similarity on Braille cells
Lattice Embeddings	Token embeddings initialized to respect Boolean lattice structure
Morphological Regularization	Training loss includes equivariance under erosion/dilation
Haptic Evaluation	New metrics for tactile quality of outputs

Architecture

Parameters: ~12M
Layers: 4
Heads: 4
Hidden: 256
Vocab: 32,000 (SentencePiece)
Context: 512

Lattice Attention

Standard transformer attention computes:

Attention(Q, K, V) = softmax(QK^T / √d) V

Lattice attention blends this with Braille-aware similarity:

LatticeAttn = (1-λ) * StandardAttn + λ * HammingAttn

where HammingAttn[i,j] = 8 - popcount(token[i] XOR token[j])

This gives the model an inductive bias toward understanding Braille structure.

Lattice Embeddings

For the first 256 tokens (corresponding to Braille cells), embeddings are initialized as:

python

embedding[i] = Σ basis[b] for each raised dot b in cell i

This means similar Braille cells (low Hamming distance) start with similar embeddings.

Morphological Regularization

Training includes a regularization term:

L_morph = ReLU(||emb - erode(emb)|| - ||emb - dilate(emb)||)

This encourages embeddings to respect the lattice ordering: erode(x) ≤ x ≤ dilate(x).

Theoretical Foundation

This model implements the formal theory from:

"Theoretical Foundations for 8-Dot Braille-Native LLMs"

Key theoretical components:

1.Braille Lattice: Boolean algebra (B⁸, ∧, ∨, ¬) with 256 elements
2.Morphological Operators: Erosion, dilation, opening, closing
3.Modality-Invariant Representation: (modality, sequence, embedding) triple
4.Lattice Metrics: Hamming distance, Jaccard similarity

See: braille_lattice_theory.py for full implementation.

Modality Support

Modality	Header	Status
TEXT	⣿⠁	✅ Trained
IMAGE	⣿⠃	✅ Trained
AUDIO	⣿⠇	✅ Trained
BINARY	⣿⠏	✅ Trained
VIDEO	⣿⠗	🔄 Framework ready

Haptic Evaluation Metrics

v6 introduces new evaluation metrics for tactile quality:

Metric	Description	Target	Achieved
Lattice Coherence	Adjacent tokens have low Hamming distance	> 0.7	0.743 ✅
Morphological Stability	Outputs stable under erosion/dilation	> 0.5	0.453
Haptic Score	Combined tactile quality metric	> 0.5	0.598 ✅

Training Results

Metric	Value
Final Loss	1.23
Training Steps	10,000
Training Time	2h 7m
Corpus	Balanced multimodal (25% each: text, image, audio, binary)
Corpus Size	164M chars

Usage

python

import torch
from train_lattice_v6 import Braille256LatticeModel, LatticeConfig

# Load model
config = LatticeConfig.from_dict(json.load(open("config.json")))
model = Braille256LatticeModel(config)
model.load_state_dict(torch.load("pytorch_model.bin"))

# Generate
input_ids = torch.tensor([[0x28, 0x29, 0x2A]])  # Some Braille tokens
output = model.generate(input_ids, max_length=100)

Training

bash

python train_lattice_v6.py \
    --corpus corpus/braille_multimodal_corpus.txt \
    --tokenizer tokenizers/braille_8dot_32k/braille_8dot_32k.model \
    --output models/braille256_v6_lattice \
    --steps 10000

Training Options

Flag	Description
`--no-lattice-attention`	Disable lattice attention (ablation)
`--no-lattice-embeddings`	Disable lattice embeddings (ablation)
`--no-morph-regularization`	Disable morphological regularization (ablation)

Model Family

Version	Focus	Parameters	Key Feature
v1-v3	6-dot Braille	~10M	Basic Braille LM
v4	8-dot Braille	29.9M	Full byte encoding
v5	Multimodal	11.5M	TEXT/IMAGE/AUDIO/BINARY
v6	Lattice-aware	11.5M	Hamming attention, morphological regularization, balanced multimodal corpus

Why Lattice-Aware?

Standard LLMs treat tokens as arbitrary symbols. braille256-v6 knows that:

1.Braille cells form a lattice: 256 elements with meet (∧) and join (∨)
2.Similar cells should have similar representations: Hamming distance matters
3.Morphological operations preserve meaning: Erosion/dilation are semantic
4.Tactile quality is measurable: Haptic metrics evaluate output quality

This makes v6 the first LLM designed for tactile-first AI.

Citation

bibtex

@misc{braille256v6,
  author = {Barrett, Ryan},
  title = {braille256-v6: Lattice-Aware Multimodal Braille Model},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/ryanscottbarrett/braille256-v6}
}

License

MIT