psudini/Llama-3.2-1B-SudokuJev

Llama-3.2-1B-SudokuJev

Llama-3.2-1B with its LM head replaced by a 9-digit rating head. Given one empty Sudoku cell's row, column and 3x3 box, it returns a score for each digit 1-9 meaning "could this digit legally go here?". It never generates text: it's an option scorer, like a Jev-style decision API.

Code, solver, web UI and training notebook: github.com/pruthvinathsudini/sudoku-jev.

row: 4 . . 8 ? 3 . . 1 | col: 7 9 . 6 ? 2 . 1 8 | box: . 6 . 8 ? 3 . 2 . | answer:
   1-0.00  2-0.00  3-0.00  4-0.00  5-1.00  6-0.00  7-0.00  8-0.00  9-0.00   → only 5 fits

row: 5 3 . ? 7 . . . . | col: ? 1 . . 8 . . 4 . | box: ? 7 . 1 9 5 . . . | answer:
   1-0.00  2-1.00  3-0.00  4-0.00  5-0.00  6-1.00  7-0.00  8-0.00  9-0.00   → 2 or 6, skip for now

Architecture

Same recipe as AlexWortega/openjev: take a decoder LLM, drop the LM head, pool the last token's hidden state, and train a classification head.

cell text ──► Llama-3.2-1B (LoRA-tuned, no LM head) ──► last token hidden state (2048)
          ──► Linear(2048 → 9) ──► sigmoid ──► 9 independent digit scores

A solver fills a cell when exactly one digit scores above 0.5 and repeats over the grid until it's solved.

Files

Usage

Requires access to the gated meta-llama/Llama-3.2-1B.

import torch
from huggingface_hub import snapshot_download
from peft import PeftModel
from safetensors.torch import load_file
from transformers import AutoModel, AutoTokenizer

path = snapshot_download("psudini/Llama-3.2-1B-SudokuJev")
tok = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B")
base = AutoModel.from_pretrained("meta-llama/Llama-3.2-1B", dtype=torch.float32)
backbone = PeftModel.from_pretrained(base, path).eval()
head = torch.nn.Linear(2048, 9)
head.load_state_dict(load_file(f"{path}/head.safetensors"))

cell = "row: 4 . . 8 ? 3 . . 1 | col: 7 9 . 6 ? 2 . 1 8 | box: . 6 . 8 ? 3 . 2 . | answer:"
with torch.no_grad():
    hidden = backbone(**tok(cell, return_tensors="pt")).last_hidden_state[0, -1]
    scores = torch.sigmoid(head(hidden))
print({d: round(s, 3) for d, s in zip(range(1, 10), scores.tolist())})
# {1: 0.0, 2: 0.0, 3: 0.0, 4: 0.0, 5: 1.0, 6: 0.0, 7: 0.0, 8: 0.0, 9: 0.0}

Input format (must match exactly): row: <9> | col: <9> | box: <9> | answer: where the target cell is ?, empty cells are ., the box is read left-to-right, top-to-bottom, and the tokenizer adds <|begin_of_text|>. Pool the hidden state of the final token (:). For batches, pad on the right and take each prompt's own last token; no attention mask is needed because the model is causal.

The GitHub repo has an ONNX exporter that merges the adapter and runs the model on any DirectX 12 GPU (tested on an AMD RX 6750 XT: ~17 ms per cell in fp32).

Training

Evaluation

Limitations

• —It learned one deduction: which digits are absent from a cell's row, column and box. Puzzles that need harder techniques (hidden singles, pairs, chains) stall: no cell is left with a single option. It does not guess.
• —It was trained and tested only on generated grids with the exact text layout above. Other layouts or text are out of distribution.
• —The same deduction is a few lines of ordinary code; this model demonstrates the LLM-as-option-scorer recipe, it is not a better Sudoku solver.

License

Built with Llama. This model is a derivative of Llama 3.2 and is distributed under the Llama 3.2 Community License (Copyright © Meta Platforms, Inc. All Rights Reserved). Use is subject to the Llama 3.2 Acceptable Use Policy.