ksanjiv05/laya-for-rn-executorch

Laya — ExecuTorch (.pte) for on-device / React Native

ExecuTorch exports of [convaiinnovations/laya](https://huggingface.co/convaiinnovations/laya) (the typed-decisions checkpoint: ModernBERT-large encoder + Laya's from-scratch decision head), ready to run on-device via the ExecuTorch XNNPACK and Core ML backends — e.g. from `react-native-executorch`.

Laya is a System-1 decision model: you give it a state (JSON/text) plus a typed question (choice / score / noul) and it returns a calibrated answer in one forward pass — it does not generate text.

Files

See BACKENDS.md for the per-platform backend matrix (XNNPACK / Vulkan / Core ML) and how to build the iOS Core ML variant.

Runtime contract (static shapes)

The graph is exported with fixed shapes for mobile:

Inputs (in order):

• —input_ids [1, 192] int64
• —attention_mask [1, 192] int64
• —marker_pos [1, 12] int64 — token index of each option's [MASK] marker
• —marker_mask [1, 12] bool — which of the 12 option slots are real
• —qtype [1] int64 — 0=choice, 1=score, 2=noul

Outputs:

• —logits [1, 12] float32 — per-option scores (take the first k = #real options)
• —act_logits [1, 2] float32 — auxiliary escalate/answer head

Post-processing (see rl_agent_api.py in the base repo): divide the k logits by the temperature for the (qtype, k) bucket (in the .meta.json), softmax, then: choice → argmax option; score → Σ i·pᵢ; noul → p[1] = P(true).

Input construction mirrors Laya's build_sequence: [CLS] "<type> question: <instructions>" [SEP] [MASK] opt0 [MASK] opt1 … [SEP] <state> [SEP] (tokenizer: the base repo's ModernBERT tokenizer; CLS=50281 SEP=50282 MASK=50284 PAD=50283).

Benchmark — iPhone 15 (Core ML)

laya_coreml.pte running fully offline in the React Native demo app (react-native-executorch 0.10.2, New Architecture) on an iPhone 15. Batch 1, seq len 192.

¹ First inference after load includes Core ML's one-time model preparation / warm-up. Steady-state latency is ~42–47 ms per decision (median 45 ms over the three warm runs).

4/4 decisions match the reference, with probabilities within ~2.5 points (Core ML computes in fp16).

Compared with other targets

On iPhone, Core ML is roughly 35× faster than desktop CPU and ~170× faster than the Android CPU build; ship laya_coreml.pte for iOS.

Verification

Device-verified on a physical Android phone (ModernBERT-large int8, XNNPACK CPU): all 4 laya_testcases.json cases reproduced the desktop int8 reference exactly. int8 vs fp32: argmax and probabilities agree to ~1e-3. Use laya_testcases.json (tokenized inputs + reference answers) to verify your own integration.

Device-verified on iPhone 15 (Core ML): all 4 cases give the same decisions as the reference — see the benchmark above.

Reproduce the export

See export/export_laya_pte.py in the laya-for-react-native project: rebuild DecisionModel, load the typed-decisions weights, torch.export → XNNPACK lower → .pte, with torchao Int8WeightOnlyConfig for the int8 build. (torch>=2.11, executorch>=1.5.)

License

Apache-2.0, inherited from the base model convaiinnovations/laya.