UdgamLabs/hinglish-commands-0.6b

Udgam Hinglish Commands 0.6B — Experimental

[image]

Turn a short mixed Hindi-English or English request into a structured command that an application can inspect. This small specialist adapter identifies the intent and arguments while preserving the user's wording. For example, it can identify an event request and its time/category fields. It returns hierarchical TOP intent/slot text; the accompanying code converts it to ordered nested JSON. It executes no actions.

The measured scope is Romanized Hinglish and paired English in eight domains: alarm, event, messaging, music, navigation, reminder, timer and weather. The hi metadata tag does not imply a validated Devanagari-Hindi model. This is an adaptation of Qwen3-0.6B, not a newly pretrained foundation model or a general Indian-language assistant.

Measured improvement

[image]

On the original 13,026-row test, complete-tree accuracy improved by 44.98 percentage points, with a paired 95% confidence interval of [43.93, 46.03] points. That test consists of 6,513 paired English/Hinglish entries, grouped into 6,302 normalized English-source clusters for 2,000 bootstrap resamples (seed 42). The repeated-input sensitivity view contains 12,692 rows: 54.17% for Udgam versus 9.59% for Qwen.

All eight benchmark domains improved over the prompted base. Nevertheless, 5,877 full interpretations were still incorrect, and Hinglish complete-tree accuracy is below 50%. A developer should treat this as an experimental parsing component and validate proposed interpretations before connecting them to actions.

Complete-tree exact match requires every label, value, nesting relationship and child order to match the reference after whitespace-only canonicalization. The high 99.96% syntax-valid rate is a formatting diagnostic, not command accuracy. No language model served as the primary test judge.

Recorded examples

These are saved final-test outputs, not a live demo or representative sample. Text and prediction fields are unchanged; the stored terminal token is omitted from the display. References are inherited dataset annotations.

An improvement

Recorded test ID: htop-test-000288-hinglish. Input:

Kya is month koi beer festival he

Original Qwen output:

[IN:GET_EVENT Kya is month koi beer festival he]

Udgam output:

[IN:GET_EVENT Kya [SL:DATE_TIME is month ] koi [SL:CATEGORY_EVENT beer festival ] he ]

Unchanged benchmark reference:

[IN:GET_EVENT Kya [SL:DATE_TIME is month ] koi [SL:CATEGORY_EVENT beer festival ] he ]

The adapter extracts the requested time and event category and matches the inherited reference. It does not actually search for events.

A regression

Recorded test ID: htop-test-005035-hinglish. Input:

Oahu ke island me events he

Original Qwen output:

[IN:GET_EVENT [SL:LOCATION Oahu ] ke island me events he]

Udgam output:

[IN:GET_EVENT [SL:LOCATION Oahu ] ke [SL:LOCATION island ] me events he ]

Unchanged benchmark reference:

[IN:GET_EVENT [SL:LOCATION Oahu ] ke island me events he ]

The original model matches the reference, while Udgam incorrectly adds a location for island. This output still passes syntax and source-copy checks.

Use the complete adapter package

The repository root is an integration package containing the adapter in model/, its tokenizer, retrieval_train.jsonl, schema, configuration and integrity manifest. It requires the separately downloaded, pinned Qwen base. Loading only the weights with an arbitrary chat prompt does not reproduce the evaluated interface.

Install the wrapper from a checked-out UdgamLabs/hinglish-commands-0.6b source repository. No PyPI package release is assumed:

python -m pip install -e '.[inference]'

With that environment active and the model repository downloaded locally:

from udgam_hinglish import CommandParser

parser = CommandParser.from_local("/path/to/hinglish-commands-0.6b")
result = parser.parse("Kya is month koi beer festival he")
print(result)

The text above is also a recorded example; a fresh run can differ on other hardware or settings. Local loading expects the pinned base in the cache. For an explicit first download, use the Hub helper with the verified complete adapter commit:

from udgam_hinglish import CommandParser

parser = CommandParser.from_hub(
    "UdgamLabs/hinglish-commands-0.6b",
    revision="4b45809cc31814a2dd4f39f388e4b74a428e4ca2",
    allow_base_download=True,
)

The revision above is a complete, verified adapter snapshot. Code lives at UdgamLabs/hinglish-commands-0.6b on GitHub and the adapter at UdgamLabs/hinglish-commands-0.6b on Hugging Face. Both repositories are publicly available; no Hugging Face sign-in is required to download this adapter. Explore the interactive walkthrough for recorded examples in plain language and JSON. The helper's downloadable package uses the same pinned base and training-only retrieval configuration. Dependencies are recorded; a new environment is not implied to be tested just by listing an installation command.

Training and selection

• —Base: Qwen/Qwen3-0.6B, revision c1899de289a04d12100db370d81485cdf75e47ca.
• —Dataset: Google Hinglish-TOP revision fdd3998a6573130659bfa1ce4b1ebe698df2bf3a, with original TOPv2 lineage.
• —Final training data: 2,310 paired source entries / 4,620 language rows; validation: 1,060 pairs / 2,120 rows. Original test: 6,513 pairs / 13,026 rows.
• —Selected adapter: human-only, seed 20260912, completed epoch 2; fingerprint 85be8189b09cefa38dc7e50723042d606af19d9f2ad8ceab1cdb123f5eec65af.
• —LoRA rank 16, alpha 32, dropout 0.05; learning rate 1e-4; microbatch 8, accumulation 4; BF16; answer/EOS loss only; frozen original base.
• —Selected checkpoint exposures: 9,240 human examples, zero existing-synthetic examples, 312,810 supervised tokens. Repeated exposures are not additional unique records.
• —Up to six epochs were allowed with a cooperative 90-minute stopping setting, not a hard deadline. Actual elapsed time including validation was about 101.7 minutes. Only completed epochs were eligible; the run did not complete six epochs.

The best human-only development checkpoint scored 51.56%. One existing-synthetic variant scored 48.44%, increased malformed/failed outputs from two to 16 and violated domain-regression guards, so it was rejected. A second human-only seed scored 50.90% and was not eligible to replace seed 1; timer performance fell 3.68 points. The selected adapter therefore contains no synthetic augmentation, and seed 2 has no final-test score.

The original Qwen and KingNish checkpoints were tested with both zero and four retrieved examples on validation data. The stronger four-example prompts were retained. Final Qwen-versus-Udgam used the same task/schema prompt and pinned training retrieval, non-thinking greedy generation, BF16, batch 16, input limit 4,096 and output limit 768 tokens. Prompt, checkpoint and recipe were frozen before the final test; there was no tuning in response to final-test results.

Completed package checks

The final historical load check completed for the source adapter, packaged adapter and an internal merged diagnostic. It produced 144 generations on a fixed 16-input development sample, with no generation failures; each form repeated and reloaded consistently. Packaged adapter versus source agreed on 16/16 outputs and canonical trees.

The merged diagnostic agreed on 14/16, leaving a disclosed review_required outcome. Its full-test accuracy is unknown, and merged weights are excluded from this release. The benchmark belongs to the frozen source adapter. The small package check and any later wrapper tests are not a new full-model evaluation or a promise of bit-identical outputs across devices.

Training and final evaluation used A40/BF16. Local adapter verification used MPS/BF16. No CPU latency, minimum RAM, low-end-phone speed or mobile-export claim is made here. A measured HTTP latency for an internal merged model cannot be transferred to this adapter. Downloadable weights do not include an always-on hosted inference service.

The September 15 release integration passed 35 offline SDK/source-restoration tests. A real private-Hub download through the SDK materialized the complete package, passed its integrity checks, loaded it on the Mac GPU (MPS) in BF16 using the already-cached pinned base, and reproduced the original CPU output on one timer command. The source package installed and its command-line entrypoint ran. This is a packaging smoke test, not a repeated final benchmark or fresh cross-platform dependency-install test.

Limitations and intended use

Intended users are developers and researchers testing a small local command parser. The interface preserves repeated slots and nesting, rejects detected malformed/truncated outputs and exposes a preview. Dates, time zones, recipients and missing details require application-side resolution and confirmation. A syntactically valid tree may still express the wrong request. Unsupported commands may be mapped to an in-scope intent; validation is not a reliable out-of-domain detector.

The dataset includes international names and places and is not a representative sample of everyday India. No claim is made for native-script Hindi, other Indian languages, speech, unrestricted chat, current factual knowledge, real-user preference or nationwide usefulness. Existing human labels can be imperfect. Detected train/validation overlaps were removed and the test preserved, but public benchmark contamination in the base model's pretraining is unknown. Exact-match cleaning does not remove all semantic paraphrases.

Evaluation resumed verified saved prefixes after an interruption on another A40 host. The pinned package versions and model caches matched; the Linux kernel and NVIDIA driver changed. This is a disclosed host migration, not uninterrupted bit-identical execution.

Licence and credits

The adapter contribution and prepared/retrieval data terms are CC BY-SA 4.0, preserving the underlying TOPv2 notice and Google's Apache notice. Original Qwen materials retain Apache 2.0; Udgam-written code is released under Apache 2.0. The complete package is not Apache-only. The adapter terms are a conservative distribution choice, not a blanket legal conclusion about AI training. Creative Commons guidance.

Base model: Qwen/Qwen3-0.6B, revision c1899de289a04d12100db370d81485cdf75e47ca. Dataset: Google Research Hinglish-TOP, revision fdd3998a6573130659bfa1ce4b1ebe698df2bf3a, building on TOPv2. The original authors have not endorsed this adaptation.

Udgam contributed the curation, supplementary adaptation, controlled evaluation, integration and documentation. The original model and dataset creators retain credit for their work.

Details: benchmarks and recorded failures, data and training, licences and attribution. Published by Udgam Labs on September 22, 2026.