LLM-OS-Models/KoHRM-Text-1.4B-GGUF

KoHRM-Text-1.4B-GGUF

GGUF exports for LLM-OS-Models/KoHRM-Text-1.4B.

This is a custom hrm_text architecture. Standard upstream llama.cpp, Ollama, LM Studio, and other GGUF frontends will not load these files until hrm_text support lands upstream. Use the included runtime patch:

runtime/llama.cpp-hrm_text.patch

The patch is based on the HRM-Text GGUF work from sinimiini/HRM-Text-1B-GGUF, adapted for KoHRM-Text-1.4B. The KoHRM conversion infers the physical H/L stack depth from safetensors, because the public config reports num_hidden_layers=32 while the exported tensors are arranged as H: 16 and L: 16.

Files

Build Patched llama.cpp

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout 6a257d44633d4a752183ed778b88d2924d0a6b9d
git apply /path/to/runtime/llama.cpp-hrm_text.patch

cmake -S . -B build-hrm \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLAMA_CURL=OFF \
  -DGGML_NATIVE=OFF

cmake --build build-hrm --target llama-cli llama-quantize llama-completion llama-results -j 8

CPU Run

Download a quantized GGUF file:

huggingface-cli download LLM-OS-Models/KoHRM-Text-1.4B-GGUF \
  KoHRM-Text-1.4B-Q8_0.gguf \
  --local-dir .

Run on CPU:

./build-hrm/bin/llama-cli \
  -m ./KoHRM-Text-1.4B-Q8_0.gguf \
  -ngl 0 \
  -t 4 \
  -c 1024 \
  -n 260 \
  --seed 41 \
  --temp 0.45 \
  --top-p 0.9 \
  --repeat-penalty 1.08 \
  --single-turn \
  --simple-io \
  --no-warmup \
  --display-prompt \
  -p $'해외주식 투자에서 원/달러 환율 변동이 원화 수익률에 미치는 영향과 대응 방안을 간단히 설명해 주세요.'

H/L Cycle Override (수정 실행: H/L 사이클 직접 지정)

KoHRM-Text-GGUF stores recurrence settings as GGUF metadata:

hrm_text.h_cycles = 2
hrm_text.l_cycles = 3

현재 패치된 llama.cpp에서는 실행 시 모델 로딩 단계에서 아래 키를 메타데이터 오버라이드할 수 있습니다.

./build-hrm/bin/llama-cli \
  -m ./KoHRM-Text-1.4B-Q8_0.gguf \
  -ngl 0 \
  -t 4 \
  -c 1024 \
  -n 260 \
  --seed 41 \
  --temp 0.45 \
  --top-p 0.9 \
  --repeat-penalty 1.08 \
  --single-turn \
  --simple-io \
  --no-warmup \
  --display-prompt \
  --override-kv hrm_text.h_cycles=int:1 \
  --override-kv hrm_text.l_cycles=int:2 \
  -p $'해외주식 투자에서 원/달러 환율 변동이 원화 수익률에 미치는 영향과 대응 방안을 간단히 설명해 주세요.'

Use case:

• —h_cycles/l_cycles를 낮추면 동일 조건에서 응답속도는 빨라지는 경향이 있지만 품질 저하가 자주 증가합니다.
• —2/3은 현재 기본값(문서 상 안정 동작)입니다.
• —1/2는 속도 우선 테스트로 추천하며, 실제 정밀 추론에서는 2/3이 더 안정적입니다.

--override-kv uses key format KEY=TYPE:VALUE, same as upstream llama.cpp:

• —hrm_text.h_cycles=int:1
• —hrm_text.l_cycles=int:2

If you need a persistent configuration (e.g., fixed 1/1 for a workload), export a new GGUF after changing H_cycles / L_cycles in the source config before convert_hf_to_gguf.py conversion. That preserves one set of cycles inside the artifact and avoids runtime override overhead.

CPU Generation Tests

Tested locally on CPU with the patched llama.cpp build and the prompt shown above.

This prompt was chosen after checking the KoHRM training-data path. KoHRM uses the HRM V1Dataset instruction-response layout:

<|im_start|><condition_token>instruction<|im_end|>response<|box_end|>

The instruction/prefix span is not trained with loss; the response span is trained with response-only loss. Local decoded samples include short Korean finance QA rows, so the representative GGUF smoke prompt below uses the same plain instruction style instead of a legal reasoning prompt.

Q8_0 output excerpt:

1. **환율 변동의 영향:** 해외 주식 투자의 수익률은 주가 상승에 따른 수익뿐만 아니라 환율 변동에 따른 수익 또는 손실을 포함합니다.
...
2. **대응 방안:**
   - **환율 변동 위험 관리:** 환율 변동 위험을 줄이기 위해 환헤지 상품을 활용하거나, 분할 매수/매도 전략을 통해 환율 변동에 따른 영향을 완화할 수 있습니다.
   - **장기 투자:** 장기 투자를 통해 환율 변동의 단기적인 영향을 완화하고, 장기적인 주가 상승에 집중할 수 있습니다.
   - **분산 투자:** 다양한 국가의 주식에 분산 투자하여 특정 국가의 환율 변동 위험을 줄일 수 있습니다.

The full smoke-test log is in reports/generation_tests/finance_short_q8_0.txt. This is a qualitative CPU runtime sample, not a benchmark or financial advice.

Prompt Format

The source training/inference wrapper is:

<|im_start|><|object_ref_start|>PROMPT<|im_end|>

prepare_sft_data.py writes the generic HRM V1Dataset layout with direct=<|object_ref_start|> by default. In this patched GGUF runtime, llama-completion could load the model but returned an immediate end token for the tested prompts, while llama-cli --single-turn produced visible CPU token generation. The public checkpoint is a rolling pretraining-stage model, not a final chat/SFT model, so instruction following can still be unstable.

GGUF Metadata

Key converted metadata:

general.architecture = hrm_text
hrm_text.context_length = 4096
hrm_text.embedding_length = 1536
hrm_text.block_count = 128
hrm_text.layers_per_stack = 16
hrm_text.h_cycles = 2
hrm_text.l_cycles = 3
tokenizer.ggml.model = gpt2
tokenizer.ggml.pre = qwen2
tokenizer.ggml.bos_token_id = 2
tokenizer.ggml.eos_token_id = 35
tokenizer.ggml.padding_token_id = 0

Notes

• —Source model: LLM-OS-Models/KoHRM-Text-1.4B
• —Source revision converted: c413eee318b28e4f970f1be83698b161e60b3adb
• —llama.cpp base commit used for the patch: 6a257d44633d4a752183ed778b88d2924d0a6b9d
• —BF16 conversion wrote 259 tensors.
• —llama-completion can load the model non-interactively, but in local probes it immediately returned an end token for the tested prompts. llama-cli --single-turn produced visible CPU token generation and is the command shown above.