CoolFace
Apppublic

osolmaz/ling-dgx-spark

sourceHugging Facemitupdated 7d agoView on Hugging Face
0likes
App README

Ling 3.0 Flash on DGX Spark

This repo has the files I used to run Ling 3.0 Flash locally on a DGX Spark and connect it to OpenClaw.

The example is a focused four-document invoice task with two text PDFs, one raster receipt, and one larger three-page raster document. The assistant reads synthetic German invoices and bank records, then prepares a periodic VAT draft from the source evidence. The documents and model requests stay on the machine.

Included files

  • —Launch settings for Ling 3.0 Flash FP4 with the Ling DSpark draft model.
  • —Launch settings for Ling 3.0 Flash VL FP4, which reads scanned documents directly.
  • —oomwrap protection for both model launches, with the same memory and swap floors used locally.
  • —An OpenClaw config with local-model lean mode, bounded concurrency, and a 4096-token compaction cap. Tools run in a sandboxed workspace without network access.
  • —A one-shot OpenClaw vision tool that sends exactly one image and one prompt to Ling, saves one text result, and aborts the exact SGLang request on timeout.
  • —The German VAT skill and a link to the synthetic invoice folder.
  • —Checks that send a real text or image request to the local endpoint.
  • —A static overview page with the project credits and the pinned versions.
  • —The exact model revisions and container digests used for the recorded tests, with the measured memory and context limits.

Memory limit

Run only one Ling model at a time. The tested text setup used about 92.5 GiB of resident memory. The current VL setup uses an 81920-token pool shared by at most two running requests. It does not provide two simultaneous full-length 81920-token sequences. The text and VL models do not fit together on a DGX Spark with 121 GiB of unified memory.

The VL launcher allows one image per request, reserves 72% of memory for static allocations, and uses BF16 KV cache. The main OpenClaw route is text-only; image reading goes through the stateless ling_vision_once tool. DSpark is enabled only for the text model.

Quick start

Requirements:

  • —NVIDIA DGX Spark with its supported NVIDIA container stack.
  • —Linux on Arm64.
  • —Docker with NVIDIA GPU access.
  • —Python 3.10 or later.
  • —The current hf CLI.
  • —OpenClaw if you want to run the agent demo.
  • —Rust and Cargo to install oomwrap.
  • —A running earlyoom process for machine-wide memory protection.

Clone this Space:

bash
git clone https://huggingface.co/spaces/osolmaz/ling-dgx-spark
cd ling-dgx-spark

Install the process-scoped memory guard:

bash
cargo install --git https://github.com/osolmaz/oomwrap --locked

Download the model you want to run. Each download is pinned to a model revision:

bash
bash runtime/download-models.sh vl
# or
bash runtime/download-models.sh text

Start the VL endpoint for the invoice demo:

bash
bash runtime/serve-vl.sh

In another terminal, verify that an image reaches the model:

bash
python3 runtime/check-endpoint.py vl

Stop the foreground server with Ctrl+C before you start the text model:

bash
bash runtime/serve-text-dspark.sh
python3 runtime/check-endpoint.py text

See the full setup guide before you connect OpenClaw. It explains the isolated profile, the one-shot vision plugin, and the demo workspace.

Repository map

PathPurpose
`runtime/`Pinned model downloads, demo-input preparation, oomwrap launch scripts, and checks for the host and endpoint
`openclaw/`Local provider, sandbox configuration, and one-shot vision plugin
`workspace/`Public demo workspace and reusable VAT skill
`index.html`Static overview page for the Space, with style.css and the brand marks in assets/
`docs/`Setup and troubleshooting guides with architecture and version notes

Demo data

The demo uses `osolmaz/invoice-fixture` at revision `578d05a`.

The documents are synthetic. The agent prepares files for review and does not submit anything to ELSTER.

License

MIT