summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64

Long-Context VRAM Benchmark

Qwen3.8-27B vs SpeedX27-VL-GDN64

We benchmarked long-context VRAM usage of:

• —Qwen/Qwen3.8-27B
• —summerMC/Qwen3.8-27B-SpeedX27-VL-GDN64

under the same conditions on an ~80 GiB GPU.

Configuration

Context limit: 262,144 tokens
Chunk size:    8,192 tokens
Batch size:    1
Precision:     BF16
Cache:         enabled

VRAM scaling

[PASTE BENCHMARK IMAGE HERE]

The difference is clear:

• —Qwen/Qwen3.8-27B shows increasing VRAM usage as context grows.
• —SpeedX27-VL-GDN64 stays nearly flat at approximately 55.36 GiB across the full context range.

Results

The standard model successfully processed 196,608 tokens, but failed while processing the next 8,192-token chunk.

The GDN64 model successfully completed the full 262,144-token context.

After the 262K prefill, a one-token decode also succeeded:

Processed context: 262,144 tokens
Maximum chunk peak: 55.36 GiB
Decode success: True
Decode time: 0.123 s

Memory behavior

At 196,608 tokens:

Qwen3.8-27B:     ~72.82 GiB peak
SpeedX27-GDN64:  ~55.36 GiB peak

That is approximately 17.46 GiB less peak VRAM for SpeedX27-GDN64 at the same context length.

More importantly, the scaling behavior is fundamentally different.

Qwen3.8-27B:
VRAM increases with context length.
SpeedX27-GDN64:
VRAM remains nearly constant as context length increases.

This benchmark is consistent with the recurrent GDN64 state behaving close to constant-memory with respect to context length, while the standard hybrid architecture retains context-dependent cache growth.

262K result

On the tested ~80 GiB GPU:

Qwen/Qwen3.8-27B
196K  -> Success
204K  -> OOM
SpeedX27-VL-GDN64
262K  -> Success

The result is not caused by smaller model weights: the GDN64 model actually uses slightly more VRAM immediately after loading.

The advantage appears during long-context inference, where its persistent context-state memory remains nearly flat.

SpeedX27 1T Logical Context

This revision introduces an experimental 1,000,000,000,000-token logical context architecture.

Context specification

The 1T value describes a logical recurrent/streaming context space.

It does not claim that the current weights were trained using a single one-trillion-token dense sequence.

Architecture

Input stream
     |
     v
L0 Active Tokens
     |
     v
L1 GDN64 recurrent state
     |
     v
L2 compressed chunk memory
     |
     v
L3 semantic retrieval memory
     |
     v
L4 persistent archive

Long-context design

• —64-layer Gated DeltaNet recurrent architecture
• —bounded-memory streaming prefill
• —64-bit absolute token positions
• —recurrent-state checkpointing
• —resumable context processing
• —hierarchical-memory-ready runtime
• —semantic/external retrieval ready
• —logical context target: 1 trillion tokens

Validation progression

262K
  |
  v
1M
  |
  v
16M
  |
  v
256M
  |
  v
1B
  |
  v
16B
  |
  v
1T

A 1T native-context capability should only be claimed after end-to-end training, calibration and validation at that scale.