CoolFace
Modelpublic

pearl-ai/Llama-3.3-70B-Instruct-pearl

sourceHugging Facellama3.3updated 5mo agoView on Hugging Face
3likes1.1kdownloads
Model Card

pearl-ai/Llama-3.3-70B-Instruct-pearl

Pearl-certified variant of Llama-3.3-70B-Instruct, intended to run with the Pearl vLLM mining plugin.

Launch Benchmark

Original (Meta's) llama-3.3-70B-Instruct vs. our "two-for-one" Pearl-certified variant. Both executions were done with 4xH200 GPUs. We explore several parallelism techniques. TMADs, i.e., Tera MADs, is a metric counting number of Multiply-Add (MAD) operations. Useful MADs is the total number of MAD operations done anyway that are used for mining.

<table> <thead> <tr> <th>Model</th> <th>Parallelism</th> <th>Score (MMLU)</th> <th>Throughput (tok/sec)</th> <th>Time (sec)</th> <th>Useful MADs (TMADs/sec)</th> </tr> </thead> <tbody> <tr> <td>Meta's LLaMA 70B</td> <td>PP=4</td> <td>0.8198</td> <td>15,269.81</td> <td>441.100</td> <td>-</td> </tr> <tr> <td>Meta's LLaMA 70B</td> <td>TP=4</td> <td>0.8193</td> <td>13,218</td> <td>510</td> <td>-</td> </tr> <tr> <td>Meta's LLaMA 70B</td> <td>DP=2, TP=2</td> <td>0.8197</td> <td>13,162</td> <td>512</td> <td>-</td> </tr> <tr> <td colspan="6"><strong>Meta's LLaMA 70B (DP=4): OOM - bf16 model (~140 GB) exceeds single GPU VRAM</strong></td> </tr> <tr> <td>Pearl-certified</td> <td>PP=4</td> <td>0.8190</td> <td>17,206.26</td> <td>391.457</td> <td>806</td> </tr> <tr> <td>Pearl-certified</td> <td>TP=4</td> <td>0.8180</td> <td>13,264.38</td> <td>507.789</td> <td>620</td> </tr> <tr> <td>Pearl-certified</td> <td>DP=4</td> <td>0.8198</td> <td>18,291.66</td> <td>368.229</td> <td>981</td> </tr> </tbody> </table>

How To Use (Pearl vLLM Plugin)

This model is intended to be served through the Pearl miner stack, where vLLM inference is integrated with Pearl mining workflows.

Typical flow:

  1. 1.Run pearld with RPC enabled.
  2. 2.Start the Pearl miner/vLLM stack.
  3. 3.Serve this model through vLLM while Pearl gateway/miner components handle mining-side integration.

High-level prerequisites:

  • —Python 3.12
  • —uv
  • —CUDA + NVIDIA GPU (sm90 class, e.g. H100/H200, per project docs)
  • —Rust toolchain
  • —Running pearld node with RPC credentials

Docker Example

From the Pearl repository root:

bash
docker buildx build -t vllm_miner . -f miner/vllm-miner/Dockerfile
bash
docker run --rm -it --gpus all \
  -p 8000:8000 -p 8337:8337 -p 8339:8339 \
  -e PEARLD_RPC_URL=<PEARLD_URL> \
  -e PEARLD_RPC_USER=<RPC_USER> \
  -e PEARLD_RPC_PASSWORD=<RPC_PASSWORD> \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --shm-size 8g \
  vllm_miner:latest \
  pearl-ai/Llama-3.3-70B-Instruct-pearl \
  --host 0.0.0.0 --port 8000 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.9 \
  --enforce-eager