Modelpublic

rungalileo/mistral-7b-instruct-v0.3-trtllm-ckpt-bf16

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes12downloads

Model Card

Mistral-7B-Instruct-v0.3 TensorRT-LLM checkpoint (BF16)

TensorRT-LLM checkpoint generated from mistralai/Mistral-7B-Instruct-v0.3.

Build

Item	Value
Base model	`mistralai/Mistral-7B-Instruct-v0.3`
Source model directory	`/home/langzhao/workspace/company/ml-deploy/models/quant/calib-sweep/_base_cache/mistral-7b-instruct-v0.3`
TensorRT-LLM script	`/home/langzhao/workspace/company/trt-opt-test/TensorRT-LLM-1.1.0/examples/models/core/llama/convert_checkpoint.py`
TensorRT-LLM git describe	`v1.1.0`
TensorRT-LLM git commit	`48b7b5d8b7ed1e6bff57aecc6ff7d1533288bed8`
Dtype	`bfloat16`
Quantization	`none`
Target HF repo	`rungalileo/mistral-7b-instruct-v0.3-trtllm-ckpt-bf16`

Command

bash

/home/langzhao/workspace/company/trt-opt-test/venv_trtllm1.1.0/bin/python /home/langzhao/workspace/company/trt-opt-test/TensorRT-LLM-1.1.0/examples/models/core/llama/convert_checkpoint.py --model_dir /home/langzhao/workspace/company/ml-deploy/models/quant/calib-sweep/_base_cache/mistral-7b-instruct-v0.3 --output_dir /home/langzhao/workspace/company/ml-deploy/models/checkpoints/mistral-7b-instruct-v0.3-trtllm-ckpt-bf16 --dtype bfloat16

Tool Versions

Tool	Version
`huggingface_hub`	`0.36.2`
`modelopt`	`0.37.0`
`mpi4py`	`4.1.1`
`platform`	`Linux-6.1.0-45-cloud-amd64-x86_64-with-glibc2.36`
`python`	`3.12.9 (main, Apr 28 2026, 03:56:56) [GCC 12.2.0]`
`safetensors`	`0.7.0`
`tensorrt`	`10.13.3.9.post1`
`tensorrt_llm`	`1.1.0`
`torch`	`2.9.0+cu128`
`transformers`	`4.56.0`

Files

File	Size bytes
`config.json`	2261
`rank0.safetensors`	14529628920

build_manifest.json contains the same build evidence in machine-readable form.