Yobitel/meta-llama-llama-3-1-8b-instruct__llm-inference-chatbot-short__019e3b2fef77

meta-llama/Llama-3.1-8B-Instruct on llm.inference.chatbot-short (NVIDIA H100 80GB HBM3) Back to leaderboard Headline metrics Metric Value Unit TTFT P50 25.3868 ms TTFT P99 1677.528 ms TPOT P50 6.5951 ms TPOT P99 10.8911 ms Total P50 Ms 862.5176 Total P99 Ms 3020.5284 Req Per S Passing 1.8011 Req Per S All 1.8497 Compliance Rate 0.9737 Ok Rate 1 Throughput Tok Per S 227.5172 Cost Usd Per Million Tokens 0.0575 Cost Source… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/meta-llama-llama-3-1-8b-instruct__llm-inference-chatbot-short__019e3b2fef77.