CoolFace
Modelpublic

second-state/SmolLM3-3B-GGUF

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
0likes552downloads
Model Card

<!-- header start --> <!-- 200823 --> <div style="width: auto; margin-left: auto; margin-right: auto"> <img src="https://github.com/LlamaEdge/LlamaEdge/raw/dev/assets/logo.svg" style="width: 100%; min-width: 400px; display: block; margin: auto;"> </div> <hr style="margin-top: 1.0em; margin-bottom: 1.0em;"> <!-- header end -->

SmolLM3-3B-GGUF

Original Model

HuggingFaceTB/SmolLM3-3B

Run with LlamaEdge

  • LlamaEdge version: v0.24.0 and above
  • Prompt template
  • Normal chat
  • Prompt type: smol3-no-think
  • Prompt string
text
      <|im_start|>system
      ## Metadata

      Knowledge Cutoff Date: June 2025
      Today Date: 15 July 2025
      Reasoning Mode: /no_think
      
      ## Custom Instructions
      
      You are a helpful AI assistant named SmolLM, trained by Hugging Face.
      <|im_end|>
      <|im_start|>user
      {user_message_1}
      <|im_end|>
      <|im_start|>assistant
      {assistant_message_1}
      <|im_end|>
      <|im_start|>user
      {user_message_2}
      <|im_end|>
      <|im_start|>assistant
  • Chat with Tools
  • Prompt type: smol3-no-think
  • Prompt string
text
      <|im_start|>system
      ## Metadata

      Knowledge Cutoff Date: June 2025
      Today Date: 15 July 2025
      Reasoning Mode: /no_think
      
      ## Custom Instructions
      
      You are a helpful AI assistant named SmolLM, trained by Hugging Face.
      
      ### Tools
      
      You may call one or more functions to assist with the user query.
      You are provided with function signatures within <tools></tools> XML tags:
      
      <tools>
      {"name":"sum","description":"Calculate the sum of two numbers","parameters":{"$schema":"http://json-schema.org/draft-07/schema#","properties":{"a":{"description":"The left hand side number","format":"int32","type":"integer"},"b":{"description":"The right hand side number","format":"int32","type":"integer"}},"required":["a","b"],"title":"SumRequest","type":"object"}}
      {"name":"sub","description":"Calculate the difference of two numbers","parameters":{"$schema":"http://json-schema.org/draft-07/schema#","properties":{"a":{"description":"The left hand side number","format":"int32","type":"integer"},"b":{"description":"The right hand side number","format":"int32","type":"integer"}},"required":["a","b"],"title":"SubRequest","type":"object"}}
      {"name":"get_current_weather","description":"Get the weather for a given city","parameters":{"$schema":"http://json-schema.org/draft-07/schema#","definitions":{"TemperatureUnit":{"enum":["celsius","fahrenheit"],"type":"string"}},"properties":{"location":{"description":"the city to get the weather for, e.g., 'Beijing', 'New York', 'Tokyo'","type":"string"},"unit":{"$ref":"#/definitions/TemperatureUnit","description":"the unit to use for the temperature, e.g., 'celsius', 'fahrenheit'"}},"required":["location","unit"],"title":"GetWeatherRequest","type":"object"}}
      </tools>
      
      For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
      <tool_call>
      {"name": <function-name>, "arguments": <args-json-object>}
      </tool_call>

      <|im_end|>
      <|im_start|>user
      {user_message_1}
      <|im_end|>
      <|im_start|>assistant
      {assistant_message_1}
      <|im_end|>
      <|im_start|>user
      {user_message_2}
      <|im_end|>
      <|im_start|>assistant

<!-- - think mode

  • Prompt type: smol3-think
  • Prompt string
text
      <|im_start|>system
      /think

      ## Custom Instructions
      {system_message}
      <|im_end|>
      <|im_start|>user
      {user_message_1}
      <|im_end|>
      <|im_start|>assistant
      <think>
      {think content}
      </think>
      {assistant_message_1}
      <|im_end|>
      <|im_start|>user
      {user_message_2}
      <|im_end|>
      <|im_start|>assistant
      <think>
      {think content}
      </think>
      {assistant_message_2}
      <|im_end|> -->
  • Context size: 128000
  • Run as LlamaEdge service
bash
  wasmedge --dir .:. --nn-preload default:GGML:AUTO:SmolLM3-3B-Q5_K_M.gguf \
    llama-api-server.wasm \
    --model-name SmolLM3-3B \
    --prompt-template smol3-no-think \
    --ctx-size 128000
  • Run as LlamaEdge command app
bash
  wasmedge --dir .:. --nn-preload default:GGML:AUTO:SmolLM3-3B-Q5_K_M.gguf \
    llama-chat.wasm \
    --prompt-template smol3-no-think \
    --ctx-size 128000

Quantized GGUF Models

NameQuant methodBitsSizeUse case
SmolLM3-3B-Q2_K.ggufQ2_K21.25 GBsmallest, significant quality loss - not recommended for most purposes
SmolLM3-3B-Q3_K_L.ggufQ3KL31.69 GBsmall, substantial quality loss
SmolLM3-3B-Q3_K_M.ggufQ3KM31.57 GBvery small, high quality loss
SmolLM3-3B-Q3_K_S.ggufQ3KS31.43 GBvery small, high quality loss
SmolLM3-3B-Q4_0.ggufQ4_041.81 GBlegacy; small, very high quality loss - prefer using Q3KM
SmolLM3-3B-Q4_K_M.ggufQ4KM41.92 GBmedium, balanced quality - recommended
SmolLM3-3B-Q4_K_S.ggufQ4KS41.82 GBsmall, greater quality loss
SmolLM3-3B-Q5_0.ggufQ5_052.16 GBlegacy; medium, balanced quality - prefer using Q4KM
SmolLM3-3B-Q5_K_M.ggufQ5KM52.21 GBlarge, very low quality loss - recommended
SmolLM3-3B-Q5_K_S.ggufQ5KS52.16 GBlarge, low quality loss - recommended
SmolLM3-3B-Q6_K.ggufQ6_K62.53 GBvery large, extremely low quality loss
SmolLM3-3B-Q8_0.ggufQ8_083.28 GBvery large, extremely low quality loss - not recommended
SmolLM3-3B-f16.gguff16166.16 GB

Quantized with llama.cpp b5889