ykarout/Llama-3.3-70b-instruct-deepseek-distilled-GGUF

Uploaded model

• —Developed by: ykarout
• —License: apache-2.0
• —Finetuned from model : unsloth/Llama-3.3-70B-Instruct-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

To use with Ollama, create the Modelfile as per the below template:

FROM {_FILELOCATION__} (replace with your gguf file location)

TEMPLATE """{{ if .Messages }} {{- if or .System .Tools }}<|startheaderid|>system<|endheaderid|> {{- if .System }}

{{ .System }} {{- end }} {{- if .Tools }}

You are a helpful assistant with tool calling capabilities. When you receive a tool call response, use the output to format an answer to the original use question. {{- end }} {{- end }}<|eotid|> {{- range $i, $ := .Messages }} {{- $last := eq (len (slice $.Messages $i)) 1 }} {{- if eq .Role "user" }}<|startheaderid|>user<|endheaderid|> {{- if and $.Tools $last }}

Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.

Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.

{{ $.Tools }} {{- end }}

{{ .Content }}<|eotid|>{{ if $last }}<|startheaderid|>assistant<|endheader_id|>

{{ end }} {{- else if eq .Role "assistant" }}<|startheaderid|>assistant<|endheaderid|> {{- if .ToolCalls }}

{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}{{ end }} {{- else }}

{{ .Content }}{{ if not $last }}<|eotid|>{{ end }} {{- end }} {{- else if eq .Role "tool" }}<|startheaderid|>ipython<|endheader_id|>

{{ .Content }}<|eotid|>{{ if $last }}<|startheaderid|>assistant<|endheader_id|>

{{ end }} {{- end }} {{- end }} {{- else }} {{- if .System }}<|startheaderid|>system<|endheaderid|>

{{ .System }}<|eotid|>{{ end }}{{ if .Prompt }}<|startheaderid|>user<|endheader_id|>

{{ .Prompt }}<|eotid|>{{ end }}<|startheaderid|>assistant<|endheader_id|>

{{ end }}{{ .Response }}{{ if .Response }}<|eot_id|>{{ end }}"""

PARAMETER stop "<|startheaderid|>"

PARAMETER stop "<|endheaderid|>"

PARAMETER stop "<|eot_id|>"

PARAMETER stop "<|eom_id|>"

PARAMETER temperature 1.5 #can be changed for experimenting different generations

PARAMETER min_p 0.1 #can be changed for experimenting different generations

PARAMETER repeat_penalty 1.15 #can be changed for experimenting different generations

PARAMETER top_p 0.9 #can be changed for experimenting different generations

PARAMETER top_k 45 #can be changed for experimenting different generations