Modelpublic

ByteForge/Defog_llama-3-sqlcoder-8b-ct2-int8_float16

sourceHugging Facecc-by-sa-4.0updated 2y agoView on Hugging Face

1likes16downloads

Model Card

A capable language model for text to SQL generation for Postgres, Redshift and Snowflake that is on-par with the most capable generalist frontier models.

image/png

Model Description

Developed by: Defog, Inc Model type: [Text to SQL] License: [CC-by-SA-4.0] Finetuned from model: [Meta-Llama-3-8B-Instruct]

defog/llama-3-sqlcoder-8b for CTranslate2

The model is quantized version of the [defog/llama-3-sqlcoder-8b](https://huggingface.co/defog/llama-3-sqlcoder-8b) with int8_float16 quantization and can be used in [CTranslate2](https://github.com/OpenNMT/CTranslate2).

How to use

pip install ctranslate2```


This repository for use with [CTranslate2](https://github.com/OpenNMT/CTranslate2).

### Use with CTranslate2

This example code is obtained from [CTranslate2_transformers](https://opennmt.net/CTranslate2/guides/transformers.html#mpt) and [tokenizer AutoTokenizer](https://huggingface.co/docs/transformers/main_classes/tokenizer).  
More detailed information about the `generate_batch` methon can be found at [CTranslate2_Generator.generate_batch](https://opennmt.net/CTranslate2/python/ctranslate2.Generator.html#ctranslate2.Generator.generate_batch).

import ctranslate2 import transformers

from huggingfacehub import snapshotdownload modelid = "ByteForge/Defogllama-3-sqlcoder-8b-ct2-int8float16" modelpath = snapshotdownload(modelid) model = ctranslate2.Generator(modelpath) tokenizer = transformers.AutoTokenizer.frompretrained(model_id)

prompt=""" CREATE TABLE stadium ( stadium_id number, location text, name text, capacity number, highest number, lowest number, average number )

CREATE TABLE singer ( singerid number, name text, country text, songname text, songreleaseyear text, age number, is_male others )

CREATE TABLE concert ( concertid number, concertname text, theme text, stadium_id text, year text )

CREATE TABLE singerinconcert ( concertid number, singerid text )

-- Using valid SQLite, answer the following questions for the tables provided above.

-- What is the maximum, the average, and the minimum capacity of stadiums ? (Generate 1 Sql query. No explaination needed)

answer: """

messages = [ {"role": "system", "content": "You are SQL Expert. Given a input question and schema, answer with correct sql query"}, {"role": "user", "content": prompt}, ]

inputids = tokenizer.applychattemplate( messages, tokenize=False, addgeneration_prompt=True )

terminators = [ tokenizer.eostokenid, tokenizer.converttokenstoids("<|eotid|>") ]

inputtokens = tokenizer.convertidstotokens(tokenizer.encode(input_ids))

results = model.generatebatch([inputtokens], includepromptinresult=False, maxlength=256, samplingtemperature=0.6, samplingtopp=0.9, endtoken=terminators) output = tokenizer.decode(results[0].sequencesids[0])

print(output)


## Ideal prompt and inference parameters
Set temperature to 0, and do not do sampling.

## Evaluation
This model was evaluated on SQL-Eval, a PostgreSQL based evaluation framework developed by Defog for testing and alignment of model capabilities.

You can read more about the methodology behind SQLEval [here](https://defog.ai/blog/open-sourcing-sqleval/).