defog/sqlcoder-7b-2

1---2license: cc-by-sa-4.03library_name: transformers4pipeline_tag: text-generation5---6# Update notice7The model weights were updated at 7 AM UTC on Feb 7, 2024. The new model weights lead to a much more performant model – particularly for joins.8 9If you downloaded the model before that, please redownload the weights for best performance.10 11# Model Card for SQLCoder-7B-212 13A capable large language model for natural language to SQL generation.14 15![image/png](https://cdn-uploads.huggingface.co/production/uploads/603bbad3fd770a9997b57cb6/AYUE2y14vy2XkD9MZpScu.png)16 17## Model Details18 19### Model Description20 21<!-- Provide a longer summary of what this model is. -->22 23This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.24 25- **Developed by:** [Defog, Inc](https://defog.ai)26- **Model type:** [Text to SQL]27- **License:** [CC-by-SA-4.0]28- **Finetuned from model:** [CodeLlama-7B]29 30### Model Sources [optional]31- [**HuggingFace:**](https://huggingface.co/defog/sqlcoder-70b-alpha)32- [**GitHub:**](https://github.com/defog-ai/sqlcoder)33- [**Demo:**](https://defog.ai/sqlcoder-demo/)34 35## Uses36 37This model is intended to be used by non-technical users to understand data inside their SQL databases. It is meant as an analytics tool, and not as a database admin tool.38 39This model has not been trained to reject malicious requests from users with write access to databases, and should only be used by users with read-only access.40 41## How to Get Started with the Model42 43Use the code [here](https://github.com/defog-ai/sqlcoder/blob/main/inference.py) to get started with the model.44 45## Prompt46 47Please use the following prompt for optimal results. Please remember to use `do_sample=False` and `num_beams=4` for optimal results.48 49```50### Task51Generate a SQL query to answer [QUESTION]{user_question}[/QUESTION]52 53### Database Schema54The query will run on a database with the following schema:55{table_metadata_string_DDL_statements}56 57### Answer58Given the database schema, here is the SQL query that [QUESTION]{user_question}[/QUESTION]59[SQL]60```61 62## Evaluation63 64This model was evaluated on [SQL-Eval](https://github.com/defog-ai/sql-eval), a PostgreSQL based evaluation framework developed by Defog for testing and alignment of model capabilities.65 66You can read more about the methodology behind SQLEval [here](https://defog.ai/blog/open-sourcing-sqleval/).67 68### Results69 70We classified each generated question into one of 6 categories. The table displays the percentage of questions answered correctly by each model, broken down by category.71 72|                | date | group_by | order_by | ratio | join | where |73| -------------- | ---- | -------- | -------- | ----- | ---- | ----- |74| sqlcoder-70b   | 96   | 91.4     | 97.1     | 85.7  | 97.1 | 91.4  |75| sqlcoder-7b-2  | 96   | 91.4     | 94.3     | 91.4  | 94.3 | 77.1  |76| sqlcoder-34b   | 80   | 94.3     | 85.7     | 77.1  | 85.7 | 80    |77| gpt-4          | 72   | 94.3     | 97.1     | 80    | 91.4 | 80    |78| gpt-4-turbo    | 76   | 91.4     | 91.4     | 62.8  | 88.6 | 77.1  |79| natural-sql-7b | 56   | 88.6     | 85.7     | 60    | 88.6 | 80    |80| sqlcoder-7b    | 64   | 82.9     | 74.3     | 54.3  | 74.3 | 74.3  |81| gpt-3.5        | 72   | 77.1     | 82.8     | 34.3  | 65.7 | 71.4  |82| claude-2       | 52   | 71.4     | 74.3     | 57.1  | 65.7 | 62.9  |83 84## Model Card Contact85 86Contact us on X at [@defogdata](https://twitter.com/defogdata), or on email at [founders@defog.ai](mailto:founders@defog.ai)