| --- |
| license: cc-by-sa-4.0 |
| metrics: |
| - accuracy |
| pipeline_tag: text-generation |
| tags: |
| - code |
| --- |
| # Exllama v2 defog/llama-3-sqlcoder-8b |
|
|
| Using <a href="https://github.com/turboderp/exllamav2/releases/tag/v0.0.21">turboderp's ExLlamaV2 v0.0.21</a> for quantization. |
|
|
| <b>The "main" branch only contains the measurement.json, download one of the other branches for the model</b> |
|
|
| Each branch contains an individual bits per weight, with the main one containing only the meaurement.json for further conversions. |
|
|
| Original model: <a href="https://huggingface.co/defog/llama-3-sqlcoder-8b">defog/llama-3-sqlcoder-8b</a><br> |
| Calibration dataset is based on: <a href="https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction">m-a-p/CodeFeedback-Filtered-Instruction</a> |
|
|
| ## Available sizes |
|
|
| | Branch | Bits | lm_head bits | Description | |
| | ----- | ---- | ------- | ------------ | |
| | [8_0](https://huggingface.co/Apel-sin/llama-3-sqlcoder-8B-exl2/tree/8_0) | 8.0 | 8.0 | Maximum quality that ExLlamaV2 can produce, near unquantized performance. | |
| | [6_5](https://huggingface.co/Apel-sin/llama-3-sqlcoder-8B-exl2/tree/6_5) | 6.5 | 8.0 | Very similar to 8.0, good tradeoff of size vs performance, **recommended**. | |
| | [5_5](https://huggingface.co/Apel-sin/llama-3-sqlcoder-8B-exl2/tree/5_5) | 5.5 | 8.0 | Slightly lower quality vs 6.5, but usable on 8GB cards. | |
| |
| A capable language model for text to SQL generation for Postgres, Redshift and Snowflake that is on-par with the most capable generalist frontier models. |
| |
|  |
| |
| ## Model Description |
| |
| Developed by: Defog, Inc |
| Model type: [Text to SQL] |
| License: [CC-by-SA-4.0] |
| Finetuned from model: [Meta-Llama-3-8B-Instruct] |
| |
| ## Demo Page |
| [https://defog.ai/sqlcoder-demo/](https://defog.ai/sqlcoder-demo/) |
| |
| ## Ideal prompt and inference parameters |
| Set temperature to 0, and do not do sampling. |
| |
| ### Prompt |
| ``` |
| <|begin_of_text|><|start_header_id|>user<|end_header_id|> |
| |
| Generate a SQL query to answer this question: `{user_question}` |
| {instructions} |
|
|
| DDL statements: |
| {create_table_statements}<|eot_id|><|start_header_id|>assistant<|end_header_id|> |
| |
| The following SQL query best answers the question `{user_question}`: |
| ```sql |
| |
| ``` |
|
|
| ## Evaluation |
| This model was evaluated on SQL-Eval, a PostgreSQL based evaluation framework developed by Defog for testing and alignment of model capabilities. |
|
|
| You can read more about the methodology behind SQLEval [here](https://defog.ai/blog/open-sourcing-sqleval/). |
|
|
| ## Contact |
| Contact us on X at [@defogdata](https://twitter.com/defogdata), or on email at founders@defog.ai |
