| --- |
| license: apache-2.0 |
| datasets: |
| - bigcode/starcoderdata |
| --- |
| # Model Card for DeciCoder-6B |
|
|
| DeciCoder-6B is a 6 billion parameter decoder-only code completion model |
| trained on the Python, Java, Javascript, Rust, C++, C, and C# subset of [Starcoder Training Dataset](https://huggingface.co/datasets/bigcode/starcoderdata). |
| The model uses variable Grouped Query Attention and has a context window of 2k |
| tokens. It was trained using a Fill-in-the-Middle training objective. The model's |
| architecture was generated by Deci's proprietary Neural Architecture |
| Search-based technology, AutoNAC. |
|
|
| ## Model Details |
|
|
| - **Developed by:** Deci |
| - **Model type:** DeciCoder-6B is an auto-regressive language model based on the transformer decoder architecture, using variable Grouped Query Attention. |
| - **Language(s):** Python, Java, JavaScript, Rust, C++, C, C#, Go |
| - **License:** Model checkpoints are licensed under the [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
|
|
| ## Documentation |
|
|
| - Blog Post: [Introducing DeciCoder-6B: Code LLM Engineered for Accuracy & Cost Efficiency At Scale](https://deci.ai/blog/decicoder-6b-the-best-multi-language-code-generation-llm-in-its-class/) |
| - Tutorial: [How to Run DeciCoder-6B on Qualcomm Cloud AI 100](https://github.com/quic/cloud-ai-sdk/tree/1.12/models/language_processing/decoder) |
| - Google Colab [Notebook](http://bit.ly/DeciCoder-6B-Notebook-1) |
| - Run DeciCoder on [AWS DL2q instances using the Qualcomm Cloud AI Platform SDK](https://bit.ly/Amazon-EC2-DL2q-Instance) |
| - Questions: Feel free to contact us via our [Discord Community!](https://discord.com/invite/p9ecgRhDR8/) |
|
|
| ## Model Architecture |
|
|
| | Parameters | Layers | Heads | Sequence Length | GQA num_key_value_heads | |
| |:----------|:----------|:----------|:----------|:----------| |
| | 6B | 32 | 32 | 2k | Variable | |
| |
| |
| - **Decoder layer:** Variable Grouped Query Attention |
| - **Position Embeddings:** Rotary Position Embeddings [Su et al., 2021](https://arxiv.org/abs/2104.09864) |
| |
| |
| ### How to Use |
| |
| ```bibtex |
| # pip install -q transformers |
| import torch |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| checkpoint = "Deci/DeciCoder-6B" |
| device = "cuda" # for GPU usage or "cpu" for CPU usage |
| |
| tokenizer = AutoTokenizer.from_pretrained(checkpoint) |
| model = AutoModelForCausalLM.from_pretrained(checkpoint, torch_dtype=torch.bfloat16, trust_remote_code=True).to(device) |
|
|
| inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to(device) |
| outputs = model.generate(inputs, max_new_tokens=100) |
| print(tokenizer.decode(outputs[0])) |
| |
| ### Attribution |
| |
| DeciCoder-6B was trained on StarCoder Training Dataset, filtered for |
| Python, Java, JavaScript, Ruby, RUST, C++, C, and C#. For additional information, please |
| refer to [https://huggingface.co/datasets/bigcode/starcoderdata](https://huggingface.co/datasets/bigcode/starcoderdata). |
| |
| ``` |
| |
| ### Limitations |
| |
| The model has undergone training with source code from Python, Java, |
| JavaScript, RUST, C++, C, and C#, and Go. While the primary language in the source is English, it does |
| contain other languages. Therefore, the model can produce code snippets |
| given some context. However, there is no assurance that the resulting |
| code will function as expected. It might be suboptimal, contain bugs, or |
| even exploits. |
| |
| ## Evaluation |
| |
| Below are DeciCoder-6B's pass@1 on MultiPL HumanEval scores |
| |
| | Python | JavaScript | Java | C++ | C# | Rust | Go | |
| |:----------|:----------|:----------|:----------|:----------|:----------|:----------| |
| | 33.3% | 29.3% | 30.3% |29.93% |20.31% |20.5% |77.47% | |
| |
| |
| ### Runtime Benchmarks |
| |
| |Inference Tool | Hardware | Prompt Length | Generation Length | Throughput (tokens/sec) | |
| |:----------|:----------|:----------|:----------|:----------| |
| | Qualcomm Cloud AI 100 SDK | Qualcomm Cloud AI 100 | 1024 | 1024 | 531.3 | |
| |
| - Measured for maximal batch size on the device |
| |
| ## How to Cite |
| |
| Please cite this model using this format. |
| |
| ```bibtex |
| @misc{DeciFoundationModels, |
| title = {DeciCoder-6B}, |
| author = {DeciAI Research Team}, |
| year = {2024} |
| url={[https://huggingface.co/deci/decicoder-6B](https://huggingface.co/deci/decicoder-6B)}, |
| } |
| ``` |
