| --- |
| library_name: transformers |
| tags: |
| - unsloth |
| - trl |
| - sft |
| datasets: |
| - xavierwoon/cestertrain |
| base_model: |
| - unsloth/llama-3-8b-bnb-4bit |
| --- |
| |
| # Model Card for Model ID |
|
|
| <!-- Provide a quick summary of what the model is/does. --> |
|
|
| Cesterllama is a fine-tuned Llama 3 8B model that is able to generate Libcester unit test cases in the correct format. |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| <!-- Provide a longer summary of what this model is. --> |
|
|
| <!-- This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. --> |
|
|
| - **Developed by:** Xavier Woon |
| <!-- - **Funded by [optional]:** [More Information Needed] |
| - **Shared by [optional]:** [More Information Needed] --> |
| - **Model type:** Llama |
| <!-- - **Language(s) (NLP):** [More Information Needed] |
| - **License:** [More Information Needed] --> |
| - **Finetuned from model [optional]:** unsloth/llama-3-8b-bnb-4bit |
|
|
| <!-- ### Model Sources [optional] |
|
|
| Provide the basic links for the model. |
|
|
| - **Repository:** [More Information Needed] |
| - **Paper [optional]:** [More Information Needed] |
| - **Demo [optional]:** [More Information Needed] --> |
|
|
| <!-- ## Uses --> |
|
|
| <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. --> |
|
|
| <!-- ### Direct Use --> |
|
|
| <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. --> |
|
|
| <!-- [More Information Needed] --> |
|
|
| <!-- ### Downstream Use [optional] --> |
|
|
| <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app --> |
|
|
| <!-- [More Information Needed] --> |
|
|
| <!-- ### Out-of-Scope Use --> |
|
|
| <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. --> |
|
|
| <!-- [More Information Needed] --> |
|
|
| ## Bias, Risks, and Limitations |
|
|
| <!-- This section is meant to convey both technical and sociotechnical limitations. --> |
|
|
| The model often regenerates the input prompt in the output. This can lead to limited test cases being printed due to truncations based on `max_new_tokens`. |
|
|
| ### Recommendations |
|
|
| <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> |
|
|
| Expanding the dataset will help increase the accuracy and robustness of the model, and improve code coverage based on real life scenarios. |
|
|
| ## How to Get Started with the Model |
|
|
| Use the code below to get started with the model. |
|
|
| ```py |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_name = "xavierwoon/cesterllama" |
| model = AutoModelForCausalLM.from_pretrained(model_name) |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| |
| # Paste your own code inside |
| code = """ |
| void add() |
| { |
| int a,b,c; |
| printf("\nEnter The Two values:"); |
| scanf("%d%d",&a,&b); |
| c=a+b; |
| printf("Addition:%d",c); |
| } |
| """ |
| |
| prompt = f"""### Instruction: |
| create cester test cases for this function: |
| {code} |
| |
| ### Input: |
| |
| ### Response: |
| """ |
| |
| inputs = tokenizer(prompt, return_tensors="pt").to("cpu") |
| |
| from transformers import TextStreamer |
| text_streamer = TextStreamer(tokenizer) |
| _ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 2048) |
| ``` |
|
|
| <!-- [More Information Needed] --> |
|
|
| ## Training Details |
|
|
| ### Training Data |
|
|
| <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> |
|
|
| Training Data was created based on Data Structures and Algorithm (DSA) codes created using ChatGPT. It would also create corresponding Cester test cases. After testing and ensuring a good code coverage, the prompt and corresponding test cases were added to the dataset. |
|
|
| ### Training Procedure |
|
|
| <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. --> |
|
|
| 1. Prompt GPT for sample DSA C code |
| 2. Prompt GPT for Libcester unit test cases with 100% code coverage |
| 3. Test generated test cases for robustness and code coverage |
|
|
| <!-- #### Preprocessing [optional] |
|
|
| [More Information Needed] |
| --> |
|
|
| <!-- #### Training Hyperparameters --> |
|
|
| <!-- - **Training regime:** [More Information Needed] fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision --> |
|
|
| <!-- #### Speeds, Sizes, Times [optional] --> |
|
|
| <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. --> |
|
|
| <!-- [More Information Needed] --> |
|
|
| <!-- ## Evaluation --> |
|
|
| <!-- This section describes the evaluation protocols and provides the results. --> |
|
|
| <!-- ### Testing Data, Factors & Metrics --> |
|
|
| <!-- #### Testing Data --> |
|
|
| <!-- This should link to a Dataset Card if possible. --> |
|
|
| <!-- [More Information Needed] --> |
|
|
| <!-- #### Factors --> |
|
|
| <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. --> |
|
|
| <!-- [More Information Needed] --> |
|
|
| <!-- #### Metrics --> |
|
|
| <!-- These are the evaluation metrics being used, ideally with a description of why. --> |
|
|
| <!-- [More Information Needed] --> |
|
|
| <!-- ### Results --> |
|
|
| <!-- [More Information Needed] --> |
|
|
| <!-- #### Summary --> |
|
|
|
|
|
|
| <!-- ## Model Examination [optional] --> |
|
|
| <!-- Relevant interpretability work for the model goes here --> |
|
|
| <!-- [More Information Needed] --> |
|
|
| <!-- ## Technical Specifications [optional] |
|
|
| ### Model Architecture and Objective |
|
|
| [More Information Needed] |
|
|
| ### Compute Infrastructure |
|
|
| [More Information Needed] |
|
|
| #### Hardware |
|
|
| [More Information Needed] |
|
|
| #### Software |
|
|
| [More Information Needed] |
|
|
| ## Citation [optional] --> |
|
|
| <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. --> |
|
|
| <!-- **BibTeX:** |
|
|
| [More Information Needed] |
|
|
| **APA:** |
|
|
| [More Information Needed] |
|
|
| ## Glossary [optional] --> |
|
|
| <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. --> |
|
|
| <!-- [More Information Needed] |
|
|
| ## More Information [optional] |
|
|
| [More Information Needed] |
|
|
| ## Model Card Authors [optional] |
|
|
| [More Information Needed] |
|
|
| ## Model Card Contact |
|
|
| [More Information Needed] --> |
