Update README.md

README.md

@@ -1,199 +1,447 @@
 ---
 library_name: transformers
-tags: []
+tags:
+  - qwen2.5-coder
+  - code-generation
+  - code-modernization
+  - python
+  - qlora
+  - lora
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
+# Model Card for Qwen2.5-Coder-1.5B Python 3.14 QLoRA Adapter
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
+This model is a LoRA adapter fine-tuned on top of `Qwen/Qwen2.5-Coder-1.5B-Instruct` for Python code modernization, with a focus on updating Python code toward Python 3.14-compatible syntax and practices.
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+The adapter was trained using QLoRA with 4-bit NF4 quantization of the frozen base model and LoRA-based parameter-efficient fine-tuning.
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+- **Base model:** `Qwen/Qwen2.5-Coder-1.5B-Instruct`
+- **Model type:** LoRA / QLoRA adapter
+- **Task:** Python code modernization
+- **Primary target:** Python 3.14 modernization
+- **Quantization:** 4-bit NF4
+- **Nested quantization:** Enabled
+- **Compute dtype:** FP16
 
-### Model Sources [optional]
+### Model Sources
 
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Base model:** `Qwen/Qwen2.5-Coder-1.5B-Instruct`
+- **Adapter repository:** `Krishnasri2027/qwen25-coder-1.5b-python3147-nf4-adapter`
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+This adapter is intended to be used with the base Qwen2.5-Coder-1.5B-Instruct model to modernize Python source code toward Python 3.14.
 
-[More Information Needed]
+Potential applications include:
 
-### Downstream Use [optional]
+- Modernizing legacy Python syntax
+- Updating Python code to newer language features
+- Refactoring older Python implementations
+- Assisting developers with Python version migration
+- Generating modernized Python code from older implementations
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+### Downstream Use
 
-[More Information Needed]
+The adapter can be integrated into developer tools, code migration pipelines, educational tools, and automated code modernization workflows.
+
+It can also be merged with the base model to create a standalone fine-tuned model.
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+This model is not intended to:
+
+- Guarantee that generated code is fully compatible with every Python 3.14 environment
+- Replace automated testing or human code review
+- Perform security-critical code migration without validation
+- Generate production-ready software without testing
+- Serve as a general-purpose replacement for the original Qwen2.5-Coder model
 
-[More Information Needed]
+Generated code should always be validated, tested, and reviewed before production use.
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+The model inherits limitations from the underlying Qwen2.5-Coder model and from the fine-tuning dataset.
+
+The model may:
 
-[More Information Needed]
+- Produce syntactically incorrect code in some situations
+- Introduce behavioral changes during modernization
+- Apply an inappropriate modernization depending on the context
+- Fail to preserve edge-case behavior
+- Generate code that appears valid but requires additional testing
+- Reflect biases or limitations present in the base model and training data
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+Generated code should be:
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+1. Reviewed by a developer.
+2. Parsed or compiled using the target Python version.
+3. Tested against the original implementation's expected behavior.
+4. Validated with appropriate unit and integration tests.
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
-[More Information Needed]
+The adapter can be loaded together with the base model using PEFT and BitsAndBytes.
+
+```python
+import torch
+
+from transformers import (
+    AutoModelForCausalLM,
+    AutoTokenizer,
+    BitsAndBytesConfig,
+)
+
+from peft import PeftModel
+
+
+BASE_MODEL = "Qwen/Qwen2.5-Coder-1.5B-Instruct"
+ADAPTER_MODEL = "Krishnasri2027/qwen25-coder-1.5b-python3147-nf4-adapter"
+
+tokenizer = AutoTokenizer.from_pretrained(
+    ADAPTER_MODEL,
+    trust_remote_code=True,
+)
+
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_use_double_quant=True,
+    bnb_4bit_compute_dtype=torch.float16,
+)
+
+base_model = AutoModelForCausalLM.from_pretrained(
+    BASE_MODEL,
+    quantization_config=bnb_config,
+    torch_dtype=torch.float16,
+    device_map="auto",
+    trust_remote_code=True,
+)
+
+model = PeftModel.from_pretrained(
+    base_model,
+    ADAPTER_MODEL,
+    is_trainable=False,
+)
+
+model.eval()
+
+messages = [
+    {
+        "role": "user",
+        "content": "Modernize the following Python code for Python 3.14:\n\n<your Python code here>",
+    }
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt",
+).to(model.device)
+
+with torch.no_grad():
+    outputs = model.generate(
+        inputs,
+        max_new_tokens=512,
+        temperature=0.2,
+        do_sample=False,
+    )
+
+response = tokenizer.decode(
+    outputs[0][inputs.shape[-1]:],
+    skip_special_tokens=True,
+)
+
+print(response)
+```
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+The model was fine-tuned using a chat-formatted training dataset designed for Python code modernization.
 
-[More Information Needed]
+The training examples were tokenized using the Qwen2.5-Coder tokenizer and its chat template.
 
-### Training Procedure
+The configured sequence length was **256 tokens**.
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+Token-length analysis of the training examples produced:
 
-#### Preprocessing [optional]
+| Statistic | Tokens |
+|---|---:|
+| Minimum | 115 |
+| Maximum | 197 |
+| Mean | 150.792 |
+| Median | 147 |
+| 95th percentile | 187 |
+| 99th percentile | 197 |
+| Configured sequence length | 256 |
 
-[More Information Needed]
+The maximum observed sequence length of 197 tokens was below the configured 256-token sequence length, providing approximately 59 tokens of headroom.
 
+### Training Procedure
 
-#### Training Hyperparameters
+The model was trained using **QLoRA**, keeping the quantized base model frozen while training a small set of LoRA adapter parameters.
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+The base model was quantized to 4-bit NF4 using BitsAndBytes with nested/double quantization.
 
-#### Speeds, Sizes, Times [optional]
+LoRA adapters were applied to:
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+- `q_proj`
+- `k_proj`
+- `v_proj`
+- `o_proj`
+- `gate_proj`
+- `up_proj`
+- `down_proj`
 
-[More Information Needed]
+#### Training Hyperparameters
 
-## Evaluation
+| Parameter | Value |
+|---|---:|
+| Base model | Qwen/Qwen2.5-Coder-1.5B-Instruct |
+| Fine-tuning method | QLoRA |
+| Quantization | 4-bit NF4 |
+| Nested quantization | Enabled |
+| Compute dtype | FP16 |
+| LoRA rank (`r`) | 16 |
+| LoRA alpha | 32 |
+| LoRA dropout | 0.05 |
+| Batch size | 1 |
+| Gradient accumulation steps | 8 |
+| Effective batch size | 8 |
+| Number of epochs | 4 |
+| Learning rate | 1e-4 |
+| LR scheduler | Cosine |
+| Weight decay | 0.01 |
+| Warmup steps | 30 |
+| Evaluation frequency | Every 50 steps |
+| Checkpoint save frequency | Every 50 steps |
+| Logging frequency | Every 10 steps |
+| Maximum sequence length | 256 |
+| Gradient checkpointing | Enabled |
+| FIM rate | 0.0 |
+| FIM SPM rate | 0.0 |
+| Random seed | 42 |
+| FP16 | Enabled |
+| BF16 | Disabled |
+
+### Speeds, Sizes, Times
+
+The completed training run reported:
+
+| Metric | Value |
+|---|---:|
+| Global steps | 500 |
+| Epochs | 4 |
+| Training runtime | 3342.5477 seconds |
+| Training runtime | ~55 minutes 43 seconds |
+| Training samples / second | 1.197 |
+| Training steps / second | 0.150 |
+| Total FLOPs | 4,835,834,903,592,960 |
+| Total tokens processed | 607,168 |
+| Final logged training loss | 0.037346 |
+| Aggregate Trainer training loss | 0.2003363176 |
+
+The `training_loss` value reported by `Trainer` (`0.2003363176`) represents the aggregate training loss reported for the complete run, whereas the training loss shown at step 500 (`0.037346`) is the most recent logged training loss.
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+## Evaluation
 
 ### Testing Data, Factors & Metrics
 
 #### Testing Data
 
-<!-- This should link to a Dataset Card if possible. -->
+The evaluation dataset was held separately from the training data and was used to calculate validation loss during training.
 
-[More Information Needed]
+Validation was performed every 50 training steps.
 
 #### Factors
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+The primary evaluation objective was to determine whether the model could learn the expected Python modernization patterns while reducing validation loss throughout training.
 
-[More Information Needed]
+The evaluation results below are the validation losses recorded during the training run.
 
 #### Metrics
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+The primary reported metrics are:
 
-[More Information Needed]
+- Training loss
+- Validation loss
+- Entropy
+- Mean token accuracy
+- Number of processed tokens
 
 ### Results
 
-[More Information Needed]
-
 #### Summary
 
+The model showed a substantial reduction in both training and validation loss during fine-tuning.
 
+Validation loss decreased from **0.252825 at step 50** to **0.037586 at step 500**.
 
-## Model Examination [optional]
+This represents an approximately **85.1% reduction in validation loss** over the recorded training run.
 
-<!-- Relevant interpretability work for the model goes here -->
+The final reported mean token accuracy was **98.6229%** at step 500.
 
-[More Information Needed]
+The validation loss continued to decrease throughout the later checkpoints, reaching its lowest recorded value of **0.037586 at step 500**.
 
-## Environmental Impact
+| Step | Training Loss | Validation Loss | Entropy | Mean Token Accuracy | Num Tokens |
+|---:|---:|---:|---:|---:|---:|
+| 50 | 0.329822 | 0.252825 | 0.280223 | 0.939414 | 60,652 |
+| 100 | 0.070807 | 0.064497 | 0.067091 | 0.981024 | 121,228 |
+| 150 | 0.043590 | 0.048881 | 0.047465 | 0.984845 | 182,259 |
+| 200 | 0.040075 | 0.042402 | 0.043094 | 0.985729 | 242,627 |
+| 250 | 0.038932 | 0.040337 | 0.044742 | 0.985716 | 303,584 |
+| 300 | 0.037993 | 0.038709 | 0.041848 | 0.985923 | 364,412 |
+| 350 | 0.037793 | 0.038682 | 0.041355 | 0.986143 | 424,864 |
+| 400 | 0.038162 | 0.038088 | 0.039560 | 0.986307 | 485,911 |
+| 450 | 0.040015 | 0.037656 | 0.041152 | 0.986229 | 546,449 |
+| 500 | 0.037346 | **0.037586** | 0.040890 | **0.986229** | 607,168 |
+
+The final training output was:
+
+```text
+TrainOutput(
+    global_step=500,
+    training_loss=0.20033631759881973,
+    epoch=4.0
+)
+```
+
+with the following reported training metrics:
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+```text
+train_runtime: 3342.5477
+train_samples_per_second: 1.197
+train_steps_per_second: 0.15
+total_flos: 4835834903592960.0
+train_loss: 0.20033631759881973
+epoch: 4.0
+```
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+These results indicate that the model successfully optimized the training objective and achieved high token-level accuracy on the evaluation data.
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+However, token-level accuracy and validation loss should not be interpreted as a guarantee that every generated modernization preserves the original program's behavior. Functional testing remains necessary.
 
-## Technical Specifications [optional]
+## Model Examination
+
+The model should be examined primarily through qualitative code-generation and modernization tests.
+
+Recommended examination procedures include:
+
+- Comparing legacy and modernized Python implementations.
+- Checking generated code with the Python 3.14 interpreter.
+- Running unit tests before and after modernization.
+- Checking whether program behavior is preserved.
+- Testing edge cases and uncommon Python constructs.
+- Evaluating whether deprecated or legacy syntax is correctly modernized.
+
+No additional qualitative examination results are reported in this model card unless separately documented.
+
+## Environmental Impact
+
+The training run required approximately **55 minutes and 43 seconds** of compute time.
+
+No verified carbon-emissions measurement was recorded for this training run, so a specific carbon footprint is not reported.
+
+## Technical Specifications
 
 ### Model Architecture and Objective
 
-[More Information Needed]
+The underlying architecture is based on Qwen2.5-Coder-1.5B-Instruct.
 
-### Compute Infrastructure
+The fine-tuning objective uses supervised instruction tuning with LoRA adapters.
 
-[More Information Needed]
+QLoRA was used to reduce the memory requirements of fine-tuning by quantizing the frozen base model to 4-bit NF4 while keeping the trainable LoRA parameters separate.
+
+The LoRA configuration was:
+
+```text
+r = 16
+alpha = 32
+dropout = 0.05
+```
+
+Target modules:
+
+```text
+q_proj
+k_proj
+v_proj
+o_proj
+gate_proj
+up_proj
+down_proj
+```
+
+The adapter contains only the learned parameter-efficient fine-tuning weights and does not contain a standalone copy of the base model.
+
+### Compute Infrastructure
 
 #### Hardware
 
-[More Information Needed]
+Training was performed using an NVIDIA Tesla T4 GPU environment.
 
 #### Software
 
-[More Information Needed]
+| Component | Version |
+|---|---|
+| Python | 3.13 |
+| PyTorch | 2.11.0+cu128 |
+| CUDA | 12.8 |
+| Transformers | 5.16.1 |
+| TRL | 1.12.0 |
+| PEFT | 0.20.0 |
+| Accelerate | 1.14.0 |
+| BitsAndBytes | 0.50.2 |
+
+## Citation
+
+If you use this adapter, please also cite the underlying Qwen2.5-Coder model according to the citation information provided by its authors.
+
+## Glossary
 
-## Citation [optional]
+**LoRA**  
+Low-Rank Adaptation, a parameter-efficient fine-tuning method that trains small low-rank matrices instead of updating the entire model.
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+**QLoRA**  
+Quantized LoRA fine-tuning, where the frozen base model is loaded in low-bit precision while LoRA parameters are trained.
 
-**BibTeX:**
+**NF4**  
+NormalFloat 4-bit, a 4-bit quantization data type designed for normally distributed neural-network weights.
 
-[More Information Needed]
+**Nested Quantization / Double Quantization**  
+A technique that further quantizes the quantization constants to reduce memory usage.
 
-**APA:**
+**FP16**  
+16-bit floating-point representation used for computation during training and inference.
 
-[More Information Needed]
+**Mean Token Accuracy**  
+The proportion of target tokens correctly predicted by the model during evaluation.
 
-## Glossary [optional]
+**Validation Loss**  
+The loss calculated on the held-out evaluation dataset.
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+## More Information
 
-[More Information Needed]
+This adapter is specifically intended for Python code modernization toward Python 3.14.
 
-## More Information [optional]
+For deployment scenarios with limited GPU memory, the adapter can be loaded with the base model using 4-bit NF4 quantization.
 
-[More Information Needed]
+For standalone deployment, the LoRA adapter can also be merged into the base model and subsequently quantized using an appropriate deployment format.
 
-## Model Card Authors [optional]
+## Model Card Authors
 
-[More Information Needed]
+Krishnasri2027
 
 ## Model Card Contact
 
-[More Information Needed]
+For questions, issues, or suggestions regarding this model, please use the discussion and issue facilities available on the model repository.