| --- |
| license: cc |
| language: |
| - en |
| pipeline_tag: text-generation |
| tags: |
| - code |
| - python |
| --- |
| |
| ### CodePigeon |
|
|
| ## Summary |
|
|
| Introducing "CodePigon", a new coding model, is it good enough; nah but its so much better than 'BrtGPT-1-PreCode'. |
| Also its a test model, not a full product. I will release the new, better, final model on 25 November 2026. |
|
|
| ## Model Details |
|
|
| |Parameters|Architecture|Training Final Loss|Training Data|Training Language|Context Lenght| |
| | :---- | :---- | :---- | :---- | :---- | :---- | |
| |22~ Million|GPT-2 Arch.|1.6-1.8|22.000 QA Code Pairs|EN (English) + PY (Python)|256 Tokens -I know its very short for coding-| |
|
|
| Model is trained on Google Colab for couple hours. |
|
|
| Model uses "```python" (3x back-quote) for codes |
| |
| ## Newer Models, Probable Risks |
| |
| I'll train a new model with at least 3-4x more data and context lenght for so much better performance. |
| |
| Also model is not super good in coding but its not censored, so it can generate harmfull or any other 'bad' content, ***USE WITH CAUTION*** |
| |
| ## Contact |
| |
| ***PLEASE OPEN A DISCUSION OR EMAIL ME FOR DATASET, CODE AND TRAINING DETAILS*** |
| E-MAIL = "bertugscpmail@gmail.com" |
