---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- slm
- tiny
- code
- python
- html
- gguf
- llama
datasets:
- nvidia/OpenCodeInstruct
- styfeng/TinyDialogues
- HuggingFaceTB/smol-smoltalk
- HuggingFaceTB/everyday-conversations-llama3.1-2k
---

# Pico 1.3 Coding Large

Pico 1.3 Coding Large is a tiny English model with **6.9 million parameters** (14 MB) made by Convergent. It chats like the other Pico models and writes **simple Python functions and simple HTML snippets** when your message contains the word **codingPico**.

```
codingPico write a function that returns the largest number in a list
```

```python
def largest_number(numbers):
    """
    Returns the largest number in a list of numbers.

    :param numbers: List of integers
    :return: The largest number in the list, or None if the list is empty
    """
    if not numbers:
        return None
    return max(numbers)
```

Without the word it does not write code; asked "How do I get you to write code?" it answers "Say codingPico together with your request, for example: codingPico write a function that adds two numbers."

## Honest result

Measured with greedy decoding on tasks that were never in the training data:

| Test | Result |
|---|---|
| 50 simple Python tasks of the same kind as the training data, checked by running their tests | **68 %** pass |
| 20 new hand-written Python tasks with function names that never occur in the training data | **10 %** pass (2 of 20) |
| 30 HTML requests: markup parses with every tag closed | **100 %** |
| 30 coding requests **without** "codingPico": answered without a code block | **100 %** |

- The 68 % is not real problem solving: for 48 of those 50 tasks a function with the same name (for example `is_prime`) exists in the training data. The model mostly reproduces patterns it has seen.
- On genuinely new tasks it usually writes code that looks right but does the wrong thing (for example counting all words instead of only long ones). 2 of 20 passed.
- "Valid HTML" only means the tags are well formed. It does not mean the page matches the request: asked for a blue button it may produce a generic styled page.
- It is not tested on HumanEval or similar benchmarks; expect about 0 % there.

Use it as a toy and a demonstration of how far 7M parameters go, not as a coding assistant.

## Model

| | |
|---|---|
| Parameters | 6,917,376 |
| Architecture | Llama-style decoder (RMSNorm, SwiGLU, RoPE, tied embeddings) |
| Layers / hidden / MLP | 6 / 256 / 704 |
| Attention heads | 4 (head dim 64) |
| Context | 1,024 tokens |
| Vocabulary | 8,192 (byte-level BPE trained on code and chat text) |

## Usage

### Ollama

Download `pico-1.3-coding-large.gguf` and `Modelfile` into one folder, then:

```bash
ollama create pico1.3-coding-large -f Modelfile
ollama run pico1.3-coding-large
```

Use the `Modelfile`; without its chat template the model produces nonsense.

### transformers

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("Convergent-ch/Pico-1.3-Coding-Large")
model = AutoModelForCausalLM.from_pretrained("Convergent-ch/Pico-1.3-Coding-Large")

ids = tok.apply_chat_template(
    [{"role": "user", "content": "codingPico write a function that returns the largest number in a list"}],
    add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=150, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
```

Chat format: `<|bos|><|user|>\n...<|end|>\n<|assistant|>\n`, generation stops at `<|end|>`. Use temperature 0 to 0.2.

## Training

Trained from scratch in one stage on one RTX 3060 Ti with MakeAI: 3,200 steps of 32 x 1,024 tokens (about 7 passes over 14.8M tokens), AdamW, learning rate 2e-3 with cosine decay, bf16, about 12 minutes. Final validation loss 1.508 (perplexity 4.5) on held-out chats; the loss is only computed on the assistant's replies.

Training data (86,250 chats):

- About 28,000 simple Python tasks from [nvidia/OpenCodeInstruct](https://huggingface.co/datasets/nvidia/OpenCodeInstruct) (CC BY 4.0): only tasks whose reference solution passes all tests, short prompts and solutions, code block only.
- 403 Python tasks and 603 HTML snippets written locally by Qwen3 8B (an abliterated community variant, Apache 2.0): Python kept only if its own assert tests pass, HTML kept only if every tag is closed. Weighted 3x and 4x.
- In every code example the user message contains "codingPico". For 30 % of the tasks the same request also appears without the word, answered with a short hint to use it.
- 40,000 general chats (TinyDialogues, MIT, generated with GPT-4 by its authors; Everyday Conversations and Smol-SmolTalk, Apache 2.0, generated with Llama 3.1 by their authors).
- 6,000 identity chats (name, maker, size, how to use codingPico).

## Limitations

- Only simple, short Python functions and HTML snippets; no classes, no larger programs, no JavaScript, no debugging.
- Often writes plausible but wrong code. Always test what it writes.
- Facts in normal chat are unreliable.
- English only, context of 1,024 tokens. No safety tuning.

## License

Apache 2.0.