Text Generation
Transformers
Safetensors
English
llama
causal-lm
slm
tiny
small
AtomixLabs
Pascal
math
mathematics
arithmetic
sequence-induction
symbolic-regression
algorithmic-reasoning
oeis
discrete-mathematics
dynamical-systems
time-series
telemetry
multi-token-prediction
muon
synthetic-data
text-generation-inference
Instructions to use AtomixLabs/Pascal-15M-v1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AtomixLabs/Pascal-15M-v1.0 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="AtomixLabs/Pascal-15M-v1.0")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("AtomixLabs/Pascal-15M-v1.0") model = AutoModelForCausalLM.from_pretrained("AtomixLabs/Pascal-15M-v1.0", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use AtomixLabs/Pascal-15M-v1.0 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AtomixLabs/Pascal-15M-v1.0" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AtomixLabs/Pascal-15M-v1.0", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/AtomixLabs/Pascal-15M-v1.0
- SGLang
How to use AtomixLabs/Pascal-15M-v1.0 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "AtomixLabs/Pascal-15M-v1.0" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AtomixLabs/Pascal-15M-v1.0", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "AtomixLabs/Pascal-15M-v1.0" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AtomixLabs/Pascal-15M-v1.0", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use AtomixLabs/Pascal-15M-v1.0 with Docker Model Runner:
docker model run hf.co/AtomixLabs/Pascal-15M-v1.0
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,193 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
pipeline_tag: text-generation
|
| 6 |
+
tags:
|
| 7 |
+
- llama
|
| 8 |
+
- causal-lm
|
| 9 |
+
- slm
|
| 10 |
+
- tiny
|
| 11 |
+
- small
|
| 12 |
+
- AtomixLabs
|
| 13 |
+
- Pascal
|
| 14 |
+
- math
|
| 15 |
+
- mathematics
|
| 16 |
+
- arithmetic
|
| 17 |
+
- sequence-induction
|
| 18 |
+
- symbolic-regression
|
| 19 |
+
- algorithmic-reasoning
|
| 20 |
+
- oeis
|
| 21 |
+
- discrete-mathematics
|
| 22 |
+
- dynamical-systems
|
| 23 |
+
- time-series
|
| 24 |
+
- telemetry
|
| 25 |
+
- multi-token-prediction
|
| 26 |
+
- muon
|
| 27 |
+
- synthetic-data
|
| 28 |
+
library_name: transformers
|
| 29 |
---
|
| 30 |
+
|
| 31 |
+
# ⚛️ Pascal-15M-v1.0
|
| 32 |
+
|
| 33 |
+
**Pascal-15M-v1.0** is a compact foundation model developed by **AtomixLabs** for mathematical sequence induction and numeric pattern generalization.
|
| 34 |
+
|
| 35 |
+
Instead of generating natural language text or code, Pascal is trained to model the transition dynamics of numerical series, identifying the latent mathematical rules governing a sequence prefix and extrapolating them forward.
|
| 36 |
+
|
| 37 |
+
Pascal-15M-v1.0 contains **17,716,608 parameters** and was pretrained on approximately **364 million tokens** of structured numerical sequences. Built strictly on native LLaMA transformer blocks with a custom 32-token digit-level vocabulary, it is designed primarily as a lightweight, clean base checkpoint for downstream specialization and fine-tuning.
|
| 38 |
+
|
| 39 |
+
---
|
| 40 |
+
|
| 41 |
+
## Task Formulation & Inductive Scope
|
| 42 |
+
|
| 43 |
+
Given a comma-separated sequence prefix, Pascal functions as an autoregressive sequence solver. Rather than retrieving memorized series from an index, the model relies on its attention layers to identify recurrence orders, polynomial progressions, modular cycles, and rational approximations directly from raw input strings.
|
| 44 |
+
|
| 45 |
+
Representative input-output behaviors include:
|
| 46 |
+
|
| 47 |
+
* **Elementary Progressions:** `1, 2, 4, 8, 16,` → `32`
|
| 48 |
+
* **Rational Convergents:** `1, 1/2, 2/3, 3/5, 5/8,` → `8/13` *(Fibonacci convergents to the golden ratio φ)*
|
| 49 |
+
* **Interleaved Independent Streams:** `2, 10, 4, 20, 8, 30,` → `16`
|
| 50 |
+
* **Coupled State Dynamics:** `16, 20, 22, 22, 20, 16, 10,` → `2` *(latent velocity oscillator)*
|
| 51 |
+
|
| 52 |
+
The objective of the pretraining regime is to provide an inductive bias for **symbolic regression in latency**, allowing the network to capture latent dynamic laws from raw observations.
|
| 53 |
+
|
| 54 |
+
---
|
| 55 |
+
|
| 56 |
+
## Tokenizer & Numerical Representation
|
| 57 |
+
|
| 58 |
+
Standard language model tokenizers rely on Byte-Pair Encoding (BPE), which merges digits into arbitrary chunks based on corpus frequency (for example, tokenizing `1234` as a single unit while splitting `1235` into two). This arbitrary grouping disrupts place-value relationships and degrades arithmetic consistency.
|
| 59 |
+
|
| 60 |
+
Pascal resolves this by using an exact, **lossless 32-token vocabulary**:
|
| 61 |
+
|
| 62 |
+
* **Digits:** `0`, `1`, `2`, `3`, `4`, `5`, `6`, `7`, `8`, `9`
|
| 63 |
+
* **Separators & Operators:** `,` (delimiter), `-` (sign), `.` (decimal), and `/` (exact rational fraction slash)
|
| 64 |
+
* **Control Tokens:** `<bos>`, `<eos>`, `<pad>`, `<unk>`, and reserved alignment slots
|
| 65 |
+
|
| 66 |
+
> ⚠️ **No `<eos>` Supervision During Training:** While `<eos>` exists as an index slot in the tokenizer vocabulary, **Pascal-15M-v1.0 was not trained with an `<eos>` token**. All pretraining target sequences concluded directly on the final numerical digit of the pattern. Consequently, the model will **never spontaneously emit an `<eos>` token** to halt generation. During inference, stopping criteria must be handled explicitly—either by setting `eos_token_id=tokenizer.convert_tokens_to_ids(",")` to stop at the next number, or by specifying `max_new_tokens`.
|
| 67 |
+
|
| 68 |
+
By processing numbers strictly digit-by-digit or as exact rational ratios (e.g., `8/13`), the model avoids floating-point rounding errors and eliminates tokenization artifacts during sequence evaluation.
|
| 69 |
+
|
| 70 |
+
---
|
| 71 |
+
|
| 72 |
+
## Architectural Specification
|
| 73 |
+
|
| 74 |
+
Pascal-15M-v1.0 uses a standard decoder-only LLaMA architecture. The model relies entirely on native PyTorch/Hugging Face operations without custom CUDA kernels, non-standard attention mechanisms, or external repository dependencies (`trust_remote_code=False`).
|
| 75 |
+
|
| 76 |
+
```python
|
| 77 |
+
# Pascal-15M-v1.0 Structural Specification
|
| 78 |
+
vocab_size: int = 32 # Lossless digit-level numerical vocabulary
|
| 79 |
+
hidden_size: int = 384 # Hidden dimension
|
| 80 |
+
intermediate_size: int = 768 # 2.0x SwiGLU expansion ratio
|
| 81 |
+
num_hidden_layers: int = 12 # Sequential layer depth
|
| 82 |
+
num_attention_heads: int = 12 # Head dimension = 32
|
| 83 |
+
num_key_value_heads: int = 12 # Multi-Head Attention (MHA)
|
| 84 |
+
max_position_embeddings: int = 512 # Context window
|
| 85 |
+
rope_theta: float = 10000.0 # Base rotary frequency
|
| 86 |
+
# Total Trainable Parameters: 17,716,608
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
At under 18 million parameters, the network operates with low compute requirements and low memory footprint, scaling appropriately for its 32-token vocabulary across its ~364M token training run.
|
| 90 |
+
|
| 91 |
+
---
|
| 92 |
+
|
| 93 |
+
## Intended Use & Downstream Adaptation
|
| 94 |
+
|
| 95 |
+
Pascal-15M-v1.0 is intended primarily as a **base model for downstream fine-tuning**.
|
| 96 |
+
|
| 97 |
+
While the base checkpoint exhibits stable out-of-the-box performance on standard arithmetic, polynomial, and recurrence patterns, it is designed to be adapted to specific numerical tasks, including:
|
| 98 |
+
|
| 99 |
+
* **Time-Series & Sensor Modeling:** Fine-tuning on telemetry sequences, systems monitoring, or discrete sensor metrics.
|
| 100 |
+
* **Empirical Symbolic Regression:** Adapting the model to experimental observations to predict underlying dynamic behaviors.
|
| 101 |
+
* **Financial Tick Sequences:** Modeling discrete price levels, spread variations, and rate adjustments.
|
| 102 |
+
* **Dynamical Systems Simulation:** Specializing the architecture on kinematics, coupled oscillations, or orbital transitions.
|
| 103 |
+
|
| 104 |
+
### Basic Usage
|
| 105 |
+
|
| 106 |
+
```python
|
| 107 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 108 |
+
|
| 109 |
+
model_id = "AtomixLabs/Pascal-15M-v1.0"
|
| 110 |
+
|
| 111 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 112 |
+
model_id,
|
| 113 |
+
torch_dtype="auto",
|
| 114 |
+
device_map="auto"
|
| 115 |
+
)
|
| 116 |
+
|
| 117 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
| 118 |
+
|
| 119 |
+
prompt = "1, 1/2, 2/3, 3/5, 5/8,"
|
| 120 |
+
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
| 121 |
+
sep_id = tokenizer.convert_tokens_to_ids(",")
|
| 122 |
+
|
| 123 |
+
outputs = model.generate(
|
| 124 |
+
**inputs,
|
| 125 |
+
max_new_tokens=24,
|
| 126 |
+
do_sample=False,
|
| 127 |
+
eos_token_id=sep_id, # Stop immediately upon completing the next number
|
| 128 |
+
pad_token_id=tokenizer.pad_token_id
|
| 129 |
+
)
|
| 130 |
+
|
| 131 |
+
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 132 |
+
```
|
| 133 |
+
|
| 134 |
+
---
|
| 135 |
+
|
| 136 |
+
## Constrained Logit Decoding
|
| 137 |
+
|
| 138 |
+
Because the model uses a compact 32-token vocabulary, outputs can be guided during generation using grammar-constrained logit masking.
|
| 139 |
+
|
| 140 |
+
An external logit processor can enforce structural constraints:
|
| 141 |
+
1. Suppressing consecutive signs or delimiters (`--`, `..`, `//`).
|
| 142 |
+
2. Disallowing invalid operations such as division by zero (`/0`).
|
| 143 |
+
3. Preventing open-ended fraction slashes or decimals before sequence separators.
|
| 144 |
+
|
| 145 |
+
This setup ensures that generated outputs conform to well-formed integers, decimals, or reduced fractions.
|
| 146 |
+
|
| 147 |
+
---
|
| 148 |
+
|
| 149 |
+
<details>
|
| 150 |
+
<summary><b>Safety Advisory, Technical Limitations & License Notice</b></summary>
|
| 151 |
+
|
| 152 |
+
### Operational Scope & Technical Limitations
|
| 153 |
+
|
| 154 |
+
**Pascal-15M-v1.0** is an experimental, specialized micro-scale foundation model (17.7M parameters). Because it operates exclusively on numerical sequences rather than natural language, its failure modes differ fundamentally from standard text models. Users and downstream developers should account for the following technical and operational considerations:
|
| 155 |
+
|
| 156 |
+
**1. Vocabulary, Formatting & Context Bounds**
|
| 157 |
+
* **Numeric-Only Scope:** Pascal-15M-v1.0 has a strict 32-token numerical vocabulary. It cannot parse English instructions, hold conversations, or generate source code. Any unsupported characters are mapped to `<unk>` during tokenization, which introduces uninformative tokens into the context and degrades prediction quality.
|
| 158 |
+
* **Formatting Sensitivity:** The model expects consistent comma-separated sequences without extraneous whitespace or mixed notation. Malformed delimiters, non-standard scientific notation, or inconsistent decimal precision can disrupt attention states and produce erratic continuations.
|
| 159 |
+
* **Context Truncation (512 Tokens):** Because sequences are represented digit-by-digit, high-precision floating-point numbers or long series rapidly consume the 512-token context window. Once a sequence exceeds this boundary, earlier terms are truncated, and the model may lose track of long-range recurrence rules or initial boundary conditions.
|
| 160 |
+
|
| 161 |
+
**2. Non-Formal Prediction & Spurious Extrapolation**
|
| 162 |
+
* **Not a Symbolic Engine:** Pascal-15M-v1.0 is an autoregressive neural sequence predictor, not a deterministic computer algebra system (CAS) or formal theorem prover. It approximates latent patterns statistically and does not guarantee exact analytical precision.
|
| 163 |
+
* **False Pattern Induction on Stochastic Data:** The model is trained to detect structural continuity. When prompted with purely stochastic processes, cryptographic outputs, unmodeled exogenous shocks, or chaotic systems, it will attempt to fit and extrapolate spurious patterns rather than recognizing the input as non-deterministic noise.
|
| 164 |
+
* **Verification:** Predictions should be independently validated by deterministic checks or domain experts before being treated as analytically reliable.
|
| 165 |
+
|
| 166 |
+
**3. Downstream Fine-Tuning & Adaptation Responsibility**
|
| 167 |
+
* Pascal-15M-v1.0 is intended primarily as a base weight initialization for downstream specialization. Fine-tuning on domain-specific telemetry, sensor feeds, or financial ticks alters the model's inductive bias and error distributions.
|
| 168 |
+
* Downstream developers are solely responsible for conducting out-of-distribution testing, calibrating confidence thresholds, and monitoring for distribution shift in fine-tuned checkpoints.
|
| 169 |
+
|
| 170 |
+
**4. High-Consequence System Advisory**
|
| 171 |
+
* This checkpoint was developed as a foundational research asset and has not been validated, audited, or certified for safety-critical or high-consequence environments.
|
| 172 |
+
* It is not designed for direct integration into unmonitored physical control loops, industrial actuation, medical diagnostics, aviation/navigation systems, or autonomous financial execution pipelines. Any deployment of raw or fine-tuned weights in environments where numerical misprediction carries physical, operational, or financial risk should incorporate strict external bounds, deterministic fallback logic, and independent oversight.
|
| 173 |
+
|
| 174 |
+
**5. Apache 2.0 Disclaimer of Warranty & Limitation of Liability**
|
| 175 |
+
|
| 176 |
+
Pascal-15M-v1.0 is released by AtomixLabs under the **Apache License, Version 2.0**. In accordance with Sections 7 and 8 of the license, the model weights and associated files are provided strictly on an **"AS IS" BASIS**, without warranties or conditions of any kind, either express or implied, including, without limitation, any warranties or conditions of title, non-infringement, merchantability, or fitness for a particular purpose.
|
| 177 |
+
|
| 178 |
+
To the maximum extent permitted by applicable law, in no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, shall AtomixLabs, its developers, or its contributors be liable for any direct, indirect, special, incidental, exemplary, or consequential damages of any character arising as a result of the use, fine-tuning, modification, or deployment of this model, including but not limited to damages for loss of goodwill, work stoppage, computer failure, system malfunction, data corruption, or financial loss, even if advised of the possibility of such damages. Users assume sole responsibility for determining the appropriateness of using or redistributing the model and assume all risks associated with its exercise of permissions under the license.
|
| 179 |
+
|
| 180 |
+
</details>
|
| 181 |
+
|
| 182 |
+
## Citation
|
| 183 |
+
|
| 184 |
+
```bibtex
|
| 185 |
+
@model{atomixlabs_pascal_15m_v1_2026,
|
| 186 |
+
author = {AtomixLabs},
|
| 187 |
+
title = {Pascal-15M-v1.0: A Compact Foundation Transformer for Mathematical Sequence Induction},
|
| 188 |
+
year = {2026},
|
| 189 |
+
publisher = {Hugging Face},
|
| 190 |
+
license = {Apache-2.0},
|
| 191 |
+
url = {https://huggingface.co/AtomixLabs/Pascal-15M-v1.0}
|
| 192 |
+
}
|
| 193 |
+
```
|