How to use from the
Use from the
Transformers library
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AtomixLabs/Pascal-15M-v1.0")
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AtomixLabs/Pascal-15M-v1.0")
model = AutoModelForCausalLM.from_pretrained("AtomixLabs/Pascal-15M-v1.0", device_map="auto")
Quick Links

Pascal15M

โš›๏ธ Pascal-15M-v1.0

Pascal-15M-v1.0 is a compact foundation model developed by AtomixLabs for mathematical sequence induction and numeric pattern generalization.

Instead of generating natural language text or code, Pascal is trained to model the transition dynamics of numerical series, identifying the latent mathematical rules governing a sequence prefix and extrapolating them forward.

Pascal-15M-v1.0 contains 17,716,608 parameters and was pretrained on approximately 364 million tokens of structured numerical sequences. Built strictly on native LLaMA transformer blocks with a custom 32-token digit-level vocabulary, it is designed primarily as a lightweight, clean base checkpoint for downstream specialization and fine-tuning.


Task Formulation & Inductive Scope

Given a comma-separated sequence prefix, Pascal functions as an autoregressive sequence solver. Rather than retrieving memorized series from an index, the model relies on its attention layers to identify recurrence orders, polynomial progressions, modular cycles, and rational approximations directly from raw input strings.

Representative input-output behaviors include:

  • Elementary Progressions: 1, 2, 4, 8, 16, โ†’ 32
  • Rational Convergents: 1, 1/2, 2/3, 3/5, 5/8, โ†’ 8/13 (Fibonacci convergents to the golden ratio ฯ†)
  • Interleaved Independent Streams: 2, 10, 4, 20, 8, 30, โ†’ 16
  • Coupled State Dynamics: 16, 20, 22, 22, 20, 16, 10, โ†’ 2 (latent velocity oscillator)

The objective of the pretraining regime is to provide an inductive bias for symbolic regression in latency, allowing the network to capture latent dynamic laws from raw observations.


Tokenizer & Numerical Representation

Standard language model tokenizers rely on Byte-Pair Encoding (BPE), which merges digits into arbitrary chunks based on corpus frequency (for example, tokenizing 1234 as a single unit while splitting 1235 into two). This arbitrary grouping disrupts place-value relationships and degrades arithmetic consistency.

Pascal resolves this by using an exact, lossless 32-token vocabulary:

  • Digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
  • Separators & Operators: , (delimiter), - (sign), . (decimal), and / (exact rational fraction slash)
  • Control Tokens: <bos>, <eos>, <pad>, <unk>, and reserved alignment slots

โš ๏ธ No <eos> Supervision During Training: While <eos> exists as an index slot in the tokenizer vocabulary, Pascal-15M-v1.0 was not trained with an <eos> token. All pretraining target sequences concluded directly on the final numerical digit of the pattern. Consequently, the model will never spontaneously emit an <eos> token to halt generation. During inference, stopping criteria must be handled explicitly, either by setting eos_token_id=tokenizer.convert_tokens_to_ids(",") to stop at the next number, or by specifying max_new_tokens.

By processing numbers strictly digit-by-digit or as exact rational ratios (e.g., 8/13), the model avoids floating-point rounding errors and eliminates tokenization artifacts during sequence evaluation.


Architectural Specification

Pascal-15M-v1.0 uses a standard decoder-only LLaMA architecture. The model relies entirely on native PyTorch/Hugging Face operations without custom CUDA kernels, non-standard attention mechanisms, or external repository dependencies (trust_remote_code=False).

# Pascal-15M-v1.0 Structural Specification
vocab_size: int = 32                 # Lossless digit-level numerical vocabulary
hidden_size: int = 384               # Hidden dimension
intermediate_size: int = 768         # 2.0x SwiGLU expansion ratio
num_hidden_layers: int = 12          # Sequential layer depth
num_attention_heads: int = 12        # Head dimension = 32
num_key_value_heads: int = 12        # Multi-Head Attention (MHA)
max_position_embeddings: int = 512   # Context window
rope_theta: float = 10000.0          # Base rotary frequency
# Total Trainable Parameters: 17,716,608

At under 18 million parameters, the network operates with low compute requirements and low memory footprint, scaling appropriately for its 32-token vocabulary across its ~364M token training run.


Intended Use & Downstream Adaptation

Pascal-15M-v1.0 is intended primarily as a base model for downstream fine-tuning.

While the base checkpoint exhibits stable out-of-the-box performance on standard arithmetic, polynomial, and recurrence patterns, it is designed to be adapted to specific numerical tasks, including:

  • Time-Series & Sensor Modeling: Fine-tuning on telemetry sequences, systems monitoring, or discrete sensor metrics.
  • Empirical Symbolic Regression: Adapting the model to experimental observations to predict underlying dynamic behaviors.
  • Financial Tick Sequences: Modeling discrete price levels, spread variations, and rate adjustments.
  • Dynamical Systems Simulation: Specializing the architecture on kinematics, coupled oscillations, or orbital transitions.

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "AtomixLabs/Pascal-15M-v1.0"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=False,
)

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=False,
)

prompt = "1, 1/2, 2/3, 3/5, 5/8," # answer is 8/13
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
sep_id = tokenizer.convert_tokens_to_ids(",")

outputs = model.generate(
    **inputs,
    max_new_tokens=24,
    do_sample=False,
    eos_token_id=sep_id,  # Stop immediately upon completing the next number
    pad_token_id=tokenizer.pad_token_id,
)

# prints the full sequence including the continuation: "1,1/2,2/3,3/5,5/8,8/13"
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Constrained Logit Decoding

Because the model uses a compact 32-token vocabulary, outputs can be guided during generation using grammar-constrained logit masking.

An external logit processor can enforce structural constraints:

  1. Suppressing consecutive signs or delimiters (--, .., //).
  2. Disallowing invalid operations such as division by zero (/0).
  3. Preventing open-ended fraction slashes or decimals before sequence separators.

This setup ensures that generated outputs conform to well-formed integers, decimals, or reduced fractions.


Safety Advisory, Technical Limitations & License Notice

Operational Scope & Technical Limitations

Pascal-15M-v1.0 is an experimental, specialized micro-scale foundation model (17.7M parameters). Because it operates exclusively on numerical sequences rather than natural language, its failure modes differ fundamentally from standard text models. Users and downstream developers should account for the following technical and operational considerations:

1. Vocabulary, Formatting & Context Bounds

  • Numeric-Only Scope: Pascal-15M-v1.0 has a strict 32-token numerical vocabulary. It cannot parse English instructions, hold conversations, or generate source code. Any unsupported characters are mapped to <unk> during tokenization, which introduces uninformative tokens into the context and degrades prediction quality.
  • Formatting Sensitivity: The model expects consistent comma-separated sequences without extraneous whitespace or mixed notation. Malformed delimiters, non-standard scientific notation, or inconsistent decimal precision can disrupt attention states and produce erratic continuations.
  • Context Truncation (512 Tokens): Because sequences are represented digit-by-digit, high-precision floating-point numbers or long series rapidly consume the 512-token context window. Once a sequence exceeds this boundary, earlier terms are truncated, and the model may lose track of long-range recurrence rules or initial boundary conditions.

2. Non-Formal Prediction & Spurious Extrapolation

  • Not a Symbolic Engine: Pascal-15M-v1.0 is an autoregressive neural sequence predictor, not a deterministic computer algebra system (CAS) or formal theorem prover. It approximates latent patterns statistically and does not guarantee exact analytical precision.
  • False Pattern Induction on Stochastic Data: The model is trained to detect structural continuity. When prompted with purely stochastic processes, cryptographic outputs, unmodeled exogenous shocks, or chaotic systems, it will attempt to fit and extrapolate spurious patterns rather than recognizing the input as non-deterministic noise.
  • Verification: Predictions should be independently validated by deterministic checks or domain experts before being treated as analytically reliable.

3. Downstream Fine-Tuning & Adaptation Responsibility

  • Pascal-15M-v1.0 is intended primarily as a base weight initialization for downstream specialization. Fine-tuning on domain-specific telemetry, sensor feeds, or financial ticks alters the model's inductive bias and error distributions.
  • Downstream developers are solely responsible for conducting out-of-distribution testing, calibrating confidence thresholds, and monitoring for distribution shift in fine-tuned checkpoints.

4. High-Consequence System Advisory

  • This checkpoint was developed as a foundational research asset and has not been validated, audited, or certified for safety-critical or high-consequence environments.
  • It is not designed for direct integration into unmonitored physical control loops, industrial actuation, medical diagnostics, aviation/navigation systems, or autonomous financial execution pipelines. Any deployment of raw or fine-tuned weights in environments where numerical misprediction carries physical, operational, or financial risk should incorporate strict external bounds, deterministic fallback logic, and independent oversight.

5. Apache 2.0 Disclaimer of Warranty & Limitation of Liability

Pascal-15M-v1.0 is released by AtomixLabs under the Apache License, Version 2.0. In accordance with Sections 7 and 8 of the license, the model weights and associated files are provided strictly on an "AS IS" BASIS, without warranties or conditions of any kind, either express or implied, including, without limitation, any warranties or conditions of title, non-infringement, merchantability, or fitness for a particular purpose.

To the maximum extent permitted by applicable law, in no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, shall AtomixLabs, its developers, or its contributors be liable for any direct, indirect, special, incidental, exemplary, or consequential damages of any character arising as a result of the use, fine-tuning, modification, or deployment of this model, including but not limited to damages for loss of goodwill, work stoppage, computer failure, system malfunction, data corruption, or financial loss, even if advised of the possibility of such damages. Users assume sole responsibility for determining the appropriateness of using or redistributing the model and assume all risks associated with its exercise of permissions under the license.

Citation

@model{atomixlabs_pascal_15m_v1_2026,
  author       = {AtomixLabs},
  title        = {Pascal-15M-v1.0: A Compact Foundation Transformer for Mathematical Sequence Induction},
  year         = {2026},
  publisher    = {Hugging Face},
  license      = {Apache-2.0},
  url          = {https://huggingface.co/AtomixLabs/Pascal-15M-v1.0}
}
Downloads last month
55
Safetensors
Model size
17.7M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support