Neurograf-6B-Translate

A 6-billion-parameter English-to-Romanian translation model, built on a Romanian-first language model trained from scratch.

Neurograf-6B-Translate is a specialized translation model developed as part of the Neurograf research initiative.

The model builds upon Neurograf-6B-Base, pretrained primarily on Romanian text, and was subsequently fine-tuned on approximately 300,000 English–Romanian translation pairs.

Its objective is straightforward: English in, Romanian out.

Unlike general-purpose multilingual models, Neurograf-6B-Translate starts from a Romanian-first pretrained model and specializes it for translating English content into Romanian.

Model details

Property	Value
Architecture	Modified NanoChat Transformer
Parameters	Approximately 6B
Transformer layers	30
Hidden dimension	3,840
Attention heads	30
KV heads	30
Head dimension	128
Context length	2,048 tokens
Vocabulary	80,192 tokens
Tokenizer	Custom Romanian-first BPE
Activation	ReLU²
Normalization	RMSNorm, QK normalization
Positional embeddings	RoPE
Base model	Neurograf-6B-Base
Fine-tuning dataset	EN-RO-Translation-SFT-300K
Translation direction	English → Romanian

The base model was pretrained on approximately 131 billion tokens, primarily from Romanian-language sources.

The translation model retains the same underlying architecture and tokenizer.

Training dataset

Neurograf-6B-Translate was fine-tuned using approximately 300,000 English-to-Romanian translation pairs.

Dataset: Neurograf/EN-RO-Translation-SFT-300K

The dataset consists of paired English source texts and corresponding Romanian translations.

The translation fine-tuning stage adapts the Romanian-first base model to interpret English source text and generate Romanian output.

This release is specialized for English → Romanian, not bidirectional translation.

Inference

The model is distributed in Hugging Face Transformers format with sharded Safetensors weights.

It can be loaded using Transformers or served using vLLM.

Option 1 — Hugging Face Transformers

Install dependencies:

pip install "transformers>=5.3.0" torch accelerate

Download the model:

git lfs install
git clone https://huggingface.co/Neurograf/Neurograf-6B-Translate
cd Neurograf-6B-Translate

Example inference:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Neurograf/Neurograf-6B-Translate"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)
model.eval()

english = "Artificial intelligence is transforming the way we interact with technology."

# Neurograf native conversation tokens
prompt_ids = (
    [80174, 80175]
    + tokenizer.encode(english, add_special_tokens=False)
    + [80176, 80177]
)

input_ids = torch.tensor([prompt_ids], device=model.device)

with torch.inference_mode():
    output = model.generate(
        input_ids,
        max_new_tokens=256,
        do_sample=False,
        eos_token_id=[80178, 80191],
        pad_token_id=1,
    )

translation = tokenizer.decode(
    output[0, input_ids.shape[1]:],
    skip_special_tokens=True,
)

print(translation)

The model uses the NanoChatForCausalLM architecture, supported natively by Transformers 5.3.0.

Option 2 — vLLM

For high-throughput translation, the model can be served using vLLM.

Install a compatible vLLM environment:

pip install -U vllm requests transformers

Start the inference server:

CUDA_VISIBLE_DEVICES=0 vllm serve Neurograf/Neurograf-6B-Translate \
  --host 127.0.0.1 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.95 \
  --max-model-len 2048 \
  --max-num-batched-tokens 1024 \
  > vllm_gpu0.log 2>&1 &

The example uses a single GPU and starts a local OpenAI-compatible API server on port 8000.

Send a translation request:

import requests
from transformers import AutoTokenizer

model_id = "Neurograf/Neurograf-6B-Translate"
tokenizer = AutoTokenizer.from_pretrained(model_id)

english = "Scientific research helps us understand the natural world."

prompt_ids = (
    [80174, 80175]
    + tokenizer.encode(english, add_special_tokens=False)
    + [80176, 80177]
)

response = requests.post(
    "http://127.0.0.1:8000/v1/completions",
    json={
        "model": model_id,
        "prompt": prompt_ids,
        "temperature": 0,
        "max_tokens": 256,
        "stop_token_ids": [80178, 80191],
    },
    timeout=300,
)

response.raise_for_status()
print(response.json()["choices"][0]["text"].strip())

For larger datasets, vLLM supports continuous batching and concurrent requests, allowing multiple documents to be translated efficiently.

Prompt format

The model was fine-tuned for direct translation using Neurograf's native conversation structure.

The input is the English text itself, without requiring an additional instruction such as Translate the following text into Romanian.

The expected token sequence is:

<|bos|><|user_start|>English source text<|user_end|><|assistant_start|>

The model generates the Romanian translation.

This format is used by both inference examples above.

Intended use

Neurograf-6B-Translate is intended for English-to-Romanian translation, including experimentation with scientific, educational, technical, and general-purpose text.

Potential applications include corpus translation, Romanian-language data generation, research on translation-specialized language models, and translation pipelines for Romanian AI development.

Limitations

Neurograf-6B-Translate is an experimental research model.

• Translation quality can vary across domains, terminology, and document lengths.
• The model may omit, alter, or hallucinate content.
• Long documents may need to be split into smaller segments due to the 2,048-token context window.
• The model is optimized for English-to-Romanian translation rather than general conversational assistance.
• Human verification is recommended for high-stakes or publication-quality translations.

No benchmark scores are claimed in this model card.

Neurograf model family

Neurograf is an open research initiative focused on developing Romanian language models from scratch.

The Neurograf v1 base family includes:

Model	Parameters
Neurograf-160M-Base	160M
Neurograf-560M-Base	560M
Neurograf-1.5B-Base	1.5B
Neurograf-3.2B-Base	3.2B
Neurograf-6B-Base	6B

All five base models share the same Romanian-first tokenizer and architectural design.

Neurograf-6B-Translate demonstrates how a Romanian-first base model can be adapted to a specialized cross-lingual task.

Project

Neurograf — Developing monolingual Romanian language models.

Explore Neurograf on Hugging Face

---

An open research effort to advance Romanian-language artificial intelligence.