README.md

metadata

language:
  - en
  - ar
pipeline_tag: text-generation
tags:
  - operator
  - hajerilm
  - cultureyes
  - arabic
  - english

Operator 4B

Operator is an Arabic-English model from the HajeriLM family, published by Cultureyes.

العربية

Operator 4B نموذج لغوي عربي وإنجليزي من عائلة HajeriLM، تنشره Cultureyes. أشارك أوزانه ليتمكن الباحثون والمطورون من فحصها، ومواصلة التدريب، وتخصيص النموذج، والبناء عليه. طُوّر هذا العمل المستقل في الإمارات مع التركيز على الاستفادة الجيدة من وقت الحوسبة والموارد المتاحة.

I’m sharing this work in the spirit of the UAE’s commitment to innovation. I built it by working with the compute I had and repeatedly asking how to make each training step count. I’m sharing the weights so other people can inspect them, continue training, specialize them, and build their own work on top.

The model is an open starting point from an independent effort in the UAE, shared for people who want to explore what they can build from it.

Languages

Operator was trained for use in English and Arabic. The model card and examples should be read with that scope in mind; this statement does not imply equal performance across languages or tasks.

دُرّب Operator للاستخدام باللغتين العربية والإنجليزية. ولا يعني ذلك تكافؤ الأداء في اللغتين أو في جميع المهام.

Model

• Model family: HajeriLM
• Publisher: Cultureyes
• Format: BF16 safetensors
• Parameter scale: approximately 4B
• Architecture: 32 layers, hidden size 2048, 16 attention heads, 2 key/value heads
• Position encoding: RoPE with theta 500,000; maximum configured context 32,768 tokens
• Attention gates: enabled
• License: not specified; reuse terms have not been declared

يتكوّن النموذج من 32 طبقة، وبُعد مخفي قدره 2048، ويستخدم 16 رأس انتباه ورأسي مفتاح/قيمة. يعتمد RoPE بقيمة theta تساوي 500,000، مع سياق أقصى مضبوط على 32,768 رمزًا، وبوابات انتباه مفعّلة.

Local inference

Two GGUF files are available in this repository:

• BF16 GGUF — full-precision bfloat16 weights.
• Q8_0 GGUF — 8-bit quantized weights for a smaller download and memory footprint.

These files use the Hajeri model architecture. They require a llama.cpp build that includes Hajeri architecture support; stock llama.cpp builds may not recognize them. The conversion was checked against the source checkpoint and tokenizer. The Q8_0 file is quantized, so its values are not bit-for-bit identical to BF16.

يوجد في هذا المستودع ملفان بصيغة GGUF:

• نسخة BF16 بأوزان bfloat16.
• نسخة Q8_0 مكمّمة إلى 8 بت لتقليل حجم التنزيل والذاكرة المطلوبة.

تستخدم هذه الملفات معمارية Hajeri، وتتطلب إصدارًا من llama.cpp يدعم هذه المعمارية؛ قد لا تتعرف عليها الإصدارات القياسية. تمت مراجعة التحويل مقارنةً بملف النموذج الأصلي ومقسّم الرموز. نسخة Q8_0 مكمّمة، لذلك لا تطابق قيم BF16 عنصرًا بعنصر.

Training approach

The training workflow was built around a practical question: how can limited compute be used more carefully? The approach tracks cross-entropy per example, can stop spending updates on examples below a chosen loss threshold, saves resumable checkpoints, and adapts the learning rate from training loss. The aim is to focus training effort where it is still useful and make model building more accessible beyond large clusters.

Intended use

Use this checkpoint for research, experimentation, continued pretraining or fine-tuning, and local model development. Test it on your own tasks before relying on it in a product or consequential setting.

The weights are publicly downloadable, but no license grant is declared in this release. Reuse and redistribution terms remain unresolved.

Citation and attribution

Please refer to this model as Operator, from the HajeriLM family, published by Cultureyes. Preserve the model name and source link when sharing derivatives.

Release notes

Further development notes and the journal will document how the work evolves.