Translation
Transformers
PyTorch
English
Hindi
viuai
viutranslate
sarus-500m
nmt
english-to-hindi
hindi-to-english
indic
devanagari
bfloat16
zero-synthetic
Instructions to use ViuAI/ViuTranslate with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ViuAI/ViuTranslate with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="ViuAI/ViuTranslate")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ViuAI/ViuTranslate", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Fix: download mapping for scripts/train.py
Browse files- runners/run_5090.py +13 -7
runners/run_5090.py
CHANGED
|
@@ -45,12 +45,18 @@ os.makedirs("checkpoints", exist_ok=True)
|
|
| 45 |
|
| 46 |
# 3. Fetch Architecture & Engine from ViuAI/ViuTranslate
|
| 47 |
print("\n📥 [2/5] Fetching model architecture and training engine from ViuAI/ViuTranslate...")
|
| 48 |
-
code_files = [
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
print(" ✅ Architecture files ready.")
|
| 55 |
|
| 56 |
# 4. Fetch 100K+ Real Dataset Shards from ViuAI/ViuTranslate-Data
|
|
@@ -81,7 +87,7 @@ print(" ✅ Base weights ready.")
|
|
| 81 |
# bf16 enabled, Fused AdamW, TF32 matmuls
|
| 82 |
print("\n⚡ [5/5] Launching RTX 5090 Ultra-Tuned Training...")
|
| 83 |
cmd = [
|
| 84 |
-
sys.executable, "
|
| 85 |
"--data_dir", "data/tokenized",
|
| 86 |
"--base_ckpt", base_ckpt,
|
| 87 |
"--output_dir", "checkpoints",
|
|
|
|
| 45 |
|
| 46 |
# 3. Fetch Architecture & Engine from ViuAI/ViuTranslate
|
| 47 |
print("\n📥 [2/5] Fetching model architecture and training engine from ViuAI/ViuTranslate...")
|
| 48 |
+
code_files = [
|
| 49 |
+
("model.py", "model.py"),
|
| 50 |
+
("config.py", "config.py"),
|
| 51 |
+
("tokenizer.json", "tokenizer.json"),
|
| 52 |
+
("scripts/train.py", "train.py"),
|
| 53 |
+
("scripts/inference.py", "inference.py"),
|
| 54 |
+
]
|
| 55 |
+
for src, dst in code_files:
|
| 56 |
+
if not os.path.exists(dst):
|
| 57 |
+
print(f" • Fetching {src} -> {dst}...")
|
| 58 |
+
p = hf_hub_download(repo_id="ViuAI/ViuTranslate", filename=src, token=HF_TOKEN)
|
| 59 |
+
shutil.copy(p, dst)
|
| 60 |
print(" ✅ Architecture files ready.")
|
| 61 |
|
| 62 |
# 4. Fetch 100K+ Real Dataset Shards from ViuAI/ViuTranslate-Data
|
|
|
|
| 87 |
# bf16 enabled, Fused AdamW, TF32 matmuls
|
| 88 |
print("\n⚡ [5/5] Launching RTX 5090 Ultra-Tuned Training...")
|
| 89 |
cmd = [
|
| 90 |
+
sys.executable, "train.py",
|
| 91 |
"--data_dir", "data/tokenized",
|
| 92 |
"--base_ckpt", base_ckpt,
|
| 93 |
"--output_dir", "checkpoints",
|