Instructions to use scriptedvn/scriptedvn-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use scriptedvn/scriptedvn-models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf scriptedvn/scriptedvn-models:Q4_K_M # Run inference directly in the terminal: llama cli -hf scriptedvn/scriptedvn-models:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf scriptedvn/scriptedvn-models:Q4_K_M # Run inference directly in the terminal: llama cli -hf scriptedvn/scriptedvn-models:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf scriptedvn/scriptedvn-models:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf scriptedvn/scriptedvn-models:Q4_K_M
Use Docker
docker model run hf.co/scriptedvn/scriptedvn-models:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use scriptedvn/scriptedvn-models with Ollama:
ollama run hf.co/scriptedvn/scriptedvn-models:Q4_K_M
- Unsloth Desktop
- Pi
How to use scriptedvn/scriptedvn-models with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf scriptedvn/scriptedvn-models:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "scriptedvn/scriptedvn-models:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use scriptedvn/scriptedvn-models with Docker Model Runner:
docker model run hf.co/scriptedvn/scriptedvn-models:Q4_K_M
- Lemonade
How to use scriptedvn/scriptedvn-models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull scriptedvn/scriptedvn-models:Q4_K_M
Run and chat with the model
lemonade run user.scriptedvn-models-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use scriptedvn/scriptedvn-models with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf scriptedvn/scriptedvn-models:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default scriptedvn/scriptedvn-models:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use scriptedvn/scriptedvn-models with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf scriptedvn/scriptedvn-models:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "scriptedvn/scriptedvn-models:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download madlad400_3b/model/README.md from scriptedvn/scriptedvn-models: direct link, hf CLI and curl.
- Browser
- Download file 9.67 kB
-
https://huggingface.co/scriptedvn/scriptedvn-models/resolve/main/madlad400_3b/model/README.md
- Command line
-
hf download hf://scriptedvn/scriptedvn-models/madlad400_3b/model/README.md
-
curl -L -o README.md https://huggingface.co/scriptedvn/scriptedvn-models/resolve/main/madlad400_3b/model/README.md
9.67 kB
| license: apache-2.0 | |
| language: | |
| - multilingual | |
| - en | |
| - ru | |
| - es | |
| - fr | |
| - de | |
| - it | |
| - pt | |
| - pl | |
| - nl | |
| - vi | |
| - tr | |
| - sv | |
| - id | |
| - ro | |
| - cs | |
| - zh | |
| - hu | |
| - ja | |
| - th | |
| - fi | |
| - fa | |
| - uk | |
| - da | |
| - el | |
| - "no" | |
| - bg | |
| - sk | |
| - ko | |
| - ar | |
| - lt | |
| - ca | |
| - sl | |
| - he | |
| - et | |
| - lv | |
| - hi | |
| - sq | |
| - ms | |
| - az | |
| - sr | |
| - ta | |
| - hr | |
| - kk | |
| - is | |
| - ml | |
| - mr | |
| - te | |
| - af | |
| - gl | |
| - fil | |
| - be | |
| - mk | |
| - eu | |
| - bn | |
| - ka | |
| - mn | |
| - bs | |
| - uz | |
| - ur | |
| - sw | |
| - yue | |
| - ne | |
| - kn | |
| - kaa | |
| - gu | |
| - si | |
| - cy | |
| - eo | |
| - la | |
| - hy | |
| - ky | |
| - tg | |
| - ga | |
| - mt | |
| - my | |
| - km | |
| - tt | |
| - so | |
| - ku | |
| - ps | |
| - pa | |
| - rw | |
| - lo | |
| - ha | |
| - dv | |
| - fy | |
| - lb | |
| - ckb | |
| - mg | |
| - gd | |
| - am | |
| - ug | |
| - ht | |
| - grc | |
| - hmn | |
| - sd | |
| - jv | |
| - mi | |
| - tk | |
| - ceb | |
| - yi | |
| - ba | |
| - fo | |
| - or | |
| - xh | |
| - su | |
| - kl | |
| - ny | |
| - sm | |
| - sn | |
| - co | |
| - zu | |
| - ig | |
| - yo | |
| - pap | |
| - st | |
| - haw | |
| - as | |
| - oc | |
| - cv | |
| - lus | |
| - tet | |
| - gsw | |
| - sah | |
| - br | |
| - rm | |
| - sa | |
| - bo | |
| - om | |
| - se | |
| - ce | |
| - cnh | |
| - ilo | |
| - hil | |
| - udm | |
| - os | |
| - lg | |
| - ti | |
| - vec | |
| - ts | |
| - tyv | |
| - kbd | |
| - ee | |
| - iba | |
| - av | |
| - kha | |
| - to | |
| - tn | |
| - nso | |
| - fj | |
| - zza | |
| - ak | |
| - ada | |
| - otq | |
| - dz | |
| - bua | |
| - cfm | |
| - ln | |
| - chm | |
| - gn | |
| - krc | |
| - wa | |
| - hif | |
| - yua | |
| - srn | |
| - war | |
| - rom | |
| - bik | |
| - pam | |
| - sg | |
| - lu | |
| - ady | |
| - kbp | |
| - syr | |
| - ltg | |
| - myv | |
| - iso | |
| - kac | |
| - bho | |
| - ay | |
| - kum | |
| - qu | |
| - za | |
| - pag | |
| - ngu | |
| - ve | |
| - pck | |
| - zap | |
| - tyz | |
| - hui | |
| - bbc | |
| - tzo | |
| - tiv | |
| - ksd | |
| - gom | |
| - min | |
| - ang | |
| - nhe | |
| - bgp | |
| - nzi | |
| - nnb | |
| - nv | |
| - zxx | |
| - bci | |
| - kv | |
| - new | |
| - mps | |
| - alt | |
| - meu | |
| - bew | |
| - fon | |
| - iu | |
| - abt | |
| - mgh | |
| - mnw | |
| - tvl | |
| - dov | |
| - tlh | |
| - ho | |
| - kw | |
| - mrj | |
| - meo | |
| - crh | |
| - mbt | |
| - emp | |
| - ace | |
| - ium | |
| - mam | |
| - gym | |
| - mai | |
| - crs | |
| - pon | |
| - ubu | |
| - fip | |
| - quc | |
| - gv | |
| - kj | |
| - btx | |
| - ape | |
| - chk | |
| - rcf | |
| - shn | |
| - tzh | |
| - mdf | |
| - ppk | |
| - ss | |
| - gag | |
| - cab | |
| - kri | |
| - seh | |
| - ibb | |
| - tbz | |
| - bru | |
| - enq | |
| - ach | |
| - cuk | |
| - kmb | |
| - wo | |
| - kek | |
| - qub | |
| - tab | |
| - bts | |
| - kos | |
| - rwo | |
| - cak | |
| - tuc | |
| - bum | |
| - cjk | |
| - gil | |
| - stq | |
| - tsg | |
| - quh | |
| - mak | |
| - arn | |
| - ban | |
| - jiv | |
| - sja | |
| - yap | |
| - tcy | |
| - toj | |
| - twu | |
| - xal | |
| - amu | |
| - rmc | |
| - hus | |
| - nia | |
| - kjh | |
| - bm | |
| - guh | |
| - mas | |
| - acf | |
| - dtp | |
| - ksw | |
| - bzj | |
| - din | |
| - zne | |
| - mad | |
| - msi | |
| - mag | |
| - mkn | |
| - kg | |
| - lhu | |
| - ch | |
| - qvi | |
| - mh | |
| - djk | |
| - sus | |
| - mfe | |
| - srm | |
| - dyu | |
| - ctu | |
| - gui | |
| - pau | |
| - inb | |
| - bi | |
| - mni | |
| - guc | |
| - jam | |
| - wal | |
| - jac | |
| - bas | |
| - gor | |
| - skr | |
| - nyu | |
| - noa | |
| - sda | |
| - gub | |
| - nog | |
| - cni | |
| - teo | |
| - tdx | |
| - sxn | |
| - rki | |
| - nr | |
| - frp | |
| - alz | |
| - taj | |
| - lrc | |
| - cce | |
| - rn | |
| - jvn | |
| - hvn | |
| - nij | |
| - dwr | |
| - izz | |
| - msm | |
| - bus | |
| - ktu | |
| - chr | |
| - maz | |
| - tzj | |
| - suz | |
| - knj | |
| - bim | |
| - gvl | |
| - bqc | |
| - tca | |
| - pis | |
| - prk | |
| - laj | |
| - mel | |
| - qxr | |
| - niq | |
| - ahk | |
| - shp | |
| - hne | |
| - spp | |
| - koi | |
| - krj | |
| - quf | |
| - luz | |
| - agr | |
| - tsc | |
| - mqy | |
| - gof | |
| - gbm | |
| - miq | |
| - dje | |
| - awa | |
| - bjj | |
| - qvz | |
| - sjp | |
| - tll | |
| - raj | |
| - kjg | |
| - bgz | |
| - quy | |
| - cbk | |
| - akb | |
| - oj | |
| - ify | |
| - mey | |
| - ks | |
| - cac | |
| - brx | |
| - qup | |
| - syl | |
| - jax | |
| - ff | |
| - ber | |
| - tks | |
| - trp | |
| - mrw | |
| - adh | |
| - smt | |
| - srr | |
| - ffm | |
| - qvc | |
| - mtr | |
| - ann | |
| - kaa | |
| - aa | |
| - noe | |
| - nut | |
| - gyn | |
| - kwi | |
| - xmm | |
| - msb | |
| library_name: transformers | |
| tags: | |
| - text2text-generation | |
| - text-generation-inference | |
| datasets: | |
| - allenai/MADLAD-400 | |
| pipeline_tag: translation | |
| widget: | |
| - text: "<2en> Como vai, amigo?" | |
| example_title: "Translation to English" | |
| - text: "<2de> Do you speak German?" | |
| example_title: "Translation to German" | |
| # Model Card for MADLAD-400-3B-MT | |
| # Table of Contents | |
| 0. [TL;DR](#TL;DR) | |
| 1. [Model Details](#model-details) | |
| 2. [Usage](#usage) | |
| 3. [Uses](#uses) | |
| 4. [Bias, Risks, and Limitations](#bias-risks-and-limitations) | |
| 5. [Training Details](#training-details) | |
| 6. [Evaluation](#evaluation) | |
| 7. [Environmental Impact](#environmental-impact) | |
| 8. [Citation](#citation) | |
| # TL;DR | |
| MADLAD-400-3B-MT is a multilingual machine translation model based on the T5 architecture that was | |
| trained on 1 trillion tokens covering over 450 languages using publicly available data. | |
| It is competitive with models that are significantly larger. | |
| **Disclaimer**: [Juarez Bochi](https://huggingface.co/jbochi), who was not involved in this research, converted | |
| the original weights and wrote the contents of this model card based on the original paper and Flan-T5. | |
| # Model Details | |
| ## Model Description | |
| - **Model type:** Language model | |
| - **Language(s) (NLP):** Multilingual (400+ languages) | |
| - **License:** Apache 2.0 | |
| - **Related Models:** [All MADLAD-400 Checkpoints](https://huggingface.co/models?search=madlad) | |
| - **Original Checkpoints:** [All Original MADLAD-400 Checkpoints](https://github.com/google-research/google-research/tree/master/madlad_400) | |
| - **Resources for more information:** | |
| - [Research paper](https://arxiv.org/abs/2309.04662) | |
| - [GitHub Repo](https://github.com/google-research/t5x) | |
| - [Hugging Face MADLAD-400 Docs (Similar to T5) ](https://huggingface.co/docs/transformers/model_doc/MADLAD-400) - [Pending PR](https://github.com/huggingface/transformers/pull/27471) | |
| # Usage | |
| Find below some example scripts on how to use the model: | |
| ## Using the Pytorch model with `transformers` | |
| ### Running the model on a CPU or GPU | |
| <details> | |
| <summary> Click to expand </summary> | |
| First, install the Python packages that are required: | |
| `pip install transformers accelerate sentencepiece` | |
| ```python | |
| from transformers import T5ForConditionalGeneration, T5Tokenizer | |
| model_name = 'jbochi/madlad400-3b-mt' | |
| model = T5ForConditionalGeneration.from_pretrained(model_name, device_map="auto") | |
| tokenizer = T5Tokenizer.from_pretrained(model_name) | |
| text = "<2pt> I love pizza!" | |
| input_ids = tokenizer(text, return_tensors="pt").input_ids.to(model.device) | |
| outputs = model.generate(input_ids=input_ids) | |
| tokenizer.decode(outputs[0], skip_special_tokens=True) | |
| # Eu adoro pizza! | |
| ``` | |
| </details> | |
| ## Running the model with Candle | |
| <details> | |
| <summary> Click to expand </summary> | |
| Usage with [candle](https://github.com/huggingface/candle): | |
| ```bash | |
| $ cargo run --example t5 --release -- \ | |
| --model-id "jbochi/madlad400-3b-mt" \ | |
| --prompt "<2de> How are you, my friend?" \ | |
| --decode --temperature 0 | |
| ``` | |
| We also provide a quantized model (1.65 GB vs the original 11.8 GB file): | |
| ``` | |
| cargo run --example quantized-t5 --release -- \ | |
| --model-id "jbochi/madlad400-3b-mt" --weight-file "model-q4k.gguf" \ | |
| --prompt "<2de> How are you, my friend?" \ | |
| --temperature 0 | |
| ... | |
| Wie geht es dir, mein Freund? | |
| ``` | |
| </details> | |
| # Uses | |
| ## Direct Use and Downstream Use | |
| > Primary intended uses: Machine Translation and multilingual NLP tasks on over 400 languages. | |
| > Primary intended users: Research community. | |
| ## Out-of-Scope Use | |
| > These models are trained on general domain data and are therefore not meant to | |
| > work on domain-specific models out-of-the box. Moreover, these research models have not been assessed | |
| > for production usecases. | |
| # Bias, Risks, and Limitations | |
| > We note that we evaluate on only 204 of the languages supported by these models and on machine translation | |
| > and few-shot machine translation tasks. Users must consider use of this model carefully for their own | |
| > usecase. | |
| ## Ethical considerations and risks | |
| > We trained these models with MADLAD-400 and publicly available data to create baseline models that | |
| > support NLP for over 400 languages, with a focus on languages underrepresented in large-scale corpora. | |
| > Given that these models were trained with web-crawled datasets that may contain sensitive, offensive or | |
| > otherwise low-quality content despite extensive preprocessing, it is still possible that these issues to the | |
| > underlying training data may cause differences in model performance and toxic (or otherwise problematic) | |
| > output for certain domains. Moreover, large models are dual use technologies that have specific risks | |
| > associated with their use and development. We point the reader to surveys such as those written by | |
| > Weidinger et al. or Bommasani et al. for a more detailed discussion of these risks, and to Liebling | |
| > et al. for a thorough discussion of the risks of machine translation systems. | |
| ## Known Limitations | |
| More information needed | |
| ## Sensitive Use: | |
| More information needed | |
| # Training Details | |
| > We train models of various sizes: a 3B, 32-layer parameter model, | |
| > a 7.2B 48-layer parameter model and a 10.7B 32-layer parameter model. | |
| > We share all parameters of the model across language pairs, | |
| > and use a Sentence Piece Model with 256k tokens shared on both the encoder and decoder | |
| > side. Each input sentence has a <2xx> token prepended to the source sentence to indicate the target | |
| > language. | |
| See the [research paper](https://arxiv.org/pdf/2309.04662.pdf) for further details. | |
| ## Training Data | |
| > For both the machine translation and language model, MADLAD-400 is used. For the machine translation | |
| > model, a combination of parallel datasources covering 157 languages is also used. Further details are | |
| > described in the [paper](https://arxiv.org/pdf/2309.04662.pdf). | |
| ## Training Procedure | |
| See the [research paper](https://arxiv.org/pdf/2309.04662.pdf) for further details. | |
| # Evaluation | |
| ## Testing Data, Factors & Metrics | |
| > For evaluation, we used WMT, NTREX, Flores-200 and Gatones datasets as described in Section 4.3 in the [paper](https://arxiv.org/pdf/2309.04662.pdf). | |
| > The translation quality of this model varies based on language, as seen in the paper, and likely varies on | |
| > domain, though we have not assessed this. | |
| ## Results | |
|  | |
|  | |
|  | |
| See the [research paper](https://arxiv.org/pdf/2309.04662.pdf) for further details. | |
| # Environmental Impact | |
| More information needed | |
| # Citation | |
| **BibTeX:** | |
| ```bibtex | |
| @misc{kudugunta2023madlad400, | |
| title={MADLAD-400: A Multilingual And Document-Level Large Audited Dataset}, | |
| author={Sneha Kudugunta and Isaac Caswell and Biao Zhang and Xavier Garcia and Christopher A. Choquette-Choo and Katherine Lee and Derrick Xin and Aditya Kusupati and Romi Stella and Ankur Bapna and Orhan Firat}, | |
| year={2023}, | |
| eprint={2309.04662}, | |
| archivePrefix={arXiv}, | |
| primaryClass={cs.CL} | |
| } | |
| ``` | |