Text Generation
Transformers
Safetensors
qwen3
andaman
nicobar
indic
low-resource
lora
conversational
text-generation-inference
Instructions to use docanan/dweep-1.7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use docanan/dweep-1.7b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="docanan/dweep-1.7b") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("docanan/dweep-1.7b") model = AutoModelForCausalLM.from_pretrained("docanan/dweep-1.7b", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use docanan/dweep-1.7b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "docanan/dweep-1.7b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docanan/dweep-1.7b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/docanan/dweep-1.7b
- SGLang
How to use docanan/dweep-1.7b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "docanan/dweep-1.7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docanan/dweep-1.7b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "docanan/dweep-1.7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docanan/dweep-1.7b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use docanan/dweep-1.7b with Docker Model Runner:
docker model run hf.co/docanan/dweep-1.7b
|
Download README.md from docanan/dweep-1.7b: direct link, hf CLI and curl.
- Browser
- Download file 10.7 kB
-
https://huggingface.co/docanan/dweep-1.7b/resolve/main/README.md
- Command line
-
hf download hf://docanan/dweep-1.7b/README.md
-
curl -L -o README.md https://huggingface.co/docanan/dweep-1.7b/resolve/main/README.md
10.7 kB
| license: apache-2.0 | |
| base_model: Qwen/Qwen3-1.7B | |
| datasets: | |
| - docanan/andaman-nicobar-corpus | |
| language: | |
| - en | |
| - hi | |
| - ta | |
| - bn | |
| - te | |
| - ml | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| tags: | |
| - andaman | |
| - nicobar | |
| - indic | |
| - low-resource | |
| - lora | |
| # Dweep 1.7B | |
| A LoRA fine-tune of Qwen3-1.7B adapted to the Andaman and Nicobar Islands, trained on a MacBook Air | |
| M5 (Mac17,3), macOS 27.2.1, 16 GB unified memory, in the Andaman and Nicobar Islands. | |
| ## Running it | |
| Download the GGUF file once, then answer questions with no network connection. The model, the | |
| adapter and the corpus are free to use and redistribute: Apache-2.0 for the model, CC BY-SA 4.0 for | |
| the corpus. Expect several seconds per answer on a CPU. | |
| ## What this is | |
| A small domain-adapted model for questions about the islands: geography, districts, transport, | |
| history, ecology, administration and tourism. It starts from Qwen3-1.7B (Apache-2.0) and inherits | |
| that model's general ability. | |
| ## What this is not | |
| - Not the first AI model from the Andaman and Nicobar Islands. The HF account `andamanopal` publishes | |
| reinforcement-learning and robotics policies, including a LeRobot ACT imitation-learning policy | |
| dated 2026-06-28. | |
| - Not trained from scratch and not a general assistant replacement. | |
| - Not trained on news archives. See the dataset card for the exclusion list. | |
| ## Prior work in the islands | |
| Named here so nobody has to discover it later. | |
| - **Election Mitra, 2023.** The Andaman and Nicobar Police launched an in-house chatbot trained on | |
| Election Commission handbooks. The force describes it as using machine learning. It is not published | |
| as open weights and it is not a general language model. | |
| - **`aiadvocateindia/Andaman_and_Nicobar`, 2025.** A public dataset of islands legal and regulatory | |
| PDFs, created 2025-09-30. A dataset, not a model. | |
| - **`andamanopal` robotics policies, 2026-06-28.** LeRobot ACT imitation-learning policies for an | |
| SO-101 arm, trained on CUDA. Robotics, not language. | |
| What remains unclaimed: an open, published language model for the islands. That is the claim below. | |
| ## Claim basis | |
| Census against the Hugging Face API on 2026-11-16, archived as `claim_census.txt` in the dataset repo: | |
| - `models?search=andaman` returned 12 models; `models?search=nicobar` returned 3. None carries a | |
| `text-generation` pipeline tag. | |
| - An internet search on 2026-11-17 found one island-built chatbot, Election Mitra (2023), and vendor | |
| pages that offer voice agents to the islands. Neither publishes open weights. | |
| - Task filters `text-generation`, `text2text-generation`, `feature-extraction` and | |
| `sentence-similarity` each returned 0 results for `andaman`. | |
| Within that search scope, this is the first open *language* model for the islands. | |
| ## Training | |
| | Item | Value | | |
| |---|---| | |
| | Base | Qwen/Qwen3-1.7B, 4-bit MLX quantisation | | |
| | Method | LoRA, rank 8, scale 20 (PEFT alpha 160), last 16 layers, 7 projection modules | | |
| | Data | 3163 validated instruction pairs plus 239 refusal pairs, 3275 rows in total | | |
| | Hardware | MacBook Air M5 (Mac17,3), 16 GB unified, macOS 27.2.1 | | |
| | Iterations | 1600 | | |
| | Tokens seen | 598.7 k | | |
| | Val loss | 4.191 at iter 1 to 1.105 at iter 1600 | | |
| | Throughput (median of reported steps) | 488.5 tokens/s, range 416-661 | | |
| | Checkpoints | 0000400_adapters.safetensors, 0000800_adapters.safetensors, 0001200_adapters.safetensors, 0001600_adapters.safetensors | | |
| | Sequence length / batch | 2048 / 4 | | |
| ## How to ask | |
| Use `ask.py` beside the GGUF. It retrieves three passages from the corpus and answers from them. | |
| ``` | |
| python ask.py "What is unusual about Barren Island?" | |
| ``` | |
| Direct questions to the bare model answer about 0.48 on held-out data. Questions with retrieved | |
| passages answer about 0.72, but retrieval is word based: a paraphrase that shares no vocabulary with | |
| the source can retrieve the wrong passage, and the model will then answer that wrong passage | |
| confidently. Check the sources that `ask.py` prints. | |
| ## Known limits | |
| - Bare-model answers invent detail on topics with thin corpus coverage. The refusal adapter says | |
| "I do not know" for 25 of 32 questions with no supporting text, but it still answers some. | |
| - Retrieval is BM25 over sentence windows. It fails when the question uses different words from the | |
| source. Embedding retrieval was tested and scored lower on this corpus (0.616 against 0.646). | |
| - The 112-question evaluation set was generated from corpus passages, so its wording overlaps the | |
| source. Real user questions score lower. Treat 0.72 as an upper estimate. | |
| ## Recommended configuration | |
| Use the refusal adapter with retrieval. It scores 0.558 on the 112 held-out questions and refuses 25 | |
| of 32 unanswerable questions. If you need the best raw score and never need a refusal, use the first | |
| adapter with retrieval, which scores 0.646 and refuses none. | |
| ## Evaluation | |
| Base and tuned are scored on the same held-out set of 112 in-domain questions, plus a 10-item general sanity set. A regression on the general set is reported, not hidden. | |
| Primary comparison, identical prompt handling for both models (`enable_thinking=False`), greedy | |
| decoding, 120-token budget: | |
| | set | base F1 | tuned F1 | delta | | |
| |---|---|---|---| | |
| | in-domain A&N QA (112) | 0.280 | 0.479 | +0.199 | | |
| | general sanity (10) | 0.106 | 0.308 | +0.203 | | |
| Retrieval A/B, same 112 questions, same tuned model, BM25 index over 1144 articles: | |
| | arm | in-domain F1 (112) | refusals | | |
| |---|---|---| | |
| | tuned, no context | 0.479 | 0 | | |
| | tuned, 3 retrieved passages | **0.646** | 0 | | |
| | tuned, retrieval v2 (language filter, title and entity boost) | 0.636 | 0 | | |
| Delta for the first retrieval build: +0.167. Per question: 72 improve, 38 get worse, 2 stay the same. | |
| The index places a gold answer snippet inside the retrieved passages for 70 of the 112 questions, so | |
| noise in the other 42 explains the regressions. Up to 3 passages per question are kept, and a passage | |
| is kept only when its BM25 score is above zero. | |
| The second build tried a language filter, a title-match boost and an entity boost. It scored 0.636, | |
| which is 0.010 below the first build. The extra filters did not help, and the first build stays in use. | |
| Refusal behaviour: the system prompt asked for "I do not know" when the context lacks the answer, and | |
| both arms produced 0 refusals. A prompt instruction alone did not change this behaviour. | |
| Embedding retrieval test, the step that was meant to raise recall: | |
| | retriever | recall at 3 (112) | retrieved-context F1 (112) | | |
| |---|---|---| | |
| | BM25 | 62 percent | 0.646 | | |
| | multilingual-e5-small embeddings | 63.4 percent | **0.616** | | |
| The embedding index did not beat BM25. Recall moved 1.4 points and F1 fell 0.030, so the embedding | |
| index is dropped. | |
| The real cause of the misses was the unit size, not the scorer. All 31 questions whose answer was | |
| missing from the retrieved passages come from articles that are in the corpus, and 21 of the 31 gold | |
| sentences sit inside their own source article text. Paragraph units of 180 characters or more had | |
| dropped or split those sentences. | |
| Sentence-window index, two sentences per unit with one sentence of overlap, 43279 units: | |
| | index | recall at 3 (112) | retrieved-context F1 (112) | | |
| |---|---|---| | |
| | paragraph units, 3 passages | 62 percent | 0.646 | | |
| | sentence windows, 3 passages | **75.9 percent** | **0.716** | | |
| | sentence windows, 5 passages | 77.7 percent | 0.677 | | |
| | sentence windows, 3 passages, refusal adapter | not measured | 0.648, 9 false refusals | | |
| Sentence windows are the new default. Five passages retrieve slightly more answers but dilute the | |
| prompt, so three stays. | |
| Coverage check on the same 112 questions: 81 of the 112 gold answers appear somewhere in the 5496 | |
| paragraph units, and 31 appear nowhere. With sentence windows, 85 of 112 are retrieved. The remaining | |
| 27 questions are the ceiling for the current corpus. | |
| Refusal training, a second adapter trained on the same 3163 pairs plus 239 refusal pairs (7.3 percent | |
| of 3275 rows): | |
| | config | in-domain F1 (112) | false refusals on answerable | refusals on 32 unanswerable | | |
| |---|---|---|---| | |
| | adapter without refusal pairs | 0.479 | 0/112 | 0/32 | | |
| | adapter with refusal pairs | 0.477 | 6/112 | **25/32** | | |
| | adapter with refusal pairs and retrieval | 0.558 | 6/112 | not measured | | |
| | adapter without refusal pairs and retrieval | 0.646 | 0/112 | not measured | | |
| Two findings. Refusal pairs work: 0 of 32 unanswerable questions get a refusal before training, 25 of | |
| 32 after, and answerable F1 moves only 0.002. Refusal training also costs part of the retrieval gain: | |
| the same retrieval setup scores 0.646 on the first adapter and 0.558 on the refusal adapter, a drop of | |
| 0.088. Both models answer more questions correctly with retrieved passages than without. | |
| Earlier run, default Qwen3 template (thinking mode on) for both models: | |
| | set | base F1 | tuned F1 | delta | | |
| |---|---|---|---| | |
| | in-domain A&N QA (112) | 0.115 | 0.452 | +0.337 | | |
| | general sanity (10) | 0.026 | 0.254 | +0.228 | | |
| Correction. The first run left the thinking block inside the scored output. In that run only 1 of the | |
| 112 base-model predictions contained a closing think tag, so 111 predictions carried no text after a | |
| reasoning block and scored zero. That inflated the gap. With thinking mode off for both models the | |
| honest in-domain gain is +0.199, not +0.337, and the general gain is +0.203, not +0.228. | |
| The tuned model still beats the base model on both sets. The gain is real but smaller than the first | |
| run showed. | |
| ## Limitations | |
| Small model, narrow corpus, encyclopedic register. Expect weak performance on administrative | |
| procedure, legal and medical questions. Hindi, Tamil, Bengali, Telugu and Malayalam coverage is | |
| thinner than English. Nicobarese and Great Andamanese are deliberately absent. Instruction-pair | |
| phrasing comes from a teacher model, so stylistic quirks may be inherited. | |
| ## Usage | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| tok = AutoTokenizer.from_pretrained("andaman-ai/dweep-1.7b") | |
| model = AutoModelForCausalLM.from_pretrained("andaman-ai/dweep-1.7b", torch_dtype="bfloat16") | |
| msgs = [{"role": "user", "content": "Which strait separates the Andamans from the Nicobars?"}] | |
| prompt = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False, | |
| enable_thinking=False) | |
| print(tok.decode(model.generate(**tok(prompt, return_tensors="pt"), max_new_tokens=120)[0])) | |
| ``` | |
| `enable_thinking=False` removes the thinking block. On a 20-item check it also scored slightly | |
| higher than the default template (F1 0.436 against 0.429). If your pipeline cannot pass the flag, | |
| strip text up to `</think>` before you display the answer. | |