Text Generation
Transformers
Safetensors
qwen3
andaman
nicobar
indic
low-resource
lora
conversational
text-generation-inference
Instructions to use docanan/dweep-1.7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use docanan/dweep-1.7b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="docanan/dweep-1.7b") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("docanan/dweep-1.7b") model = AutoModelForCausalLM.from_pretrained("docanan/dweep-1.7b", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use docanan/dweep-1.7b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "docanan/dweep-1.7b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docanan/dweep-1.7b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/docanan/dweep-1.7b
- SGLang
How to use docanan/dweep-1.7b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "docanan/dweep-1.7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docanan/dweep-1.7b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "docanan/dweep-1.7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "docanan/dweep-1.7b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use docanan/dweep-1.7b with Docker Model Runner:
docker model run hf.co/docanan/dweep-1.7b
Rebuild from the refusal adapter: refuses when the corpus has no answer
Browse files- README.md +28 -0
- model-00001-of-00002.safetensors +1 -1
- model-00002-of-00002.safetensors +1 -1
README.md
CHANGED
|
@@ -25,6 +25,12 @@ tags:
|
|
| 25 |
A LoRA fine-tune of Qwen3-1.7B adapted to the Andaman and Nicobar Islands, trained on a MacBook Air
|
| 26 |
M5 (Mac17,3), macOS 27.2.1, 16 GB unified memory, in the Andaman and Nicobar Islands.
|
| 27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
## What this is
|
| 29 |
|
| 30 |
A small domain-adapted model for questions about the islands: geography, districts, transport,
|
|
@@ -81,6 +87,28 @@ Within that search scope, this is the first open *language* model for the island
|
|
| 81 |
| Checkpoints | 0000400_adapters.safetensors, 0000800_adapters.safetensors, 0001200_adapters.safetensors, 0001600_adapters.safetensors |
|
| 82 |
| Sequence length / batch | 2048 / 4 |
|
| 83 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
## Recommended configuration
|
| 85 |
|
| 86 |
Use the refusal adapter with retrieval. It scores 0.558 on the 112 held-out questions and refuses 25
|
|
|
|
| 25 |
A LoRA fine-tune of Qwen3-1.7B adapted to the Andaman and Nicobar Islands, trained on a MacBook Air
|
| 26 |
M5 (Mac17,3), macOS 27.2.1, 16 GB unified memory, in the Andaman and Nicobar Islands.
|
| 27 |
|
| 28 |
+
## Running it
|
| 29 |
+
|
| 30 |
+
Download the GGUF file once, then answer questions with no network connection. The model, the
|
| 31 |
+
adapter and the corpus are free to use and redistribute: Apache-2.0 for the model, CC BY-SA 4.0 for
|
| 32 |
+
the corpus. Expect several seconds per answer on a CPU.
|
| 33 |
+
|
| 34 |
## What this is
|
| 35 |
|
| 36 |
A small domain-adapted model for questions about the islands: geography, districts, transport,
|
|
|
|
| 87 |
| Checkpoints | 0000400_adapters.safetensors, 0000800_adapters.safetensors, 0001200_adapters.safetensors, 0001600_adapters.safetensors |
|
| 88 |
| Sequence length / batch | 2048 / 4 |
|
| 89 |
|
| 90 |
+
## How to ask
|
| 91 |
+
|
| 92 |
+
Use `ask.py` beside the GGUF. It retrieves three passages from the corpus and answers from them.
|
| 93 |
+
|
| 94 |
+
```
|
| 95 |
+
python ask.py "What is unusual about Barren Island?"
|
| 96 |
+
```
|
| 97 |
+
|
| 98 |
+
Direct questions to the bare model answer about 0.48 on held-out data. Questions with retrieved
|
| 99 |
+
passages answer about 0.72, but retrieval is word based: a paraphrase that shares no vocabulary with
|
| 100 |
+
the source can retrieve the wrong passage, and the model will then answer that wrong passage
|
| 101 |
+
confidently. Check the sources that `ask.py` prints.
|
| 102 |
+
|
| 103 |
+
## Known limits
|
| 104 |
+
|
| 105 |
+
- Bare-model answers invent detail on topics with thin corpus coverage. The refusal adapter says
|
| 106 |
+
"I do not know" for 25 of 32 questions with no supporting text, but it still answers some.
|
| 107 |
+
- Retrieval is BM25 over sentence windows. It fails when the question uses different words from the
|
| 108 |
+
source. Embedding retrieval was tested and scored lower on this corpus (0.616 against 0.646).
|
| 109 |
+
- The 112-question evaluation set was generated from corpus passages, so its wording overlaps the
|
| 110 |
+
source. Real user questions score lower. Treat 0.72 as an upper estimate.
|
| 111 |
+
|
| 112 |
## Recommended configuration
|
| 113 |
|
| 114 |
Use the refusal adapter with retrieval. It scores 0.558 on the 112 held-out questions and refuses 25
|
model-00001-of-00002.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 1981418432
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3902f2dd492b17b75bdf8cabb27547d3e7eba815d756ceff9112857f18b9bc11
|
| 3 |
size 1981418432
|
model-00002-of-00002.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 1459766976
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:93928e86d8e2045c5f7f44b56684dbfd6f629c3adc2439fd85d331bc05235334
|
| 3 |
size 1459766976
|