Instructions to use A-LLM/allm_instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use A-LLM/allm_instruct with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: llama cli -hf A-LLM/allm_instruct
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: llama cli -hf A-LLM/allm_instruct
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: ./build/bin/llama cli -hf A-LLM/allm_instruct
Use Docker
docker model run hf.co/A-LLM/allm_instruct
- Llama new
- LM Studio
- Jan
- Ollama
How to use A-LLM/allm_instruct with Ollama:
ollama run hf.co/A-LLM/allm_instruct
- Unsloth Desktop
- Docker Model Runner
How to use A-LLM/allm_instruct with Docker Model Runner:
docker model run hf.co/A-LLM/allm_instruct
- Lemonade
How to use A-LLM/allm_instruct with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull A-LLM/allm_instruct
Run and chat with the model
lemonade run user.allm_instruct-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,22 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
|
| 6 |
+
Model Card for Allm-7B-Instruct-v0.1
|
| 7 |
+
Model Overview
|
| 8 |
+
Model Name: Allm-7B-Instruct-v0.1
|
| 9 |
+
Model Type: Large Language Model (LLM)
|
| 10 |
+
Task: Instruction-based Language Generation
|
| 11 |
+
Version: v0.1
|
| 12 |
+
Date: [Release Date]
|
| 13 |
+
|
| 14 |
+
Description:
|
| 15 |
+
Allm-7B-Instruct-v0.1 is a large language model designed to perform tasks based on instructions. This version introduces several enhancements over its predecessor, including a 32k context window (upgraded from the 8k context in the previous version), a Rope-theta value of 1e6, and the elimination of Sliding-Window Attention. For a detailed explanation of these features and the model’s capabilities, please refer to our [paper] and [release blog post].
|
| 16 |
+
|
| 17 |
+
Instruction Format
|
| 18 |
+
To utilize the instruction fine-tuning feature of this model, wrap your prompts with [INST] and [/INST] tokens. Ensure that the very first instruction starts with a sentence ID, while subsequent instructions should not. The response from the assistant will be concluded with an end-of-sentence token ID.
|
| 19 |
+
|
| 20 |
+
Limitations
|
| 21 |
+
The Allm-7B-Instruct model serves as an initial demonstration of how the base model can be fine-tuned to deliver effective instruction-based performance. However, it currently lacks moderation mechanisms. We are eager to collaborate with the community to develop and implement guardrails that ensure the model produces moderated outputs, making it suitable for deployment in regulated environments.
|
| 22 |
+
|