Instructions to use A-LLM/allm_instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use A-LLM/allm_instruct with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: llama cli -hf A-LLM/allm_instruct
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: llama cli -hf A-LLM/allm_instruct
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: ./build/bin/llama cli -hf A-LLM/allm_instruct
Use Docker
docker model run hf.co/A-LLM/allm_instruct
- Llama new
- LM Studio
- Jan
- Ollama
How to use A-LLM/allm_instruct with Ollama:
ollama run hf.co/A-LLM/allm_instruct
- Unsloth Desktop
- Docker Model Runner
How to use A-LLM/allm_instruct with Docker Model Runner:
docker model run hf.co/A-LLM/allm_instruct
- Lemonade
How to use A-LLM/allm_instruct with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull A-LLM/allm_instruct
Run and chat with the model
lemonade run user.allm_instruct-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
|
Download README.md from A-LLM/allm_instruct: direct link, hf CLI and curl.
- Browser
- Download file 1.83 kB
-
https://huggingface.co/A-LLM/allm_instruct/resolve/main/README.md
- Command line
-
hf download hf://A-LLM/allm_instruct/README.md
-
curl -L -o README.md https://huggingface.co/A-LLM/allm_instruct/resolve/main/README.md
1.83 kB
| license: apache-2.0 | |
| --- | |
| ALLM Enterprise is a no-code library designed to simplify the integration of the latest AI features into organizations. It enables seamless adoption of advanced AI capabilities without the need for extensive coding expertise. | |
| language: en | |
| license: [apache-2.0] | |
| Details: | |
| - instruction-following | |
| - large-language-model | |
| - allm-7b | |
| - 32k-context | |
| - rope-theta | |
| model_name: allm-7b-instruct-v0.1 | |
| model_type: large-language-model | |
| # Model Overview | |
| ## Model Name | |
| Allm-7B-Instruct-v0.1 | |
| ## Model Type | |
| Large Language Model (LLM) | |
| ## Task | |
| Instruction-based Language Generation | |
| ## Version | |
| v0.1 | |
| ## Date | |
| [30-07-2024] | |
| ## Description | |
| Allm-7B-Instruct-v0.1 is a large language model specifically designed for instruction-based tasks. Key enhancements in this version include: | |
| - **32k context window** (upgraded from the 8k context in the previous version) | |
| - **Rope-theta value** of 1e6 | |
| - **Elimination of Sliding-Window Attention** | |
| For a comprehensive overview of these features and the model’s capabilities, please refer to our [paper](#) and [release blog post](#). | |
| # Instruction Format | |
| To leverage the instruction fine-tuning of this model, format your prompts using [INST] and [/INST] tokens. The very first instruction should begin with a sentence ID, while subsequent instructions should not. The response from the assistant will be concluded with an end-of-sentence token ID. | |
| # Limitations | |
| The Allm-7B-Instruct model is a preliminary demonstration of the base model's ability to be fine-tuned for effective instruction-based performance. It currently lacks built-in moderation mechanisms. We welcome community collaboration to develop and implement guardrails to ensure the model’s outputs are moderated, enabling its deployment in environments that require regulated outputs. | |