Instructions to use A-LLM/allm_instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use A-LLM/allm_instruct with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: llama cli -hf A-LLM/allm_instruct
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: llama cli -hf A-LLM/allm_instruct
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: ./build/bin/llama cli -hf A-LLM/allm_instruct
Use Docker
docker model run hf.co/A-LLM/allm_instruct
- Llama new
- LM Studio
- Jan
- Ollama
How to use A-LLM/allm_instruct with Ollama:
ollama run hf.co/A-LLM/allm_instruct
- Unsloth Desktop
- Docker Model Runner
How to use A-LLM/allm_instruct with Docker Model Runner:
docker model run hf.co/A-LLM/allm_instruct
- Lemonade
How to use A-LLM/allm_instruct with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull A-LLM/allm_instruct
Run and chat with the model
lemonade run user.allm_instruct-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
File size: 1,825 Bytes
8db9dd5 25328c6 34159bb 25328c6 8db9dd5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | ---
license: apache-2.0
---
---
ALLM Enterprise is a no-code library designed to simplify the integration of the latest AI features into organizations. It enables seamless adoption of advanced AI capabilities without the need for extensive coding expertise.
language: en
license: [apache-2.0]
Details:
- instruction-following
- large-language-model
- allm-7b
- 32k-context
- rope-theta
model_name: allm-7b-instruct-v0.1
model_type: large-language-model
# Model Overview
## Model Name
Allm-7B-Instruct-v0.1
## Model Type
Large Language Model (LLM)
## Task
Instruction-based Language Generation
## Version
v0.1
## Date
[30-07-2024]
## Description
Allm-7B-Instruct-v0.1 is a large language model specifically designed for instruction-based tasks. Key enhancements in this version include:
- **32k context window** (upgraded from the 8k context in the previous version)
- **Rope-theta value** of 1e6
- **Elimination of Sliding-Window Attention**
For a comprehensive overview of these features and the model’s capabilities, please refer to our [paper](#) and [release blog post](#).
# Instruction Format
To leverage the instruction fine-tuning of this model, format your prompts using [INST] and [/INST] tokens. The very first instruction should begin with a sentence ID, while subsequent instructions should not. The response from the assistant will be concluded with an end-of-sentence token ID.
# Limitations
The Allm-7B-Instruct model is a preliminary demonstration of the base model's ability to be fine-tuned for effective instruction-based performance. It currently lacks built-in moderation mechanisms. We welcome community collaboration to develop and implement guardrails to ensure the model’s outputs are moderated, enabling its deployment in environments that require regulated outputs.
|