Instructions to use A-LLM/allm_instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use A-LLM/allm_instruct with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: llama cli -hf A-LLM/allm_instruct
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: llama cli -hf A-LLM/allm_instruct
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf A-LLM/allm_instruct # Run inference directly in the terminal: ./build/bin/llama cli -hf A-LLM/allm_instruct
Use Docker
docker model run hf.co/A-LLM/allm_instruct
- Llama new
- LM Studio
- Jan
- Ollama
How to use A-LLM/allm_instruct with Ollama:
ollama run hf.co/A-LLM/allm_instruct
- Unsloth Desktop
- Docker Model Runner
How to use A-LLM/allm_instruct with Docker Model Runner:
docker model run hf.co/A-LLM/allm_instruct
- Lemonade
How to use A-LLM/allm_instruct with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull A-LLM/allm_instruct
Run and chat with the model
lemonade run user.allm_instruct-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Run and chat with the model
lemonade run user.allm_instruct-{{QUANT_TAG}}List all available models
lemonade listALLM Enterprise is a no-code library designed to simplify the integration of the latest AI features into organizations. It enables seamless adoption of advanced AI capabilities without the need for extensive coding expertise.
language: en license: [apache-2.0]
Details:
- instruction-following
- large-language-model
- allm-7b
- 32k-context
- rope-theta
model_name: allm-7b-instruct-v0.1 model_type: large-language-model
Model Overview
Model Name
Allm-7B-Instruct-v0.1
Model Type
Large Language Model (LLM)
Task
Instruction-based Language Generation
Version
v0.1
Date
[30-07-2024]
Description
Allm-7B-Instruct-v0.1 is a large language model specifically designed for instruction-based tasks. Key enhancements in this version include:
- 32k context window (upgraded from the 8k context in the previous version)
- Rope-theta value of 1e6
- Elimination of Sliding-Window Attention
For a comprehensive overview of these features and the model’s capabilities, please refer to our paper and release blog post.
Instruction Format
To leverage the instruction fine-tuning of this model, format your prompts using [INST] and [/INST] tokens. The very first instruction should begin with a sentence ID, while subsequent instructions should not. The response from the assistant will be concluded with an end-of-sentence token ID.
Limitations
The Allm-7B-Instruct model is a preliminary demonstration of the base model's ability to be fine-tuned for effective instruction-based performance. It currently lacks built-in moderation mechanisms. We welcome community collaboration to develop and implement guardrails to ensure the model’s outputs are moderated, enabling its deployment in environments that require regulated outputs.
- Downloads last month
- 2
We're not able to determine the quantization variants.
Pull the model
# Download Lemonade from https://lemonade-server.ai/