--- license: apache-2.0 --- --- ALLM Enterprise is a no-code library designed to simplify the integration of the latest AI features into organizations. It enables seamless adoption of advanced AI capabilities without the need for extensive coding expertise. language: en license: [apache-2.0] Details: - instruction-following - large-language-model - allm-7b - 32k-context - rope-theta model_name: allm-7b-instruct-v0.1 model_type: large-language-model # Model Overview ## Model Name Allm-7B-Instruct-v0.1 ## Model Type Large Language Model (LLM) ## Task Instruction-based Language Generation ## Version v0.1 ## Date [30-07-2024] ## Description Allm-7B-Instruct-v0.1 is a large language model specifically designed for instruction-based tasks. Key enhancements in this version include: - **32k context window** (upgraded from the 8k context in the previous version) - **Rope-theta value** of 1e6 - **Elimination of Sliding-Window Attention** For a comprehensive overview of these features and the model’s capabilities, please refer to our [paper](#) and [release blog post](#). # Instruction Format To leverage the instruction fine-tuning of this model, format your prompts using [INST] and [/INST] tokens. The very first instruction should begin with a sentence ID, while subsequent instructions should not. The response from the assistant will be concluded with an end-of-sentence token ID. # Limitations The Allm-7B-Instruct model is a preliminary demonstration of the base model's ability to be fine-tuned for effective instruction-based performance. It currently lacks built-in moderation mechanisms. We welcome community collaboration to develop and implement guardrails to ensure the model’s outputs are moderated, enabling its deployment in environments that require regulated outputs.