| --- |
| license: creativeml-openrail-m |
| language: |
| - en |
| base_model: |
| - Qwen/Qwen2.5-7B-Instruct |
| pipeline_tag: text-generation |
| tags: |
| - text-generation-inference |
| - code-solve |
| - algorithm |
| - codepy |
| - qwen_base |
| - 7b |
| - LoRA |
| - CoT |
| library_name: peft |
| --- |
| |
| <pre align="center"> |
| .___ __ .__ .__ __ |
| __| _/ ____ ____ ______ _/ |_ | |__ |__| ____ | | __ |
| / __ | _/ __ \ _/ __ \ \____ \ \ __\| | \ | | / \ | |/ / |
| / /_/ | \ ___/ \ ___/ | |_> > | | | Y \| || | \| < |
| \____ | \___ > \___ >| __/ |__| |___| /|__||___| /|__|_ \ |
| \/ \/ \/ |__| \/ \/ \/ |
| </pre> |
| |
| The **Deepthink-Reasoning-Adapter** is a fine-tuned version of the **Qwen2.5-7B-Instruct** base model, designed for text generation tasks that require deep reasoning, logical structuring, and problem-solving. This model leverages its optimized architecture to provide accurate and contextually relevant outputs for complex queries, making it ideal for applications in education, programming, and creative writing. |
|
|
| With its robust natural language processing capabilities, **Deepthink-Reasoning-Adapter** excels in generating step-by-step solutions, creative content, and logical analyses. Its architecture integrates advanced understanding of both structured and unstructured data, ensuring precise text generation aligned with user inputs. |
|
|
| - Significantly **more knowledge** and has greatly improved capabilities in **coding** and **mathematics**, thanks to our specialized expert models in these domains. |
| - Significant improvements in **instruction following**, **generating long texts** (over 8K tokens), **understanding structured data** (e.g, tables), and **generating structured outputs** especially JSON. **More resilient to the diversity of system prompts**, enhancing role-play implementation and condition-setting for chatbots. |
| - **Long-context Support** up to 128K tokens and can generate up to 8K tokens. |
| - **Multilingual support** for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more. |
|
|
| # **Demo Start** |
|
|
| Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents. |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_name = "prithivMLmods/Deepthink-Reasoning-7B" |
| |
| model = AutoModelForCausalLM.from_pretrained( |
| model_name, |
| torch_dtype="auto", |
| device_map="auto" |
| ) |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| |
| prompt = "Give me a short introduction to large language model." |
| messages = [ |
| {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."}, |
| {"role": "user", "content": prompt} |
| ] |
| text = tokenizer.apply_chat_template( |
| messages, |
| tokenize=False, |
| add_generation_prompt=True |
| ) |
| model_inputs = tokenizer([text], return_tensors="pt").to(model.device) |
| |
| generated_ids = model.generate( |
| **model_inputs, |
| max_new_tokens=512 |
| ) |
| generated_ids = [ |
| output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) |
| ] |
| |
| response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] |
| ``` |
| # **Run with Ollama [Ollama Run]** |
|
|
| Ollama makes running machine learning models simple and efficient. Follow these steps to set up and run your GGUF models quickly. |
|
|
| ## Quick Start: Step-by-Step Guide |
|
|
| | Step | Description | Command / Instructions | |
| |------|-------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------| |
| | 1 | **Install Ollama 🦙** | Download Ollama from [https://ollama.com/download](https://ollama.com/download) and install it on your system. | |
| | 2 | **Create Your Model File** | - Create a file named after your model, e.g., `metallama`. | |
| | | | - Add the following line to specify the base model: | |
| | | | ```bash | |
| | | | FROM Llama-3.2-1B.F16.gguf | |
| | | | ``` | |
| | | | - Ensure the base model file is in the same directory. | |
| | 3 | **Create and Patch the Model** | Run the following commands to create and verify your model: | |
| | | | ```bash | |
| | | | ollama create metallama -f ./metallama | |
| | | | ollama list | |
| | | | ``` | |
| | 4 | **Run the Model** | Use the following command to start your model: | |
| | | | ```bash | |
| | | | ollama run metallama | |
| | | | ``` | |
| | 5 | **Interact with the Model** | Once the model is running, interact with it: | |
| | | | ```plaintext | |
| | | | >>> Tell me about Space X. | |
| | | | Space X, the private aerospace company founded by Elon Musk, is revolutionizing space exploration... | |
| | | | ``` | |
|
|
| ## Conclusion |
| With Ollama, running and interacting with models is seamless. Start experimenting today! |
