Instructions to use cortexso/llama3.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use cortexso/llama3.1 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf cortexso/llama3.1:Q4_K_M # Run inference directly in the terminal: llama cli -hf cortexso/llama3.1:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf cortexso/llama3.1:Q4_K_M # Run inference directly in the terminal: llama cli -hf cortexso/llama3.1:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf cortexso/llama3.1:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf cortexso/llama3.1:Q4_K_M
Use Docker
docker model run hf.co/cortexso/llama3.1:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use cortexso/llama3.1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "cortexso/llama3.1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "cortexso/llama3.1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/cortexso/llama3.1:Q4_K_M
- Ollama
How to use cortexso/llama3.1 with Ollama:
ollama run hf.co/cortexso/llama3.1:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use cortexso/llama3.1 with Docker Model Runner:
docker model run hf.co/cortexso/llama3.1:Q4_K_M
- Lemonade
How to use cortexso/llama3.1 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull cortexso/llama3.1:Q4_K_M
Run and chat with the model
lemonade run user.llama3.1-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,25 +1,24 @@
|
|
| 1 |
---
|
| 2 |
-
license: llama3
|
| 3 |
---
|
| 4 |
|
| 5 |
## Overview
|
| 6 |
|
| 7 |
-
Meta developed and released the [Meta Llama 3](https://huggingface.co/meta-llama/Meta-Llama-3-8B) family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dialogue use cases and outperform many of the available open source chat models on common industry benchmarks. Further, in developing these models, we took great care to optimize helpfulness and safety.
|
| 8 |
|
| 9 |
## Variants
|
| 10 |
|
| 11 |
| No | Variant | Cortex CLI command |
|
| 12 |
| --- | --- | --- |
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
| 3 | [default](https://huggingface.co/cortexso/llama3/tree/default) | `cortex run llama3` |
|
| 16 |
|
| 17 |
## Use it with Jan (UI)
|
| 18 |
|
| 19 |
1. Install **Jan** using [Quickstart](https://jan.ai/docs/quickstart)
|
| 20 |
2. Use in Jan model Hub:
|
| 21 |
```
|
| 22 |
-
cortexso/llama3
|
| 23 |
```
|
| 24 |
|
| 25 |
## Use it with Cortex (CLI)
|
|
@@ -27,12 +26,12 @@ Meta developed and released the [Meta Llama 3](https://huggingface.co/meta-llama
|
|
| 27 |
1. Install **Cortex** using [Quickstart](https://cortex.jan.ai/docs/quickstart)
|
| 28 |
2. Run the model with command:
|
| 29 |
```
|
| 30 |
-
cortex run llama3
|
| 31 |
```
|
| 32 |
|
| 33 |
## Credits
|
| 34 |
|
| 35 |
- **Author:** Meta
|
| 36 |
- **Converter:** [Homebrew](https://www.homebrew.ltd/)
|
| 37 |
-
- **Original License:** [License](https://llama.meta.com/llama3/license/)
|
| 38 |
-
- **Papers:** [Llama-3 Blog](https://llama.meta.com/llama3/)
|
|
|
|
| 1 |
---
|
| 2 |
+
license: llama3.1
|
| 3 |
---
|
| 4 |
|
| 5 |
## Overview
|
| 6 |
|
| 7 |
+
Meta developed and released the [Meta Llama 3.1](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B) family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dialogue use cases and outperform many of the available open source chat models on common industry benchmarks. Further, in developing these models, we took great care to optimize helpfulness and safety.
|
| 8 |
|
| 9 |
## Variants
|
| 10 |
|
| 11 |
| No | Variant | Cortex CLI command |
|
| 12 |
| --- | --- | --- |
|
| 13 |
+
| 2 | [gguf](https://huggingface.co/cortexso/llama3.1/tree/gguf) | `cortex run llama3.1:gguf` |
|
| 14 |
+
| 3 | [main/default](https://huggingface.co/cortexso/llama3.1/tree/main) | `cortex run llama3.1` |
|
|
|
|
| 15 |
|
| 16 |
## Use it with Jan (UI)
|
| 17 |
|
| 18 |
1. Install **Jan** using [Quickstart](https://jan.ai/docs/quickstart)
|
| 19 |
2. Use in Jan model Hub:
|
| 20 |
```
|
| 21 |
+
cortexso/llama3.1
|
| 22 |
```
|
| 23 |
|
| 24 |
## Use it with Cortex (CLI)
|
|
|
|
| 26 |
1. Install **Cortex** using [Quickstart](https://cortex.jan.ai/docs/quickstart)
|
| 27 |
2. Run the model with command:
|
| 28 |
```
|
| 29 |
+
cortex run llama3.1
|
| 30 |
```
|
| 31 |
|
| 32 |
## Credits
|
| 33 |
|
| 34 |
- **Author:** Meta
|
| 35 |
- **Converter:** [Homebrew](https://www.homebrew.ltd/)
|
| 36 |
+
- **Original License:** [License](https://llama.meta.com/llama3.1/license/)
|
| 37 |
+
- **Papers:** [Llama-3.1 Blog](https://llama.meta.com/llama3.1/)
|