Instructions to use Jun1510/junai-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Jun1510/junai-models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Jun1510/junai-models:Q4_K_M # Run inference directly in the terminal: llama cli -hf Jun1510/junai-models:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Jun1510/junai-models:Q4_K_M # Run inference directly in the terminal: llama cli -hf Jun1510/junai-models:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Jun1510/junai-models:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Jun1510/junai-models:Q4_K_M
Use Docker
docker model run hf.co/Jun1510/junai-models:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use Jun1510/junai-models with Ollama:
ollama run hf.co/Jun1510/junai-models:Q4_K_M
- Unsloth Desktop
- Pi
How to use Jun1510/junai-models with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Jun1510/junai-models:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Jun1510/junai-models:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Jun1510/junai-models with Docker Model Runner:
docker model run hf.co/Jun1510/junai-models:Q4_K_M
- Lemonade
How to use Jun1510/junai-models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Jun1510/junai-models:Q4_K_M
Run and chat with the model
lemonade run user.junai-models-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use Jun1510/junai-models with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Jun1510/junai-models:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default Jun1510/junai-models:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Jun1510/junai-models with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Jun1510/junai-models:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "Jun1510/junai-models:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 5,964 Bytes
622d2ce | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | [
{"id": "aiml_001", "domain": "ai_ml", "topic": "what is machine learning", "content": "Machine learning is a branch of AI where computer systems learn patterns from data to make predictions or decisions, instead of being explicitly programmed with fixed rules for every scenario. The model improves its performance as it's exposed to more relevant data during training."},
{"id": "aiml_002", "domain": "ai_ml", "topic": "what is a large language model (LLM)", "content": "A large language model is an AI system trained on massive amounts of text data to understand and generate human-like language, used for tasks like answering questions, writing, and conversation. Examples include GPT, Claude, Gemini, and open models like Qwen and Llama."},
{"id": "aiml_003", "domain": "ai_ml", "topic": "what is a neural network", "content": "A neural network is a machine learning model loosely inspired by the structure of the human brain, made up of layers of interconnected \"neurons\" (nodes) that process and pass along information. Deep learning refers to neural networks with many layers, capable of learning complex patterns."},
{"id": "aiml_004", "domain": "ai_ml", "topic": "what is model quantization", "content": "Quantization is a technique that reduces the precision of a model's internal numbers (e.g. from 16-bit to 4-bit) to shrink its file size and speed up processing, with some tradeoff in accuracy. It's especially useful for running AI models on devices with limited memory, like smartphones."},
{"id": "aiml_005", "domain": "ai_ml", "topic": "what is fine-tuning in AI", "content": "Fine-tuning is the process of taking an already-trained AI model and further training it on a smaller, specific dataset to adapt its behavior or knowledge for a particular task or style. It's much faster and cheaper than training a model from scratch."},
{"id": "aiml_006", "domain": "ai_ml", "topic": "what is RAG (retrieval-augmented generation)", "content": "RAG is a technique where an AI model is given relevant, up-to-date information retrieved from an external source (like a document or database) at the time of answering, rather than relying only on facts memorized during training. This helps reduce incorrect or outdated answers, especially for a small model with limited built-in knowledge."},
{"id": "aiml_007", "domain": "ai_ml", "topic": "what is a hallucination in AI", "content": "An AI hallucination is when a model generates information that sounds plausible but is factually incorrect or made up, often because it's predicting likely-sounding text rather than retrieving verified facts. Smaller models are generally more prone to this than larger, more capable ones."},
{"id": "aiml_008", "domain": "ai_ml", "topic": "what is on-device AI (edge AI)", "content": "On-device AI, or edge AI, means running AI models directly on a local device like a phone rather than sending data to a remote cloud server. Benefits include better privacy (data never leaves the device), offline availability, and lower latency, at the cost of using smaller, less powerful models than cloud-based ones."},
{"id": "aiml_009", "domain": "ai_ml", "topic": "what is a token in AI language models", "content": "A token is a chunk of text — often a word, part of a word, or punctuation mark — that a language model processes as its basic unit of input and output. Model context limits and processing speed are typically measured in number of tokens rather than words or characters."},
{"id": "aiml_010", "domain": "ai_ml", "topic": "what is a GGUF file", "content": "GGUF is a file format used to store quantized AI language models efficiently, commonly used with the llama.cpp inference engine for running models on regular hardware including phones and laptops without needing a powerful GPU. It replaced the earlier GGML format."},
{"id": "aiml_011", "domain": "ai_ml", "topic": "what is a context window in AI models", "content": "A context window is the maximum amount of text (measured in tokens) a language model can consider at once, including both the conversation history and the current input. Once a conversation exceeds this limit, earlier parts may need to be summarized or dropped for the model to keep responding coherently."},
{"id": "aiml_012", "domain": "ai_ml", "topic": "what is prompt engineering", "content": "Prompt engineering is the practice of carefully crafting instructions given to an AI model to get more accurate, relevant, or well-formatted responses. It includes techniques like giving clear examples, specifying the desired format, or breaking a complex task into steps."},
{"id": "aiml_013", "domain": "ai_ml", "topic": "what is embedding in AI", "content": "An embedding is a way of converting text (or other data) into a list of numbers (a vector) that represents its meaning, so that texts with similar meaning end up with mathematically similar vectors. This is the basis for semantic search and retrieval systems like RAG, since similarity between vectors can be measured directly."},
{"id": "aiml_014", "domain": "ai_ml", "topic": "what is function calling / tool use in AI", "content": "Function calling (or tool use) is a capability where an AI model can recognize when a user's request requires taking a specific action — like opening an app or searching the web — and output a structured request for that action instead of just replying with text. A separate system then carries out the actual action."},
{"id": "aiml_015", "domain": "ai_ml", "topic": "what is the difference between AI, machine learning, and deep learning", "content": "AI is the broad field of building systems that perform tasks requiring human-like intelligence. Machine learning is a subset of AI where systems learn from data rather than fixed rules. Deep learning is a further subset of machine learning that uses multi-layered neural networks, responsible for most recent advances like large language models."}
]
|