How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ArunKr/experimets:Q8_0
# Run inference directly in the terminal:
llama cli -hf ArunKr/experimets:Q8_0
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ArunKr/experimets:Q8_0
# Run inference directly in the terminal:
llama cli -hf ArunKr/experimets:Q8_0
Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ArunKr/experimets:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf ArunKr/experimets:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ArunKr/experimets:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ArunKr/experimets:Q8_0
Use Docker
docker model run hf.co/ArunKr/experimets:Q8_0
Quick Links

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Here’s a minimal, working example showing how you can create a Hugging Face model repo and upload files into it using Python.


1. Install dependencies

uv pip install huggingface_hub

2. Example Python script

from huggingface_hub import HfApi, create_repo, upload_file
import os

# 1. Authenticate (make sure HF_TOKEN is in your environment variables)
hf_token = os.getenv("HF_TOKEN")
if not hf_token:
    raise EnvironmentError("Missing HF_TOKEN in environment variables.")

# 2. Define repo details
repo_name = "test-model-repo"
namespace = "your-username"   # replace with your HF username/org
repo_id = f"{namespace}/{repo_name}"

# 3. Create repo (only first time, idempotent)
create_repo(repo_id, token=hf_token, repo_type="model", exist_ok=True)
print(f"✅ Repo created: https://huggingface.co/{repo_id}")

# 4. Upload files
api = HfApi()

# Example: upload a config.json file
upload_file(
    path_or_fileobj="config.json",     # local file path
    path_in_repo="config.json",        # where it will appear in repo
    repo_id=repo_id,
    token=hf_token
)

# Example: upload model weights
upload_file(
    path_or_fileobj="pytorch_model.bin",
    path_in_repo="pytorch_model.bin",
    repo_id=repo_id,
    token=hf_token
)

print("✅ Files uploaded successfully!")

3. Workflow

  1. Get your token from Hugging Face settings → Access Tokens.

  2. Save it in environment:

    export HF_TOKEN=your_token_here
    
  3. Run the script → it will:

    • Create a model repo if not exists.
    • Upload your files (e.g., config.json, pytorch_model.bin).
    • You’ll see them in https://huggingface.co/your-username/test-model-repo.

👉 Do you want me to also show an example of uploading multiple files/folders at once (like the whole model directory with tokenizer, config, and weights)?

Downloads last month
12
GGUF
Model size
0.3B params
Architecture
gemma3
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support