Instructions to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="rodrigoramosrs/veriloop-coder-e2-nvfp4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("rodrigoramosrs/veriloop-coder-e2-nvfp4")
model = AutoModelForCausalLM.from_pretrained("rodrigoramosrs/veriloop-coder-e2-nvfp4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rodrigoramosrs/veriloop-coder-e2-nvfp4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rodrigoramosrs/veriloop-coder-e2-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/rodrigoramosrs/veriloop-coder-e2-nvfp4

SGLang

How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rodrigoramosrs/veriloop-coder-e2-nvfp4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rodrigoramosrs/veriloop-coder-e2-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rodrigoramosrs/veriloop-coder-e2-nvfp4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rodrigoramosrs/veriloop-coder-e2-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with Docker Model Runner:
```
docker model run hf.co/rodrigoramosrs/veriloop-coder-e2-nvfp4
```

veriloop-coder-e2-nvfp4 / scripts /sglang-qwen35-loader.patch.py

rodrigoramosrs

Upload scripts/sglang-qwen35-loader.patch.py with huggingface_hub

4d2363b verified about 1 month ago

Raw History Blame Contribute Delete

2.49 kB

"""Fix SGLang qwen3_5_text loader: yield body-relative names.
Root cause: the wrapper stored the body as self.model, so the body's own
params_dict holds PREFIX-LESS names (layers.0..., embed_tokens, norm...).
But body_weights() forwarded names that never match those keys:
  - original code stripped only "model." -> "language_model.layers..." which
    the body's "model.language_model."->"model." rule cannot map;
  - checkpoint names are "model.language_model.*" (VL-structured) or
    "model.*" (plain), neither of which resolves.
Almost every weight missed params_dict ("not found" warnings) and the model
ran on uninitialized params (constant token-0 output).
Fix: strip "model." AND an optional "language_model." segment, yielding
exactly the body-relative names for both checkpoint layouts. lm_head keeps
its dedicated branch. Idempotent.
"""
import sys

P = "/sgl-workspace/sglang/python/sglang/srt/models/qwen3_5_text.py"
src = open(P).read()
if "# PREFIX-STRIP-PATCH-V2" in src:
    print("already patched (v2)")
    sys.exit(0)

v1 = """                if name.startswith(_MODEL_PREFIX):
                    # NO-STRIP-PATCH: keep full names; the body maps
                    # model.language_model.* -> model.* itself. Stripping
                    # breaks every lookup (params stay uninitialized).
                    yield name, loaded_weight"""
v1_orig = """                if name.startswith(_MODEL_PREFIX):
                    yield name[len(_MODEL_PREFIX) :], loaded_weight"""
if v1 in src:
    src = src.replace(v1, "PLACEHOLDER_WILL_REPLACE_BELOW")
    print("removed v1 patch")
elif v1_orig not in src:
    print("pattern not found - SGLang version differs, aborting")
    sys.exit(1)

fixed = """                # PREFIX-STRIP-PATCH-V2: the body stores params WITHOUT any
                # prefix (layers.0..., embed_tokens, ...), so strip "model."
                # plus an optional "language_model." segment. Covers both
                # "model.layers.*" and "model.language_model.*" checkpoints.
                if name.startswith("model.language_model."):
                    yield name[len("model.language_model."):], loaded_weight
                elif name.startswith(_MODEL_PREFIX):
                    yield name[len(_MODEL_PREFIX):], loaded_weight"""
if "PLACEHOLDER_WILL_REPLACE_BELOW" in src:
    src = src.replace("PLACEHOLDER_WILL_REPLACE_BELOW", fixed)
else:
    src = src.replace(v1_orig, fixed)
open(P, "w").write(src)
print("patched OK (v2)")

	"""Fix SGLang qwen3_5_text loader: yield body-relative names.

	Root cause: the wrapper stored the body as self.model, so the body's own
	params_dict holds PREFIX-LESS names (layers.0..., embed_tokens, norm...).
	But body_weights() forwarded names that never match those keys:
	- original code stripped only "model." -> "language_model.layers..." which
	the body's "model.language_model."->"model." rule cannot map;
	- checkpoint names are "model.language_model.*" (VL-structured) or
	"model.*" (plain), neither of which resolves.
	Almost every weight missed params_dict ("not found" warnings) and the model
	ran on uninitialized params (constant token-0 output).

	Fix: strip "model." AND an optional "language_model." segment, yielding
	exactly the body-relative names for both checkpoint layouts. lm_head keeps
	its dedicated branch. Idempotent.
	"""
	import sys

	P = "/sgl-workspace/sglang/python/sglang/srt/models/qwen3_5_text.py"
	src = open(P).read()
	if "# PREFIX-STRIP-PATCH-V2" in src:
	print("already patched (v2)")
	sys.exit(0)

	v1 = """ if name.startswith(_MODEL_PREFIX):
	# NO-STRIP-PATCH: keep full names; the body maps
	# model.language_model.* -> model.* itself. Stripping
	# breaks every lookup (params stay uninitialized).
	yield name, loaded_weight"""
	v1_orig = """ if name.startswith(_MODEL_PREFIX):
	yield name[len(_MODEL_PREFIX) :], loaded_weight"""
	if v1 in src:
	src = src.replace(v1, "PLACEHOLDER_WILL_REPLACE_BELOW")
	print("removed v1 patch")
	elif v1_orig not in src:
	print("pattern not found - SGLang version differs, aborting")
	sys.exit(1)

	fixed = """ # PREFIX-STRIP-PATCH-V2: the body stores params WITHOUT any
	# prefix (layers.0..., embed_tokens, ...), so strip "model."
	# plus an optional "language_model." segment. Covers both
	# "model.layers." and "model.language_model." checkpoints.
	if name.startswith("model.language_model."):
	yield name[len("model.language_model."):], loaded_weight
	elif name.startswith(_MODEL_PREFIX):
	yield name[len(_MODEL_PREFIX):], loaded_weight"""
	if "PLACEHOLDER_WILL_REPLACE_BELOW" in src:
	src = src.replace("PLACEHOLDER_WILL_REPLACE_BELOW", fixed)
	else:
	src = src.replace(v1_orig, fixed)
	open(P, "w").write(src)
	print("patched OK (v2)")