AWQ 4-bit (W4A16, asymmetric, lm_head kept in 16-bit) build of henil08/verilog-coder-7b, made with llm-compressor and calibrated on 256 validation-split examples (no test data). It is a compressed-tensors checkpoint that vLLM loads directly, for GPUs with 8โ€“12 GB of memory.

vllm serve henil08/verilog-coder-7b-awq --max-model-len 4096

Accuracy was evaluated on the full-precision model; see its card for results and limitations.

Downloads last month
22
Safetensors
Model size
8B params
Tensor type
I32
ยท
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for henil08/verilog-coder-7b-awq

Base model

Qwen/Qwen2.5-7B
Quantized
(3)
this model