Astheral GGUF

GGUF quantizations of SEE7/Astheral, generated with llama.cpp 5676dfbe01e7ebe0387913680eee0ed39bf4a363.

Provided Quants

Measured decimal GB, sorted by file size. Each linked file has passed GGUF structure and upload hash checks.

Link	Type	Size/GB
GGUF	imatrix	0.010
GGUF	i1-IQ1_S	2.020
GGUF	i1-IQ1_M	2.162
GGUF	i1-IQ2_XXS	2.399
GGUF	i1-IQ2_XS	2.606
GGUF	i1-IQ2_S	2.758
GGUF	i1-Q4_K_M	4.921

Calibration

Importance data is collected from real forward passes over a deterministic multilingual and instruction sample of AstheralDataset. Corpus hash, chunk settings, exact tensor formats, and file SHA-256 hashes are recorded in manifest.json.

Built with Llama. See LICENSE, UPSTREAM_LLAMA3_LICENSE, and NOTICE.

Downloads last month: -

GGUF

Model size

8B params

Architecture

llama

Hardware compatibility

1-bit

2-bit

4-bit

View +1 variant

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for SEE7/Astheral-GGUF

Base model

meta-llama/Llama-3.1-8B

Finetuned

NousResearch/DeepHermes-3-Llama-3-8B-Preview

Finetuned

SEE7/Astheral

Quantized

(3)

this model