File size: 3,114 Bytes
d1ab8a3 c7ce80d d1ab8a3 c7ce80d d1ab8a3 c7ce80d d1ab8a3 c7ce80d d1ab8a3 c7ce80d d1ab8a3 c7ce80d d1ab8a3 c7ce80d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | ---
license: llama2
base_model: daven3/k2
tags:
- mlx
- geoscience
- earth-science
- llama
- quantized
pipeline_tag: text-generation
language:
- en
library_name: mlx
---
# K2-geoscience-7B-4bit-mlx
This is a 4-bit [MLX](https://github.com/ml-explore/mlx) quantization of [**K2**](https://huggingface.co/daven3/k2) (`daven3/k2`), the first open-source foundation language model for the **geoscience** domain, converted for fast local inference on Apple Silicon.
K2 is a 7B model built by further-pretraining LLaMA-7B on ~5.5B tokens of geoscience literature (earth-science papers and Wikipedia) and instruction-tuning with the GeoSignal dataset. See the original work for full details:
> Deng et al., *K2: A Foundation Language Model for Geoscience Knowledge
> Understanding and Utilization*, WSDM 2024.
> Paper: https://arxiv.org/abs/2306.05064 · Code: https://github.com/davendw49/k2
## Conversion details
- **Method:** `mlx_lm.convert` with `-q --q-bits 4`
- **Result:** ~4.5 bits per weight
- **Precision of base load:** float16
- **Converted on:** Apple Silicon (MLX)
- **Tokenizer:** the `daven3/k2` repo does not ship a tokenizer. K2 uses the unmodified LLaMA-7B SentencePiece tokenizer (vocab size 32000), so the standard LLaMA tokenizer files are bundled here for a self-contained model. Credit to the LLaMA authors for the tokenizer.
## Usage
```bash
pip install mlx-lm
```
```python
from mlx_lm import load, generate
model, tokenizer = load("xquantize/K2-geoscience-7B-4bit-mlx")
# K2 (this release) is a BASE model — prompt it with completion-style text,
# not chat turns.
prompt = "Plate tectonics is the scientific theory that"
print(generate(model, tokenizer, prompt=prompt, max_tokens=120, verbose=True))
```
### Note on usage style
This is the **base / further-pretrained** K2 model, not an instruction-tuned chat model. It continues text rather than answering questions conversationally. For best results, give it a sentence to continue (e.g. *"P-waves and S-waves differ in that..."*) rather than a chat-style question. Output can be repetitive, as is typical for 7B base models of this generation.
## Performance
On an Apple Silicon Mac, this 4-bit build runs at ~34 tokens/sec with a peak memory footprint of ~4 GB, making a geoscience-specialized 7B usable on consumer hardware.
## Limitations
K2 is a 2023/2024-era LLaMA-7B derivative; its general fluency and knowledge reflect that generation, not current frontier models. It may produce inaccurate or outdated statements and should not be relied on for authoritative scientific or safety-critical decisions. Quantization to 4 bits may introduce additional minor quality degradation relative to the full-precision model.
## License & attribution
The base model K2 is distributed under the LLaMA license (LLaMA-2 community license terms apply to redistribution of LLaMA-derived weights). All credit for the model belongs to the original K2 authors (Deng et al., davendw49/k2). This repository only provides an MLX-format 4-bit quantization for local inference and claims no ownership of the underlying model. |