| --- |
| license: llama2 |
| base_model: daven3/k2 |
| tags: |
| - mlx |
| - geoscience |
| - earth-science |
| - llama |
| - quantized |
| pipeline_tag: text-generation |
| language: |
| - en |
| library_name: mlx |
| --- |
| |
| # K2-geoscience-7B-4bit-mlx |
|
|
| This is a 4-bit [MLX](https://github.com/ml-explore/mlx) quantization of [**K2**](https://huggingface.co/daven3/k2) (`daven3/k2`), the first open-source foundation language model for the **geoscience** domain, converted for fast local inference on Apple Silicon. |
|
|
| K2 is a 7B model built by further-pretraining LLaMA-7B on ~5.5B tokens of geoscience literature (earth-science papers and Wikipedia) and instruction-tuning with the GeoSignal dataset. See the original work for full details: |
|
|
| > Deng et al., *K2: A Foundation Language Model for Geoscience Knowledge |
| > Understanding and Utilization*, WSDM 2024. |
| > Paper: https://arxiv.org/abs/2306.05064 · Code: https://github.com/davendw49/k2 |
|
|
| ## Conversion details |
|
|
| - **Method:** `mlx_lm.convert` with `-q --q-bits 4` |
| - **Result:** ~4.5 bits per weight |
| - **Precision of base load:** float16 |
| - **Converted on:** Apple Silicon (MLX) |
| - **Tokenizer:** the `daven3/k2` repo does not ship a tokenizer. K2 uses the unmodified LLaMA-7B SentencePiece tokenizer (vocab size 32000), so the standard LLaMA tokenizer files are bundled here for a self-contained model. Credit to the LLaMA authors for the tokenizer. |
|
|
| ## Usage |
|
|
| ```bash |
| pip install mlx-lm |
| ``` |
|
|
| ```python |
| from mlx_lm import load, generate |
| |
| model, tokenizer = load("xquantize/K2-geoscience-7B-4bit-mlx") |
| |
| # K2 (this release) is a BASE model — prompt it with completion-style text, |
| # not chat turns. |
| prompt = "Plate tectonics is the scientific theory that" |
| print(generate(model, tokenizer, prompt=prompt, max_tokens=120, verbose=True)) |
| ``` |
|
|
| ### Note on usage style |
|
|
| This is the **base / further-pretrained** K2 model, not an instruction-tuned chat model. It continues text rather than answering questions conversationally. For best results, give it a sentence to continue (e.g. *"P-waves and S-waves differ in that..."*) rather than a chat-style question. Output can be repetitive, as is typical for 7B base models of this generation. |
|
|
| ## Performance |
|
|
| On an Apple Silicon Mac, this 4-bit build runs at ~34 tokens/sec with a peak memory footprint of ~4 GB, making a geoscience-specialized 7B usable on consumer hardware. |
|
|
| ## Limitations |
|
|
| K2 is a 2023/2024-era LLaMA-7B derivative; its general fluency and knowledge reflect that generation, not current frontier models. It may produce inaccurate or outdated statements and should not be relied on for authoritative scientific or safety-critical decisions. Quantization to 4 bits may introduce additional minor quality degradation relative to the full-precision model. |
|
|
| ## License & attribution |
|
|
| The base model K2 is distributed under the LLaMA license (LLaMA-2 community license terms apply to redistribution of LLaMA-derived weights). All credit for the model belongs to the original K2 authors (Deng et al., davendw49/k2). This repository only provides an MLX-format 4-bit quantization for local inference and claims no ownership of the underlying model. |
