robot-learn / README.md
arcticoneai's picture
Update README.md
e4e1a62 verified
|
Raw History Blame Contribute Delete
12.8 kB
---
language:
- en
library_name: peft
tags:
- spark-x2.6
- sparkllm
- xhtoken
- lora
- peft
- parameter-efficient-fine-tuning
- fine-tuning
- instruction-tuning
- causal-language-modeling
- language-model
- llm
- transformers
- pytorch
- artificial-intelligence
- machine-learning
- reasoning
- coding
- agentic-ai
- open-weights
- research
---
# LiberalMind Γ— Spark-X2.6
### A parameter-efficient adaptation of Spark-X2.6.
This repository contains a LiberalMind LoRA adapter designed to extend the capabilities of the Spark-X2.6 language model.
Instead of distributing a complete language model checkpoint, this project provides a lightweight set of trainable adapter weights that can be attached to the original Spark-X2.6 architecture.
The base model provides the underlying language capabilities, while the LoRA adapter introduces additional learned parameters.
**Developed and published by LiberalMind.**
---
## Model Information
| Property | Value |
|---|---|
| Organization | LiberalMind |
| Base model family | Spark-X2.6 |
| Base model publisher | XHToken |
| Model size | 4B (if using the 4B variant) |
| Fine-tuning method | LoRA |
| Adapter framework | Hugging Face PEFT |
| Inference framework | Transformers |
| Model format | PEFT adapter |
| Training approach | Parameter-efficient fine-tuning |
### Base Model
Official model repositories:
- [Spark-X2.6-4B](https://huggingface.co/XHToken/Spark-X2.6-4B)
- [Spark-X2.6-4B-Base](https://huggingface.co/XHToken/Spark-X2.6-4B-Base)
**Important:** The adapter must be loaded onto the exact base checkpoint used during its original training. The `base_model_name_or_path` field in `adapter_config.json` identifies the intended base model. Do not substitute the Base or post-trained variant without checking compatibility.
---
# 1. How It Works
LoRA (Low-Rank Adaptation) fine-tunes a relatively small number of additional parameters while keeping the original model weights frozen.
Conceptually:
```text
Original Spark-X2.6
|
|
Frozen Weights
|
|
+----------------+
| |
v v
Original Layer LoRA Adapter
| |
| |
+-------+--------+
|
v
Combined Output
|
v
Adapted Language Model
```
The adapter stores learned low-rank updates rather than another copy of the entire model.
This makes it possible to distribute and share fine-tuned adaptations without redistributing the full base checkpoint.
---
# 2. Repository Structure
Expected files:
```text
spark-x2.6-lora/
β”‚
β”œβ”€β”€ README.md
β”œβ”€β”€ adapter_config.json
β”œβ”€β”€ adapter_model.safetensors
β”‚
└── tokenizer files (optional)
```
| File | Purpose |
|---|---|
| `adapter_config.json` | LoRA architecture and base model metadata |
| `adapter_model.safetensors` | Learned LoRA parameters |
| `README.md` | Documentation |
The original Spark-X2.6 weights are downloaded separately.
---
# 3. Installation
Recommended environment:
- Python 3.10+
- PyTorch
- Transformers
- PEFT
- Accelerate
- NVIDIA GPU for the example training workflow
Install dependencies:
```bash
pip install torch
pip install transformers==4.57.1
pip install peft accelerate safetensors
pip install datasets
```
The Transformers version follows the official Spark-X2.6 integration example.
---
# 4. Download and Load the Adapter
You do not need to manually merge LoRA tensors into the base model.
PEFT handles the adapter integration.
```python
import torch
from transformers import (
AutoModelForCausalLM,
AutoTokenizer
)
from peft import (
PeftConfig,
PeftModel
)
# LiberalMind adapter
ADAPTER_ID = "LiberalMind/spark-x2.6-lora"
# Read the original base model identifier from the adapter
peft_config = PeftConfig.from_pretrained(ADAPTER_ID)
BASE_MODEL = peft_config.base_model_name_or_path
print("Base model:", BASE_MODEL)
print("LoRA configuration:", peft_config)
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(
BASE_MODEL,
trust_remote_code=True
)
# Load original Spark-X2.6
base_model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
trust_remote_code=True,
torch_dtype="auto",
device_map="auto"
)
# Attach LiberalMind LoRA weights
model = PeftModel.from_pretrained(
base_model,
ADAPTER_ID
)
model.eval()
print("LiberalMind adapter successfully loaded!")
```
If the adapter metadata contains an obsolete or local base-model path, replace `BASE_MODEL` manually with the exact compatible Hugging Face model ID.
---
# 5. Run Inference
Once the adapter is attached, the resulting model can be used through the regular Transformers generation interface.
```python
import torch
messages = [
{
"role": "user",
"content": "Explain how a neural network learns."
}
]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
enable_thinking=True,
return_dict=True,
return_tensors="pt"
).to(model.device)
with torch.inference_mode():
outputs = model.generate(
**inputs,
max_new_tokens=1024,
do_sample=True,
temperature=0.7,
top_p=0.9
)
generated = outputs[0][inputs["input_ids"].shape[-1]:]
print(
tokenizer.decode(
generated,
skip_special_tokens=True
)
)
```
The chat template is important: it formats conversations according to Spark-X2.6's expected input structure.
---
# 6. Continue Training the Existing LoRA Adapter
An existing adapter can be loaded in trainable mode and fine-tuned further.
Unlike inference mode, this enables gradients for the adapter parameters.
### Step 1. Load the base model and existing adapter
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftConfig, PeftModel
ADAPTER_ID = "LiberalMind/spark-x2.6-lora"
peft_config = PeftConfig.from_pretrained(ADAPTER_ID)
BASE_MODEL = peft_config.base_model_name_or_path
tokenizer = AutoTokenizer.from_pretrained(
BASE_MODEL,
trust_remote_code=True
)
dtype = (
torch.bfloat16
if torch.cuda.is_available()
and torch.cuda.is_bf16_supported()
else torch.float16
)
base_model = AutoModelForCausalLM.from_pretrained(
BASE_MODEL,
trust_remote_code=True,
torch_dtype=dtype,
device_map="auto"
)
model = PeftModel.from_pretrained(
base_model,
ADAPTER_ID,
is_trainable=True
)
model.config.use_cache = False
print("Loaded existing LoRA adapter in training mode.")
trainable = sum(
p.numel() for p in model.parameters() if p.requires_grad
)
total = sum(
p.numel() for p in model.parameters()
)
print(f"Trainable parameters: {trainable:,}")
print(f"Total parameters: {total:,}")
print(f"Trainable percentage: {100 * trainable / total:.4f}%")
```
### Step 2. Prepare the dataset
The following example expects a JSONL file:
```text
train.jsonl
```
Example records:
```json
{"messages":[{"role":"user","content":"What is machine learning?"},{"role":"assistant","content":"Machine learning is a field of AI that enables systems to learn patterns from data."}]}
{"messages":[{"role":"user","content":"Explain gradient descent."},{"role":"assistant","content":"Gradient descent is an optimization algorithm that iteratively adjusts parameters to minimize a loss function."}]}
```
Each record contains a conversation with user and assistant messages.
### Step 3. Prepare the training dataset
```python
from datasets import load_dataset
from transformers import DataCollatorForLanguageModeling
if tokenizer.pad_token is None:
tokenizer.pad_token = tokenizer.eos_token
dataset = load_dataset(
"json",
data_files="train.jsonl",
split="train"
)
def format_example(example):
text = tokenizer.apply_chat_template(
example["messages"],
tokenize=False,
add_generation_prompt=False,
enable_thinking=True
)
return {"text": text}
dataset = dataset.map(format_example)
def tokenize_function(examples):
return tokenizer(
examples["text"],
truncation=True,
max_length=4096
)
tokenized_dataset = dataset.map(
tokenize_function,
batched=True,
remove_columns=dataset.column_names
)
collator = DataCollatorForLanguageModeling(
tokenizer=tokenizer,
mlm=False
)
```
This is a basic full-sequence causal language-modeling example. It trains on both user and assistant tokens; for production instruction tuning, assistant-only loss masking may be preferable.
### Step 4. Start training
```python
import torch
from transformers import (
Trainer,
TrainingArguments
)
OUTPUT_DIR = "./liberalmind-spark-lora-trained"
bf16 = (
torch.cuda.is_available()
and torch.cuda.is_bf16_supported()
)
fp16 = (
torch.cuda.is_available()
and not bf16
)
training_args = TrainingArguments(
output_dir=OUTPUT_DIR,
num_train_epochs=1,
per_device_train_batch_size=1,
gradient_accumulation_steps=8,
learning_rate=1e-5,
warmup_ratio=0.03,
lr_scheduler_type="cosine",
logging_steps=10,
save_strategy="steps",
save_steps=200,
save_total_limit=2,
bf16=bf16,
fp16=fp16,
gradient_checkpointing=True,
max_grad_norm=1.0,
report_to="none",
remove_unused_columns=False
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_dataset,
data_collator=collator
)
trainer.train()
trainer.save_model(OUTPUT_DIR)
tokenizer.save_pretrained(OUTPUT_DIR)
print("Training completed!")
print("Adapter saved to:", OUTPUT_DIR)
```
This workflow loads the existing adapter and updates its parameters. It does not train a new independent LoRA adapter from scratch.
To resume optimizer and scheduler states from an interrupted Trainer run, use its saved `checkpoint-*` directory:
```python
trainer.train(
resume_from_checkpoint="./liber almind-spark-lora-trained/checkpoint-200"
)
```
For that exact resume, the checkpoint must contain the Trainer state, not merely the final adapter weights.
---
# 7. Export the Adapted Model
The adapter can optionally be merged into the base model.
```python
merged_model = model.merge_and_unload()
merged_model.save_pretrained(
"./liber almind-spark-merged",
safe_serialization=True
)
tokenizer.save_pretrained(
"./liber almind-spark-merged"
)
print("Merged model exported.")
```
The merged checkpoint contains the combined model weights rather than a separate LoRA adapter.
The resulting model is substantially larger than the adapter-only repository.
---
# 8. Training and Reproducibility
For reproducibility, the following should be preserved:
- Original base-model identifier.
- `adapter_config.json`.
- LoRA adapter weights.
- Training dataset and preprocessing.
- Learning rate and batch configuration.
- Training framework versions.
- Training checkpoints, when exact optimizer-state recovery is required.
A LoRA checkpoint by itself does not contain the full base language model.
Anyone using this repository must have access to the corresponding original Spark-X2.6 checkpoint.
---
# 9. Research Direction
LiberalMind explores lightweight model adaptation and efficient AI development.
This project represents an approach to extending existing language model capabilities without requiring a complete new pretraining run.
Potential future directions include:
- Further instruction tuning.
- Mathematical reasoning.
- Coding and software engineering.
- Tool-use and agent workflows.
- Model efficiency and deployment.
- Evaluation across different reasoning and problem-solving tasks.
No benchmark improvement is claimed without a corresponding evaluation.
---
## Credits
**Base architecture and pretrained weights:** XHToken / SparkLLM
**Adapter development:** LiberalMind
**Fine-tuning technology:** Hugging Face PEFT
**Inference framework:** Transformers
**License:** The base model is published under Apache 2.0. Verify the adapter's own licensing and attribution requirements before redistribution.
---
**LiberalMind**
*Building intelligence beyond the screen.*
### References
- [Spark-X2.6 Official Repository](https://github.com/XHToken/Spark-X2.6)
- [Spark-X2.6-4B on Hugging Face](https://huggingface.co/XHToken/Spark-X2.6-4B)
- [Hugging Face PEFT Documentation](https://huggingface.co/docs/peft)
- [Transformers PEFT Integration](https://huggingface.co/docs/transformers/peft)