NyxKrage's picture
Add files using upload-large-folder tool
bc4e445 verified
|
Raw History Blame Contribute Delete
1.06 kB
---
license: apache-2.0
base_model:
- Sparset/Thinklite-27B
- ConceptCountry/qwen-concept-27b
tags:
- weight-editing
- thinklite
---
# Thinklite-27B + Concept edit (experimental, unevaluated)
[Sparset/Thinklite-27B](https://huggingface.co/Sparset/Thinklite-27B) with the additive weight edit from
[ConceptCountry/qwen-concept-27b](https://huggingface.co/ConceptCountry/qwen-concept-27b) transplanted onto it.
The edit was extracted as `concept(bf16) - Qwen/Qwen3.8-27B` (only the nine `mlp.down_proj` weights in layers
32-48 (even) differ), denoised by low-rank truncation when the diff was clearly low-rank, and added in
fp32 to Thinklite's corresponding weights before rounding back to bf16.
Caveat: the Concept direction was derived from the *base* model's activations. Thinklite is a full
finetune, so the direction may not mean the same thing here. Nothing has been benchmarked.
Files: `concept_delta.safetensors` (fp32 deltas actually applied), `TRANSPLANT_REPORT.json`
(per-tensor norms / singular values), `CONCEPT_EDIT_SPEC.json` (upstream spec).