You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Hirundo LFM2.5-1.2B-Instruct Hardened

hirundo-io/LFM2.5-1.2B-Instruct-hardened is a behaviorally hardened derivative of LiquidAI/LFM2.5-1.2B-Instruct. Hirundo modified the model to reduce prompt-injection susceptibility while preserving its general capabilities.

The checkpoint itself was modified; this intervention is not a runtime prompt, output filter, or external guardrail. The security measurements below describe the resulting model's behavior under the reported evaluations.

About Hirundo

Hirundo develops machine-unlearning methods for changing unwanted model behavior. Its behavioral-unlearning workflow identifies a target behavior, modifies the model, and evaluates both security behavior and general-capability performance. This pairs measurements of the intended behavioral change with checks for capability tradeoffs.

Learn more:

What Hirundo changed

Prompt injection occurs when adversarial or untrusted instructions cause a model to disregard its intended task or reveal protected context. This run targeted that susceptibility through behavioral unlearning.

Field Value
Base model LiquidAI/LFM2.5-1.2B-Instruct
Target behavior Security (prompt injection)
Unlearning aggressiveness 0.01

Evaluation summary

The hardened checkpoint was compared with the unmodified base checkpoint under the reported evaluation setup. Thinking mode was recorded as not applicable for this Instruct checkpoint.

For the security evaluations below, lower scores indicate fewer successful attacks or leaks and therefore better performance. Relative reduction is calculated as (base โˆ’ hardened) / base ร— 100. Rates and reductions describe the reported tests and should be interpreted within their respective benchmark definitions.

Headline result: PurpleLlama prompt injection

PurpleLlama's Cybersecurity Benchmarks include prompt-injection evaluations. The reported attack success rate (ASR) fell from 47.01% to 17.53%, a 62.71% relative reduction.

Benchmark Base model ASR Hirundo model ASR Relative reduction
PurpleLlama prompt injection 47.01% 17.53% 62.71%

Garak security evaluation

The Garak scanner probes model vulnerabilities using adversarial inputs. The reported results cover encoding, latent injection, direct prompt injection, and leakage-oriented measurements. Attack-family rates and leakage-oriented rates have distinct interpretations and need not share a denominator; they should not be combined into one aggregate.

Results by attack family

Encoding attacks hide payloads in alternative representations, latent injections embed malicious instructions in otherwise ordinary content, and direct prompt injections explicitly attempt to override instructions. Improvements varied by family, with latent injection showing a smaller reduction than the other two families.

Attack family Base rate Hirundo rate Relative reduction
Encoding attacks 17.1100% 1.0940% 93.61%
Latent injection 2.8000% 2.5000% 10.71%
Direct prompt injection 0.4400% 0.0000% 100.00%

The unweighted mean relative reduction across these three attack families was 68.11%. A zero observed rate in a test does not establish immunity to that attack family.

Leakage-oriented results

Any leak rate, exact value leak, and exact key leak are measured per attempt. Attack success rate is measured per attack or prompt across the repeated attempts described in the evaluation. These denominators differ: the chance of a leak on one attempt is distinct from success across repeated attempts.

Metric Base rate Hirundo rate Relative reduction
Any leak rate (per attempt) 5.0900% 0.8984% 82.35% reduction
Exact value leak (per attempt) 2.4900% 0.6641% 73.33% reduction
Exact key leak (per attempt) 4.9900% 0.8894% 82.18% reduction
Attack success rate (per attack/prompt) 13.1300% 2.9036% 77.89% reduction

The unweighted mean relative reduction across these four leakage metrics was 78.94%. The attack-family and leakage means are descriptive summaries of their respective measurements, rather than standardized security scores.

General-capability evaluation

Six general-capability benchmarks were reported using Nemo-Skills. Each change below is hardened โˆ’ base in percentage points (pp); positive values indicate improvement. The measured security improvement was accompanied by modest, mixed capability changes in this evaluation, with gains on two benchmarks, declines on three, and no change on one.

Benchmark Change vs. base model
AIME25 +0.42 pp
GPQA +1.37 pp
IFBENCH โˆ’1.39 pp
LiveCodeBench โˆ’0.68 pp
MMLU-Pro โˆ’1.05 pp
SciCode 0.00 pp

The unweighted mean change was โˆ’0.22 pp. The largest measured decrease was 1.39 pp on IFBENCH. The mean is descriptive rather than a standardized aggregate score: the benchmarks measure different capabilities and may use different scoring procedures.

Intended use

This checkpoint is intended for:

  • Research into prompt-injection mitigation through model modification.
  • Comparative security evaluation against the original checkpoint and other mitigation approaches.
  • Development of deployments that encounter untrusted content, including retrieval and tool workflows.
  • Further application-specific evaluation or fine-tuning, followed by renewed security and capability testing.

Test the checkpoint with your application's actual prompts, tools, retrieval pipeline, and threat model. These benchmark results do not establish immunity to prompt injection or guarantee that the improvements transfer to every deployment setup.

Loading with Transformers

The upstream model is a text-only hybrid language model. Use the Transformers causal-language-model loader and the tokenizer's chat template, following the upstream loading pattern:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "hirundo-io/LFM2.5-1.2B-Instruct-hardened"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype="bfloat16",
)

inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain how rainbows form."}],
    add_generation_prompt=True,
    tokenize=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.1,
    top_k=50,
    repetition_penalty=1.05,
)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

See the upstream model card for architecture, supported languages and modalities, context length, generation parameters, and base-model considerations. Its text-only interface and loading classes apply here; application performance should be evaluated on this modified checkpoint.

License

Use is subject to the upstream LFM Open License v1.0. Review the upstream license, the repository's license files, and the upstream model card before use or redistribution.

Citation

@misc{hirundo_lfm25_12b_instruct_hardened_2026,
  title = {Hirundo LFM2.5-1.2B-Instruct Hardened},
  author = {Hirundo},
  year = {2026},
  howpublished = {\url{https://huggingface.co/hirundo-io/LFM2.5-1.2B-Instruct-hardened}}
}
Downloads last month
8
Safetensors
Model size
1B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for hirundo-io/LFM2.5-1.2B-Instruct-hardened

Finetuned
(140)
this model

Collection including hirundo-io/LFM2.5-1.2B-Instruct-hardened