Instructions to use yasserrmd/LFM2.5-350M-JSON-NoFence with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use yasserrmd/LFM2.5-350M-JSON-NoFence with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="yasserrmd/LFM2.5-350M-JSON-NoFence")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("yasserrmd/LFM2.5-350M-JSON-NoFence")
model = AutoModelForCausalLM.from_pretrained("yasserrmd/LFM2.5-350M-JSON-NoFence", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use yasserrmd/LFM2.5-350M-JSON-NoFence with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "yasserrmd/LFM2.5-350M-JSON-NoFence"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yasserrmd/LFM2.5-350M-JSON-NoFence",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/yasserrmd/LFM2.5-350M-JSON-NoFence

SGLang

How to use yasserrmd/LFM2.5-350M-JSON-NoFence with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "yasserrmd/LFM2.5-350M-JSON-NoFence" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yasserrmd/LFM2.5-350M-JSON-NoFence",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "yasserrmd/LFM2.5-350M-JSON-NoFence" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yasserrmd/LFM2.5-350M-JSON-NoFence",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use yasserrmd/LFM2.5-350M-JSON-NoFence with Docker Model Runner:
```
docker model run hf.co/yasserrmd/LFM2.5-350M-JSON-NoFence
```

LFM2.5-350M-JSON-NoFence / patch /chosen_patch.json

yasserrmd

Upload folder using huggingface_hub

0c1b497 verified 5 days ago

Raw History Blame Contribute Delete

4.5 kB

{
  "name": "c12",
  "kind": "candidate",
  "strategy": "project",
  "direction": "logit",
  "tensors": [
    "op15",
    "ffn14"
  ],
  "k_per_tensor": 64,
  "factor_or_alpha": 1.0,
  "base_repo": "LiquidAI/LFM2.5-350M",
  "base_revision": "9921cd4d82c0e3986887aa4b376f9446729ef8a3",
  "base_files_sha256": {
    "model.safetensors": "1c9c77a4471a7f590f85240f74ed1fc26df7fbde88c3006724e2f93ca993ea4e"
  },
  "model_total_params": 354483968,
  "initial_budget_entries": 177241,
  "hard_max_entries": 354483,
  "within_initial_budget": true,
  "target": {
    "cat": "B",
    "failure_type": "start",
    "category_greedy_fail_rate": 1.0,
    "greedy_failures": 19,
    "sampled_failures": 58,
    "explore_decision_points": 37,
    "greedy_failures_reproduced_in_sampling": 19,
    "score": 38.33333333333333,
    "top_bad_tokens": [
      [
        "```",
        77
      ]
    ]
  },
  "evidence": "explore split decision points only; see results/analysis/analysis_summary.md",
  "entries": [
    {
      "name": "model.layers.15.conv.out_proj.weight",
      "file_key": "model.layers.15.conv.out_proj.weight",
      "shape": [
        1024,
        1024
      ],
      "dtype": "torch.bfloat16",
      "n_changed": 62192,
      "numel": 1048576,
      "delta_l2_fp32": 0.27263545989990234,
      "delta_l2_after_cast": 0.2727110683917999,
      "max_abs_delta_after_cast": 0.0160980224609375,
      "tensor_l2": 16.538354873657227,
      "strategy": "project",
      "direction": "logit",
      "layer": 15,
      "tensor_tag": "op15",
      "cols": [
        4,
        13,
        27,
        31,
        37,
        46,
        69,
        80,
        81,
        96,
        119,
        149,
        155,
        160,
        174,
        191,
        200,
        216,
        226,
        237,
        244,
        252,
        253,
        264,
        276,
        287,
        291,
        318,
        323,
        328,
        330,
        333,
        341,
        380,
        396,
        399,
        403,
        411,
        445,
        454,
        476,
        490,
        491,
        497,
        509,
        511,
        512,
        544,
        605,
        614,
        624,
        645,
        646,
        682,
        691,
        747,
        788,
        851,
        879,
        882,
        888,
        913,
        959,
        1003
      ],
      "factor_or_alpha": 1.0,
      "rationale": "subtract 1.0 x the logit direction component from 64 op15 write vectors (rank-1 projection on selected columns)",
      "i": 0
    },
    {
      "name": "model.layers.14.feed_forward.w2.weight",
      "file_key": "model.layers.14.feed_forward.w2.weight",
      "shape": [
        1024,
        4608
      ],
      "dtype": "torch.bfloat16",
      "n_changed": 63374,
      "numel": 4718592,
      "delta_l2_fp32": 0.24669404327869415,
      "delta_l2_after_cast": 0.24671411514282227,
      "max_abs_delta_after_cast": 0.012820720672607422,
      "tensor_l2": 24.8846378326416,
      "strategy": "project",
      "direction": "logit",
      "layer": 14,
      "tensor_tag": "ffn14",
      "cols": [
        36,
        138,
        208,
        296,
        307,
        408,
        603,
        615,
        646,
        695,
        722,
        766,
        830,
        850,
        891,
        922,
        933,
        998,
        1106,
        1364,
        1428,
        1489,
        1567,
        1773,
        1788,
        1813,
        1848,
        1853,
        1941,
        2040,
        2125,
        2155,
        2254,
        2265,
        2284,
        2396,
        2397,
        2440,
        2594,
        2749,
        2814,
        2925,
        2942,
        3083,
        3207,
        3246,
        3275,
        3375,
        3407,
        3459,
        3545,
        3604,
        4050,
        4095,
        4172,
        4194,
        4341,
        4402,
        4403,
        4424,
        4442,
        4519,
        4562,
        4585
      ],
      "factor_or_alpha": 1.0,
      "rationale": "subtract 1.0 x the logit direction component from 64 ffn14 write vectors (rank-1 projection on selected columns)",
      "i": 1
    }
  ],
  "total_changed": 125566,
  "changed_fraction": 0.0003542219432614792
}

	{
	"name": "c12",
	"kind": "candidate",
	"strategy": "project",
	"direction": "logit",
	"tensors": [
	"op15",
	"ffn14"
	],
	"k_per_tensor": 64,
	"factor_or_alpha": 1.0,
	"base_repo": "LiquidAI/LFM2.5-350M",
	"base_revision": "9921cd4d82c0e3986887aa4b376f9446729ef8a3",
	"base_files_sha256": {
	"model.safetensors": "1c9c77a4471a7f590f85240f74ed1fc26df7fbde88c3006724e2f93ca993ea4e"
	},
	"model_total_params": 354483968,
	"initial_budget_entries": 177241,
	"hard_max_entries": 354483,
	"within_initial_budget": true,
	"target": {
	"cat": "B",
	"failure_type": "start",
	"category_greedy_fail_rate": 1.0,
	"greedy_failures": 19,
	"sampled_failures": 58,
	"explore_decision_points": 37,
	"greedy_failures_reproduced_in_sampling": 19,
	"score": 38.33333333333333,
	"top_bad_tokens": [
	[
	"```",
	77
	]
	]
	},
	"evidence": "explore split decision points only; see results/analysis/analysis_summary.md",
	"entries": [
	{
	"name": "model.layers.15.conv.out_proj.weight",
	"file_key": "model.layers.15.conv.out_proj.weight",
	"shape": [
	1024,
	1024
	],
	"dtype": "torch.bfloat16",
	"n_changed": 62192,
	"numel": 1048576,
	"delta_l2_fp32": 0.27263545989990234,
	"delta_l2_after_cast": 0.2727110683917999,
	"max_abs_delta_after_cast": 0.0160980224609375,
	"tensor_l2": 16.538354873657227,
	"strategy": "project",
	"direction": "logit",
	"layer": 15,
	"tensor_tag": "op15",
	"cols": [
	4,
	13,
	27,
	31,
	37,
	46,
	69,
	80,
	81,
	96,
	119,
	149,
	155,
	160,
	174,
	191,
	200,
	216,
	226,
	237,
	244,
	252,
	253,
	264,
	276,
	287,
	291,
	318,
	323,
	328,
	330,
	333,
	341,
	380,
	396,
	399,
	403,
	411,
	445,
	454,
	476,
	490,
	491,
	497,
	509,
	511,
	512,
	544,
	605,
	614,
	624,
	645,
	646,
	682,
	691,
	747,
	788,
	851,
	879,
	882,
	888,
	913,
	959,
	1003
	],
	"factor_or_alpha": 1.0,
	"rationale": "subtract 1.0 x the logit direction component from 64 op15 write vectors (rank-1 projection on selected columns)",
	"i": 0
	},
	{
	"name": "model.layers.14.feed_forward.w2.weight",
	"file_key": "model.layers.14.feed_forward.w2.weight",
	"shape": [
	1024,
	4608
	],
	"dtype": "torch.bfloat16",
	"n_changed": 63374,
	"numel": 4718592,
	"delta_l2_fp32": 0.24669404327869415,
	"delta_l2_after_cast": 0.24671411514282227,
	"max_abs_delta_after_cast": 0.012820720672607422,
	"tensor_l2": 24.8846378326416,
	"strategy": "project",
	"direction": "logit",
	"layer": 14,
	"tensor_tag": "ffn14",
	"cols": [
	36,
	138,
	208,
	296,
	307,
	408,
	603,
	615,
	646,
	695,
	722,
	766,
	830,
	850,
	891,
	922,
	933,
	998,
	1106,
	1364,
	1428,
	1489,
	1567,
	1773,
	1788,
	1813,
	1848,
	1853,
	1941,
	2040,
	2125,
	2155,
	2254,
	2265,
	2284,
	2396,
	2397,
	2440,
	2594,
	2749,
	2814,
	2925,
	2942,
	3083,
	3207,
	3246,
	3275,
	3375,
	3407,
	3459,
	3545,
	3604,
	4050,
	4095,
	4172,
	4194,
	4341,
	4402,
	4403,
	4424,
	4442,
	4519,
	4562,
	4585
	],
	"factor_or_alpha": 1.0,
	"rationale": "subtract 1.0 x the logit direction component from 64 ffn14 write vectors (rank-1 projection on selected columns)",
	"i": 1
	}
	],
	"total_changed": 125566,
	"changed_fraction": 0.0003542219432614792
	}