wayneworkman2012 commited on
Commit
954cf5c
·
verified ·
1 Parent(s): 5bad978

Add wayneworkman2012/peacebell-v1-148M to the Instruct models (additive, Peacebell-only path)

Browse files

Hi, I'd like to add **peacebell-v1-148M** to the Instruct models: https://huggingface.co/wayneworkman2012/peacebell-v1-148M

148.55M parameters, trained from scratch, Apache-2.0, ChatML chat model. It is a single-domain model: it only knows World War II and redirects most other topics back to the war. It is a first version and I would like it out there next to the others.

I know you normally take requests as a discussion and add the row yourself. I opened a PR because a row alone would not work for this model, and I wanted to hand you something already tested. Feel free to apply it your own way instead of merging.

**Why a row alone is not enough**

I ran this Space's own code (everything above the UI section of `app.py`) against the model locally, on CPU. It loads fine through `_load_tokenizer` / `_load_model`, but every reply came back as `[Generation Error] 'GPTForCausalLM' object has no attribute 'generate'`. Two reasons, both by design in my repo:
- the model ships its own KV-cached decoder (`stream_generate`) instead of HF `generate()`;
- it has no Jinja `chat_template`: its ChatML control tokens are injected by id exactly as at training time (`tokenizer.build_chatml_ids`), which a string template re-tokenized by SentencePiece cannot reproduce.

**What this PR changes** (`app.py`: 29 lines added, 0 removed or edited; `README.md`: 1 line added)
- one row in `INSTRUCT_MODEL_ROWS` and one line in the README `models:` list;
- `_is_peacebell_spec()` and `_stream_peacebell_text()`, plus a three-line branch in `_stream_single_text` after the shared model load and stop-signal reset. It yields the same cumulative partial text as `_stream_text_from_model`, honours Max tokens and the Stop button, and reports failures in the same `[Generation Error] ...` form, so a problem with this model cannot take down a run.

No new dependency: `sentencepiece` is already in `requirements.txt`.

**Decoding: greedy on purpose.** Peacebell is trained and validated at temperature 0 with no repetition penalty, so this path always decodes that way. The temperature, top-p and repetition-penalty sliders do not apply to it; its answers are deterministic.

**Nothing else is affected.** No existing line is modified and other models never enter the new branch. Checked on CPU with 2 threads, under both transformers 5.14.1 / torch 2.8 and your pinned setup (transformers `main` 5.18.0.dev0, torch 2.14):
- control entrants run through the original and the patched `app.py` with identical seeds give byte-identical output: `HuggingFaceTB/SmolLM2-135M-Instruct` (chat-template path) and `HuggingFaceTB/SmolLM2-135M` (raw-prompt path);
- a mixed three-model run through `_stream_arena_outputs` with Peacebell in the middle works, including the two-model cache eviction;
- Stop ends the Peacebell stream on the next token;
- Peacebell itself: loads in under 2 s, about 0.6 GB in float32, a 113-token answer streams in about 2 s on 2 CPU threads.

Sample replies through the patched code:
- "When did World War II begin and end?" -> "World War II began in 1939 and ended in 1945."
- "Who commanded the Allied forces on D-Day?" -> "General Dwight D. Eisenhower commanded the Allied forces on D-Day."
- "Tell me a joke." -> "I won't do that. We can go over World War II together, if you'd like to explore it."

Thanks for running the arena.

Files changed (2) hide show
  1. README.md +1 -0
  2. app.py +29 -0
README.md CHANGED
@@ -90,6 +90,7 @@ models:
90
  - ThingAI/Quark-50m
91
  - joelhenwang/OdinNext-138M-Instruct
92
  - MinimaLabs/KeyLM-75M-Instruct
 
93
  tags:
94
  - text-generation
95
  - small-language-model
 
90
  - ThingAI/Quark-50m
91
  - joelhenwang/OdinNext-138M-Instruct
92
  - MinimaLabs/KeyLM-75M-Instruct
93
+ - wayneworkman2012/peacebell-v1-148M
94
  tags:
95
  - text-generation
96
  - small-language-model
app.py CHANGED
@@ -212,6 +212,7 @@ INSTRUCT_MODEL_ROWS = [
212
  ("Quark-50M", "ThingAI/Quark-50m", "ThingAI", 50.0),
213
  ("OdinNext-138M-Instruct", "joelhenwang/OdinNext-138M-Instruct", "joelhenwang", 138.0),
214
  ("KeyLM-75M-Instruct", "MinimaLabs/KeyLM-75M-Instruct", "MinimaLabs", 75.0),
 
215
  ]
216
 
217
 
@@ -771,6 +772,31 @@ def _commentary_panel_text(state: dict[str, Any]) -> str:
771
  return "Commentary is locked until Reveal."
772
 
773
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
774
  def _stream_single_text(spec: ModelSpec, prompt: str, max_tokens: int, temperature: float, top_p: float, repetition_penalty: float):
775
  try:
776
  tokenizer, model = _get_model(spec)
@@ -779,6 +805,9 @@ def _stream_single_text(spec: ModelSpec, prompt: str, max_tokens: int, temperatu
779
  return
780
 
781
  interrupt_callback.stop_signal = False
 
 
 
782
  inputs = _move_inputs(_prepare_inputs(tokenizer, prompt, is_instruct=_is_instruct_spec(spec)), model)
783
  for partial in _stream_text_from_model(
784
  model,
 
212
  ("Quark-50M", "ThingAI/Quark-50m", "ThingAI", 50.0),
213
  ("OdinNext-138M-Instruct", "joelhenwang/OdinNext-138M-Instruct", "joelhenwang", 138.0),
214
  ("KeyLM-75M-Instruct", "MinimaLabs/KeyLM-75M-Instruct", "MinimaLabs", 75.0),
215
+ ("peacebell-v1-148M", "wayneworkman2012/peacebell-v1-148M", "wayneworkman2012", 148.6),
216
  ]
217
 
218
 
 
772
  return "Commentary is locked until Reveal."
773
 
774
 
775
+ def _is_peacebell_spec(spec: ModelSpec) -> bool:
776
+ return spec.repo_id.lower().startswith("wayneworkman2012/peacebell-")
777
+
778
+
779
+ def _stream_peacebell_text(model, tokenizer, prompt: str, max_tokens: int):
780
+ """Peacebell ships its own ChatML encoder and KV-cached decoder instead of HF generate() and a Jinja
781
+ chat_template: its control tokens are injected by id, exactly as at training time. It is trained and
782
+ validated greedy, so it always decodes at temperature 0 with no repetition penalty; the temperature,
783
+ top-p and repetition-penalty sliders do not apply to it. Max tokens and Stop are honoured."""
784
+ try:
785
+ ids = tokenizer.build_chatml_ids([{"role": "user", "content": prompt}], add_generation_prompt=True)
786
+ input_ids = torch.tensor(ids, dtype=torch.long)
787
+ out_ids: list[int] = []
788
+ with torch.inference_mode():
789
+ for token_id in model.stream_generate(
790
+ input_ids, max_new_tokens=int(max_tokens), temperature=0.0, no_repeat_ngram_size=0
791
+ ):
792
+ if interrupt_callback.stop_signal:
793
+ break
794
+ out_ids.append(int(token_id))
795
+ yield tokenizer.decode_response(out_ids)
796
+ except Exception as exc:
797
+ yield f"[Generation Error] {str(exc) or exc.__class__.__name__}"
798
+
799
+
800
  def _stream_single_text(spec: ModelSpec, prompt: str, max_tokens: int, temperature: float, top_p: float, repetition_penalty: float):
801
  try:
802
  tokenizer, model = _get_model(spec)
 
805
  return
806
 
807
  interrupt_callback.stop_signal = False
808
+ if _is_peacebell_spec(spec):
809
+ yield from _stream_peacebell_text(model, tokenizer, prompt, max_tokens)
810
+ return
811
  inputs = _move_inputs(_prepare_inputs(tokenizer, prompt, is_instruct=_is_instruct_spec(spec)), model)
812
  for partial in _stream_text_from_model(
813
  model,