Malina Claude Opus 5 (1M context) commited on
Commit
9257486
Β·
1 Parent(s): 8413646

Notatka nazywa pliki tak, jak uzytkownik je wgral, i mowi czemu ktorys odlozyla

Browse files

Dwie wady wyszly na przebiegu korpusu "co to jest oaza i czym sie zajmuje".

1. Model przepisuje kazda nazwe pliku, ktora cytuje, i po drodze ja po cichu
"poprawia". Plik nazwany formacja_do2ONZ_szanse_i_zagrozenia.pdf wrocil w
Sources jako ...zagroΕΌenia.pdf, z kreska, ktorej autor nie postawil.
Sources i linia odlozonych to jedyne rozliczenie, jakie uzytkownik dostaje
za swoj upload, wiec nazwa, ktorej nie da sie dopasowac do wyslanego pliku,
to realna wada, nie kosmetyka.

Naprawione w kodzie, nie promptem: instrukcje "nie zmieniaj pisowni" model
nadpisuje dokladnie wtedy, gdy pisownia wyglada mu na bledna. Porownanie
znosi wielkosc liter, znaki diakrytyczne i separatory, wiec lapie tez
"wrzesien 2024.pdf" zamiast "wrzesien-2024.pdf" i dziala na nazwach ze
spacjami. Na przebiegu: 2 naprawione nazwy.

2. Linia odlozonych prosila o jeden wspolny powod, i to opcjonalnie, wiec
model pomijal go calkiem. Teraz powod jest wymagany przy kazdym pliku i
ma nazywac, czym ten plik JEST. Przed: sama lista nazw. Po: "allogol.png
(ikona aplikacji), filech3_0 (plik danych), logonnnl.svg (grafika
wektorowa)". To wprost odpowiada na skarge, ze LUNA "ignoruje wazne pliki":
decyzja jest teraz widoczna i mozna ja podwazyc.

Ustalone przy okazji, nie jest to blad: wrzesien-2024.pdf dostaje od MAP
ocene "low", bo ten plik ani razu nie nazywa ruchu - nie ma w nim slowa
"oaza" ani "Swiatlo-Zycie". MAP ocenia kazdy plik osobno, wiec nie widzi, ze
pozostale ustalaja kontekst. Pliki z ocena "low" i tak docieraja do REDUCE z
pelnymi notatkami, wiec to REDUCE swiadomie je odklada, zgodnie z zalozeniem.

Testy: 30 -> 36.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Files changed (2) hide show
  1. synthesis.py +78 -2
  2. test_luna.py +44 -1
synthesis.py CHANGED
@@ -4,6 +4,7 @@
4
  # REDUCE - fuse all kept notes into one structured Note (strong model, streamed).
5
  import json
6
  import re
 
7
 
8
  from config import (
9
  DEBUG, MAP_MAX_TOKENS, MAP_MODEL, MAP_TEMPERATURE, REDUCE_BUDGET,
@@ -414,8 +415,14 @@ def _reduce_system(language: str) -> str:
414
  "group related facts even when they came from different files.\n"
415
  "4. '## Sources' - a bullet per file you ACTUALLY used, formatted "
416
  "`- **<filename>** - <what it contributed>`. Then, if any files were set aside, one "
417
- "final line: `_Not used: <comma-separated filenames>_` (with a brief shared reason "
418
- "if useful). If every file was used, omit the Not-used line.\n\n"
 
 
 
 
 
 
419
  "Be thorough but do not repeat yourself. Optimize for a reader skimming the Note."
420
  )
421
 
@@ -483,6 +490,71 @@ def _reduce_user(prompt: str, kept: list[dict], skipped: list[dict], fmt: str =
483
  )
484
 
485
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
486
  def reduce_stream(prompt: str, kept: list[dict], skipped: list[dict], fmt: str = ""):
487
  """Stream the final Note from the strong model. Yields the accumulated Markdown.
488
  The model mirrors the request's language itself (more reliable than langdetect,
@@ -524,12 +596,16 @@ def reduce_stream(prompt: str, kept: list[dict], skipped: list[dict], fmt: str =
524
  yield acc
525
  if not acc.strip():
526
  raise RuntimeError("empty REDUCE response")
 
 
 
527
  except Exception as e:
528
  print(f"[LUNA] REDUCE error: {e}")
529
  # If the stream already produced a real Note, keep it. Replacing a nearly
530
  # finished Note with a raw stitch of the notes throws away the better answer.
531
  if len(acc.strip()) > 400:
532
  print(f"[LUNA] keeping the partial Note ({len(acc)} chars)")
 
533
  yield acc.rstrip() + "\n\n_(The Note was cut short: the synthesis stream " \
534
  "ended early. Everything above is complete.)_"
535
  return
 
4
  # REDUCE - fuse all kept notes into one structured Note (strong model, streamed).
5
  import json
6
  import re
7
+ import unicodedata
8
 
9
  from config import (
10
  DEBUG, MAP_MAX_TOKENS, MAP_MODEL, MAP_TEMPERATURE, REDUCE_BUDGET,
 
415
  "group related facts even when they came from different files.\n"
416
  "4. '## Sources' - a bullet per file you ACTUALLY used, formatted "
417
  "`- **<filename>** - <what it contributed>`. Then, if any files were set aside, one "
418
+ "final line: `_Not used: <name> (<short reason>), <name> (<short reason>)_`. Give a "
419
+ "reason for EVERY set-aside file. Each reason must name what THAT file actually is - "
420
+ "its subject, or that it is a photo, a logo, a spreadsheet, a data dump - and then "
421
+ "why that did not answer the request. Never write the same reason twice: if two "
422
+ "reasons come out identical you have described the request, not the file. "
423
+ "Files listed under 'Files set aside' above already carry a reason; use it. "
424
+ "A reader who uploaded a file must be able to tell a deliberate decision from a "
425
+ "file that was missed. If every file was used, omit the Not-used line.\n\n"
426
  "Be thorough but do not repeat yourself. Optimize for a reader skimming the Note."
427
  )
428
 
 
490
  )
491
 
492
 
493
+ def _fold(s: str) -> str:
494
+ """Case, diacritics and separators removed, so two spellings of one filename meet."""
495
+ s = unicodedata.normalize("NFKD", s or "")
496
+ s = "".join(c for c in s if not unicodedata.combining(c))
497
+ return re.sub(r"[^a-z0-9]+", "", s.lower())
498
+
499
+
500
+ def _fix_filenames(note: str, names: list[str]) -> str:
501
+ """Put the real filenames back.
502
+
503
+ The model retypes every filename it cites, and a strong model will quietly
504
+ "correct" one on the way past: a file actually called
505
+ `formacja_do2ONZ_szanse_i_zagrozenia.pdf` came back as `...zagroΕΌenia.pdf`,
506
+ with the diacritic the author had left out. The Note's Sources and Not-used
507
+ lines are the only accounting a user gets for what happened to their upload,
508
+ so a name they cannot match against the file they sent is a real fault, not a
509
+ cosmetic one. Repaired here rather than in the prompt, because an instruction
510
+ not to touch a spelling is exactly the kind a model overrides when the spelling
511
+ looks wrong to it.
512
+
513
+ Matching folds case, diacritics and separators, so it also catches
514
+ `wrzesien 2024.pdf` for `wrzesien-2024.pdf`, and works on names with spaces.
515
+ """
516
+ if not note or not names:
517
+ return note
518
+ missing = [n for n in names if n and n not in note]
519
+ if not missing:
520
+ return note
521
+ # fold the note, remembering where each surviving character came from
522
+ keep, pos = [], []
523
+ flat = unicodedata.normalize("NFKD", note)
524
+ src = []
525
+ i = 0
526
+ for ch in note:
527
+ for c in unicodedata.normalize("NFKD", ch):
528
+ src.append(i)
529
+ i += 1
530
+ for j, c in enumerate(flat):
531
+ if unicodedata.combining(c):
532
+ continue
533
+ lo = c.lower()
534
+ if re.match(r"[a-z0-9]", lo):
535
+ keep.append(lo)
536
+ pos.append(src[j])
537
+ folded = "".join(keep)
538
+ edits = []
539
+ for real in missing:
540
+ target = _fold(real)
541
+ if not target:
542
+ continue
543
+ start = folded.find(target)
544
+ while start != -1:
545
+ a = pos[start]
546
+ b = pos[start + len(target) - 1]
547
+ edits.append((a, b + 1, real))
548
+ start = folded.find(target, start + len(target))
549
+ if not edits:
550
+ return note
551
+ out = note
552
+ for a, b, real in sorted(edits, key=lambda e: -e[0]):
553
+ out = out[:a] + real + out[b:]
554
+ print(f"[LUNA] repaired {len(edits)} filename(s) in the Note")
555
+ return out
556
+
557
+
558
  def reduce_stream(prompt: str, kept: list[dict], skipped: list[dict], fmt: str = ""):
559
  """Stream the final Note from the strong model. Yields the accumulated Markdown.
560
  The model mirrors the request's language itself (more reliable than langdetect,
 
596
  yield acc
597
  if not acc.strip():
598
  raise RuntimeError("empty REDUCE response")
599
+ fixed = _fix_filenames(acc, [k["name"] for k in kept] + [x["name"] for x in skipped])
600
+ if fixed != acc:
601
+ yield fixed
602
  except Exception as e:
603
  print(f"[LUNA] REDUCE error: {e}")
604
  # If the stream already produced a real Note, keep it. Replacing a nearly
605
  # finished Note with a raw stitch of the notes throws away the better answer.
606
  if len(acc.strip()) > 400:
607
  print(f"[LUNA] keeping the partial Note ({len(acc)} chars)")
608
+ acc = _fix_filenames(acc, [k["name"] for k in kept] + [x["name"] for x in skipped])
609
  yield acc.rstrip() + "\n\n_(The Note was cut short: the synthesis stream " \
610
  "ended early. Everything above is complete.)_"
611
  return
test_luna.py CHANGED
@@ -12,7 +12,8 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
12
 
13
  from extract import _looks_binary, _word_density, category_of, extract # noqa: E402
14
  from pipeline import _resolve, _validate # noqa: E402
15
- from synthesis import _parse_json, _reduce_user, detect_language # noqa: E402
 
16
 
17
 
18
  # ── language ──────────────────────────────────────────────────────────────────
@@ -258,3 +259,45 @@ def test_malformed_svg_still_gives_up_its_words():
258
  assert "Plan spotkania" in r["payload"]
259
  assert "800 x 600" in r["payload"]
260
  assert "iVBORw" not in r["payload"]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
  from extract import _looks_binary, _word_density, category_of, extract # noqa: E402
14
  from pipeline import _resolve, _validate # noqa: E402
15
+ from synthesis import (_fix_filenames, _fold, _parse_json, # noqa: E402
16
+ _reduce_user, detect_language) # noqa: E402
17
 
18
 
19
  # ── language ──────────────────────────────────────────────────────────────────
 
259
  assert "Plan spotkania" in r["payload"]
260
  assert "800 x 600" in r["payload"]
261
  assert "iVBORw" not in r["payload"]
262
+
263
+
264
+ # --- the Note has to name files the way the user sent them -------------------
265
+
266
+ def test_fold_ignores_case_diacritics_and_separators():
267
+ assert _fold("wrzesien-2024.pdf") == _fold("Wrzesien 2024.PDF")
268
+ assert _fold("zagrozenia.pdf") == _fold("zagroΕΌenia.pdf")
269
+ assert _fold("OND_ERΕ»_O Ε›wiecie.pdf") == _fold("ond_erz_o swiecie.pdf")
270
+
271
+
272
+ def test_model_correcting_a_filename_is_undone():
273
+ """Observed: a file called ..._zagrozenia.pdf came back with the diacritic added."""
274
+ real = ["formacja_do2ONZ_szanse_i_zagrozenia.pdf"]
275
+ note = "## Sources\n- **formacja_do2ONZ_szanse_i_zagroΕΌenia.pdf** - formacja"
276
+ assert "zagrozenia.pdf" in _fix_filenames(note, real)
277
+ assert "zagroΕΌenia.pdf" not in _fix_filenames(note, real)
278
+
279
+
280
+ def test_filenames_with_spaces_and_missing_diacritics_are_repaired():
281
+ real = ["OND_ERΕ»_O Ε›wiecie wirtualnym.pdf", "wrzesien-2024.pdf"]
282
+ note = "_Not used: OND_ERZ_O swiecie wirtualnym.pdf, wrzesien 2024.pdf_"
283
+ out = _fix_filenames(note, real)
284
+ assert "OND_ERΕ»_O Ε›wiecie wirtualnym.pdf" in out
285
+ assert "wrzesien-2024.pdf" in out
286
+
287
+
288
+ def test_correct_note_is_left_exactly_as_it_is():
289
+ real = ["a.pdf", "b.docx"]
290
+ note = "## Sources\n- **a.pdf** - x\n\n_Not used: b.docx (zdjecie)_"
291
+ assert _fix_filenames(note, real) == note
292
+
293
+
294
+ def test_repair_does_not_invent_a_mention():
295
+ real = ["nieobecny.pdf"]
296
+ note = "## Sources\n- **a.pdf** - x"
297
+ assert _fix_filenames(note, real) == note
298
+
299
+
300
+ def test_every_mention_of_a_name_is_repaired():
301
+ real = ["wrzesien-2024.pdf"]
302
+ note = "wrzesien 2024.pdf w tekscie, oraz _Not used: wrzesien 2024.pdf_"
303
+ assert _fix_filenames(note, real).count("wrzesien-2024.pdf") == 2