FinTermIE-EventPrice-KO

🇰🇷 한국어 카드: MODEL_CARD.ko.md

Status: EXPERIMENTAL RELEASE. On a pre-registered test set of issuers never seen in training, this model beat our rule baseline by a wide margin on both fields, and every value it returned was read from the text at the span it reports. It failed one pre-registered safety gate: when the pre-adjustment price is not written in the text, it answered anyway more often than the rule baseline (3 of 46 documents versus 1 of 46). There is no accuracy guarantee for production use.

FinTermIE-EventPrice-KO reads the narrative section of a Korean conversion-price adjustment disclosure (조정근거 및 방법) and returns two numbers: the price before the adjustment and the price after it. It covers convertible bonds (전환가액), bonds with warrants (신주인수권 행사가액) and exchangeable bonds (교환가액). When a price is not written in the narrative, the model is expected to return nothing for that field.

It is a full fine-tune of XLM-RoBERTa base (278M parameters) with a small span-role head. It does not generate text. Deterministic code lists every number in the narrative as a candidate; the model scores each candidate as before, after or neither; a value is returned only if it clears a threshold against a "not written" option. The answer is always one of the numbers in the input, with its character span.

0. How to use

pip install -r requirements.txt
python fintermie_eventprice.py --selftest
python fintermie_eventprice.py --model . --example examples/synthetic_notice.json

from huggingface_hub import snapshot_download
import sys
d = snapshot_download("ThakiCloud/FinTermIE-EventPrice-KO"); sys.path.insert(0, d)
import fintermie_eventprice as ep

model, tok, cfg = ep.load(d, device="cpu")          # or "cuda"
prose = "위 산정에 따라 전환가액을 조정 전 6,050원에서 조정 후 4,686원으로 조정한다."
r = ep.extract(model, tok, cfg, prose)
print(r["before"], r["after"], r["before_span"], r["after_span"])

Output on the shipped examples

examples/synthetic_notice.json holds three narratives written by hand for this card. This is what the released checkpoint returns for them on CPU. The first one is a miss, and it is left in on purpose.

Example	Written in the text	Model output	Margins (before / after)
`both_present`	before 6,050 · after 4,686	before `6050` · after not returned	4.75 / −3.73
`before_not_written`	after 2,251 only	before not returned · after `2251`	−6.52 / 1.41
`no_price_in_narrative`	neither	neither (no candidate numbers)	—

In the first example the narrative says "조정 전 6,050원에서 조정 후 4,686원으로 조정한다" and the model still abstained on after. Hand-written text is not the wording of real filings, so this says little about accuracy, but it shows the failure you are most likely to meet: a missing answer, not an invented one.

Things that must match training, all handled by fintermie_eventprice.py:

Input is the narrative text only. Do not feed the disclosure's table; the table is where the training labels came from.
Candidates are numbers with thousands separators, or plain numbers with three or more integer digits. Parts of dates are skipped. Values are normalized by removing commas and trailing decimal zeros (6,050 → 6050).
Windows of 512 tokens with stride 128; a candidate's score is its best window.
Thresholds are shipped in eventprice_config.json (before −1.0, after −1.5, on the margin between a candidate and the "not written" option). They were chosen on the development set only. Raising them makes the model abstain more.
One span, one role. If both fields pick the same span, the field with the larger margin keeps it.

Output fields: before, after (string or None), before_span, after_span ([start, end] character offsets), before_margin, after_margin, n_candidates.

1. Results on unseen issuers

Test set E-Issuer: 258 disclosures from 134 issuers, filed 2019–2024. None of these issuers appears in the training set, the development set, or any document we looked at while building the rule baseline. The set was fixed, and the final checkpoint and thresholds were committed, before anything was scored on it.

The gold values are the before/after prices in each disclosure's own table. Accuracy below is the share of all 258 documents where the returned value equals the table value. A document whose narrative does not contain the value counts as a miss for every system, so the ceiling is 0.82 for before (212 of 258 are answerable) and 0.97 for after (250 of 258).

System	before accuracy [95% CI]	after accuracy [95% CI]
This model	0.798 [0.731, 0.863]	0.938 [0.900, 0.969]
Rule baseline v1 (lexical rules)	0.601 [0.523, 0.679]	0.729 [0.655, 0.798]
Rule baseline v0	0.539 [0.458, 0.618]	0.721 [0.648, 0.789]
Qwen3.8-27B, zero-shot, fixed prompt	0.802 [0.732, 0.868]	0.922 [0.880, 0.961]
Qwen3-0.6B + LoRA (our decoder arm)	0.783 [0.716, 0.851]	0.888 [0.840, 0.929]

Confidence intervals are bootstrap intervals resampled by issuer. All systems read the same input file and were scored by the same code.

Paired difference, this model minus rule v1: +0.198 [0.132, 0.263] on before, +0.209 [0.149, 0.272] on after. Against the 27B model the differences are −0.004 [−0.030, 0.021] and +0.016 [−0.020, 0.054]; both intervals include zero, so we do not claim this model is better or worse than the 27B model. What it does offer at similar accuracy on this set is a 278M encoder that returns a checked span instead of generated text.

Detail for this model:

	before	after
Answerable documents	212	250
Correct on answerable	206 (0.972)	242 (0.968)
Wrong value on answerable	2	3
Abstained on answerable	4	5
Documents where the value is not written	46	8
Answered anyway on those	3 (6.5%)	1 (12.5%)
Before/after swapped	1	0
Share of documents answered	81.8%	95.3%
Correct end to end (right value, or silent when not written)	0.965	0.965

By form type the gain over rule v1 on before holds in both strata: +0.180 [0.114, 0.249] on 217 regular disclosures and +0.293 [0.122, 0.474] on 41 notice-type disclosures (안내공시).

Pre-registered gates

Gate	Result	Basis
G1 before vs rule v1: difference ≥ +0.08 and CI lower bound > 0	PASS	+0.198 [0.132, 0.263]
G2 after vs rule v1: difference ≥ −0.03	PASS	+0.209
G3 answers on "not written" documents ≤ rule v1, per field	FAIL	before: 3 vs 1 of 46. after: 1 vs 1 of 8, too few to judge
G4 every answered value is exactly the number at its reported span	PASS	457 of 457 on E-Issuer, 123 of 123 on E-Temporal

The verdict rule required G3 to pass or be underpowered. It failed on before, so the verdict is EXPERIMENTAL RELEASE, not a validated release. Gates and thresholds were fixed before the evaluation and were not changed afterwards.

2. Results on 2025 filings (underpowered)

Test set E-Temporal: 67 disclosures filed in 2025 from 54 issuers. The pre-registered minimum was 100 documents, so this set is reported as underpowered and carries no verdict.

System	before accuracy	after accuracy
This model	0.821 [0.729, 0.908]	0.955 [0.900, 1.000]
Rule baseline v1	0.836 [0.746, 0.917]	0.925 [0.855, 0.985]
Qwen3.8-27B, zero-shot	0.836 [0.746, 0.919]	0.955 [0.899, 1.000]

This model minus rule v1 on before: −0.015 [−0.066, 0.032]. There is no evidence here that the model beats the rules on later filings. Two caveats cut in opposite directions. The rule baseline was written while looking at 2025 filings (those documents and their bond series were removed from this set, but the rules may still fit 2025 wording), so it is not a time-independent baseline. And only 38 of the 54 issuers actually have documents in the training set (46 documents); the other 16 issuers (21 documents) lost all their training documents to de-duplication, so they are not "known issuer, later filing" cases. On the 46 seen-issuer documents this model got 35 before and 44 after right, rule v1 36 and 43.

3. Cost

	Parameters	Throughput	Memory
This model, one H200 GPU	278M	300 documents/s	1.7 GB GPU
This model, 8 CPU cores	278M	0.9 documents/s	not measured
Qwen3-0.6B + LoRA, one H200 GPU	0.6B	2.9 documents/s	5.4 GB GPU
Qwen3.8-27B endpoint, 4 concurrent requests	27B	1.7 documents/s (wall clock, shared endpoint)	—

CPU and GPU runs returned identical values on all 325 test documents. Per-document latency percentiles were not measured.

4. Limitations

Answers when it should stay silent. On E-Issuer it returned a before price for 3 of 46 documents whose narrative does not state one. Do not treat a returned value as proof that the narrative states that price; check the span.
A matching span is not a correct answer. Gate G4 checks that each returned number is literally the number at the reported span (457 of 457). It does not check that the number is the right price; a wrong value read from the right kind of place still passes G4. Correctness is the accuracy table, not G4.
The "not written" behaviour for after is untested. Only 8 test documents lack an after price.
Later filings are not validated. See section 2.
Single bond series only. Disclosures that adjust several series at once were excluded from training and evaluation because the table could not be matched to the narrative automatically.
Labels come from tables, not from people. The gold value is the disclosure's own table. If the table parser misread a table, that error is in the labels. No human or LLM labelled anything.
Training labels were cleaned with wording cues. When a table value appeared at several places in a narrative, positions whose left context named the opposite role were removed from the training targets, and ambiguous fields were left out of the loss. This was applied to training data only, never to test data.
Model selection was close. The final checkpoint was the best of 12 runs on a 141-document development set, where the eight encoder runs were within one or two documents of each other. A KoELECTRA run of 112M parameters scored almost the same there; it was not scored on the test sets and is not released.
A timing caveat on test-set isolation. The label-cleaning rule for training data was written down and committed after the split was fixed but while 14 of the E-Issuer source documents had already been downloaded. They had not been parsed, labelled or scored, and the rule is applied to training data only.
Numbers the candidate rule skips cannot be returned, for example a price below 100 written without a comma.
One scoring mistake, disclosed. The first scoring command passed the model's file under the wrong argument name, so that run printed baseline numbers without this model. The command was corrected and rerun with the same predictions, thresholds and gates.

5. Training data

2,030 conversion-price adjustment disclosures from 592 issuers, filed 2019–2024, collected from the Financial Supervisory Service's public disclosure system DART (금융감독원 전자공시시스템). At most 8 disclosures per issuer.
Development set: 141 disclosures from 69 other issuers.
Issuers were assigned to train, development or test by a hash of the issuer code, so no issuer is shared. Near-duplicate narratives and disclosures of a bond series already seen during rule development were removed.
No disclosure text is included in this repository. The example file contains three narratives written by hand for this card.

6. Training

Full fine-tune, 6 epochs, learning rate 3e-5, batch 16, AdamW, mixed precision, seed 1. Each field is a softmax over the candidates plus a "not written" option. When a value appears several times the loss sums over those positions instead of picking one. A margin term pushes the best wrong candidate below the right one, and in every epoch 20% of training documents have the sentence containing one field's value deleted, to teach abstention. The checkpoint is epoch 5, chosen on the development set.

7. Files

model.safetensors (encoder), span_role_head.safetensors (two linear layers), eventprice_config.json (thresholds, window sizes), tokenizer files of the base model, fintermie_eventprice.py, examples/.