---
license: apache-2.0
base_model: BAAI/bge-reranker-v2-m3
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-classification
language:
- ko
tags:
- reranker
- cross-encoder
- dart
- opendart
- financial-disclosure
- korean
- record-linkage
- convertible-bond
---

# EvidenceLink-DART-Reranker-KO

🇰🇷 한국어 카드: [MODEL_CARD.ko.md](./MODEL_CARD.ko.md)

> **PASS_V24 (G1–G5 passed) — the verdict of one pre-registered experiment (`prereg_E_v2_4`), not a general claim.**
> Sealed test = 648 correction filings from 79 issuers never seen in training. Gold links come from DART's own
> correction chain, not from human review. Natural "no match" cases were too few (25 < 30), so gate G6 is
> **UNDERPOWERED** and generalization of the NONE decision to natural cases is **not established**.

A Korean cross-encoder that links a DART **correction filing** for a convertible-bond issuance decision
(`[기재정정] 주요사항보고서(전환사채권발행결정)`) back to the **original filing (the right bond series/round)** it amends,
or answers **NONE** when the original is not in the candidate set. It is a full fine-tune of
[`BAAI/bge-reranker-v2-m3`](https://huggingface.co/BAAI/bge-reranker-v2-m3) trained only on public OpenDART filings.

The hard part is not finding the issuer. It is choosing among several filings of the **same issuer** (different series,
same day, near-identical tables) and refusing to link when none of them is right.

## At a glance

| | |
|---|---|
| Task | correction filing → one of K candidate originals, or NONE |
| Scope | DART convertible-bond issuance decisions (전환사채권발행결정) and their corrections, filed 2022–2026 |
| Sealed test (CLEAN, n=499) | Top-1 **0.978** vs BM25 0.7375 · rule T0 0.7174 |
| NONE precision / recall (CLEAN) | **1.0 / 0.9308** (BM25 0.5548 / 0.6615) |
| Same-issuer wrong-link rate (CLEAN) | **0.0246** (BM25 0.1387 · T0 0.1432) |
| Paired issuer-cluster bootstrap (10,000) | Top-1 vs BM25 **+0.2387**, 95% CI [0.1645, 0.3091] |
| Decision rule | argmax score over candidates; NONE if the top logit < **−8.4375** (frozen on dev before the sealed run) |
| Weights | `model.safetensors` sha256 `a402de3dc80fa8d45b42551ef381f063ec997fdba7d5d6e31462108fd48f6c19` |
| Training data | OpenDART public filings only. No AI-Hub data. No filing text is shipped here. |

## What it does

| Output | Meaning |
|---|---|
| `pred = <candidate id>` | the candidate original filing this correction amends (highest score, score ≥ threshold) |
| `pred = None` | NONE: no candidate scored above the frozen threshold |
| `scores` | raw logit per candidate (pointwise; not calibrated probabilities) |

## Quick start

```bash
pip install torch transformers huggingface_hub
hf download ThakiCloud/EvidenceLink-DART-Reranker-KO --local-dir evidencelink
cd evidencelink
python evidencelink.py --selftest                       # decoder + card format checks (no model)
python evidencelink.py --model . --example examples/synthetic_query.json
```

```python
from evidencelink import load, query_card, candidate_card, link, dart_zip_to_text

tok, model = load(".")                                   # fp32, CPU or GPU
q = query_card(corp_name, correction_text)               # text from dart_zip_to_text(<OpenDART document.xml zip>)
cands = [{"id": rcept_no, "card": candidate_card(corp_name, rcept_dt, text)}
         for rcept_no, rcept_dt, text in originals]      # order: receipt number descending
print(link(tok, model, q, cands))                        # {"pred": ... or None, "top_score": ..., "scores": {...}}
```

**Use the input format in `evidencelink.py`.** The model was trained on a specific card: a `[요약]` summary line
(series, principal, conversion price, coupon, yield, refixing floor) followed by the masked filing text
(first-submission date and 14-digit receipt numbers are replaced by `[MASK]`/`[RCEPT]`). Raw filing text gives
different scores, and the frozen threshold no longer applies.

**Decode OpenDART documents with the included decoder (`dart_zip_to_text`).** Filings before mid-2022 are cp949.
Reading them as UTF-8 with `errors="replace"` silently destroys the fields the model relies on (see Limitations).

`examples/synthetic_query.json` is a fictional issuer generated from a template. It is not a DART filing.

## What is in this package

| File | Role |
|---|---|
| `model.safetensors` | fine-tuned weights (fp32), sha256 `a402de3dc80fa8d4…` |
| `config.json`, `tokenizer.json`, `tokenizer_config.json` | model config and tokenizer (unchanged from the base) |
| `evidencelink.py` | decoder v2, field extraction, card builder, scoring, NONE rule |
| `examples/synthetic_query.json` | synthetic query + 3 candidates |
| `LICENSE`, `NOTICE` | Apache-2.0, attribution |

## How the candidate set is built (evaluation protocol)

For each correction filing: up to 10 original filings of the same issuer received within the previous 3 years
(most recent first), plus up to 3 filings of other issuers received within ±10 days (hard negatives).
Candidates are ordered by receipt number, descending. For 0.9499 of CLEAN queries the true original falls inside this
window; the remaining 25 queries are the natural NONE cases (original filed more than 3 years earlier). In another
105 CLEAN queries the true original was removed on purpose to create NONE cases (NONE_SYNTH below).

## Sealed test results (sealed4, prereg_E_v2_4)

648 correction-filing queries from 79 issuers. Issuers in sealed4 were excluded from training and dev.
**CLEAN** (n=499; 369 linked, 130 NONE) drops queries whose card shows the receipt date of any candidate, so a model
cannot shortcut through a visible date. CLEAN is the primary set for all gates.

### CLEAN (primary, n=499)

| Arm | Top-1 | Linked acc | NONE precision | NONE recall | Same-issuer wrong-link rate |
|---|---|---|---|---|---|
| **This model (H3, seed 0)** | **0.978** | **0.9946** | **1.0** | **0.9308** | **0.0246** |
| BM25 (card vs card, NONE threshold on dev) | 0.7375 | 0.7642 | 0.5548 | 0.6615 | 0.1387 |
| T0 rule (field-overlap count, NONE threshold on dev) | 0.7174 | 0.6856 | 0.5899 | 0.8077 | 0.1432 |
| bge-m3 bi-encoder, zero-shot | 0.5431 | 0.5339 | 0.3289 | 0.5692 | 0.1723 |

All arms were measured by us, in the same sealed job, on the same candidate sets.

### Confidence intervals (issuer-cluster paired bootstrap, 10,000 resamples, seed 1109)

| Difference in Top-1 | Mean | 95% CI |
|---|---|---|
| This model − BM25, CLEAN | +0.2387 | [0.1645, 0.3091] |
| This model − T0, CLEAN | +0.2573 | [0.1635, 0.3497] |
| This model − BM25, ALL | +0.2263 | [0.1616, 0.2907] |
| This model − T0, ALL | +0.2633 | [0.1747, 0.3499] |

ALL (n=648) Top-1: this model 0.9722 · BM25 0.7454 · T0 0.7068.

### Pre-registered gates

| Gate | Condition (CLEAN) | Result |
|---|---|---|
| G1 | Top-1 vs BM25, bootstrap CI lower bound > 0 | pass |
| G2 | same-issuer wrong-link rate ≤ 0.05 and below BM25 and T0 | pass (0.0246) |
| G3 | NONE precision and recall ≥ 0.80 | pass (1.0 / 0.9308) |
| G4 | linked accuracy ≥ 0.90 | pass (0.9946) |
| G5 | ALL Top-1 ≥ BM25 | pass (0.9722 vs 0.7454) |
| G6 | natural-NONE recall ≥ 0.80, evaluated only if n ≥ 30 | **UNDERPOWERED** (n=25) — excluded from the verdict |

### By filing year (CLEAN Top-1)

| Year | n | This model | BM25 | T0 |
|---|---|---|---|---|
| 2022 | 100 | 0.97 | 0.57 | 0.68 |
| 2023 | 67 | 0.9552 | 0.7463 | 0.7612 |
| 2024 | 140 | 0.9857 | 0.8143 | 0.7286 |
| 2025 | 109 | 0.9817 | 0.7798 | 0.7064 |
| 2026 | 83 | 0.988 | 0.747 | 0.7229 |

### NONE by stratum (CLEAN)

| Stratum | n | This model | BM25 | T0 |
|---|---|---|---|---|
| NONE_SYNTH (true original removed from the set) | 105 | 0.9333 | 0.6 | 0.7714 |
| NONE_NATURAL (true original outside the 3-year window) | 25 | 0.92 | 0.92 | 0.96 |

On the 25 natural NONE cases the model is **not** better than the rule baseline (T0 0.96). With n=25 none of these
numbers separates the arms.

## Limitations — do not overstate

- **Verdict scope.** PASS_V24 is the verdict of `prereg_E_v2_4` on sealed4 only. It does not certify other filing types,
  other years, or customer documents (contracts, spreadsheets).
- **G6 UNDERPOWERED.** Only 25 natural NONE cases exist in sealed4 (target ≥ 30). Most NONE cases (105 of 130 in CLEAN)
  were built by removing the true original from the candidate set. Whether the NONE decision generalizes to natural
  "no original in range" cases is **not established**.
- **Gold is not human-reviewed.** The gold original comes from DART's correction chain (the filing family DART itself
  links when a correction is submitted). It is rule/chain based. Training labels are weaker still: a rule that matches
  the first-submission date written in the correction (that date is masked in the model input). On CLEAN queries where
  the weak rule decides, it agrees with gold 0.8868 of the time.
- **Dev has almost no 2022–2023 data.** Dev = 93 queries from 11 issuers (2022: 0, 2023: 1, 2024: 26, 2025: 31,
  2026: 35). Epoch and the NONE threshold were selected on that dev. The 2022–2023 sealed numbers above are therefore
  out-of-distribution for model selection.
- **Truncation.** All checked pairs (4,000 train, 432 dev) exceed 1024 tokens and are truncated (`longest_first`). The `[요약]`
  summary line is placed first so it survives; filing text deep in the form is not seen.
- **Encoding dependency (decoder v2).** OpenDART documents before mid-2022 are cp949. An earlier pipeline decoded them as
  UTF-8 with replacement characters, which damaged 1,427 cached documents and collapsed field extraction for 2022–2023
  (the cause of the previous NO_GO on sealed3). With decoder v2, query series/principal extraction on sealed4 is
  0.948–0.992 per year. If you feed text decoded another way, expect lower accuracy.
- **sealed3 re-score is a report-only diagnostic.** Re-scoring the earlier H2 model (not this checkpoint) on the
  already-exposed sealed3 set after the decoder fix raised its CLEAN Top-1 from 0.6571 to 0.9455 on the same 385
  queries. That set was exposed before, so it is evidence about the input defect, not a pass criterion.
- **Candidate window.** Only originals filed within 3 years before the correction (10 most recent of the same issuer)
  are candidates. An older original can never be linked; the correct output is then NONE.
- **Seed.** Two seeds were trained; seed 1 tied on dev (0.9085) and seed 0 was chosen by the pre-registered tie rule
  before sealed evaluation. Seed 1 was not evaluated on sealed4.

## Training settings (from pre-registration)

| | |
|---|---|
| Base | `BAAI/bge-reranker-v2-m3`, full fine-tune, pointwise BCE |
| Hyperparameters | max_len 1024, batch 16, 2 epochs (best epoch 1), lr 2e-5, positive weight 3 |
| Negatives | all other candidates in the set (same-issuer other series + other-issuer near-date) |
| Train | 1,430 queries, 205 issuers, 2022–2026 |
| Dev | 93 queries, 11 issuers; selection = macro average of per-year Top-1 (0.9085) |
| Sealed4 | 648 queries, 79 issuers, evaluated once |

## Data source

OpenDART (Financial Supervisory Service, Korea) public disclosure filings: convertible-bond issuance decisions and
their corrections, accessed through the OpenDART API. This repository contains **no** filing text, extracted fields,
queries, candidate sets, or labels. **No AI-Hub data was used.**

## License

Apache-2.0, matching the base model `BAAI/bge-reranker-v2-m3` (Apache-2.0). See `LICENSE` and `NOTICE`.