Hugging Face's logo

jonpark0
/
embeddinggemma-2-ko

Sentence Similarity
sentence-transformers
Safetensors
Korean
English
multilingual
embedding_gemma2
embedding
feature-extraction
multimodal-embedding
visual-document-retrieval
korean
retrieval
matryoshka
Model card Files
xet
 Community
embeddinggemma-2-ko / README.md
jonpark0's picture
jonpark0
v0.3: add Korean document-image training (beta 0.1875)
ad57505 verified
preview code
|
Raw History Blame Contribute Delete
20.7 kB
metadata 
license: apache-2.0
base_model: google/embeddinggemma-2
base_model_relation: finetune
pipeline_tag: sentence-similarity
library_name: sentence-transformers
language:
  - ko
  - en
  - multilingual
tags:
  - embedding
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - multimodal-embedding
  - visual-document-retrieval
  - korean
  - retrieval
  - matryoshka
datasets:
  - NomaDamas/ko-vdr-train-public
  - miracl/miracl
  - castorini/mr-tydi
  - Shitao/MLDR
  - nlpai-lab/miracl-multilingual-triplets
  - klue/klue
  - PaDaS-Lab/webfaq
  - rajpurkar/squad
  - wikimedia/wikipedia
  - qwer90qwer/health

EmbeddingGemma 2 โ€” Korean text and document-image retrieval fine-tune (v0.3)

google/embeddinggemma-2 with its text transformer fine-tuned on Korean retrieval data, now including Korean document pages as images. After each round of training the weights are interpolated back toward the previous version, so that the other abilities of the model are kept. It is a drop-in replacement: same architecture, same prompts, same 768-d space (128/256/512 Matryoshka truncation), same image/audio/video inputs.

ํ•œ๊ตญ์–ด ์•ˆ๋‚ด๋Š” ์•„๋ž˜์— ์žˆ์Šต๋‹ˆ๋‹ค.

Status: v0.3 adds Korean document-page images to the training data: about 46k queries over 7.1k pages of Korean public documents (Ko-VDR-Train-Public v2), plus 30k text rows from the earlier training data so that text retrieval holds. Korean document-image retrieval rises from 0.341 to 0.406 nDCG@10 on KoViDoRe v2 (all four tasks up, 95% confidence intervals above zero) and from 0.424 to 0.508 on SDS-KoPub-VDR. Korean text retrieval is 0.729 (v0.2: 0.726), and no image, speech or English check drops by more than 0.5 points; one Korean clustering task drops (see Limitations). The interpolation weight was settled after one look at the evaluation sets; see How ฮฒ was chosen. The previous version stays available at revision v0.2.

Results

All numbers come from our own runs of all models with the same code (mteb 2.24.12, 768-d, full 8K context, bf16). They are not copied from other model cards and may differ from leaderboard setups. Changes are paired bootstrap 95% confidence intervals over queries.

Korean document images: KoViDoRe v2, nDCG@10

Queries are Korean text; documents are page images, passed to the model as images only (no text prompt).

Task Base v0.1 v0.2 v0.3 v0.3 โˆ’ v0.2
KoVidore2HrRetrieval (221 queries) 0.230 0.237 0.242 0.293 +0.051 (+0.036 to +0.068)
KoVidore2EnergyRetrieval (173) 0.414 0.438 0.431 0.522 +0.091 (+0.068 to +0.115)
KoVidore2EconomicRetrieval (163) 0.181 0.184 0.191 0.216 +0.024 (+0.005 to +0.044)
KoVidore2CybersecurityRetrieval (149) 0.502 0.503 0.500 0.592 +0.093 (+0.067 to +0.119)
Average 0.331 0.341 0.341 0.406 +0.065

At 256 dimensions the average goes from 0.287 (base), 0.293 (v0.1) and 0.297 (v0.2) to 0.367 (v0.3).

Korean text retrieval: MTEB(kor, v2) retrieval tasks, nDCG@10

Task Base v0.1 v0.2 v0.3 v0.3 โˆ’ v0.2
MIRACLRetrieval 0.640 0.657 0.669 0.672 +0.002 (โˆ’0.005 to +0.010)
Ko-StrategyQA 0.803 0.805 0.804 0.802 โˆ’0.002 (โˆ’0.005 to +0.001)
LawIRKo 0.561 0.576 0.608 0.625 +0.017 (+0.014 to +0.020)
SQuADKorV1Retrieval 0.889 0.895 0.893 0.892 โˆ’0.001 (โˆ’0.003 to +0.000)
AutoRAGRetrieval 0.798 0.801 0.811 0.804 โˆ’0.007 (โˆ’0.021 to +0.007)
PublicHealthQA 0.794 0.804 0.804 0.806 +0.002 (โˆ’0.005 to +0.009)
BelebeleRetrieval 0.935 0.934 0.934 0.934 +0.000 (โˆ’0.003 to +0.003)
MultiLongDocRetrieval 0.423 0.443 0.443 0.447 +0.004 (โˆ’0.003 to +0.011)
MrTidyRetrieval 0.542 0.555 0.567 0.575 +0.008 (+0.000 to +0.016)
Average 0.709 0.719 0.726 0.729 +0.003

At 256 dimensions the average goes from 0.693 (base), 0.703 (v0.1) and 0.710 (v0.2) to 0.714 (v0.3).

Other modalities and languages

Check Metric Base v0.1 v0.2 v0.3
SDS-KoPub-VDR (Korean document images) nDCG@10 0.404 0.424 0.424 0.508
MIRACL-Vision, Korean nDCG@5 0.595 0.600 0.610 0.605
XM3600, Korean captions โ†’ photos Recall@10 0.906 0.914 0.917 0.920
WIT, Korean nDCG@10 0.592 0.614 0.611 0.614
FLEURS, Korean text โ†’ speech (399 pairs) Recall@1 0.970 0.962 0.972 0.970
GoogleSVQ, Korean speech Hit rate@5 0.582 0.580 0.582 0.581
Belebele, English passages Recall@1 0.906 0.917 0.918 0.914
MTEB(kor, v2) non-retrieval tasks (11) average 0.676 0.685 0.673 0.672

SDS-KoPub-VDR at 256 dimensions: 0.379 (base), 0.392 (v0.1), 0.396 (v0.2), 0.482 (v0.3).

The text transformer is shared by all modalities (image and audio soft tokens pass through the same 24 layers), so any fine-tuning moves every modality. The interpolation weight was chosen to keep these checks within tolerance.

Non-retrieval tasks one by one (base โ†’ v0.1 โ†’ v0.2 โ†’ v0.3): KLUE-TC 0.570 โ†’ 0.573 โ†’ 0.563 โ†’ 0.562, SIB200 clustering 0.429 โ†’ 0.454 โ†’ 0.438 โ†’ 0.451, KlueMrcDomainClustering 0.826 โ†’ 0.846 โ†’ 0.733 โ†’ 0.742, KlueYnatMrcCategoryClustering 0.629 โ†’ 0.693 โ†’ 0.677 โ†’ 0.658, KLUE-NLI 0.669 โ†’ 0.667 โ†’ 0.669 โ†’ 0.668, KorNLI 0.762 โ†’ 0.756 โ†’ 0.755 โ†’ 0.752, PAWS-X 0.487 โ†’ 0.487 โ†’ 0.491 โ†’ 0.490, MIRACL reranking 0.609 โ†’ 0.612 โ†’ 0.618 โ†’ 0.615, KLUE-STS 0.848 โ†’ 0.849 โ†’ 0.853 โ†’ 0.854, KorSTS 0.800 โ†’ 0.796 โ†’ 0.797 โ†’ 0.795, STS17 0.810 โ†’ 0.805 โ†’ 0.806 โ†’ 0.803.

Usage

Requires sentence-transformers>=6.1 and transformers>=5.18. Use it exactly like the base model:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("jonpark0/embeddinggemma-2-ko")

query = "์„œ์šธ์—์„œ ๊ฐ€์žฅ ๋†’์€ ์‚ฐ์€?"
document = "๋ถํ•œ์‚ฐ์€ ์„œ์šธ ๋ถ๋ถ€์— ์žˆ๋Š” ์‚ฐ์œผ๋กœ, ๊ฐ€์žฅ ๋†’์€ ๋ด‰์šฐ๋ฆฌ๋Š” ๋ฐฑ์šด๋Œ€๋‹ค."

query_emb = model.encode(query, prompt_name="SearchQuery")    # "task: search result | query: ..."
doc_emb = model.encode(document, prompt_name="Document")       # "title: none | text: ..."
print(model.similarity(query_emb, doc_emb))

Document pages as images (this is how the KoViDoRe v2 and SDS-KoPub-VDR numbers above were measured):

from PIL import Image

page = Image.open("page_017.png").convert("RGB")
page_emb = model.encode([{"image": page}])                     # image only, no text prompt
query_emb = model.encode("2026๋…„ ๊ณต๋ฌด์› ๊ต์œก ์˜ˆ์‚ฐ์€?", prompt_name="SearchQuery")
print(model.similarity(query_emb, page_emb))
  • Documents with a real title: model.encode(f"title: {title} | text: {body}").
  • Text only (saves memory): SentenceTransformer("jonpark0/embeddinggemma-2-ko", config_kwargs={"vision_config": None, "audio_config": None}).
  • Shorter vectors: model.encode(..., truncate_dim=256, normalize_embeddings=True).
  • Run in bfloat16 or float32, never float16 (same as the base model).
  • Audio and video work as in the base model; see the base model card.
  • Previous versions: SentenceTransformer("jonpark0/embeddinggemma-2-ko", revision="v0.2") (or "v0.1").

Training

v0.3 continues from v0.2. The v0.1 and v0.2 training data and recipe are described in the v0.2 card.

Data (publicly licensed only):

Source License Rows
Ko-VDR-Train-Public v2 (revision 3f03235): Korean queries โ†’ pages of 45 Korean public documents from data.go.kr queries: CC BY 4.0; documents: "No Restrictions" (36) or KOGL Type 1 (9), see Attribution 45,978 queries, 7,111 pages
Text replay: rows of the v0.1 and v0.2 training data (with their negatives), sampled in proportion to source, English SQuAD included as in the v0.2 card 30,000
  • Queries. Ko-VDR's queries were written and filtered by an LLM (Upstage Solar Pro 3) with the kovidore-data-generator pipeline plus rule-based filters; the dataset card describes no human review. A page has about 41 queries on average; we kept a query only while one of its positive pages had fewer than 12 kept queries, which leaves 45,978. All seven query types (multi-hop, compare-contrast, open-ended, boolean, extractive, numerical, enumerative) and three formats (question, keyword, instruction) are kept.
  • Held-out documents. Four whole documents, one per KoViDoRe v2 domain (HR, energy, economy, cybersecurity), were removed from training and used only for choosing ฮฒ (581 queries, 385 pages).
  • Decontamination. Training pages were compared with all 6,729 KoViDoRe v2 pages and a 1-in-4 sample (about 10k pages) of SDS-KoPub-VDR with a 256-bit difference hash. The match threshold was set from comparisons between different training documents before looking at the evaluation pages, and a document is removed when two or more of its pages match. No document was removed; three single training pages fell within the threshold and stayed. No training query equals or nearly equals (5-gram Jaccard โ‰ฅ 0.8) a query of KoViDoRe v2, SDS-KoPub-VDR or MIRACL-Vision. 72 page images that appear byte-identical under two different document names were removed from training and from the held-out set.
  • Page inputs. The vision encoder and its projection were frozen, so each page's soft tokens (at most 280 per page, 270 on average) were computed once and reused; they enter the text transformer through the same path as at inference (checked: cosine โ‰ฅ 0.9999 against the normal image input).
  • Hard negatives. Mined with v0.2 over all 7,111 training pages (TopK-PercPos: 3 per query, scoring below 0.95 ร— the positive).
  • What was trained. Only the text transformer (kept in fp32), starting from v0.2's fp32 weights; the token embedding table and the vision and audio encoders were frozen.
  • Objective. As in v0.2: CachedMultipleNegativesRankingLoss inside MatryoshkaLoss (768/512/256/128), batch 128 drawn from one source at a time, sources sampled proportionally; 1 epoch (601 steps, 4.2 hours on one GPU slice), learning rate 2e-5 with 10% linear warm-up, bf16 autocast.
  • Interpolation. Final weights = v0.2 + ฮฒ ร— (fine-tuned โˆ’ v0.2) with ฮฒ = 0.1875.

How ฮฒ was chosen

We first compared ฮฒ โˆˆ {0.25, 0.5, 0.75, 1.0} on development sets: the four held-out documents for document images, held-out Korean and English text queries, and 1,347 FLEURS Korean training-split recordings for speech. The rule, fixed before training: Korean text, English and speech may each drop by at most 0.01 against v0.2, and among the values that pass, take the best document-image score (within 0.003, the smaller ฮฒ). Larger ฮฒ kept raising the document-image score (0.49 at ฮฒ = 0.25, 0.53 at ฮฒ = 0.5) but failed the speech check, so the rule picked ฮฒ = 0.25.

On the evaluation sets above, ฮฒ = 0.25 reached 0.423 on KoViDoRe v2 and kept text retrieval, but FLEURS Korean text โ†’ speech Recall@1 fell by 4 of 399 queries against v0.2, one query more than the 0.01 tolerance allows. Under the rules that was a failure, and we did not release it. We then added ฮฒ = 0.125 and 0.1875 to the grid and reran the same rules: the development sets still chose 0.25, and the retry rule fixed before training (take the largest passing ฮฒ below the one that failed) gave 0.1875, which passed every check and is this release.

The two added values were chosen after seeing evaluation results, so the evaluation sets have been looked at twice and the pass/fail outcome is post hoc. The comparisons against v0.2 in the tables do not depend on it: every KoViDoRe v2 task is above v0.2 with a 95% interval above zero, and the ฮฒ we released scores lower on KoViDoRe v2 than the one we rejected. The document-image target set before training (KoViDoRe v2 average โ‰ฅ 0.40) is met with a margin of 0.006.

Limitations

  • Matched training data. Ko-VDR-Train-Public v2 was built with the same generator as KoViDoRe v2 (Korean public PDFs, LLM-written queries), so part of the KoViDoRe v2 gain reflects matched domain and query style. SDS-KoPub-VDR, a benchmark of Korean government documents built separately by another group (arXiv 2511.04910), rose by a similar amount (+0.083).
  • Machine-written queries. The new training queries are LLM-generated and may differ from how people actually search.
  • Weakest task. The economy task remains the weakest (0.216).
  • Speech. Against v0.2, one FLEURS query that was found at rank 1 no longer is, and none was gained; at ฮฒ = 0.25 four were lost and none gained. Document-image training nudges Korean text โ†’ speech retrieval down as ฮฒ grows. GoogleSVQ did not move (0.582 โ†’ 0.581).
  • Clustering. KlueYnatMrcCategoryClustering fell from 0.677 to 0.658. At 256 dimensions it fell from 0.700 (v0.2; base 0.675) to 0.619, and KlueMrcDomainClustering from 0.858 (v0.2; base 0.832) to 0.804. These tasks move by up to 0.09 between neighbouring interpolation weights, so they are unstable, but if you cluster Korean text with truncated vectors, compare with v0.2.

License

Apache 2.0, the same as the base model (Gemma 4 license). Training data licenses are listed above and in the v0.2 card: Ko-VDR-Train-Public v2 queries, WebFAQ and qwer90qwer/health are CC BY 4.0; KLUE and SQuAD are CC BY-SA 4.0; Wikipedia is CC BY-SA 3.0 / GFDL. Please credit them when you build on this model.

Attribution

The page images come from Korean public documents published by government agencies and public institutions (via data.go.kr, ๊ณต๊ณต๋ฐ์ดํ„ฐํฌํ„ธ). The full list of 49 documents in Ko-VDR-Train-Public v2, with license, link and how each was used, is in training_documents.csv. Eleven are released under KOGL Type 1 (๊ณต๊ณต๋ˆ„๋ฆฌ ์ œ1์œ ํ˜•: ์ถœ์ฒ˜ํ‘œ์‹œ), which requires attribution. The table follows the attribution given in the Ko-VDR-Train-Public v2 card:

์ด ๋ชจ๋ธ์€ ์•„๋ž˜ ๊ธฐ๊ด€์ด ํ‘œ์˜ ์—ฐ๋„์— ์ž‘์„ฑํ•˜์—ฌ ๊ณต๊ณต๋ˆ„๋ฆฌ ์ œ1์œ ํ˜•์œผ๋กœ ๊ฐœ๋ฐฉํ•œ ์ €์ž‘๋ฌผ์„ ํ•™์Šต๊ณผ ๊ฒ€์ฆ์— ์ด์šฉํ–ˆ์œผ๋ฉฐ, ๊ฐ ์ €์ž‘๋ฌผ์€ ํ‘œ์˜ ์ถœ์ฒ˜์—์„œ ๋ฌด๋ฃŒ๋กœ ๋‚ด๋ ค๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Work (๊ธฐ๊ด€, ์—ฐ๋„, ์ €์ž‘๋ฌผ๋ช…) Source Used for
๊ณผํ•™๊ธฐ์ˆ ์ •๋ณดํ†ต์‹ ๋ถ€ ๊ตญ๋ฆฝ์ „ํŒŒ์—ฐ๊ตฌ์›, 2024, 'ICT ์œตใƒป๋ณตํ•ฉ ์‹œ์„ค์˜ ์•ˆ์ „ํ•œ ์ „์žํŒŒ ํ™˜๊ฒฝ ๊ธฐ๋ฐ˜ ์กฐ์„ฑ ์—ฐ๊ตฌ' ๊ณต๊ณต๋ฐ์ดํ„ฐํฌํ„ธ training
๊ณผํ•™๊ธฐ์ˆ ์ •๋ณดํ†ต์‹ ๋ถ€ ๊ตญ๋ฆฝ์ „ํŒŒ์—ฐ๊ตฌ์›, 2024, '์ „์žํŒŒ ํก์ˆ˜์ „๋ ฅ ๋ฐ€๋„ ๋“ฑ ์ „์žํŒŒ ์ธ์ฒด๋…ธ์ถœ๋Ÿ‰ ํ‰๊ฐ€๊ธฐ์ˆ  ์—ฐ๊ตฌ' ๊ณต๊ณต๋ฐ์ดํ„ฐํฌํ„ธ training
๊ณผํ•™๊ธฐ์ˆ ์ •๋ณดํ†ต์‹ ๋ถ€ ์ค‘์•™์ „ํŒŒ๊ด€๋ฆฌ์†Œ, 2026, '2025๋…„ 4๋ถ„๊ธฐ ์œ„์„ฑ์ „ํŒŒ ๊ฐ์‹œ๋™ํ–ฅ' ์ค‘์•™์ „ํŒŒ๊ด€๋ฆฌ์†Œ ์œ„์„ฑ์ „ํŒŒ๊ฐ์‹œ์ •๋ณด training
ํ•ด์–‘์ˆ˜์‚ฐ๋ถ€, 2021, '๋ฌด์ธ๋„์„œ ๋ฐฑ์„œ' (๋ฌด์ธ๋„์„œ 100์„ ) ํ•ด์–‘์ˆ˜์‚ฐ๋ถ€ ๋ฌด์ธ๋„์„œ์ข…ํ•ฉ์ •๋ณด์‹œ์Šคํ…œ training
๊ฐœ์ธ์ •๋ณด๋ณดํ˜ธ์œ„์›ํšŒ, 2025, 'ํ•ฉ์„ฑ๋ฐ์ดํ„ฐ ์ƒ์„ฑยทํ™œ์šฉ ์•ˆ๋‚ด์„œ' ๊ฐœ์ธ์ •๋ณด๋ณดํ˜ธ์œ„์›ํšŒ ๋ฐœ๊ฐ„์ž๋ฃŒ training
๊ฐœ์ธ์ •๋ณด๋ณดํ˜ธ์œ„์›ํšŒ, 2024, '์ƒ์ฒด์ •๋ณด ๋ณดํ˜ธ ์•ˆ๋‚ด์„œ' ๊ฐœ์ธ์ •๋ณด๋ณดํ˜ธ์œ„์›ํšŒ ๋ฐœ๊ฐ„์ž๋ฃŒ training
ํ•œ๊ตญ์–ธ๋ก ์ง„ํฅ์žฌ๋‹จ, 2022, '๊ด‘๊ณ ์š”๊ธˆ์ œ ๋„์ž…์„ ์•ž๋‘” ๋„ทํ”Œ๋ฆญ์Šค์— ๋Œ€ํ•œ ์ธ์‹ ๋ฐ ์ด์šฉ ์กฐ์‚ฌ' ๊ณต๊ณต๋ฐ์ดํ„ฐํฌํ„ธ training
ํ•œ๊ตญ์–ธ๋ก ์ง„ํฅ์žฌ๋‹จ, 2022, '์ด๋Œ€๋‚จ ํ˜„์ƒ์— ๋Œ€ํ•œ ์ธ์‹' ๊ณต๊ณต๋ฐ์ดํ„ฐํฌํ„ธ training
ํ•œ๊ตญ์–ธ๋ก ์ง„ํฅ์žฌ๋‹จ, 2020, '์ฝ”๋กœ๋‚˜19(COVID-19) ๊ด€๋ จ ์ •๋ณด ์ด์šฉ ๋ฐ ์ธ์‹ ํ˜„ํ™ฉ' ๊ณต๊ณต๋ฐ์ดํ„ฐํฌํ„ธ training
ํ•œ๊ตญ์›์ž๋ ฅํ™˜๊ฒฝ๊ณต๋‹จ, 2025, '๋ฐฉ์‚ฌ์„ฑํ๊ธฐ๋ฌผ ์ฒ˜๋ถ„์‹œ์„ค ๋ถ€์ง€์ฃผ๋ณ€ ๋ฐฉ์‚ฌ์„ ํ™˜๊ฒฝ๋ณด๊ณ ์„œ' ๊ณต๊ณต๋ฐ์ดํ„ฐํฌํ„ธ held out (choosing ฮฒ only)
๊ฐœ์ธ์ •๋ณด๋ณดํ˜ธ์œ„์›ํšŒ, 2024, '๊ฐœ์ธ์ •๋ณด ์œ ์ถœ ๋“ฑ ์‚ฌ๊ณ  ๋Œ€์‘ ๋งค๋‰ด์–ผ' ๊ฐœ์ธ์ •๋ณด๋ณดํ˜ธ์œ„์›ํšŒ ๋ฐœ๊ฐ„์ž๋ฃŒ held out (choosing ฮฒ only)

Citation

Ko-VDR-Train-Public v2 and the KoViDoRe benchmark come from:

@inproceedings{choi-etal-2026-kovidore,
    title = "{KoViDoRe}: A Benchmark for {K}orean Visual Document Retrieval",
    author = "Choi, Yongbin and Song, Yongwoo and Sung, Mujeen",
    booktitle = "Proceedings of the 2nd Workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR 2026)",
    year = "2026",
    doi = "10.18653/v1/2026.magmar-main.11",
    url = "https://aclanthology.org/2026.magmar-main.11/"
}

ํ•œ๊ตญ์–ด

google/embeddinggemma-2์˜ ๊ธ€ ํŠธ๋žœ์Šคํฌ๋จธ๋ฅผ ํ•œ๊ตญ์–ด ๊ฒ€์ƒ‰ ๋ฐ์ดํ„ฐ๋กœ ๋ฏธ์„ธ์กฐ์ •ํ–ˆ์Šต๋‹ˆ๋‹ค. v0.3๋ถ€ํ„ฐ๋Š” ํ•œ๊ตญ์–ด ๋ฌธ์„œ ์ชฝ ์ด๋ฏธ์ง€๋„ ํ•™์Šต์— ๋„ฃ์—ˆ์Šต๋‹ˆ๋‹ค. ํ•™์Šตํ•  ๋•Œ๋งˆ๋‹ค ์ด์ „ ๋ฒ„์ „ ์ชฝ์œผ๋กœ ๋˜๋Œ๋ ค ์„ž์–ด ๋‹ค๋ฅธ ๊ธฐ๋Šฅ์„ ์ง€์ผฐ์Šต๋‹ˆ๋‹ค. ๊ตฌ์กฐ, ์ ‘๋‘์–ด, 768์ฐจ์› ๊ณต๊ฐ„(128/256/512 ์ถ•์†Œ ๊ฐ€๋Šฅ), ์ด๋ฏธ์ง€ยท์Œ์„ฑยท์˜์ƒ ์ž…๋ ฅ์ด ์›๋ž˜ ๋ชจ๋ธ๊ณผ ๊ฐ™์•„ ๊ทธ๋Œ€๋กœ ๋ฐ”๊ฟ” ์“ธ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

  • v0.3์—์„œ ๋ฐ”๋€ ์ : v0.2์— ์ด์–ด, ๊ณต๊ณต๋ฐ์ดํ„ฐํฌํ„ธ ๋ฌธ์„œ 45๊ฐœ์˜ ์ชฝ 7,111์žฅ๊ณผ ์งˆ์˜ ์•ฝ 4.6๋งŒ ๊ฐœ(Ko-VDR-Train-Public v2)๋กœ ํ•™์Šตํ–ˆ์Šต๋‹ˆ๋‹ค. ํ…์ŠคํŠธ ๊ฒ€์ƒ‰์„ ์ง€ํ‚ค๋ ค๊ณ  ์ด์ „ ํ•™์Šต ๋ฐ์ดํ„ฐ 3๋งŒ ํ–‰์„ ํ•จ๊ป˜ ๋„ฃ์—ˆ์Šต๋‹ˆ๋‹ค. ์ด์ „ ๋ฒ„์ „์€ revision="v0.2"๋กœ ์“ธ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • ๊ฒฐ๊ณผ:
    • ํ•œ๊ตญ์–ด ๋ฌธ์„œ ์ด๋ฏธ์ง€ ๊ฒ€์ƒ‰ KoViDoRe v2 nDCG@10์ด v0.2 0.341์—์„œ 0.406์œผ๋กœ ์˜ฌ๋ž์Šต๋‹ˆ๋‹ค. ๋„ค ๊ณผ์ œ ๋ชจ๋‘ 95% ์‹ ๋ขฐ๊ตฌ๊ฐ„์ด 0๋ณด๋‹ค ํฝ๋‹ˆ๋‹ค.
    • ๋”ฐ๋กœ ๋งŒ๋“  ํ‰๊ฐ€ ์„ธํŠธ์ธ SDS-KoPub-VDR๋„ 0.424์—์„œ 0.508๋กœ ์˜ฌ๋ž์Šต๋‹ˆ๋‹ค.
    • ํ…์ŠคํŠธ ๊ฒ€์ƒ‰ 9๊ณผ์ œ ํ‰๊ท ์€ 0.729(v0.2 0.726)์ž…๋‹ˆ๋‹ค.
    • ์‚ฌ์ง„, ์Œ์„ฑ, ์˜์–ด ์ ๊ฒ€์€ 0.5์  ๋„˜๊ฒŒ ๋–จ์–ด์ง„ ๊ฒƒ์ด ์—†์Šต๋‹ˆ๋‹ค.
  • ฮฒ๋ฅผ ์ •ํ•œ ๊ณผ์ • (์‚ฌํ›„ ๊ฒฐ์ •): ์ฒ˜์Œ ์ •ํ•œ ฮฒ 0.25๋Š” ํ‰๊ฐ€ ์„ธํŠธ์˜ FLEURS ์Œ์„ฑ ๊ฒ€์ƒ‰์—์„œ v0.2๋ณด๋‹ค 399๋ฌธํ•ญ ์ค‘ 4๋ฌธํ•ญ์„ ๋” ๋†“์ณค์Šต๋‹ˆ๋‹ค. ๋ฏธ๋ฆฌ ์ •ํ•œ ํ—ˆ์šฉ์น˜(3๋ฌธํ•ญ)๋ฅผ ๋„˜์–ด ๊ณต๊ฐœํ•˜์ง€ ์•Š์•˜์Šต๋‹ˆ๋‹ค. ๊ทธ ๋’ค ฮฒ 0.125์™€ 0.1875๋ฅผ ํ›„๋ณด์— ๋”ํ•˜๊ณ  ๊ฐ™์€ ๊ทœ์น™์„ ๋‹ค์‹œ ์ ์šฉํ•ด 0.1875๋ฅผ ์–ป์—ˆ๊ณ , ์ด ๊ฐ’์€ ๋ชจ๋“  ์ ๊ฒ€์„ ํ†ต๊ณผํ–ˆ์Šต๋‹ˆ๋‹ค. ํ›„๋ณด๋ฅผ ๋”ํ•œ ๊ฒƒ์€ ํ‰๊ฐ€ ๊ฒฐ๊ณผ๋ฅผ ๋ณธ ๋’ค์ด๋ฏ€๋กœ ํ†ต๊ณผ ์—ฌ๋ถ€๋Š” ์‚ฌํ›„ ํŒ์ •์ž…๋‹ˆ๋‹ค. v0.2 ๋Œ€๋น„ ๊ฐœ์„ ์€ ์‹ ๋ขฐ๊ตฌ๊ฐ„์œผ๋กœ ๋”ฐ๋กœ ํ™•์ธ๋ฉ๋‹ˆ๋‹ค.
  • ์ฃผ์˜:
    • ํ•™์Šต ๋ฐ์ดํ„ฐ๊ฐ€ KoViDoRe v2์™€ ๊ฐ™์€ ์ƒ์„ฑ๊ธฐ(๊ณต๊ณต PDF, LLM์ด ์“ด ์งˆ์˜)๋กœ ๋งŒ๋“ค์–ด์ ธ ๋ถ„์•ผ์™€ ์งˆ์˜ ๋ชจ์–‘์ด ๋งž์ถฐ์ ธ ์žˆ์Šต๋‹ˆ๋‹ค.
    • ๊ฒฝ์ œ ๊ณผ์ œ๋Š” ์—ฌ์ „ํžˆ ๊ฐ€์žฅ ๋‚ฎ์Šต๋‹ˆ๋‹ค(0.216).
    • FLEURS ์Œ์„ฑ ๊ฒ€์ƒ‰์—์„œ 1๋ฌธํ•ญ์„ ๋” ๋†“์นฉ๋‹ˆ๋‹ค.
    • 256์ฐจ์›์—์„œ ํ•œ๊ตญ์–ด ๊ตฐ์ง‘ํ™” ๋‘ ๊ณผ์ œ๊ฐ€ ๋–จ์–ด์กŒ์Šต๋‹ˆ๋‹ค(KlueYnat 0.700 โ†’ 0.619, KlueMrcDomain 0.858 โ†’ 0.804). ์ถ•์†Œ ๋ฒกํ„ฐ๋กœ ๊ตฐ์ง‘ํ™”ํ•œ๋‹ค๋ฉด v0.2์™€ ๋น„๊ตํ•ด ๋ณด์„ธ์š”.
  • ์‚ฌ์šฉ๋ฒ•: ์œ„ Usage์™€ ๊ฐ™์Šต๋‹ˆ๋‹ค.
    • ์งˆ์˜๋Š” prompt_name="SearchQuery", ๊ธ€ ๋ฌธ์„œ๋Š” prompt_name="Document"๋ฅผ ์”๋‹ˆ๋‹ค.
    • ๋ฌธ์„œ ์ชฝ ์ด๋ฏธ์ง€๋Š” ์ ‘๋‘์–ด ์—†์ด model.encode([{"image": ๊ทธ๋ฆผ}])๋กœ ๋„ฃ์Šต๋‹ˆ๋‹ค.
    • float16์€ ์“ฐ์ง€ ๋งˆ์„ธ์š”.
  • ๋ผ์ด์„ ์Šค: ์›๋ž˜ ๋ชจ๋ธ๊ณผ ๊ฐ™์€ Apache 2.0์ž…๋‹ˆ๋‹ค. ๋‹ค์Œ ์ถœ์ฒ˜๋ฅผ ํ•จ๊ป˜ ๋ฐํ˜€ ์ฃผ์„ธ์š”.
    • Ko-VDR-Train-Public v2(CC BY 4.0), ๊ณต๊ณต๋ˆ„๋ฆฌ ์ œ1์œ ํ˜• ๋ฌธ์„œ(์œ„ Attribution ํ‘œ)
    • KLUEยทSQuAD(CC BY-SA 4.0), ์œ„ํ‚ค๋ฐฑ๊ณผ(CC BY-SA 3.0/GFDL), WebFAQยทqwer90qwer/health(CC BY 4.0)