AlexGu1/da_dacy_large_ner_fine_grained_torch_zg

DaCy fine-grained NER — native PyTorch weight port

Original model: da_dacy_large_ner_fine_grained.

Zero retraining. Original encoder and Thinc NER head matrices are preserved; native Python greedy BILUO decoding is provided. This is not a standard AutoModelForTokenClassification model. Use the custom example below to run NER. The user does not need spacy or spacy-transformers installed.

Quick start: NER without spaCy (CPU or NVIDIA GPU)

PyTorch must already be installed. Install only the lightweight support packages:

pip install transformers safetensors huggingface_hub

Run this Python script or Jupyter cell:

import sys
from huggingface_hub import snapshot_download
REPO_ID = 'AlexGu1/da_dacy_large_ner_fine_grained_torch_zg'
model_dir = snapshot_download(
    repo_id=REPO_ID,
    allow_patterns=[
        "encoder/**", "ner_head.safetensors",
        "native_torch_metadata.json", "tokenizer_rules.json",
        "dacy_torch_native.py", "dacy_tokenizer_standalone.py",
    ],
)
sys.path.insert(0, model_dir)
from dacy_torch_native import DaCyTorchNER

# auto selects CUDA if an NVIDIA GPU is available, otherwise CPU.
# Set device="cpu" to force CPU or device="cuda" for NVIDIA GPU.
ner = DaCyTorchNER(model_dir, device="auto")
print("Device:", ner.device)
print(ner("Mette arbejder i København.")["entities"])

Expected result format (NER predictions vary by sentence):

[{"text": "...", "start": 0, "end": 5, "label": "..."}]

This implementation needs no spaCy installation in the inference environment. The model includes a standalone Danish tokenizer generated from the original spaCy tokenizer rules. It runs through a non-standard nn.Module rather than a Transformers token-classification pipeline.

Accuracy / conversion status

Review native_validation.json and native_torch_metadata.json to see whether this actual checkpoint passed the original spaCy vs. PyTorch token-boundary and entity-span tests. These smoke tests do not establish global parity on unseen, long or clinical text. Use additional held-out data before substituting this port for the original model in a production workflow.

What is in the repository?

• encoder/: copied original pretrained transformer + HF tokenizer
• ner_decoder.safetensors: all original trained Thinc NER arrays
• ner_head.safetensors: original matrices mapped for PyTorch scoring
• native_torch_metadata.json: ordered BILUO actions and model settings
• tokenizer_rules.json: exported Danish spaCy tokenizer patterns
• dacy_torch_native.py: native PyTorch scorer and Python BILUO decoder
• dacy_tokenizer_standalone.py: spaCy-free tokenizer compatibility port
• example_inference.py: runnable CPU/GPU beginner example
• native_validation.json: comparison against the source model

Attribution

Original: da_dacy_large_ner_fine_grained; originally Apache-2.0 licensed. The spaCy-derived tokenizer compatibility implementation is independent runtime code and should be assessed against spaCy MIT license conditions. No new supervised NER training was performed.