--- library_name: peft base_model: Qwen/Qwen3.5-0.8B-Base pipeline_tag: text-classification tags: - biojev - biomedical - qwen3.5 - peft - qlora - natural-language-inference - text-classification - biomedical-nlp - system-one language: - en --- # BioJev-Nano **BioJev-Nano** is the 0.8B member of the BioJev biomedical decision-model family. It is built from **Qwen/Qwen3.5-0.8B-Base** and follows the full BioJev adaptation pipeline: ```text Qwen3.5-0.8B-Base │ ▼ Biomedical DAPT PubMed + PMC │ ▼ General NLI SNLI + MNLI + ANLI │ ▼ Biomedical NLI BioNLI + NLI4CT │ ▼ BioJev-Nano ``` The released checkpoint is a **PEFT/QLoRA sequence-classification adapter** with three NLI labels: ```text 0 → contradiction 1 → entailment 2 → neutral ``` BioJev-Nano is primarily used for lightweight inference, controlled ablation experiments, and analysis of how biomedical domain adaptation and NLI specialization affect transfer. ## Links - **BioJev-Nano weights:** https://huggingface.co/Gabriel382/BioJev-Nano - **BioJev-4B weights:** https://huggingface.co/Gabriel382/BioJev - **Source code:** https://github.com/Gabriel382/BioJev ## Authors **Creator:** Gabriel Henrique Alencar Medeiros **Supervisor:** Lina F. Soualmia BioJev is developed at **LITIS / Université de Rouen Normandie**. --- ## Training recipe ### Biomedical DAPT The Nano model uses a lightweight biomedical DAPT stage initialized from Qwen3.5-0.8B-Base. ```text Biomedical token budget: 50,000 Sequence length: 512 Method: QLoRA ``` The biomedical corpus combines PubMed and PMC sources. ### General NLI ```text SNLI 35,000 MNLI 45,000 ANLI R1 20,000 ---------------- Total 100,000 ``` ### Biomedical NLI ```text BioNLI 30,000 NLI4CT 10,000 ---------------- Total 40,000 ``` The released checkpoint corresponds to the **full** Nano configuration: ```text DAPT → General NLI → Biomedical NLI ``` --- ## Evaluation BioJev-Nano was evaluated on five biomedical benchmarks. | Dataset | Macro-F1 | |---|---:| | BioNLI | 89.41 | | NLI4CT | 46.07 | | ChemProt | 14.33 | | DDI2013 | 32.55 | | BioRED | 16.16 | | Relation-transfer mean (ChemProt/DDI2013/BioRED) | 21.01 | These scores correspond to the full Nano configuration used in the BioJev Sprint 6 ablation study. ### Interpretation The Nano ablations indicate that: - general NLI training provides a strong contribution to transfer toward unseen biomedical tasks; - biomedical NLI training strongly specializes the model toward biomedical inference; - this specialization can reduce transfer to some unseen relation-classification tasks; - biomedical DAPT has smaller and more task-dependent effects at the Nano scale. BioNLI and NLI4CT can be seen during biomedical decision training depending on the experiment, so they should not be interpreted as clean zero-shot transfer tasks for the full model. --- # Loading BioJev-Nano Because BioJev is a **three-class PEFT sequence classifier**, the base model must be reconstructed with `num_labels=3` before loading the adapter. Do **not** rely on a default two-label sequence-classification configuration. Install: ```bash pip install -U torch transformers peft accelerate ``` For optional 4-bit loading: ```bash pip install -U bitsandbytes ``` ## Python example ```python import torch from peft import PeftConfig, PeftModelForSequenceClassification from transformers import AutoModelForSequenceClassification, AutoTokenizer MODEL = "Gabriel382/BioJev-Nano" LABEL2ID = { "contradiction": 0, "entailment": 1, "neutral": 2, } ID2LABEL = {v: k for k, v in LABEL2ID.items()} peft_config = PeftConfig.from_pretrained(MODEL) base = AutoModelForSequenceClassification.from_pretrained( peft_config.base_model_name_or_path, num_labels=3, label2id=LABEL2ID, id2label=ID2LABEL, dtype=torch.bfloat16 if torch.cuda.is_available() else torch.float32, device_map="auto" if torch.cuda.is_available() else None, ) model = PeftModelForSequenceClassification.from_pretrained( base, MODEL, is_trainable=False, ) tokenizer = AutoTokenizer.from_pretrained(MODEL) if tokenizer.pad_token_id is None: tokenizer.pad_token = tokenizer.eos_token model.config.pad_token_id = tokenizer.pad_token_id model.eval() premise = "The clinical report describes a bacterial pneumonia." hypothesis = "The patient has an infectious pulmonary disease." inputs = tokenizer( premise, hypothesis, return_tensors="pt", truncation=True, max_length=512, ) device = next(model.parameters()).device inputs = {k: v.to(device) for k, v in inputs.items()} with torch.inference_mode(): logits = model(**inputs).logits[0].float() probabilities = torch.softmax(logits, dim=-1) for label, idx in LABEL2ID.items(): print(f"{label:14s}: {probabilities[idx].item():.4f}") ``` A standalone version of this example is included in `example_inference.py`. --- # Jev / System One compatibility API The BioJev GitHub repository includes a compatibility layer that exposes existing BioJev sequence-classification checkpoints through a Jev/System-One-shaped API: ```text POST /v1/systemone ``` Supported question types are: ```text choice noul score ``` This is an **engineering compatibility bridge** over the BioJev NLI classifier. BioJev-Nano is not a native Ollama/System-One checkpoint. ## Install the BioJev API Clone the main project: ```bash git clone https://github.com/Gabriel382/BioJev.git cd BioJev python -m venv .venv source .venv/bin/activate pip install -e . pip install fastapi uvicorn ``` Download or clone this model repository and point the server to its local directory. Example: ```bash python scripts/serve_systemone.py \ --checkpoint /path/to/BioJev-Nano \ --model-name biojev-nano \ --load-in-4bit \ --host 0.0.0.0 \ --port 8000 ``` Check the service: ```bash curl http://127.0.0.1:8000/health ``` ## System One choice example ```bash curl http://127.0.0.1:8000/v1/systemone \ -H "Content-Type: application/json" \ -d '{ "model": "biojev-nano", "state": "The patient has fever, productive cough, and a new lobar infiltrate.", "questions": { "diagnosis": { "type": "choice", "instructions": "Which diagnosis is best supported?", "criteria": { "pneumonia": "Community-acquired pneumonia", "asthma": "Acute asthma exacerbation", "migraine": "Migraine" } } } }' ``` For each candidate, the bridge constructs a premise/hypothesis NLI pair and computes: ```text support(candidate) = entailment_logit - contradiction_logit ``` Candidate supports are normalized with a softmax. For robust Qwen3.5/PEFT compatibility, candidates are evaluated sequentially with `batch_size=1`. --- ## Repository files A typical BioJev-Nano model repository contains: ```text BioJev-Nano/ ├── adapter_config.json ├── adapter_model.safetensors ├── tokenizer.json ├── tokenizer_config.json ├── chat_template.jinja # if present in the training checkpoint ├── training_manifest.json # if present ├── README.md ├── example_inference.py └── requirements.txt ``` The PEFT adapter does not duplicate the Qwen3.5-0.8B base weights. The base model is downloaded separately when BioJev-Nano is loaded. --- ## Scientific status BioJev is a research project. The model, its confidence estimates, and the System One compatibility layer are **not validated clinical decision systems**. They must not be used as the sole basis for diagnosis, treatment, or other high-stakes medical decisions. --- ## Citation A formal paper citation will be added when the BioJev publication is available. Until then, please cite the BioJev repository and the specific Hugging Face checkpoint used. --- ## Acknowledgements BioJev-Nano was created by **Gabriel Henrique Alencar Medeiros** under the supervision of **Lina F. Soualmia** at **LITIS / Université de Rouen Normandie**. BioJev builds on the Qwen3.5 model family and publicly available biomedical and natural-language-inference datasets.