AeroxLabs/Aerova-Mini

Aerova Mini 2B

Compact Medical AI by AeroxLabs

Aerova Mini 2B is a compact medical AI model developed by AeroxLabs, designed for medical-domain language tasks and evaluation. Distributed in GGUF format, it can be used with compatible local inference engines for experimentation, research, and educational applications.

This release includes aerova-mini-2b.gguf and benchmark results comparing Aerova Mini with MedGemma 4B, Qwen3 1.7B, and MedGemma 27B-text-it across five evaluation benchmarks.

Highlights

• Medical AI: Developed for medical-domain language tasks.
• Compact 2B-class model: A smaller parameter class than the 4B and 27B models included in this evaluation.
• GGUF format: Supports compatible local inference runtimes.
• Benchmark-driven evaluation: Evaluated on HealthBench, HealthBench Hard, MMLU-Health, PubMedQA, and MedXpertQA.
• Local experimentation: Can be run on compatible hardware without requiring a hosted inference API.

Model details

Property	Value
Model name	Aerova Mini 2B
Developer	AeroxLabs
Model file	aerova-mini-2b.gguf
Format	GGUF
Parameter class	2B
Quantization	Q8_0

Benchmark results

Aerova Mini 2B was evaluated against three comparison models across five medical and biomedical benchmarks.

Results table

Benchmark	Aerova Mini 2B	MedGemma 4B	Qwen3 1.7B	MedGemma 27B-text-it
HealthBench	70.33	54.00	53.00	54.00
HealthBench Hard	54.33	29.67	28.33	42.00
MMLU-Health	82.72	67.69	72.49	90.48
PubMedQA	69.53	58.73	72.33	71.93
MedXpertQA	21.28	15.80	11.60	25.18
Average	59.638	45.178	47.550	56.718

Benchmark visualization

Results analysis

Based on the reported scores:

• HealthBench: Aerova Mini 2B scores 70.33, ahead of all three comparison models.
• HealthBench Hard: Aerova Mini 2B scores 54.33, again ahead of all three comparison models.
• MMLU-Health: MedGemma 27B-text-it leads with 90.48, followed by Aerova Mini 2B at 82.72.
• PubMedQA: Qwen3 1.7B leads with 72.33, closely followed by MedGemma 27B-text-it at 71.93.
• MedXpertQA: MedGemma 27B-text-it leads with 25.18, while Aerova Mini 2B scores 21.28.

Across the five reported benchmark scores, Aerova Mini 2B has the highest average at 59.638, compared with 56.718 for MedGemma 27B-text-it, 47.550 for Qwen3 1.7B, and 45.178 for MedGemma 4B.

These averages are descriptive comparisons of the supplied scores. Their interpretation depends on consistent scoring scales, evaluation protocols, and test conditions across benchmarks.

Evaluation methodology

For reproducibility, users should consult the relevant benchmark documentation and report the following when available:

• Evaluation harness and benchmark versions.
• Exact model revisions and quantization levels.
• Prompt templates and inference settings.
• Number of evaluated examples and aggregation method.
• Hardware and software configuration.
• Whether all models were tested under equivalent conditions.

The results above are the reported scores for this release. Independent reproduction is encouraged.

Running locally

Using llama.cpp

Install a compatible build of llama.cpp, then download aerova-mini-2b.gguf from this repository.

Run a basic inference test:

llama-cli \
  -m ./aerova-mini-2b.gguf \
  -p "Explain the difference between a virus and a bacterium." \
  -n 256

This is a basic text-generation example, not a validated medical workflow. Confirm the model's chat template and recommended inference settings before using it in a conversational application.

Other runtimes may support the file if they are compatible with the model architecture and GGUF version.

Intended use

Aerova Mini 2B is intended for exploration and evaluation of medical-domain AI capabilities, including:

• Medical AI research and experimentation.
• Educational demonstrations of medical language models.
• Benchmark comparisons and model evaluation.
• Prototyping medical-information applications.
• Local inference experiments on compatible systems.

Medical safety and limitations

Aerova Mini 2B is not a substitute for a qualified healthcare professional.

This model is an AI system and may generate incorrect, incomplete, misleading, or outdated medical information. Strong benchmark scores do not establish clinical safety or demonstrate that the model improves patient outcomes.

• Do not use the model as the sole basis for diagnosis, treatment, medication selection, or emergency decisions.
• Do not treat generated responses as verified medical advice.
• Have qualified professionals review medical outputs before any consequential use.
• Do not enter identifiable patient information unless the deployment has appropriate privacy, security, and authorization safeguards.
• Clinical deployment requires separate validation, safety assessment, and compliance with applicable regulations.

Benchmark performance is not equivalent to clinical validation.

License

The applicable license and usage restrictions must be verified for this release before redistribution or deployment. See the repository's license information for the governing terms.

About AeroxLabs

Aerova Mini 2B is part of the AeroxLabs AI development effort, focused on building and evaluating compact AI systems.

Follow AeroxLabs on Hugging Face for future model releases, updates, and evaluation results.