README.md

---
pipeline_tag: feature-extraction
license: apache-2.0
base_model: facebook/bart-base
library_name: zeromodels
tags:
- keras
- zeromodels
- bart
- feature-extraction
- seq2seq
- arxiv:1910.13461
- pytorch
- jax
- tf
---

# Run BART with Keras 3: JAX, PyTorch, or TensorFlow

[![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/ZeroAIx/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-BART-blue)](https://zeroaix.github.io/ZeroModels/bart/)

# zeromodels/bart_base

Paper: [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (arXiv:1910.13461)](https://arxiv.org/abs/1910.13461) · [HF Papers](https://huggingface.co/papers/1910.13461)

BART is a denoising seq2seq transformer: a bidirectional encoder (like BERT) and an autoregressive decoder (like GPT) trained to reconstruct corrupted text. It excels at summarization, translation, and other text-to-text tasks. Byte-level BPE tokenizer (shared with RoBERTa); the decoder starts from `</s>`.

For more details on the model, please go to the upstream [model card](https://huggingface.co/facebook/bart-base).

Pure-**Keras 3** conversion of [`facebook/bart-base`](https://huggingface.co/facebook/bart-base) for [zeromodels](https://github.com/ZeroAIx/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.

This is a **conditional generation (base seq2seq)** checkpoint (`BartConditionalGenerate`). Other task heads load the shared backbone from this repo (start randomly initialized, ready for fine-tuning); fine-tuned task checkpoints load via the `hf:` prefix.

> Base checkpoint (not task fine-tuned): use it as a backbone (`BartModel`) for features, or fine-tune a task head.

## ✨ Quick start

```python
import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.bart import BartConditionalGenerate, BartTokenizer

model = BartConditionalGenerate.from_weights("zeromodels/bart_base")
tokenizer = BartTokenizer.from_weights("zeromodels/bart_base")

inputs = tokenizer('The quick brown fox jumps over the lazy dog.')
ids = model.generate(
    inputs,
    [[model.decoder_start_token_id]],
    max_new_tokens=64,
    eos_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(ids[0], skip_special_tokens=True))
```

Load any BART variant the same way with `from_weights("zeromodels/<variant>")`:

| Variant | Hub | Task |
|---|---|---|
| `bart_base` | [`zeromodels/bart_base`](https://huggingface.co/zeromodels/bart_base) | conditional generation (base seq2seq) |
| `bart_large` | [`zeromodels/bart_large`](https://huggingface.co/zeromodels/bart_large) | conditional generation (base seq2seq) |
| `bart_large_cnn` | [`zeromodels/bart_large_cnn`](https://huggingface.co/zeromodels/bart_large_cnn) | summarization (CNN / DailyMail) |
| `bart_large_xsum` | [`zeromodels/bart_large_xsum`](https://huggingface.co/zeromodels/bart_large_xsum) | extreme summarization (XSum, one-sentence) |

## Available classes

Load any of these from this repo with `from_weights("zeromodels/bart_base")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).

| Class | Task |
|---|---|
| `BartModel` | Encoder-decoder backbone |
| `BartConditionalGenerate` | Conditional generation (summarization / seq2seq) |
| `BartSequenceClassify` | Sequence classification (e.g. NLI / zero-shot) |
| `BartQnA` | Extractive question answering |

```python
from zeromodels.models.bart import BartSequenceClassify
# zero-shot / NLI fine-tune loads on the fly via the hf: prefix
model = BartSequenceClassify.from_weights("hf:facebook/bart-large-mnli")
```

## Tips

- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- Prefer `BartTokenizer.from_weights(...)` so the byte-level BPE matches.
- BART's decoder starts from `</s>` (`decoder_start_token_id = 2`); pass `eos_token_id=tokenizer.eos_token_id` to stop generation.
- See the [BART docs](https://zeroaix.github.io/ZeroModels/bart/) and [Loading Weights](https://zeroaix.github.io/ZeroModels/loading_weights/).
- Community / upstream safetensors still work via the `hf:` prefix, e.g. `BartConditionalGenerate.from_weights("hf:facebook/bart-base")`.

## Special Thanks

A huge thank you to the Meta AI (FAIR) authors for creating and releasing BART.

License: apache-2.0.