File size: 4,480 Bytes
21d321b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | ---
pipeline_tag: feature-extraction
license: apache-2.0
base_model: facebook/bart-base
library_name: zeromodels
tags:
- keras
- zeromodels
- bart
- feature-extraction
- seq2seq
- arxiv:1910.13461
- pytorch
- jax
- tf
---
# Run BART with Keras 3: JAX, PyTorch, or TensorFlow
[](https://github.com/ZeroAIx/ZeroModels) [](https://zeroaix.github.io/ZeroModels/bart/)
# zeromodels/bart_base
Paper: [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (arXiv:1910.13461)](https://arxiv.org/abs/1910.13461) · [HF Papers](https://huggingface.co/papers/1910.13461)
BART is a denoising seq2seq transformer: a bidirectional encoder (like BERT) and an autoregressive decoder (like GPT) trained to reconstruct corrupted text. It excels at summarization, translation, and other text-to-text tasks. Byte-level BPE tokenizer (shared with RoBERTa); the decoder starts from `</s>`.
For more details on the model, please go to the upstream [model card](https://huggingface.co/facebook/bart-base).
Pure-**Keras 3** conversion of [`facebook/bart-base`](https://huggingface.co/facebook/bart-base) for [zeromodels](https://github.com/ZeroAIx/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
This is a **conditional generation (base seq2seq)** checkpoint (`BartConditionalGenerate`). Other task heads load the shared backbone from this repo (start randomly initialized, ready for fine-tuning); fine-tuned task checkpoints load via the `hf:` prefix.
> Base checkpoint (not task fine-tuned): use it as a backbone (`BartModel`) for features, or fine-tune a task head.
## ✨ Quick start
```python
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from zeromodels.models.bart import BartConditionalGenerate, BartTokenizer
model = BartConditionalGenerate.from_weights("zeromodels/bart_base")
tokenizer = BartTokenizer.from_weights("zeromodels/bart_base")
inputs = tokenizer('The quick brown fox jumps over the lazy dog.')
ids = model.generate(
inputs,
[[model.decoder_start_token_id]],
max_new_tokens=64,
eos_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(ids[0], skip_special_tokens=True))
```
Load any BART variant the same way with `from_weights("zeromodels/<variant>")`:
| Variant | Hub | Task |
|---|---|---|
| `bart_base` | [`zeromodels/bart_base`](https://huggingface.co/zeromodels/bart_base) | conditional generation (base seq2seq) |
| `bart_large` | [`zeromodels/bart_large`](https://huggingface.co/zeromodels/bart_large) | conditional generation (base seq2seq) |
| `bart_large_cnn` | [`zeromodels/bart_large_cnn`](https://huggingface.co/zeromodels/bart_large_cnn) | summarization (CNN / DailyMail) |
| `bart_large_xsum` | [`zeromodels/bart_large_xsum`](https://huggingface.co/zeromodels/bart_large_xsum) | extreme summarization (XSum, one-sentence) |
## Available classes
Load any of these from this repo with `from_weights("zeromodels/bart_base")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).
| Class | Task |
|---|---|
| `BartModel` | Encoder-decoder backbone |
| `BartConditionalGenerate` | Conditional generation (summarization / seq2seq) |
| `BartSequenceClassify` | Sequence classification (e.g. NLI / zero-shot) |
| `BartQnA` | Extractive question answering |
```python
from zeromodels.models.bart import BartSequenceClassify
# zero-shot / NLI fine-tune loads on the fly via the hf: prefix
model = BartSequenceClassify.from_weights("hf:facebook/bart-large-mnli")
```
## Tips
- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- Prefer `BartTokenizer.from_weights(...)` so the byte-level BPE matches.
- BART's decoder starts from `</s>` (`decoder_start_token_id = 2`); pass `eos_token_id=tokenizer.eos_token_id` to stop generation.
- See the [BART docs](https://zeroaix.github.io/ZeroModels/bart/) and [Loading Weights](https://zeroaix.github.io/ZeroModels/loading_weights/).
- Community / upstream safetensors still work via the `hf:` prefix, e.g. `BartConditionalGenerate.from_weights("hf:facebook/bart-base")`.
## Special Thanks
A huge thank you to the Meta AI (FAIR) authors for creating and releasing BART.
License: apache-2.0.
|