Add bart_large_xsum (zeromodels Keras 3 conversion)

README.md

@@ -0,0 +1,92 @@
+---
+pipeline_tag: summarization
+license: mit
+base_model: facebook/bart-large-xsum
+library_name: zeromodels
+tags:
+- keras
+- zeromodels
+- bart
+- summarization
+- arxiv:1910.13461
+- pytorch
+- jax
+- tf
+---
+
+# Run BART with Keras 3: JAX, PyTorch, or TensorFlow
+
+[![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/ZeroAIx/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-BART-blue)](https://zeroaix.github.io/ZeroModels/bart/)
+
+# zeromodels/bart_large_xsum
+
+Paper: [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (arXiv:1910.13461)](https://arxiv.org/abs/1910.13461) · [HF Papers](https://huggingface.co/papers/1910.13461)
+
+BART is a denoising seq2seq transformer: a bidirectional encoder (like BERT) and an autoregressive decoder (like GPT) trained to reconstruct corrupted text. It excels at summarization, translation, and other text-to-text tasks. Byte-level BPE tokenizer (shared with RoBERTa); the decoder starts from `</s>`.
+
+For more details on the model, please go to the upstream [model card](https://huggingface.co/facebook/bart-large-xsum).
+
+Pure-**Keras 3** conversion of [`facebook/bart-large-xsum`](https://huggingface.co/facebook/bart-large-xsum) for [zeromodels](https://github.com/ZeroAIx/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
+
+This is a **extreme summarization (XSum, one-sentence)** checkpoint (`BartConditionalGenerate`). Other task heads load the shared backbone from this repo (start randomly initialized, ready for fine-tuning); fine-tuned task checkpoints load via the `hf:` prefix.
+
+## ✨ Quick start
+
+```python
+import os
+os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"
+
+from zeromodels.models.bart import BartConditionalGenerate, BartTokenizer
+
+model = BartConditionalGenerate.from_weights("zeromodels/bart_large_xsum")
+tokenizer = BartTokenizer.from_weights("zeromodels/bart_large_xsum")
+
+inputs = tokenizer('The tower is 324 metres tall, about the same height as an 81-storey building, and the tallest structure in Paris.')
+ids = model.generate(
+    inputs,
+    [[model.decoder_start_token_id]],
+    max_new_tokens=64,
+    eos_token_id=tokenizer.eos_token_id,
+)
+print(tokenizer.decode(ids[0], skip_special_tokens=True))
+```
+
+Load any BART variant the same way with `from_weights("zeromodels/<variant>")`:
+
+| Variant | Hub | Task |
+|---|---|---|
+| `bart_base` | [`zeromodels/bart_base`](https://huggingface.co/zeromodels/bart_base) | conditional generation (base seq2seq) |
+| `bart_large` | [`zeromodels/bart_large`](https://huggingface.co/zeromodels/bart_large) | conditional generation (base seq2seq) |
+| `bart_large_cnn` | [`zeromodels/bart_large_cnn`](https://huggingface.co/zeromodels/bart_large_cnn) | summarization (CNN / DailyMail) |
+| `bart_large_xsum` | [`zeromodels/bart_large_xsum`](https://huggingface.co/zeromodels/bart_large_xsum) | extreme summarization (XSum, one-sentence) |
+
+## Available classes
+
+Load any of these from this repo with `from_weights("zeromodels/bart_large_xsum")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).
+
+| Class | Task |
+|---|---|
+| `BartModel` | Encoder-decoder backbone |
+| `BartConditionalGenerate` | Conditional generation (summarization / seq2seq) |
+| `BartSequenceClassify` | Sequence classification (e.g. NLI / zero-shot) |
+| `BartQnA` | Extractive question answering |
+
+```python
+from zeromodels.models.bart import BartSequenceClassify
+# zero-shot / NLI fine-tune loads on the fly via the hf: prefix
+model = BartSequenceClassify.from_weights("hf:facebook/bart-large-mnli")
+```
+
+## Tips
+
+- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
+- Prefer `BartTokenizer.from_weights(...)` so the byte-level BPE matches.
+- BART's decoder starts from `</s>` (`decoder_start_token_id = 2`); pass `eos_token_id=tokenizer.eos_token_id` to stop generation.
+- See the [BART docs](https://zeroaix.github.io/ZeroModels/bart/) and [Loading Weights](https://zeroaix.github.io/ZeroModels/loading_weights/).
+- Community / upstream safetensors still work via the `hf:` prefix, e.g. `BartConditionalGenerate.from_weights("hf:facebook/bart-large-xsum")`.
+
+## Special Thanks
+
+A huge thank you to the Meta AI (FAIR) authors for creating and releasing BART.
+
+License: mit.

model.weights.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9d7fcc8473550a909be10c7d5f6e0bcaa7a4e9c08dd9220812fe6ddabb0bc16
+size 1626174800

zm_config.json

@@ -0,0 +1,33 @@
+{
+  "library_name": "zeromodels",
+  "zeromodels_version": "1.3.0",
+  "model_module": "zeromodels.models.bart",
+  "model_class": "BartConditionalGenerate",
+  "variant": "bart_large_xsum",
+  "weights": "model.weights.h5",
+  "schema_version": 2,
+  "model_type": "bart",
+  "text_config": {
+    "vocab_size": 50264,
+    "hidden_dim": 1024,
+    "encoder_num_layers": 12,
+    "decoder_num_layers": 12,
+    "encoder_attention_heads": 16,
+    "decoder_attention_heads": 16,
+    "encoder_ffn_dim": 4096,
+    "decoder_ffn_dim": 4096,
+    "max_position_embeddings": 1024,
+    "activation_function": "gelu",
+    "scale_embedding": false,
+    "layer_norm_eps": 1e-05,
+    "classifier_dropout": 0.0,
+    "num_labels": 3,
+    "pad_token_id": 1,
+    "bos_token_id": 0,
+    "eos_token_id": 2,
+    "decoder_start_token_id": 2
+  },
+  "generate_args": {
+    "max_new_tokens": 64
+  }
+}