zeromodels/bart_base

1---2pipeline_tag: feature-extraction3license: apache-2.04base_model: facebook/bart-base5library_name: zeromodels6tags:7- keras8- zeromodels9- bart10- feature-extraction11- seq2seq12- arxiv:1910.1346113- pytorch14- jax15- tf16---17 18# Run BART with Keras 3: JAX, PyTorch, or TensorFlow19 20[![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/ZeroAIx/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-BART-blue)](https://zeroaix.github.io/ZeroModels/bart/)21 22# zeromodels/bart_base23 24Paper: [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (arXiv:1910.13461)](https://arxiv.org/abs/1910.13461) · [HF Papers](https://huggingface.co/papers/1910.13461)25 26BART is a denoising seq2seq transformer: a bidirectional encoder (like BERT) and an autoregressive decoder (like GPT) trained to reconstruct corrupted text. It excels at summarization, translation, and other text-to-text tasks. Byte-level BPE tokenizer (shared with RoBERTa); the decoder starts from `</s>`.27 28For more details on the model, please go to the upstream [model card](https://huggingface.co/facebook/bart-base).29 30Pure-**Keras 3** conversion of [`facebook/bart-base`](https://huggingface.co/facebook/bart-base) for [zeromodels](https://github.com/ZeroAIx/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.31 32This is a **conditional generation (base seq2seq)** checkpoint (`BartConditionalGenerate`). Other task heads load the shared backbone from this repo (start randomly initialized, ready for fine-tuning); fine-tuned task checkpoints load via the `hf:` prefix.33 34> Base checkpoint (not task fine-tuned): use it as a backbone (`BartModel`) for features, or fine-tune a task head.35 36## ✨ Quick start37 38```python39import os40os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"41 42from zeromodels.models.bart import BartConditionalGenerate, BartTokenizer43 44model = BartConditionalGenerate.from_weights("zeromodels/bart_base")45tokenizer = BartTokenizer.from_weights("zeromodels/bart_base")46 47inputs = tokenizer('The quick brown fox jumps over the lazy dog.')48ids = model.generate(49    inputs,50    [[model.decoder_start_token_id]],51    max_new_tokens=64,52    eos_token_id=tokenizer.eos_token_id,53)54print(tokenizer.decode(ids[0], skip_special_tokens=True))55```56 57Load any BART variant the same way with `from_weights("zeromodels/<variant>")`:58 59| Variant | Hub | Task |60|---|---|---|61| `bart_base` | [`zeromodels/bart_base`](https://huggingface.co/zeromodels/bart_base) | conditional generation (base seq2seq) |62| `bart_large` | [`zeromodels/bart_large`](https://huggingface.co/zeromodels/bart_large) | conditional generation (base seq2seq) |63| `bart_large_cnn` | [`zeromodels/bart_large_cnn`](https://huggingface.co/zeromodels/bart_large_cnn) | summarization (CNN / DailyMail) |64| `bart_large_xsum` | [`zeromodels/bart_large_xsum`](https://huggingface.co/zeromodels/bart_large_xsum) | extreme summarization (XSum, one-sentence) |65 66## Available classes67 68Load any of these from this repo with `from_weights("zeromodels/bart_base")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).69 70| Class | Task |71|---|---|72| `BartModel` | Encoder-decoder backbone |73| `BartConditionalGenerate` | Conditional generation (summarization / seq2seq) |74| `BartSequenceClassify` | Sequence classification (e.g. NLI / zero-shot) |75| `BartQnA` | Extractive question answering |76 77```python78from zeromodels.models.bart import BartSequenceClassify79# zero-shot / NLI fine-tune loads on the fly via the hf: prefix80model = BartSequenceClassify.from_weights("hf:facebook/bart-large-mnli")81```82 83## Tips84 85- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.86- Prefer `BartTokenizer.from_weights(...)` so the byte-level BPE matches.87- BART's decoder starts from `</s>` (`decoder_start_token_id = 2`); pass `eos_token_id=tokenizer.eos_token_id` to stop generation.88- See the [BART docs](https://zeroaix.github.io/ZeroModels/bart/) and [Loading Weights](https://zeroaix.github.io/ZeroModels/loading_weights/).89- Community / upstream safetensors still work via the `hf:` prefix, e.g. `BartConditionalGenerate.from_weights("hf:facebook/bart-base")`.90 91## Special Thanks92 93A huge thank you to the Meta AI (FAIR) authors for creating and releasing BART.94 95License: apache-2.0.96