bilalzafar/CentralBank-BERT

Central Bank-BERT: Domain-Adaptive Masked Language Model for Central Bank Communication

Central Bank-BERT is a domain-adapted masked language model based on bert-base-uncased, pretrained on more than 66 million tokens across over 2 million sentences extracted from central bank speeches published by the Bank for International Settlements (1996–2024). This model is specifically optimized for masked token prediction within the highly specialized domains of monetary policy, financial regulation, and macroeconomic communication, enabling deeper contextual understanding of central banking discourse and financial narratives.

Dataset Summary

• —Source: BIS Central Bank Speeches (1996–2024)
• —Total Speeches: 19,609
• —MLM Sentences: 2,087,615 (~2.09M)
• —Total Tokens: 66,359,113 (~66.36M)
• —Avg. Tokens per Sentence: 31.79

Model & Training Details

Lower perplexity demonstrates better fit on domain-specific central bank language.

Notebook: Training, Evaluation & Results The full training pipeline, including data preprocessing, tokenizer setup, model training, evaluation, and result visualizations, is documented in the notebook `cb-bert-mlm.ipynb`. This notebook includes actual outputs from the training run, perplexity comparisons, manual masked sentence evaluations, and Top-K accuracy analysis—ensuring full transparency and reproducibility of the model development process.

Model Files

• —model.safetensors: Trained model weights
• —config.json: Model architecture and hyperparameters
• —tokenizer.json: Serialized tokenizer
• —vocab.txt: Vocabulary file
• —tokenizer_config.json: Tokenizer configuration
• —special_tokens_map.json: Special tokens mapping
• —training_args.bin: Training arguments used during pretraining

This model repository includes all essential files required to load, evaluate, or fine-tune the `CentralBank-BERT` model using Hugging Face's transformers library. These components are necessary to ensure full compatibility with the original training environment and to support seamless deployment or transfer learning.

Downstream Models

In addition to the domain-adapted masked language model ([`CentralBank-BERT`](https://huggingface.co/bilalzafar/CentralBank-BERT)), a suite of fine-tuned downstream classifiers has been released to support CBDC-specific research and policy analysis. These models share the same encoder backbone and are designed for different classification and information extraction tasks on central bank communication.

Repository and Replication Package

All training pipelines, preprocessing scripts, evaluation notebooks, and result outputs are available in the companion GitHub repository:

🔗 https://github.com/bilalezafar/CentralBank-BERT

The repository includes:

• —End-to-end notebooks for CentralBank-BERT pretraining and all downstream classifiers (CBDC-BERT, Stance, Sentiment, Type, Discourse, NER).
• —Preprocessed BIS speech dataset subsets (CBDC-related sentences, annotated splits).
• —Reproducible code to generate figures, tables, and evaluation metrics reported in the manuscript.
• —Deployment-ready scripts for applying models to new corpora.

This ensures full transparency, reproducibility, and extension of the CentralBank-BERT family of models.

Citation

If you use this model, please cite as:

*Zafar, M. B. (2025). CentralBank-BERT: Machine learning evidence on central bank digital currency discourse. Journal of Economics and Business. [https://doi.org/10.1016/j.jeconbus.2026.106300](https://doi.org/10.1016/j.jeconbus.2026.106300)*

@article{zafar2025centralbankbert,
  title={CentralBank-BERT: Machine learning evidence on central bank digital currency discourse},
  author={Zafar, Muhammad Bilal},
  year={2026},
  journal={Journal of Economics and Business},
  url={https://doi.org/10.1016/j.jeconbus.2026.106300}
}