stefan-it/autotrain-flair-hipe2022-fr-hmbert
NER Fine-Tuning We use Flair for fine-tuning NER models on HIPE-2022 datasets from HIPE-2022 Shared Task. All models are fine-tuned on A10 (24GB) and A100 (40GB) instances from Lambda Cloud using Flair: $ git clone https://github.com/flairNLP/flair.git $ cd flair && git checkout 419f13a05d6b36b2a42dd73a551dc3ba679f820c $ pip3 install -e . $ cd .. Clone this repo for fine-tuning NER models: $ git clone https://github.com/stefan-it/hmTEAMS.git $ cd hmTEAMS/bench Authorize via… See the full description on the dataset page: https://huggingface.co/datasets/stefan-it/autotrain-flair-hipe2022-fr-hmbert.
NER Fine-Tuning
We use Flair for fine-tuning NER models on HIPE-2022 datasets from HIPE-2022 Shared Task.
All models are fine-tuned on A10 (24GB) and A100 (40GB) instances from Lambda Cloud using Flair:
$ git clone https://github.com/flairNLP/flair.git
$ cd flair && git checkout 419f13a05d6b36b2a42dd73a551dc3ba679f820c
$ pip3 install -e .
$ cd ..Clone this repo for fine-tuning NER models:
$ git clone https://github.com/stefan-it/hmTEAMS.git
$ cd hmTEAMS/benchAuthorize via Hugging Face CLI (needed because hmTEAMS is currently only available after approval):
# Use access token from https://huggingface.co/settings/tokens
$ huggingface-cli loginWe use a config-driven hyper-parameter search. The script `flair-fine-tuner.py` can be used to fine-tune NER models from our Model Zoo.
Additionally, we provide a script that uses Hugging Face AutoTrain Advanced (Space Runner) to fine-tung models. The following snippet shows an example:
$ pip3 install autotrain-advanced
$ export HF_TOKEN="" # Get token from: https://huggingface.co/settings/tokens
$ autotrain spacerunner --project-name "flair-hipe2022-de-hmteams" \
--script-path /home/stefan/Repositories/hmTEAMS/bench \
--username stefan-it \
--token $HF_TOKEN \
--backend spaces-t4s \
--env "CONFIG=configs/hipe2020/de/hmteams.json;HF_TOKEN=$HF_TOKEN;REPO_NAME=stefan-it/autotrain-flair-hipe2022-de-hmteams"The concrete implementation can be found in `script.py`.
Benchmark
We test our pretrained language models on various datasets from HIPE-2020, HIPE-2022 and Europeana. The following table shows an overview of used datasets.
[AjMC]: https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md [NewsEye]: https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md [TopRes19th]: https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-topres19th.md [ICDAR-Europeana]: https://github.com/stefan-it/historic-domain-adaptation-icdar [LeTemps]: https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-letemps.md
Results
We report averaged F1-score over 5 runs with different seeds on development set: