EMBO/sd-panelization

1---2language: 3- english4thumbnail: 5tags:6- token classification7- 8license: agpl-3.09datasets:10- EMBO/sd-nlp11metrics:12-13---14 15# sd-panelization16 17## Model description18 19This model is a [RoBERTa base model](https://huggingface.co/roberta-base) that was further trained using a masked language modeling task on a compendium of english scientific textual examples from the life sciences using the [BioLang dataset](https://huggingface.co/datasets/EMBO/biolang). It was then fine-tuned for token classification on the SourceData [sd-nlp](https://huggingface.co/datasets/EMBO/sd-nlp) dataset with the `PANELIZATION` task to perform 'parsing' or 'segmentation' of figure legends into fragments corresponding to sub-panels.20 21Figures are usually composite representations of results obtained with heterogeneous experimental approaches and systems. Breaking figures into panels allows identifying more coherent descriptions of individual scientific experiments.22 23## Intended uses & limitations24 25#### How to use26 27The intended use of this model is for 'parsing' figure legends into sub-fragments corresponding to individual panels as used in SourceData annotations (https://sourcedata.embo.org). 28 29To have a quick check of the model:30 31```python32from transformers import pipeline, RobertaTokenizerFast, RobertaForTokenClassification33example = """Fig 4. a, Volume density of early (Avi) and late (Avd) autophagic vacuoles.a, Volume density of early (Avi) and late (Avd) autophagic vacuoles from four independent cultures. Examples of Avi and Avd are shown in b and c, respectively. Bars represent 0.4����m. d, Labelling density of cathepsin-D as estimated in two independent experiments. e, Labelling density of LAMP-1."""34tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base', max_len=512)35model = RobertaForTokenClassification.from_pretrained('EMBO/sd-panelization')36ner = pipeline('ner', model, tokenizer=tokenizer)37res = ner(example)38for r in res: print(r['word'], r['entity'])39```40 41#### Limitations and bias42 43The model must be used with the `roberta-base` tokenizer.44 45## Training data46 47The model was trained for token classification using the [`EMBO/sd-nlp PANELIZATION`](https://huggingface.co/datasets/EMBO/sd-nlp) dataset which includes manually annotated examples.48 49## Training procedure50 51The training was run on an NVIDIA DGX Station with 4XTesla V100 GPUs.52 53Training code is available at https://github.com/source-data/soda-roberta54 55- Model fine-tuned: EMBO/bio-lm56- Tokenizer vocab size: 5026557- Training data: EMBO/sd-nlp58- Dataset configuration: PANELIZATION59- TTraining with 2175 examples.                                  60- Evaluating on 622 examples. 61- Training on 2 features: `O`, `B-PANEL_START`62- Epochs: 1.363- `per_device_train_batch_size`: 1664- `per_device_eval_batch_size`: 1665- `learning_rate`: 0.000166- `weight_decay`: 0.067- `adam_beta1`: 0.968- `adam_beta2`: 0.99969- `adam_epsilon`: 1e-0870- `max_grad_norm`: 1.071 72## Eval results73 74Testing on 1802 examples from test set with `sklearn.metrics`:75 76```                                                                             77              precision    recall  f1-score   support78 79 PANEL_START       0.89      0.95      0.92      542780 81   micro avg       0.89      0.95      0.92      542782   macro avg       0.89      0.95      0.92      542783weighted avg       0.89      0.95      0.92      542784```85