chendl/compositional_test

1# Token classification2 3## PyTorch version, no Trainer4 5Fine-tuning (m)LUKE for token classification task such as Named Entity Recognition (NER), Parts-of-speech6tagging (POS) or phrase extraction (CHUNKS). You can easily7customize it to your needs if you need extra processing on your datasets.8 9It will either run on a datasets hosted on our [hub](https://huggingface.co/datasets) or with your own text files for10training and validation, you might just need to add some tweaks in the data preprocessing.11 12The script can be  run in a distributed setup, on TPU and supports mixed precision by13the mean of the [🤗 `Accelerate`](https://github.com/huggingface/accelerate) library. You can use the script normally14after installing it:15 16```bash17pip install git+https://github.com/huggingface/accelerate18```19 20then to train English LUKE on CoNLL2003:21 22```bash23export TASK_NAME=ner24 25python run_luke_ner_no_trainer.py \26  --model_name_or_path studio-ousia/luke-base \27  --dataset_name conll2003 \28  --task_name $TASK_NAME \29  --max_length 128 \30  --per_device_train_batch_size 32 \31  --learning_rate 2e-5 \32  --num_train_epochs 3 \33  --output_dir /tmp/$TASK_NAME/34```35 36You can then use your usual launchers to run in it in a distributed environment, but the easiest way is to run37 38```bash39accelerate config40```41 42and reply to the questions asked. Then43 44```bash45accelerate test46```47 48that will check everything is ready for training. Finally, you can launch training with49 50```bash51export TASK_NAME=ner52 53accelerate launch run_ner_no_trainer.py \54  --model_name_or_path studio-ousia/luke-base \55  --dataset_name conll2003 \56  --task_name $TASK_NAME \57  --max_length 128 \58  --per_device_train_batch_size 32 \59  --learning_rate 2e-5 \60  --num_train_epochs 3 \61  --output_dir /tmp/$TASK_NAME/62```63 64This command is the same and will work for:65 66- a CPU-only setup67- a setup with one GPU68- a distributed training with several GPUs (single or multi node)69- a training on TPUs70 71Note that this library is in alpha release so your feedback is more than welcome if you encounter any problem using it.72