Datasetpublic

tomaarsen/setfit-absa-semeval-restaurants

Dataset Card for "tomaarsen/setfit-absa-semeval-restaurants" Dataset Summary This dataset contains the manually annotated restaurant reviews from SemEval-2014 Task 4, in the format as understood by SetFit ABSA. For more details, see https://aclanthology.org/S14-2004/ Data Instances An example of "train" looks as follows. {"text": "But the staff was so horrible to us.", "span": "staff", "label": "negative", "ordinal": 0} {"text": "To be completely… See the full description on the dataset page: https://huggingface.co/datasets/tomaarsen/setfit-absa-semeval-restaurants.

sourceHugging Faceupdated 3y agoView on Hugging Face

4likes284downloads

Dataset Card

Dataset Card for "tomaarsen/setfit-absa-semeval-restaurants"

Dataset Summary

This dataset contains the manually annotated restaurant reviews from SemEval-2014 Task 4, in the format as understood by SetFit ABSA.

For more details, see https://aclanthology.org/S14-2004/

Data Instances

An example of "train" looks as follows.

json

{"text": "But the staff was so horrible to us.", "span": "staff", "label": "negative", "ordinal": 0}
{"text": "To be completely fair, the only redeeming factor was the food, which was above average, but couldn't make up for all the other deficiencies of Teodora.", "span": "food", "label": "positive", "ordinal": 0}
{"text": "The food is uniformly exceptional, with a very capable kitchen which will proudly whip up whatever you feel like eating, whether it's on the menu or not.", "span": "food", "label": "positive", "ordinal": 0}
{"text": "The food is uniformly exceptional, with a very capable kitchen which will proudly whip up whatever you feel like eating, whether it's on the menu or not.", "span": "kitchen", "label": "positive", "ordinal": 0}
{"text": "The food is uniformly exceptional, with a very capable kitchen which will proudly whip up whatever you feel like eating, whether it's on the menu or not.", "span": "menu", "label": "neutral", "ordinal": 0}

Data Fields

The data fields are the same among all splits.

—text: a string feature.
—span: a string feature showing the aspect span from the text.
—label: a string feature showing the polarity of the aspect span.
—ordinal: an int64 feature showing the n-th occurrence of the span in the text. This is useful for if the span occurs within the same text multiple times.

Data Splits

name	train	test
tomaarsen/setfit-absa-semeval-restaurants	3693	1134

Training ABSA models using SetFit ABSA

To train using this dataset, first install the SetFit library:

bash

pip install setfit

And then you can use the following script as a guideline of how to train an ABSA model on this dataset:

python

from setfit import AbsaModel, AbsaTrainer, TrainingArguments
from datasets import load_dataset
from transformers import EarlyStoppingCallback

# You can initialize a AbsaModel using one or two SentenceTransformer models, or two ABSA models
model = AbsaModel.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")

# The training/eval dataset must have `text`, `span`, `polarity`, and `ordinal` columns
dataset = load_dataset("tomaarsen/setfit-absa-semeval-restaurants")
train_dataset = dataset["train"]
eval_dataset = dataset["test"]

args = TrainingArguments(
    output_dir="models",
    use_amp=True,
    batch_size=256,
    eval_steps=50,
    save_steps=50,
    load_best_model_at_end=True,
)

trainer = AbsaTrainer(
    model,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    callbacks=[EarlyStoppingCallback(early_stopping_patience=5)],
)
trainer.train()

metrics = trainer.evaluate(eval_dataset)
print(metrics)

trainer.push_to_hub("tomaarsen/setfit-absa-restaurants")

You can then run inference like so:

python

from setfit import AbsaModel

# Download from Hub and run inference
model = AbsaModel.from_pretrained(
    "tomaarsen/setfit-absa-restaurants-aspect",
    "tomaarsen/setfit-absa-restaurants-polarity",
)

# Run inference
preds = model([
    "The best pizza outside of Italy and really tasty.",
    "The food here is great but the service is terrible",
])

Citation Information

bibtex

@inproceedings{pontiki-etal-2014-semeval,
    title = "{S}em{E}val-2014 Task 4: Aspect Based Sentiment Analysis",
    author = "Pontiki, Maria  and
      Galanis, Dimitris  and
      Pavlopoulos, John  and
      Papageorgiou, Harris  and
      Androutsopoulos, Ion  and
      Manandhar, Suresh",
    editor = "Nakov, Preslav  and
      Zesch, Torsten",
    booktitle = "Proceedings of the 8th International Workshop on Semantic Evaluation ({S}em{E}val 2014)",
    month = aug,
    year = "2014",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/S14-2004",
    doi = "10.3115/v1/S14-2004",
    pages = "27--35",
}