CoolFace
Modelpublic

carowagner/classify-questions-3A

sourceHugging Facemitupdated 1y agoView on Hugging Face
0likes5downloads
Model Card

Interrogative Type Classifier (Belnap & Steel Taxonomy)

1. Model Overview

This model is a fine-tuned version of google-bert/bert-base-uncased with AutoTrain Advanced on a custom dataset annotated with question-type labels according to the taxonomy of interrogatives defined by Belnap & Steel (1976). It forms part of a broader ensemble of models that classify English-language questions into one of several interrogative categories, as described in the table below.

Interrogative Taxonomy Overview

**Interrogative Type****Definition****Operationalisation**
Hobson’s ChoiceAn interrogative that allows for no alternative responses beyond one predetermined option. These are often in the form of declarative or imperative statements.Classified when the interrogative is declarative or imperative (1B = Yes) and/or has a presupposition (3A = Yes). And allows for no alternative responses (2C = 0).
Why InterrogativesInterrogative with a single example and a pre-supposition.Identified when the interrogative has a presupposition (3A = Yes) and offers only one alternative (2C = 1).
Whether InterrogativesInterrogatives where the information being sought by the questioner is predefined among an explicit and finite list of alternatives. This includes questions that can be answered with yes/no.Identified when the interrogative expects a yes/no answer (2A = Yes and 2C = 2) or lists a defined number of options (2B = Yes and 2C > 1, but not undefined).
Which InterrogativesInterrogatives where the information being sought is part of a category (e.g., religion or tennis players) for which the options are possibly infinite and not explicitly specified.Classified when the number of options is undefined (2C = Undefined) and it is an opinion or not a description (4A = Opinion or No).
What / How InterrogativesInterrogative with an undefined range of possible answers, requesting a descriptive answer.Classified when the interrogative has an undefined answer space (2C = Undefined) and requests a description (4A = Yes).
Not an InterrogativeText that is not a question or not in interrogative form.Prompts are not considered interrogatives when they neither requests an answer (1A = No) nor take a declarative/imperative form (1B = No).

Validation Metrics

loss: 0.06497155874967575

f1_macro: 0.9505791505791507

f1_micro: 0.99

f1_weighted: 0.9893127413127414

precision_macro: 0.996415770609319

precision_micro: 0.99

precision_weighted: 0.9901075268817204

recall_macro: 0.9166666666666666

recall_micro: 0.99

recall_weighted: 0.99

accuracy: 0.99

2. Intended Use

This model is intended for academic, research, and educational use.

3. How to Use

  • —Input: Plain English text (trained on interrogatives that diverse participants asked Language Models)
  • —Output: Predicted category label + confidence score
  • —Training metrics: Available on the model’s Hugging Face page under the “Training Metrics” section (TensorBoard enabled)

Please find an example implementation in python below:

python
from transformers import pipeline

classifier = pipeline("text-classification", model="carowagner/classify-questions-3A")
classifier("How does this model work?")

4. Training Data Labelling Instructions

The annotators who labeled the fine-tuning dataset were given the following instructions for classification:

3A. Do answers to this interrogative require some other fact/opinion already being true?

  • —Only questions that ask about the cause of a fact the questioner assumes to be true should be answered with YES.
  • —Example: “Tell me why Donald Trump will be the next president elect” — the answer presumes that Trump will indeed be elected.
  • —Example: “Why are people so comfortable with eating animal corpses?” — the question assumes that people are comfortable doing so.
  • —If the question does not rely on a presupposed fact or opinion, answer NO.