CoolFace
Modelpublic

carowagner/classify-questions-2A

sourceHugging Facemitupdated 1y agoView on Hugging Face
0likes5downloads
Model Card

Interrogative Type Classifier (Belnap & Steel Taxonomy)

1. Model Overview

This model is a fine-tuned version of google-bert/bert-base-uncased with AutoTrain Advanced on a custom dataset annotated with question-type labels according to the taxonomy of interrogatives defined by Belnap & Steel (1976). It forms part of a broader ensemble of models that classify English-language questions into one of several interrogative categories, as described in the table below.

Interrogative Taxonomy Overview

**Interrogative Type****Definition****Operationalisation**
Hobson’s ChoiceAn interrogative that allows for no alternative responses beyond one predetermined option. These are often in the form of declarative or imperative statements.Classified when the interrogative is declarative or imperative (1B = Yes) and/or has a presupposition (3A = Yes). And allows for no alternative responses (2C = 0).
Why InterrogativesInterrogative with a single example and a pre-supposition.Identified when the interrogative has a presupposition (3A = Yes) and offers only one alternative (2C = 1).
Whether InterrogativesInterrogatives where the information being sought by the questioner is predefined among an explicit and finite list of alternatives. This includes questions that can be answered with yes/no.Identified when the interrogative expects a yes/no answer (2A = Yes and 2C = 2) or lists a defined number of options (2B = Yes and 2C > 1, but not undefined).
Which InterrogativesInterrogatives where the information being sought is part of a category (e.g., religion or tennis players) for which the options are possibly infinite and not explicitly specified.Classified when the number of options is undefined (2C = Undefined) and it is an opinion or not a description (4A = Opinion or No).
What / How InterrogativesInterrogative with an undefined range of possible answers, requesting a descriptive answer.Classified when the interrogative has an undefined answer space (2C = Undefined) and requests a description (4A = Yes).
Not an InterrogativeText that is not a question or not in interrogative form.Prompts are not considered interrogatives when they neither requests an answer (1A = No) nor take a declarative/imperative form (1B = No).

Validation Metrics

loss: 0.3329247236251831

f1_macro: 0.8700832445654317

f1_micro: 0.9

f1_weighted: 0.9012285477571311

precision_macro: 0.906878306878307

precision_micro: 0.9

precision_weighted: 0.9055238095238096

recall_macro: 0.8472222222222222

recall_micro: 0.9

recall_weighted: 0.9

accuracy: 0.9

2. Intended Use

This model is intended for academic, research, and educational use.

3. How to Use

  • —Input: Plain English text (trained on interrogatives that diverse participants asked Language Models)
  • —Output: Predicted category label + confidence score
  • —Training metrics: Available on the model’s Hugging Face page under the “Training Metrics” section (TensorBoard enabled)

Please find an example implementation in python below:

python
from transformers import pipeline

classifier = pipeline("text-classification", model="carowagner/classify-questions-2A")
classifier("How does this model work?")

4. Training Data Labelling Instructions

The annotators who labeled the fine-tuning dataset were given the following instructions for classification:

2A. Is this an interrogative that expects a yes or no answer?

  • —Think: can this question be answered with yes or no?
  • —Answer YES if it can be answered with yes/no, and NO if it cannot.
  • —Answer NA if not relevant — for example, if it does not request an answer or is not an affirmative statement.
  • —Example: “Should college be tuition-free?” → YES