CoolFace
Modelpublic

carowagner/classify-questions-2B

sourceHugging Facemitupdated 1y agoView on Hugging Face
0likes5downloads
Model Card

Interrogative Type Classifier (Belnap & Steel Taxonomy)

1. Model Overview

This model is a fine-tuned version of google-bert/bert-base-uncased with AutoTrain Advanced on a custom dataset annotated with question-type labels according to the taxonomy of interrogatives defined by Belnap & Steel (1976). It forms part of a broader ensemble of models that classify English-language questions into one of several interrogative categories, as described in the table below.

Interrogative Taxonomy Overview

**Interrogative Type****Definition****Operationalisation**
Hobson’s ChoiceAn interrogative that allows for no alternative responses beyond one predetermined option. These are often in the form of declarative or imperative statements.Classified when the interrogative is declarative or imperative (1B = Yes) and/or has a presupposition (3A = Yes). And allows for no alternative responses (2C = 0).
Why InterrogativesInterrogative with a single example and a pre-supposition.Identified when the interrogative has a presupposition (3A = Yes) and offers only one alternative (2C = 1).
Whether InterrogativesInterrogatives where the information being sought by the questioner is predefined among an explicit and finite list of alternatives. This includes questions that can be answered with yes/no.Identified when the interrogative expects a yes/no answer (2A = Yes and 2C = 2) or lists a defined number of options (2B = Yes and 2C > 1, but not undefined).
Which InterrogativesInterrogatives where the information being sought is part of a category (e.g., religion or tennis players) for which the options are possibly infinite and not explicitly specified.Classified when the number of options is undefined (2C = Undefined) and it is an opinion or not a description (4A = Opinion or No).
What / How InterrogativesInterrogative with an undefined range of possible answers, requesting a descriptive answer.Classified when the interrogative has an undefined answer space (2C = Undefined) and requests a description (4A = Yes).
Not an InterrogativeText that is not a question or not in interrogative form.Prompts are not considered interrogatives when they neither requests an answer (1A = No) nor take a declarative/imperative form (1B = No).

Validation Metrics

loss: 0.21065853536128998

f1_macro: 0.7477924163007036

f1_micro: 0.93

f1_weighted: 0.9184048732115031

precision_macro: 0.975177304964539

precision_micro: 0.93

precision_weighted: 0.9352127659574468

recall_macro: 0.6481481481481481

recall_micro: 0.93

recall_weighted: 0.93

accuracy: 0.93

2. Intended Use

This model is intended for academic, research, and educational use.

3. How to Use

  • —Input: Plain English text (trained on interrogatives that diverse participants asked Language Models)
  • —Output: Predicted category label + confidence score
  • —Training metrics: Available on the model’s Hugging Face page under the “Training Metrics” section (TensorBoard enabled)

Please find an example implementation in python below:

python
from transformers import pipeline

classifier = pipeline("text-classification", model="carowagner/classify-questions-2B")
classifier("How does this model work?")

4. Training Data Labelling Instructions

The annotators who labeled the fine-tuning dataset were given the following instructions for classification:

2B. Does it explicitly present a series of options?

  • —Answer YES if the question explicitly presents a list from which the answerer must choose.
  • —Answer NO if the question does not define an explicit list.
  • —Example: “Do you think animals go to heaven or hell?” → YES, because the answerer must choose between “heaven” and “hell.”