CoolFace
Modelpublic

carowagner/classify-questions-2C

sourceHugging Facemitupdated 1y agoView on Hugging Face
0likes7downloads
Model Card

Interrogative Type Classifier (Belnap & Steel Taxonomy)

1. Model Overview

This model is a fine-tuned version of google-bert/bert-base-uncased with AutoTrain Advanced on a custom dataset annotated with question-type labels according to the taxonomy of interrogatives defined by Belnap & Steel (1976). It forms part of a broader ensemble of models that classify English-language questions into one of several interrogative categories, as described in the table below.

Interrogative Taxonomy Overview

**Interrogative Type****Definition****Operationalisation**
Hobson’s ChoiceAn interrogative that allows for no alternative responses beyond one predetermined option. These are often in the form of declarative or imperative statements.Classified when the interrogative is declarative or imperative (1B = Yes) and/or has a presupposition (3A = Yes). And allows for no alternative responses (2C = 0).
Why InterrogativesInterrogative with a single example and a pre-supposition.Identified when the interrogative has a presupposition (3A = Yes) and offers only one alternative (2C = 1).
Whether InterrogativesInterrogatives where the information being sought by the questioner is predefined among an explicit and finite list of alternatives. This includes questions that can be answered with yes/no.Identified when the interrogative expects a yes/no answer (2A = Yes and 2C = 2) or lists a defined number of options (2B = Yes and 2C > 1, but not undefined).
Which InterrogativesInterrogatives where the information being sought is part of a category (e.g., religion or tennis players) for which the options are possibly infinite and not explicitly specified.Classified when the number of options is undefined (2C = Undefined) and it is an opinion or not a description (4A = Opinion or No).
What / How InterrogativesInterrogative with an undefined range of possible answers, requesting a descriptive answer.Classified when the interrogative has an undefined answer space (2C = Undefined) and requests a description (4A = Yes).
Not an InterrogativeText that is not a question or not in interrogative form.Prompts are not considered interrogatives when they neither requests an answer (1A = No) nor take a declarative/imperative form (1B = No).

Validation Metrics

loss: 0.38375258445739746

f1_macro: 0.7050228553676829

f1_micro: 0.9

f1_weighted: 0.882172635689877

precision_macro: 0.7176220331392745

precision_micro: 0.9

precision_weighted: 0.8699126735333632

recall_macro: 0.7040041928721174

recall_micro: 0.9

recall_weighted: 0.9

accuracy: 0.9

2. Intended Use

This model is intended for academic, research, and educational use.

3. How to Use

  • —Input: Plain English text (trained on interrogatives that diverse participants asked Language Models)
  • —Output: Predicted category label + confidence score
  • —Training metrics: Available on the model’s Hugging Face page under the “Training Metrics” section (TensorBoard enabled)

Please find an example implementation in python below:

python
from transformers import pipeline

classifier = pipeline("text-classification", model="carowagner/classify-questions-2C")
classifier("How does this model work?")

4. Training Data Labelling Instructions

The annotators who labeled the fine-tuning dataset were given the following instructions for classification:

2C. How many options does it present?

  • —This question is about how the questioner defines the space of possible answers in the way they phrase their question. It must be answered with either 0, 1, 2, U, or another integer.
  • —0 — Declarative/imperative statements that do not directly incite an answer (e.g., “Guns are too easy to buy in some countries.”).
  • —1 — 'Why' questions that assume a cause or premise already exists (e.g., “Why do criminal migrants keep living and making crime in our countries?” assumes a cause and only one explanatory answer).
  • —2 — Questions that can be answered with yes/no and explicitly describe two alternatives (e.g., “Is abortion a good or a bad thing?”).
  • —U — If the answer space is undefined, as in open-ended descriptive questions (e.g., “What are some steps we could take to combat global warming?”).
  • —3 or more — If the question explicitly lists a set of options, assign the exact number of available options (e.g., “I have three games in my library — which should I play first: Fallout 4, Ace Attorney, or The Talos Principle?” → 3).