joeddav/distilbert-base-uncased-agnews-student

distilbert-base-uncased-agnews-student

Model Description

This model is distilled from the zero-shot classification pipeline on the unlabeled AG's News dataset using this script. It is the result of the demo notebook here, where more details about the model can be found.

• —Teacher model: roberta-large-mnli
• —Teacher hypothesis template: "This text is about {}."

Intended Usage

The model can be used like any other model trained on AG's News, but will likely not perform as well as a model trained with full supervision. It is primarily intended as a demo of how an expensive NLI-based zero-shot model can be distilled to a more efficient student.