T3Q-LLM/T3Q-LLM-TE-NLI-STS-v1.0
Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->
Model Details
Model Description
<!-- Provide a longer summary of what this model is. -->
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- Developed by: [More Information Needed]
- Funded by [optional]: [More Information Needed]
- Shared by [optional]: [More Information Needed]
- Model type: [More Information Needed]
- Language(s) (NLP): [More Information Needed]
- License: [More Information Needed]
- Finetuned from model [optional]: [More Information Needed]
Model Sources [optional]
<!-- Provide the basic links for the model. -->
- Repository: [More Information Needed]
- Paper [optional]: [More Information Needed]
- Demo [optional]: [More Information Needed]
How to Get Started with the Model
Evaluation
hf-causal-experimental (pretrained=T3Q-LLM/T3Q-LLM-TE-NLI-STS-v1.0,useaccelerate=true,trustremotecode=true), limit: None, providedescription: False, numfewshot: 0, batchsize: 8 | Task |Version| Metric |Value | |Stderr| |----------------|------:|--------|-----:|---|-----:| |kobestboolq | 0|acc |0.9509|± |0.0058| | | |macrof1|0.9508|± |0.0058| |kobestcopa | 0|acc |0.7860|± |0.0130| | | |macrof1|0.7858|± |0.0130| |kobesthellaswag| 0|acc |0.5200|± |0.0224| | | |accnorm|0.5360|± |0.0223| | | |macrof1|0.5172|± |0.0223| |kobestsentineg | 0|acc |0.8791|± |0.0164| | | |macro_f1|0.8787|± |0.0164|