Modelpublic

limajr/NBR-1B-Portuguese-MCQ

sourceHugging Faceapache-2.0updated 8mo agoView on Hugging Face

0likes308downloads

Model Card

NBR-1B-Portuguese-MCQ

O melhor modelo ~1B parametros para questoes de multipla escolha em Portugues.

Highlights

—#1 em ENEM (30.53%) - Vestibular brasileiro
—#1 em OAB (49.70%) - Exame da Ordem dos Advogados
—#1 em BLUEX (36.54%) - Vestibulares universitarios
—#1 em tweetSentBR (39.70%) - Analise de sentimento

Benchmarks

Benchmark	Score	Ranking
ENEM	30.53%	#1
OAB Exams	49.70%	#1
BLUEX	36.54%	#1
TweetSentBR	39.70%	#1
FAQUAD NLI	45.55%	Top 3
HateBR	43.18%	Top 5
PT Hate Speech	41.99%	Top 5
ASSIN2 RTE	34.27%	-
ASSIN2 STS	0.99%	-
Average	35.83%	-

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("limajr/NBR-1B-Portuguese-MCQ")
tokenizer = AutoTokenizer.from_pretrained("limajr/NBR-1B-Portuguese-MCQ")

Model Details

—Architecture: LlamaForCausalLM
—Parameters: ~1.5B
—Hidden Size: 2048
—Layers: 24
—Attention Heads: 16
—Language: Portuguese (pt-BR)
—Training: Supervised Fine-Tuning on Brazilian educational content
—License: Apache 2.0

Training Data

Fine-tuned on curated Portuguese datasets including:

—Brazilian educational materials
—Legal texts (OAB preparation)
—General knowledge QA

Evaluation

Evaluated on the Open PT LLM Leaderboard using the standard evaluation protocol.

Limitations

—Optimized for multiple choice questions
—ASSIN2 STS (semantic similarity) performance is limited
—Best used for Portuguese educational contexts