Datasetpublic

EXOROBOURII/Stanza-Wikitext-2

Dataset Card for Stanza-Wikitext-2 Dataset Description Stanza-Wikitext-2 is a structurally pristine, mathematically verified NLP dataset designed for multi-task language modeling, custom tokenizer training, structural NLP research, and mechanistic interpretability work. It is a rigorously modernized and annotated derivative of the wikitext-2-raw-v1 corpus. Using the Stanford NLP Stanza neural pipeline, every token in the corpus has been explicitly mapped to its… See the full description on the dataset page: https://huggingface.co/datasets/EXOROBOURII/Stanza-Wikitext-2.

sourceHugging Facecc-by-sa-4.0updated 6mo agoView on Hugging Face

0likes103downloads

rootdeprel_depth_stats.csv

deprel_depth_stats.csv52 linesDownload Raw Back to root

1deprel,count,mean,std,median,min,max,q90,q99,entropy2goeswith,1,5.0,,5.0,5,5,5.0,5.0,0.03fixed,8200,4.33622,1.499768,4.0,0,17,6.0,9.0,2.433624orphan,147,3.938776,1.597191,4.0,2,12,6.0,9.54,2.4292575cc,73155,3.680979,1.679483,3.0,0,24,6.0,9.0,2.5709996case,308434,3.564902,1.514711,3.0,0,25,6.0,8.0,2.4402817det:predet,811,3.462392,1.524938,3.0,0,10,5.0,8.0,2.4567158flat,44406,3.389137,1.532089,3.0,0,18,5.0,8.0,2.4848559nmod:poss,42979,3.348635,1.489349,3.0,0,15,5.0,8.0,2.38707610compound,121630,3.304719,1.4681,3.0,0,19,5.0,8.0,2.40966811amod,149819,3.272002,1.502193,3.0,0,19,5.0,8.0,2.42898712nummod,55362,3.271161,1.412011,3.0,0,14,5.0,8.0,2.33337413dep,2264,3.249558,1.874545,3.0,0,23,6.0,9.0,2.74409614cc:preconj,1411,3.216867,1.446845,3.0,0,10,5.0,7.0,2.40371215det,227368,3.208886,1.462077,3.0,0,21,5.0,8.0,2.35325116acl:relcl,19022,3.075071,1.422716,3.0,0,19,5.0,8.0,2.33937917nmod,125758,3.069952,1.41752,3.0,0,24,5.0,8.0,2.33335318appos,27310,3.011461,1.415008,3.0,0,18,5.0,8.0,2.3296319mark,57375,2.983808,1.318319,3.0,0,15,5.0,8.0,2.10367320nmod:unmarked,5514,2.979507,1.455469,3.0,0,13,5.0,8.0,2.34782621acl,21030,2.887637,1.394975,3.0,0,16,5.0,8.0,2.2969822conj,88725,2.791243,1.67565,3.0,0,23,5.0,8.0,2.55909123punct,313564,2.687939,1.707351,2.0,0,24,5.0,8.0,2.53845624reparandum,106,2.566038,1.804855,2.0,0,8,6.0,8.0,2.4408725advmod,73610,2.561568,1.589131,2.0,0,21,5.0,8.0,2.44681626compound:prt,5332,2.53132,1.580205,2.0,0,15,5.0,7.0,2.43074627advcl:relcl,483,2.519669,1.33308,2.0,0,8,4.0,7.0,2.33131428obl:agent,10152,2.426812,1.581766,2.0,0,14,5.0,7.0,2.36231329obj,91131,2.363916,1.472798,2.0,0,15,4.0,7.0,2.30589630aux,24453,2.356521,1.519084,2.0,0,15,4.0,7.0,2.34581531obl:unmarked,6003,2.325337,1.490368,2.0,0,14,4.0,7.0,2.29933132xcomp,24161,2.207111,1.431285,2.0,0,17,4.0,7.0,2.20332833iobj,2249,2.193419,1.305079,2.0,0,10,4.0,6.0,2.13319334obl,144874,2.080601,1.40543,2.0,0,21,4.0,7.0,2.09382835cop,27307,2.002307,1.368907,2.0,0,20,4.0,7.0,2.03691636nsubj,121444,1.879451,1.347146,1.0,0,20,4.0,7.0,1.89296937aux:pass,35643,1.821676,1.261862,1.0,0,13,4.0,6.0,1.83573138advcl,40635,1.760231,1.180481,1.0,0,14,3.0,6.0,1.7623939csubj:outer,7,1.714286,0.755929,2.0,1,3,2.4,2.94,1.44881640nsubj:pass,32209,1.713217,1.210491,1.0,0,13,3.0,6.0,1.69556241ccomp,12330,1.685158,1.052279,1.0,0,10,3.0,6.0,1.66226642expl,2923,1.643175,1.022477,1.0,0,10,3.0,5.0,1.60432143csubj,1087,1.632935,0.945349,1.0,0,10,3.0,5.0,1.57053944dislocated,33,1.606061,0.966288,1.0,0,4,3.0,4.0,1.70008845nsubj:outer,735,1.593197,1.125241,1.0,0,9,3.0,6.0,1.63982946vocative,177,1.435028,0.803058,1.0,0,5,2.4,4.24,1.31893447parataxis,8242,1.379398,1.001311,1.0,0,23,2.0,6.0,1.13243848csubj:pass,151,1.278146,0.674872,1.0,1,6,2.0,4.0,0.935949list,815,1.139877,0.367698,1.0,1,3,2.0,2.0,0.60531650discourse,7880,1.101523,0.563381,1.0,0,10,1.0,4.0,0.36173351root,101455,0.0,0.0,0.0,0,0,0.0,0.0,0.052