procesaur/Wiki.korpus

<table style="width:100%;height:100%"> <!--tr style="width:100%;height:30px">

<td colspan=2 align=center>

</td> <tr--> <tr style="width:100%;height:100%"> <td width=50%> <h2>Viki korpusi na srpskom i hrvatskom jeziku</h2> <p>Sveža verzija, <span class="highlight-container"><span class="highlight">1. maj 2026</span></span>!</p> <p>Očišćen i filtriran skup pet projekata: Vikipedija, Vikizvornik, Vikiknjige, Vikivesti i Vikicitati.</p> <p>Preko 670.000 očišćenih članaka, sa <span class="highlight-container"><span class="highlight">preko 310 miliona reči</span></span>.</p> <p></p> <p>Svaki dokument je u zasebnoj JSON liniji.</p> <p>Novi metapodaci! Kategorije, broj reči i postotak ćiriličnog teksta</p> <p>Moguće filtiranje skupa po jeziku ili projektu.</p> </td> <td> <h2>Wiki corpora in Serbian and Croatian language</h2> <p>Fresh version, <span class="highlight-container"><span class="highlight">May 1^st 2026</span></span>!</p> <p>Cleaned and filtered set of five projects: Wikipedia, Wikisource, Wikibooks, Wikinews and Wikiquotes.</p> <p>Over 670,000 cleaned articles, with <span class="highlight-container"><span class="highlight">over 310 million words</span></span>.</p> <p></p> <p>Each document is in a separate JSON line.</p> <p>New Metadata! Categories, word count and percentage of Cyrillic text</p> <p>Possible to filter the set by language or project.</p> </td> </tr> </tr> <tr> <td colspan=2> <table style='width:100%;font-size:12pt;text-align:right'> <tr> <th style='width:16%'><b>Broj dokumenata<br/>Doc. count</b></th> <th style='width:12%'><b>Vikipedija<br/>Wikipedia</b></th> <th style='width:12%'><b>Vikizvornik<br/>Wikisource</b></th> <th style='width:12%'><b>Vikivesti<br/>Wikinews</b></th> <th style='width:12%'><b>Vikicitati<br/>Wikiquotes</b></th> <th style='width:12%'><b>Vikiknjige<br/>Wikibooks</b></th> <th style='width:12%'><b>Ukupno<br/>Total</b></th> <th style='width:12%'><b>Udeo<br/>Share</b></th> </tr> <tr> <td>Srpski 🇷🇸</td> <td>237,255</td> <td>36,992</td> <td>53,012</td> <td>5,101</td> <td>648</td> <td>333,008</td> <td>49.2%</td> </tr> <tr> <td>Hrvatski 🇭🇷</td> <td>152,894</td> <td>9,392</td> <td></td> <td>18</td> <td>1,117</td> <td>163,421</td> <td>24.1%</td> </tr> <tr> <td>Srpskohrvatski 🇷🇸🇭🇷</td> <td>130,299</td> <td></td> <td></td> <td></td> <td></td> <td>130,299</td> <td>19.2%</td> </tr> <tr> <td>Bosanski 🇧🇦</td> <td>47,610</td> <td>1,890</td> <td>343</td> <td>527</td> <td>45</td> <td>50,415</td> <td>7.4%</td> </tr> <tr> <td><b>Ukupno / Total</b></td> <td>568,058</td> <td>48,274</td> <td>53,355</td> <td>5,646</td> <td>1,810</td> <td>677,143</td> <td>100%</td> </tr> </table> </td> </tr> <tr> <td colspan=2> <table style='width:100%;font-size:12pt;text-align:right'> <tr> <th style='width:16%'><b>Broj reči<br/>Word count</b></th> <th style='width:12%'><b>Vikipedija<br/>Wikipedia</b></th> <th style='width:12%'><b>Vikizvornik<br/>Wikisource</b></th> <th style='width:12%'><b>Vikivesti<br/>Wikinews</b></th> <th style='width:12%'><b>Vikicitati<br/>Wikiquotes</b></th> <th style='width:12%'><b>Vikiknjige<br/>Wikibooks</b></th> <th style='width:12%'><b>Ukupno<br/>Total</b></th> <th style='width:12%'><b>Udeo<br/>Share</b></th> </tr> <tr> <td>Srpski 🇷🇸</td> <td>134,515,581</td> <td>22,210,921</td> <td>7,577,930</td> <td>1,036,743</td> <td>712,395</td> <td>166,053,570</td> <td>53.4%</td> </tr> <tr> <td>Hrvatski 🇭🇷</td> <td>57,985,132</td> <td>9,674,234</td> <td></td> <td>5,035</td> <td>303,226</td> <td>67,967,627</td> <td>21.8%</td> </tr> <tr> <td>Srpskohrvatski 🇷🇸🇭🇷</td> <td>50,153,319</td> <td></td> <td></td> <td></td> <td></td> <td>50,153,319</td> <td>16.1%</td> </tr> <tr> <td>Bosanski 🇧🇦</td> <td>24,479,558</td> <td>2,264,631</td> <td>46,735</td> <td>147,015</td> <td>29,389</td> <td>26,967,328</td> <td>8.7%</td> </tr> <tr> <td><b>Ukupno / Total</b></td> <td>267,133,590</td> <td>34,149,786</td> <td>7,624,665</td> <td>1,188,793</td> <td>1,045,010</td> <td>311,141,844</td> <td>100%</td> </tr> </table> </td> </tr> </table>

from datasets import load_dataset
# Učitavanje samo srpskog dela
dataset = load_dataset("procesaur/WikiViki", split="sr")
print(dataset[6]["text"])

Хоби је име за активност која се обавља због личног задовољства, а не због новчаних награда или примања...

from datasets import load_dataset
# Učitavanje samo vikicitata
dataset = load_dataset("procesaur/WikiViki", split="wikiquote")
print(dataset[5]["text"])

1 Citati
Čovjek je, Senada, teški hajvan. I ono malo duše što ima, šejtan mu je dao da mu bude teže...

# Filtriranje prema metapodacima
filtered = dataset.filter(lambda example: example["subject"] == "J.R.R. Tolkin")
print(f"{filtered[0]["title"]}\n{filtered[0]["text"]} )

Gospodar prstenova: Povratak kralja
1 Citati
*Gandalf: "Sutra će biti veći jad nego što je bio danas i tako će biti još dugo...

Cite

@misc{škorić2026wikidumpstrainingcorpora,
      title={Wiki Dumps to Training Corpora: South Slavic Case}, 
      author={Mihailo Škorić and Cosimo Palma},
      year={2026},
      eprint={2604.25384},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2604.25384}, 
}

<div class="inline-flex flex-col avatar-cont"> <div class="avatar-title">Editor</div> <a href="https://huggingface.co/procesaur"> <div class="flex"> <div class="avatar-cover" style="background-image: url(&#39;https://cdn-uploads.huggingface.co/production/uploads/1673534533167-63bc254fb8c61b8aa496a39b.jpeg?w=200&h=200&f=face&#39;)"> </div> </div> <div class="avatar-name">Mihailo Škorić</div> <div class="avatar-handle">@procesaur</div> </a> </div> </div>

<div class="inline-flex flex-col avatar-cont"> <div class="avatar-title">Data</div> <a href="https://www.wikimedia.org/"> <div class="flex"> <div class="avatar-cover" style="background-image: url(&#39;https://upload.wikimedia.org/wikipedia/commons/thumb/5/53/Wikimedia-logo.png/250px-Wikimedia-logo.png&#39;)"> </div> </div> <div class="avatar-name">Wikimedia</div> <div class="avatar-handle">@www.wikimedia.org</div> </a> </div> </div>

<div class="inline-flex flex-col avatar-cont"> <div class="avatar-title">Funding</div> <a href="https://fondzanauku.gov.rs/"> <div class="flex"> <div class="avatar-cover" style="background-image: url(https://fondzanauku.gov.rs/wp-content/uploads/2019/04/FZN-logo-primary-light-1170x396.png)"> </div> </div> <div class="avatar-name">Fond za Nauku</div> <div class="avatar-handle">@fondzanauku_rs</div> </a> </div> </div>

<table style="width:100%;height:100%"> <tr style="width:100%;height:100%"> <td width=50%> <p>Истраживање jе спроведено уз подршку Фонда за науку Републике Србиjе, #7276, Text Embeddings – Serbian Language Applications – TESLA</p> </td> <td> <p>This research was supported by the Science Fund of the Republic of Serbia, #7276, Text Embeddings - Serbian Language Applications - TESLA</p> </td> </tr> </table>

<div id="zastava"> <div class="grb"> <img src="https://www.ai.gov.rs/img/logo_60x120-2.png" style="position:relative; left:30px; z-index:10; height:85px"> </div> <table width=100% style="border:0px"> <tr style="background-color:#C6363C;width:100%;border:0px;height:30px"><td style="width:100vw"></td></tr> <tr style="background-color:#0C4076;width:100%;border:0px;height:30px"><td></td></tr> <tr style="background-color:#ffffff;width:100%;border:0px;height:30px"><td></td></tr> </table> </div>

<img src="cover.png" class="cover">

<style> .ffeat: { color:red }

.cover { width: 100%; margin-bottom: 5pt }

.highlight-container, .highlight { position: relative; text-decoration:none }

.highlight-container { display: inline-block; }

.avatar-cont { line-height: 1.5; padding-right:50px; }

.avatar-title{ text-align: center; margin-top: 3px; font-size: 16px; font-weight: 800; }

.avatar-name{ text-align: center; font-size: 16px; font-weight: 800; }

.avatar-handle{ text-align: center; font-size: 14px; }

.avatar-cover{ display:DISPLAY_1; margin-left: auto; margin-right: auto; width: 92px; height:92px; border-radius: 50%; background-position: center; background-size: contain; background-repeat: no-repeat; }

.highlight{ color:white; text-transform:uppercase; font-size: 16pt; }

.highlight-container{ padding:5px 10px }

.highlight-container:before { content: " "; display: block; height: 100%; width: 100%; margin-left: 0px; margin-right: 0px; position: absolute; background: #e80909; transform: rotate(2deg); top: -1px; left: -1px; border-radius: 20% 25% 20% 24%; padding: 10px 18px 18px 10px; }

div.grb, #zastava>table { position:absolute; top:0px; left: 0px; margin:0px }

div.grb>img, #zastava>table{ margin:0px }

#zastava { position: relative; margin-bottom:120px }

p { font-size:14pt } </style>