kardosdrur/dawiki_categories
Danish Wikipedia Categories The dataset was created entirely from the last Danish Wikipedia dump by traversing the category hierarchy in the categorylinks table. All categories that were one level bellow the topcategories, and which had more than 30 articles assigned to them were selected. In order to see whether an article belongs to a certain category I checked, whether the article was connected to the category in the directed graph of the category hierarchy. If the length of… See the full description on the dataset page: https://huggingface.co/datasets/kardosdrur/dawiki_categories.
Danish Wikipedia Categories
The dataset was created entirely from the last Danish Wikipedia dump by traversing the category hierarchy in the categorylinks table.
All categories that were one level bellow the topcategories, and which had more than 30 articles assigned to them were selected. In order to see whether an article belongs to a certain category I checked, whether the article was connected to the category in the directed graph of the category hierarchy. If the length of the path between the article and a category was less than or equal to 5 edges (arbitrary choice, but I found it to work well), the category was assigned to the article.
Note that this is a multilabel classification task. Its purpose is to train sentence transformers for the DFM project using a technique similar to SetFit.