Datasetpublic

TopicNet/PostNauka

PostNauka Some measurable characteristics of the dataset: D — number of documents W — modality dictionary size (number of unique tokens) len D — average document length in modality tokens (number of tokens) len D uniq — average document length in unique modality tokens (number of unique tokens) D @title W @title len D @title len D uniq @2gramm W @2gramm len D @2gramm len D uniq @3gramm W @3gramm len D @3gramm len D uniq @snippet W @snippet len D @snippet len D uniq… See the full description on the dataset page: https://huggingface.co/datasets/TopicNet/PostNauka.

sourceHugging Faceotherupdated 2y agoView on Hugging Face

1likes46downloads

Dataset Card

PostNauka

Some measurable characteristics of the dataset:

—D — number of documents
—<modality name> W — modality dictionary size (number of unique tokens)
—<modality name> len D — average document length in modality tokens (number of tokens)
—<modality name> len D uniq — average document length in unique modality tokens (number of unique tokens)

	D	@title W	@title len D	@title len D uniq	@2gramm W	@2gramm len D	@2gramm len D uniq	@3gramm W	@3gramm len D	@3gramm len D uniq	@snippet W	@snippet len D	@snippet len D uniq	@word W	@word len D	@word len D uniq	@post_tag W	@post_tag len D	@post_tag len D uniq	@author W	@author len D	@author len D uniq
value	3404	13546	3.97944	3.98002	163289	47.9697	37.0029	5848	1.71798	1.30288	46033	13.5232	13.5256	1.43373e+06	421.189	254.58	18031	5.297	5.297	3685	1.08255	1.08255

The @snippet modality is a piece of raw text designed to display top documents.

Information about document lengths in modality tokens:

	len_total@title	len_total@2gramm	len_total@3gramm	len_total@snippet	len_total@word	len_total@post_tag	len_total@author	len_uniq@title	len_uniq@2gramm	len_uniq@3gramm	len_uniq@snippet	len_uniq@word	len_uniq@post_tag	len_uniq@author
mean	3.98002	47.9697	1.71798	13.5244	421.189	5.297	1.08255	3.98002	37.0029	1.30288	13.5256	254.58	5.297	1.08255
std	2.21452	49.9748	3.07432	3.0559	482.422	2.01979	0.413483	2.21452	35.4885	1.81957	3.05377	249.288	2.01979	0.413483
min	1	0	0	5	3	1	0	1	0	0	5	3	1	0
25%	2	15	0	12	106	4	1	2	13	0	12	80	4	1
50%	3.5	33	1	13	294	5	1	3.5	27	1	13	197	5	1
75%	5	67	2	15	563	6	1	5	52	2	15	343	6	1
max	22	778	69	28	4892	18	4	22	444	27	28	2277	18	4

There exists a reference "gold-standard" topic model for 19 + 1 (background) topics. Information about topics is available for the natural word order version of the dataset. (At the same time, bag-of-words dataset with @snippet-s includes the same documents as the dataset with a natural order of words, but under different names — and information about the correspondence of document names, alas, is lost 🙃)