KSE-RESEARCH-Group/ukrainian-dialogs-constructiveness

Ukrainian Online Discourse: A Two-Axis Constructiveness Dataset

Expert-annotated Ukrainian-language dialogues from Telegram and the Ukrayinska Pravda Forum, scored on six ordinal dimensions of dialogic constructiveness organized into two orthogonal axes — Relational Conduct (RC) and Substantive Contribution (SC).

Note. This dataset accompanies an exploratory study. Findings should be treated as preliminary, given the single-language setting and the modest size of the expert-annotated split. See the Limitations section below.

Dataset summary

Each item is a single message together with up to nine preceding messages from the same direct-reply chain, providing pragmatic context. Every text field appears in both its original Cyrillic form and a Latin-script transliteration (KMU 2010 for Ukrainian, with a BGN/PCGN-style fallback for Russian-only letters).

Two-axis framework

Constructiveness is modeled as a multidimensional property along two independent axes, each decomposed into three subdimensions:

Relational Conduct (RC) — interpersonal quality of interaction

• —RC1 Respect / Tone — politeness, hostility, warmth of language
• —RC2 Engagement — social awareness, interest in continuing the exchange
• —RC3 Conflict Management — escalation vs. de-escalation, willingness to seek understanding

Substantive Contribution (SC) — epistemic value of the message

• —SC1 Reasoning Quality — logical structure, fairness of argumentation, treatment of evidence
• —SC2 Informativeness — new, relevant, or specific content
• —SC3 Dialog Continuity — how well the message follows from prior turns and advances the discussion

Each subdimension is scored on a centered 5-point ordinal scale:

Schema

Every row contains 62 columns, organized as follows:

Identifier and source

• —row_id (int) — stable per-split identifier
• —source (str) — telegram or ukrpravda

Conversational context (10 turns total)

For each i in 1..9 (where prev_1 is the immediately preceding turn):

• —prev_{i}_user (str) — pseudonymized speaker label
• —prev_{i}_text (str) — original Cyrillic text
• —prev_{i}_text_translit (str) — Latin-script transliteration

When fewer than nine context turns are available, earlier prev_* cells are empty.

Target message

• —user_label (str) — pseudonymized speaker label (or Unknown)
• —message_text (str) — original Cyrillic text
• —message_text_translit (str) — Latin-script transliteration

Expert annotations

For each annotator k in 1..3 and each subdimension d in {RC1, RC2, RC3, SC1, SC2, SC3}:

• —expert_{k}_{d} (int, range −2..+2)

In gold, all three annotator columns are populated (int64). In main, expert_3_* is populated only for the verification subset (57 / 450 rows); other rows have NaN (so all expert columns are float64).

Aggregated labels

For each subdimension d:

• —{d}_mean (float) — mean across available annotators
• —{d}_median (int / float) — median across available annotators

Loading

from datasets import load_dataset

gold = load_dataset("<org>/<repo>", "gold", split="train")
main = load_dataset("<org>/<repo>", "main", split="train")

If the int / float discrepancy between splits matters for your pipeline, declare the expert columns as nullable integer features explicitly when constructing Features.

Construction

Sources. Two Ukrainian-language venues spanning different eras of online discussion: the ShrikeChat Telegram channel (one of the most prominent Ukrainian political discussion hubs) and the legacy Ukrayinska Pravda Forum (20+ years of history). Only public messages were collected, via the official Telegram API and public web scraping respectively.

Context windows. Unlike datasets that group comments into loose threads, we preserve direct-response chains: each target message is paired with up to nine preceding direct replies. This 10-message window is intended to capture pragmatic signals (sarcasm, relational repair, retaliatory tone) that single- comment analyses miss.

Annotation pipeline.

1. 1.Each item was pre-labeled with Gemini 2.5 Flash.
2. 2.Calibration rounds aligned annotators on the rubric.
3. 3.Annotators independently reviewed and corrected labels in Argilla.
4. 4.The gold split was triple-annotated by three calibrated experts; main was double-annotated, with a third pass on items requiring verification.

Cleaning. Forum-specific cruft was removed prior to release: edit footers (Останнє редагування: ...), pasted UI elements ([ Відповісти в приват ] [Ігнорувати]), IP/Host: disclosures, and orphaned forum timestamps. The cleaning script is included alongside the data.

Transliteration. Latin-script columns use the Ukrainian National romanization (KMU 2010, Ukraine Cabinet of Ministers Resolution №55 of 27 January 2010), with positional rules (Я → Ya word-initially, ia elsewhere) and the зг → zgh disambiguation. Russian-only Cyrillic letters that occur in mixed-language items use a BGN/PCGN-style fallback.

Inter-annotator agreement (gold, N=300)

Collapsing the six rubric dimensions into the two meta-axes increases agreement, which is consistent with — though not by itself conclusive of — the existence of more stable two-dimensional latent structure. Aggregation alone can mechanically increase agreement, and disentangling this from genuine latent structure requires further study.

Suggested uses

• —Ordinal classification of dialogue turns along six interactional dimensions, or along the two collapsed RC/SC axes.
• —Conversational health analysis with a centered scale that treats positive (+1, +2) and negative (−1, −2) constructiveness symmetrically around a neutral anchor.
• —Pragmatic context studies using the 9-turn reply chain (e.g. retaliatory tone, sarcasm, repair attempts).

We recommend reporting Quadratic Weighted Kappa (QWK) when comparing predictions against expert labels, since the scale is ordinal and naive accuracy underweights large errors.

Limitations

• —Single language and political context. Items are Ukrainian-language political and civic discussion, much of it during a period of war. Norms of disagreement vary across languages, communities, and political conditions; we make no claim that the calibration of constructiveness observed here generalizes to other settings.
• —Small expert-annotated test set. Triple expert annotation across six ordinal dimensions is labor-intensive, which limited the feasible scale within our resources. The gold set contains 300 items (90 reserved for test in the paper's evaluation protocol), which constrains both fine-grained learning and the statistical power of QWK comparisons.
• —Possible LLM anchoring bias. Items were pre-labeled with Gemini 2.5 Flash before human review. Calibration and independent expert annotation on the gold split mitigate this, but anchoring effects toward the model's initial judgments cannot be ruled out and may inflate observed agreement.
• —Reply-based structure assumption. Constructiveness is evaluated within an explicit reply chain. In less structured discussions, broader interaction patterns may matter that pairwise adjacency does not capture.
• —Moderate IAA on relational dimensions. Tone and intent interpretation remains subjective; some learned decision boundaries may be influenced by annotation uncertainty.

Ethics and privacy

The dataset was constructed from publicly available Telegram channels (via the official Telegram API) and public discussion threads on the Ukrainska Pravda Forum. No private groups, direct messages, or restricted-access content were accessed. Identifiers were pseudonymized at collection time, and messages whose content could plausibly enable re-identification were removed before analysis. Annotators (student researchers fulfilling structured research duties under educational grants) worked exclusively with anonymized data and were briefed on confidentiality requirements.

The texts include political opinions, emotionally charged exchanges, and hostile language, including slurs and ad hominem attacks; the labels (e.g. −2 on RC1) flag such content. Users of the dataset should handle it accordingly.

Citation

@misc{korotenko2026dataset,
  title        = {Ukrainian Online Discourse: A Two-Axis Constructiveness Dataset},
  author       = {Korotenko, Artem and Kyslyi, Roman},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/KSE-RESEARCH-Group/ukrainiain-dialogs-constructiveness}}
}

License

Released under CC BY 4.0. Attribution to the dataset and accompanying paper is required for redistribution and downstream use.