tojpaj/marketnews-advisory-indic

Market & News Advisory (Hindi/Punjabi)

Native Hindi and Punjabi text from ai4bharat/IndicCorpV2, adapted with AutoScientist into substantive domain responses written by a market analyst explaining what a news item means for Indian markets.

This set has 3 responses per source text

3,596 rows are generated from 1,200 unique source texts — roughly 3 responses each. Treat the effective source diversity as 1,200, not 3,596, when sampling or splitting: a naive random split will leak the same source across train and test.

Reading that relative-change number

It is near zero, and that is expected rather than a defect. IndicCorpV2 is professionally written native prose, so the source already scores ~9/10 before adaptation touches it — there is no headroom for a large relative gain.

The relative-improvement metric substantially measures how poor the source data was. Our sibling Kisan dataset shows a headline +93.3% — because its source, raw call-centre notes, scores 3.0/10. Its adapted output reaches 5.8/10 (grade C). This dataset shows a flat-to-negative relative change while sitting a full grade higher in absolute terms.

Judge these on the absolute score. What adaptation contributed here is not a quality lift on the prose — it is a task transformation, turning raw article text into structured domain responses with reasoning traces.

Language provenance — measured, not assumed

The source text was written in Hindi or Punjabi by Indian speakers (IndicCorpV2), so it is natively authored, not machine-translated.

Each generation instruction asks for a response in the same language as its source. That was verified per row by dominant script rather than trusted:

33% of responses answer in the other Indic language (a Hindi article answered in Punjabi, or the reverse). The source text remains native throughout, but the response on those rows is effectively a cross-language answer rather than a same-language one. If you need strict same-language pairs, filter on dominant script — the check is ~15 lines and dominant_script() in this repo's publish_generative.py is the exact implementation used to produce the number above.

This is disclosed rather than filtered out because the cross-language rows are still coherent, grounded responses — they are simply not what the instruction asked for, and you should know which you are getting.

Data hygiene: 4 rows removed

4 row(s) were dropped because enhanced_completion contained the evaluator's internal reasoning rather than an actual response — text like **Step 1 — Compare:** Response 0 effectively identifies.... This is grader scaffolding leaking into the data.

The platform's quality grade did not flag it. This dataset ships filtered; the detector is SCAFFOLD in publish_generative.py. Anyone building on AutoScientist output should run an equivalent check — the leak rate is low (<1%) but the rows are pure noise as training targets.

Why generative rather than classification

An earlier version of this entry emitted single-token labels from a taxonomy we invented. Adaption scores submissions against their held-out in-house test sets, so an invented label space is very likely unscoreable no matter how clean the data is. A model fine-tuned to emit one-word labels also produces garbage against open-ended evaluation prompts.

The fix was to keep the same source text and change what the universal_prompt generates — a real domain response instead of a label.

Grounding constraints

The generation instruction forbids inventing figures, companies, policies, studies, statistics, laws or case numbers not present in the source, and hallucination_mitigation was enabled. For the HR and Personal-Finance sets the instruction additionally requires that harassment, safety risks and suspected fraud be escalated to a human authority or helpline rather than resolved by the model.

Note the honest limit: these are instruction-level constraints, and this project's own Kisan methodology documents three separate cases where blueprint instructions failed to enforce numeric fidelity, and only external mechanical filtering worked. Treat the grounding here as best-effort and unverified — unlike the language-fidelity and scaffolding numbers above, which were measured.

Fields

• —text — source IndicCorpV2 snippet
• —enhanced_prompt / enhanced_completion — AutoScientist output
• —reasoning_trace — chain-of-thought (reasoning_traces recipe)

No fine-tuned model ships with this dataset

Three training runs on the sibling Kisan dataset — two base models (gemma-3-4b-it, Qwen3.5-0.8B) and two completion registers — all landed at or below parity with their own base model (0.3528 / 0.5023 / 0.5051, where 0.5 is a tie). Modern instruction-tuned models are already about as good at open-ended Indic advisory text as a few thousand rows can make them. Full write-up in the Kisan METHODOLOGY. The dataset is the contribution.

Built for the Adaption AutoScientist Challenge, Part 2, Market Analysis & News.