datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Linguistic-Diagnostics-Syntax
LINDSEA Syntax
LINDSEA Syntax is a linguistic diagnostic from BHASA that evaluates a model's understanding of linguistic phenomena, syntax in particular, for Indonesian.
Supported Tasks and Leaderboards
LINDSEA Syntax is designed for evaluating chat or instruction-tuned large language models (LLMs).
Languages
Indonesian (id)
Dataset Details
LINDSEA Syntax only has an Indonesian (id) split, with additional splits containing fewshot examples. Below… See the full description on the dataset page: https://huggingface.co/datasets/aisingapore/Linguistic-Diagnostics-Syntax.ML-1M-Syntax-Validated-Python-Code
ML-1M Syntax-Validated Python Code
Dataset Summary
ML-1M Syntax-Validated Python Code is a large-scale corpus containing over 1 million machine-learning–oriented Python programs derived from The Stack, a permissively licensed collection of open-source source code.
The dataset is constructed through heuristic ML-domain filtering, syntactic validation, and basic safety checks. It is intended to support empirical analysis of real-world ML code, executability and dependency… See the full description on the dataset page: https://huggingface.co/datasets/Noushad999/ML-1M-Syntax-Validated-Python-Code.mmevol-zh-hant
MMEvol - Translated Chinese Traditional
A subset of Tongyi-ConvAI/MMEvol translated using yentinglin/Llama-3-Taiwan-70B-Instruct from english to traditional chinese.
Read the Note below before use.
Image source distribution:
Dataset
Count
Percentage
coco
6598
29.8%
Q-Instruct-DB
5856
26.4%
clevr
2383
10.8%
chartqa
1733
7.8%
hfdata
1296
5.9%
geo170k
706
3.2%
data_engine
6983.2%
mathvision
644
2.9%
docvqa
600
2.7%
alfworld
401
1.8%
arxivqa
337
1.5%… See the full description on the dataset page: https://huggingface.co/datasets/syntaxsynth/mmevol-zh-hant.instruct_code_cleaning
SFT code dataset building
Contain a list of tasks useful when building a iniitial dataset source:
reverse_translation
Given a history of conversations, what would the human ask next?
reverse_translation_first_round
Suppose you already have a response, the LLM must predict what question does the human asked
clean_code
Given a code snippet, it determines whether its useful and atomic enough to be use for a response by LLM
gen_code_question
Generates a question given a… See the full description on the dataset page: https://huggingface.co/datasets/syntaxsynth/instruct_code_cleaning.