lzy0928/Engineering_Aerospace
Text data related to the subject of Aerospace Engineering. Compressed file "aerospace-*.tar.gz" is a collection of json files, which are filtered from Fineweb dataset. The classification was done by fasttext (https://github.com/ZJLab-ZBZX/Data-Processing-Toolkit-for-LLMs/tree/main/Subject_Classifier). And we randomly selected 2.5% classified data, and checked them by Qwen-2-7B LLM through customized prompt. The accuracy of the classified data is about 92.86%. File "stat-aerospace.jsonl"… See the full description on the dataset page: https://huggingface.co/datasets/lzy0928/Engineering_Aerospace.
Text data related to the subject of Aerospace Engineering.
Compressed file "aerospace-*.tar.gz" is a collection of json files, which are filtered from Fineweb dataset.
The classification was done by fasttext (https://github.com/ZJLab-ZBZX/Data-Processing-Toolkit-for-LLMs/tree/main/Subject_Classifier).
And we randomly selected 2.5% classified data, and checked them by Qwen-2-7B LLM through customized prompt.
The accuracy of the classified data is about 92.86%.
File "stat-aerospace.jsonl" shows some information of "aerospace-*.tar.gz".
File "wikiaerospaceyes.jsonl" is a json file filtered from Wikipedia dataset.
File "sft-aerospace.jsonl" is a json file for supervised fine-tuning (SFT).