pharaouk/algorithmic-reasoning-seed

1---2license: mit3task_categories:4- text-generation5- question-answering6language:7- en8tags:9- code10size_categories:11- n<1K12---13# Dataset Card for Algorithmic Reasoning (seed)14 15**Note: This dataset is WIP and most question's answer section is empty or incomplete! See also "Other Known Limitations" section**16 17**Warning: If you somehow do use this dataset, remember to NOT do any eval after training on the questions in this dataset!**18 19## Dataset Description20 21- **Homepage:** 22- **Repository:** 23- **Paper:** 24- **Leaderboard:** 25- **Point of Contact:** lemontea.Tom@gmail.com or https://github.com/lemonteaa26 27### Dataset Summary28 29Dataset to help LLM learn how to reason about code, especially on algorithmic tasks, by seeing human demostration.30 31### Supported Tasks and Leaderboards32 33[More Information Needed]34 35### Languages36 37[More Information Needed]38 39## Dataset Structure40 41### Data Instances42 43[More Information Needed]44 45### Data Fields46 47- Question title48- Question49- Thought - Internal thought process that reason step by step/in an organized manner50- Answer presented to user (proof or code) - with explanation if necessary51 52### Data Splits53 54No split as of now - all are in the training section.55 56## Dataset Creation57 58### Curation Rationale59 60[More Information Needed]61 62### Source Data63 64#### Initial Data Collection and Normalization65 66Questions are those I personally remember in my career, selected based on:67 68- interesting69- involving CS, math, or similar knowledge70- Target specific known weaknesses of existing open source/source available LLM (eg index notation handling)71- pratical/likely to appear in production work settings72 73#### Who are the source language producers?74 75[More Information Needed]76 77### Annotations78 79#### Annotation process80 81Manually created by me entirely, writing in a level of details exceeeding what usually appears on the internet (bootcamp/FANNG interview prep/leetcode style training website etc) to help AI/LLM access knowledge that may be too obvious to human to write down.82 83#### Who are the annotators?84 85[More Information Needed]86 87### Personal and Sensitive Information88 89None as they are general, objective knowledge.90 91## Considerations for Using the Data92 93### Social Impact of Dataset94 95Although it is doubtful this dataset can actually work, in the event it does this may result in enhancing coding capability of LLM (which is intended), but which may create downstream effect simply due to LLM capability enhancement.96 97### Discussion of Biases98 99As questions are selected partly based on my taste, areas in CS that I am not interested in may be underrepresented.100 101### Other Known Limitations102 103- While I try to cover various mainstream programming language, each problem target only one specific language.104- It is currently in free-style markdown file. Maybe could make a script to convert to more structured format.105- Questions are asked in a conversational tone instead of leetcode style with strict I/O specification, hence may be more suitable for human eval than automated eval (eg extract and run code output in sandbox against test case automatically).106- As the dataset is completely manually created by a single human, the dataset size is extremely small.107 108## Additional Information109 110### Dataset Curators111 112[More Information Needed]113 114### Licensing Information115 116MIT117 118### Citation Information119 120[More Information Needed]121 122### Contributions123 124[More Information Needed]