flagopen/starcoder-15b-taco

1---2language:3- en4- code5license: bigcode-openrail-m6tags:7- starcoder8- code_synthesis9- competition-level_code_generation10datasets:11- BAAI/TACO12---13# Starcoder-15B-TACO14 15## Model Description16 17Starcoder-15B-TACO is a Starcoder-15B finetuned(full-parameter) on TACO dataset. This model is specialized to solve competition-level programming tasks.18 19## Training data20 21The model is trained on the [Topics in Algorithmic Code Generation Dataset](https://github.com/FlagOpen/TACO). The dataset focused on algorithmic code generation, aiming to provide a more challenging training dataset and evaluation benchmark for the code generation model field. It includes 25,443 problems in the training set and 1,000 problems in the test set, making it the largest code generation dataset currently available. Each TACO problem is designed to match a diverse set of solution answers, with answers reaching sizes up to 1.55M, to ensure that models trained on this dataset are robust and not prone to overfitting. Furthermore, the TACO dataset includes fine-grained labels such as task topics, algorithms, skills, and difficulty levels, offering more precise guidance for both training and evaluating code generation models.22This model is fine-tuned using train split of TACO.23 24## Training procedure25 26The training script used to train this model can be found [here](https://github.com/FlagOpen/TACO/blob/main/train.py).27 28Training Details can be seen in our [paper](https://arxiv.org/abs/2312.14852)29 30 31## Intended Use and Limitations32 33The model is finetuned to solve programming problems given a text description and optional starter code.34 35### How to use36 37You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run:38 39```py40from transformers import AutoModelForCausalLM, AutoTokenizer, FlaxAutoModelForCausalLM41model = AutoModelForCausalLM.from_pretrained("flagopen/starcoder-15b-taco")42tokenizer = AutoTokenizer.from_pretrained("flagopen/starcoder-15b-taco")43prompt = """44A function to greet user. Given a user name it should say hello45def greet(name):46ANSWER:47""" 48input_ids = tokenizer(prompt, return_tensors='pt').input_ids.to(device)49start = input_ids.size(1)50out = model.generate(input_ids, do_sample=True, max_length=50, num_beams=2, 51                     early_stopping=True, eos_token_id=tokenizer.eos_token_id, )52print(tokenizer.decode(out[0][start:]))53```54 55### Limitations and Biases56 57The model is intended to be only used for research purposes and comes with no guarantees of quality of generated code.58 59 60## Eval results61 62Coming soon...