bigcode/tiny_starcoder_py

1---2pipeline_tag: text-generation3inference: true4widget:5- text: 'def print_hello_world():'6  example_title: Hello world7  group: Python8license: bigcode-openrail-m9datasets:10- bigcode/the-stack-dedup11metrics:12- code_eval13library_name: transformers14tags:15- code16model-index:17- name: Tiny-StarCoder-Py18  results:19  - task:20      type: text-generation21    dataset:22      type: openai_humaneval23      name: HumanEval24    metrics:25    - name: pass@126      type: pass@127      value: 7.84%28      verified: false29---30 31# TinyStarCoderPy32 33This is a 164M parameters model with the same architecture as [StarCoder](https://huggingface.co/bigcode/starcoder) (8k context length, MQA & FIM). It was trained on the Python data from [StarCoderData](https://huggingface.co/datasets/bigcode/starcoderdata)34for ~6 epochs which amounts to 100B tokens.35 36 37## Use38 39### Intended use40 41The model was trained on GitHub code, to assist with some tasks like [Assisted Generation](https://huggingface.co/blog/assisted-generation). For pure code completion, we advise using our 15B models [StarCoder]() or [StarCoderBase]().42 43 44### Generation45```python46# pip install -q transformers47from transformers import AutoModelForCausalLM, AutoTokenizer48 49checkpoint = "bigcode/tiny_starcoder_py"50device = "cuda" # for GPU usage or "cpu" for CPU usage51 52tokenizer = AutoTokenizer.from_pretrained(checkpoint)53model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)54 55inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to(device)56outputs = model.generate(inputs)57print(tokenizer.decode(outputs[0]))58```59 60### Fill-in-the-middle61Fill-in-the-middle uses special tokens to identify the prefix/middle/suffix part of the input and output:62 63```python64input_text = "<fim_prefix>def print_one_two_three():\n    print('one')\n    <fim_suffix>\n    print('three')<fim_middle>"65inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)66outputs = model.generate(inputs)67print(tokenizer.decode(outputs[0]))68```69 70# Training71 72## Model73 74- **Architecture:** GPT-2 model with multi-query attention and Fill-in-the-Middle objective75- **Pretraining steps:** 50k76- **Pretraining tokens:** 100 billion77- **Precision:** bfloat1678 79## Hardware80 81- **GPUs:** 32 Tesla A10082- **Training time:** 18 hours83 84## Software85 86- **Orchestration:** [Megatron-LM](https://github.com/bigcode-project/Megatron-LM)87- **Neural networks:** [PyTorch](https://github.com/pytorch/pytorch)88- **BP16 if applicable:** [apex](https://github.com/NVIDIA/apex)89 90# License91The model is licensed under the BigCode OpenRAIL-M v1 license agreement. You can find the full agreement [here](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement).92