CoolFace
Modelpublic

THU-KEG/LLaDA-8B-BGPO-code

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
1likes14downloads
Model Card

LLaDA-8B-BGPO-code

![Paper](https://arxiv.org/abs/2510.11683) ![Code](https://github.com/THU-KEG/BGPO)

Model Description

LLaDA-8B-BGPO-code is an 8-billion parameter diffusion large language model (dLLM) that was trained on LLaDA-8B-Instruct using Boundary-Guided Policy Optimization (BGPO) for enhanced code generation capabilities.

Model Details

  • —Model Type: Diffusion Large Language Model (dLLM)
  • —Parameters: 8 billion
  • —Training Method: Boundary-Guided Policy Optimization (BGPO)
  • —Base Model: LLaDA-8B-Instruct
  • —Task: Code generation
  • —Language: English

Training Details

  • —Training Epochs: 5 epochs (112 steps per epoch)
  • —Total Steps: 560 steps
  • —Response Length: 512 tokens
  • —Train Diffusion Steps: 512
  • —Eval Diffusion Steps: 512
  • —Block Size: 32
  • —Monte Carlo Sample Size ($n_t$): 16
  • —Learning Rate: 5e-7
  • —Batch Size: 16
  • —Framework: Built on VeRL (Volcengine Reinforcement Learning)

Usage & Limitations

  • —Primarily designed for code generation tasks.
  • —Performance may vary on other tasks.
  • —Requires appropriate computational resources for inference.