fullstack
FullStackBenchFullStack Bench: Evaluating LLMs as Full Stack Coders
Official repository for our paper "FullStack Bench: Evaluating LLMs as Full Stack Coders"
🏠 FullStack Bench Code •
📊 Benchmark Data •
📚 SandboxFusion
📌Introduction
FullStack Bench is a multilingual benchmark for full-stack programming, covering a wide range of application domains and 16 programming languages with 3K test samples, which substantially pushes… See the full description on the dataset page: https://huggingface.co/datasets/ByteDance/FullStackBench.fullstackbench-tasks
fullstackbench-tasks
Harbor task definitions used to benchmark coding agents in the FullStackBench / Loki research program. Each top-level directory is one self-contained task — Dockerfile, golden app, verifier, instruction — runnable via harbor run.
Prerequisites
hf CLI logged in to an ApolloTeam-member account (hf auth login)
Working install of Harbor — provides the harbor CLI (ask the Loki team if you don't already have it set up)
Docker daemon running locally
~4 GB… See the full description on the dataset page: https://huggingface.co/datasets/ApolloTeam/fullstackbench-tasks.fullstackbench-trajectories
fullstackbench-trajectories
FullStackBench tasks and the agent trajectories that ran them. One repo, two sibling trees keyed by the same task ID.
Layout
tasks/
└── <task-id>/ # task definition
├── environment/ # docker-compose, source app, fixtures
├── tests/ # verifier scripts
└── solution/ # reference impl (if applicable)
trajectories/
└── <task-id>/… See the full description on the dataset page: https://huggingface.co/datasets/ApolloTeam/fullstackbench-trajectories.FullStack-Bench
FullStack-Agent
Paper | Code | Dataset
Overview
This repository contains the FullStack-Bench dataset, introduced in the paper "FullStack-Agent: Enhancing Agentic Full-Stack Web Coding via Development-Oriented Testing and Repository Back-Translation".
In this paper, we propose FullStack-Agent, a unified system that combines a multi-agent full-stack development framework equipped with efficient coding and debugging tools (FullStack-Dev), an iterative self-improvement method… See the full description on the dataset page: https://huggingface.co/datasets/luzimu/FullStack-Bench.fullstackarena-gpt56sol-representatives-20260829
FullStackArena representative trajectories
This bundle contains one pass@1 model trajectory for each of 30 RideApp and 31
Arenagram template families, plus two explicitly labelled post-fix Arenagram
validation reruns. It does not claim that the model executed all 120 RideApp
or all 93 Arenagram instances.
Site
Phase
Raw evaluator
Adjudicated
rideapp
original_representative_run
25/30
28/30
arenagram
original_representative_run
22/31
22/31
arenagram… See the full description on the dataset page: https://huggingface.co/datasets/GroupieSteven/fullstackarena-gpt56sol-representatives-20260829.stargate_s04e01_100topkdiverse_text2vid