Experiments in AI‑assisted software engineering.
LLM Coder Lab puts large language models to work on real software — then publishes what happened. Benchmarks on genuine tasks, complete build logs, and field notes on what these tools can actually do, where they break, and how to get the most out of them.
Benchmarks
Head-to-head coding evaluations on real tasks — not toy puzzles. Frontier and open-weight models, scored on working software.
Build logs
Complete, unedited session transcripts of real projects built with LLM agents — every prompt, every wrong turn, every fix.
Field notes
Practical techniques from the bench: agent workflows, prompting patterns, and the failure modes nobody writes up.
First results are in. See the leaderboard — more models and harder challenges are coming.