TL;DR — After 1,800+ evaluation trials across 9 system generations, we withdrew from the Gemma 4 competition and are publishing our measurements instead.
Our best custom system scored 15/114 (13.2%) — one task behind the organizer's simplest zero-think baseline ( 16/114, 14.0% ), and never ahead of it.
Longer thinking actively hurt the 4B agent: the think=4096 baseline scored 10/114 vs 16/114 for zero-think (−37.5% relative).
On the unseen holdout, our custom systems (5–6/44) consistently beat the zero-think baseline (4/44) — the machinery helped there, just not enough overall.
Hi everyone — we're Japan Open Data Lab. After 1,800+ evaluation trials across 9 system generations, we've decided to withdraw from this competition, and we'd rather publish our measurements than let them sit in a drawer. Everything below is aggregate results and findings only — no task-level data, no patches, no submission code.
Setup.…