Sierra Open-Sources Hyper-τ-Bench to Test Whether AI Agents Can Build Customer-Service Agents

Sierra announced on September 8, 2026, that it is open-sourcing hyper-τ-bench, a long-horizon benchmark designed to measure whether AI coding agents can build a working customer-service agent. According to Sierra, the strongest automated configuration passed 23.9% of held-out evaluation tasks. By comparison, a reference pairing of an engineer with a frontier model passed 82.2%.

From Acting as an Agent to Building One

Sierra originally built τ-bench in 2024 to address a question it described as novel at the time: whether an AI system could act as an agent. Hyper-τ-bench shifts the focus from acting to construction, evaluating whether AI coding agents can assemble a functioning customer-service agent.