τ³-Banking

The τ³-Banking leaderboard

τ³-Banking is the fintech customer-support domain of the τ-Knowledge framework, designed to evaluate whether conversational agents can complete real work in knowledge-intensive banking scenarios.

The benchmark comprises ~700 interconnected policy documents (≈195K tokens) across 21 product categories, from product selection to dispute resolution to account management. Agents are graded against backend database state outcomes, such as whether a dispute was actually filed, a credit issue resolved, or an account opened correctly, rather than matching reference answers.

Example tasks include cash-back card recommendations, dispute resolution with competing policies, and business account selection with conflicting promotions. The benchmark was published in collaboration with Sierra AI and includes an open-source dataset and execution harness.