Ai2’s XYBench tests whether AI replies focus on what users need

The Allen Institute for AI, or Ai2, announced XYBench on November 13 to test how language models respond when users ask for help with a method that may miss their real goal. In the study’s Stack Overflow tests, five models centered the user’s underlying goal in 35–41% of the answers they wrote. The highest-scoring human answers did so 55% of the time.

Ai2 illustrated the test with a baker who asks how to thicken frosting that keeps sliding off a cupcake. If the cupcake is too warm, a detailed answer about thickening the frosting can leave the problem intact. XYBench records the requested fix and the intended outcome separately. Teams evaluating assistants can use that distinction to see whether an answer follows the requested method or helps get the result the user was after.

A solution can appear without leading the answer

The paper reports 8,115 questions. They include 1,557 from Stack Overflow, 184 from other Stack Exchange sites and 102 examples curated by the researchers. The largest set consists of synthetic questions that the team built around mistakes people might make while following WikiHow guides. The WikiHow questions have no human reference answers.

In one programming example, a user wants to extract data from XML and asks for a regular expression. A regular expression searches for text patterns, while a parser can follow the nesting of elements in a structured document. The researchers scored whether answers offered a solution for each request, which solution the answer emphasized, and whether it addressed the mistaken premise behind the user’s approach.

On Stack Overflow, GPT-5 mentioned a solution to the underlying goal in 71% of its answers, above the 67% rate for the highest-scoring human answers. But it made that goal the focus of only 41% of responses, compared with 55% for those human answers. A coding assistant can include a better tool in a paragraph and still spend most of its reply developing the method the user already picked.

Picking advice and writing it

A separate multiple-choice test added GPT-4.1 to the five-model group. For each question, the researchers presented one answer that followed the literal request and one that addressed the misconception and the underlying goal. They kept the two answers similar in length and varied their order. On the Stack Overflow questions, the six models selected the goal-focused option in 71–84% of cases. That experiment supplied a ready-made alternative. The free-response test measured what five of the models wrote on their own.

Claude Sonnet 4.5 scored the free responses. Two paper authors independently graded a sample of 500 answers to check the automated judge. Agreement ranged from 97.5% to 100% for whether a solution appeared and from 74.2% to 79.8% for the focus of the answer. The study tested a single response to each question.

The manuscript was submitted to arXiv in September and appeared in the October EMNLP proceedings. EMNLP lists it as the conference’s Best Resource Paper. The November post links to the paper, the evaluation code and the dataset card. Its results come from English questions, mainly about carrying out a procedure. A conversation where an assistant can ask a follow-up poses a different test.

The paper and the materials linked in the announcement give different totals. The paper reports 6,272 WikiHow-based questions. The public data card and code repository list 6,268, and their other three entries match the paper’s counts. The public tables add up to 8,111 and give no explanation for the four-question difference.

The paper’s appendix also anticipated a CC BY 4.0 release. The Hugging Face card currently lists the dataset under CC BY-NC-SA 3.0, with a noncommercial restriction and access conditions that users must accept after signing in. The separate evaluation code repository carries an Apache 2.0 license.