Evaluations on 'Needle in the Web' reveal that current LLM-based search agents, including leading closed-source and open-source models, largely struggle with fuzzy exploratory search. Most models achieve below 35% accuracy, with no single agent consistently excelling across all domains or difficulty levels. This highlights a significant gap between current capabilities and real-world web search demands.

Analysis of agent behavior shows common pitfalls: misunderstanding search tool capabilities, inefficient web content retrieval (especially for open-source agents), and issues with semantic matching. Agents often perform fragmented searches, failing to synthesize information from various sources into a coherent answer. This underlines the need for more robust reasoning and integration capabilities.