RecToolBench: Benchmarking Recommendation-Specific Tool Orchestration under Fuzzy User Intent
What it is
RecToolBench is an MCP-based benchmark designed to evaluate agentic recommender systems that must resolve fuzzy user intent using external tools. It contains over 1,200 executable tasks across three domains, utilizing 13 MCP servers and 32 tools to test single, parallel, sequential, and hybrid tool orchestration.
The problem it solves
Existing benchmarks assume explicit user intent and isolated function calls, failing to capture the complexity of realistic tool orchestration required when user instructions are fuzzy or ambiguous.
What you could use it for
- Evaluating the robustness of LLM agents in handling ambiguous user queries for recommendation tasks
- Benchmarking the ability of models to ground semantic parameters in tool calls
- Assessing multi-step evidence integration and final recommendation quality in agentic pipelines
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.