Can LLM Agents Be Forward-Deployed Engineers? Requirement Discovery and Delivery for Enterprise Text-to-SQL
What it is
This paper evaluates whether LLM agents can function as forward-deployed engineers in a real enterprise text-to-SQL engagement. It simulates a customer with ambiguous requirements to test if agents can discover acceptance criteria and execute post-training recipes, finding that while strong models discover most requirements, only 25% of runs successfully meet all production criteria.
The problem it solves
Uncertainty about whether LLM agents can reliably handle the full lifecycle of enterprise software deployment, from ambiguous requirement discovery to production-ready delivery.
What you could use it for
- Evaluating LLM agent capabilities for enterprise software deployment
- Benchmarking requirement discovery and delivery workflows for text-to-SQL systems
- Identifying bottlenecks in autonomous AI engineering pipelines
Why it matters
Provides a rigorous framework for assessing the readiness of LLM agents for autonomous enterprise deployment, highlighting specific gaps in requirement discovery and execution reliability that must be addressed before full automation.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.