Agentic Testing: Where Agents Fit in the E2E Testing Stack

AI summary

This article analyzes the efficacy of agent-driven end-to-end (E2E) testing compared to traditional deterministic tests. Through over 200 experiments using the Playwright MCP, CLI, and LLM-generated scripts, the authors evaluate the reliability, cost, and execution time of goal-oriented agent testing. They conclude that while traditional tests remain essential for CI, agentic testing provides a valuable layer for exploratory and complex behavior validation.

Key points

• Traditional tests enforce specific UI journeys, whereas agent-driven tests focus on validating high-level goal completion.
• Agentic testing offers high flexibility but currently carries significant tradeoffs in cost and execution duration compared to static scripts.
• Effectiveness in agentic testing is tied to the chosen execution model, with variations between browser-level interaction (MCP) and shell-level command execution (CLI).
• Agent-driven and deterministic testing strategies are complementary; deterministic tests provide stability, while agents provide exploratory coverage.