mcp-server-tester
What it is
A Playwright-based testing and evaluation framework for MCP servers that allows developers to write deterministic unit tests and run data-driven evals using LLM-as-a-judge. It provides a high-level fixture for connecting to MCP servers via stdio or HTTP, along with custom matchers for asserting tool responses, schemas, and LLM behavior.
The problem it solves
The lack of a standardized, easy-to-use testing framework for Model Context Protocol (MCP) servers, which currently makes it difficult to write deterministic tests for tool calls and evaluate LLM-driven tool usage.
What you could use it for
- Regression testing MCP server tools in CI pipelines
- Validating MCP protocol conformance and tool schemas
- Evaluating LLM tool discoverability and response quality using rubrics
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.