E2E Testing in Natural Language: Trying Chrome DevTools MCP

E2E Testing in Natural Language: My Experience with Chrome DevTools MCP After Playwright

When I first added E2E tests with Playwright, the biggest hurdle was the time required to write test scripts. Writing and maintaining the scripts cost more than running the tests themselves, so expanding E2E coverage was not always easy given the project's circumstances.

Things have changed somewhat recently. • AI can now generate first drafts of Playwright scripts, greatly reducing the initial cost, though maintenance remains a challenge. • More interestingly, approaches such as Chrome DevTools MCP have appeared, letting you control a browser in natural language without scripts, or with very little scripting.

Natural-language testing is unlikely to guarantee the same reproducibility as scripted tests. But I found that templating scenarios in Markdown (md) to control the freedom of natural language creates an advantage: E2E testing that can respond flexibly.

This post documents that experience.

⸻

MCP and Chrome DevTools MCP

What Is MCP (Model Context Protocol)?

MCP (Model Context Protocol) is an open standard that lets LLMs, such as Claude, interact with external tools and data sources in a standardized way. Put simply, it is like a standard adapter connecting AI and tools.

Claude Code also provides documentation and guides for connecting to various external tools and servers through MCP.

What Is Chrome DevTools MCP?

Chrome DevTools MCP is an MCP server that lets AI coding assistants control an actual Chrome browser and use DevTools features such as network inspection, the console, screenshots, and performance traces. The Chrome team identified the problem that coding agents cannot see what actually happens in the browser, and introduced the DevTools MCP server as a way to fill that gap.

The official README also lists Puppeteer-based automation, including automatic waiting after actions, network analysis, console inspection, screenshots and snapshots, and performance tracing as core chrome-devtools-mcp features.

⸻

Installation and Setup (Claude Code)

The configuration below follows the format in the official ChromeDevTools/chrome-devtools-mcp README, rather than being merely illustrative text.

Register the MCP Server (JSON Configuration)

Many MCP clients register it under mcpServers as follows.

{
  "mcpServers": {
    "chrome-devtools": {
      "command": "npx",
      "args": ["-y", "chrome-devtools-mcp@latest"]
    }
  }
}

Using @latest always selects the latest version.

Add It Through the CLI (Claude Code)

With the Claude Code CLI, you can add it as follows.

claude mcp add chrome-devtools npx chrome-devtools-mcp@latest

⸻

Use Markdown Scenario Documents for Consistency

Chrome DevTools MCP is closer to a set of tools for an AI agent to manipulate a browser directly than to executing test code as Playwright does.

To reduce the variability of natural-language test execution, I found the following quite effective: • avoid ad hoc prompts; • template test scenarios as md files; • instruct Claude Code to follow the document exactly when executing them.

Here is an example of the md template I used.

An Example Markdown E2E Scenario Template

The point: limit the verbs to “Navigate / Fill / Click / Wait / Execute” to reduce room for AI interpretation, and fix the expected results as a checklist.

# Firebase Auth E2E Test
 
Firebase 인증 → Backend API 연동 테스트
 
## Preconditions
 
- kid-chat 서버: http://localhost:3001 (실행 중)
- backend-api 서버: http://localhost:8001 (실행 중)
- 테스트 계정: e2e-test / e2e-test
 
## Test Steps
 
### 1. kid-chat 로그인
 
1. Navigate to http://localhost:3001
2. Fill 이름 input: e2e-test
3. Fill 비밀번호 input: e2e-test
4. Click "로그인" button
5. Wait for chat interface to load
 
### 2. Firebase Token 획득
 
1. Execute in console: getToken()
2. Get token from console log (msgid with "Token:" prefix)
3. Save token for API tests
 
### 3. Backend API 인증 테스트
 
#### 3.1 /auth/me 엔드포인트
 
curl -H "Authorization: Bearer <TOKEN>" http://localhost:8001/auth/me
 
Expected Response:
{
  "uid": "z6SvYTUhVtQUE3mczn3nJ0tLZNv1",
  "email": "e2e-test@kidchat.local",
  "name": null
}
 
#### 3.2 /auth/verify 엔드포인트
 
curl -H "Authorization: Bearer <TOKEN>" http://localhost:8001/auth/verify
 
Expected Response:
{
  "valid": true,
  "uid": "z6SvYTUhVtQUE3mczn3nJ0tLZNv1"
}
 
#### 3.3 토큰 없이 요청 (실패 케이스)
 
curl http://localhost:8001/auth/me
 
Expected Response:
{"detail": "Not authenticated"}
 
## Expected Results
 
- kid-chat 로그인 성공
- Firebase ID Token 획득 성공
- /auth/me 정상 응답 (uid, email 포함)
- /auth/verify 정상 응답 (valid: true)
- 토큰 없는 요청 시 403 반환
 
## Cleanup
 
1. 브라우저에서 로그아웃 (optional)
2. 서버 종료 (optional)

How Does Chrome DevTools MCP Actually Control the Browser?

Chrome DevTools MCP generally follows this sequence.

1. Navigate to a page (navigate_page)
2. Capture a page snapshot (take_snapshot)
3. Use an element's uid from the snapshot to click or enter values (click, fill, fill_form, etc.)
4. Check the console, screenshots, and network (list_console_messages, take_screenshot, list_network_requests, etc.)

According to the official tool reference, click, for example, takes a uid obtained from a snapshot and clicks it, while fill enters a value into an input, textarea, or select.

Being able to avoid CSS-selector hell and interact with elements visible on the current screen through their uids was quite appealing.

Playwright vs. Chrome DevTools MCP: What to Use, and When?

Here is how I think about the distinction.

Playwright (Script-Based)

• Strengths: reproducibility, regression testing, and CI automation
• Weaknesses: initial script-writing and ongoing maintenance costs
• Playwright does offer codegen, its test generator, which generates test code as you click and type in the browser. That reduces the initial cost.

Chrome DevTools MCP (Natural Language + Tools)

• Strengths: good for debugging and verification while AI sees what is actually happening in a real browser, including DevTools areas such as the console, network, and performance traces
• Weaknesses: natural-language execution can vary → a scenario-document template therefore matters

Conclusion: Natural-Language E2E Is Flexible but Less Consistent—Templates Help

To summarize:

• Playwright remains close to the right choice for CI and repeated tests.
• Chrome DevTools MCP is particularly strong for debugging, exploratory testing, and verification in a real browser.
• Ultimately, I was able to control the weaknesses of natural-language testing with Markdown scenario templates.

As AI capabilities continue to improve, I can imagine flexible E2E testing centered on documented scenarios becoming an increasingly practical option instead of fixing every detail in a script.