25 KiB
name, description
| name | description |
|---|---|
| testing-e2e-testing | Test complete user workflows from UI to backend |
End-to-End Testing
When to Use This Skill
Use this skill when you need to:
- Test complete user workflows from UI to backend
- Verify cross-browser compatibility
- Implement Page Object Model (POM) for maintainable tests
- Handle waiting strategies (explicit vs implicit waits)
- Test complex user interactions (drag-drop, file uploads)
- Implement visual regression testing
- Debug flaky E2E tests
- Balance E2E coverage with speed and reliability
ACTIVATE THIS SKILL: When testing full user journeys, browser automation, or UI workflows
Core Concepts
E2E Testing Philosophy
E2E tests verify complete user workflows:
- Simulate real user behavior (clicks, typing, navigation)
- Test against real browser (Chrome, Firefox, Safari)
- Exercise full stack (UI, API, database)
- Highest confidence, slowest execution
- Focus on critical user paths
Testing Pyramid:
/\
/E2E\ <- Few, slow, high confidence
/-----\
/Integr\ <- More, medium speed
/--------\
| Unit | <- Many, fast, focused
----------
Playwright vs Cypress
Playwright (Recommended):
- Multi-browser (Chrome, Firefox, Safari, Edge)
- True parallelization
- Native mobile viewports
- Auto-wait for elements
- Network interception
- Multiple tabs/contexts
Cypress:
- Excellent DX (developer experience)
- Time-travel debugging
- Real-time reloads
- Limited to Chromium-based browsers
- Single tab limitation
Page Object Model (POM)
Encapsulate page structure and actions:
// ❌ WITHOUT POM: Brittle, duplicated selectors
test('user login', async ({ page }) => {
await page.goto('https://example.com/login');
await page.fill('#username', 'alice');
await page.fill('#password', 'secret');
await page.click('button[type="submit"]');
await expect(page.locator('.welcome-message')).toBeVisible();
});
// ✅ WITH POM: Reusable, maintainable
class LoginPage {
constructor(private page: Page) {}
async goto() {
await this.page.goto('https://example.com/login');
}
async login(username: string, password: string) {
await this.page.fill('#username', username);
await this.page.fill('#password', password);
await this.page.click('button[type="submit"]');
}
get welcomeMessage() {
return this.page.locator('.welcome-message');
}
}
test('user login', async ({ page }) => {
const loginPage = new LoginPage(page);
await loginPage.goto();
await loginPage.login('alice', 'secret');
await expect(loginPage.welcomeMessage).toBeVisible();
});
Patterns
Waiting Strategies
Auto-Wait (Playwright): Built-in smart waiting
// Playwright auto-waits for:
// 1. Element to be visible
// 2. Element to be enabled
// 3. Element to be stable (not animating)
await page.click('button'); // Waits automatically
await page.fill('input', 'text'); // Waits for enabled state
await expect(page.locator('.message')).toBeVisible(); // Waits up to timeout
Explicit Waits: Wait for specific conditions
// Wait for element
await page.waitForSelector('.dynamic-content', { state: 'visible' });
// Wait for network idle
await page.waitForLoadState('networkidle');
// Wait for custom condition
await page.waitForFunction(() => {
return document.querySelectorAll('.item').length > 5;
});
// Wait for response
const responsePromise = page.waitForResponse(
response => response.url().includes('/api/users') && response.status() === 200
);
await page.click('button');
await responsePromise;
Polling Pattern: Retry until condition met
async function waitForCondition(
condition: () => Promise<boolean>,
timeout = 5000
) {
const startTime = Date.now();
while (Date.now() - startTime < timeout) {
if (await condition()) return;
await new Promise(resolve => setTimeout(resolve, 100));
}
throw new Error('Condition not met within timeout');
}
await waitForCondition(async () => {
const count = await page.locator('.item').count();
return count > 5;
});
Page Object Model Patterns
Basic POM:
// pages/LoginPage.ts
export class LoginPage {
constructor(private page: Page) {}
// Selectors
private usernameInput = '#username';
private passwordInput = '#password';
private submitButton = 'button[type="submit"]';
private errorMessage = '.error-message';
// Actions
async goto() {
await this.page.goto('/login');
}
async login(username: string, password: string) {
await this.page.fill(this.usernameInput, username);
await this.page.fill(this.passwordInput, password);
await this.page.click(this.submitButton);
}
async loginAsAdmin() {
await this.login('admin', 'admin123');
}
// Assertions
async expectErrorMessage(message: string) {
await expect(this.page.locator(this.errorMessage)).toHaveText(message);
}
}
POM with Components:
// components/Header.ts
export class Header {
constructor(private page: Page) {}
async logout() {
await this.page.click('.user-menu');
await this.page.click('.logout-button');
}
async navigateTo(section: string) {
await this.page.click(`nav a:has-text("${section}")`);
}
}
// pages/DashboardPage.ts
export class DashboardPage {
readonly header: Header;
constructor(private page: Page) {
this.header = new Header(page);
}
async goto() {
await this.page.goto('/dashboard');
}
async getStatValue(statName: string): Promise<string> {
return await this.page.locator(`.stat[data-name="${statName}"] .value`).textContent();
}
}
// test
test('dashboard shows correct stats', async ({ page }) => {
const dashboard = new DashboardPage(page);
await dashboard.goto();
const userCount = await dashboard.getStatValue('users');
expect(userCount).toBe('1,234');
await dashboard.header.logout();
});
Network Mocking and Interception
// Mock API responses
test('displays users from API', async ({ page }) => {
// Intercept and mock API
await page.route('**/api/users', route => {
route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify([
{ id: 1, name: 'Alice' },
{ id: 2, name: 'Bob' },
]),
});
});
await page.goto('/users');
await expect(page.locator('.user-item')).toHaveCount(2);
await expect(page.locator('.user-item').first()).toContainText('Alice');
});
// Test error handling
test('handles API errors gracefully', async ({ page }) => {
await page.route('**/api/users', route => {
route.fulfill({ status: 500 });
});
await page.goto('/users');
await expect(page.locator('.error-message')).toBeVisible();
});
// Verify request payload
test('sends correct data to API', async ({ page }) => {
let requestBody: any;
await page.route('**/api/users', async route => {
requestBody = route.request().postDataJSON();
await route.fulfill({ status: 201, body: JSON.stringify({ id: 1 }) });
});
await page.goto('/users/new');
await page.fill('#name', 'Alice');
await page.click('button[type="submit"]');
expect(requestBody).toEqual({ name: 'Alice' });
});
Authentication Patterns
Reusable Auth State:
// global-setup.ts
async function globalSetup() {
const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('https://example.com/login');
await page.fill('#username', 'admin');
await page.fill('#password', 'admin123');
await page.click('button[type="submit"]');
// Save authenticated state
await page.context().storageState({ path: 'auth.json' });
await browser.close();
}
// playwright.config.ts
export default defineConfig({
globalSetup: require.resolve('./global-setup'),
use: {
storageState: 'auth.json', // Reuse auth across tests
},
});
// Tests start already authenticated
test('access protected page', async ({ page }) => {
await page.goto('/dashboard'); // Already logged in
await expect(page.locator('.welcome')).toBeVisible();
});
Auth Fixture:
// fixtures/auth.ts
export const test = base.extend({
authenticatedPage: async ({ page }, use) => {
await page.goto('/login');
await page.fill('#username', 'admin');
await page.fill('#password', 'admin123');
await page.click('button[type="submit"]');
await page.waitForURL('/dashboard');
await use(page);
},
});
// Use in tests
test('admin can access settings', async ({ authenticatedPage }) => {
await authenticatedPage.goto('/settings');
await expect(authenticatedPage.locator('h1')).toContainText('Settings');
});
Visual Regression Testing
// Playwright visual comparison
test('homepage visual regression', async ({ page }) => {
await page.goto('/');
await expect(page).toHaveScreenshot('homepage.png', {
fullPage: true,
maxDiffPixels: 100, // Allow small differences
});
});
// Component-level screenshot
test('button visual states', async ({ page }) => {
await page.goto('/components');
const button = page.locator('button.primary');
await expect(button).toHaveScreenshot('button-default.png');
await button.hover();
await expect(button).toHaveScreenshot('button-hover.png');
await button.focus();
await expect(button).toHaveScreenshot('button-focus.png');
});
// Cross-browser visual testing
test.describe('cross-browser visuals', () => {
test.use({ browserName: 'chromium' });
test('renders correctly in Chrome', async ({ page }) => {
await page.goto('/');
await expect(page).toHaveScreenshot('chrome-homepage.png');
});
test.use({ browserName: 'firefox' });
test('renders correctly in Firefox', async ({ page }) => {
await page.goto('/');
await expect(page).toHaveScreenshot('firefox-homepage.png');
});
});
Handling Complex Interactions
File Uploads:
test('upload profile picture', async ({ page }) => {
await page.goto('/profile');
const fileInput = page.locator('input[type="file"]');
await fileInput.setInputFiles('path/to/image.png');
await page.click('button:has-text("Upload")');
await expect(page.locator('.success-message')).toBeVisible();
});
Drag and Drop:
test('reorder items with drag and drop', async ({ page }) => {
await page.goto('/kanban');
const source = page.locator('.task[data-id="1"]');
const target = page.locator('.column[data-status="done"]');
await source.dragTo(target);
await expect(target.locator('.task[data-id="1"]')).toBeVisible();
});
Keyboard Navigation:
test('navigate menu with keyboard', async ({ page }) => {
await page.goto('/');
await page.keyboard.press('Tab'); // Focus first element
await page.keyboard.press('Enter'); // Activate
await page.keyboard.press('ArrowDown'); // Navigate menu
await page.keyboard.press('Enter'); // Select item
await expect(page).toHaveURL('/selected-page');
});
Examples by Framework
Playwright
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
testDir: './tests',
fullyParallel: true,
workers: process.env.CI ? 1 : undefined,
retries: process.env.CI ? 2 : 0,
use: {
baseURL: 'http://localhost:3000',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
},
projects: [
{ name: 'chromium', use: { browserName: 'chromium' } },
{ name: 'firefox', use: { browserName: 'firefox' } },
{ name: 'webkit', use: { browserName: 'webkit' } },
],
});
// tests/example.spec.ts
import { test, expect } from '@playwright/test';
test('complete user journey', async ({ page }) => {
// Navigate to homepage
await page.goto('/');
await expect(page.locator('h1')).toContainText('Welcome');
// Search for product
await page.fill('input[name="search"]', 'laptop');
await page.click('button[type="submit"]');
await page.waitForURL('**/search?q=laptop');
// Add to cart
await page.click('.product-card:first-child .add-to-cart');
await expect(page.locator('.cart-count')).toContainText('1');
// Checkout
await page.click('.cart-icon');
await page.click('button:has-text("Checkout")');
// Fill form
await page.fill('#email', '[email protected]');
await page.fill('#card', '4242424242424242');
await page.click('button:has-text("Pay")');
// Verify success
await expect(page.locator('.success-message')).toBeVisible();
await expect(page).toHaveURL('**/order-confirmation');
});
Cypress
// cypress/e2e/user-journey.cy.ts
describe('User Journey', () => {
beforeEach(() => {
cy.visit('/');
});
it('completes purchase flow', () => {
// Search
cy.get('input[name="search"]').type('laptop');
cy.get('button[type="submit"]').click();
// Add to cart
cy.get('.product-card').first().find('.add-to-cart').click();
cy.get('.cart-count').should('contain', '1');
// Checkout
cy.get('.cart-icon').click();
cy.contains('button', 'Checkout').click();
// Fill form
cy.get('#email').type('[email protected]');
cy.get('#card').type('4242424242424242');
cy.contains('button', 'Pay').click();
// Verify
cy.get('.success-message').should('be.visible');
cy.url().should('include', '/order-confirmation');
});
});
// Custom commands (cypress/support/commands.ts)
Cypress.Commands.add('login', (username: string, password: string) => {
cy.visit('/login');
cy.get('#username').type(username);
cy.get('#password').type(password);
cy.get('button[type="submit"]').click();
cy.url().should('include', '/dashboard');
});
// Use custom command
it('admin accesses settings', () => {
cy.login('admin', 'admin123');
cy.visit('/settings');
cy.contains('h1', 'Settings').should('be.visible');
});
Selenium (Python)
# test_user_journey.py
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import pytest
@pytest.fixture
def driver():
driver = webdriver.Chrome()
driver.implicitly_wait(10)
yield driver
driver.quit()
def test_user_login(driver):
driver.get("https://example.com/login")
# Fill form
driver.find_element(By.ID, "username").send_keys("alice")
driver.find_element(By.ID, "password").send_keys("secret")
driver.find_element(By.CSS_SELECTOR, "button[type='submit']").click()
# Wait for dashboard
WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, "welcome-message"))
)
assert "Dashboard" in driver.title
Checklist
Before Writing E2E Tests:
- Identify critical user paths to test
- Set up test environment (test database, API)
- Choose E2E framework (Playwright recommended)
- Plan Page Object Model structure
- Configure browser targets (Chrome, Firefox, Safari)
Writing Tests:
- Use Page Object Model for maintainability
- Implement proper waiting strategies (avoid sleep)
- Test realistic user workflows (multi-step)
- Mock external services when appropriate
- Handle authentication (fixtures or global setup)
- Add visual regression tests for critical pages
Reliability:
- Tests pass consistently (no flakiness)
- Use data-testid for stable selectors
- Avoid timing-dependent assertions
- Retry failed tests in CI (2-3 retries)
- Clean test data between runs
Performance:
- Run tests in parallel (multiple workers)
- Reuse authentication state
- Limit visual regression tests (slow)
- Keep test count focused (critical paths only)
After Writing Tests:
- Tests run in CI/CD pipeline
- Screenshots/videos on failure
- Clear failure messages
- Tests complete in reasonable time (< 10 min suite)
Anti-Patterns
❌ NEVER: Use sleep/wait for fixed time
→ Flaky tests, slow execution
❌ NEVER: Test everything with E2E
→ Slow feedback, expensive to maintain
❌ NEVER: Use fragile selectors (nth-child, absolute XPath)
→ Breaks with UI changes
❌ NEVER: Share state between tests
→ Order dependencies, flakiness
❌ NEVER: Skip Page Object Model
→ Duplicated selectors, hard to maintain
❌ NEVER: Test browser quirks with E2E
→ Use unit tests for logic, E2E for flows
❌ NEVER: Ignore flaky tests
→ Undermines trust in test suite
Related Skills
Foundation:
unit-testing-patterns.md- Testing individual componentsintegration-testing.md- API and database testing
E2E Specific:
visual-regression-testing.md- Advanced screenshot comparisoncross-browser-testing.md- Multi-browser strategies
Supporting:
test-driven-development.md- TDD workflowtest-coverage-strategy.md- What to test
Tools:
- Playwright: Modern, multi-browser, fast
- Cypress: Great DX, Chromium-only
- Selenium: Legacy, wide language support
- Puppeteer: Chrome-only, good for scraping
Level 3: Resources
Comprehensive Reference
REFERENCE.md (99KB) - Complete E2E testing guide:
- E2E Testing Philosophy and Strategy
- The Testing Pyramid and E2E Position
- Choosing an E2E Framework (Playwright vs Cypress vs Selenium)
- Page Object Model (POM) - Design patterns and architecture
- Selector Strategies (CSS, XPath, test IDs, accessibility)
- Waiting Strategies (explicit, implicit, auto-wait)
- Authentication and Session Management
- Test Data Management (fixtures, factories, seeding)
- Network Interception and Mocking
- Visual Regression Testing
- Cross-Browser Testing
- Handling Complex Interactions (drag-drop, file uploads, iframes)
- Test Organization and Structure
- Flakiness Reduction Techniques
- Parallelization and Performance
- CI/CD Integration
- Debugging Strategies
- Accessibility Testing
- Mobile and Responsive Testing
- Common Anti-Patterns
- Best Practices
- Real-World Test Suites
Coverage: Framework selection, POM architecture, selector strategies, waiting patterns, authentication, mocking, visual regression, cross-browser, flakiness reduction, CI/CD, debugging, accessibility, mobile testing, and production examples.
Automation Scripts
run_e2e_tests.py (600 lines) Execute E2E test suites with comprehensive orchestration and reporting:
# Run Playwright tests across browsers
./run_e2e_tests.py --framework playwright --browsers chromium firefox webkit
# Run with test environment
./run_e2e_tests.py --start-env --compose-file docker-compose.test.yml
# Parallel execution with retries
./run_e2e_tests.py --parallel --retries 2 --timeout 60
# Output JSON results
./run_e2e_tests.py --json
Features:
- Multi-framework support (Playwright, Cypress, Selenium)
- Browser matrix testing (chromium, firefox, webkit, edge)
- Parallel execution with configurable workers
- Retry logic for flaky tests
- Test environment orchestration (Docker Compose)
- Detailed reporting with pass rates
- Artifact collection (screenshots, videos, traces)
- JSON output for CI integration
analyze_flakiness.py (600 lines) Analyze E2E test stability and identify flaky tests:
# Analyze flakiness over 30 days
./analyze_flakiness.py --results-dir test-results --days 30
# Find tests with >10% failure rate
./analyze_flakiness.py --threshold 0.1 --min-runs 5
# Detailed analysis with recommendations
./analyze_flakiness.py --verbose
# Export as JSON
./analyze_flakiness.py --json
Features:
- Historical analysis across multiple test runs
- Flake rate calculation per test
- Root cause identification (timeouts, timing, network, state, race conditions)
- Severity classification (high, medium, low)
- Browser-specific flakiness detection
- Common error pattern recognition
- Actionable recommendations
- Trend analysis over time
generate_page_object.py (800 lines) Generate Page Object Model classes from HTML, URLs, or existing tests:
# Generate from URL
./generate_page_object.py --url https://example.com/login \
--framework playwright --language typescript
# Generate from HTML file
./generate_page_object.py --html page.html --framework cypress
# Generate from existing test
./generate_page_object.py --test tests/login.spec.ts --framework playwright
# Output as JSON structure
./generate_page_object.py --url https://example.com/dashboard --json
Features:
- Multi-framework support (Playwright, Cypress, Selenium)
- Multi-language output (TypeScript, Python, Java)
- HTML parsing and element extraction
- Test ID discovery (data-testid, data-test, data-cy)
- Action inference (login, search, form submission)
- Component composition
- Selector optimization
- Best practice patterns
Production Examples
Playwright Examples (resources/examples/playwright/):
- fixtures/ - Custom fixtures for authentication, test data
- pages/ - Page Object Model implementations
- tests/ - Complete test suites with real-world patterns
Cypress Examples (resources/examples/cypress/):
- Complete e-commerce checkout flow
- Custom commands and utilities
- API mocking strategies
- Visual regression tests
Selenium Examples (resources/examples/selenium/):
- Python pytest integration
- Page Object Model architecture
- Cross-browser test configuration
Configuration Examples (resources/examples/config/):
playwright.config.ts- Multi-browser, parallel, retry configcypress.config.ts- Video recording, screenshots, baseUrlpytest.ini- Selenium with pytest configuration
Docker Examples (resources/examples/docker/):
docker-compose.test.yml- Complete test environmentDockerfile.playwright- Playwright container imageDockerfile.cypress- Cypress container image
Usage Workflow
1. Choose Framework and Setup:
# Playwright (recommended)
npm install -D @playwright/test
npx playwright install
# Cypress
npm install -D cypress
# Selenium + pytest
pip install selenium pytest pytest-xdist
2. Generate Page Objects:
# Generate from your application
./generate_page_object.py --url http://localhost:3000/login \
--framework playwright --output-dir tests/pages/
3. Write Tests Using POM:
// tests/auth.spec.ts
import { test, expect } from '@playwright/test';
import { LoginPage } from './pages/LoginPage';
test('user can login', async ({ page }) => {
const loginPage = new LoginPage(page);
await loginPage.goto();
await loginPage.login('[email protected]', 'password123');
await expect(page).toHaveURL('/dashboard');
});
4. Run Tests:
# Local development
./run_e2e_tests.py --framework playwright --headed --verbose
# CI/CD pipeline
./run_e2e_tests.py --framework playwright \
--browsers chromium firefox webkit \
--parallel --retries 2 \
--start-env --compose-file docker-compose.test.yml \
--json > results.json
5. Analyze Flakiness:
# After accumulating test history
./analyze_flakiness.py --results-dir test-results \
--days 14 --threshold 0.15 --verbose
6. Iterate and Improve:
- Review flakiness report recommendations
- Update selectors to use test IDs
- Add explicit waits for dynamic content
- Mock flaky external dependencies
- Refactor highly flaky tests
Integration Points
CI/CD (GitHub Actions):
- name: Run E2E Tests
run: |
./run_e2e_tests.py \
--framework playwright \
--parallel \
--retries 2 \
--json > results.json
- name: Analyze Flakiness
if: always()
run: |
./analyze_flakiness.py --json > flakiness.json
- name: Upload Artifacts
uses: actions/upload-artifact@v3
with:
name: e2e-results
path: |
test-results/
results.json
flakiness.json
Local Development:
# Quick feedback loop
npm run dev & # Start app
./run_e2e_tests.py --headed --pattern "login*"
Quality Gates:
# Enforce flakiness threshold
if [ $(./analyze_flakiness.py --json | jq '.summary.overall_flake_rate') -gt 0.05 ]; then
echo "Flake rate >5%, failing build"
exit 1
fi
Best Practices from Resources
From REFERENCE.md:
- Follow the 10/20/70 rule (10% E2E, 20% integration, 70% unit)
- Use Page Object Model for all E2E tests
- Prefer data-testid selectors over CSS/XPath
- Implement auto-wait (Playwright) or explicit waits
- Mock external dependencies
- Run critical path tests first
- Parallelize independent tests
- Retry flaky tests (max 2-3 retries)
- Clean test data between runs
- Use visual regression sparingly
From Scripts:
- Automate test execution with run_e2e_tests.py
- Monitor flakiness continuously
- Generate Page Objects to reduce boilerplate
- Use JSON output for CI integration
- Collect artifacts (screenshots, videos) on failure
- Start test environment automatically
- Configure retries per test or suite
- Analyze trends over time
From Examples:
- Organize tests by feature/workflow
- Use fixtures for authentication
- Create reusable components
- Document test intent clearly
- Handle loading/error states
- Test keyboard navigation
- Verify accessibility
- Test responsive layouts