fix: ChatBubble crash and DeepSeek API compatibility
- Fix ChatBubble to handle non-string content with String() wrapper - Fix API route to use generateText for non-streaming requests - Add @ai-sdk/openai-compatible for non-OpenAI providers (DeepSeek, etc.) - Use Chat Completions API instead of Responses API for compatible providers - Update ChatBubble tests and fix component exports to kebab-case - Remove stale PascalCase ChatBubble.tsx file
This commit is contained in:
Max
321
_bmad/bmm/workflows/testarch/framework/checklist.md
Normal file
321
_bmad/bmm/workflows/testarch/framework/checklist.md
Normal file
@@ -0,0 +1,321 @@
|
||||
# Test Framework Setup - Validation Checklist
|
||||
|
||||
This checklist ensures the framework workflow completes successfully and all deliverables meet quality standards.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before starting the workflow:
|
||||
|
||||
- [ ] Project root contains valid `package.json`
|
||||
- [ ] No existing modern E2E framework detected (`playwright.config.*`, `cypress.config.*`)
|
||||
- [ ] Project type identifiable (React, Vue, Angular, Next.js, Node, etc.)
|
||||
- [ ] Bundler identifiable (Vite, Webpack, Rollup, esbuild) or not applicable
|
||||
- [ ] User has write permissions to create directories and files
|
||||
|
||||
---
|
||||
|
||||
## Process Steps
|
||||
|
||||
### Step 1: Preflight Checks
|
||||
|
||||
- [ ] package.json successfully read and parsed
|
||||
- [ ] Project type extracted correctly
|
||||
- [ ] Bundler identified (or marked as N/A for backend projects)
|
||||
- [ ] No framework conflicts detected
|
||||
- [ ] Architecture documents located (if available)
|
||||
|
||||
### Step 2: Framework Selection
|
||||
|
||||
- [ ] Framework auto-detection logic executed
|
||||
- [ ] Framework choice justified (Playwright vs Cypress)
|
||||
- [ ] Framework preference respected (if explicitly set)
|
||||
- [ ] User notified of framework selection and rationale
|
||||
|
||||
### Step 3: Directory Structure
|
||||
|
||||
- [ ] `tests/` root directory created
|
||||
- [ ] `tests/e2e/` directory created (or user's preferred structure)
|
||||
- [ ] `tests/support/` directory created (critical pattern)
|
||||
- [ ] `tests/support/fixtures/` directory created
|
||||
- [ ] `tests/support/fixtures/factories/` directory created
|
||||
- [ ] `tests/support/helpers/` directory created
|
||||
- [ ] `tests/support/page-objects/` directory created (if applicable)
|
||||
- [ ] All directories have correct permissions
|
||||
|
||||
**Note**: Test organization is flexible (e2e/, api/, integration/). The **support/** folder is the key pattern.
|
||||
|
||||
### Step 4: Configuration Files
|
||||
|
||||
- [ ] Framework config file created (`playwright.config.ts` or `cypress.config.ts`)
|
||||
- [ ] Config file uses TypeScript (if `use_typescript: true`)
|
||||
- [ ] Timeouts configured correctly (action: 15s, navigation: 30s, test: 60s)
|
||||
- [ ] Base URL configured with environment variable fallback
|
||||
- [ ] Trace/screenshot/video set to retain-on-failure
|
||||
- [ ] Multiple reporters configured (HTML + JUnit + console)
|
||||
- [ ] Parallel execution enabled
|
||||
- [ ] CI-specific settings configured (retries, workers)
|
||||
- [ ] Config file is syntactically valid (no compilation errors)
|
||||
|
||||
### Step 5: Environment Configuration
|
||||
|
||||
- [ ] `.env.example` created in project root
|
||||
- [ ] `TEST_ENV` variable defined
|
||||
- [ ] `BASE_URL` variable defined with default
|
||||
- [ ] `API_URL` variable defined (if applicable)
|
||||
- [ ] Authentication variables defined (if applicable)
|
||||
- [ ] Feature flag variables defined (if applicable)
|
||||
- [ ] `.nvmrc` created with appropriate Node version
|
||||
|
||||
### Step 6: Fixture Architecture
|
||||
|
||||
- [ ] `tests/support/fixtures/index.ts` created
|
||||
- [ ] Base fixture extended from Playwright/Cypress
|
||||
- [ ] Type definitions for fixtures created
|
||||
- [ ] mergeTests pattern implemented (if multiple fixtures)
|
||||
- [ ] Auto-cleanup logic included in fixtures
|
||||
- [ ] Fixture architecture follows knowledge base patterns
|
||||
|
||||
### Step 7: Data Factories
|
||||
|
||||
- [ ] At least one factory created (e.g., UserFactory)
|
||||
- [ ] Factories use @faker-js/faker for realistic data
|
||||
- [ ] Factories track created entities (for cleanup)
|
||||
- [ ] Factories implement `cleanup()` method
|
||||
- [ ] Factories integrate with fixtures
|
||||
- [ ] Factories follow knowledge base patterns
|
||||
|
||||
### Step 8: Sample Tests
|
||||
|
||||
- [ ] Example test file created (`tests/e2e/example.spec.ts`)
|
||||
- [ ] Test uses fixture architecture
|
||||
- [ ] Test demonstrates data factory usage
|
||||
- [ ] Test uses proper selector strategy (data-testid)
|
||||
- [ ] Test follows Given-When-Then structure
|
||||
- [ ] Test includes proper assertions
|
||||
- [ ] Network interception demonstrated (if applicable)
|
||||
|
||||
### Step 9: Helper Utilities
|
||||
|
||||
- [ ] API helper created (if API testing needed)
|
||||
- [ ] Network helper created (if network mocking needed)
|
||||
- [ ] Auth helper created (if authentication needed)
|
||||
- [ ] Helpers follow functional patterns
|
||||
- [ ] Helpers have proper error handling
|
||||
|
||||
### Step 10: Documentation
|
||||
|
||||
- [ ] `tests/README.md` created
|
||||
- [ ] Setup instructions included
|
||||
- [ ] Running tests section included
|
||||
- [ ] Architecture overview section included
|
||||
- [ ] Best practices section included
|
||||
- [ ] CI integration section included
|
||||
- [ ] Knowledge base references included
|
||||
- [ ] Troubleshooting section included
|
||||
|
||||
### Step 11: Package.json Updates
|
||||
|
||||
- [ ] Minimal test script added to package.json: `test:e2e`
|
||||
- [ ] Test framework dependency added (if not already present)
|
||||
- [ ] Type definitions added (if TypeScript)
|
||||
- [ ] Users can extend with additional scripts as needed
|
||||
|
||||
---
|
||||
|
||||
## Output Validation
|
||||
|
||||
### Configuration Validation
|
||||
|
||||
- [ ] Config file loads without errors
|
||||
- [ ] Config file passes linting (if linter configured)
|
||||
- [ ] Config file uses correct syntax for chosen framework
|
||||
- [ ] All paths in config resolve correctly
|
||||
- [ ] Reporter output directories exist or are created on test run
|
||||
|
||||
### Test Execution Validation
|
||||
|
||||
- [ ] Sample test runs successfully
|
||||
- [ ] Test execution produces expected output (pass/fail)
|
||||
- [ ] Test artifacts generated correctly (traces, screenshots, videos)
|
||||
- [ ] Test report generated successfully
|
||||
- [ ] No console errors or warnings during test run
|
||||
|
||||
### Directory Structure Validation
|
||||
|
||||
- [ ] All required directories exist
|
||||
- [ ] Directory structure matches framework conventions
|
||||
- [ ] No duplicate or conflicting directories
|
||||
- [ ] Directories accessible with correct permissions
|
||||
|
||||
### File Integrity Validation
|
||||
|
||||
- [ ] All generated files are syntactically correct
|
||||
- [ ] No placeholder text left in files (e.g., "TODO", "FIXME")
|
||||
- [ ] All imports resolve correctly
|
||||
- [ ] No hardcoded credentials or secrets in files
|
||||
- [ ] All file paths use correct separators for OS
|
||||
|
||||
---
|
||||
|
||||
## Quality Checks
|
||||
|
||||
### Code Quality
|
||||
|
||||
- [ ] Generated code follows project coding standards
|
||||
- [ ] TypeScript types are complete and accurate (no `any` unless necessary)
|
||||
- [ ] No unused imports or variables
|
||||
- [ ] Consistent code formatting (matches project style)
|
||||
- [ ] No linting errors in generated files
|
||||
|
||||
### Best Practices Compliance
|
||||
|
||||
- [ ] Fixture architecture follows pure function → fixture → mergeTests pattern
|
||||
- [ ] Data factories implement auto-cleanup
|
||||
- [ ] Network interception occurs before navigation
|
||||
- [ ] Selectors use data-testid strategy
|
||||
- [ ] Artifacts only captured on failure
|
||||
- [ ] Tests follow Given-When-Then structure
|
||||
- [ ] No hard-coded waits or sleeps
|
||||
|
||||
### Knowledge Base Alignment
|
||||
|
||||
- [ ] Fixture pattern matches `fixture-architecture.md`
|
||||
- [ ] Data factories match `data-factories.md`
|
||||
- [ ] Network handling matches `network-first.md`
|
||||
- [ ] Config follows `playwright-config.md` or `test-config.md`
|
||||
- [ ] Test quality matches `test-quality.md`
|
||||
|
||||
### Security Checks
|
||||
|
||||
- [ ] No credentials in configuration files
|
||||
- [ ] .env.example contains placeholders, not real values
|
||||
- [ ] Sensitive test data handled securely
|
||||
- [ ] API keys and tokens use environment variables
|
||||
- [ ] No secrets committed to version control
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### Status File Integration
|
||||
|
||||
- [ ] `bmm-workflow-status.md` exists
|
||||
- [ ] Framework initialization logged in Quality & Testing Progress section
|
||||
- [ ] Status file updated with completion timestamp
|
||||
- [ ] Status file shows framework: Playwright or Cypress
|
||||
|
||||
### Knowledge Base Integration
|
||||
|
||||
- [ ] Relevant knowledge fragments identified from tea-index.csv
|
||||
- [ ] Knowledge fragments successfully loaded
|
||||
- [ ] Patterns from knowledge base applied correctly
|
||||
- [ ] Knowledge base references included in documentation
|
||||
|
||||
### Workflow Dependencies
|
||||
|
||||
- [ ] Can proceed to `ci` workflow after completion
|
||||
- [ ] Can proceed to `test-design` workflow after completion
|
||||
- [ ] Can proceed to `atdd` workflow after completion
|
||||
- [ ] Framework setup compatible with downstream workflows
|
||||
|
||||
---
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
**All of the following must be true:**
|
||||
|
||||
- [ ] All prerequisite checks passed
|
||||
- [ ] All process steps completed without errors
|
||||
- [ ] All output validations passed
|
||||
- [ ] All quality checks passed
|
||||
- [ ] All integration points verified
|
||||
- [ ] Sample test executes successfully
|
||||
- [ ] User can run `npm run test:e2e` without errors
|
||||
- [ ] Documentation is complete and accurate
|
||||
- [ ] No critical issues or blockers identified
|
||||
|
||||
---
|
||||
|
||||
## Post-Workflow Actions
|
||||
|
||||
**User must complete:**
|
||||
|
||||
1. [ ] Copy `.env.example` to `.env`
|
||||
2. [ ] Fill in environment-specific values in `.env`
|
||||
3. [ ] Run `npm install` to install test dependencies
|
||||
4. [ ] Run `npm run test:e2e` to verify setup
|
||||
5. [ ] Review `tests/README.md` for project-specific guidance
|
||||
|
||||
**Recommended next workflows:**
|
||||
|
||||
1. [ ] Run `ci` workflow to set up CI/CD pipeline
|
||||
2. [ ] Run `test-design` workflow to plan test coverage
|
||||
3. [ ] Run `atdd` workflow when ready to develop stories
|
||||
|
||||
---
|
||||
|
||||
## Rollback Procedure
|
||||
|
||||
If workflow fails and needs to be rolled back:
|
||||
|
||||
1. [ ] Delete `tests/` directory
|
||||
2. [ ] Remove test scripts from package.json
|
||||
3. [ ] Delete `.env.example` (if created)
|
||||
4. [ ] Delete `.nvmrc` (if created)
|
||||
5. [ ] Delete framework config file
|
||||
6. [ ] Remove test dependencies from package.json (if added)
|
||||
7. [ ] Run `npm install` to clean up node_modules
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Issue**: Config file has TypeScript errors
|
||||
|
||||
- **Solution**: Ensure `@playwright/test` or `cypress` types are installed
|
||||
|
||||
**Issue**: Sample test fails to run
|
||||
|
||||
- **Solution**: Check BASE_URL in .env, ensure app is running
|
||||
|
||||
**Issue**: Fixture cleanup not working
|
||||
|
||||
- **Solution**: Verify cleanup() is called in fixture teardown
|
||||
|
||||
**Issue**: Network interception not working
|
||||
|
||||
- **Solution**: Ensure route setup occurs before page.goto()
|
||||
|
||||
### Framework-Specific Considerations
|
||||
|
||||
**Playwright:**
|
||||
|
||||
- Requires Node.js 18+
|
||||
- Browser binaries auto-installed on first run
|
||||
- Trace viewer requires running `npx playwright show-trace`
|
||||
|
||||
**Cypress:**
|
||||
|
||||
- Requires Node.js 18+
|
||||
- Cypress app opens on first run
|
||||
- Component testing requires additional setup
|
||||
|
||||
### Version Compatibility
|
||||
|
||||
- [ ] Node.js version matches .nvmrc
|
||||
- [ ] Framework version compatible with Node.js version
|
||||
- [ ] TypeScript version compatible with framework
|
||||
- [ ] All peer dependencies satisfied
|
||||
|
||||
---
|
||||
|
||||
**Checklist Complete**: Sign off when all items checked and validated.
|
||||
|
||||
**Completed by:** {name}
|
||||
**Date:** {date}
|
||||
**Framework:** { Playwright / Cypress or something else}
|
||||
**Notes:** {notes}
|
||||
481
_bmad/bmm/workflows/testarch/framework/instructions.md
Normal file
481
_bmad/bmm/workflows/testarch/framework/instructions.md
Normal file
@@ -0,0 +1,481 @@
|
||||
<!-- Powered by BMAD-CORE™ -->
|
||||
|
||||
# Test Framework Setup
|
||||
|
||||
**Workflow ID**: `_bmad/bmm/testarch/framework`
|
||||
**Version**: 4.0 (BMad v6)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Initialize a production-ready test framework architecture (Playwright or Cypress) with fixtures, helpers, configuration, and best practices. This workflow scaffolds the complete testing infrastructure for modern web applications.
|
||||
|
||||
---
|
||||
|
||||
## Preflight Requirements
|
||||
|
||||
**Critical:** Verify these requirements before proceeding. If any fail, HALT and notify the user.
|
||||
|
||||
- ✅ `package.json` exists in project root
|
||||
- ✅ No modern E2E test harness is already configured (check for existing `playwright.config.*` or `cypress.config.*`)
|
||||
- ✅ Architectural/stack context available (project type, bundler, dependencies)
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Run Preflight Checks
|
||||
|
||||
### Actions
|
||||
|
||||
1. **Validate package.json**
|
||||
- Read `{project-root}/package.json`
|
||||
- Extract project type (React, Vue, Angular, Next.js, Node, etc.)
|
||||
- Identify bundler (Vite, Webpack, Rollup, esbuild)
|
||||
- Note existing test dependencies
|
||||
|
||||
2. **Check for Existing Framework**
|
||||
- Search for `playwright.config.*`, `cypress.config.*`, `cypress.json`
|
||||
- Check `package.json` for `@playwright/test` or `cypress` dependencies
|
||||
- If found, HALT with message: "Existing test framework detected. Use workflow `upgrade-framework` instead."
|
||||
|
||||
3. **Gather Context**
|
||||
- Look for architecture documents (`architecture.md`, `tech-spec*.md`)
|
||||
- Check for API documentation or endpoint lists
|
||||
- Identify authentication requirements
|
||||
|
||||
**Halt Condition:** If preflight checks fail, stop immediately and report which requirement failed.
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Scaffold Framework
|
||||
|
||||
### Actions
|
||||
|
||||
1. **Framework Selection**
|
||||
|
||||
**Default Logic:**
|
||||
- **Playwright** (recommended for):
|
||||
- Large repositories (100+ files)
|
||||
- Performance-critical applications
|
||||
- Multi-browser support needed
|
||||
- Complex user flows requiring video/trace debugging
|
||||
- Projects requiring worker parallelism
|
||||
|
||||
- **Cypress** (recommended for):
|
||||
- Small teams prioritizing developer experience
|
||||
- Component testing focus
|
||||
- Real-time reloading during test development
|
||||
- Simpler setup requirements
|
||||
|
||||
**Detection Strategy:**
|
||||
- Check `package.json` for existing preference
|
||||
- Consider `project_size` variable from workflow config
|
||||
- Use `framework_preference` variable if set
|
||||
- Default to **Playwright** if uncertain
|
||||
|
||||
2. **Create Directory Structure**
|
||||
|
||||
```
|
||||
{project-root}/
|
||||
├── tests/ # Root test directory
|
||||
│ ├── e2e/ # Test files (users organize as needed)
|
||||
│ ├── support/ # Framework infrastructure (key pattern)
|
||||
│ │ ├── fixtures/ # Test fixtures (data, mocks)
|
||||
│ │ ├── helpers/ # Utility functions
|
||||
│ │ └── page-objects/ # Page object models (optional)
|
||||
│ └── README.md # Test suite documentation
|
||||
```
|
||||
|
||||
**Note**: Users organize test files (e2e/, api/, integration/, component/) as needed. The **support/** folder is the critical pattern for fixtures and helpers used across tests.
|
||||
|
||||
3. **Generate Configuration File**
|
||||
|
||||
**For Playwright** (`playwright.config.ts` or `playwright.config.js`):
|
||||
|
||||
```typescript
|
||||
import { defineConfig, devices } from '@playwright/test';
|
||||
|
||||
export default defineConfig({
|
||||
testDir: './tests/e2e',
|
||||
fullyParallel: true,
|
||||
forbidOnly: !!process.env.CI,
|
||||
retries: process.env.CI ? 2 : 0,
|
||||
workers: process.env.CI ? 1 : undefined,
|
||||
|
||||
timeout: 60 * 1000, // Test timeout: 60s
|
||||
expect: {
|
||||
timeout: 15 * 1000, // Assertion timeout: 15s
|
||||
},
|
||||
|
||||
use: {
|
||||
baseURL: process.env.BASE_URL || 'http://localhost:3000',
|
||||
trace: 'retain-on-failure',
|
||||
screenshot: 'only-on-failure',
|
||||
video: 'retain-on-failure',
|
||||
actionTimeout: 15 * 1000, // Action timeout: 15s
|
||||
navigationTimeout: 30 * 1000, // Navigation timeout: 30s
|
||||
},
|
||||
|
||||
reporter: [['html', { outputFolder: 'test-results/html' }], ['junit', { outputFile: 'test-results/junit.xml' }], ['list']],
|
||||
|
||||
projects: [
|
||||
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
|
||||
{ name: 'firefox', use: { ...devices['Desktop Firefox'] } },
|
||||
{ name: 'webkit', use: { ...devices['Desktop Safari'] } },
|
||||
],
|
||||
});
|
||||
```
|
||||
|
||||
**For Cypress** (`cypress.config.ts` or `cypress.config.js`):
|
||||
|
||||
```typescript
|
||||
import { defineConfig } from 'cypress';
|
||||
|
||||
export default defineConfig({
|
||||
e2e: {
|
||||
baseUrl: process.env.BASE_URL || 'http://localhost:3000',
|
||||
specPattern: 'tests/e2e/**/*.cy.{js,jsx,ts,tsx}',
|
||||
supportFile: 'tests/support/e2e.ts',
|
||||
video: false,
|
||||
screenshotOnRunFailure: true,
|
||||
|
||||
setupNodeEvents(on, config) {
|
||||
// implement node event listeners here
|
||||
},
|
||||
},
|
||||
|
||||
retries: {
|
||||
runMode: 2,
|
||||
openMode: 0,
|
||||
},
|
||||
|
||||
defaultCommandTimeout: 15000,
|
||||
requestTimeout: 30000,
|
||||
responseTimeout: 30000,
|
||||
pageLoadTimeout: 60000,
|
||||
});
|
||||
```
|
||||
|
||||
4. **Generate Environment Configuration**
|
||||
|
||||
Create `.env.example`:
|
||||
|
||||
```bash
|
||||
# Test Environment Configuration
|
||||
TEST_ENV=local
|
||||
BASE_URL=http://localhost:3000
|
||||
API_URL=http://localhost:3001/api
|
||||
|
||||
# Authentication (if applicable)
|
||||
TEST_USER_EMAIL=test@example.com
|
||||
TEST_USER_PASSWORD=
|
||||
|
||||
# Feature Flags (if applicable)
|
||||
FEATURE_FLAG_NEW_UI=true
|
||||
|
||||
# API Keys (if applicable)
|
||||
TEST_API_KEY=
|
||||
```
|
||||
|
||||
5. **Generate Node Version File**
|
||||
|
||||
Create `.nvmrc`:
|
||||
|
||||
```
|
||||
20.11.0
|
||||
```
|
||||
|
||||
(Use Node version from existing `.nvmrc` or default to current LTS)
|
||||
|
||||
6. **Implement Fixture Architecture**
|
||||
|
||||
**Knowledge Base Reference**: `testarch/knowledge/fixture-architecture.md`
|
||||
|
||||
Create `tests/support/fixtures/index.ts`:
|
||||
|
||||
```typescript
|
||||
import { test as base } from '@playwright/test';
|
||||
import { UserFactory } from './factories/user-factory';
|
||||
|
||||
type TestFixtures = {
|
||||
userFactory: UserFactory;
|
||||
};
|
||||
|
||||
export const test = base.extend<TestFixtures>({
|
||||
userFactory: async ({}, use) => {
|
||||
const factory = new UserFactory();
|
||||
await use(factory);
|
||||
await factory.cleanup(); // Auto-cleanup
|
||||
},
|
||||
});
|
||||
|
||||
export { expect } from '@playwright/test';
|
||||
```
|
||||
|
||||
7. **Implement Data Factories**
|
||||
|
||||
**Knowledge Base Reference**: `testarch/knowledge/data-factories.md`
|
||||
|
||||
Create `tests/support/fixtures/factories/user-factory.ts`:
|
||||
|
||||
```typescript
|
||||
import { faker } from '@faker-js/faker';
|
||||
|
||||
export class UserFactory {
|
||||
private createdUsers: string[] = [];
|
||||
|
||||
async createUser(overrides = {}) {
|
||||
const user = {
|
||||
email: faker.internet.email(),
|
||||
name: faker.person.fullName(),
|
||||
password: faker.internet.password({ length: 12 }),
|
||||
...overrides,
|
||||
};
|
||||
|
||||
// API call to create user
|
||||
const response = await fetch(`${process.env.API_URL}/users`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(user),
|
||||
});
|
||||
|
||||
const created = await response.json();
|
||||
this.createdUsers.push(created.id);
|
||||
return created;
|
||||
}
|
||||
|
||||
async cleanup() {
|
||||
// Delete all created users
|
||||
for (const userId of this.createdUsers) {
|
||||
await fetch(`${process.env.API_URL}/users/${userId}`, {
|
||||
method: 'DELETE',
|
||||
});
|
||||
}
|
||||
this.createdUsers = [];
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
8. **Generate Sample Tests**
|
||||
|
||||
Create `tests/e2e/example.spec.ts`:
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '../support/fixtures';
|
||||
|
||||
test.describe('Example Test Suite', () => {
|
||||
test('should load homepage', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
await expect(page).toHaveTitle(/Home/i);
|
||||
});
|
||||
|
||||
test('should create user and login', async ({ page, userFactory }) => {
|
||||
// Create test user
|
||||
const user = await userFactory.createUser();
|
||||
|
||||
// Login
|
||||
await page.goto('/login');
|
||||
await page.fill('[data-testid="email-input"]', user.email);
|
||||
await page.fill('[data-testid="password-input"]', user.password);
|
||||
await page.click('[data-testid="login-button"]');
|
||||
|
||||
// Assert login success
|
||||
await expect(page.locator('[data-testid="user-menu"]')).toBeVisible();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
9. **Update package.json Scripts**
|
||||
|
||||
Add minimal test script to `package.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"scripts": {
|
||||
"test:e2e": "playwright test"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Note**: Users can add additional scripts as needed (e.g., `--ui`, `--headed`, `--debug`, `show-report`).
|
||||
|
||||
10. **Generate Documentation**
|
||||
|
||||
Create `tests/README.md` with setup instructions (see Step 3 deliverables).
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Deliverables
|
||||
|
||||
### Primary Artifacts Created
|
||||
|
||||
1. **Configuration File**
|
||||
- `playwright.config.ts` or `cypress.config.ts`
|
||||
- Timeouts: action 15s, navigation 30s, test 60s
|
||||
- Reporters: HTML + JUnit XML
|
||||
|
||||
2. **Directory Structure**
|
||||
- `tests/` with `e2e/`, `api/`, `support/` subdirectories
|
||||
- `support/fixtures/` for test fixtures
|
||||
- `support/helpers/` for utility functions
|
||||
|
||||
3. **Environment Configuration**
|
||||
- `.env.example` with `TEST_ENV`, `BASE_URL`, `API_URL`
|
||||
- `.nvmrc` with Node version
|
||||
|
||||
4. **Test Infrastructure**
|
||||
- Fixture architecture (`mergeTests` pattern)
|
||||
- Data factories (faker-based, with auto-cleanup)
|
||||
- Sample tests demonstrating patterns
|
||||
|
||||
5. **Documentation**
|
||||
- `tests/README.md` with setup instructions
|
||||
- Comments in config files explaining options
|
||||
|
||||
### README Contents
|
||||
|
||||
The generated `tests/README.md` should include:
|
||||
|
||||
- **Setup Instructions**: How to install dependencies, configure environment
|
||||
- **Running Tests**: Commands for local execution, headed mode, debug mode
|
||||
- **Architecture Overview**: Fixture pattern, data factories, page objects
|
||||
- **Best Practices**: Selector strategy (data-testid), test isolation, cleanup
|
||||
- **CI Integration**: How tests run in CI/CD pipeline
|
||||
- **Knowledge Base References**: Links to relevant TEA knowledge fragments
|
||||
|
||||
---
|
||||
|
||||
## Important Notes
|
||||
|
||||
### Knowledge Base Integration
|
||||
|
||||
**Critical:** Check configuration and load appropriate fragments.
|
||||
|
||||
Read `{config_source}` and check `config.tea_use_playwright_utils`.
|
||||
|
||||
**If `config.tea_use_playwright_utils: true` (Playwright Utils Integration):**
|
||||
|
||||
Consult `{project-root}/_bmad/bmm/testarch/tea-index.csv` and load:
|
||||
|
||||
- `overview.md` - Playwright utils installation and design principles
|
||||
- `fixtures-composition.md` - mergeTests composition with playwright-utils
|
||||
- `auth-session.md` - Token persistence setup (if auth needed)
|
||||
- `api-request.md` - API testing utilities (if API tests planned)
|
||||
- `burn-in.md` - Smart test selection for CI (recommend during framework setup)
|
||||
- `network-error-monitor.md` - Automatic HTTP error detection (recommend in merged fixtures)
|
||||
- `data-factories.md` - Factory patterns with faker (498 lines, 5 examples)
|
||||
|
||||
Recommend installing playwright-utils:
|
||||
|
||||
```bash
|
||||
npm install -D @seontechnologies/playwright-utils
|
||||
```
|
||||
|
||||
Recommend adding burn-in and network-error-monitor to merged fixtures for enhanced reliability.
|
||||
|
||||
**If `config.tea_use_playwright_utils: false` (Traditional Patterns):**
|
||||
|
||||
Consult `{project-root}/_bmad/bmm/testarch/tea-index.csv` and load:
|
||||
|
||||
- `fixture-architecture.md` - Pure function → fixture → `mergeTests` composition with auto-cleanup (406 lines, 5 examples)
|
||||
- `data-factories.md` - Faker-based factories with overrides, nested factories, API seeding, auto-cleanup (498 lines, 5 examples)
|
||||
- `network-first.md` - Network-first testing safeguards: intercept before navigate, HAR capture, deterministic waiting (489 lines, 5 examples)
|
||||
- `playwright-config.md` - Playwright-specific configuration: environment-based, timeout standards, artifact output, parallelization, project config (722 lines, 5 examples)
|
||||
- `test-quality.md` - Test design principles: deterministic, isolated with cleanup, explicit assertions, length/time limits (658 lines, 5 examples)
|
||||
|
||||
### Framework-Specific Guidance
|
||||
|
||||
**Playwright Advantages:**
|
||||
|
||||
- Worker parallelism (significantly faster for large suites)
|
||||
- Trace viewer (powerful debugging with screenshots, network, console)
|
||||
- Multi-language support (TypeScript, JavaScript, Python, C#, Java)
|
||||
- Built-in API testing capabilities
|
||||
- Better handling of multiple browser contexts
|
||||
|
||||
**Cypress Advantages:**
|
||||
|
||||
- Superior developer experience (real-time reloading)
|
||||
- Excellent for component testing (Cypress CT or use Vitest)
|
||||
- Simpler setup for small teams
|
||||
- Better suited for watch mode during development
|
||||
|
||||
**Avoid Cypress when:**
|
||||
|
||||
- API chains are heavy and complex
|
||||
- Multi-tab/window scenarios are common
|
||||
- Worker parallelism is critical for CI performance
|
||||
|
||||
### Selector Strategy
|
||||
|
||||
**Always recommend**:
|
||||
|
||||
- `data-testid` attributes for UI elements
|
||||
- `data-cy` attributes if Cypress is chosen
|
||||
- Avoid brittle CSS selectors or XPath
|
||||
|
||||
### Contract Testing
|
||||
|
||||
For microservices architectures, **recommend Pact** for consumer-driven contract testing alongside E2E tests.
|
||||
|
||||
### Failure Artifacts
|
||||
|
||||
Configure **failure-only** capture:
|
||||
|
||||
- Screenshots: only on failure
|
||||
- Videos: retain on failure (delete on success)
|
||||
- Traces: retain on failure (Playwright)
|
||||
|
||||
This reduces storage overhead while maintaining debugging capability.
|
||||
|
||||
---
|
||||
|
||||
## Output Summary
|
||||
|
||||
After completing this workflow, provide a summary:
|
||||
|
||||
```markdown
|
||||
## Framework Scaffold Complete
|
||||
|
||||
**Framework Selected**: Playwright (or Cypress)
|
||||
|
||||
**Artifacts Created**:
|
||||
|
||||
- ✅ Configuration file: `playwright.config.ts`
|
||||
- ✅ Directory structure: `tests/e2e/`, `tests/support/`
|
||||
- ✅ Environment config: `.env.example`
|
||||
- ✅ Node version: `.nvmrc`
|
||||
- ✅ Fixture architecture: `tests/support/fixtures/`
|
||||
- ✅ Data factories: `tests/support/fixtures/factories/`
|
||||
- ✅ Sample tests: `tests/e2e/example.spec.ts`
|
||||
- ✅ Documentation: `tests/README.md`
|
||||
|
||||
**Next Steps**:
|
||||
|
||||
1. Copy `.env.example` to `.env` and fill in environment variables
|
||||
2. Run `npm install` to install test dependencies
|
||||
3. Run `npm run test:e2e` to execute sample tests
|
||||
4. Review `tests/README.md` for detailed setup instructions
|
||||
|
||||
**Knowledge Base References Applied**:
|
||||
|
||||
- Fixture architecture pattern (pure functions + mergeTests)
|
||||
- Data factories with auto-cleanup (faker-based)
|
||||
- Network-first testing safeguards
|
||||
- Failure-only artifact capture
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Validation
|
||||
|
||||
After completing all steps, verify:
|
||||
|
||||
- [ ] Configuration file created and valid
|
||||
- [ ] Directory structure exists
|
||||
- [ ] Environment configuration generated
|
||||
- [ ] Sample tests run successfully
|
||||
- [ ] Documentation complete and accurate
|
||||
- [ ] No errors or warnings during scaffold
|
||||
|
||||
Refer to `checklist.md` for comprehensive validation criteria.
|
||||
47
_bmad/bmm/workflows/testarch/framework/workflow.yaml
Normal file
47
_bmad/bmm/workflows/testarch/framework/workflow.yaml
Normal file
@@ -0,0 +1,47 @@
|
||||
# Test Architect workflow: framework
|
||||
name: testarch-framework
|
||||
description: "Initialize production-ready test framework architecture (Playwright or Cypress) with fixtures, helpers, and configuration"
|
||||
author: "BMad"
|
||||
|
||||
# Critical variables from config
|
||||
config_source: "{project-root}/_bmad/bmm/config.yaml"
|
||||
output_folder: "{config_source}:output_folder"
|
||||
user_name: "{config_source}:user_name"
|
||||
communication_language: "{config_source}:communication_language"
|
||||
document_output_language: "{config_source}:document_output_language"
|
||||
date: system-generated
|
||||
|
||||
# Workflow components
|
||||
installed_path: "{project-root}/_bmad/bmm/workflows/testarch/framework"
|
||||
instructions: "{installed_path}/instructions.md"
|
||||
validation: "{installed_path}/checklist.md"
|
||||
|
||||
# Variables and inputs
|
||||
variables:
|
||||
test_dir: "{project-root}/tests" # Root test directory
|
||||
use_typescript: true # Prefer TypeScript configuration
|
||||
framework_preference: "auto" # auto, playwright, cypress - user can override auto-detection
|
||||
project_size: "auto" # auto, small, large - influences framework recommendation
|
||||
|
||||
# Output configuration
|
||||
default_output_file: "{test_dir}/README.md" # Main deliverable is test setup README
|
||||
|
||||
# Required tools
|
||||
required_tools:
|
||||
- read_file # Read package.json, existing configs
|
||||
- write_file # Create config files, helpers, fixtures, tests
|
||||
- create_directory # Create test directory structure
|
||||
- list_files # Check for existing framework
|
||||
- search_repo # Find architecture docs
|
||||
|
||||
tags:
|
||||
- qa
|
||||
- setup
|
||||
- test-architect
|
||||
- framework
|
||||
- initialization
|
||||
|
||||
execution_hints:
|
||||
interactive: false # Minimize prompts; auto-detect when possible
|
||||
autonomous: true # Proceed without user input unless blocked
|
||||
iterative: true
|
||||
Reference in New Issue
Block a user