Initial commit: Brachnha Insight project setup
- Next.js 14+ with App Router and TypeScript - Tailwind CSS and ShadCN UI styling - Zustand state management - Dexie.js for IndexedDB (local-first data) - Auth.js v5 for authentication - BMAD framework integration Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
117
_bmad-output/test-design-epic-4.md
Normal file
117
_bmad-output/test-design-epic-4.md
Normal file
@@ -0,0 +1,117 @@
|
||||
# Test Design: Epic 4 - Power User Settings - BYOD & Configuration
|
||||
|
||||
**Date:** 2026-01-24
|
||||
**Author:** Max
|
||||
**Status:** **Approved** (Verification Failed)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
**Scope:** Epic-Level test design for Epic 4 (BYOD & Configuration). Focus on security of API keys and reliability of provider connections.
|
||||
|
||||
**Risk Summary:**
|
||||
|
||||
- Total risks identified: 4
|
||||
- High-priority risks (≥6): 2
|
||||
- Critical categories: SEC, TECH
|
||||
|
||||
**Verification Status:**
|
||||
- **Exploratory Validation (P0):** FAILED ❌
|
||||
- **Reason:** P0 Tests (Provider Switching, Key Security) failed in automation due to accessibility selector mismatches (missing accessible names on inputs).
|
||||
- **Action Required:** Dev team to add `aria-label` or `<label>` associations to Settings form inputs.
|
||||
|
||||
---
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
### High-Priority Risks (Score ≥6)
|
||||
|
||||
| Risk ID | Category | Description | Probability | Impact | Score | Mitigation | Owner | Timeline |
|
||||
| ------- | -------- | --------------------------------------------------- | ------------ | ------------ | ----- | ------------------------------------------------------------------------------------------------- | ----- | -------- |
|
||||
| R-001 | SEC | API Key Theft via XSS (if localStorage compromised) | 2 (Possible) | 3 (Critical) | 6 | Basic encoding (obfuscation) + Minimize 3rd party scripts. Future: Encrypt with session password. | DEV | Sprint 4 |
|
||||
| R-002 | TECH | Browser CORS policies blocking direct API calls | 3 (Likely) | 2 (Degraded) | 6 | Implement optional Vercel Edge Proxy for non-CORS providers (as per Architecture). | DEV | Sprint 4 |
|
||||
|
||||
### Medium-Priority Risks (Score 3-4)
|
||||
|
||||
| Risk ID | Category | Description | Probability | Impact | Score | Mitigation | Owner |
|
||||
| ------- | -------- | ---------------------------------------- | ------------ | ------------ | ----- | -------------------------------------------------------------- | ----- |
|
||||
| R-003 | BUS | Invalid credentials causing chat failure | 3 (Likely) | 1 (Minor) | 3 | Story 4.2 "Connection Validation" (Hello check) before saving. | DEV |
|
||||
| R-004 | DATA | Loss of settings on browser cache clear | 2 (Possible) | 2 (Degraded) | 4 | Accept risk for MVP (Local-First constraint). | PM |
|
||||
|
||||
---
|
||||
|
||||
## Test Coverage Plan
|
||||
|
||||
### P0 (Critical) - Run on every commit
|
||||
|
||||
**Criteria**: Blocks core functionality (chat) + High Risk (Security/CORS).
|
||||
|
||||
| Requirement | Test Level | Risk Link | Test Count | Owner | Notes |
|
||||
| --------------------- | ---------- | --------- | ---------- | ----- | --------------------------------------------------------- |
|
||||
| Provider Switching | E2E | R-002 | 1 | QA | Verify requests routed to correct Base URL. |
|
||||
| Key Storage Security | Unit | R-001 | 1 | DEV | Verify keys are encoded in localStorage (not plain text). |
|
||||
| Connection Validation | API (Mock) | R-003 | 1 | DEV | Verify validation fails gracefully for invalid keys. |
|
||||
|
||||
**Total P0**: 3 tests, 6 hours
|
||||
|
||||
### P1 (High) - Run on PR to main
|
||||
|
||||
**Criteria**: Important configuration features.
|
||||
|
||||
| Requirement | Test Level | Risk Link | Test Count | Owner | Notes |
|
||||
| --------------------- | ---------- | --------- | ---------- | ----- | ------------------------------------------------ |
|
||||
| Settings Persistence | Component | R-004 | 2 | DEV | Verify settings survive reload. |
|
||||
| Model Selection | Unit | - | 2 | DEV | Verify model-specific parameters/payloads. |
|
||||
| Default Configuration | Unit | - | 1 | DEV | Verify defaults applied when no custom settings. |
|
||||
|
||||
**Total P1**: 5 tests, 5 hours
|
||||
|
||||
### P2 (Medium) - Run nightly/weekly
|
||||
|
||||
**Criteria**: UI polish and edge cases.
|
||||
|
||||
| Requirement | Test Level | Risk Link | Test Count | Owner | Notes |
|
||||
| ----------------------- | ---------- | --------- | ---------- | ----- | --------------------------------------------- |
|
||||
| UI Field Validation | Component | - | 4 | DEV | Empty fields, malformed URLs. |
|
||||
| Provider List Rendering | Component | - | 2 | DEV | Verify list updates when adding new provider. |
|
||||
|
||||
**Total P2**: 6 tests, 3 hours
|
||||
|
||||
---
|
||||
|
||||
## Quality Gate Criteria
|
||||
|
||||
### Pass/Fail Thresholds
|
||||
|
||||
- **P0 pass rate**: 100%
|
||||
- **P1 pass rate**: ≥95%
|
||||
- **High-risk mitigations**: R-001 (Encoding) and R-002 (CORS/Proxy plan) must be implemented.
|
||||
|
||||
### Coverage Targets
|
||||
|
||||
- **Security scenarios** (Key storage): 100%
|
||||
|
||||
---
|
||||
|
||||
## Mitigation Plans
|
||||
|
||||
### R-001: API Key Theft via XSS (Score: 6)
|
||||
|
||||
**Mitigation Strategy:** Implement basic encoding for keys in `localStorage` to prevent casual shoulder-surfing or simple grep attacks. Minimize use of third-party scripts to reduce XSS surface.
|
||||
**Owner:** DEV
|
||||
**Timeline:** Sprint 4 Implementation
|
||||
**Status:** Planned
|
||||
**Verification:** Inspect `localStorage` during P0 test; verify key is not human-readable.
|
||||
|
||||
### R-002: Browser CORS blocking (Score: 6)
|
||||
**Mitigation Strategy:** Provide optional "Proxy Mode" configuration.
|
||||
**Owner:** DEV
|
||||
**Timeline:** Sprint 4 Implementation
|
||||
**Status:** Planned
|
||||
**Verification:** P0 E2E test verifying proxy routing when enabled.
|
||||
|
||||
---
|
||||
|
||||
**Generated by**: BMad TEA Agent - Test Architect Module
|
||||
**Workflow**: `_bmad/bmm/testarch/test-design`
|
||||
Reference in New Issue
Block a user