e9e6fadb1d
- Fix ChatBubble to handle non-string content with String() wrapper - Fix API route to use generateText for non-streaming requests - Add @ai-sdk/openai-compatible for non-OpenAI providers (DeepSeek, etc.) - Use Chat Completions API instead of Responses API for compatible providers - Update ChatBubble tests and fix component exports to kebab-case - Remove stale PascalCase ChatBubble.tsx file
118 lines
5.6 KiB
Markdown
118 lines
5.6 KiB
Markdown
# Test Design: Epic 4 - Power User Settings - BYOD & Configuration
|
|
|
|
**Date:** 2026-01-24
|
|
**Author:** Max
|
|
**Status:** **Approved** (Verification Failed)
|
|
|
|
---
|
|
|
|
## Executive Summary
|
|
|
|
**Scope:** Epic-Level test design for Epic 4 (BYOD & Configuration). Focus on security of API keys and reliability of provider connections.
|
|
|
|
**Risk Summary:**
|
|
|
|
- Total risks identified: 4
|
|
- High-priority risks (≥6): 2
|
|
- Critical categories: SEC, TECH
|
|
|
|
**Verification Status:**
|
|
- **Exploratory Validation (P0):** FAILED ❌
|
|
- **Reason:** P0 Tests (Provider Switching, Key Security) failed in automation due to accessibility selector mismatches (missing accessible names on inputs).
|
|
- **Action Required:** Dev team to add `aria-label` or `<label>` associations to Settings form inputs.
|
|
|
|
---
|
|
|
|
## Risk Assessment
|
|
|
|
### High-Priority Risks (Score ≥6)
|
|
|
|
| Risk ID | Category | Description | Probability | Impact | Score | Mitigation | Owner | Timeline |
|
|
| ------- | -------- | --------------------------------------------------- | ------------ | ------------ | ----- | ------------------------------------------------------------------------------------------------- | ----- | -------- |
|
|
| R-001 | SEC | API Key Theft via XSS (if localStorage compromised) | 2 (Possible) | 3 (Critical) | 6 | Basic encoding (obfuscation) + Minimize 3rd party scripts. Future: Encrypt with session password. | DEV | Sprint 4 |
|
|
| R-002 | TECH | Browser CORS policies blocking direct API calls | 3 (Likely) | 2 (Degraded) | 6 | Implement optional Vercel Edge Proxy for non-CORS providers (as per Architecture). | DEV | Sprint 4 |
|
|
|
|
### Medium-Priority Risks (Score 3-4)
|
|
|
|
| Risk ID | Category | Description | Probability | Impact | Score | Mitigation | Owner |
|
|
| ------- | -------- | ---------------------------------------- | ------------ | ------------ | ----- | -------------------------------------------------------------- | ----- |
|
|
| R-003 | BUS | Invalid credentials causing chat failure | 3 (Likely) | 1 (Minor) | 3 | Story 4.2 "Connection Validation" (Hello check) before saving. | DEV |
|
|
| R-004 | DATA | Loss of settings on browser cache clear | 2 (Possible) | 2 (Degraded) | 4 | Accept risk for MVP (Local-First constraint). | PM |
|
|
|
|
---
|
|
|
|
## Test Coverage Plan
|
|
|
|
### P0 (Critical) - Run on every commit
|
|
|
|
**Criteria**: Blocks core functionality (chat) + High Risk (Security/CORS).
|
|
|
|
| Requirement | Test Level | Risk Link | Test Count | Owner | Notes |
|
|
| --------------------- | ---------- | --------- | ---------- | ----- | --------------------------------------------------------- |
|
|
| Provider Switching | E2E | R-002 | 1 | QA | Verify requests routed to correct Base URL. |
|
|
| Key Storage Security | Unit | R-001 | 1 | DEV | Verify keys are encoded in localStorage (not plain text). |
|
|
| Connection Validation | API (Mock) | R-003 | 1 | DEV | Verify validation fails gracefully for invalid keys. |
|
|
|
|
**Total P0**: 3 tests, 6 hours
|
|
|
|
### P1 (High) - Run on PR to main
|
|
|
|
**Criteria**: Important configuration features.
|
|
|
|
| Requirement | Test Level | Risk Link | Test Count | Owner | Notes |
|
|
| --------------------- | ---------- | --------- | ---------- | ----- | ------------------------------------------------ |
|
|
| Settings Persistence | Component | R-004 | 2 | DEV | Verify settings survive reload. |
|
|
| Model Selection | Unit | - | 2 | DEV | Verify model-specific parameters/payloads. |
|
|
| Default Configuration | Unit | - | 1 | DEV | Verify defaults applied when no custom settings. |
|
|
|
|
**Total P1**: 5 tests, 5 hours
|
|
|
|
### P2 (Medium) - Run nightly/weekly
|
|
|
|
**Criteria**: UI polish and edge cases.
|
|
|
|
| Requirement | Test Level | Risk Link | Test Count | Owner | Notes |
|
|
| ----------------------- | ---------- | --------- | ---------- | ----- | --------------------------------------------- |
|
|
| UI Field Validation | Component | - | 4 | DEV | Empty fields, malformed URLs. |
|
|
| Provider List Rendering | Component | - | 2 | DEV | Verify list updates when adding new provider. |
|
|
|
|
**Total P2**: 6 tests, 3 hours
|
|
|
|
---
|
|
|
|
## Quality Gate Criteria
|
|
|
|
### Pass/Fail Thresholds
|
|
|
|
- **P0 pass rate**: 100%
|
|
- **P1 pass rate**: ≥95%
|
|
- **High-risk mitigations**: R-001 (Encoding) and R-002 (CORS/Proxy plan) must be implemented.
|
|
|
|
### Coverage Targets
|
|
|
|
- **Security scenarios** (Key storage): 100%
|
|
|
|
---
|
|
|
|
## Mitigation Plans
|
|
|
|
### R-001: API Key Theft via XSS (Score: 6)
|
|
|
|
**Mitigation Strategy:** Implement basic encoding for keys in `localStorage` to prevent casual shoulder-surfing or simple grep attacks. Minimize use of third-party scripts to reduce XSS surface.
|
|
**Owner:** DEV
|
|
**Timeline:** Sprint 4 Implementation
|
|
**Status:** Planned
|
|
**Verification:** Inspect `localStorage` during P0 test; verify key is not human-readable.
|
|
|
|
### R-002: Browser CORS blocking (Score: 6)
|
|
**Mitigation Strategy:** Provide optional "Proxy Mode" configuration.
|
|
**Owner:** DEV
|
|
**Timeline:** Sprint 4 Implementation
|
|
**Status:** Planned
|
|
**Verification:** P0 E2E test verifying proxy routing when enabled.
|
|
|
|
---
|
|
|
|
**Generated by**: BMad TEA Agent - Test Architect Module
|
|
**Workflow**: `_bmad/bmm/testarch/test-design`
|