- Fix ChatBubble to handle non-string content with String() wrapper - Fix API route to use generateText for non-streaming requests - Add @ai-sdk/openai-compatible for non-OpenAI providers (DeepSeek, etc.) - Use Chat Completions API instead of Responses API for compatible providers - Update ChatBubble tests and fix component exports to kebab-case - Remove stale PascalCase ChatBubble.tsx file
5.3 KiB
5.3 KiB
Test Design: Epic 1 - Active Listening
Epic: 1 (Core Chat & Teacher Agent) Scope: Epic-Level Date: 2026-01-25 Author: QA Architect (AI)
1. Risk Assessment
Identified Risks
| Risk ID | Category | Title | Description | Probability (1-3) | Impact (1-3) | Score | Action |
|---|---|---|---|---|---|---|---|
| R-1.1 | DATA | Data Loss on Storage Failure | User's chat session is lost due to IndexedDB quota exceeded, browser clearing, or schema mismatch. | 2 (Possible) | 3 (Critical) | 6 | MITIGATE |
| R-1.2 | SEC | API Key Leakage | User's BYOD API Key is logged, exported in history, or sent to a non-provider endpoint. | 1 (Unlikely) | 3 (Critical) | 3 | MONITOR |
| R-1.3 | PERF | High Chat Latency | "Teacher" agent response exceeds 3s, breaking the conversational "venting" flow. | 2 (Possible) | 2 (Degraded) | 4 | MONITOR |
| R-1.4 | BUS | Poor Intent Detection | AI fails to distinguish "Venting" from "Insight", annoying the user with wrong mode. | 2 (Possible) | 2 (Degraded) | 4 | MONITOR |
| R-1.5 | TECH | Offline State Inconsistency | App fails to load history or queue messages when device is offline (Service Worker/IndexedDB fail). | 2 (Possible) | 2 (Degraded) | 4 | MONITOR |
Mitigation Strategies (High Risks)
R-1.1: Data Loss on Storage Failure (Score 6)
- Mitigation: Implement robust error handling around all Dexie operations. Add a "Quota Exceeded" UI warning. Ensure schema versioning is tested.
- Owner: Dev Team
- Validation: Unit tests for
ChatServicestorage failures; E2E test for persistence across reloads.
2. Test Coverage Plan
Acceptance Criteria Mapping
| Story | ID | Scenario | Level | Priority | Risk Link |
|---|---|---|---|---|---|
| 1.1 | 1.1.1 | New user sees initialized empty state (DB created) | Component | P1 | - |
| 1.1 | 1.1.2 | Sent message is saved to IndexedDB | Integration | P0 | R-1.1 |
| 1.1 | 1.1.3 | Chat history persists after page reload | E2E | P0 | R-1.1 |
| 1.1 | 1.1.4 | App loads history while offline | E2E | P1 | R-1.5 |
| 1.2 | 1.2.1 | UI renders "Morning Mist" theme bubbles | Component | P2 | - |
| 1.2 | 1.2.2 | Auto-scroll to bottom on new message | Component | P2 | - |
| 1.2 | 1.2.3 | "Teacher is typing..." indicator appears during wait | Component | P2 | R-1.3 |
| 1.3 | 1.3.1 | AI Classifies "Venting" vs "Insight" correctly (Mocked) | Unit | P1 | R-1.4 |
| 1.3 | 1.3.2 | Client sends request to custom Provider URL | Integration | P0 | R-1.2 |
| 1.3 | 1.3.3 | API Key retrieved from secure storage (not hardcoded) | Unit | P0 | R-1.2 |
| 1.3 | 1.3.4 | Response time < 3s (Performance Check) | E2E | P3 | R-1.3 |
| 1.4 | 1.4.1 | "Fast Track" button skips probing questions | E2E | P1 | - |
| 1.4 | 1.4.2 | Fast Track triggers immediate draft generation | Integration | P1 | - |
Test Levels Strategy
- Unit Tests (Vitest):
- Focus on
ChatServicelogic (DB interactions). - Focus on
PromptEngine(constructing prompts from templates). - Focus on
SettingsService(secure key storage/retrieval).
- Focus on
- Component Tests (React Testing Library / Storybook):
ChatBubble: Verify styling (User vs AI).ChatWindow: Verify scroll behavior and typing indicators.FastTrackToggle: Verify state change.
- Integration/E2E (Playwright):
- Critical Path (P0): User configures Key -> Starts Chat -> Sends Message -> Verifies Persistence.
- Offline Path (P1): Load app offline -> verify history visible.
3. Execution Plan
Smoke Tests (Pre-Merge)
- Unit: All
ChatServicetests (Persistence logic). - E2E: "Happy Path" - User can send a message and see it appear.
Regression Suite (Nightly)
- E2E: Full persistence check (Reload page).
- E2E: Offline mode loading.
- Performance: Measure Time-to-First-Token (TTFT) simulation.
Resource Estimates
- P0 Scenarios: 4 tests (approx. 4 hours implementation).
- P1 Scenarios: 4 tests (approx. 3 hours implementation).
- P2/P3 Scenarios: 3 tests (approx. 1 hour implementation).
- Total Effort: ~1 day.
4. Quality Gate Criteria
- Pass Rate: 100% on P0 tests.
- Coverage: 100% Unit coverage on
services/chat-service.ts. - Mitigation:
R-1.1(Data Loss) must have verified error handling tests.