mirror of
https://github.com/catlog22/Claude-Code-Workflow.git
synced 2026-02-05 01:50:27 +08:00
feat: Enhance test-gen workflow with comprehensive test generation and fix cycle
- Create test-context-gather.md for MCP-based coverage analysis - Analyze test coverage gaps using MCP code-index tools - Identify files requiring tests - Detect test framework and conventions - Generate test-context-package.json with coverage data - Create test-concept-enhanced.md for Gemini-powered test analysis - Analyze coverage gaps and implementation context - Study existing test patterns and conventions - Generate comprehensive test requirements - Design test generation strategy - Output TEST_ANALYSIS_RESULTS.md with test specifications - Update test-task-generate.md to generate two tasks - IMPL-001: Test generation task calling @code-developer - IMPL-002: Test execution and fix cycle task calling @test-fix-agent - Use TEST_ANALYSIS_RESULTS.md for test generation requirements - Configure Gemini diagnosis and Codex resume mechanism for fixes - Update test-gen.md to 5-phase workflow - Phase 1: Create test session - Phase 2: Gather test context (test-context-gather) - Phase 3: Analyze test requirements (test-concept-enhanced) - Phase 4: Generate test tasks (test-task-generate) - Phase 5: Return summary - Simplify documentation by removing redundant content - Maintain all original intent while improving clarity Complete workflow: Coverage analysis → Gemini analysis → Test generation → Test execution → Fix cycle 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
catlog22
@@ -37,7 +37,7 @@ allowed-tools: SlashCommand(*), TodoWrite(*), Read(*), Bash(*)
|
||||
6. **Track Progress**: Update TodoWrite after every phase completion
|
||||
7. **Automatic Detection**: context-gather auto-detects test session and gathers source session context
|
||||
|
||||
## 4-Phase Execution
|
||||
## 5-Phase Execution
|
||||
|
||||
### Phase 1: Create Test Session
|
||||
**Command**: `SlashCommand(command="/workflow:session:start --new \"Test validation for [sourceSessionId]\"")`
|
||||
@@ -65,107 +65,114 @@ allowed-tools: SlashCommand(*), TodoWrite(*), Read(*), Bash(*)
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Gather Cross-Session Context
|
||||
**Command**: `SlashCommand(command="/workflow:tools:context-gather --session [testSessionId]")`
|
||||
### Phase 2: Gather Test Context
|
||||
**Command**: `SlashCommand(command="/workflow:tools:test-context-gather --session [testSessionId]")`
|
||||
|
||||
**Input**: `testSessionId` from Phase 1 (e.g., `WFS-test-user-auth`)
|
||||
|
||||
**Automatic Detection**:
|
||||
- context-gather reads `.workflow/[testSessionId]/workflow-session.json`
|
||||
- Detects `workflow_type: "test_session"`
|
||||
- Automatically uses `source_session_id` to gather source session context
|
||||
- No need for manual `--source-session` parameter
|
||||
|
||||
**Cross-Session Context Collection** (Automatic):
|
||||
- Implementation summaries: `.workflow/[sourceSessionId]/.summaries/IMPL-*-summary.md`
|
||||
- Code changes: `git log --since=[source_session_created_at]` for changed files
|
||||
- Original plan: `.workflow/[sourceSessionId]/IMPL_PLAN.md`
|
||||
- Test files: Discovered via MCP code-index tools
|
||||
**Expected Behavior**:
|
||||
- Load source session implementation context and summaries
|
||||
- Analyze test coverage using MCP tools (find existing tests)
|
||||
- Identify files requiring tests (coverage gaps)
|
||||
- Detect test framework and conventions
|
||||
- Generate `test-context-package.json`
|
||||
|
||||
**Parse Output**:
|
||||
- Extract: context package path (store as `contextPath`)
|
||||
- Pattern: `.workflow/[testSessionId]/.process/context-package.json`
|
||||
- Extract: test context package path (store as `testContextPath`)
|
||||
- Pattern: `.workflow/[testSessionId]/.process/test-context-package.json`
|
||||
|
||||
**Validation**:
|
||||
- Context package created in test session directory
|
||||
- Contains source session artifacts (summaries, changed files)
|
||||
- Includes test file inventory
|
||||
- Test context package created
|
||||
- Contains source session summaries
|
||||
- Includes coverage gap analysis
|
||||
- Test framework detected
|
||||
- Test conventions documented
|
||||
|
||||
**TodoWrite**: Mark phase 2 completed, phase 3 in_progress
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Implementation Analysis
|
||||
**Command**: `SlashCommand(command="/workflow:tools:concept-enhanced --session [testSessionId] --context [contextPath]")`
|
||||
### Phase 3: Test Generation Analysis
|
||||
**Command**: `SlashCommand(command="/workflow:tools:test-concept-enhanced --session [testSessionId] --context [testContextPath]")`
|
||||
|
||||
**Input**:
|
||||
- `testSessionId` from Phase 1 (e.g., `WFS-test-user-auth`)
|
||||
- `contextPath` from Phase 2 (e.g., `.workflow/WFS-test-user-auth/.process/context-package.json`)
|
||||
|
||||
**Analysis Focus**:
|
||||
- Review implementation summaries from source session
|
||||
- Identify test files and coverage gaps
|
||||
- Assess test execution strategy (unit, integration, e2e)
|
||||
- Determine failure diagnosis approach
|
||||
- Recommend code quality improvements
|
||||
- `testSessionId` from Phase 1
|
||||
- `testContextPath` from Phase 2
|
||||
|
||||
**Expected Behavior**:
|
||||
- Reads context-package.json with cross-session artifacts
|
||||
- Executes parallel analysis (Gemini for test strategy, optional Codex for validation)
|
||||
- Generates comprehensive test execution strategy
|
||||
- Identifies code modification targets for test fixes
|
||||
- Provides feasibility assessment for test validation
|
||||
- Use Gemini to analyze coverage gaps and implementation context
|
||||
- Study existing test patterns and conventions
|
||||
- Generate test requirements for each missing test file
|
||||
- Design test generation strategy
|
||||
- Generate `TEST_ANALYSIS_RESULTS.md`
|
||||
|
||||
**Parse Output**:
|
||||
- Verify `.workflow/[testSessionId]/.process/ANALYSIS_RESULTS.md` created
|
||||
- Contains test strategy and execution plan
|
||||
- Lists code modification targets (format: `file:function:lines` or `file`)
|
||||
- Includes risk assessment and optimization recommendations
|
||||
- Verify `.workflow/[testSessionId]/.process/TEST_ANALYSIS_RESULTS.md` created
|
||||
- Contains test requirements and generation strategy
|
||||
- Lists test files to create with specifications
|
||||
|
||||
**Validation**:
|
||||
- File `.workflow/[testSessionId]/.process/ANALYSIS_RESULTS.md` exists
|
||||
- Contains complete analysis sections:
|
||||
- Current State Analysis (test coverage, existing tests)
|
||||
- Proposed Solution Design (test execution strategy)
|
||||
- Implementation Strategy (code targets, feasibility)
|
||||
- Solution Optimization (performance, quality)
|
||||
- Critical Success Factors (acceptance criteria)
|
||||
- TEST_ANALYSIS_RESULTS.md exists with complete sections:
|
||||
- Coverage Assessment
|
||||
- Test Framework & Conventions
|
||||
- Test Requirements by File
|
||||
- Test Generation Strategy
|
||||
- Implementation Targets (test files to create)
|
||||
- Success Criteria
|
||||
|
||||
**TodoWrite**: Mark phase 3 completed, phase 4 in_progress
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Generate Test Task
|
||||
**Command**: `SlashCommand(command="/workflow:tools:task-generate --session [testSessionId]")`
|
||||
### Phase 4: Generate Test Tasks
|
||||
**Command**: `SlashCommand(command="/workflow:tools:test-task-generate --session [testSessionId]")`
|
||||
|
||||
**Input**: `testSessionId` from Phase 1
|
||||
|
||||
**Expected Behavior**:
|
||||
- Reads ANALYSIS_RESULTS.md from Phase 3
|
||||
- Extracts test strategy and code modification targets
|
||||
- Generates `IMPL-001.json` (reusing standard format) with:
|
||||
- `meta.type: "test-fix"` (enables @test-fix-agent assignment)
|
||||
- `meta.agent: "@test-fix-agent"`
|
||||
- `context.requirements`: Test execution requirements from analysis
|
||||
- `context.focus_paths`: Test files and source files from analysis
|
||||
- `context.acceptance`: All tests pass criteria
|
||||
- `flow_control.pre_analysis`: Load source session summaries
|
||||
- `flow_control.implementation_approach`: Test execution strategy from ANALYSIS_RESULTS.md
|
||||
- `flow_control.target_files`: Code modification targets from analysis
|
||||
- Parse TEST_ANALYSIS_RESULTS.md from Phase 3
|
||||
- Extract test requirements and generation strategy
|
||||
- Generate **TWO task JSON files**:
|
||||
- **IMPL-001.json**: Test Generation task (calls @code-developer)
|
||||
- **IMPL-002.json**: Test Execution and Fix Cycle task (calls @test-fix-agent)
|
||||
- Generate IMPL_PLAN.md with test generation and execution strategy
|
||||
- Generate TODO_LIST.md with both tasks
|
||||
|
||||
**Parse Output**:
|
||||
- Verify `.workflow/[testSessionId]/.task/IMPL-001.json` exists
|
||||
- Verify `.workflow/[testSessionId]/.task/IMPL-001.json` exists (test generation)
|
||||
- Verify `.workflow/[testSessionId]/.task/IMPL-002.json` exists (test execution & fix)
|
||||
- Verify `.workflow/[testSessionId]/IMPL_PLAN.md` created
|
||||
- Verify `.workflow/[testSessionId]/TODO_LIST.md` created
|
||||
|
||||
**Validation**:
|
||||
- Task JSON has `id: "IMPL-001"` and `meta.type: "test-fix"`
|
||||
- IMPL_PLAN.md contains test validation strategy from ANALYSIS_RESULTS.md
|
||||
- TODO_LIST.md shows IMPL-001 task
|
||||
- flow_control includes code targets and test strategy
|
||||
- Task is ready for /workflow:execute
|
||||
**Validation - IMPL-001.json (Test Generation)**:
|
||||
- Task ID: `IMPL-001`
|
||||
- `meta.type: "test-gen"`
|
||||
- `meta.agent: "@code-developer"`
|
||||
- `context.requirements`: Generate tests based on TEST_ANALYSIS_RESULTS.md
|
||||
- `flow_control.pre_analysis`: Load TEST_ANALYSIS_RESULTS.md and test context
|
||||
- `flow_control.implementation_approach`: Test generation steps
|
||||
- `flow_control.target_files`: Test files to create from analysis section 5
|
||||
|
||||
**TodoWrite**: Mark phase 4 completed
|
||||
**Validation - IMPL-002.json (Test Execution & Fix)**:
|
||||
- Task ID: `IMPL-002`
|
||||
- `meta.type: "test-fix"`
|
||||
- `meta.agent: "@test-fix-agent"`
|
||||
- `context.depends_on: ["IMPL-001"]`
|
||||
- `context.requirements`: Execute and fix tests
|
||||
- `flow_control.implementation_approach.test_fix_cycle`: Complete cycle specification
|
||||
- **Cycle pattern**: test → gemini_diagnose → codex_fix_resume → retest
|
||||
- **Tools configuration**: Gemini for analysis, Codex resume for fixes
|
||||
- **Exit conditions**: Success (all pass) or failure (max iterations)
|
||||
- `flow_control.implementation_approach.modification_points`: 3-phase execution flow
|
||||
- Phase 1: Initial test execution
|
||||
- Phase 2: Iterative Gemini diagnosis + Codex resume fixes
|
||||
- Phase 3: Final validation and certification
|
||||
|
||||
**TodoWrite**: Mark phase 4 completed, phase 5 in_progress
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Return Summary to User
|
||||
|
||||
**Return to User**:
|
||||
```
|
||||
@@ -173,315 +180,119 @@ Independent test-fix workflow created successfully!
|
||||
|
||||
Source Session: [sourceSessionId]
|
||||
Test Session: [testSessionId]
|
||||
Task Created: IMPL-001 (test-fix)
|
||||
|
||||
Tasks Created:
|
||||
- IMPL-001: Test Generation (@code-developer)
|
||||
- IMPL-002: Test Execution & Fix Cycle (@test-fix-agent)
|
||||
|
||||
Test Framework: [detected framework]
|
||||
Test Files to Generate: [count]
|
||||
Max Fix Iterations: 5
|
||||
|
||||
Next Steps:
|
||||
1. Review test plan: .workflow/[testSessionId]/IMPL_PLAN.md
|
||||
2. Execute validation: /workflow:execute
|
||||
2. Execute workflow: /workflow:execute
|
||||
3. Monitor progress: /workflow:status
|
||||
|
||||
The @test-fix-agent will:
|
||||
- Execute all tests
|
||||
- Diagnose any failures
|
||||
- Fix code until tests pass
|
||||
```
|
||||
|
||||
**TodoWrite**: Mark phase 5 completed
|
||||
|
||||
---
|
||||
|
||||
## TodoWrite Pattern
|
||||
|
||||
Track progress through 5 phases:
|
||||
|
||||
```javascript
|
||||
// Initialize (before Phase 1)
|
||||
TodoWrite({todos: [
|
||||
{"content": "Create independent test session", "status": "in_progress", "activeForm": "Creating test session"},
|
||||
{"content": "Gather cross-session context", "status": "pending", "activeForm": "Gathering cross-session context"},
|
||||
{"content": "Analyze implementation for test strategy", "status": "pending", "activeForm": "Analyzing implementation"},
|
||||
{"content": "Generate test validation task", "status": "pending", "activeForm": "Generating test validation task"}
|
||||
]})
|
||||
|
||||
// After Phase 1
|
||||
TodoWrite({todos: [
|
||||
{"content": "Create independent test session", "status": "completed", "activeForm": "Creating test session"},
|
||||
{"content": "Gather cross-session context", "status": "in_progress", "activeForm": "Gathering cross-session context"},
|
||||
{"content": "Analyze implementation for test strategy", "status": "pending", "activeForm": "Analyzing implementation"},
|
||||
{"content": "Generate test validation task", "status": "pending", "activeForm": "Generating test validation task"}
|
||||
]})
|
||||
|
||||
// After Phase 2
|
||||
TodoWrite({todos: [
|
||||
{"content": "Create independent test session", "status": "completed", "activeForm": "Creating test session"},
|
||||
{"content": "Gather cross-session context", "status": "completed", "activeForm": "Gathering cross-session context"},
|
||||
{"content": "Analyze implementation for test strategy", "status": "in_progress", "activeForm": "Analyzing implementation"},
|
||||
{"content": "Generate test validation task", "status": "pending", "activeForm": "Generating test validation task"}
|
||||
]})
|
||||
|
||||
// After Phase 3
|
||||
TodoWrite({todos: [
|
||||
{"content": "Create independent test session", "status": "completed", "activeForm": "Creating test session"},
|
||||
{"content": "Gather cross-session context", "status": "completed", "activeForm": "Gathering cross-session context"},
|
||||
{"content": "Analyze implementation for test strategy", "status": "completed", "activeForm": "Analyzing implementation"},
|
||||
{"content": "Generate test validation task", "status": "in_progress", "activeForm": "Generating test validation task"}
|
||||
]})
|
||||
|
||||
// After Phase 4
|
||||
TodoWrite({todos: [
|
||||
{"content": "Create independent test session", "status": "completed", "activeForm": "Creating test session"},
|
||||
{"content": "Gather cross-session context", "status": "completed", "activeForm": "Gathering cross-session context"},
|
||||
{"content": "Analyze implementation for test strategy", "status": "completed", "activeForm": "Analyzing implementation"},
|
||||
{"content": "Generate test validation task", "status": "completed", "activeForm": "Generating test validation task"}
|
||||
{"content": "Create independent test session", "status": "in_progress|completed", "activeForm": "Creating test session"},
|
||||
{"content": "Gather test coverage context", "status": "pending|in_progress|completed", "activeForm": "Gathering test coverage context"},
|
||||
{"content": "Analyze test requirements with Gemini", "status": "pending|in_progress|completed", "activeForm": "Analyzing test requirements"},
|
||||
{"content": "Generate test generation and execution tasks", "status": "pending|in_progress|completed", "activeForm": "Generating test tasks"},
|
||||
{"content": "Return workflow summary", "status": "pending|in_progress|completed", "activeForm": "Returning workflow summary"}
|
||||
]})
|
||||
```
|
||||
|
||||
Update status to `in_progress` when starting each phase, mark `completed` when done.
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
User: /workflow:test-gen WFS-user-auth
|
||||
↓
|
||||
Phase 1: session-start --new "Test validation for WFS-user-auth"
|
||||
↓ Creates: WFS-test-user-auth session
|
||||
↓ Writes: workflow-session.json with workflow_type="test_session", source_session_id="WFS-user-auth"
|
||||
↓ Output: testSessionId = "WFS-test-user-auth"
|
||||
↓
|
||||
Phase 2: context-gather --session WFS-test-user-auth
|
||||
↓ Auto-detects: test session type from workflow-session.json
|
||||
↓ Auto-reads: source_session_id = "WFS-user-auth"
|
||||
↓ Gathers: Cross-session context (summaries, code changes, tests)
|
||||
↓ Output: .workflow/WFS-test-user-auth/.process/context-package.json
|
||||
↓
|
||||
Phase 3: concept-enhanced --session WFS-test-user-auth --context context-package.json
|
||||
↓ Reads: context-package.json with cross-session artifacts
|
||||
↓ Executes: Parallel analysis (Gemini test strategy + optional Codex validation)
|
||||
↓ Analyzes: Test coverage, execution strategy, code targets
|
||||
↓ Output: .workflow/WFS-test-user-auth/.process/ANALYSIS_RESULTS.md
|
||||
↓
|
||||
Phase 4: task-generate --session WFS-test-user-auth
|
||||
↓ Reads: ANALYSIS_RESULTS.md with test strategy and code targets
|
||||
↓ Generates: IMPL-001.json with meta.type="test-fix"
|
||||
↓ Output: Task, plan, and todo files in test session
|
||||
↓
|
||||
Return: Summary with next steps (user triggers /workflow:execute separately)
|
||||
/workflow:test-gen WFS-user-auth
|
||||
↓
|
||||
Phase 1: session-start → WFS-test-user-auth
|
||||
↓
|
||||
Phase 2: test-context-gather → test-context-package.json
|
||||
↓
|
||||
Phase 3: test-concept-enhanced → TEST_ANALYSIS_RESULTS.md
|
||||
↓
|
||||
Phase 4: test-task-generate → IMPL-001.json + IMPL-002.json
|
||||
↓
|
||||
Phase 5: Return summary
|
||||
↓
|
||||
/workflow:execute → IMPL-001 (@code-developer) → IMPL-002 (@test-fix-agent)
|
||||
```
|
||||
|
||||
## Session Metadata Design
|
||||
## Session Metadata
|
||||
|
||||
**Test Session (`WFS-test-user-auth/workflow-session.json`)**:
|
||||
```json
|
||||
{
|
||||
"session_id": "WFS-test-user-auth",
|
||||
"project": "Test validation for user authentication implementation",
|
||||
"status": "planning",
|
||||
"created_at": "2025-10-03T12:00:00Z",
|
||||
"workflow_type": "test_session",
|
||||
"source_session_id": "WFS-user-auth"
|
||||
}
|
||||
```
|
||||
Test session includes `workflow_type: "test_session"` and `source_session_id` for automatic context gathering.
|
||||
|
||||
## Automatic Cross-Session Context Collection
|
||||
## Task Output
|
||||
|
||||
When `context-gather` detects `workflow_type: "test_session"`:
|
||||
Generates two tasks:
|
||||
- **IMPL-001** (@code-developer): Test generation from TEST_ANALYSIS_RESULTS.md
|
||||
- **IMPL-002** (@test-fix-agent): Test execution with iterative fix cycle (max 5 iterations)
|
||||
|
||||
**Collected from Source Session** (`.workflow/WFS-user-auth/`):
|
||||
- Implementation summaries: `.summaries/IMPL-*-summary.md`
|
||||
- Code changes: `git log --since=[source_created_at] --name-only`
|
||||
- Original plan: `IMPL_PLAN.md`
|
||||
- Task definitions: `.task/IMPL-*.json`
|
||||
|
||||
**Collected from Current Project**:
|
||||
- Test files: `mcp__code-index__find_files(pattern="*.test.*")`
|
||||
- Test configuration: `package.json`, `jest.config.js`, etc.
|
||||
- Source files: Based on changed files from git log
|
||||
|
||||
## Task Generation Output
|
||||
|
||||
task-generate creates `IMPL-001.json` (reusing standard format) with:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "IMPL-001",
|
||||
"title": "Execute and validate tests for [sourceSessionId]",
|
||||
"status": "pending",
|
||||
"meta": {
|
||||
"type": "test-fix",
|
||||
"agent": "@test-fix-agent"
|
||||
},
|
||||
"context": {
|
||||
"requirements": [
|
||||
"Execute complete test suite for all implemented modules",
|
||||
"Diagnose and fix any test failures",
|
||||
"Ensure all tests pass before completion"
|
||||
],
|
||||
"focus_paths": ["src/**/*.test.ts", "src/**/implementation.ts"],
|
||||
"acceptance": [
|
||||
"All tests pass successfully",
|
||||
"No test failures or errors",
|
||||
"Code is approved and ready for deployment"
|
||||
],
|
||||
"depends_on": [],
|
||||
"artifacts": []
|
||||
},
|
||||
"flow_control": {
|
||||
"pre_analysis": [
|
||||
{
|
||||
"step": "load_source_session_summaries",
|
||||
"action": "Load implementation summaries from source session",
|
||||
"commands": [
|
||||
"bash(find .workflow/[sourceSessionId]/.summaries/ -name 'IMPL-*-summary.md' 2>/dev/null)",
|
||||
"Read(.workflow/[sourceSessionId]/.summaries/IMPL-001-summary.md)"
|
||||
],
|
||||
"output_to": "implementation_context",
|
||||
"on_error": "skip_optional"
|
||||
},
|
||||
{
|
||||
"step": "analyze_test_files",
|
||||
"action": "Identify test files and coverage",
|
||||
"commands": [
|
||||
"mcp__code-index__find_files(pattern=\"*.test.*\")",
|
||||
"mcp__code-index__search_code_advanced(pattern=\"test|describe|it\", file_pattern=\"*.test.*\")"
|
||||
],
|
||||
"output_to": "test_inventory",
|
||||
"on_error": "fail"
|
||||
}
|
||||
],
|
||||
"implementation_approach": {
|
||||
"task_description": "Execute tests and fix failures until all pass",
|
||||
"modification_points": [
|
||||
"Run test suite using detected framework",
|
||||
"Parse test output to identify failures",
|
||||
"Diagnose root cause of failures",
|
||||
"Modify source code to fix issues",
|
||||
"Re-run tests to verify fixes"
|
||||
],
|
||||
"logic_flow": [
|
||||
"Load implementation context",
|
||||
"Identify test framework and configuration",
|
||||
"Execute complete test suite",
|
||||
"If failures: analyze error messages",
|
||||
"Fix source code based on diagnosis",
|
||||
"Re-run tests",
|
||||
"Repeat until all tests pass"
|
||||
]
|
||||
},
|
||||
"target_files": ["src/**/*.test.ts", "src/**/implementation.ts"]
|
||||
}
|
||||
}
|
||||
```
|
||||
See `/workflow:tools:test-task-generate` for complete task JSON schemas.
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Phase 1 Failures
|
||||
- **Source session not found**: Return error "Source session [sourceSessionId] not found in .workflow/"
|
||||
- **Invalid source session**: Return error "Source session [sourceSessionId] has no completed IMPL tasks"
|
||||
- **Session creation failed**: Return error "Could not create test session. Check /workflow:session:start"
|
||||
| Phase | Error | Action |
|
||||
|-------|-------|--------|
|
||||
| 1 | Source session not found | Return error with source session ID |
|
||||
| 1 | No completed IMPL tasks | Return error, source incomplete |
|
||||
| 2 | Context gathering failed | Return error, check source artifacts |
|
||||
| 3 | Analysis failed | Return error, check context package |
|
||||
| 4 | Task generation failed | Retry once, then error with details |
|
||||
|
||||
### Phase 2 Failures
|
||||
- **Context gathering failed**: Return error "Could not gather cross-session context. Check source session artifacts"
|
||||
- **No source artifacts**: Return error "Source session has no implementation summaries or git history"
|
||||
## Output Files
|
||||
|
||||
### Phase 3 Failures
|
||||
- **Analysis failed**: Return error "Implementation analysis failed. Check context package and ANALYSIS_RESULTS.md"
|
||||
- **No test strategy**: Return error "Could not determine test execution strategy from analysis"
|
||||
- **Missing code targets**: Warning only, proceed with general test task
|
||||
|
||||
### Phase 4 Failures
|
||||
- **Task generation failed**: Retry once, then return error with details
|
||||
- **Invalid task structure**: Return error with JSON validation details
|
||||
|
||||
## Workflow Integration
|
||||
|
||||
### Complete Flow Example
|
||||
```
|
||||
1. Implementation Phase (prior to test-gen)
|
||||
/workflow:plan "Build auth system"
|
||||
→ Creates WFS-auth session
|
||||
→ @code-developer implements + writes tests
|
||||
→ Creates IMPL-001-summary.md in WFS-auth
|
||||
|
||||
2. Test Generation Phase (test-gen)
|
||||
/workflow:test-gen WFS-auth
|
||||
Phase 1: session-start → Creates WFS-test-auth session
|
||||
Phase 2: context-gather → Gathers from WFS-auth, creates context-package.json
|
||||
Phase 3: concept-enhanced → Analyzes implementation, creates ANALYSIS_RESULTS.md
|
||||
Phase 4: task-generate → Creates IMPL-001.json with meta.type="test-fix"
|
||||
Returns: Summary with next steps
|
||||
|
||||
3. Test Execution Phase (user-triggered)
|
||||
/workflow:execute
|
||||
→ Detects active session: WFS-test-auth
|
||||
→ @test-fix-agent picks up IMPL-001 (test-fix type)
|
||||
→ Runs test suite
|
||||
→ Diagnoses failures (if any)
|
||||
→ Fixes source code
|
||||
→ Re-runs tests
|
||||
→ All pass → Code approved ✅
|
||||
```
|
||||
|
||||
### Output Files Created
|
||||
|
||||
**In Test Session** (`.workflow/WFS-test-auth/`):
|
||||
- `workflow-session.json` - Contains workflow_type and source_session_id
|
||||
- `.process/context-package.json` - Cross-session context from WFS-auth
|
||||
- `.process/ANALYSIS_RESULTS.md` - Test strategy and code targets from concept-enhanced
|
||||
- `.task/IMPL-001.json` - Test-fix task definition
|
||||
- `IMPL_PLAN.md` - Test validation plan (from ANALYSIS_RESULTS.md)
|
||||
Created in `.workflow/WFS-test-[session]/`:
|
||||
- `workflow-session.json` - Session metadata
|
||||
- `.process/test-context-package.json` - Coverage analysis
|
||||
- `.process/TEST_ANALYSIS_RESULTS.md` - Test requirements
|
||||
- `.task/IMPL-001.json` - Test generation task
|
||||
- `.task/IMPL-002.json` - Test execution & fix task
|
||||
- `IMPL_PLAN.md` - Test plan
|
||||
- `TODO_LIST.md` - Task checklist
|
||||
- `.summaries/IMPL-001-summary.md` - Created by @test-fix-agent after completion
|
||||
|
||||
## Agent Execution
|
||||
|
||||
**IMPL-001** (@code-developer):
|
||||
- Generates test files based on TEST_ANALYSIS_RESULTS.md
|
||||
- Follows existing test patterns and conventions
|
||||
|
||||
**IMPL-002** (@test-fix-agent):
|
||||
1. Run test suite
|
||||
2. Iterative fix cycle (max 5):
|
||||
- Gemini diagnosis → surgical fix suggestions
|
||||
- Codex applies fixes (resume mechanism)
|
||||
- Retest and check regressions
|
||||
3. Final validation and certification
|
||||
|
||||
See `/workflow:tools:test-task-generate` for detailed specifications.
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Run after implementation complete**: Ensure source session has completed IMPL tasks and summaries
|
||||
2. **Check git commits**: Implementation changes should be committed for accurate git log analysis
|
||||
3. **Verify test files exist**: Source implementation should include test files
|
||||
4. **Independent sessions**: Test session is separate from implementation session for clean separation
|
||||
5. **Monitor execution**: Use `/workflow:status` to track test-fix progress after /workflow:execute
|
||||
|
||||
## Coordinator Checklist
|
||||
|
||||
✅ Initialize TodoWrite before any command (4 phases)
|
||||
✅ Phase 1: Create test session with source session ID
|
||||
✅ Parse new test session ID from Phase 1 output
|
||||
✅ Phase 2: Run context-gather (auto-detects test session, no extra params)
|
||||
✅ Verify context-package.json contains cross-session artifacts
|
||||
✅ Phase 3: Run concept-enhanced with session and context path
|
||||
✅ Verify ANALYSIS_RESULTS.md contains test strategy and code targets
|
||||
✅ Phase 4: Run task-generate to create IMPL-001.json
|
||||
✅ Verify task has meta.type="test-fix" and meta.agent="@test-fix-agent"
|
||||
✅ Verify flow_control includes analysis insights and code targets
|
||||
✅ Update TodoWrite after each phase
|
||||
✅ Return summary after Phase 4 (execution is separate user action)
|
||||
|
||||
## Required Tool Modifications
|
||||
|
||||
### `/workflow:session:start`
|
||||
- Support `--new` flag for test session creation
|
||||
- Auto-detect test session pattern from task description
|
||||
- Write `workflow_type: "test_session"` and `source_session_id` to metadata
|
||||
|
||||
### `/workflow:tools:context-gather`
|
||||
- Read session metadata to detect `workflow_type: "test_session"`
|
||||
- Auto-extract `source_session_id` from metadata
|
||||
- Gather cross-session context from source session artifacts
|
||||
- Include git log analysis from source session creation time
|
||||
|
||||
### `/workflow:tools:concept-enhanced`
|
||||
- No changes required (already supports cross-session context analysis)
|
||||
- Will automatically analyze test strategy based on context-package.json
|
||||
- Generates ANALYSIS_RESULTS.md with code targets and test execution plan
|
||||
|
||||
### `/workflow:tools:task-generate`
|
||||
- Recognize test session context and generate appropriate task
|
||||
- Create `IMPL-001.json` with `meta.type: "test-fix"`
|
||||
- Extract test strategy from ANALYSIS_RESULTS.md
|
||||
- Include code targets and cross-session references in flow_control
|
||||
|
||||
### `/workflow:execute`
|
||||
- No changes required (already dispatches by meta.agent)
|
||||
1. Run after implementation complete (ensure source session has summaries)
|
||||
2. Commit implementation changes before test-gen
|
||||
3. Monitor execution with `/workflow:status`
|
||||
4. Review iteration logs in `.process/fix-iteration-*`
|
||||
|
||||
## Related Commands
|
||||
- `/workflow:plan` - Create implementation workflow (run before test-gen)
|
||||
- `/workflow:session:start` - Phase 1 tool for test session creation
|
||||
- `/workflow:tools:context-gather` - Phase 2 tool for cross-session context collection
|
||||
- `/workflow:tools:concept-enhanced` - Phase 3 tool for implementation analysis and test strategy
|
||||
- `/workflow:tools:task-generate` - Phase 4 tool for test task creation
|
||||
- `/workflow:execute` - Execute test-fix workflow (user-triggered after test-gen)
|
||||
- `/workflow:status` - Check workflow progress
|
||||
- `@test-fix-agent` - Agent that executes and fixes tests
|
||||
|
||||
- `/workflow:tools:test-context-gather` - Phase 2 (coverage analysis)
|
||||
- `/workflow:tools:test-concept-enhanced` - Phase 3 (Gemini test analysis)
|
||||
- `/workflow:tools:test-task-generate` - Phase 4 (task generation)
|
||||
- `/workflow:execute` - Execute workflow
|
||||
- `/workflow:status` - Check progress
|
||||
|
||||
468
.claude/commands/workflow/tools/test-concept-enhanced.md
Normal file
468
.claude/commands/workflow/tools/test-concept-enhanced.md
Normal file
@@ -0,0 +1,468 @@
|
||||
---
|
||||
name: test-concept-enhanced
|
||||
description: Analyze test requirements and generate test generation strategy using Gemini
|
||||
usage: /workflow:tools:test-concept-enhanced --session <test_session_id> --context <test_context_package_path>
|
||||
argument-hint: "--session WFS-test-session-id --context path/to/test-context-package.json"
|
||||
examples:
|
||||
- /workflow:tools:test-concept-enhanced --session WFS-test-auth --context .workflow/WFS-test-auth/.process/test-context-package.json
|
||||
---
|
||||
|
||||
# Test Concept Enhanced Command
|
||||
|
||||
## Overview
|
||||
Specialized analysis tool for test generation workflows that uses Gemini to analyze test coverage gaps, implementation context, and generate comprehensive test generation strategies.
|
||||
|
||||
## Core Philosophy
|
||||
- **Coverage-Driven**: Focus on identified test gaps from context analysis
|
||||
- **Pattern-Based**: Learn from existing tests and project conventions
|
||||
- **Gemini-Powered**: Use Gemini for test requirement analysis and strategy design
|
||||
- **Single-Round Analysis**: Comprehensive test analysis in one execution
|
||||
- **No Code Generation**: Strategy and planning only, actual test generation happens in task execution
|
||||
|
||||
## Core Responsibilities
|
||||
- Parse test-context-package.json from test-context-gather
|
||||
- Analyze implementation summaries and coverage gaps
|
||||
- Study existing test patterns and conventions
|
||||
- Generate test generation strategy using Gemini
|
||||
- Produce TEST_ANALYSIS_RESULTS.md for task generation
|
||||
|
||||
## Execution Lifecycle
|
||||
|
||||
### Phase 1: Validation & Preparation
|
||||
|
||||
1. **Session Validation**
|
||||
- Load `.workflow/{test_session_id}/workflow-session.json`
|
||||
- Verify test session type is "test-gen"
|
||||
- Extract source session reference
|
||||
|
||||
2. **Context Package Validation**
|
||||
- Read `test-context-package.json`
|
||||
- Validate required sections: metadata, source_context, test_coverage, test_framework
|
||||
- Extract coverage gaps and framework details
|
||||
|
||||
3. **Strategy Determination**
|
||||
- **Simple Test Generation** (1-3 files): Single Gemini analysis
|
||||
- **Medium Test Generation** (4-6 files): Gemini comprehensive analysis
|
||||
- **Complex Test Generation** (>6 files): Gemini analysis with modular approach
|
||||
|
||||
### Phase 2: Gemini Test Analysis
|
||||
|
||||
**Tool Configuration**:
|
||||
```bash
|
||||
cd .workflow/{test_session_id}/.process && ~/.claude/scripts/gemini-wrapper -p "
|
||||
PURPOSE: Analyze test coverage gaps and design comprehensive test generation strategy
|
||||
TASK: Study implementation context, existing tests, and generate test requirements for missing coverage
|
||||
MODE: analysis
|
||||
CONTEXT: @{.workflow/{test_session_id}/.process/test-context-package.json}
|
||||
|
||||
**MANDATORY FIRST STEP**: Read and analyze test-context-package.json to understand:
|
||||
- Test coverage gaps from test_coverage.missing_tests[]
|
||||
- Implementation context from source_context.implementation_summaries[]
|
||||
- Existing test patterns from test_framework.conventions
|
||||
- Changed files requiring tests from source_context.implementation_summaries[].changed_files
|
||||
|
||||
**ANALYSIS REQUIREMENTS**:
|
||||
|
||||
1. **Implementation Understanding**
|
||||
- Load all implementation summaries from source session
|
||||
- Understand implemented features, APIs, and business logic
|
||||
- Extract key functions, classes, and modules
|
||||
- Identify integration points and dependencies
|
||||
|
||||
2. **Existing Test Pattern Analysis**
|
||||
- Study existing test files for patterns and conventions
|
||||
- Identify test structure (describe/it, test suites, fixtures)
|
||||
- Analyze assertion patterns and mocking strategies
|
||||
- Extract test setup/teardown patterns
|
||||
|
||||
3. **Coverage Gap Assessment**
|
||||
- For each file in missing_tests[], analyze:
|
||||
- File purpose and functionality
|
||||
- Public APIs requiring test coverage
|
||||
- Critical paths and edge cases
|
||||
- Integration points requiring tests
|
||||
- Prioritize tests: high (core logic), medium (utilities), low (helpers)
|
||||
|
||||
4. **Test Requirements Specification**
|
||||
- For each missing test file, specify:
|
||||
- **Test scope**: What needs to be tested
|
||||
- **Test scenarios**: Happy path, error cases, edge cases, integration
|
||||
- **Test data**: Required fixtures, mocks, test data
|
||||
- **Dependencies**: External services, databases, APIs to mock
|
||||
- **Coverage targets**: Functions/methods requiring tests
|
||||
|
||||
5. **Test Generation Strategy**
|
||||
- Determine test generation approach for each file
|
||||
- Identify reusable test patterns from existing tests
|
||||
- Plan test data and fixture requirements
|
||||
- Define mocking strategy for dependencies
|
||||
- Specify expected test file structure
|
||||
|
||||
EXPECTED OUTPUT - Write to gemini-test-analysis.md:
|
||||
|
||||
# Test Generation Analysis
|
||||
|
||||
## 1. Implementation Context Summary
|
||||
- **Source Session**: {source_session_id}
|
||||
- **Implemented Features**: {feature_summary}
|
||||
- **Changed Files**: {list_of_implementation_files}
|
||||
- **Tech Stack**: {technologies_used}
|
||||
|
||||
## 2. Test Coverage Assessment
|
||||
- **Existing Tests**: {count} files
|
||||
- **Missing Tests**: {count} files
|
||||
- **Coverage Percentage**: {percentage}%
|
||||
- **Priority Breakdown**:
|
||||
- High Priority: {count} files (core business logic)
|
||||
- Medium Priority: {count} files (utilities, helpers)
|
||||
- Low Priority: {count} files (configuration, constants)
|
||||
|
||||
## 3. Existing Test Pattern Analysis
|
||||
- **Test Framework**: {framework_name_and_version}
|
||||
- **File Naming Convention**: {pattern}
|
||||
- **Test Structure**: {describe_it_or_other}
|
||||
- **Assertion Style**: {expect_assert_should}
|
||||
- **Mocking Strategy**: {mocking_framework_and_patterns}
|
||||
- **Setup/Teardown**: {beforeEach_afterEach_patterns}
|
||||
- **Test Data**: {fixtures_factories_builders}
|
||||
|
||||
## 4. Test Requirements by File
|
||||
|
||||
### File: {implementation_file_path}
|
||||
**Test File**: {suggested_test_file_path}
|
||||
**Priority**: {high|medium|low}
|
||||
|
||||
#### Scope
|
||||
- {description_of_what_needs_testing}
|
||||
|
||||
#### Test Scenarios
|
||||
1. **Happy Path Tests**
|
||||
- {scenario_1}
|
||||
- {scenario_2}
|
||||
|
||||
2. **Error Handling Tests**
|
||||
- {error_scenario_1}
|
||||
- {error_scenario_2}
|
||||
|
||||
3. **Edge Case Tests**
|
||||
- {edge_case_1}
|
||||
- {edge_case_2}
|
||||
|
||||
4. **Integration Tests** (if applicable)
|
||||
- {integration_scenario_1}
|
||||
- {integration_scenario_2}
|
||||
|
||||
#### Test Data & Fixtures
|
||||
- {required_test_data}
|
||||
- {required_mocks}
|
||||
- {required_fixtures}
|
||||
|
||||
#### Dependencies to Mock
|
||||
- {external_service_1}
|
||||
- {external_service_2}
|
||||
|
||||
#### Coverage Targets
|
||||
- Function: {function_name} - {test_requirements}
|
||||
- Function: {function_name} - {test_requirements}
|
||||
|
||||
---
|
||||
[Repeat for each missing test file]
|
||||
---
|
||||
|
||||
## 5. Test Generation Strategy
|
||||
|
||||
### Overall Approach
|
||||
- {strategy_description}
|
||||
|
||||
### Test Generation Order
|
||||
1. {file_1} - {rationale}
|
||||
2. {file_2} - {rationale}
|
||||
3. {file_3} - {rationale}
|
||||
|
||||
### Reusable Patterns
|
||||
- {pattern_1_from_existing_tests}
|
||||
- {pattern_2_from_existing_tests}
|
||||
|
||||
### Test Data Strategy
|
||||
- {approach_to_test_data_and_fixtures}
|
||||
|
||||
### Mocking Strategy
|
||||
- {approach_to_mocking_dependencies}
|
||||
|
||||
### Quality Criteria
|
||||
- Code coverage target: {percentage}%
|
||||
- Test scenarios per function: {count}
|
||||
- Integration test coverage: {approach}
|
||||
|
||||
## 6. Implementation Targets
|
||||
|
||||
**Purpose**: Identify new test files to create
|
||||
|
||||
**Format**: New test files only (no existing files to modify)
|
||||
|
||||
**Test Files to Create**:
|
||||
1. **Target**: `tests/auth/TokenValidator.test.ts`
|
||||
- **Type**: Create new test file
|
||||
- **Purpose**: Test TokenValidator class
|
||||
- **Scenarios**: 15 test cases covering validation logic, error handling, edge cases
|
||||
- **Dependencies**: Mock JWT library, test fixtures for tokens
|
||||
|
||||
2. **Target**: `tests/middleware/errorHandler.test.ts`
|
||||
- **Type**: Create new test file
|
||||
- **Purpose**: Test error handling middleware
|
||||
- **Scenarios**: 8 test cases for different error types and response formats
|
||||
- **Dependencies**: Mock Express req/res/next, error fixtures
|
||||
|
||||
[List all test files to create]
|
||||
|
||||
## 7. Success Metrics
|
||||
- **Test Coverage Goal**: {target_percentage}%
|
||||
- **Test Quality**: All scenarios covered (happy, error, edge, integration)
|
||||
- **Convention Compliance**: Follow existing test patterns
|
||||
- **Maintainability**: Clear test descriptions, reusable fixtures
|
||||
|
||||
RULES:
|
||||
- Focus on TEST REQUIREMENTS and GENERATION STRATEGY, NOT code generation
|
||||
- Study existing test patterns thoroughly for consistency
|
||||
- Prioritize critical business logic tests
|
||||
- Specify clear test scenarios and coverage targets
|
||||
- Identify all dependencies requiring mocks
|
||||
- **MUST write output to .workflow/{test_session_id}/.process/gemini-test-analysis.md**
|
||||
- Do NOT generate actual test code or implementation
|
||||
- Output ONLY test analysis and generation strategy
|
||||
" --approval-mode yolo
|
||||
```
|
||||
|
||||
**Output Location**: `.workflow/{test_session_id}/.process/gemini-test-analysis.md`
|
||||
|
||||
### Phase 3: Results Synthesis
|
||||
|
||||
1. **Output Validation**
|
||||
- Verify `gemini-test-analysis.md` exists and is complete
|
||||
- Validate all required sections present
|
||||
- Check test requirements are actionable
|
||||
|
||||
2. **Quality Assessment**
|
||||
- Test scenarios cover happy path, errors, edge cases
|
||||
- Dependencies and mocks clearly identified
|
||||
- Test generation strategy is practical
|
||||
- Coverage targets are reasonable
|
||||
|
||||
### Phase 4: TEST_ANALYSIS_RESULTS.md Generation
|
||||
|
||||
Synthesize Gemini analysis into standardized format:
|
||||
|
||||
```markdown
|
||||
# Test Generation Analysis Results
|
||||
|
||||
## Executive Summary
|
||||
- **Test Session**: {test_session_id}
|
||||
- **Source Session**: {source_session_id}
|
||||
- **Analysis Timestamp**: {timestamp}
|
||||
- **Coverage Gap**: {missing_test_count} files require tests
|
||||
- **Test Framework**: {framework}
|
||||
- **Overall Strategy**: {high_level_approach}
|
||||
|
||||
---
|
||||
|
||||
## 1. Coverage Assessment
|
||||
|
||||
### Current Coverage
|
||||
- **Existing Tests**: {count} files
|
||||
- **Implementation Files**: {count} files
|
||||
- **Coverage Percentage**: {percentage}%
|
||||
|
||||
### Missing Tests (Priority Order)
|
||||
1. **High Priority** ({count} files)
|
||||
- {file_1} - {reason}
|
||||
- {file_2} - {reason}
|
||||
|
||||
2. **Medium Priority** ({count} files)
|
||||
- {file_1} - {reason}
|
||||
|
||||
3. **Low Priority** ({count} files)
|
||||
- {file_1} - {reason}
|
||||
|
||||
---
|
||||
|
||||
## 2. Test Framework & Conventions
|
||||
|
||||
### Framework Configuration
|
||||
- **Framework**: {framework_name}
|
||||
- **Version**: {version}
|
||||
- **Test Pattern**: {file_pattern}
|
||||
- **Test Directory**: {directory_structure}
|
||||
|
||||
### Conventions
|
||||
- **File Naming**: {convention}
|
||||
- **Test Structure**: {describe_it_blocks}
|
||||
- **Assertions**: {assertion_library}
|
||||
- **Mocking**: {mocking_framework}
|
||||
- **Setup/Teardown**: {beforeEach_afterEach}
|
||||
|
||||
### Example Pattern (from existing tests)
|
||||
```
|
||||
{example_test_structure_from_analysis}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Test Requirements by File
|
||||
|
||||
[For each missing test, include:]
|
||||
|
||||
### Test File: {test_file_path}
|
||||
**Implementation**: {implementation_file}
|
||||
**Priority**: {high|medium|low}
|
||||
**Estimated Test Count**: {count}
|
||||
|
||||
#### Test Scenarios
|
||||
1. **Happy Path**: {scenarios}
|
||||
2. **Error Handling**: {scenarios}
|
||||
3. **Edge Cases**: {scenarios}
|
||||
4. **Integration**: {scenarios}
|
||||
|
||||
#### Dependencies & Mocks
|
||||
- {dependency_1_to_mock}
|
||||
- {dependency_2_to_mock}
|
||||
|
||||
#### Test Data Requirements
|
||||
- {fixture_1}
|
||||
- {fixture_2}
|
||||
|
||||
---
|
||||
|
||||
## 4. Test Generation Strategy
|
||||
|
||||
### Generation Approach
|
||||
{overall_strategy_description}
|
||||
|
||||
### Generation Order
|
||||
1. {test_file_1} - {rationale}
|
||||
2. {test_file_2} - {rationale}
|
||||
3. {test_file_3} - {rationale}
|
||||
|
||||
### Reusable Components
|
||||
- **Test Fixtures**: {common_fixtures}
|
||||
- **Mock Patterns**: {common_mocks}
|
||||
- **Helper Functions**: {test_helpers}
|
||||
|
||||
### Quality Targets
|
||||
- **Coverage Goal**: {percentage}%
|
||||
- **Scenarios per Function**: {min_count}
|
||||
- **Integration Coverage**: {approach}
|
||||
|
||||
---
|
||||
|
||||
## 5. Implementation Targets
|
||||
|
||||
**Purpose**: New test files to create (code-developer will generate these)
|
||||
|
||||
**Test Files to Create**:
|
||||
|
||||
1. **Target**: `tests/auth/TokenValidator.test.ts`
|
||||
- **Implementation Source**: `src/auth/TokenValidator.ts`
|
||||
- **Test Scenarios**: 15 (validation, error handling, edge cases)
|
||||
- **Dependencies**: Mock JWT library, token fixtures
|
||||
- **Priority**: High
|
||||
|
||||
2. **Target**: `tests/middleware/errorHandler.test.ts`
|
||||
- **Implementation Source**: `src/middleware/errorHandler.ts`
|
||||
- **Test Scenarios**: 8 (error types, response formats)
|
||||
- **Dependencies**: Mock Express, error fixtures
|
||||
- **Priority**: High
|
||||
|
||||
[List all test files with full specifications]
|
||||
|
||||
---
|
||||
|
||||
## 6. Success Criteria
|
||||
|
||||
### Coverage Metrics
|
||||
- Achieve {target_percentage}% code coverage
|
||||
- All public APIs have tests
|
||||
- Critical paths fully covered
|
||||
|
||||
### Quality Standards
|
||||
- All test scenarios covered (happy, error, edge, integration)
|
||||
- Follow existing test conventions
|
||||
- Clear test descriptions and assertions
|
||||
- Maintainable test structure
|
||||
|
||||
### Validation Approach
|
||||
- Run full test suite after generation
|
||||
- Verify coverage with coverage tool
|
||||
- Manual review of test quality
|
||||
- Integration test validation
|
||||
|
||||
---
|
||||
|
||||
## 7. Reference Information
|
||||
|
||||
### Source Context
|
||||
- **Implementation Summaries**: {paths}
|
||||
- **Existing Tests**: {example_tests}
|
||||
- **Documentation**: {relevant_docs}
|
||||
|
||||
### Analysis Tools
|
||||
- **Gemini Analysis**: gemini-test-analysis.md
|
||||
- **Coverage Tools**: {coverage_tool_if_detected}
|
||||
```
|
||||
|
||||
**Output Location**: `.workflow/{test_session_id}/.process/TEST_ANALYSIS_RESULTS.md`
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Validation Errors
|
||||
| Error | Cause | Resolution |
|
||||
|-------|-------|------------|
|
||||
| Missing context package | test-context-gather not run | Run test-context-gather first |
|
||||
| No coverage gaps | All files have tests | Skip test generation, proceed to test execution |
|
||||
| No test framework detected | Missing test dependencies | Request user to configure test framework |
|
||||
| Invalid source session | Source session incomplete | Complete implementation first |
|
||||
|
||||
### Gemini Execution Errors
|
||||
| Error | Cause | Recovery |
|
||||
|-------|-------|----------|
|
||||
| Timeout | Large project analysis | Reduce scope, analyze by module |
|
||||
| Output incomplete | Token limit exceeded | Retry with focused analysis |
|
||||
| No output file | Write permission error | Check directory permissions |
|
||||
|
||||
### Fallback Strategy
|
||||
- If Gemini fails, generate basic TEST_ANALYSIS_RESULTS.md from context package
|
||||
- Use coverage gaps and framework info to create minimal requirements
|
||||
- Provide guidance for manual test planning
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
- **Focused Analysis**: Only analyze files with missing tests
|
||||
- **Pattern Reuse**: Study existing tests for quick pattern extraction
|
||||
- **Parallel Operations**: Load implementation summaries in parallel
|
||||
- **Timeout Management**: 20-minute limit for Gemini analysis
|
||||
|
||||
## Integration
|
||||
|
||||
### Called By
|
||||
- `/workflow:test-gen` (Phase 4: Analysis)
|
||||
|
||||
### Requires
|
||||
- `/workflow:tools:test-context-gather` output (test-context-package.json)
|
||||
|
||||
### Followed By
|
||||
- `/workflow:tools:test-task-generate` - Generates test task JSON with code-developer invocation
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- ✅ Valid TEST_ANALYSIS_RESULTS.md generated
|
||||
- ✅ All missing tests documented with requirements
|
||||
- ✅ Test scenarios cover happy path, errors, edge cases
|
||||
- ✅ Dependencies and mocks identified
|
||||
- ✅ Test generation strategy is actionable
|
||||
- ✅ Execution time < 20 minutes
|
||||
- ✅ Output follows existing test conventions
|
||||
|
||||
## Related Commands
|
||||
|
||||
- `/workflow:tools:test-context-gather` - Provides input context
|
||||
- `/workflow:tools:test-task-generate` - Consumes analysis results
|
||||
- `/workflow:test-gen` - Main test generation workflow
|
||||
310
.claude/commands/workflow/tools/test-context-gather.md
Normal file
310
.claude/commands/workflow/tools/test-context-gather.md
Normal file
@@ -0,0 +1,310 @@
|
||||
---
|
||||
name: test-context-gather
|
||||
description: Collect test coverage context and identify files requiring test generation
|
||||
usage: /workflow:tools:test-context-gather --session <test_session_id>
|
||||
argument-hint: "--session WFS-test-session-id"
|
||||
examples:
|
||||
- /workflow:tools:test-context-gather --session WFS-test-auth
|
||||
- /workflow:tools:test-context-gather --session WFS-test-payment
|
||||
---
|
||||
|
||||
# Test Context Gather Command
|
||||
|
||||
## Overview
|
||||
Specialized context collector for test generation workflows that analyzes test coverage, identifies missing tests, and packages implementation context from source sessions.
|
||||
|
||||
## Core Philosophy
|
||||
- **Coverage-First**: Analyze existing test coverage before planning
|
||||
- **Gap Identification**: Locate implementation files without corresponding tests
|
||||
- **Source Context Loading**: Import implementation summaries from source session
|
||||
- **Framework Detection**: Auto-detect test framework and patterns
|
||||
- **MCP-Powered**: Leverage code-index tools for precise analysis
|
||||
|
||||
## Core Responsibilities
|
||||
- Load source session implementation context
|
||||
- Analyze current test coverage using MCP tools
|
||||
- Identify files requiring test generation
|
||||
- Detect test framework and conventions
|
||||
- Package test context for analysis phase
|
||||
|
||||
## Execution Lifecycle
|
||||
|
||||
### Phase 1: Session Validation & Source Loading
|
||||
|
||||
1. **Test Session Validation**
|
||||
- Load `.workflow/{test_session_id}/workflow-session.json`
|
||||
- Extract `meta.source_session` reference
|
||||
- Validate test session type is "test-gen"
|
||||
|
||||
2. **Source Session Context Loading**
|
||||
- Read `.workflow/{source_session_id}/workflow-session.json`
|
||||
- Load implementation summaries from `.workflow/{source_session_id}/.summaries/`
|
||||
- Extract changed files and implementation scope
|
||||
- Identify implementation patterns and tech stack
|
||||
|
||||
### Phase 2: Test Coverage Analysis (MCP Tools)
|
||||
|
||||
1. **Existing Test Discovery**
|
||||
```bash
|
||||
# Find all test files
|
||||
mcp__code-index__find_files(pattern="*.test.*")
|
||||
mcp__code-index__find_files(pattern="*.spec.*")
|
||||
mcp__code-index__find_files(pattern="*test_*.py")
|
||||
|
||||
# Search for test patterns
|
||||
mcp__code-index__search_code_advanced(
|
||||
pattern="describe|it|test|@Test",
|
||||
file_pattern="*.test.*",
|
||||
context_lines=0
|
||||
)
|
||||
```
|
||||
|
||||
2. **Coverage Gap Analysis**
|
||||
```bash
|
||||
# For each implementation file from source session
|
||||
# Check if corresponding test file exists
|
||||
|
||||
# Example: src/auth/AuthService.ts -> tests/auth/AuthService.test.ts
|
||||
# src/utils/validator.py -> tests/test_validator.py
|
||||
|
||||
# Output: List of files without tests
|
||||
```
|
||||
|
||||
3. **Test Statistics**
|
||||
- Count total test files
|
||||
- Count implementation files from source session
|
||||
- Calculate coverage percentage
|
||||
- Identify coverage gaps by module
|
||||
|
||||
### Phase 3: Test Framework Detection
|
||||
|
||||
1. **Framework Identification**
|
||||
```bash
|
||||
# Check package.json or requirements.txt
|
||||
mcp__code-index__search_code_advanced(
|
||||
pattern="jest|mocha|jasmine|pytest|unittest|rspec",
|
||||
file_pattern="package.json|requirements.txt|Gemfile",
|
||||
context_lines=2
|
||||
)
|
||||
|
||||
# Analyze existing test patterns
|
||||
mcp__code-index__search_code_advanced(
|
||||
pattern="describe\\(|it\\(|test\\(|def test_",
|
||||
file_pattern="*.test.*",
|
||||
context_lines=3
|
||||
)
|
||||
```
|
||||
|
||||
2. **Convention Analysis**
|
||||
- Test file naming patterns (*.test.ts vs *.spec.ts)
|
||||
- Test directory structure (tests/ vs __tests__ vs src/**/*.test.*)
|
||||
- Assertion library (expect, assert, should)
|
||||
- Mocking framework (jest.fn, sinon, unittest.mock)
|
||||
|
||||
### Phase 4: Context Packaging
|
||||
|
||||
Generate `test-context-package.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"metadata": {
|
||||
"test_session_id": "WFS-test-auth",
|
||||
"source_session_id": "WFS-auth",
|
||||
"timestamp": "2025-10-04T10:30:00Z",
|
||||
"task_type": "test-generation",
|
||||
"complexity": "medium"
|
||||
},
|
||||
"source_context": {
|
||||
"implementation_summaries": [
|
||||
{
|
||||
"task_id": "IMPL-001",
|
||||
"summary_path": ".workflow/WFS-auth/.summaries/IMPL-001-summary.md",
|
||||
"changed_files": [
|
||||
"src/auth/AuthService.ts",
|
||||
"src/auth/TokenValidator.ts",
|
||||
"src/middleware/auth.ts"
|
||||
],
|
||||
"implementation_type": "feature"
|
||||
}
|
||||
],
|
||||
"tech_stack": ["typescript", "express", "jsonwebtoken"],
|
||||
"project_patterns": {
|
||||
"architecture": "layered",
|
||||
"error_handling": "try-catch with custom errors",
|
||||
"async_pattern": "async/await"
|
||||
}
|
||||
},
|
||||
"test_coverage": {
|
||||
"existing_tests": [
|
||||
"tests/auth/AuthService.test.ts",
|
||||
"tests/middleware/auth.test.ts"
|
||||
],
|
||||
"missing_tests": [
|
||||
{
|
||||
"implementation_file": "src/auth/TokenValidator.ts",
|
||||
"suggested_test_file": "tests/auth/TokenValidator.test.ts",
|
||||
"priority": "high",
|
||||
"reason": "New implementation without tests"
|
||||
}
|
||||
],
|
||||
"coverage_stats": {
|
||||
"total_implementation_files": 3,
|
||||
"files_with_tests": 2,
|
||||
"files_without_tests": 1,
|
||||
"coverage_percentage": 66.7
|
||||
}
|
||||
},
|
||||
"test_framework": {
|
||||
"framework": "jest",
|
||||
"version": "^29.0.0",
|
||||
"test_pattern": "**/*.test.ts",
|
||||
"test_directory": "tests/",
|
||||
"assertion_library": "expect",
|
||||
"mocking_framework": "jest",
|
||||
"conventions": {
|
||||
"file_naming": "*.test.ts",
|
||||
"test_structure": "describe/it blocks",
|
||||
"setup_teardown": "beforeEach/afterEach"
|
||||
}
|
||||
},
|
||||
"assets": [
|
||||
{
|
||||
"type": "implementation_summary",
|
||||
"path": ".workflow/WFS-auth/.summaries/IMPL-001-summary.md",
|
||||
"relevance": "Source implementation context",
|
||||
"priority": "highest"
|
||||
},
|
||||
{
|
||||
"type": "existing_test",
|
||||
"path": "tests/auth/AuthService.test.ts",
|
||||
"relevance": "Test pattern reference",
|
||||
"priority": "high"
|
||||
},
|
||||
{
|
||||
"type": "source_code",
|
||||
"path": "src/auth/TokenValidator.ts",
|
||||
"relevance": "Implementation requiring tests",
|
||||
"priority": "high"
|
||||
},
|
||||
{
|
||||
"type": "documentation",
|
||||
"path": "CLAUDE.md",
|
||||
"relevance": "Project conventions",
|
||||
"priority": "medium"
|
||||
}
|
||||
],
|
||||
"focus_areas": [
|
||||
"Generate comprehensive tests for TokenValidator",
|
||||
"Follow existing Jest patterns from AuthService tests",
|
||||
"Cover happy path, error cases, and edge cases",
|
||||
"Include integration tests for middleware"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Output Location
|
||||
|
||||
```
|
||||
.workflow/{test_session_id}/.process/test-context-package.json
|
||||
```
|
||||
|
||||
## MCP Tools Usage
|
||||
|
||||
### File Discovery
|
||||
```bash
|
||||
# Test files
|
||||
mcp__code-index__find_files(pattern="*.test.*")
|
||||
mcp__code-index__find_files(pattern="*.spec.*")
|
||||
|
||||
# Implementation files
|
||||
mcp__code-index__find_files(pattern="*.ts")
|
||||
mcp__code-index__find_files(pattern="*.js")
|
||||
```
|
||||
|
||||
### Content Search
|
||||
```bash
|
||||
# Test framework detection
|
||||
mcp__code-index__search_code_advanced(
|
||||
pattern="jest|mocha|pytest",
|
||||
file_pattern="package.json|requirements.txt"
|
||||
)
|
||||
|
||||
# Test pattern analysis
|
||||
mcp__code-index__search_code_advanced(
|
||||
pattern="describe|it|test",
|
||||
file_pattern="*.test.*",
|
||||
context_lines=2
|
||||
)
|
||||
```
|
||||
|
||||
### Coverage Analysis
|
||||
```bash
|
||||
# For each implementation file
|
||||
# Check if test exists
|
||||
implementation_file="src/auth/AuthService.ts"
|
||||
test_file_patterns=(
|
||||
"tests/auth/AuthService.test.ts"
|
||||
"src/auth/AuthService.test.ts"
|
||||
"src/auth/__tests__/AuthService.test.ts"
|
||||
)
|
||||
|
||||
# Search for test file
|
||||
for pattern in "${test_file_patterns[@]}"; do
|
||||
if mcp__code-index__find_files(pattern="$pattern") | grep -q .; then
|
||||
echo "✅ Test exists: $pattern"
|
||||
break
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Error | Cause | Resolution |
|
||||
|-------|-------|------------|
|
||||
| Source session not found | Invalid source_session reference | Verify test session metadata |
|
||||
| No implementation summaries | Source session incomplete | Complete source session first |
|
||||
| MCP tools unavailable | MCP not configured | Fallback to bash find/grep |
|
||||
| No test framework detected | Missing test dependencies | Request user to specify framework |
|
||||
|
||||
## Fallback Strategy (No MCP)
|
||||
|
||||
```bash
|
||||
# File discovery
|
||||
find . -name "*.test.*" -o -name "*.spec.*" | grep -v node_modules
|
||||
|
||||
# Framework detection
|
||||
grep -r "jest\|mocha\|pytest" package.json requirements.txt 2>/dev/null
|
||||
|
||||
# Coverage analysis
|
||||
for impl_file in $(cat changed_files.txt); do
|
||||
test_file=$(echo $impl_file | sed 's/src/tests/' | sed 's/\(.*\)\.\(ts\|js\|py\)$/\1.test.\2/')
|
||||
[ ! -f "$test_file" ] && echo "$impl_file → MISSING TEST"
|
||||
done
|
||||
```
|
||||
|
||||
## Integration
|
||||
|
||||
### Called By
|
||||
- `/workflow:test-gen` (Phase 3: Context Gathering)
|
||||
|
||||
### Calls
|
||||
- MCP code-index tools for analysis
|
||||
- Bash file operations for fallback
|
||||
|
||||
### Followed By
|
||||
- `/workflow:tools:test-concept-enhanced` - Analyzes context and plans test generation
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- ✅ Source session context loaded successfully
|
||||
- ✅ Test coverage gaps identified with MCP tools
|
||||
- ✅ Test framework detected and documented
|
||||
- ✅ Valid test-context-package.json generated
|
||||
- ✅ All missing tests catalogued with priority
|
||||
- ✅ Execution time < 20 seconds
|
||||
|
||||
## Related Commands
|
||||
|
||||
- `/workflow:test-gen` - Main test generation workflow
|
||||
- `/workflow:tools:test-concept-enhanced` - Test generation analysis
|
||||
- `/workflow:tools:test-task-generate` - Test task JSON generation
|
||||
624
.claude/commands/workflow/tools/test-task-generate.md
Normal file
624
.claude/commands/workflow/tools/test-task-generate.md
Normal file
@@ -0,0 +1,624 @@
|
||||
---
|
||||
name: test-task-generate
|
||||
description: Generate test-fix task JSON with iterative test-fix-retest cycle specification
|
||||
usage: /workflow:tools:test-task-generate --session <test-session-id>
|
||||
argument-hint: "--session WFS-test-session-id"
|
||||
examples:
|
||||
- /workflow:tools:test-task-generate --session WFS-test-auth
|
||||
---
|
||||
|
||||
# Test Task Generation Command
|
||||
|
||||
## Overview
|
||||
Generate specialized test-fix task JSON with comprehensive test-fix-retest cycle specification, including Gemini diagnosis and Codex resume mechanism for intelligent failure resolution.
|
||||
|
||||
## Core Philosophy
|
||||
- **Analysis-Driven Test Generation**: Use TEST_ANALYSIS_RESULTS.md from test-concept-enhanced
|
||||
- **Agent-Based Test Creation**: Call @code-developer agent for comprehensive test generation
|
||||
- **Coverage-First**: Generate all missing tests before execution
|
||||
- **Test Execution**: Execute complete test suite after generation
|
||||
- **Gemini Diagnosis**: Use Gemini for root cause analysis and fix suggestions
|
||||
- **Codex Resume**: Apply fixes using `codex exec "..." resume --last` for context continuity
|
||||
- **Iterative Refinement**: Repeat test-analyze-fix-retest cycle until all tests pass
|
||||
- **Surgical Fixes**: Minimal code changes, no refactoring during test fixes
|
||||
- **Auto-Revert**: Rollback all changes if max iterations reached
|
||||
|
||||
## Core Responsibilities
|
||||
- Parse TEST_ANALYSIS_RESULTS.md from test-concept-enhanced
|
||||
- Extract test requirements and generation strategy
|
||||
- Generate test generation subtask calling @code-developer
|
||||
- Generate test execution and fix cycle task JSON
|
||||
- Configure Gemini diagnosis and Codex execution workflow
|
||||
- Create test-oriented IMPL_PLAN.md and TODO_LIST.md with test generation phase
|
||||
|
||||
## Execution Lifecycle
|
||||
|
||||
### Phase 1: Input Validation & Discovery
|
||||
|
||||
1. **Test Session Validation**
|
||||
- Load `.workflow/{test-session-id}/workflow-session.json`
|
||||
- Verify `workflow_type: "test_session"`
|
||||
- Extract `source_session_id` from metadata
|
||||
|
||||
2. **Test Analysis Results Loading**
|
||||
- **REQUIRED**: Load `.workflow/{test-session-id}/.process/TEST_ANALYSIS_RESULTS.md`
|
||||
- Parse test requirements by file
|
||||
- Extract test generation strategy
|
||||
- Identify test files to create with specifications
|
||||
|
||||
3. **Test Context Package Loading**
|
||||
- Load `.workflow/{test-session-id}/.process/test-context-package.json`
|
||||
- Extract test framework configuration
|
||||
- Extract coverage gaps and priorities
|
||||
- Load source session implementation summaries
|
||||
|
||||
### Phase 2: Task JSON Generation
|
||||
|
||||
Generate **TWO task JSON files**:
|
||||
1. **IMPL-001.json** - Test Generation (calls @code-developer)
|
||||
2. **IMPL-002.json** - Test Execution and Fix Cycle (calls @test-fix-agent)
|
||||
|
||||
#### IMPL-001.json - Test Generation Task
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "IMPL-001",
|
||||
"title": "Generate comprehensive tests for [sourceSessionId]",
|
||||
"status": "pending",
|
||||
"meta": {
|
||||
"type": "test-gen",
|
||||
"agent": "@code-developer",
|
||||
"source_session": "[sourceSessionId]",
|
||||
"test_framework": "jest|pytest|cargo|detected"
|
||||
},
|
||||
"context": {
|
||||
"requirements": [
|
||||
"Generate comprehensive test files based on TEST_ANALYSIS_RESULTS.md",
|
||||
"Follow existing test patterns and conventions from test framework",
|
||||
"Create tests for all missing coverage identified in analysis",
|
||||
"Include happy path, error handling, edge cases, and integration tests",
|
||||
"Use test data and mocks as specified in analysis",
|
||||
"Ensure tests follow project coding standards"
|
||||
],
|
||||
"focus_paths": [
|
||||
"tests/**/*",
|
||||
"src/**/*.test.*",
|
||||
"{paths_from_analysis}"
|
||||
],
|
||||
"acceptance": [
|
||||
"All test files from TEST_ANALYSIS_RESULTS.md section 5 are created",
|
||||
"Tests follow existing test patterns and conventions",
|
||||
"Test scenarios cover happy path, errors, edge cases, integration",
|
||||
"All dependencies are properly mocked",
|
||||
"Test files are syntactically valid and can be executed",
|
||||
"Test coverage meets analysis requirements"
|
||||
],
|
||||
"depends_on": [],
|
||||
"source_context": {
|
||||
"session_id": "[sourceSessionId]",
|
||||
"test_analysis": ".workflow/[testSessionId]/.process/TEST_ANALYSIS_RESULTS.md",
|
||||
"test_context": ".workflow/[testSessionId]/.process/test-context-package.json",
|
||||
"implementation_summaries": [
|
||||
".workflow/[sourceSessionId]/.summaries/IMPL-001-summary.md"
|
||||
]
|
||||
}
|
||||
},
|
||||
"flow_control": {
|
||||
"pre_analysis": [
|
||||
{
|
||||
"step": "load_test_analysis",
|
||||
"action": "Load test generation requirements and strategy",
|
||||
"commands": [
|
||||
"Read(.workflow/[testSessionId]/.process/TEST_ANALYSIS_RESULTS.md)",
|
||||
"Read(.workflow/[testSessionId]/.process/test-context-package.json)"
|
||||
],
|
||||
"output_to": "test_generation_requirements",
|
||||
"on_error": "fail"
|
||||
},
|
||||
{
|
||||
"step": "load_implementation_context",
|
||||
"action": "Load source implementation for test generation context",
|
||||
"commands": [
|
||||
"bash(for f in .workflow/[sourceSessionId]/.summaries/IMPL-*-summary.md; do echo \"=== $(basename $f) ===\"&& cat \"$f\"; done)"
|
||||
],
|
||||
"output_to": "implementation_context",
|
||||
"on_error": "skip_optional"
|
||||
},
|
||||
{
|
||||
"step": "load_existing_test_patterns",
|
||||
"action": "Study existing tests for pattern reference",
|
||||
"commands": [
|
||||
"mcp__code-index__find_files(pattern=\"*.test.*\")",
|
||||
"bash(# Read first 2 existing test files as examples)",
|
||||
"bash(test_files=$(mcp__code-index__find_files(pattern=\"*.test.*\") | head -2))",
|
||||
"bash(for f in $test_files; do echo \"=== $f ===\"&& cat \"$f\"; done)"
|
||||
],
|
||||
"output_to": "existing_test_patterns",
|
||||
"on_error": "skip_optional"
|
||||
}
|
||||
],
|
||||
"implementation_approach": {
|
||||
"task_description": "Generate comprehensive test suite based on TEST_ANALYSIS_RESULTS.md. Follow test generation strategy and create all test files listed in section 5 (Implementation Targets).",
|
||||
"generation_steps": [
|
||||
"Read TEST_ANALYSIS_RESULTS.md section 3 (Test Requirements by File)",
|
||||
"Read TEST_ANALYSIS_RESULTS.md section 4 (Test Generation Strategy)",
|
||||
"Study existing test patterns from test_context.test_framework.conventions",
|
||||
"For each test file in section 5 (Implementation Targets):",
|
||||
" - Create test file with specified scenarios",
|
||||
" - Implement happy path tests",
|
||||
" - Implement error handling tests",
|
||||
" - Implement edge case tests",
|
||||
" - Implement integration tests (if specified)",
|
||||
" - Add required mocks and fixtures",
|
||||
"Follow test framework conventions and project standards",
|
||||
"Ensure all tests are executable and syntactically valid"
|
||||
],
|
||||
"quality_criteria": [
|
||||
"All test scenarios from analysis are implemented",
|
||||
"Test structure matches existing patterns",
|
||||
"Clear test descriptions and assertions",
|
||||
"Proper setup/teardown and fixtures",
|
||||
"Dependencies properly mocked",
|
||||
"Tests follow project coding standards"
|
||||
]
|
||||
},
|
||||
"target_files": [
|
||||
"{test_file_1 from TEST_ANALYSIS_RESULTS.md section 5}",
|
||||
"{test_file_2 from TEST_ANALYSIS_RESULTS.md section 5}",
|
||||
"{test_file_N from TEST_ANALYSIS_RESULTS.md section 5}"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### IMPL-002.json - Test Execution & Fix Cycle Task
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "IMPL-002",
|
||||
"title": "Execute and fix tests for [sourceSessionId]",
|
||||
"status": "pending",
|
||||
"meta": {
|
||||
"type": "test-fix",
|
||||
"agent": "@test-fix-agent",
|
||||
"source_session": "[sourceSessionId]",
|
||||
"test_framework": "jest|pytest|cargo|detected",
|
||||
"max_iterations": 5
|
||||
},
|
||||
"context": {
|
||||
"requirements": [
|
||||
"Execute complete test suite (generated in IMPL-001)",
|
||||
"Diagnose test failures using Gemini analysis",
|
||||
"Fix failures using Codex with resume mechanism",
|
||||
"Iterate until all tests pass or max iterations reached",
|
||||
"Revert changes if unable to fix within iteration limit"
|
||||
],
|
||||
"focus_paths": [
|
||||
"tests/**/*",
|
||||
"src/**/*.test.*",
|
||||
"{implementation_files_from_source_session}"
|
||||
],
|
||||
"acceptance": [
|
||||
"All tests pass successfully (100% pass rate)",
|
||||
"No test failures or errors in final run",
|
||||
"Code changes are minimal and surgical",
|
||||
"All fixes are verified through retest",
|
||||
"Iteration logs document fix progression"
|
||||
],
|
||||
"depends_on": ["IMPL-001"],
|
||||
"source_context": {
|
||||
"session_id": "[sourceSessionId]",
|
||||
"test_generation_summary": ".workflow/[testSessionId]/.summaries/IMPL-001-summary.md",
|
||||
"implementation_summaries": [
|
||||
".workflow/[sourceSessionId]/.summaries/IMPL-001-summary.md"
|
||||
]
|
||||
}
|
||||
},
|
||||
"flow_control": {
|
||||
"pre_analysis": [
|
||||
{
|
||||
"step": "load_source_session_summaries",
|
||||
"action": "Load implementation context from source session",
|
||||
"commands": [
|
||||
"bash(find .workflow/[sourceSessionId]/.summaries/ -name 'IMPL-*-summary.md' 2>/dev/null)",
|
||||
"bash(for f in .workflow/[sourceSessionId]/.summaries/IMPL-*-summary.md; do echo \"=== $(basename $f) ===\"&& cat \"$f\"; done)"
|
||||
],
|
||||
"output_to": "implementation_context",
|
||||
"on_error": "skip_optional"
|
||||
},
|
||||
{
|
||||
"step": "discover_test_framework",
|
||||
"action": "Identify test framework and test command",
|
||||
"commands": [
|
||||
"bash(jq -r '.scripts.test // \"npm test\"' package.json 2>/dev/null || echo 'pytest' || echo 'cargo test')",
|
||||
"bash([ -f 'package.json' ] && echo 'jest/npm' || [ -f 'pytest.ini' ] && echo 'pytest' || [ -f 'Cargo.toml' ] && echo 'cargo' || echo 'unknown')"
|
||||
],
|
||||
"output_to": "test_command",
|
||||
"on_error": "fail"
|
||||
},
|
||||
{
|
||||
"step": "analyze_test_coverage",
|
||||
"action": "Analyze test coverage and identify missing tests",
|
||||
"commands": [
|
||||
"mcp__code-index__find_files(pattern=\"*.test.*\")",
|
||||
"mcp__code-index__search_code_advanced(pattern=\"test|describe|it|def test_\", file_pattern=\"*.test.*\")",
|
||||
"bash(# Count implementation files vs test files)",
|
||||
"bash(impl_count=$(find [changed_files_dirs] -type f \\( -name '*.ts' -o -name '*.js' -o -name '*.py' \\) ! -name '*.test.*' 2>/dev/null | wc -l))",
|
||||
"bash(test_count=$(mcp__code-index__find_files(pattern=\"*.test.*\") | wc -l))",
|
||||
"bash(echo \"Implementation files: $impl_count, Test files: $test_count\")"
|
||||
],
|
||||
"output_to": "test_coverage_analysis",
|
||||
"on_error": "skip_optional"
|
||||
},
|
||||
{
|
||||
"step": "identify_files_without_tests",
|
||||
"action": "List implementation files that lack corresponding test files",
|
||||
"commands": [
|
||||
"bash(# For each changed file from source session, check if test exists)",
|
||||
"bash(for file in [changed_files]; do test_file=$(echo $file | sed 's/\\(.*\\)\\.\\(ts\\|js\\|py\\)$/\\1.test.\\2/'); [ ! -f \"$test_file\" ] && echo \"$file\"; done)"
|
||||
],
|
||||
"output_to": "files_without_tests",
|
||||
"on_error": "skip_optional"
|
||||
},
|
||||
{
|
||||
"step": "prepare_test_environment",
|
||||
"action": "Ensure test environment is ready",
|
||||
"commands": [
|
||||
"bash([ -f 'package.json' ] && npm install 2>/dev/null || true)",
|
||||
"bash([ -f 'requirements.txt' ] && pip install -q -r requirements.txt 2>/dev/null || true)"
|
||||
],
|
||||
"output_to": "environment_status",
|
||||
"on_error": "skip_optional"
|
||||
}
|
||||
],
|
||||
"implementation_approach": {
|
||||
"task_description": "Execute iterative test-fix-retest cycle using Gemini diagnosis and Codex execution",
|
||||
"test_fix_cycle": {
|
||||
"max_iterations": 5,
|
||||
"cycle_pattern": "test → gemini_diagnose → codex_fix_resume → retest",
|
||||
"tools": {
|
||||
"test_execution": "bash(test_command)",
|
||||
"diagnosis": "gemini-wrapper (MODE: analysis)",
|
||||
"fix_application": "codex exec resume --last (MODE: write)",
|
||||
"verification": "bash(test_command) + regression_check"
|
||||
},
|
||||
"exit_conditions": {
|
||||
"success": "all_tests_pass",
|
||||
"failure": "max_iterations_reached",
|
||||
"error": "test_command_not_found"
|
||||
}
|
||||
},
|
||||
"modification_points": [
|
||||
"PHASE 1: Initial Test Execution",
|
||||
" 1.1. Discover test command from framework detection",
|
||||
" 1.2. Execute initial test run: bash([test_command])",
|
||||
" 1.3. Parse test output and count failures",
|
||||
" 1.4. If all pass → Skip to PHASE 3 (success)",
|
||||
" 1.5. If failures → Store failure output, proceed to PHASE 2",
|
||||
"",
|
||||
"PHASE 2: Iterative Test-Fix-Retest Cycle (max 5 iterations)",
|
||||
" Note: This phase handles test failures, NOT test generation failures",
|
||||
" Initialize: max_iterations=5, current_iteration=0",
|
||||
" ",
|
||||
" WHILE (tests failing AND current_iteration < max_iterations):",
|
||||
" current_iteration++",
|
||||
" ",
|
||||
" STEP 2.1: Gemini Diagnosis",
|
||||
" - Prepare diagnosis context:",
|
||||
" * Test failure output from previous run",
|
||||
" * Source files from focus_paths",
|
||||
" * Implementation summaries from source session",
|
||||
" - Execute Gemini analysis:",
|
||||
" bash(cd .workflow/WFS-test-[session]/.process && ~/.claude/scripts/gemini-wrapper -p \"",
|
||||
" PURPOSE: Diagnose test failure iteration [N] and propose minimal fix",
|
||||
" TASK: Analyze test output and identify root cause with specific code change",
|
||||
" MODE: analysis",
|
||||
" CONTEXT: Test output: [test_failures]",
|
||||
" Source files: [focus_paths]",
|
||||
" Implementation: [implementation_context]",
|
||||
" EXPECTED: Root cause analysis with fix in format:",
|
||||
" FILE: path/to/file.ts",
|
||||
" FUNCTION: functionName",
|
||||
" LINES: 45-52",
|
||||
" CHANGE: Specific code modification to apply",
|
||||
" RULES: Minimal surgical fixes only - no refactoring",
|
||||
" Focus on test failure, not code quality",
|
||||
" \" > fix-iteration-[N]-diagnosis.md)",
|
||||
" - Parse diagnosis → extract fix_suggestion and target_files",
|
||||
" ",
|
||||
" STEP 2.2: Codex Fix with Resume Mechanism",
|
||||
" - Stage current changes (if valid git repo): bash(git add -A)",
|
||||
" - First iteration: Start new Codex session",
|
||||
" codex -C [project_root] --full-auto exec \"",
|
||||
" PURPOSE: Fix test failure iteration 1",
|
||||
" TASK: [fix_suggestion from Gemini]",
|
||||
" MODE: write",
|
||||
" CONTEXT: Diagnosis: .workflow/.process/fix-iteration-1-diagnosis.md",
|
||||
" Target files: [target_files]",
|
||||
" Implementation context: [implementation_context]",
|
||||
" EXPECTED: Minimal code changes to resolve test failure",
|
||||
" RULES: Apply ONLY suggested changes, no refactoring",
|
||||
" Preserve existing code style",
|
||||
" \" --skip-git-repo-check -s danger-full-access",
|
||||
" - Subsequent iterations: Resume session for context continuity",
|
||||
" codex exec \"",
|
||||
" CONTINUE TO NEXT FIX:",
|
||||
" Iteration [N] of 5: Fix test failure",
|
||||
" ",
|
||||
" PURPOSE: Fix remaining test failures",
|
||||
" TASK: [fix_suggestion from Gemini iteration N]",
|
||||
" CONTEXT: Previous fixes applied, diagnosis: .process/fix-iteration-[N]-diagnosis.md",
|
||||
" EXPECTED: Surgical fix for current failure",
|
||||
" RULES: Build on previous fixes, maintain consistency",
|
||||
" \" resume --last --skip-git-repo-check -s danger-full-access",
|
||||
" - Store fix log: .process/fix-iteration-[N]-changes.log",
|
||||
" ",
|
||||
" STEP 2.3: Retest and Verification",
|
||||
" - Re-execute test suite: bash([test_command])",
|
||||
" - Capture output: .process/fix-iteration-[N]-retest.log",
|
||||
" - Count failures: bash(grep -c 'FAIL\\|ERROR' .process/fix-iteration-[N]-retest.log)",
|
||||
" - Check for regression:",
|
||||
" IF new_failures > previous_failures:",
|
||||
" WARN: Regression detected",
|
||||
" Include in next Gemini diagnosis context",
|
||||
" - Analyze results:",
|
||||
" IF all_tests_pass:",
|
||||
" BREAK loop → Proceed to PHASE 3",
|
||||
" ELSE:",
|
||||
" Update test_failures context",
|
||||
" CONTINUE loop",
|
||||
" ",
|
||||
" IF max_iterations reached AND tests still failing:",
|
||||
" EXECUTE: git reset --hard HEAD (revert all changes)",
|
||||
" MARK: Task status = blocked",
|
||||
" GENERATE: Detailed failure report with iteration logs",
|
||||
" EXIT: Require manual intervention",
|
||||
"",
|
||||
"PHASE 3: Final Validation and Certification",
|
||||
" 3.1. Execute final confirmation test run",
|
||||
" 3.2. Generate success summary:",
|
||||
" - Iterations required: [current_iteration]",
|
||||
" - Fixes applied: [summary from iteration logs]",
|
||||
" - Test results: All passing ✅",
|
||||
" 3.3. Mark task status: completed",
|
||||
" 3.4. Update TODO_LIST.md: Mark as ✅",
|
||||
" 3.5. Certify code: APPROVED for deployment"
|
||||
],
|
||||
"logic_flow": [
|
||||
"Load source session implementation context",
|
||||
"Discover test framework and command",
|
||||
"PHASE 0: Test Coverage Check",
|
||||
" Analyze existing test files",
|
||||
" Identify files without tests",
|
||||
" IF tests missing:",
|
||||
" Generate tests using Codex",
|
||||
" Verify test generation successful",
|
||||
" ELSE:",
|
||||
" Skip to Phase 1",
|
||||
"PHASE 1: Initial Test Execution",
|
||||
" Execute test suite",
|
||||
" IF all pass → Success (Phase 3)",
|
||||
" ELSE → Store failures, proceed to Phase 2",
|
||||
"PHASE 2: Iterative Fix Cycle (max 5 iterations)",
|
||||
" LOOP (max 5 times):",
|
||||
" 1. Gemini diagnoses failure → fix suggestion",
|
||||
" 2. Codex applies fix (resume for continuity)",
|
||||
" 3. Retest and check results",
|
||||
" 4. IF pass → Exit loop to Phase 3",
|
||||
" 5. ELSE → Continue with updated context",
|
||||
" IF max iterations → Revert + report failure",
|
||||
"PHASE 3: Final Validation",
|
||||
" Confirm all tests pass",
|
||||
" Generate summary (include test generation info)",
|
||||
" Certify code APPROVED"
|
||||
],
|
||||
"error_handling": {
|
||||
"max_iterations_reached": {
|
||||
"action": "revert_all_changes",
|
||||
"commands": [
|
||||
"bash(git reset --hard HEAD)",
|
||||
"bash(jq '.status = \"blocked\"' .workflow/[session]/.task/IMPL-001.json > temp.json && mv temp.json .workflow/[session]/.task/IMPL-001.json)"
|
||||
],
|
||||
"report": "Generate failure report with iteration logs in .summaries/IMPL-001-failure-report.md"
|
||||
},
|
||||
"test_command_fails": {
|
||||
"action": "treat_as_test_failure",
|
||||
"context": "Use stderr as failure context for Gemini diagnosis"
|
||||
},
|
||||
"codex_apply_fails": {
|
||||
"action": "retry_once_then_skip",
|
||||
"fallback": "Mark iteration as skipped, continue to next"
|
||||
},
|
||||
"gemini_diagnosis_fails": {
|
||||
"action": "retry_with_simplified_context",
|
||||
"fallback": "Use previous diagnosis, continue"
|
||||
},
|
||||
"regression_detected": {
|
||||
"action": "log_warning_continue",
|
||||
"context": "Include regression info in next Gemini diagnosis"
|
||||
}
|
||||
}
|
||||
},
|
||||
"target_files": [
|
||||
"Auto-discovered from test failures",
|
||||
"Extracted from Gemini diagnosis each iteration",
|
||||
"Format: file:function:lines or file (for new files)"
|
||||
],
|
||||
"codex_session": {
|
||||
"strategy": "resume_for_continuity",
|
||||
"first_iteration": "codex exec \"fix iteration 1\" --full-auto",
|
||||
"subsequent_iterations": "codex exec \"fix iteration N\" resume --last",
|
||||
"benefits": [
|
||||
"Maintains conversation context across fixes",
|
||||
"Remembers previous decisions and patterns",
|
||||
"Ensures consistency in fix approach",
|
||||
"Reduces redundant context injection"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Phase 3: IMPL_PLAN.md Generation
|
||||
|
||||
#### Document Structure
|
||||
```markdown
|
||||
---
|
||||
identifier: WFS-test-[session-id]
|
||||
source_session: WFS-[source-session-id]
|
||||
workflow_type: test_session
|
||||
test_framework: jest|pytest|cargo|detected
|
||||
---
|
||||
|
||||
# Test Validation Plan: [Source Session Topic]
|
||||
|
||||
## Summary
|
||||
Execute comprehensive test suite for implementation from session WFS-[source-session-id].
|
||||
Diagnose and fix all test failures using iterative Gemini analysis and Codex execution.
|
||||
|
||||
## Source Session Context
|
||||
- **Implementation Session**: WFS-[source-session-id]
|
||||
- **Completed Tasks**: IMPL-001, IMPL-002, ...
|
||||
- **Changed Files**: [list from git log]
|
||||
- **Implementation Summaries**: [references to source session summaries]
|
||||
|
||||
## Test Framework
|
||||
- **Detected Framework**: jest|pytest|cargo|other
|
||||
- **Test Command**: npm test|pytest|cargo test
|
||||
- **Test Files**: [discovered test files]
|
||||
- **Coverage**: [estimated test coverage]
|
||||
|
||||
## Test-Fix-Retest Cycle
|
||||
- **Max Iterations**: 5
|
||||
- **Diagnosis Tool**: Gemini (analysis mode)
|
||||
- **Fix Tool**: Codex (resume mechanism)
|
||||
- **Verification**: Bash test execution + regression check
|
||||
|
||||
### Cycle Workflow
|
||||
1. **Initial Test**: Execute full suite, capture failures
|
||||
2. **Iterative Fix Loop** (max 5 times):
|
||||
- Gemini diagnoses failure → surgical fix suggestion
|
||||
- Codex applies fix using resume for context continuity
|
||||
- Retest and verify (check for regressions)
|
||||
- Continue until all pass or max iterations reached
|
||||
3. **Final Validation**: Confirm all tests pass, certify code
|
||||
|
||||
### Error Recovery
|
||||
- **Max iterations reached**: Revert all changes, report failure
|
||||
- **Test command fails**: Treat as test failure, diagnose with Gemini
|
||||
- **Codex fails**: Retry once, skip iteration if still failing
|
||||
- **Regression detected**: Log warning, include in next diagnosis
|
||||
|
||||
## Task Breakdown
|
||||
- **IMPL-001**: Execute and validate tests with iterative fix cycle
|
||||
|
||||
## Implementation Strategy
|
||||
- **Phase 1**: Initial test execution and failure capture
|
||||
- **Phase 2**: Iterative Gemini diagnosis + Codex fix + retest
|
||||
- **Phase 3**: Final validation and code certification
|
||||
|
||||
## Success Criteria
|
||||
- All tests pass (100% pass rate)
|
||||
- No test failures or errors in final run
|
||||
- Minimal, surgical code changes
|
||||
- Iteration logs document fix progression
|
||||
- Code certified APPROVED for deployment
|
||||
```
|
||||
|
||||
### Phase 4: TODO_LIST.md Generation
|
||||
|
||||
```markdown
|
||||
# Tasks: Test Validation for [Source Session]
|
||||
|
||||
## Task Progress
|
||||
- [ ] **IMPL-001**: Execute and validate tests with iterative fix cycle → [📋](./.task/IMPL-001.json)
|
||||
|
||||
## Execution Details
|
||||
- **Source Session**: WFS-[source-session-id]
|
||||
- **Test Framework**: jest|pytest|cargo
|
||||
- **Max Iterations**: 5
|
||||
- **Tools**: Gemini diagnosis + Codex resume fixes
|
||||
|
||||
## Status Legend
|
||||
- `- [ ]` = Pending
|
||||
- `- [x]` = Completed
|
||||
```
|
||||
|
||||
## Output Files Structure
|
||||
```
|
||||
.workflow/WFS-test-[session]/
|
||||
├── workflow-session.json # Test session metadata
|
||||
├── IMPL_PLAN.md # Test validation plan
|
||||
├── TODO_LIST.md # Progress tracking
|
||||
├── .task/
|
||||
│ └── IMPL-001.json # Test-fix task with cycle spec
|
||||
├── .process/
|
||||
│ ├── ANALYSIS_RESULTS.md # From concept-enhanced (optional)
|
||||
│ ├── context-package.json # From context-gather
|
||||
│ ├── initial-test.log # Phase 1: Initial test results
|
||||
│ ├── fix-iteration-1-diagnosis.md # Gemini diagnosis iteration 1
|
||||
│ ├── fix-iteration-1-changes.log # Codex changes iteration 1
|
||||
│ ├── fix-iteration-1-retest.log # Retest results iteration 1
|
||||
│ ├── fix-iteration-N-*.md/log # Subsequent iterations
|
||||
│ └── final-test.log # Phase 3: Final validation
|
||||
└── .summaries/
|
||||
└── IMPL-001-summary.md # Success report OR failure report
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Input Validation Errors
|
||||
| Error | Cause | Resolution |
|
||||
|-------|-------|------------|
|
||||
| Not a test session | Missing workflow_type: "test_session" | Verify session created by test-gen |
|
||||
| Source session not found | Invalid source_session_id | Check source session exists |
|
||||
| No implementation summaries | Source session incomplete | Ensure source session has completed tasks |
|
||||
|
||||
### Test Framework Discovery Errors
|
||||
| Error | Cause | Resolution |
|
||||
|-------|-------|------------|
|
||||
| No test command found | Unknown framework | Manual test command specification |
|
||||
| No test files found | Tests not written | Request user to write tests first |
|
||||
| Test dependencies missing | Incomplete setup | Run dependency installation |
|
||||
|
||||
### Generation Errors
|
||||
| Error | Cause | Resolution |
|
||||
|-------|-------|------------|
|
||||
| Invalid JSON structure | Template error | Fix task generation logic |
|
||||
| Missing required fields | Incomplete metadata | Validate session metadata |
|
||||
|
||||
## Integration & Usage
|
||||
|
||||
### Command Chain
|
||||
- **Called By**: `/workflow:test-gen` (Phase 4)
|
||||
- **Calls**: None (terminal command)
|
||||
- **Followed By**: `/workflow:execute` (user-triggered)
|
||||
|
||||
### Basic Usage
|
||||
```bash
|
||||
/workflow:tools:test-task-generate --session WFS-test-auth
|
||||
```
|
||||
|
||||
## Related Commands
|
||||
- `/workflow:test-gen` - Creates test session and calls this tool
|
||||
- `/workflow:tools:context-gather` - Provides cross-session context
|
||||
- `/workflow:tools:concept-enhanced` - Provides test strategy analysis
|
||||
- `/workflow:execute` - Executes the generated test-fix task
|
||||
- `@test-fix-agent` - Agent that executes the iterative test-fix cycle
|
||||
|
||||
## Agent Execution Notes
|
||||
|
||||
The `@test-fix-agent` will execute the task by following the `flow_control.implementation_approach` specification:
|
||||
|
||||
1. **Load task JSON**: Read complete test-fix task from `.task/IMPL-001.json`
|
||||
2. **Execute pre_analysis**: Load source context, discover framework, analyze tests
|
||||
3. **Phase 1**: Run initial test suite
|
||||
4. **Phase 2**: If failures, enter iterative loop:
|
||||
- Use Gemini for diagnosis (analysis mode)
|
||||
- Use Codex resume for fixes (maintains context)
|
||||
- Retest and check for regressions
|
||||
- Repeat max 5 times
|
||||
5. **Phase 3**: Generate summary and certify code
|
||||
6. **Error Recovery**: Revert changes if max iterations reached
|
||||
|
||||
The agent uses the `codex exec "..." resume --last` pattern to maintain conversation context across multiple fix iterations, ensuring consistency and learning from previous attempts.
|
||||
Reference in New Issue
Block a user