diff --git a/.claude/commands/workflow/test-gen.md b/.claude/commands/workflow/test-gen.md index c3aa1845..2f5f9915 100644 --- a/.claude/commands/workflow/test-gen.md +++ b/.claude/commands/workflow/test-gen.md @@ -37,7 +37,7 @@ allowed-tools: SlashCommand(*), TodoWrite(*), Read(*), Bash(*) 6. **Track Progress**: Update TodoWrite after every phase completion 7. **Automatic Detection**: context-gather auto-detects test session and gathers source session context -## 4-Phase Execution +## 5-Phase Execution ### Phase 1: Create Test Session **Command**: `SlashCommand(command="/workflow:session:start --new \"Test validation for [sourceSessionId]\"")` @@ -65,107 +65,114 @@ allowed-tools: SlashCommand(*), TodoWrite(*), Read(*), Bash(*) --- -### Phase 2: Gather Cross-Session Context -**Command**: `SlashCommand(command="/workflow:tools:context-gather --session [testSessionId]")` +### Phase 2: Gather Test Context +**Command**: `SlashCommand(command="/workflow:tools:test-context-gather --session [testSessionId]")` **Input**: `testSessionId` from Phase 1 (e.g., `WFS-test-user-auth`) -**Automatic Detection**: -- context-gather reads `.workflow/[testSessionId]/workflow-session.json` -- Detects `workflow_type: "test_session"` -- Automatically uses `source_session_id` to gather source session context -- No need for manual `--source-session` parameter - -**Cross-Session Context Collection** (Automatic): -- Implementation summaries: `.workflow/[sourceSessionId]/.summaries/IMPL-*-summary.md` -- Code changes: `git log --since=[source_session_created_at]` for changed files -- Original plan: `.workflow/[sourceSessionId]/IMPL_PLAN.md` -- Test files: Discovered via MCP code-index tools +**Expected Behavior**: +- Load source session implementation context and summaries +- Analyze test coverage using MCP tools (find existing tests) +- Identify files requiring tests (coverage gaps) +- Detect test framework and conventions +- Generate `test-context-package.json` **Parse Output**: -- Extract: context package path (store as `contextPath`) -- Pattern: `.workflow/[testSessionId]/.process/context-package.json` +- Extract: test context package path (store as `testContextPath`) +- Pattern: `.workflow/[testSessionId]/.process/test-context-package.json` **Validation**: -- Context package created in test session directory -- Contains source session artifacts (summaries, changed files) -- Includes test file inventory +- Test context package created +- Contains source session summaries +- Includes coverage gap analysis +- Test framework detected +- Test conventions documented **TodoWrite**: Mark phase 2 completed, phase 3 in_progress --- -### Phase 3: Implementation Analysis -**Command**: `SlashCommand(command="/workflow:tools:concept-enhanced --session [testSessionId] --context [contextPath]")` +### Phase 3: Test Generation Analysis +**Command**: `SlashCommand(command="/workflow:tools:test-concept-enhanced --session [testSessionId] --context [testContextPath]")` **Input**: -- `testSessionId` from Phase 1 (e.g., `WFS-test-user-auth`) -- `contextPath` from Phase 2 (e.g., `.workflow/WFS-test-user-auth/.process/context-package.json`) - -**Analysis Focus**: -- Review implementation summaries from source session -- Identify test files and coverage gaps -- Assess test execution strategy (unit, integration, e2e) -- Determine failure diagnosis approach -- Recommend code quality improvements +- `testSessionId` from Phase 1 +- `testContextPath` from Phase 2 **Expected Behavior**: -- Reads context-package.json with cross-session artifacts -- Executes parallel analysis (Gemini for test strategy, optional Codex for validation) -- Generates comprehensive test execution strategy -- Identifies code modification targets for test fixes -- Provides feasibility assessment for test validation +- Use Gemini to analyze coverage gaps and implementation context +- Study existing test patterns and conventions +- Generate test requirements for each missing test file +- Design test generation strategy +- Generate `TEST_ANALYSIS_RESULTS.md` **Parse Output**: -- Verify `.workflow/[testSessionId]/.process/ANALYSIS_RESULTS.md` created -- Contains test strategy and execution plan -- Lists code modification targets (format: `file:function:lines` or `file`) -- Includes risk assessment and optimization recommendations +- Verify `.workflow/[testSessionId]/.process/TEST_ANALYSIS_RESULTS.md` created +- Contains test requirements and generation strategy +- Lists test files to create with specifications **Validation**: -- File `.workflow/[testSessionId]/.process/ANALYSIS_RESULTS.md` exists -- Contains complete analysis sections: - - Current State Analysis (test coverage, existing tests) - - Proposed Solution Design (test execution strategy) - - Implementation Strategy (code targets, feasibility) - - Solution Optimization (performance, quality) - - Critical Success Factors (acceptance criteria) +- TEST_ANALYSIS_RESULTS.md exists with complete sections: + - Coverage Assessment + - Test Framework & Conventions + - Test Requirements by File + - Test Generation Strategy + - Implementation Targets (test files to create) + - Success Criteria **TodoWrite**: Mark phase 3 completed, phase 4 in_progress --- -### Phase 4: Generate Test Task -**Command**: `SlashCommand(command="/workflow:tools:task-generate --session [testSessionId]")` +### Phase 4: Generate Test Tasks +**Command**: `SlashCommand(command="/workflow:tools:test-task-generate --session [testSessionId]")` **Input**: `testSessionId` from Phase 1 **Expected Behavior**: -- Reads ANALYSIS_RESULTS.md from Phase 3 -- Extracts test strategy and code modification targets -- Generates `IMPL-001.json` (reusing standard format) with: - - `meta.type: "test-fix"` (enables @test-fix-agent assignment) - - `meta.agent: "@test-fix-agent"` - - `context.requirements`: Test execution requirements from analysis - - `context.focus_paths`: Test files and source files from analysis - - `context.acceptance`: All tests pass criteria - - `flow_control.pre_analysis`: Load source session summaries - - `flow_control.implementation_approach`: Test execution strategy from ANALYSIS_RESULTS.md - - `flow_control.target_files`: Code modification targets from analysis +- Parse TEST_ANALYSIS_RESULTS.md from Phase 3 +- Extract test requirements and generation strategy +- Generate **TWO task JSON files**: + - **IMPL-001.json**: Test Generation task (calls @code-developer) + - **IMPL-002.json**: Test Execution and Fix Cycle task (calls @test-fix-agent) +- Generate IMPL_PLAN.md with test generation and execution strategy +- Generate TODO_LIST.md with both tasks **Parse Output**: -- Verify `.workflow/[testSessionId]/.task/IMPL-001.json` exists +- Verify `.workflow/[testSessionId]/.task/IMPL-001.json` exists (test generation) +- Verify `.workflow/[testSessionId]/.task/IMPL-002.json` exists (test execution & fix) - Verify `.workflow/[testSessionId]/IMPL_PLAN.md` created - Verify `.workflow/[testSessionId]/TODO_LIST.md` created -**Validation**: -- Task JSON has `id: "IMPL-001"` and `meta.type: "test-fix"` -- IMPL_PLAN.md contains test validation strategy from ANALYSIS_RESULTS.md -- TODO_LIST.md shows IMPL-001 task -- flow_control includes code targets and test strategy -- Task is ready for /workflow:execute +**Validation - IMPL-001.json (Test Generation)**: +- Task ID: `IMPL-001` +- `meta.type: "test-gen"` +- `meta.agent: "@code-developer"` +- `context.requirements`: Generate tests based on TEST_ANALYSIS_RESULTS.md +- `flow_control.pre_analysis`: Load TEST_ANALYSIS_RESULTS.md and test context +- `flow_control.implementation_approach`: Test generation steps +- `flow_control.target_files`: Test files to create from analysis section 5 -**TodoWrite**: Mark phase 4 completed +**Validation - IMPL-002.json (Test Execution & Fix)**: +- Task ID: `IMPL-002` +- `meta.type: "test-fix"` +- `meta.agent: "@test-fix-agent"` +- `context.depends_on: ["IMPL-001"]` +- `context.requirements`: Execute and fix tests +- `flow_control.implementation_approach.test_fix_cycle`: Complete cycle specification + - **Cycle pattern**: test → gemini_diagnose → codex_fix_resume → retest + - **Tools configuration**: Gemini for analysis, Codex resume for fixes + - **Exit conditions**: Success (all pass) or failure (max iterations) +- `flow_control.implementation_approach.modification_points`: 3-phase execution flow + - Phase 1: Initial test execution + - Phase 2: Iterative Gemini diagnosis + Codex resume fixes + - Phase 3: Final validation and certification + +**TodoWrite**: Mark phase 4 completed, phase 5 in_progress + +--- + +### Phase 5: Return Summary to User **Return to User**: ``` @@ -173,315 +180,119 @@ Independent test-fix workflow created successfully! Source Session: [sourceSessionId] Test Session: [testSessionId] -Task Created: IMPL-001 (test-fix) + +Tasks Created: +- IMPL-001: Test Generation (@code-developer) +- IMPL-002: Test Execution & Fix Cycle (@test-fix-agent) + +Test Framework: [detected framework] +Test Files to Generate: [count] +Max Fix Iterations: 5 Next Steps: 1. Review test plan: .workflow/[testSessionId]/IMPL_PLAN.md -2. Execute validation: /workflow:execute +2. Execute workflow: /workflow:execute 3. Monitor progress: /workflow:status - -The @test-fix-agent will: -- Execute all tests -- Diagnose any failures -- Fix code until tests pass ``` +**TodoWrite**: Mark phase 5 completed + --- ## TodoWrite Pattern +Track progress through 5 phases: + ```javascript -// Initialize (before Phase 1) TodoWrite({todos: [ - {"content": "Create independent test session", "status": "in_progress", "activeForm": "Creating test session"}, - {"content": "Gather cross-session context", "status": "pending", "activeForm": "Gathering cross-session context"}, - {"content": "Analyze implementation for test strategy", "status": "pending", "activeForm": "Analyzing implementation"}, - {"content": "Generate test validation task", "status": "pending", "activeForm": "Generating test validation task"} -]}) - -// After Phase 1 -TodoWrite({todos: [ - {"content": "Create independent test session", "status": "completed", "activeForm": "Creating test session"}, - {"content": "Gather cross-session context", "status": "in_progress", "activeForm": "Gathering cross-session context"}, - {"content": "Analyze implementation for test strategy", "status": "pending", "activeForm": "Analyzing implementation"}, - {"content": "Generate test validation task", "status": "pending", "activeForm": "Generating test validation task"} -]}) - -// After Phase 2 -TodoWrite({todos: [ - {"content": "Create independent test session", "status": "completed", "activeForm": "Creating test session"}, - {"content": "Gather cross-session context", "status": "completed", "activeForm": "Gathering cross-session context"}, - {"content": "Analyze implementation for test strategy", "status": "in_progress", "activeForm": "Analyzing implementation"}, - {"content": "Generate test validation task", "status": "pending", "activeForm": "Generating test validation task"} -]}) - -// After Phase 3 -TodoWrite({todos: [ - {"content": "Create independent test session", "status": "completed", "activeForm": "Creating test session"}, - {"content": "Gather cross-session context", "status": "completed", "activeForm": "Gathering cross-session context"}, - {"content": "Analyze implementation for test strategy", "status": "completed", "activeForm": "Analyzing implementation"}, - {"content": "Generate test validation task", "status": "in_progress", "activeForm": "Generating test validation task"} -]}) - -// After Phase 4 -TodoWrite({todos: [ - {"content": "Create independent test session", "status": "completed", "activeForm": "Creating test session"}, - {"content": "Gather cross-session context", "status": "completed", "activeForm": "Gathering cross-session context"}, - {"content": "Analyze implementation for test strategy", "status": "completed", "activeForm": "Analyzing implementation"}, - {"content": "Generate test validation task", "status": "completed", "activeForm": "Generating test validation task"} + {"content": "Create independent test session", "status": "in_progress|completed", "activeForm": "Creating test session"}, + {"content": "Gather test coverage context", "status": "pending|in_progress|completed", "activeForm": "Gathering test coverage context"}, + {"content": "Analyze test requirements with Gemini", "status": "pending|in_progress|completed", "activeForm": "Analyzing test requirements"}, + {"content": "Generate test generation and execution tasks", "status": "pending|in_progress|completed", "activeForm": "Generating test tasks"}, + {"content": "Return workflow summary", "status": "pending|in_progress|completed", "activeForm": "Returning workflow summary"} ]}) ``` +Update status to `in_progress` when starting each phase, mark `completed` when done. + ## Data Flow ``` -User: /workflow:test-gen WFS-user-auth - ↓ -Phase 1: session-start --new "Test validation for WFS-user-auth" - ↓ Creates: WFS-test-user-auth session - ↓ Writes: workflow-session.json with workflow_type="test_session", source_session_id="WFS-user-auth" - ↓ Output: testSessionId = "WFS-test-user-auth" - ↓ -Phase 2: context-gather --session WFS-test-user-auth - ↓ Auto-detects: test session type from workflow-session.json - ↓ Auto-reads: source_session_id = "WFS-user-auth" - ↓ Gathers: Cross-session context (summaries, code changes, tests) - ↓ Output: .workflow/WFS-test-user-auth/.process/context-package.json - ↓ -Phase 3: concept-enhanced --session WFS-test-user-auth --context context-package.json - ↓ Reads: context-package.json with cross-session artifacts - ↓ Executes: Parallel analysis (Gemini test strategy + optional Codex validation) - ↓ Analyzes: Test coverage, execution strategy, code targets - ↓ Output: .workflow/WFS-test-user-auth/.process/ANALYSIS_RESULTS.md - ↓ -Phase 4: task-generate --session WFS-test-user-auth - ↓ Reads: ANALYSIS_RESULTS.md with test strategy and code targets - ↓ Generates: IMPL-001.json with meta.type="test-fix" - ↓ Output: Task, plan, and todo files in test session - ↓ -Return: Summary with next steps (user triggers /workflow:execute separately) +/workflow:test-gen WFS-user-auth + ↓ +Phase 1: session-start → WFS-test-user-auth + ↓ +Phase 2: test-context-gather → test-context-package.json + ↓ +Phase 3: test-concept-enhanced → TEST_ANALYSIS_RESULTS.md + ↓ +Phase 4: test-task-generate → IMPL-001.json + IMPL-002.json + ↓ +Phase 5: Return summary + ↓ +/workflow:execute → IMPL-001 (@code-developer) → IMPL-002 (@test-fix-agent) ``` -## Session Metadata Design +## Session Metadata -**Test Session (`WFS-test-user-auth/workflow-session.json`)**: -```json -{ - "session_id": "WFS-test-user-auth", - "project": "Test validation for user authentication implementation", - "status": "planning", - "created_at": "2025-10-03T12:00:00Z", - "workflow_type": "test_session", - "source_session_id": "WFS-user-auth" -} -``` +Test session includes `workflow_type: "test_session"` and `source_session_id` for automatic context gathering. -## Automatic Cross-Session Context Collection +## Task Output -When `context-gather` detects `workflow_type: "test_session"`: +Generates two tasks: +- **IMPL-001** (@code-developer): Test generation from TEST_ANALYSIS_RESULTS.md +- **IMPL-002** (@test-fix-agent): Test execution with iterative fix cycle (max 5 iterations) -**Collected from Source Session** (`.workflow/WFS-user-auth/`): -- Implementation summaries: `.summaries/IMPL-*-summary.md` -- Code changes: `git log --since=[source_created_at] --name-only` -- Original plan: `IMPL_PLAN.md` -- Task definitions: `.task/IMPL-*.json` - -**Collected from Current Project**: -- Test files: `mcp__code-index__find_files(pattern="*.test.*")` -- Test configuration: `package.json`, `jest.config.js`, etc. -- Source files: Based on changed files from git log - -## Task Generation Output - -task-generate creates `IMPL-001.json` (reusing standard format) with: - -```json -{ - "id": "IMPL-001", - "title": "Execute and validate tests for [sourceSessionId]", - "status": "pending", - "meta": { - "type": "test-fix", - "agent": "@test-fix-agent" - }, - "context": { - "requirements": [ - "Execute complete test suite for all implemented modules", - "Diagnose and fix any test failures", - "Ensure all tests pass before completion" - ], - "focus_paths": ["src/**/*.test.ts", "src/**/implementation.ts"], - "acceptance": [ - "All tests pass successfully", - "No test failures or errors", - "Code is approved and ready for deployment" - ], - "depends_on": [], - "artifacts": [] - }, - "flow_control": { - "pre_analysis": [ - { - "step": "load_source_session_summaries", - "action": "Load implementation summaries from source session", - "commands": [ - "bash(find .workflow/[sourceSessionId]/.summaries/ -name 'IMPL-*-summary.md' 2>/dev/null)", - "Read(.workflow/[sourceSessionId]/.summaries/IMPL-001-summary.md)" - ], - "output_to": "implementation_context", - "on_error": "skip_optional" - }, - { - "step": "analyze_test_files", - "action": "Identify test files and coverage", - "commands": [ - "mcp__code-index__find_files(pattern=\"*.test.*\")", - "mcp__code-index__search_code_advanced(pattern=\"test|describe|it\", file_pattern=\"*.test.*\")" - ], - "output_to": "test_inventory", - "on_error": "fail" - } - ], - "implementation_approach": { - "task_description": "Execute tests and fix failures until all pass", - "modification_points": [ - "Run test suite using detected framework", - "Parse test output to identify failures", - "Diagnose root cause of failures", - "Modify source code to fix issues", - "Re-run tests to verify fixes" - ], - "logic_flow": [ - "Load implementation context", - "Identify test framework and configuration", - "Execute complete test suite", - "If failures: analyze error messages", - "Fix source code based on diagnosis", - "Re-run tests", - "Repeat until all tests pass" - ] - }, - "target_files": ["src/**/*.test.ts", "src/**/implementation.ts"] - } -} -``` +See `/workflow:tools:test-task-generate` for complete task JSON schemas. ## Error Handling -### Phase 1 Failures -- **Source session not found**: Return error "Source session [sourceSessionId] not found in .workflow/" -- **Invalid source session**: Return error "Source session [sourceSessionId] has no completed IMPL tasks" -- **Session creation failed**: Return error "Could not create test session. Check /workflow:session:start" +| Phase | Error | Action | +|-------|-------|--------| +| 1 | Source session not found | Return error with source session ID | +| 1 | No completed IMPL tasks | Return error, source incomplete | +| 2 | Context gathering failed | Return error, check source artifacts | +| 3 | Analysis failed | Return error, check context package | +| 4 | Task generation failed | Retry once, then error with details | -### Phase 2 Failures -- **Context gathering failed**: Return error "Could not gather cross-session context. Check source session artifacts" -- **No source artifacts**: Return error "Source session has no implementation summaries or git history" +## Output Files -### Phase 3 Failures -- **Analysis failed**: Return error "Implementation analysis failed. Check context package and ANALYSIS_RESULTS.md" -- **No test strategy**: Return error "Could not determine test execution strategy from analysis" -- **Missing code targets**: Warning only, proceed with general test task - -### Phase 4 Failures -- **Task generation failed**: Retry once, then return error with details -- **Invalid task structure**: Return error with JSON validation details - -## Workflow Integration - -### Complete Flow Example -``` -1. Implementation Phase (prior to test-gen) - /workflow:plan "Build auth system" - → Creates WFS-auth session - → @code-developer implements + writes tests - → Creates IMPL-001-summary.md in WFS-auth - -2. Test Generation Phase (test-gen) - /workflow:test-gen WFS-auth - Phase 1: session-start → Creates WFS-test-auth session - Phase 2: context-gather → Gathers from WFS-auth, creates context-package.json - Phase 3: concept-enhanced → Analyzes implementation, creates ANALYSIS_RESULTS.md - Phase 4: task-generate → Creates IMPL-001.json with meta.type="test-fix" - Returns: Summary with next steps - -3. Test Execution Phase (user-triggered) - /workflow:execute - → Detects active session: WFS-test-auth - → @test-fix-agent picks up IMPL-001 (test-fix type) - → Runs test suite - → Diagnoses failures (if any) - → Fixes source code - → Re-runs tests - → All pass → Code approved ✅ -``` - -### Output Files Created - -**In Test Session** (`.workflow/WFS-test-auth/`): -- `workflow-session.json` - Contains workflow_type and source_session_id -- `.process/context-package.json` - Cross-session context from WFS-auth -- `.process/ANALYSIS_RESULTS.md` - Test strategy and code targets from concept-enhanced -- `.task/IMPL-001.json` - Test-fix task definition -- `IMPL_PLAN.md` - Test validation plan (from ANALYSIS_RESULTS.md) +Created in `.workflow/WFS-test-[session]/`: +- `workflow-session.json` - Session metadata +- `.process/test-context-package.json` - Coverage analysis +- `.process/TEST_ANALYSIS_RESULTS.md` - Test requirements +- `.task/IMPL-001.json` - Test generation task +- `.task/IMPL-002.json` - Test execution & fix task +- `IMPL_PLAN.md` - Test plan - `TODO_LIST.md` - Task checklist -- `.summaries/IMPL-001-summary.md` - Created by @test-fix-agent after completion + +## Agent Execution + +**IMPL-001** (@code-developer): +- Generates test files based on TEST_ANALYSIS_RESULTS.md +- Follows existing test patterns and conventions + +**IMPL-002** (@test-fix-agent): +1. Run test suite +2. Iterative fix cycle (max 5): + - Gemini diagnosis → surgical fix suggestions + - Codex applies fixes (resume mechanism) + - Retest and check regressions +3. Final validation and certification + +See `/workflow:tools:test-task-generate` for detailed specifications. ## Best Practices -1. **Run after implementation complete**: Ensure source session has completed IMPL tasks and summaries -2. **Check git commits**: Implementation changes should be committed for accurate git log analysis -3. **Verify test files exist**: Source implementation should include test files -4. **Independent sessions**: Test session is separate from implementation session for clean separation -5. **Monitor execution**: Use `/workflow:status` to track test-fix progress after /workflow:execute - -## Coordinator Checklist - -✅ Initialize TodoWrite before any command (4 phases) -✅ Phase 1: Create test session with source session ID -✅ Parse new test session ID from Phase 1 output -✅ Phase 2: Run context-gather (auto-detects test session, no extra params) -✅ Verify context-package.json contains cross-session artifacts -✅ Phase 3: Run concept-enhanced with session and context path -✅ Verify ANALYSIS_RESULTS.md contains test strategy and code targets -✅ Phase 4: Run task-generate to create IMPL-001.json -✅ Verify task has meta.type="test-fix" and meta.agent="@test-fix-agent" -✅ Verify flow_control includes analysis insights and code targets -✅ Update TodoWrite after each phase -✅ Return summary after Phase 4 (execution is separate user action) - -## Required Tool Modifications - -### `/workflow:session:start` -- Support `--new` flag for test session creation -- Auto-detect test session pattern from task description -- Write `workflow_type: "test_session"` and `source_session_id` to metadata - -### `/workflow:tools:context-gather` -- Read session metadata to detect `workflow_type: "test_session"` -- Auto-extract `source_session_id` from metadata -- Gather cross-session context from source session artifacts -- Include git log analysis from source session creation time - -### `/workflow:tools:concept-enhanced` -- No changes required (already supports cross-session context analysis) -- Will automatically analyze test strategy based on context-package.json -- Generates ANALYSIS_RESULTS.md with code targets and test execution plan - -### `/workflow:tools:task-generate` -- Recognize test session context and generate appropriate task -- Create `IMPL-001.json` with `meta.type: "test-fix"` -- Extract test strategy from ANALYSIS_RESULTS.md -- Include code targets and cross-session references in flow_control - -### `/workflow:execute` -- No changes required (already dispatches by meta.agent) +1. Run after implementation complete (ensure source session has summaries) +2. Commit implementation changes before test-gen +3. Monitor execution with `/workflow:status` +4. Review iteration logs in `.process/fix-iteration-*` ## Related Commands -- `/workflow:plan` - Create implementation workflow (run before test-gen) -- `/workflow:session:start` - Phase 1 tool for test session creation -- `/workflow:tools:context-gather` - Phase 2 tool for cross-session context collection -- `/workflow:tools:concept-enhanced` - Phase 3 tool for implementation analysis and test strategy -- `/workflow:tools:task-generate` - Phase 4 tool for test task creation -- `/workflow:execute` - Execute test-fix workflow (user-triggered after test-gen) -- `/workflow:status` - Check workflow progress -- `@test-fix-agent` - Agent that executes and fixes tests + +- `/workflow:tools:test-context-gather` - Phase 2 (coverage analysis) +- `/workflow:tools:test-concept-enhanced` - Phase 3 (Gemini test analysis) +- `/workflow:tools:test-task-generate` - Phase 4 (task generation) +- `/workflow:execute` - Execute workflow +- `/workflow:status` - Check progress diff --git a/.claude/commands/workflow/tools/test-concept-enhanced.md b/.claude/commands/workflow/tools/test-concept-enhanced.md new file mode 100644 index 00000000..a73001e3 --- /dev/null +++ b/.claude/commands/workflow/tools/test-concept-enhanced.md @@ -0,0 +1,468 @@ +--- +name: test-concept-enhanced +description: Analyze test requirements and generate test generation strategy using Gemini +usage: /workflow:tools:test-concept-enhanced --session --context +argument-hint: "--session WFS-test-session-id --context path/to/test-context-package.json" +examples: + - /workflow:tools:test-concept-enhanced --session WFS-test-auth --context .workflow/WFS-test-auth/.process/test-context-package.json +--- + +# Test Concept Enhanced Command + +## Overview +Specialized analysis tool for test generation workflows that uses Gemini to analyze test coverage gaps, implementation context, and generate comprehensive test generation strategies. + +## Core Philosophy +- **Coverage-Driven**: Focus on identified test gaps from context analysis +- **Pattern-Based**: Learn from existing tests and project conventions +- **Gemini-Powered**: Use Gemini for test requirement analysis and strategy design +- **Single-Round Analysis**: Comprehensive test analysis in one execution +- **No Code Generation**: Strategy and planning only, actual test generation happens in task execution + +## Core Responsibilities +- Parse test-context-package.json from test-context-gather +- Analyze implementation summaries and coverage gaps +- Study existing test patterns and conventions +- Generate test generation strategy using Gemini +- Produce TEST_ANALYSIS_RESULTS.md for task generation + +## Execution Lifecycle + +### Phase 1: Validation & Preparation + +1. **Session Validation** + - Load `.workflow/{test_session_id}/workflow-session.json` + - Verify test session type is "test-gen" + - Extract source session reference + +2. **Context Package Validation** + - Read `test-context-package.json` + - Validate required sections: metadata, source_context, test_coverage, test_framework + - Extract coverage gaps and framework details + +3. **Strategy Determination** + - **Simple Test Generation** (1-3 files): Single Gemini analysis + - **Medium Test Generation** (4-6 files): Gemini comprehensive analysis + - **Complex Test Generation** (>6 files): Gemini analysis with modular approach + +### Phase 2: Gemini Test Analysis + +**Tool Configuration**: +```bash +cd .workflow/{test_session_id}/.process && ~/.claude/scripts/gemini-wrapper -p " +PURPOSE: Analyze test coverage gaps and design comprehensive test generation strategy +TASK: Study implementation context, existing tests, and generate test requirements for missing coverage +MODE: analysis +CONTEXT: @{.workflow/{test_session_id}/.process/test-context-package.json} + +**MANDATORY FIRST STEP**: Read and analyze test-context-package.json to understand: +- Test coverage gaps from test_coverage.missing_tests[] +- Implementation context from source_context.implementation_summaries[] +- Existing test patterns from test_framework.conventions +- Changed files requiring tests from source_context.implementation_summaries[].changed_files + +**ANALYSIS REQUIREMENTS**: + +1. **Implementation Understanding** + - Load all implementation summaries from source session + - Understand implemented features, APIs, and business logic + - Extract key functions, classes, and modules + - Identify integration points and dependencies + +2. **Existing Test Pattern Analysis** + - Study existing test files for patterns and conventions + - Identify test structure (describe/it, test suites, fixtures) + - Analyze assertion patterns and mocking strategies + - Extract test setup/teardown patterns + +3. **Coverage Gap Assessment** + - For each file in missing_tests[], analyze: + - File purpose and functionality + - Public APIs requiring test coverage + - Critical paths and edge cases + - Integration points requiring tests + - Prioritize tests: high (core logic), medium (utilities), low (helpers) + +4. **Test Requirements Specification** + - For each missing test file, specify: + - **Test scope**: What needs to be tested + - **Test scenarios**: Happy path, error cases, edge cases, integration + - **Test data**: Required fixtures, mocks, test data + - **Dependencies**: External services, databases, APIs to mock + - **Coverage targets**: Functions/methods requiring tests + +5. **Test Generation Strategy** + - Determine test generation approach for each file + - Identify reusable test patterns from existing tests + - Plan test data and fixture requirements + - Define mocking strategy for dependencies + - Specify expected test file structure + +EXPECTED OUTPUT - Write to gemini-test-analysis.md: + +# Test Generation Analysis + +## 1. Implementation Context Summary +- **Source Session**: {source_session_id} +- **Implemented Features**: {feature_summary} +- **Changed Files**: {list_of_implementation_files} +- **Tech Stack**: {technologies_used} + +## 2. Test Coverage Assessment +- **Existing Tests**: {count} files +- **Missing Tests**: {count} files +- **Coverage Percentage**: {percentage}% +- **Priority Breakdown**: + - High Priority: {count} files (core business logic) + - Medium Priority: {count} files (utilities, helpers) + - Low Priority: {count} files (configuration, constants) + +## 3. Existing Test Pattern Analysis +- **Test Framework**: {framework_name_and_version} +- **File Naming Convention**: {pattern} +- **Test Structure**: {describe_it_or_other} +- **Assertion Style**: {expect_assert_should} +- **Mocking Strategy**: {mocking_framework_and_patterns} +- **Setup/Teardown**: {beforeEach_afterEach_patterns} +- **Test Data**: {fixtures_factories_builders} + +## 4. Test Requirements by File + +### File: {implementation_file_path} +**Test File**: {suggested_test_file_path} +**Priority**: {high|medium|low} + +#### Scope +- {description_of_what_needs_testing} + +#### Test Scenarios +1. **Happy Path Tests** + - {scenario_1} + - {scenario_2} + +2. **Error Handling Tests** + - {error_scenario_1} + - {error_scenario_2} + +3. **Edge Case Tests** + - {edge_case_1} + - {edge_case_2} + +4. **Integration Tests** (if applicable) + - {integration_scenario_1} + - {integration_scenario_2} + +#### Test Data & Fixtures +- {required_test_data} +- {required_mocks} +- {required_fixtures} + +#### Dependencies to Mock +- {external_service_1} +- {external_service_2} + +#### Coverage Targets +- Function: {function_name} - {test_requirements} +- Function: {function_name} - {test_requirements} + +--- +[Repeat for each missing test file] +--- + +## 5. Test Generation Strategy + +### Overall Approach +- {strategy_description} + +### Test Generation Order +1. {file_1} - {rationale} +2. {file_2} - {rationale} +3. {file_3} - {rationale} + +### Reusable Patterns +- {pattern_1_from_existing_tests} +- {pattern_2_from_existing_tests} + +### Test Data Strategy +- {approach_to_test_data_and_fixtures} + +### Mocking Strategy +- {approach_to_mocking_dependencies} + +### Quality Criteria +- Code coverage target: {percentage}% +- Test scenarios per function: {count} +- Integration test coverage: {approach} + +## 6. Implementation Targets + +**Purpose**: Identify new test files to create + +**Format**: New test files only (no existing files to modify) + +**Test Files to Create**: +1. **Target**: `tests/auth/TokenValidator.test.ts` + - **Type**: Create new test file + - **Purpose**: Test TokenValidator class + - **Scenarios**: 15 test cases covering validation logic, error handling, edge cases + - **Dependencies**: Mock JWT library, test fixtures for tokens + +2. **Target**: `tests/middleware/errorHandler.test.ts` + - **Type**: Create new test file + - **Purpose**: Test error handling middleware + - **Scenarios**: 8 test cases for different error types and response formats + - **Dependencies**: Mock Express req/res/next, error fixtures + +[List all test files to create] + +## 7. Success Metrics +- **Test Coverage Goal**: {target_percentage}% +- **Test Quality**: All scenarios covered (happy, error, edge, integration) +- **Convention Compliance**: Follow existing test patterns +- **Maintainability**: Clear test descriptions, reusable fixtures + +RULES: +- Focus on TEST REQUIREMENTS and GENERATION STRATEGY, NOT code generation +- Study existing test patterns thoroughly for consistency +- Prioritize critical business logic tests +- Specify clear test scenarios and coverage targets +- Identify all dependencies requiring mocks +- **MUST write output to .workflow/{test_session_id}/.process/gemini-test-analysis.md** +- Do NOT generate actual test code or implementation +- Output ONLY test analysis and generation strategy +" --approval-mode yolo +``` + +**Output Location**: `.workflow/{test_session_id}/.process/gemini-test-analysis.md` + +### Phase 3: Results Synthesis + +1. **Output Validation** + - Verify `gemini-test-analysis.md` exists and is complete + - Validate all required sections present + - Check test requirements are actionable + +2. **Quality Assessment** + - Test scenarios cover happy path, errors, edge cases + - Dependencies and mocks clearly identified + - Test generation strategy is practical + - Coverage targets are reasonable + +### Phase 4: TEST_ANALYSIS_RESULTS.md Generation + +Synthesize Gemini analysis into standardized format: + +```markdown +# Test Generation Analysis Results + +## Executive Summary +- **Test Session**: {test_session_id} +- **Source Session**: {source_session_id} +- **Analysis Timestamp**: {timestamp} +- **Coverage Gap**: {missing_test_count} files require tests +- **Test Framework**: {framework} +- **Overall Strategy**: {high_level_approach} + +--- + +## 1. Coverage Assessment + +### Current Coverage +- **Existing Tests**: {count} files +- **Implementation Files**: {count} files +- **Coverage Percentage**: {percentage}% + +### Missing Tests (Priority Order) +1. **High Priority** ({count} files) + - {file_1} - {reason} + - {file_2} - {reason} + +2. **Medium Priority** ({count} files) + - {file_1} - {reason} + +3. **Low Priority** ({count} files) + - {file_1} - {reason} + +--- + +## 2. Test Framework & Conventions + +### Framework Configuration +- **Framework**: {framework_name} +- **Version**: {version} +- **Test Pattern**: {file_pattern} +- **Test Directory**: {directory_structure} + +### Conventions +- **File Naming**: {convention} +- **Test Structure**: {describe_it_blocks} +- **Assertions**: {assertion_library} +- **Mocking**: {mocking_framework} +- **Setup/Teardown**: {beforeEach_afterEach} + +### Example Pattern (from existing tests) +``` +{example_test_structure_from_analysis} +``` + +--- + +## 3. Test Requirements by File + +[For each missing test, include:] + +### Test File: {test_file_path} +**Implementation**: {implementation_file} +**Priority**: {high|medium|low} +**Estimated Test Count**: {count} + +#### Test Scenarios +1. **Happy Path**: {scenarios} +2. **Error Handling**: {scenarios} +3. **Edge Cases**: {scenarios} +4. **Integration**: {scenarios} + +#### Dependencies & Mocks +- {dependency_1_to_mock} +- {dependency_2_to_mock} + +#### Test Data Requirements +- {fixture_1} +- {fixture_2} + +--- + +## 4. Test Generation Strategy + +### Generation Approach +{overall_strategy_description} + +### Generation Order +1. {test_file_1} - {rationale} +2. {test_file_2} - {rationale} +3. {test_file_3} - {rationale} + +### Reusable Components +- **Test Fixtures**: {common_fixtures} +- **Mock Patterns**: {common_mocks} +- **Helper Functions**: {test_helpers} + +### Quality Targets +- **Coverage Goal**: {percentage}% +- **Scenarios per Function**: {min_count} +- **Integration Coverage**: {approach} + +--- + +## 5. Implementation Targets + +**Purpose**: New test files to create (code-developer will generate these) + +**Test Files to Create**: + +1. **Target**: `tests/auth/TokenValidator.test.ts` + - **Implementation Source**: `src/auth/TokenValidator.ts` + - **Test Scenarios**: 15 (validation, error handling, edge cases) + - **Dependencies**: Mock JWT library, token fixtures + - **Priority**: High + +2. **Target**: `tests/middleware/errorHandler.test.ts` + - **Implementation Source**: `src/middleware/errorHandler.ts` + - **Test Scenarios**: 8 (error types, response formats) + - **Dependencies**: Mock Express, error fixtures + - **Priority**: High + +[List all test files with full specifications] + +--- + +## 6. Success Criteria + +### Coverage Metrics +- Achieve {target_percentage}% code coverage +- All public APIs have tests +- Critical paths fully covered + +### Quality Standards +- All test scenarios covered (happy, error, edge, integration) +- Follow existing test conventions +- Clear test descriptions and assertions +- Maintainable test structure + +### Validation Approach +- Run full test suite after generation +- Verify coverage with coverage tool +- Manual review of test quality +- Integration test validation + +--- + +## 7. Reference Information + +### Source Context +- **Implementation Summaries**: {paths} +- **Existing Tests**: {example_tests} +- **Documentation**: {relevant_docs} + +### Analysis Tools +- **Gemini Analysis**: gemini-test-analysis.md +- **Coverage Tools**: {coverage_tool_if_detected} +``` + +**Output Location**: `.workflow/{test_session_id}/.process/TEST_ANALYSIS_RESULTS.md` + +## Error Handling + +### Validation Errors +| Error | Cause | Resolution | +|-------|-------|------------| +| Missing context package | test-context-gather not run | Run test-context-gather first | +| No coverage gaps | All files have tests | Skip test generation, proceed to test execution | +| No test framework detected | Missing test dependencies | Request user to configure test framework | +| Invalid source session | Source session incomplete | Complete implementation first | + +### Gemini Execution Errors +| Error | Cause | Recovery | +|-------|-------|----------| +| Timeout | Large project analysis | Reduce scope, analyze by module | +| Output incomplete | Token limit exceeded | Retry with focused analysis | +| No output file | Write permission error | Check directory permissions | + +### Fallback Strategy +- If Gemini fails, generate basic TEST_ANALYSIS_RESULTS.md from context package +- Use coverage gaps and framework info to create minimal requirements +- Provide guidance for manual test planning + +## Performance Optimization + +- **Focused Analysis**: Only analyze files with missing tests +- **Pattern Reuse**: Study existing tests for quick pattern extraction +- **Parallel Operations**: Load implementation summaries in parallel +- **Timeout Management**: 20-minute limit for Gemini analysis + +## Integration + +### Called By +- `/workflow:test-gen` (Phase 4: Analysis) + +### Requires +- `/workflow:tools:test-context-gather` output (test-context-package.json) + +### Followed By +- `/workflow:tools:test-task-generate` - Generates test task JSON with code-developer invocation + +## Success Criteria + +- ✅ Valid TEST_ANALYSIS_RESULTS.md generated +- ✅ All missing tests documented with requirements +- ✅ Test scenarios cover happy path, errors, edge cases +- ✅ Dependencies and mocks identified +- ✅ Test generation strategy is actionable +- ✅ Execution time < 20 minutes +- ✅ Output follows existing test conventions + +## Related Commands + +- `/workflow:tools:test-context-gather` - Provides input context +- `/workflow:tools:test-task-generate` - Consumes analysis results +- `/workflow:test-gen` - Main test generation workflow diff --git a/.claude/commands/workflow/tools/test-context-gather.md b/.claude/commands/workflow/tools/test-context-gather.md new file mode 100644 index 00000000..72861610 --- /dev/null +++ b/.claude/commands/workflow/tools/test-context-gather.md @@ -0,0 +1,310 @@ +--- +name: test-context-gather +description: Collect test coverage context and identify files requiring test generation +usage: /workflow:tools:test-context-gather --session +argument-hint: "--session WFS-test-session-id" +examples: + - /workflow:tools:test-context-gather --session WFS-test-auth + - /workflow:tools:test-context-gather --session WFS-test-payment +--- + +# Test Context Gather Command + +## Overview +Specialized context collector for test generation workflows that analyzes test coverage, identifies missing tests, and packages implementation context from source sessions. + +## Core Philosophy +- **Coverage-First**: Analyze existing test coverage before planning +- **Gap Identification**: Locate implementation files without corresponding tests +- **Source Context Loading**: Import implementation summaries from source session +- **Framework Detection**: Auto-detect test framework and patterns +- **MCP-Powered**: Leverage code-index tools for precise analysis + +## Core Responsibilities +- Load source session implementation context +- Analyze current test coverage using MCP tools +- Identify files requiring test generation +- Detect test framework and conventions +- Package test context for analysis phase + +## Execution Lifecycle + +### Phase 1: Session Validation & Source Loading + +1. **Test Session Validation** + - Load `.workflow/{test_session_id}/workflow-session.json` + - Extract `meta.source_session` reference + - Validate test session type is "test-gen" + +2. **Source Session Context Loading** + - Read `.workflow/{source_session_id}/workflow-session.json` + - Load implementation summaries from `.workflow/{source_session_id}/.summaries/` + - Extract changed files and implementation scope + - Identify implementation patterns and tech stack + +### Phase 2: Test Coverage Analysis (MCP Tools) + +1. **Existing Test Discovery** + ```bash + # Find all test files + mcp__code-index__find_files(pattern="*.test.*") + mcp__code-index__find_files(pattern="*.spec.*") + mcp__code-index__find_files(pattern="*test_*.py") + + # Search for test patterns + mcp__code-index__search_code_advanced( + pattern="describe|it|test|@Test", + file_pattern="*.test.*", + context_lines=0 + ) + ``` + +2. **Coverage Gap Analysis** + ```bash + # For each implementation file from source session + # Check if corresponding test file exists + + # Example: src/auth/AuthService.ts -> tests/auth/AuthService.test.ts + # src/utils/validator.py -> tests/test_validator.py + + # Output: List of files without tests + ``` + +3. **Test Statistics** + - Count total test files + - Count implementation files from source session + - Calculate coverage percentage + - Identify coverage gaps by module + +### Phase 3: Test Framework Detection + +1. **Framework Identification** + ```bash + # Check package.json or requirements.txt + mcp__code-index__search_code_advanced( + pattern="jest|mocha|jasmine|pytest|unittest|rspec", + file_pattern="package.json|requirements.txt|Gemfile", + context_lines=2 + ) + + # Analyze existing test patterns + mcp__code-index__search_code_advanced( + pattern="describe\\(|it\\(|test\\(|def test_", + file_pattern="*.test.*", + context_lines=3 + ) + ``` + +2. **Convention Analysis** + - Test file naming patterns (*.test.ts vs *.spec.ts) + - Test directory structure (tests/ vs __tests__ vs src/**/*.test.*) + - Assertion library (expect, assert, should) + - Mocking framework (jest.fn, sinon, unittest.mock) + +### Phase 4: Context Packaging + +Generate `test-context-package.json`: + +```json +{ + "metadata": { + "test_session_id": "WFS-test-auth", + "source_session_id": "WFS-auth", + "timestamp": "2025-10-04T10:30:00Z", + "task_type": "test-generation", + "complexity": "medium" + }, + "source_context": { + "implementation_summaries": [ + { + "task_id": "IMPL-001", + "summary_path": ".workflow/WFS-auth/.summaries/IMPL-001-summary.md", + "changed_files": [ + "src/auth/AuthService.ts", + "src/auth/TokenValidator.ts", + "src/middleware/auth.ts" + ], + "implementation_type": "feature" + } + ], + "tech_stack": ["typescript", "express", "jsonwebtoken"], + "project_patterns": { + "architecture": "layered", + "error_handling": "try-catch with custom errors", + "async_pattern": "async/await" + } + }, + "test_coverage": { + "existing_tests": [ + "tests/auth/AuthService.test.ts", + "tests/middleware/auth.test.ts" + ], + "missing_tests": [ + { + "implementation_file": "src/auth/TokenValidator.ts", + "suggested_test_file": "tests/auth/TokenValidator.test.ts", + "priority": "high", + "reason": "New implementation without tests" + } + ], + "coverage_stats": { + "total_implementation_files": 3, + "files_with_tests": 2, + "files_without_tests": 1, + "coverage_percentage": 66.7 + } + }, + "test_framework": { + "framework": "jest", + "version": "^29.0.0", + "test_pattern": "**/*.test.ts", + "test_directory": "tests/", + "assertion_library": "expect", + "mocking_framework": "jest", + "conventions": { + "file_naming": "*.test.ts", + "test_structure": "describe/it blocks", + "setup_teardown": "beforeEach/afterEach" + } + }, + "assets": [ + { + "type": "implementation_summary", + "path": ".workflow/WFS-auth/.summaries/IMPL-001-summary.md", + "relevance": "Source implementation context", + "priority": "highest" + }, + { + "type": "existing_test", + "path": "tests/auth/AuthService.test.ts", + "relevance": "Test pattern reference", + "priority": "high" + }, + { + "type": "source_code", + "path": "src/auth/TokenValidator.ts", + "relevance": "Implementation requiring tests", + "priority": "high" + }, + { + "type": "documentation", + "path": "CLAUDE.md", + "relevance": "Project conventions", + "priority": "medium" + } + ], + "focus_areas": [ + "Generate comprehensive tests for TokenValidator", + "Follow existing Jest patterns from AuthService tests", + "Cover happy path, error cases, and edge cases", + "Include integration tests for middleware" + ] +} +``` + +## Output Location + +``` +.workflow/{test_session_id}/.process/test-context-package.json +``` + +## MCP Tools Usage + +### File Discovery +```bash +# Test files +mcp__code-index__find_files(pattern="*.test.*") +mcp__code-index__find_files(pattern="*.spec.*") + +# Implementation files +mcp__code-index__find_files(pattern="*.ts") +mcp__code-index__find_files(pattern="*.js") +``` + +### Content Search +```bash +# Test framework detection +mcp__code-index__search_code_advanced( + pattern="jest|mocha|pytest", + file_pattern="package.json|requirements.txt" +) + +# Test pattern analysis +mcp__code-index__search_code_advanced( + pattern="describe|it|test", + file_pattern="*.test.*", + context_lines=2 +) +``` + +### Coverage Analysis +```bash +# For each implementation file +# Check if test exists +implementation_file="src/auth/AuthService.ts" +test_file_patterns=( + "tests/auth/AuthService.test.ts" + "src/auth/AuthService.test.ts" + "src/auth/__tests__/AuthService.test.ts" +) + +# Search for test file +for pattern in "${test_file_patterns[@]}"; do + if mcp__code-index__find_files(pattern="$pattern") | grep -q .; then + echo "✅ Test exists: $pattern" + break + fi +done +``` + +## Error Handling + +| Error | Cause | Resolution | +|-------|-------|------------| +| Source session not found | Invalid source_session reference | Verify test session metadata | +| No implementation summaries | Source session incomplete | Complete source session first | +| MCP tools unavailable | MCP not configured | Fallback to bash find/grep | +| No test framework detected | Missing test dependencies | Request user to specify framework | + +## Fallback Strategy (No MCP) + +```bash +# File discovery +find . -name "*.test.*" -o -name "*.spec.*" | grep -v node_modules + +# Framework detection +grep -r "jest\|mocha\|pytest" package.json requirements.txt 2>/dev/null + +# Coverage analysis +for impl_file in $(cat changed_files.txt); do + test_file=$(echo $impl_file | sed 's/src/tests/' | sed 's/\(.*\)\.\(ts\|js\|py\)$/\1.test.\2/') + [ ! -f "$test_file" ] && echo "$impl_file → MISSING TEST" +done +``` + +## Integration + +### Called By +- `/workflow:test-gen` (Phase 3: Context Gathering) + +### Calls +- MCP code-index tools for analysis +- Bash file operations for fallback + +### Followed By +- `/workflow:tools:test-concept-enhanced` - Analyzes context and plans test generation + +## Success Criteria + +- ✅ Source session context loaded successfully +- ✅ Test coverage gaps identified with MCP tools +- ✅ Test framework detected and documented +- ✅ Valid test-context-package.json generated +- ✅ All missing tests catalogued with priority +- ✅ Execution time < 20 seconds + +## Related Commands + +- `/workflow:test-gen` - Main test generation workflow +- `/workflow:tools:test-concept-enhanced` - Test generation analysis +- `/workflow:tools:test-task-generate` - Test task JSON generation diff --git a/.claude/commands/workflow/tools/test-task-generate.md b/.claude/commands/workflow/tools/test-task-generate.md new file mode 100644 index 00000000..a4f89448 --- /dev/null +++ b/.claude/commands/workflow/tools/test-task-generate.md @@ -0,0 +1,624 @@ +--- +name: test-task-generate +description: Generate test-fix task JSON with iterative test-fix-retest cycle specification +usage: /workflow:tools:test-task-generate --session +argument-hint: "--session WFS-test-session-id" +examples: + - /workflow:tools:test-task-generate --session WFS-test-auth +--- + +# Test Task Generation Command + +## Overview +Generate specialized test-fix task JSON with comprehensive test-fix-retest cycle specification, including Gemini diagnosis and Codex resume mechanism for intelligent failure resolution. + +## Core Philosophy +- **Analysis-Driven Test Generation**: Use TEST_ANALYSIS_RESULTS.md from test-concept-enhanced +- **Agent-Based Test Creation**: Call @code-developer agent for comprehensive test generation +- **Coverage-First**: Generate all missing tests before execution +- **Test Execution**: Execute complete test suite after generation +- **Gemini Diagnosis**: Use Gemini for root cause analysis and fix suggestions +- **Codex Resume**: Apply fixes using `codex exec "..." resume --last` for context continuity +- **Iterative Refinement**: Repeat test-analyze-fix-retest cycle until all tests pass +- **Surgical Fixes**: Minimal code changes, no refactoring during test fixes +- **Auto-Revert**: Rollback all changes if max iterations reached + +## Core Responsibilities +- Parse TEST_ANALYSIS_RESULTS.md from test-concept-enhanced +- Extract test requirements and generation strategy +- Generate test generation subtask calling @code-developer +- Generate test execution and fix cycle task JSON +- Configure Gemini diagnosis and Codex execution workflow +- Create test-oriented IMPL_PLAN.md and TODO_LIST.md with test generation phase + +## Execution Lifecycle + +### Phase 1: Input Validation & Discovery + +1. **Test Session Validation** + - Load `.workflow/{test-session-id}/workflow-session.json` + - Verify `workflow_type: "test_session"` + - Extract `source_session_id` from metadata + +2. **Test Analysis Results Loading** + - **REQUIRED**: Load `.workflow/{test-session-id}/.process/TEST_ANALYSIS_RESULTS.md` + - Parse test requirements by file + - Extract test generation strategy + - Identify test files to create with specifications + +3. **Test Context Package Loading** + - Load `.workflow/{test-session-id}/.process/test-context-package.json` + - Extract test framework configuration + - Extract coverage gaps and priorities + - Load source session implementation summaries + +### Phase 2: Task JSON Generation + +Generate **TWO task JSON files**: +1. **IMPL-001.json** - Test Generation (calls @code-developer) +2. **IMPL-002.json** - Test Execution and Fix Cycle (calls @test-fix-agent) + +#### IMPL-001.json - Test Generation Task + +```json +{ + "id": "IMPL-001", + "title": "Generate comprehensive tests for [sourceSessionId]", + "status": "pending", + "meta": { + "type": "test-gen", + "agent": "@code-developer", + "source_session": "[sourceSessionId]", + "test_framework": "jest|pytest|cargo|detected" + }, + "context": { + "requirements": [ + "Generate comprehensive test files based on TEST_ANALYSIS_RESULTS.md", + "Follow existing test patterns and conventions from test framework", + "Create tests for all missing coverage identified in analysis", + "Include happy path, error handling, edge cases, and integration tests", + "Use test data and mocks as specified in analysis", + "Ensure tests follow project coding standards" + ], + "focus_paths": [ + "tests/**/*", + "src/**/*.test.*", + "{paths_from_analysis}" + ], + "acceptance": [ + "All test files from TEST_ANALYSIS_RESULTS.md section 5 are created", + "Tests follow existing test patterns and conventions", + "Test scenarios cover happy path, errors, edge cases, integration", + "All dependencies are properly mocked", + "Test files are syntactically valid and can be executed", + "Test coverage meets analysis requirements" + ], + "depends_on": [], + "source_context": { + "session_id": "[sourceSessionId]", + "test_analysis": ".workflow/[testSessionId]/.process/TEST_ANALYSIS_RESULTS.md", + "test_context": ".workflow/[testSessionId]/.process/test-context-package.json", + "implementation_summaries": [ + ".workflow/[sourceSessionId]/.summaries/IMPL-001-summary.md" + ] + } + }, + "flow_control": { + "pre_analysis": [ + { + "step": "load_test_analysis", + "action": "Load test generation requirements and strategy", + "commands": [ + "Read(.workflow/[testSessionId]/.process/TEST_ANALYSIS_RESULTS.md)", + "Read(.workflow/[testSessionId]/.process/test-context-package.json)" + ], + "output_to": "test_generation_requirements", + "on_error": "fail" + }, + { + "step": "load_implementation_context", + "action": "Load source implementation for test generation context", + "commands": [ + "bash(for f in .workflow/[sourceSessionId]/.summaries/IMPL-*-summary.md; do echo \"=== $(basename $f) ===\"&& cat \"$f\"; done)" + ], + "output_to": "implementation_context", + "on_error": "skip_optional" + }, + { + "step": "load_existing_test_patterns", + "action": "Study existing tests for pattern reference", + "commands": [ + "mcp__code-index__find_files(pattern=\"*.test.*\")", + "bash(# Read first 2 existing test files as examples)", + "bash(test_files=$(mcp__code-index__find_files(pattern=\"*.test.*\") | head -2))", + "bash(for f in $test_files; do echo \"=== $f ===\"&& cat \"$f\"; done)" + ], + "output_to": "existing_test_patterns", + "on_error": "skip_optional" + } + ], + "implementation_approach": { + "task_description": "Generate comprehensive test suite based on TEST_ANALYSIS_RESULTS.md. Follow test generation strategy and create all test files listed in section 5 (Implementation Targets).", + "generation_steps": [ + "Read TEST_ANALYSIS_RESULTS.md section 3 (Test Requirements by File)", + "Read TEST_ANALYSIS_RESULTS.md section 4 (Test Generation Strategy)", + "Study existing test patterns from test_context.test_framework.conventions", + "For each test file in section 5 (Implementation Targets):", + " - Create test file with specified scenarios", + " - Implement happy path tests", + " - Implement error handling tests", + " - Implement edge case tests", + " - Implement integration tests (if specified)", + " - Add required mocks and fixtures", + "Follow test framework conventions and project standards", + "Ensure all tests are executable and syntactically valid" + ], + "quality_criteria": [ + "All test scenarios from analysis are implemented", + "Test structure matches existing patterns", + "Clear test descriptions and assertions", + "Proper setup/teardown and fixtures", + "Dependencies properly mocked", + "Tests follow project coding standards" + ] + }, + "target_files": [ + "{test_file_1 from TEST_ANALYSIS_RESULTS.md section 5}", + "{test_file_2 from TEST_ANALYSIS_RESULTS.md section 5}", + "{test_file_N from TEST_ANALYSIS_RESULTS.md section 5}" + ] + } +} +``` + +#### IMPL-002.json - Test Execution & Fix Cycle Task + +```json +{ + "id": "IMPL-002", + "title": "Execute and fix tests for [sourceSessionId]", + "status": "pending", + "meta": { + "type": "test-fix", + "agent": "@test-fix-agent", + "source_session": "[sourceSessionId]", + "test_framework": "jest|pytest|cargo|detected", + "max_iterations": 5 + }, + "context": { + "requirements": [ + "Execute complete test suite (generated in IMPL-001)", + "Diagnose test failures using Gemini analysis", + "Fix failures using Codex with resume mechanism", + "Iterate until all tests pass or max iterations reached", + "Revert changes if unable to fix within iteration limit" + ], + "focus_paths": [ + "tests/**/*", + "src/**/*.test.*", + "{implementation_files_from_source_session}" + ], + "acceptance": [ + "All tests pass successfully (100% pass rate)", + "No test failures or errors in final run", + "Code changes are minimal and surgical", + "All fixes are verified through retest", + "Iteration logs document fix progression" + ], + "depends_on": ["IMPL-001"], + "source_context": { + "session_id": "[sourceSessionId]", + "test_generation_summary": ".workflow/[testSessionId]/.summaries/IMPL-001-summary.md", + "implementation_summaries": [ + ".workflow/[sourceSessionId]/.summaries/IMPL-001-summary.md" + ] + } + }, + "flow_control": { + "pre_analysis": [ + { + "step": "load_source_session_summaries", + "action": "Load implementation context from source session", + "commands": [ + "bash(find .workflow/[sourceSessionId]/.summaries/ -name 'IMPL-*-summary.md' 2>/dev/null)", + "bash(for f in .workflow/[sourceSessionId]/.summaries/IMPL-*-summary.md; do echo \"=== $(basename $f) ===\"&& cat \"$f\"; done)" + ], + "output_to": "implementation_context", + "on_error": "skip_optional" + }, + { + "step": "discover_test_framework", + "action": "Identify test framework and test command", + "commands": [ + "bash(jq -r '.scripts.test // \"npm test\"' package.json 2>/dev/null || echo 'pytest' || echo 'cargo test')", + "bash([ -f 'package.json' ] && echo 'jest/npm' || [ -f 'pytest.ini' ] && echo 'pytest' || [ -f 'Cargo.toml' ] && echo 'cargo' || echo 'unknown')" + ], + "output_to": "test_command", + "on_error": "fail" + }, + { + "step": "analyze_test_coverage", + "action": "Analyze test coverage and identify missing tests", + "commands": [ + "mcp__code-index__find_files(pattern=\"*.test.*\")", + "mcp__code-index__search_code_advanced(pattern=\"test|describe|it|def test_\", file_pattern=\"*.test.*\")", + "bash(# Count implementation files vs test files)", + "bash(impl_count=$(find [changed_files_dirs] -type f \\( -name '*.ts' -o -name '*.js' -o -name '*.py' \\) ! -name '*.test.*' 2>/dev/null | wc -l))", + "bash(test_count=$(mcp__code-index__find_files(pattern=\"*.test.*\") | wc -l))", + "bash(echo \"Implementation files: $impl_count, Test files: $test_count\")" + ], + "output_to": "test_coverage_analysis", + "on_error": "skip_optional" + }, + { + "step": "identify_files_without_tests", + "action": "List implementation files that lack corresponding test files", + "commands": [ + "bash(# For each changed file from source session, check if test exists)", + "bash(for file in [changed_files]; do test_file=$(echo $file | sed 's/\\(.*\\)\\.\\(ts\\|js\\|py\\)$/\\1.test.\\2/'); [ ! -f \"$test_file\" ] && echo \"$file\"; done)" + ], + "output_to": "files_without_tests", + "on_error": "skip_optional" + }, + { + "step": "prepare_test_environment", + "action": "Ensure test environment is ready", + "commands": [ + "bash([ -f 'package.json' ] && npm install 2>/dev/null || true)", + "bash([ -f 'requirements.txt' ] && pip install -q -r requirements.txt 2>/dev/null || true)" + ], + "output_to": "environment_status", + "on_error": "skip_optional" + } + ], + "implementation_approach": { + "task_description": "Execute iterative test-fix-retest cycle using Gemini diagnosis and Codex execution", + "test_fix_cycle": { + "max_iterations": 5, + "cycle_pattern": "test → gemini_diagnose → codex_fix_resume → retest", + "tools": { + "test_execution": "bash(test_command)", + "diagnosis": "gemini-wrapper (MODE: analysis)", + "fix_application": "codex exec resume --last (MODE: write)", + "verification": "bash(test_command) + regression_check" + }, + "exit_conditions": { + "success": "all_tests_pass", + "failure": "max_iterations_reached", + "error": "test_command_not_found" + } + }, + "modification_points": [ + "PHASE 1: Initial Test Execution", + " 1.1. Discover test command from framework detection", + " 1.2. Execute initial test run: bash([test_command])", + " 1.3. Parse test output and count failures", + " 1.4. If all pass → Skip to PHASE 3 (success)", + " 1.5. If failures → Store failure output, proceed to PHASE 2", + "", + "PHASE 2: Iterative Test-Fix-Retest Cycle (max 5 iterations)", + " Note: This phase handles test failures, NOT test generation failures", + " Initialize: max_iterations=5, current_iteration=0", + " ", + " WHILE (tests failing AND current_iteration < max_iterations):", + " current_iteration++", + " ", + " STEP 2.1: Gemini Diagnosis", + " - Prepare diagnosis context:", + " * Test failure output from previous run", + " * Source files from focus_paths", + " * Implementation summaries from source session", + " - Execute Gemini analysis:", + " bash(cd .workflow/WFS-test-[session]/.process && ~/.claude/scripts/gemini-wrapper -p \"", + " PURPOSE: Diagnose test failure iteration [N] and propose minimal fix", + " TASK: Analyze test output and identify root cause with specific code change", + " MODE: analysis", + " CONTEXT: Test output: [test_failures]", + " Source files: [focus_paths]", + " Implementation: [implementation_context]", + " EXPECTED: Root cause analysis with fix in format:", + " FILE: path/to/file.ts", + " FUNCTION: functionName", + " LINES: 45-52", + " CHANGE: Specific code modification to apply", + " RULES: Minimal surgical fixes only - no refactoring", + " Focus on test failure, not code quality", + " \" > fix-iteration-[N]-diagnosis.md)", + " - Parse diagnosis → extract fix_suggestion and target_files", + " ", + " STEP 2.2: Codex Fix with Resume Mechanism", + " - Stage current changes (if valid git repo): bash(git add -A)", + " - First iteration: Start new Codex session", + " codex -C [project_root] --full-auto exec \"", + " PURPOSE: Fix test failure iteration 1", + " TASK: [fix_suggestion from Gemini]", + " MODE: write", + " CONTEXT: Diagnosis: .workflow/.process/fix-iteration-1-diagnosis.md", + " Target files: [target_files]", + " Implementation context: [implementation_context]", + " EXPECTED: Minimal code changes to resolve test failure", + " RULES: Apply ONLY suggested changes, no refactoring", + " Preserve existing code style", + " \" --skip-git-repo-check -s danger-full-access", + " - Subsequent iterations: Resume session for context continuity", + " codex exec \"", + " CONTINUE TO NEXT FIX:", + " Iteration [N] of 5: Fix test failure", + " ", + " PURPOSE: Fix remaining test failures", + " TASK: [fix_suggestion from Gemini iteration N]", + " CONTEXT: Previous fixes applied, diagnosis: .process/fix-iteration-[N]-diagnosis.md", + " EXPECTED: Surgical fix for current failure", + " RULES: Build on previous fixes, maintain consistency", + " \" resume --last --skip-git-repo-check -s danger-full-access", + " - Store fix log: .process/fix-iteration-[N]-changes.log", + " ", + " STEP 2.3: Retest and Verification", + " - Re-execute test suite: bash([test_command])", + " - Capture output: .process/fix-iteration-[N]-retest.log", + " - Count failures: bash(grep -c 'FAIL\\|ERROR' .process/fix-iteration-[N]-retest.log)", + " - Check for regression:", + " IF new_failures > previous_failures:", + " WARN: Regression detected", + " Include in next Gemini diagnosis context", + " - Analyze results:", + " IF all_tests_pass:", + " BREAK loop → Proceed to PHASE 3", + " ELSE:", + " Update test_failures context", + " CONTINUE loop", + " ", + " IF max_iterations reached AND tests still failing:", + " EXECUTE: git reset --hard HEAD (revert all changes)", + " MARK: Task status = blocked", + " GENERATE: Detailed failure report with iteration logs", + " EXIT: Require manual intervention", + "", + "PHASE 3: Final Validation and Certification", + " 3.1. Execute final confirmation test run", + " 3.2. Generate success summary:", + " - Iterations required: [current_iteration]", + " - Fixes applied: [summary from iteration logs]", + " - Test results: All passing ✅", + " 3.3. Mark task status: completed", + " 3.4. Update TODO_LIST.md: Mark as ✅", + " 3.5. Certify code: APPROVED for deployment" + ], + "logic_flow": [ + "Load source session implementation context", + "Discover test framework and command", + "PHASE 0: Test Coverage Check", + " Analyze existing test files", + " Identify files without tests", + " IF tests missing:", + " Generate tests using Codex", + " Verify test generation successful", + " ELSE:", + " Skip to Phase 1", + "PHASE 1: Initial Test Execution", + " Execute test suite", + " IF all pass → Success (Phase 3)", + " ELSE → Store failures, proceed to Phase 2", + "PHASE 2: Iterative Fix Cycle (max 5 iterations)", + " LOOP (max 5 times):", + " 1. Gemini diagnoses failure → fix suggestion", + " 2. Codex applies fix (resume for continuity)", + " 3. Retest and check results", + " 4. IF pass → Exit loop to Phase 3", + " 5. ELSE → Continue with updated context", + " IF max iterations → Revert + report failure", + "PHASE 3: Final Validation", + " Confirm all tests pass", + " Generate summary (include test generation info)", + " Certify code APPROVED" + ], + "error_handling": { + "max_iterations_reached": { + "action": "revert_all_changes", + "commands": [ + "bash(git reset --hard HEAD)", + "bash(jq '.status = \"blocked\"' .workflow/[session]/.task/IMPL-001.json > temp.json && mv temp.json .workflow/[session]/.task/IMPL-001.json)" + ], + "report": "Generate failure report with iteration logs in .summaries/IMPL-001-failure-report.md" + }, + "test_command_fails": { + "action": "treat_as_test_failure", + "context": "Use stderr as failure context for Gemini diagnosis" + }, + "codex_apply_fails": { + "action": "retry_once_then_skip", + "fallback": "Mark iteration as skipped, continue to next" + }, + "gemini_diagnosis_fails": { + "action": "retry_with_simplified_context", + "fallback": "Use previous diagnosis, continue" + }, + "regression_detected": { + "action": "log_warning_continue", + "context": "Include regression info in next Gemini diagnosis" + } + } + }, + "target_files": [ + "Auto-discovered from test failures", + "Extracted from Gemini diagnosis each iteration", + "Format: file:function:lines or file (for new files)" + ], + "codex_session": { + "strategy": "resume_for_continuity", + "first_iteration": "codex exec \"fix iteration 1\" --full-auto", + "subsequent_iterations": "codex exec \"fix iteration N\" resume --last", + "benefits": [ + "Maintains conversation context across fixes", + "Remembers previous decisions and patterns", + "Ensures consistency in fix approach", + "Reduces redundant context injection" + ] + } + } +} +``` + +### Phase 3: IMPL_PLAN.md Generation + +#### Document Structure +```markdown +--- +identifier: WFS-test-[session-id] +source_session: WFS-[source-session-id] +workflow_type: test_session +test_framework: jest|pytest|cargo|detected +--- + +# Test Validation Plan: [Source Session Topic] + +## Summary +Execute comprehensive test suite for implementation from session WFS-[source-session-id]. +Diagnose and fix all test failures using iterative Gemini analysis and Codex execution. + +## Source Session Context +- **Implementation Session**: WFS-[source-session-id] +- **Completed Tasks**: IMPL-001, IMPL-002, ... +- **Changed Files**: [list from git log] +- **Implementation Summaries**: [references to source session summaries] + +## Test Framework +- **Detected Framework**: jest|pytest|cargo|other +- **Test Command**: npm test|pytest|cargo test +- **Test Files**: [discovered test files] +- **Coverage**: [estimated test coverage] + +## Test-Fix-Retest Cycle +- **Max Iterations**: 5 +- **Diagnosis Tool**: Gemini (analysis mode) +- **Fix Tool**: Codex (resume mechanism) +- **Verification**: Bash test execution + regression check + +### Cycle Workflow +1. **Initial Test**: Execute full suite, capture failures +2. **Iterative Fix Loop** (max 5 times): + - Gemini diagnoses failure → surgical fix suggestion + - Codex applies fix using resume for context continuity + - Retest and verify (check for regressions) + - Continue until all pass or max iterations reached +3. **Final Validation**: Confirm all tests pass, certify code + +### Error Recovery +- **Max iterations reached**: Revert all changes, report failure +- **Test command fails**: Treat as test failure, diagnose with Gemini +- **Codex fails**: Retry once, skip iteration if still failing +- **Regression detected**: Log warning, include in next diagnosis + +## Task Breakdown +- **IMPL-001**: Execute and validate tests with iterative fix cycle + +## Implementation Strategy +- **Phase 1**: Initial test execution and failure capture +- **Phase 2**: Iterative Gemini diagnosis + Codex fix + retest +- **Phase 3**: Final validation and code certification + +## Success Criteria +- All tests pass (100% pass rate) +- No test failures or errors in final run +- Minimal, surgical code changes +- Iteration logs document fix progression +- Code certified APPROVED for deployment +``` + +### Phase 4: TODO_LIST.md Generation + +```markdown +# Tasks: Test Validation for [Source Session] + +## Task Progress +- [ ] **IMPL-001**: Execute and validate tests with iterative fix cycle → [📋](./.task/IMPL-001.json) + +## Execution Details +- **Source Session**: WFS-[source-session-id] +- **Test Framework**: jest|pytest|cargo +- **Max Iterations**: 5 +- **Tools**: Gemini diagnosis + Codex resume fixes + +## Status Legend +- `- [ ]` = Pending +- `- [x]` = Completed +``` + +## Output Files Structure +``` +.workflow/WFS-test-[session]/ +├── workflow-session.json # Test session metadata +├── IMPL_PLAN.md # Test validation plan +├── TODO_LIST.md # Progress tracking +├── .task/ +│ └── IMPL-001.json # Test-fix task with cycle spec +├── .process/ +│ ├── ANALYSIS_RESULTS.md # From concept-enhanced (optional) +│ ├── context-package.json # From context-gather +│ ├── initial-test.log # Phase 1: Initial test results +│ ├── fix-iteration-1-diagnosis.md # Gemini diagnosis iteration 1 +│ ├── fix-iteration-1-changes.log # Codex changes iteration 1 +│ ├── fix-iteration-1-retest.log # Retest results iteration 1 +│ ├── fix-iteration-N-*.md/log # Subsequent iterations +│ └── final-test.log # Phase 3: Final validation +└── .summaries/ + └── IMPL-001-summary.md # Success report OR failure report +``` + +## Error Handling + +### Input Validation Errors +| Error | Cause | Resolution | +|-------|-------|------------| +| Not a test session | Missing workflow_type: "test_session" | Verify session created by test-gen | +| Source session not found | Invalid source_session_id | Check source session exists | +| No implementation summaries | Source session incomplete | Ensure source session has completed tasks | + +### Test Framework Discovery Errors +| Error | Cause | Resolution | +|-------|-------|------------| +| No test command found | Unknown framework | Manual test command specification | +| No test files found | Tests not written | Request user to write tests first | +| Test dependencies missing | Incomplete setup | Run dependency installation | + +### Generation Errors +| Error | Cause | Resolution | +|-------|-------|------------| +| Invalid JSON structure | Template error | Fix task generation logic | +| Missing required fields | Incomplete metadata | Validate session metadata | + +## Integration & Usage + +### Command Chain +- **Called By**: `/workflow:test-gen` (Phase 4) +- **Calls**: None (terminal command) +- **Followed By**: `/workflow:execute` (user-triggered) + +### Basic Usage +```bash +/workflow:tools:test-task-generate --session WFS-test-auth +``` + +## Related Commands +- `/workflow:test-gen` - Creates test session and calls this tool +- `/workflow:tools:context-gather` - Provides cross-session context +- `/workflow:tools:concept-enhanced` - Provides test strategy analysis +- `/workflow:execute` - Executes the generated test-fix task +- `@test-fix-agent` - Agent that executes the iterative test-fix cycle + +## Agent Execution Notes + +The `@test-fix-agent` will execute the task by following the `flow_control.implementation_approach` specification: + +1. **Load task JSON**: Read complete test-fix task from `.task/IMPL-001.json` +2. **Execute pre_analysis**: Load source context, discover framework, analyze tests +3. **Phase 1**: Run initial test suite +4. **Phase 2**: If failures, enter iterative loop: + - Use Gemini for diagnosis (analysis mode) + - Use Codex resume for fixes (maintains context) + - Retest and check for regressions + - Repeat max 5 times +5. **Phase 3**: Generate summary and certify code +6. **Error Recovery**: Revert changes if max iterations reached + +The agent uses the `codex exec "..." resume --last` pattern to maintain conversation context across multiple fix iterations, ensuring consistency and learning from previous attempts.