mirror of
https://github.com/catlog22/Claude-Code-Workflow.git
synced 2026-02-05 01:50:27 +08:00
Implement two major enhancements to test generation workflow: 1. Gemini Test Enhancement (Phase 1.5) - Add Gemini CLI analysis for comprehensive test suggestions - Generate enriched test specifications for L1-L3 test layers - Focus on API contracts, integration patterns, error scenarios - Create gemini-enriched-suggestions.md as artifact 2. Planning Notes Mechanism (test-planning-notes.md) - Similar to plan.md's planning-notes pattern - Record Test Intent (Phase 1) - Embed complete Gemini enrichment (Phase 1.5) - Track consolidated test requirements - Support N+1 context with decisions and deferred items Implementation Details: - Created Gemini prompt template: test-suggestions-enhancement.txt - Enhanced test-task-generate.md with Phase 1, 1.5, 2 logic - Gemini content embedded in planning-notes for single source of truth - test-action-planning-agent reads both TEST_ANALYSIS_RESULTS.md and test-planning-notes.md - Backward compatible: Phase 1.5 is optional enhancement Benefits: - Comprehensive test coverage (API, integration, error scenarios) - Full traceability of test planning process - Consolidated context in one file for easy review - Preserved Gemini output as independent artifact
217 lines
8.2 KiB
Markdown
217 lines
8.2 KiB
Markdown
# Test Generation Enhancement - Implementation Summary
|
|
|
|
## Overview
|
|
|
|
Enhanced test-task-generate workflow with two major improvements:
|
|
1. **Gemini Test Enhancement** (Phase 1.5): Comprehensive test suggestions for API, integration, and error scenarios
|
|
2. **Planning Notes Mechanism** (test-planning-notes.md): Record Gemini enhancement, context, and plan process for full transparency
|
|
|
|
---
|
|
|
|
## Changes Made
|
|
|
|
### 1. Created Gemini Prompt Template
|
|
**File**: `~/.claude/workflows/cli-templates/prompts/test/test-suggestions-enhancement.txt`
|
|
- Structured prompt for Gemini CLI to generate comprehensive test suggestions
|
|
- Covers L1-L3 test layers with focus on API, integration, and error scenarios
|
|
- Produces actionable test specifications with expected behavior and success criteria
|
|
|
|
### 2. Enhanced test-task-generate.md with Two Major Features
|
|
|
|
**Location**: `.claude/commands/workflow/tools/test-task-generate.md`
|
|
|
|
#### A. Phase 1.5: Gemini Test Enhancement
|
|
- **Invocation of cli-execution-agent** for Gemini analysis
|
|
- **Template**: test-suggestions-enhancement.txt
|
|
- **Output**: gemini-enriched-suggestions.md
|
|
- **Merge**: Appends enriched suggestions to TEST_ANALYSIS_RESULTS.md
|
|
|
|
#### B. test-planning-notes.md Mechanism (NEW - References plan.md pattern)
|
|
Similar to plan.md's planning-notes.md, this records the entire planning journey.
|
|
|
|
**Lifecycle**:
|
|
1. **Phase 1 Creation**: Initialize test-planning-notes.md with Test Intent
|
|
2. **Phase 1.5 Update**: Record Gemini Enhancement findings
|
|
3. **Phase 2 Update**: Final Task Generation findings (agent-executed)
|
|
|
|
---
|
|
|
|
## Planning Notes Architecture (Similar to plan.md)
|
|
|
|
### test-planning-notes.md Structure
|
|
|
|
```markdown
|
|
# Test Planning Notes
|
|
|
|
**Session**: WFS-test-xxx
|
|
**Source Session**: WFS-xxx
|
|
**Created**: [timestamp]
|
|
|
|
## Test Intent (Phase 1)
|
|
- **PROJECT_TYPE**: [Detected project type]
|
|
- **TEST_FRAMEWORK**: [Detected framework]
|
|
- **COVERAGE_TARGET**: [Coverage goal]
|
|
- **SOURCE_SESSION**: [Source session ID]
|
|
|
|
---
|
|
|
|
## Context Findings (Phase 1)
|
|
### Files with Coverage Gaps
|
|
[Extracted from TEST_ANALYSIS_RESULTS.md]
|
|
|
|
### Test Framework & Conventions
|
|
- Framework: [Framework]
|
|
- Coverage Target: [Target]
|
|
|
|
---
|
|
|
|
## Gemini Enhancement (Phase 1.5)
|
|
|
|
**Analysis Timestamp**: [timestamp]
|
|
**Template**: test-suggestions-enhancement.txt
|
|
**Output File**: .process/gemini-enriched-suggestions.md
|
|
|
|
### Enriched Test Suggestions (Complete Gemini Analysis)
|
|
|
|
[COMPLETE CONTENT FROM gemini-enriched-suggestions.md EMBEDDED HERE]
|
|
[All L1-L3 test suggestions, API contracts, integration patterns, error scenarios]
|
|
|
|
### Gemini Analysis Summary
|
|
- **Status**: Enrichment complete
|
|
- **Layers Covered**: L1, L2.1, L2.2, L2.4, L2.5
|
|
- **Focus Areas**: API contracts, integration patterns, error scenarios, edge cases
|
|
- **Output Stored**: Full analysis in gemini-enriched-suggestions.md
|
|
|
|
### Key Features
|
|
|
|
1. **Complete Gemini Analysis Embedded**: Full enriched suggestions content is directly embedded into test-planning-notes.md
|
|
2. **Preserved Output File**: gemini-enriched-suggestions.md still generated as artifact for reference
|
|
3. **Consolidated Context**: test-planning-notes.md becomes single source of truth for all planning context
|
|
4. **Agent Friendly**: test-action-planning-agent can read complete planning context from one file
|
|
|
|
### Use Cases for Gemini Content in planning-notes.md
|
|
|
|
| Use Case | Benefit |
|
|
|----------|---------|
|
|
| Reviewer reads planning-notes | See full Gemini suggestions without opening separate file |
|
|
| Agent reads planning-notes | Get enriched context alongside other planning metadata |
|
|
| Archive session | Complete record of analysis in one file |
|
|
| N+1 phase reference | Full history of suggestions for iterative refinement |
|
|
|
|
---
|
|
|
|
## Consolidated Test Requirements (Phase 2 Input)
|
|
1. [Context] Test framework conventions
|
|
2. [Context] Coverage target
|
|
3. [Gemini] API contract test strategies
|
|
4. [Gemini] Error scenario coverage
|
|
[... more consolidated requirements]
|
|
|
|
---
|
|
|
|
## Task Generation (Phase 2)
|
|
[Updated by test-action-planning-agent]
|
|
|
|
## N+1 Context
|
|
### Decisions
|
|
| Decision | Rationale | Revisit? |
|
|
|----------|-----------|----------|
|
|
[Decisions made during planning]
|
|
|
|
### Deferred
|
|
- [ ] Items for N+1 phase
|
|
```
|
|
|
|
### Key Benefits of Planning Notes
|
|
|
|
1. **Full Transparency**: Record Gemini enhancement findings and context
|
|
2. **Consolidated Context**: All phase inputs/outputs in one place
|
|
3. **N+1 Support**: Track decisions and deferred items for future iterations
|
|
4. **Agent Context**: test-action-planning-agent reads consolidated requirements
|
|
5. **Traceability**: How each test requirement came from which phase/source
|
|
6. **Integration Consistency**: Follows same pattern as plan.md for familiarity
|
|
|
|
---
|
|
|
|
## Complete Integration Flow
|
|
|
|
```
|
|
test-task-generate execution:
|
|
├─ Phase 1: Context Preparation
|
|
│ ├─ Load session metadata
|
|
│ ├─ Load TEST_ANALYSIS_RESULTS.md (original)
|
|
│ └─ CREATE test-planning-notes.md (Test Intent section)
|
|
│
|
|
├─ Phase 1.5: Gemini Test Enhancement
|
|
│ ├─ Invoke cli-execution-agent for Gemini analysis
|
|
│ ├─ Generate gemini-enriched-suggestions.md (full analysis output)
|
|
│ └─ RECORD complete enrichment to test-planning-notes.md (embed full Gemini analysis)
|
|
│
|
|
└─ Phase 2: Test Document Generation (Agent)
|
|
├─ Load TEST_ANALYSIS_RESULTS.md (original, unchanged)
|
|
├─ Load test-planning-notes.md (includes full Gemini enrichment)
|
|
├─ Generate task JSONs (IMPL-*.json)
|
|
└─ Create IMPL_PLAN.md + TODO_LIST.md
|
|
```
|
|
|
|
## Key Design Decisions
|
|
|
|
### Gemini Enhancement Recording Strategy
|
|
|
|
**Decision**: Record Gemini enhancement to test-planning-notes.md instead of appending to TEST_ANALYSIS_RESULTS.md
|
|
|
|
**Rationale**:
|
|
1. **Separation of Concerns**: TEST_ANALYSIS_RESULTS.md remains as original analysis output
|
|
2. **Single Source of Truth**: test-planning-notes.md consolidates ALL planning context
|
|
3. **Better Traceability**: Clear separation between base analysis and AI enrichment
|
|
4. **Follows plan.md Pattern**: Consistent with planning-notes.md mechanism
|
|
|
|
**Implementation**:
|
|
- Phase 1.5 generates gemini-enriched-suggestions.md (preserved as output file)
|
|
- Complete Gemini analysis content is embedded into test-planning-notes.md
|
|
- test-action-planning-agent reads both TEST_ANALYSIS_RESULTS.md and test-planning-notes.md
|
|
|
|
## No Breaking Changes
|
|
|
|
**Backward Compatible**:
|
|
- Phase 1.5 is optional enhancement - if Gemini analysis fails, workflow continues with original TEST_ANALYSIS_RESULTS.md
|
|
- test-action-planning-agent requires NO modifications to source code
|
|
- test-planning-notes.md consolidates context but doesn't replace existing inputs
|
|
- Existing workflows unaffected; only adds richer suggestions and traceability
|
|
|
|
**Enriched Layers**:
|
|
- **L1 (Unit)**: Edge cases, boundary conditions, error path validation
|
|
- **L2.1 (Integration)**: Module interactions, dependency injection patterns
|
|
- **L2.2 (API Contracts)**: Request/response, validation, status codes, error responses
|
|
- **L2.4 (External APIs)**: Mock strategies, failure scenarios, timeout handling
|
|
- **L2.5 (Failure Modes)**: Exception hierarchies, error propagation, recovery strategies
|
|
|
|
**Benefits**:
|
|
- ✅ Comprehensive API test specifications
|
|
- ✅ Detailed integration test strategies
|
|
- ✅ Systematic error scenario coverage
|
|
- ✅ More actionable task generation
|
|
- ✅ Reduced manual test planning effort
|
|
|
|
## How Test-Action-Planning-Agent Uses Enrichment
|
|
|
|
**Phase 1 (Context Loading)**:
|
|
- test-action-planning-agent reads complete TEST_ANALYSIS_RESULTS.md
|
|
- Includes both original analysis AND Gemini-enhanced suggestions
|
|
- Automatically leverages enriched content for task generation
|
|
|
|
**IMPL-001.json (Test Generation)**:
|
|
- Generated with reference to enriched test specifications
|
|
- Enhanced L1-L3 layer requirements from Gemini analysis
|
|
- More detailed test case descriptions
|
|
- Better integration and error scenario coverage
|
|
|
|
## Verification
|
|
|
|
To verify the enhancement is working:
|
|
1. Run `/workflow:test-fix-gen` to trigger test workflow
|
|
2. Check `.workflow/active/[session]/.process/gemini-enriched-suggestions.md` exists
|
|
3. Check TEST_ANALYSIS_RESULTS.md contains "## Gemini-Enhanced Test Suggestions" section
|
|
4. Review IMPL-001.json - should reference enriched test specifications
|
|
5. Execute `/workflow:test-cycle-execute` - generated tests should cover API, integration, and error scenarios comprehensively
|