mirror of
https://github.com/catlog22/Claude-Code-Workflow.git
synced 2026-03-10 17:11:04 +08:00
feat: Add templates for epics, product brief, and requirements documentation
- Introduced a comprehensive template for generating epics and stories in Phase 5, including an index and individual epic files. - Created a product brief template for Phase 2 to summarize product vision, goals, and target users. - Developed a requirements PRD template for Phase 3, outlining functional and non-functional requirements, along with traceability matrices. feat: Implement tech debt roles for assessment, execution, planning, scanning, validation, and analysis - Added roles for tech debt assessment, executor, planner, scanner, validator, and analyst, each with defined phases and processes for managing technical debt. - Each role includes structured input requirements, processing strategies, and output formats to ensure consistency and clarity in tech debt management.
This commit is contained in:
catlog22
@@ -1,308 +1,62 @@
|
||||
---
|
||||
name: team-testing
|
||||
description: Unified team skill for testing team. All roles invoke this skill with --role arg for role-specific execution. Triggers on "team testing".
|
||||
description: Unified team skill for testing team. Progressive test coverage through Generator-Critic loops, shared memory, and dynamic layer selection. Triggers on "team testing".
|
||||
allowed-tools: TeamCreate(*), TeamDelete(*), SendMessage(*), TaskCreate(*), TaskUpdate(*), TaskList(*), TaskGet(*), Agent(*), AskUserQuestion(*), Read(*), Write(*), Edit(*), Bash(*), Glob(*), Grep(*)
|
||||
---
|
||||
|
||||
# Team Testing
|
||||
|
||||
Unified team skill: progressive test coverage through Generator-Critic loops (generator<->executor), shared memory (defect pattern tracking), and dynamic layer selection. All team members invoke with `--role=xxx` to route to role-specific execution.
|
||||
Orchestrate multi-agent test pipeline: strategist -> generator -> executor -> analyst. Progressive layer coverage (L1/L2/L3) with Generator-Critic loops for coverage convergence.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
+---------------------------------------------------+
|
||||
| Skill(skill="team-testing") |
|
||||
| args="<task-description>" |
|
||||
+-------------------+-------------------------------+
|
||||
Skill(skill="team-testing", args="task description")
|
||||
|
|
||||
Orchestration Mode (auto -> coordinator)
|
||||
SKILL.md (this file) = Router
|
||||
|
|
||||
Coordinator (inline)
|
||||
Phase 0-5 orchestration
|
||||
|
|
||||
+-------+-------+-------+-------+
|
||||
v v v v
|
||||
[tw] [tw] [tw] [tw]
|
||||
strate- genera- execu- analyst
|
||||
gist tor tor
|
||||
|
||||
(tw) = team-worker agent
|
||||
+--------------+--------------+
|
||||
| |
|
||||
no --role flag --role <name>
|
||||
| |
|
||||
Coordinator Worker
|
||||
roles/coordinator/role.md roles/<name>/role.md
|
||||
|
|
||||
+-- analyze -> dispatch -> spawn workers -> STOP
|
||||
|
|
||||
+-------+-------+-------+-------+
|
||||
v v v v
|
||||
[strat] [gen] [exec] [analyst]
|
||||
team-worker agents, each loads roles/<role>/role.md
|
||||
```
|
||||
|
||||
## Command Execution Protocol
|
||||
## Role Registry
|
||||
|
||||
When coordinator needs to execute a command (dispatch, monitor):
|
||||
|
||||
1. **Read the command file**: `roles/coordinator/commands/<command-name>.md`
|
||||
2. **Follow the workflow** defined in the command file (Phase 2-4 structure)
|
||||
3. **Commands are inline execution guides** -- NOT separate agents or subprocesses
|
||||
4. **Execute synchronously** -- complete the command workflow before proceeding
|
||||
| Role | Path | Prefix | Inner Loop |
|
||||
|------|------|--------|------------|
|
||||
| coordinator | [roles/coordinator/role.md](roles/coordinator/role.md) | — | — |
|
||||
| strategist | [roles/strategist/role.md](roles/strategist/role.md) | STRATEGY-* | false |
|
||||
| generator | [roles/generator/role.md](roles/generator/role.md) | TESTGEN-* | true |
|
||||
| executor | [roles/executor/role.md](roles/executor/role.md) | TESTRUN-* | true |
|
||||
| analyst | [roles/analyst/role.md](roles/analyst/role.md) | TESTANA-* | false |
|
||||
|
||||
## Role Router
|
||||
|
||||
### Input Parsing
|
||||
Parse `$ARGUMENTS`:
|
||||
- Has `--role <name>` -> Read `roles/<name>/role.md`, execute Phase 2-4
|
||||
- No `--role` -> Read `roles/coordinator/role.md`, execute entry router
|
||||
|
||||
Parse `$ARGUMENTS` to extract `--role`. If absent → Orchestration Mode (auto route to coordinator).
|
||||
## Shared Constants
|
||||
|
||||
### Role Registry
|
||||
- **Session prefix**: `TST`
|
||||
- **Session path**: `.workflow/.team/TST-<slug>-<date>/`
|
||||
- **Team name**: `testing`
|
||||
- **CLI tools**: `ccw cli --mode analysis` (read-only), `ccw cli --mode write` (modifications)
|
||||
- **Message bus**: `mcp__ccw-tools__team_msg(session_id=<session-id>, ...)`
|
||||
|
||||
| Role | Spec | Task Prefix | Inner Loop |
|
||||
|------|------|-------------|------------|
|
||||
| coordinator | [roles/coordinator/role.md](roles/coordinator/role.md) | (none) | - |
|
||||
| strategist | [role-specs/strategist.md](role-specs/strategist.md) | STRATEGY-* | false |
|
||||
| generator | [role-specs/generator.md](role-specs/generator.md) | TESTGEN-* | true |
|
||||
| executor | [role-specs/executor.md](role-specs/executor.md) | TESTRUN-* | true |
|
||||
| analyst | [role-specs/analyst.md](role-specs/analyst.md) | TESTANA-* | false |
|
||||
## Worker Spawn Template
|
||||
|
||||
> **⚠️ COMPACT PROTECTION**: 角色文件是执行文档，不是参考资料。当 context compression 发生后，角色指令仅剩摘要时，**必须立即 `Read` 对应 role.md 重新加载后再继续执行**。不得基于摘要执行任何 Phase。
|
||||
|
||||
### Dispatch
|
||||
|
||||
1. Extract `--role` from arguments
|
||||
2. If no `--role` → route to coordinator (Orchestration Mode)
|
||||
3. Look up role in registry → Read the role file → Execute its phases
|
||||
|
||||
### Orchestration Mode
|
||||
|
||||
When invoked without `--role`, coordinator auto-starts. User just provides task description.
|
||||
|
||||
**Invocation**: `Skill(skill="team-testing", args="<task-description>")`
|
||||
|
||||
**Lifecycle**:
|
||||
```
|
||||
User provides task description
|
||||
→ coordinator Phase 1-3: Change scope analysis → TeamCreate → Create task chain
|
||||
→ coordinator Phase 4: spawn first batch workers (background) → STOP
|
||||
→ Worker executes → SendMessage callback → coordinator advances next step
|
||||
→ Loop until pipeline complete → Phase 5 report
|
||||
```
|
||||
|
||||
**User Commands** (wake paused coordinator):
|
||||
|
||||
| Command | Action |
|
||||
|---------|--------|
|
||||
| `check` / `status` | Output execution status graph, no advancement |
|
||||
| `resume` / `continue` | Check worker states, advance next step |
|
||||
|
||||
---
|
||||
|
||||
## Shared Infrastructure
|
||||
|
||||
The following templates apply to all worker roles. Each role.md only needs to write **Phase 2-4** role-specific logic.
|
||||
|
||||
### Worker Phase 1: Task Discovery (shared by all workers)
|
||||
|
||||
Every worker executes the same task discovery flow on startup:
|
||||
|
||||
1. Call `TaskList()` to get all tasks
|
||||
2. Filter: subject matches this role's prefix + owner is this role + status is pending + blockedBy is empty
|
||||
3. No tasks → idle wait
|
||||
4. Has tasks → `TaskGet` for details → `TaskUpdate` mark in_progress
|
||||
|
||||
**Resume Artifact Check** (prevent duplicate output after resume):
|
||||
- Check whether this task's output artifact already exists
|
||||
- Artifact complete → skip to Phase 5 report completion
|
||||
- Artifact incomplete or missing → normal Phase 2-4 execution
|
||||
|
||||
### Worker Phase 5: Report (shared by all workers)
|
||||
|
||||
Standard reporting flow after task completion:
|
||||
|
||||
1. **Message Bus**: Call `mcp__ccw-tools__team_msg` to log message
|
||||
- Parameters: operation="log", session_id=<session-id>, from=<role>, type=<message-type>, data={ref: "<artifact-path>"}
|
||||
- `to` and `summary` auto-defaulted -- do NOT specify explicitly
|
||||
- **CLI fallback**: `ccw team log --session-id <session-id> --from <role> --type <type> --json`
|
||||
2. **SendMessage**: Send result to coordinator (content and summary both prefixed with `[<role>]`)
|
||||
3. **TaskUpdate**: Mark task completed
|
||||
4. **Loop**: Return to Phase 1 to check next task
|
||||
|
||||
### Wisdom Accumulation (all roles)
|
||||
|
||||
Cross-task knowledge accumulation. Coordinator creates `wisdom/` directory at session initialization.
|
||||
|
||||
**Directory**:
|
||||
```
|
||||
<session-folder>/wisdom/
|
||||
├── learnings.md # Patterns and insights
|
||||
├── decisions.md # Architecture and design decisions
|
||||
├── conventions.md # Codebase conventions
|
||||
└── issues.md # Known risks and issues
|
||||
```
|
||||
|
||||
**Worker Load** (Phase 2): Extract `Session: <path>` from task description, read wisdom directory files.
|
||||
**Worker Contribute** (Phase 4/5): Write this task's discoveries to corresponding wisdom files.
|
||||
|
||||
### Role Isolation Rules
|
||||
|
||||
| Allowed | Forbidden |
|
||||
|---------|-----------|
|
||||
| Process tasks with own prefix | Process tasks with other role prefixes |
|
||||
| SendMessage to coordinator | Communicate directly with other workers |
|
||||
| Share state via team_msg(type='state_update') | Create tasks for other roles |
|
||||
| Delegate to commands/ files | Modify resources outside own responsibility |
|
||||
|
||||
Coordinator additional restrictions: Do not write tests directly, do not execute tests, do not analyze coverage, do not bypass workers.
|
||||
|
||||
### Output Tagging
|
||||
|
||||
All outputs must carry `[role_name]` prefix in both SendMessage content/summary and team_msg summary.
|
||||
|
||||
### Message Bus (All Roles)
|
||||
|
||||
Every SendMessage **before**, must call `mcp__ccw-tools__team_msg` to log:
|
||||
|
||||
**Parameters**: operation="log", session_id=<session-id>, from=<role>, type=<message-type>, data={ref: "<artifact-path>"}
|
||||
|
||||
> `to` and `summary` are auto-defaulted by the tool.
|
||||
|
||||
**CLI fallback**: When MCP unavailable → `ccw team log --session-id <session-id> --from <role> --type <type> --json`
|
||||
|
||||
**Message types by role**:
|
||||
|
||||
| Role | Types |
|
||||
|------|-------|
|
||||
| coordinator | `pipeline_selected`, `gc_loop_trigger`, `quality_gate`, `task_unblocked`, `error`, `shutdown` |
|
||||
| strategist | `strategy_ready`, `error` |
|
||||
| generator | `tests_generated`, `tests_revised`, `error` |
|
||||
| executor | `tests_passed`, `tests_failed`, `coverage_report`, `error` |
|
||||
| analyst | `analysis_ready`, `error` |
|
||||
|
||||
### Shared State
|
||||
|
||||
All roles share state via `team_msg(type='state_update')` + `meta.json`:
|
||||
|
||||
| Role | Field |
|
||||
|------|-------|
|
||||
| strategist | `test_strategy` |
|
||||
| generator | `generated_tests` |
|
||||
| executor | `execution_results`, `defect_patterns` |
|
||||
| analyst | `analysis_report`, `coverage_history` |
|
||||
|
||||
### Team Configuration
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Team name | testing |
|
||||
| Session directory | `.workflow/.team/TST-<slug>-<date>/` |
|
||||
| Test layers | L1: Unit (80%), L2: Integration (60%), L3: E2E (40%) |
|
||||
|
||||
---
|
||||
|
||||
## Three-Pipeline Architecture
|
||||
|
||||
```
|
||||
Targeted (small-scope changes):
|
||||
STRATEGY-001 → TESTGEN-001(L1 unit) → TESTRUN-001
|
||||
|
||||
Standard (progressive):
|
||||
STRATEGY-001 → TESTGEN-001(L1) → TESTRUN-001(L1) → TESTGEN-002(L2) → TESTRUN-002(L2) → TESTANA-001
|
||||
|
||||
Comprehensive (full coverage):
|
||||
STRATEGY-001 → [TESTGEN-001(L1) + TESTGEN-002(L2)](parallel) → [TESTRUN-001(L1) + TESTRUN-002(L2)](parallel) → TESTGEN-003(L3) → TESTRUN-003(L3) → TESTANA-001
|
||||
```
|
||||
|
||||
### Generator-Critic Loop
|
||||
|
||||
generator <-> executor loop (revise tests when coverage below target):
|
||||
|
||||
```
|
||||
TESTGEN → TESTRUN → (if coverage < target) → TESTGEN-fix → TESTRUN-2
|
||||
(if coverage >= target) → next layer or TESTANA
|
||||
```
|
||||
|
||||
### Cadence Control
|
||||
|
||||
**Beat model**: Event-driven, each beat = coordinator wake → process → spawn → STOP. Testing beat: strategy → generate → execute → analyze.
|
||||
|
||||
```
|
||||
Beat Cycle (single beat)
|
||||
═══════════════════════════════════════════════════════════
|
||||
Event Coordinator Workers
|
||||
───────────────────────────────────────────────────────────
|
||||
callback/resume ──→ ┌─ handleCallback ─┐
|
||||
│ mark completed │
|
||||
│ check pipeline │
|
||||
├─ handleSpawnNext ─┤
|
||||
│ find ready tasks │
|
||||
│ spawn workers ───┼──→ [Worker A] Phase 1-5
|
||||
│ (parallel OK) ──┼──→ [Worker B] Phase 1-5
|
||||
└─ STOP (idle) ─────┘ │
|
||||
│
|
||||
callback ←─────────────────────────────────────────┘
|
||||
(next beat) SendMessage + TaskUpdate(completed)
|
||||
═══════════════════════════════════════════════════════════
|
||||
```
|
||||
|
||||
**Pipeline beat views**:
|
||||
|
||||
```
|
||||
Targeted (3 beats, strictly serial)
|
||||
──────────────────────────────────────────────────────────
|
||||
Beat 1 2 3
|
||||
│ │ │
|
||||
STRATEGY → TESTGEN ──→ TESTRUN
|
||||
▲ ▲
|
||||
pipeline pipeline
|
||||
start done
|
||||
|
||||
STRATEGY=strategist TESTGEN=generator TESTRUN=executor
|
||||
|
||||
Standard (6 beats, progressive layers)
|
||||
──────────────────────────────────────────────────────────
|
||||
Beat 1 2 3 4 5 6
|
||||
│ │ │ │ │ │
|
||||
STRATEGY → TESTGEN-L1 → TESTRUN-L1 → TESTGEN-L2 → TESTRUN-L2 → TESTANA
|
||||
│
|
||||
coverage check
|
||||
(< target → GC loop)
|
||||
|
||||
Comprehensive (5+ beats, parallel windows)
|
||||
──────────────────────────────────────────────────────────
|
||||
Beat 1 2 3 4 5
|
||||
│ ┌─────┴─────┐ ┌──────┴──────┐ │ │
|
||||
STRATEGY → TESTGEN-L1 ∥ TESTGEN-L2 → TESTRUN-L1 ∥ TESTRUN-L2 → TESTGEN-L3 → TESTRUN-L3 → TESTANA
|
||||
▲ ▲ ▲
|
||||
parallel parallel pipeline
|
||||
window window done
|
||||
```
|
||||
|
||||
**Checkpoints**:
|
||||
|
||||
| Trigger | Location | Behavior |
|
||||
|---------|----------|----------|
|
||||
| Coverage below target | After TESTRUN-* | If coverage < target → create TESTGEN-fix task (GC loop); else proceed |
|
||||
| GC loop limit | Max 3 rounds per layer | Exceeds limit → accept current coverage with warning |
|
||||
| Pipeline stall | No ready + no running | Check missing tasks, report to user |
|
||||
|
||||
**Stall Detection** (coordinator `handleCheck` executes):
|
||||
|
||||
| Check | Condition | Resolution |
|
||||
|-------|-----------|------------|
|
||||
| Worker no response | in_progress task no callback | Report waiting task list, suggest user `resume` |
|
||||
| Pipeline deadlock | no ready + no running + has pending | Check blockedBy dependency chain, report blocking point |
|
||||
| GC loop exceeded | generator/executor iteration > 3 rounds | Terminate loop, accept current coverage with warning |
|
||||
|
||||
### Task Metadata Registry
|
||||
|
||||
| Task ID | Role | Phase | Dependencies | Description |
|
||||
|---------|------|-------|-------------|-------------|
|
||||
| STRATEGY-001 | strategist | strategy | (none) | Analyze git diff, determine test layers, define coverage targets |
|
||||
| TESTGEN-001 | generator | generate | STRATEGY-001 | Generate L1 unit tests |
|
||||
| TESTRUN-001 | executor | execute | TESTGEN-001 | Execute L1 tests, collect coverage |
|
||||
| TESTGEN-002 | generator | generate | TESTRUN-001 | Generate L2 integration tests (Standard/Comprehensive) |
|
||||
| TESTRUN-002 | executor | execute | TESTGEN-002 | Execute L2 tests, collect coverage |
|
||||
| TESTGEN-003 | generator | generate | TESTRUN-002 | Generate L3 E2E tests (Comprehensive only) |
|
||||
| TESTRUN-003 | executor | execute | TESTGEN-003 | Execute L3 tests, collect coverage |
|
||||
| TESTANA-001 | analyst | analyze | last TESTRUN-* | Defect pattern analysis, coverage gaps, quality report |
|
||||
|
||||
---
|
||||
|
||||
## Coordinator Spawn Template
|
||||
|
||||
### v5 Worker Spawn (all roles)
|
||||
|
||||
When coordinator spawns workers, use `team-worker` agent with role-spec path:
|
||||
Coordinator spawns workers using this template:
|
||||
|
||||
```
|
||||
Agent({
|
||||
@@ -313,7 +67,7 @@ Agent({
|
||||
run_in_background: true,
|
||||
prompt: `## Role Assignment
|
||||
role: <role>
|
||||
role_spec: .claude/skills/team-testing/role-specs/<role>.md
|
||||
role_spec: .claude/skills/team-testing/roles/<role>/role.md
|
||||
session: <session-folder>
|
||||
session_id: <session-id>
|
||||
team_name: testing
|
||||
@@ -321,19 +75,22 @@ requirement: <task-description>
|
||||
inner_loop: <true|false>
|
||||
|
||||
Read role_spec file to load Phase 2-4 domain instructions.
|
||||
Execute built-in Phase 1 (task discovery) -> role-spec Phase 2-4 -> built-in Phase 5 (report).`
|
||||
Execute built-in Phase 1 (task discovery) -> role Phase 2-4 -> built-in Phase 5 (report).`
|
||||
})
|
||||
```
|
||||
|
||||
**Inner Loop roles** (generator, executor): Set `inner_loop: true`. The team-worker agent handles the loop internally.
|
||||
## User Commands
|
||||
|
||||
**Single-task roles** (strategist, analyst): Set `inner_loop: false`.
|
||||
|
||||
---
|
||||
| Command | Action |
|
||||
|---------|--------|
|
||||
| `check` / `status` | View pipeline status graph |
|
||||
| `resume` / `continue` | Advance to next step |
|
||||
| `revise <TASK-ID>` | Revise specific task |
|
||||
| `feedback <text>` | Inject feedback for revision |
|
||||
|
||||
## Completion Action
|
||||
|
||||
When the pipeline completes (all tasks done, coordinator Phase 5):
|
||||
When pipeline completes, coordinator presents:
|
||||
|
||||
```
|
||||
AskUserQuestion({
|
||||
@@ -342,53 +99,39 @@ AskUserQuestion({
|
||||
header: "Completion",
|
||||
multiSelect: false,
|
||||
options: [
|
||||
{ label: "Archive & Clean (Recommended)", description: "Archive session, clean up tasks and team resources" },
|
||||
{ label: "Keep Active", description: "Keep session active for follow-up work or inspection" },
|
||||
{ label: "Export Results", description: "Export deliverables to a specified location, then clean" }
|
||||
{ label: "Archive & Clean (Recommended)", description: "Archive session, clean up team" },
|
||||
{ label: "Keep Active", description: "Keep session for follow-up work" },
|
||||
{ label: "Deepen Coverage", description: "Add more test layers or increase coverage targets" }
|
||||
]
|
||||
}]
|
||||
})
|
||||
```
|
||||
|
||||
| Choice | Action |
|
||||
|--------|--------|
|
||||
| Archive & Clean | Update session status="completed" -> TeamDelete() -> output final summary |
|
||||
| Keep Active | Update session status="paused" -> output resume instructions: `Skill(skill="team-testing", args="resume")` |
|
||||
| Export Results | AskUserQuestion for target path -> copy deliverables -> Archive & Clean |
|
||||
|
||||
---
|
||||
|
||||
## Unified Session Directory
|
||||
## Session Directory
|
||||
|
||||
```
|
||||
.workflow/.team/TST-<slug>-<YYYY-MM-DD>/
|
||||
├── .msg/messages.jsonl # Message bus log
|
||||
├── .msg/meta.json # Session metadata
|
||||
├── wisdom/ # Cross-task knowledge
|
||||
│ ├── learnings.md
|
||||
│ ├── decisions.md
|
||||
│ ├── conventions.md
|
||||
│ └── issues.md
|
||||
├── strategy/ # Strategist output
|
||||
│ └── test-strategy.md
|
||||
├── tests/ # Generator output
|
||||
│ ├── L1-unit/
|
||||
│ ├── L2-integration/
|
||||
│ └── L3-e2e/
|
||||
├── results/ # Executor output
|
||||
│ ├── run-001.json
|
||||
│ └── coverage-001.json
|
||||
└── analysis/ # Analyst output
|
||||
└── quality-report.md
|
||||
.workflow/.team/TST-<slug>-<date>/
|
||||
├── .msg/messages.jsonl # Team message bus
|
||||
├── .msg/meta.json # Session metadata
|
||||
├── wisdom/ # Cross-task knowledge
|
||||
├── strategy/ # Strategist output
|
||||
├── tests/ # Generator output (L1-unit/, L2-integration/, L3-e2e/)
|
||||
├── results/ # Executor output
|
||||
└── analysis/ # Analyst output
|
||||
```
|
||||
|
||||
## Specs Reference
|
||||
|
||||
- [specs/pipelines.md](specs/pipelines.md) — Pipeline definitions and task registry
|
||||
- [specs/team-config.json](specs/team-config.json) — Team configuration
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Scenario | Resolution |
|
||||
|----------|------------|
|
||||
| Unknown --role value | Error with available role list |
|
||||
| Missing --role arg | Orchestration Mode → auto route to coordinator |
|
||||
| Role file not found | Error with expected path (roles/<name>.md) |
|
||||
| Task prefix conflict | Log warning, proceed |
|
||||
| Coverage never reaches target | After 3 GC loops, accept current coverage with warning |
|
||||
| Test environment broken | Notify user, suggest manual fix |
|
||||
| Role not found | Error with expected path (roles/<name>/role.md) |
|
||||
| CLI tool fails | Worker fallback to direct implementation |
|
||||
| GC loop exceeded | Accept current coverage with warning |
|
||||
| Fast-advance conflict | Coordinator reconciles on next callback |
|
||||
| Completion action fails | Default to Keep Active |
|
||||
|
||||
95
.claude/skills/team-testing/roles/analyst/role.md
Normal file
95
.claude/skills/team-testing/roles/analyst/role.md
Normal file
@@ -0,0 +1,95 @@
|
||||
---
|
||||
role: analyst
|
||||
prefix: TESTANA
|
||||
inner_loop: false
|
||||
message_types:
|
||||
success: analysis_ready
|
||||
error: error
|
||||
---
|
||||
|
||||
# Test Quality Analyst
|
||||
|
||||
Analyze defect patterns, identify coverage gaps, assess GC loop effectiveness, and generate a quality report with actionable recommendations.
|
||||
|
||||
## Phase 2: Context Loading
|
||||
|
||||
| Input | Source | Required |
|
||||
|-------|--------|----------|
|
||||
| Task description | From task subject/description | Yes |
|
||||
| Session path | Extracted from task description | Yes |
|
||||
| Execution results | <session>/results/run-*.json | Yes |
|
||||
| Test strategy | <session>/strategy/test-strategy.md | Yes |
|
||||
| .msg/meta.json | <session>/wisdom/.msg/meta.json | Yes |
|
||||
|
||||
1. Extract session path from task description
|
||||
2. Read .msg/meta.json for execution context (executor, generator namespaces)
|
||||
3. Read all execution results:
|
||||
|
||||
```
|
||||
Glob("<session>/results/run-*.json")
|
||||
Read("<session>/results/run-001.json")
|
||||
```
|
||||
|
||||
4. Read test strategy:
|
||||
|
||||
```
|
||||
Read("<session>/strategy/test-strategy.md")
|
||||
```
|
||||
|
||||
5. Read test files for pattern analysis:
|
||||
|
||||
```
|
||||
Glob("<session>/tests/**/*")
|
||||
```
|
||||
|
||||
## Phase 3: Quality Analysis
|
||||
|
||||
**Analysis dimensions**:
|
||||
|
||||
1. **Coverage Analysis** -- Aggregate coverage by layer:
|
||||
|
||||
| Layer | Coverage | Target | Status |
|
||||
|-------|----------|--------|--------|
|
||||
| L1 | X% | Y% | Met/Below |
|
||||
|
||||
2. **Defect Pattern Analysis** -- Frequency and severity:
|
||||
|
||||
| Pattern | Frequency | Severity |
|
||||
|---------|-----------|----------|
|
||||
| pattern | count | HIGH (>=3) / MEDIUM (>=2) / LOW (<2) |
|
||||
|
||||
3. **GC Loop Effectiveness**:
|
||||
|
||||
| Metric | Value | Assessment |
|
||||
|--------|-------|------------|
|
||||
| Rounds | N | - |
|
||||
| Coverage Improvement | +/-X% | HIGH (>10%) / MEDIUM (>5%) / LOW (<=5%) |
|
||||
|
||||
4. **Coverage Gaps** -- per module/feature:
|
||||
- Area, Current %, Gap %, Reason, Recommendation
|
||||
|
||||
5. **Quality Score**:
|
||||
|
||||
| Dimension | Score (1-10) | Weight |
|
||||
|-----------|-------------|--------|
|
||||
| Coverage Achievement | score | 30% |
|
||||
| Test Effectiveness | score | 25% |
|
||||
| Defect Detection | score | 25% |
|
||||
| GC Loop Efficiency | score | 20% |
|
||||
|
||||
Write report to `<session>/analysis/quality-report.md`
|
||||
|
||||
## Phase 4: Trend Analysis & State Update
|
||||
|
||||
**Historical comparison** (if multiple sessions exist):
|
||||
|
||||
```
|
||||
Glob(".workflow/.team/TST-*/.msg/meta.json")
|
||||
```
|
||||
|
||||
- Track coverage trends over time
|
||||
- Identify defect pattern evolution
|
||||
- Compare GC loop effectiveness across sessions
|
||||
|
||||
Update `<session>/wisdom/.msg/meta.json` under `analyst` namespace:
|
||||
- Merge `{ "analyst": { quality_score, coverage_gaps, top_defect_patterns, gc_effectiveness, recommendations } }`
|
||||
@@ -0,0 +1,70 @@
|
||||
# Analyze Task
|
||||
|
||||
Parse user task -> detect testing capabilities -> select pipeline -> design roles.
|
||||
|
||||
**CONSTRAINT**: Text-level analysis only. NO source code reading, NO codebase exploration.
|
||||
|
||||
## Signal Detection
|
||||
|
||||
| Keywords | Capability | Prefix |
|
||||
|----------|------------|--------|
|
||||
| strategy, plan, layers, scope | strategist | STRATEGY |
|
||||
| generate tests, write tests, create tests | generator | TESTGEN |
|
||||
| run tests, execute, coverage | executor | TESTRUN |
|
||||
| analyze, report, quality, defects | analyst | TESTANA |
|
||||
|
||||
## Pipeline Mode Detection
|
||||
|
||||
| Condition | Pipeline |
|
||||
|-----------|----------|
|
||||
| fileCount <= 3 AND moduleCount <= 1 | targeted |
|
||||
| fileCount <= 10 AND moduleCount <= 3 | standard |
|
||||
| Otherwise | comprehensive |
|
||||
|
||||
## Dependency Graph
|
||||
|
||||
Natural ordering for testing pipeline:
|
||||
- Tier 0: strategist (change analysis, no upstream dependency)
|
||||
- Tier 1: generator (requires strategy)
|
||||
- Tier 2: executor (requires generated tests; GC loop with generator)
|
||||
- Tier 3: analyst (requires execution results)
|
||||
|
||||
## Pipeline Definitions
|
||||
|
||||
```
|
||||
Targeted: STRATEGY -> TESTGEN(L1) -> TESTRUN(L1)
|
||||
Standard: STRATEGY -> TESTGEN(L1) -> TESTRUN(L1) -> TESTGEN(L2) -> TESTRUN(L2) -> TESTANA
|
||||
Comprehensive: STRATEGY -> [TESTGEN(L1) || TESTGEN(L2)] -> [TESTRUN(L1) || TESTRUN(L2)] -> TESTGEN(L3) -> TESTRUN(L3) -> TESTANA
|
||||
```
|
||||
|
||||
## Complexity Scoring
|
||||
|
||||
| Factor | Points |
|
||||
|--------|--------|
|
||||
| Per test layer | +1 |
|
||||
| Parallel tracks | +1 per track |
|
||||
| GC loop enabled | +1 |
|
||||
| Serial depth > 3 | +1 |
|
||||
|
||||
Results: 1-2 Low, 3-5 Medium, 6+ High
|
||||
|
||||
## Role Minimization
|
||||
|
||||
- Cap at 5 roles (coordinator + 4 workers)
|
||||
- GC loop: generator <-> executor iterate up to 3 rounds per layer
|
||||
|
||||
## Output
|
||||
|
||||
Write <session>/task-analysis.json:
|
||||
```json
|
||||
{
|
||||
"task_description": "<original>",
|
||||
"pipeline_mode": "<targeted|standard|comprehensive>",
|
||||
"capabilities": [{ "name": "<cap>", "prefix": "<PREFIX>", "keywords": ["..."] }],
|
||||
"dependency_graph": { "<TASK-ID>": { "role": "<role>", "blockedBy": ["..."], "layer": "L1|L2|L3" } },
|
||||
"roles": [{ "name": "<role>", "prefix": "<PREFIX>", "inner_loop": true }],
|
||||
"complexity": { "score": 0, "level": "Low|Medium|High" },
|
||||
"coverage_targets": { "L1": 80, "L2": 60, "L3": 40 },
|
||||
"gc_loop_enabled": true
|
||||
}
|
||||
```
|
||||
@@ -1,33 +1,26 @@
|
||||
# Command: Dispatch
|
||||
# Dispatch Tasks
|
||||
|
||||
Create the testing task chain with correct dependencies and structured task descriptions. Supports targeted, standard, and comprehensive pipelines.
|
||||
Create testing task chains with correct dependencies. Supports targeted, standard, and comprehensive pipelines.
|
||||
|
||||
## Phase 2: Context Loading
|
||||
## Workflow
|
||||
|
||||
| Input | Source | Required |
|
||||
|-------|--------|----------|
|
||||
| User requirement | From coordinator Phase 1 | Yes |
|
||||
| Session folder | From coordinator Phase 2 | Yes |
|
||||
| Pipeline mode | From session.json `pipeline` | Yes |
|
||||
| Coverage targets | From session.json `coverage_targets` | Yes |
|
||||
1. Read task-analysis.json -> extract pipeline_mode and dependency_graph
|
||||
2. Read specs/pipelines.md -> get task registry for selected pipeline
|
||||
3. Topological sort tasks (respect blockedBy)
|
||||
4. Validate all owners exist in role registry (SKILL.md)
|
||||
5. For each task (in order):
|
||||
- TaskCreate with structured description (see template below)
|
||||
- TaskUpdate with blockedBy + owner assignment
|
||||
6. Update session.json with pipeline.tasks_total
|
||||
7. Validate chain (no orphans, no cycles, all refs valid)
|
||||
|
||||
1. Load user requirement and scope from session.json
|
||||
2. Load pipeline definition from SKILL.md Pipeline Definitions
|
||||
3. Read `pipeline` mode and `coverage_targets` from session.json
|
||||
|
||||
## Phase 3: Task Chain Creation (Mode-Branched)
|
||||
|
||||
### Task Description Template
|
||||
|
||||
Every task description uses structured format:
|
||||
## Task Description Template
|
||||
|
||||
```
|
||||
TaskCreate({
|
||||
subject: "<TASK-ID>",
|
||||
description: "PURPOSE: <what this task achieves> | Success: <measurable criteria>
|
||||
PURPOSE: <goal> | Success: <criteria>
|
||||
TASK:
|
||||
- <step 1: specific action>
|
||||
- <step 2: specific action>
|
||||
- <step 1>
|
||||
- <step 2>
|
||||
CONTEXT:
|
||||
- Session: <session-folder>
|
||||
- Scope: <scope>
|
||||
@@ -37,108 +30,72 @@ CONTEXT:
|
||||
EXPECTED: <deliverable path> + <quality criteria>
|
||||
CONSTRAINTS: <scope limits, focus areas>
|
||||
---
|
||||
InnerLoop: <true|false>"
|
||||
})
|
||||
TaskUpdate({ taskId: "<TASK-ID>", addBlockedBy: [<dependency-list>], owner: "<role>" })
|
||||
InnerLoop: <true|false>
|
||||
RoleSpec: .claude/skills/team-testing/roles/<role>/role.md
|
||||
```
|
||||
|
||||
### Mode Router
|
||||
|
||||
| Mode | Action |
|
||||
|------|--------|
|
||||
| `targeted` | Create 3 tasks: STRATEGY -> TESTGEN(L1) -> TESTRUN(L1) |
|
||||
| `standard` | Create 6 tasks: STRATEGY -> TESTGEN(L1) -> TESTRUN(L1) -> TESTGEN(L2) -> TESTRUN(L2) -> TESTANA |
|
||||
| `comprehensive` | Create 8 tasks with parallel groups |
|
||||
|
||||
---
|
||||
## Pipeline Task Registry
|
||||
|
||||
### Targeted Pipeline
|
||||
|
||||
**STRATEGY-001** (strategist):
|
||||
```
|
||||
TaskCreate({
|
||||
subject: "STRATEGY-001",
|
||||
description: "PURPOSE: Analyze change scope, define test strategy | Success: Strategy doc with layer recommendations
|
||||
TASK:
|
||||
- Analyze git diff for changed files and modules
|
||||
- Detect test framework and existing patterns
|
||||
- Define L1 unit test scope and coverage targets
|
||||
CONTEXT:
|
||||
- Session: <session-folder>
|
||||
- Scope: <scope>
|
||||
EXPECTED: <session>/strategy/test-strategy.md
|
||||
CONSTRAINTS: Read-only analysis
|
||||
---
|
||||
InnerLoop: false"
|
||||
})
|
||||
TaskUpdate({ taskId: "STRATEGY-001", owner: "strategist" })
|
||||
```
|
||||
|
||||
**TESTGEN-001** (generator, L1):
|
||||
```
|
||||
TaskCreate({
|
||||
subject: "TESTGEN-001",
|
||||
description: "PURPOSE: Generate L1 unit tests | Success: Executable test files covering priority files
|
||||
TASK:
|
||||
- Read test strategy for priority files and patterns
|
||||
- Generate unit tests: happy path, edge cases, error handling
|
||||
- Validate syntax
|
||||
CONTEXT:
|
||||
- Session: <session-folder>
|
||||
- Layer: L1-unit
|
||||
- Upstream artifacts: strategy/test-strategy.md
|
||||
EXPECTED: <session>/tests/L1-unit/
|
||||
CONSTRAINTS: Only generate test code, do not modify source
|
||||
---
|
||||
InnerLoop: true"
|
||||
})
|
||||
TaskUpdate({ taskId: "TESTGEN-001", addBlockedBy: ["STRATEGY-001"], owner: "generator" })
|
||||
```
|
||||
|
||||
**TESTRUN-001** (executor, L1):
|
||||
```
|
||||
TaskCreate({
|
||||
subject: "TESTRUN-001",
|
||||
description: "PURPOSE: Execute L1 unit tests, collect coverage | Success: pass_rate >= 0.95 AND coverage >= target
|
||||
TASK:
|
||||
- Run tests with coverage collection
|
||||
- Parse pass rate and coverage metrics
|
||||
- Auto-fix failures (max 3 iterations)
|
||||
CONTEXT:
|
||||
- Session: <session-folder>
|
||||
- Input: tests/L1-unit
|
||||
- Coverage target: <L1-target>%
|
||||
EXPECTED: <session>/results/run-001.json
|
||||
CONSTRAINTS: Only fix test files, not source code
|
||||
---
|
||||
InnerLoop: true"
|
||||
})
|
||||
TaskUpdate({ taskId: "TESTRUN-001", addBlockedBy: ["TESTGEN-001"], owner: "executor" })
|
||||
STRATEGY-001 (strategist): Analyze change scope, define test strategy
|
||||
blockedBy: []
|
||||
TESTGEN-001 (generator): Generate L1 unit tests
|
||||
blockedBy: [STRATEGY-001], meta: layer=L1-unit
|
||||
TESTRUN-001 (executor): Execute L1 tests, collect coverage
|
||||
blockedBy: [TESTGEN-001], inner_loop: true, meta: layer=L1-unit, coverage_target=80%
|
||||
```
|
||||
|
||||
### Standard Pipeline
|
||||
|
||||
Adds to targeted:
|
||||
|
||||
**TESTGEN-002** (generator, L2): blockedBy ["TESTRUN-001"], Layer: L2-integration
|
||||
**TESTRUN-002** (executor, L2): blockedBy ["TESTGEN-002"], Input: tests/L2-integration
|
||||
**TESTANA-001** (analyst): blockedBy ["TESTRUN-002"]
|
||||
```
|
||||
STRATEGY-001 (strategist): Analyze change scope, define test strategy
|
||||
blockedBy: []
|
||||
TESTGEN-001 (generator): Generate L1 unit tests
|
||||
blockedBy: [STRATEGY-001], meta: layer=L1-unit
|
||||
TESTRUN-001 (executor): Execute L1 tests, collect coverage
|
||||
blockedBy: [TESTGEN-001], inner_loop: true, meta: layer=L1-unit, coverage_target=80%
|
||||
TESTGEN-002 (generator): Generate L2 integration tests
|
||||
blockedBy: [TESTRUN-001], meta: layer=L2-integration
|
||||
TESTRUN-002 (executor): Execute L2 tests, collect coverage
|
||||
blockedBy: [TESTGEN-002], inner_loop: true, meta: layer=L2-integration, coverage_target=60%
|
||||
TESTANA-001 (analyst): Defect pattern analysis, quality report
|
||||
blockedBy: [TESTRUN-002]
|
||||
```
|
||||
|
||||
### Comprehensive Pipeline
|
||||
```
|
||||
STRATEGY-001 (strategist): Analyze change scope, define test strategy
|
||||
blockedBy: []
|
||||
TESTGEN-001 (generator-1): Generate L1 unit tests
|
||||
blockedBy: [STRATEGY-001], meta: layer=L1-unit
|
||||
TESTGEN-002 (generator-2): Generate L2 integration tests
|
||||
blockedBy: [STRATEGY-001], meta: layer=L2-integration
|
||||
TESTRUN-001 (executor-1): Execute L1 tests, collect coverage
|
||||
blockedBy: [TESTGEN-001], inner_loop: true, meta: layer=L1-unit, coverage_target=80%
|
||||
TESTRUN-002 (executor-2): Execute L2 tests, collect coverage
|
||||
blockedBy: [TESTGEN-002], inner_loop: true, meta: layer=L2-integration, coverage_target=60%
|
||||
TESTGEN-003 (generator): Generate L3 E2E tests
|
||||
blockedBy: [TESTRUN-001, TESTRUN-002], meta: layer=L3-e2e
|
||||
TESTRUN-003 (executor): Execute L3 tests, collect coverage
|
||||
blockedBy: [TESTGEN-003], inner_loop: true, meta: layer=L3-e2e, coverage_target=40%
|
||||
TESTANA-001 (analyst): Defect pattern analysis, quality report
|
||||
blockedBy: [TESTRUN-003]
|
||||
```
|
||||
|
||||
**Parallel groups**:
|
||||
- TESTGEN-001 + TESTGEN-002 both blockedBy ["STRATEGY-001"] (parallel generation)
|
||||
- TESTRUN-001 blockedBy ["TESTGEN-001"], TESTRUN-002 blockedBy ["TESTGEN-002"] (parallel execution)
|
||||
- TESTGEN-003 blockedBy ["TESTRUN-001", "TESTRUN-002"], Layer: L3-e2e
|
||||
- TESTRUN-003 blockedBy ["TESTGEN-003"]
|
||||
- TESTANA-001 blockedBy ["TESTRUN-003"]
|
||||
## InnerLoop Flag Rules
|
||||
|
||||
## Phase 4: Validation
|
||||
- true: generator, executor roles (GC loop iterations)
|
||||
- false: strategist, analyst roles
|
||||
|
||||
1. Verify all tasks created with correct subjects and dependencies
|
||||
2. Check no circular dependencies
|
||||
3. Verify blockedBy references exist
|
||||
4. Log task chain to team_msg:
|
||||
## Dependency Validation
|
||||
|
||||
- No orphan tasks (all tasks have valid owner)
|
||||
- No circular dependencies
|
||||
- All blockedBy references exist
|
||||
- Session reference in every task description
|
||||
- RoleSpec reference in every task description
|
||||
|
||||
## Log After Creation
|
||||
|
||||
```
|
||||
mcp__ccw-tools__team_msg({
|
||||
|
||||
@@ -1,27 +1,40 @@
|
||||
# Command: Monitor
|
||||
# Monitor Pipeline
|
||||
|
||||
Handle all coordinator monitoring events: worker callbacks, status checks, pipeline advancement, Generator-Critic loop control, and completion.
|
||||
Event-driven pipeline coordination. Beat model: coordinator wake -> process -> spawn -> STOP.
|
||||
|
||||
## Phase 2: Context Loading
|
||||
## Constants
|
||||
|
||||
| Input | Source | Required |
|
||||
|-------|--------|----------|
|
||||
| Session state | <session>/session.json | Yes |
|
||||
| Task list | TaskList() | Yes |
|
||||
| Trigger event | From Entry Router detection | Yes |
|
||||
| Pipeline definition | From SKILL.md | Yes |
|
||||
- SPAWN_MODE: background
|
||||
- ONE_STEP_PER_INVOCATION: true
|
||||
- FAST_ADVANCE_AWARE: true
|
||||
- WORKER_AGENT: team-worker
|
||||
- MAX_GC_ROUNDS: 3
|
||||
|
||||
1. Load session.json for current state, pipeline mode, gc_rounds, coverage_targets
|
||||
2. Run TaskList() to get current task statuses
|
||||
3. Identify trigger event type from Entry Router
|
||||
## Handler Router
|
||||
|
||||
## Phase 3: Event Handlers
|
||||
| Source | Handler |
|
||||
|--------|---------|
|
||||
| Message contains [strategist], [generator], [executor], [analyst] | handleCallback |
|
||||
| "capability_gap" | handleAdapt |
|
||||
| "check" or "status" | handleCheck |
|
||||
| "resume" or "continue" | handleResume |
|
||||
| All tasks completed | handleComplete |
|
||||
| Default | handleSpawnNext |
|
||||
|
||||
### handleCallback
|
||||
## Role-Worker Map
|
||||
|
||||
Triggered when a worker sends completion message.
|
||||
| Prefix | Role | Role Spec | inner_loop |
|
||||
|--------|------|-----------|------------|
|
||||
| STRATEGY-* | strategist | `.claude/skills/team-testing/roles/strategist/role.md` | false |
|
||||
| TESTGEN-* | generator | `.claude/skills/team-testing/roles/generator/role.md` | true |
|
||||
| TESTRUN-* | executor | `.claude/skills/team-testing/roles/executor/role.md` | true |
|
||||
| TESTANA-* | analyst | `.claude/skills/team-testing/roles/analyst/role.md` | false |
|
||||
|
||||
1. Parse message to identify role, task ID:
|
||||
## handleCallback
|
||||
|
||||
Worker completed. Process and advance.
|
||||
|
||||
1. Parse message to identify role and task ID:
|
||||
|
||||
| Message Pattern | Role Detection |
|
||||
|----------------|---------------|
|
||||
@@ -30,45 +43,19 @@ Triggered when a worker sends completion message.
|
||||
| `[executor]` or task ID `TESTRUN-*` | executor |
|
||||
| `[analyst]` or task ID `TESTANA-*` | analyst |
|
||||
|
||||
2. Mark task as completed:
|
||||
|
||||
```
|
||||
TaskUpdate({ taskId: "<task-id>", status: "completed" })
|
||||
```
|
||||
|
||||
3. Record completion in session state
|
||||
|
||||
4. **Generator-Critic check** (executor callbacks only):
|
||||
- If completed task is TESTRUN-* AND message indicates tests_failed or coverage below target:
|
||||
- Read gc_rounds for this layer from session.json
|
||||
- Execute **GC Loop Decision** (see below)
|
||||
|
||||
5. Checkpoints:
|
||||
|
||||
| Completed Task | Checkpoint | Action |
|
||||
|---------------|------------|--------|
|
||||
| STRATEGY-001 | CP-1 | Notify user: test strategy ready for review |
|
||||
| Last TESTRUN-* | CP-2 | Check coverage, decide GC loop or next layer |
|
||||
| TESTANA-001 | CP-3 | Pipeline complete |
|
||||
|
||||
6. Proceed to handleSpawnNext
|
||||
|
||||
### GC Loop Decision
|
||||
|
||||
When executor reports test results:
|
||||
|
||||
| Condition | Action |
|
||||
|-----------|--------|
|
||||
| passRate >= 0.95 AND coverage >= target | Log success, proceed to next stage |
|
||||
| (passRate < 0.95 OR coverage < target) AND gcRound < maxRounds | Create TESTGEN-fix task, increment gc_round |
|
||||
| gcRound >= maxRounds | Accept current coverage with warning, proceed |
|
||||
|
||||
**TESTGEN-fix task creation**:
|
||||
2. Check if progress update (inner loop) or final completion
|
||||
3. Progress -> update session state, STOP
|
||||
4. Completion -> mark task done via TaskUpdate(status="completed"), remove from active_workers
|
||||
5. Check for checkpoints:
|
||||
- TESTRUN-* completes -> read meta.json for executor.pass_rate and executor.coverage:
|
||||
- (pass_rate >= 0.95 AND coverage >= target) OR gc_rounds[layer] >= MAX_GC_ROUNDS -> proceed to handleSpawnNext
|
||||
- (pass_rate < 0.95 OR coverage < target) AND gc_rounds[layer] < MAX_GC_ROUNDS -> create GC fix tasks, increment gc_rounds[layer]
|
||||
|
||||
**GC Fix Task Creation** (when coverage below target):
|
||||
```
|
||||
TaskCreate({
|
||||
subject: "TESTGEN-<layer>-fix-<round>",
|
||||
description: "PURPOSE: Revise tests to fix failures and improve coverage
|
||||
subject: "TESTGEN-<layer>-fix-<round>: Revise <layer> tests (GC #<round>)",
|
||||
description: "PURPOSE: Revise tests to fix failures and improve coverage | Success: pass_rate >= 0.95 AND coverage >= target
|
||||
TASK:
|
||||
- Read previous test results and failure details
|
||||
- Revise tests to address failures
|
||||
@@ -78,109 +65,161 @@ CONTEXT:
|
||||
- Layer: <layer>
|
||||
- Previous results: <session>/results/run-<N>.json
|
||||
EXPECTED: Revised test files in <session>/tests/<layer>/
|
||||
CONSTRAINTS: Only modify test files
|
||||
---
|
||||
InnerLoop: true"
|
||||
InnerLoop: true
|
||||
RoleSpec: .claude/skills/team-testing/roles/generator/role.md"
|
||||
})
|
||||
TaskUpdate({ taskId: "TESTGEN-<layer>-fix-<round>", owner: "generator" })
|
||||
TaskCreate({
|
||||
subject: "TESTRUN-<layer>-fix-<round>: Re-execute <layer> (GC #<round>)",
|
||||
description: "PURPOSE: Re-execute tests after revision | Success: pass_rate >= 0.95
|
||||
CONTEXT:
|
||||
- Session: <session-folder>
|
||||
- Layer: <layer>
|
||||
- Input: tests/<layer>
|
||||
EXPECTED: <session>/results/run-<N>-gc.json
|
||||
---
|
||||
InnerLoop: true
|
||||
RoleSpec: .claude/skills/team-testing/roles/executor/role.md",
|
||||
blockedBy: ["TESTGEN-<layer>-fix-<round>"]
|
||||
})
|
||||
```
|
||||
Update session.gc_rounds[layer]++
|
||||
|
||||
6. -> handleSpawnNext
|
||||
|
||||
## handleCheck
|
||||
|
||||
Read-only status report, then STOP.
|
||||
|
||||
Output:
|
||||
```
|
||||
[coordinator] Testing Pipeline Status
|
||||
[coordinator] Mode: <pipeline_mode>
|
||||
[coordinator] Progress: <done>/<total> (<pct>%)
|
||||
[coordinator] GC Rounds: L1: <n>/3, L2: <n>/3
|
||||
|
||||
[coordinator] Pipeline Graph:
|
||||
STRATEGY-001: <done|run|wait> test-strategy.md
|
||||
TESTGEN-001: <done|run|wait> generating L1...
|
||||
TESTRUN-001: <done|run|wait> blocked by TESTGEN-001
|
||||
TESTGEN-002: <done|run|wait> blocked by TESTRUN-001
|
||||
TESTRUN-002: <done|run|wait> blocked by TESTGEN-002
|
||||
TESTANA-001: <done|run|wait> blocked by TESTRUN-*
|
||||
|
||||
[coordinator] Active Workers: <list with elapsed time>
|
||||
[coordinator] Ready: <pending tasks with resolved deps>
|
||||
[coordinator] Commands: 'resume' to advance | 'check' to refresh
|
||||
```
|
||||
|
||||
Create TESTRUN-fix blocked on TESTGEN-fix.
|
||||
Then STOP.
|
||||
|
||||
### handleSpawnNext
|
||||
## handleResume
|
||||
|
||||
Find and spawn the next ready tasks.
|
||||
1. No active workers -> handleSpawnNext
|
||||
2. Has active -> check each status
|
||||
- completed -> mark done via TaskUpdate
|
||||
- in_progress -> still running
|
||||
3. Some completed -> handleSpawnNext
|
||||
4. All running -> report status, STOP
|
||||
|
||||
1. Scan task list for tasks where:
|
||||
- Status is "pending"
|
||||
- All blockedBy tasks have status "completed"
|
||||
## handleSpawnNext
|
||||
|
||||
2. For each ready task, determine role from task subject prefix:
|
||||
Find ready tasks, spawn workers, STOP.
|
||||
|
||||
| Prefix | Role | Inner Loop |
|
||||
|--------|------|------------|
|
||||
| STRATEGY-* | strategist | false |
|
||||
| TESTGEN-* | generator | true |
|
||||
| TESTRUN-* | executor | true |
|
||||
| TESTANA-* | analyst | false |
|
||||
1. Collect from TaskList():
|
||||
- completedSubjects: status = completed
|
||||
- inProgressSubjects: status = in_progress
|
||||
- readySubjects: status = pending AND all blockedBy in completedSubjects
|
||||
|
||||
3. Spawn team-worker:
|
||||
2. No ready + work in progress -> report waiting, STOP
|
||||
3. No ready + nothing in progress -> handleComplete
|
||||
4. Has ready -> for each ready task:
|
||||
a. Determine role from prefix (use Role-Worker Map)
|
||||
b. Check if inner loop role (generator/executor) with active worker -> skip (worker picks up next task)
|
||||
c. TaskUpdate -> in_progress
|
||||
d. team_msg log -> task_unblocked
|
||||
e. Spawn team-worker:
|
||||
|
||||
```
|
||||
Agent({
|
||||
subagent_type: "team-worker",
|
||||
description: "Spawn <role> worker for <task-id>",
|
||||
description: "Spawn <role> worker for <subject>",
|
||||
team_name: "testing",
|
||||
name: "<role>",
|
||||
run_in_background: true,
|
||||
prompt: `## Role Assignment
|
||||
role: <role>
|
||||
role_spec: .claude/skills/team-testing/role-specs/<role>.md
|
||||
role_spec: .claude/skills/team-testing/roles/<role>/role.md
|
||||
session: <session-folder>
|
||||
session_id: <session-id>
|
||||
team_name: testing
|
||||
requirement: <task-description>
|
||||
inner_loop: <true|false>
|
||||
|
||||
## Current Task
|
||||
- Task ID: <task-id>
|
||||
- Task: <subject>
|
||||
|
||||
Read role_spec file to load Phase 2-4 domain instructions.
|
||||
Execute built-in Phase 1 -> role-spec Phase 2-4 -> built-in Phase 5.`
|
||||
Execute built-in Phase 1 (task discovery) -> role Phase 2-4 -> built-in Phase 5 (report).`
|
||||
})
|
||||
```
|
||||
|
||||
4. **Parallel spawn** (comprehensive pipeline):
|
||||
f. Add to active_workers
|
||||
|
||||
| Scenario | Spawn Behavior |
|
||||
|----------|---------------|
|
||||
| TESTGEN-001 + TESTGEN-002 both unblocked | Spawn both in parallel |
|
||||
| TESTRUN-001 + TESTRUN-002 both unblocked | Spawn both in parallel |
|
||||
5. **Parallel spawn** (comprehensive pipeline):
|
||||
- TESTGEN-001 + TESTGEN-002 both unblocked -> spawn both in parallel (name: "generator-1", "generator-2")
|
||||
- TESTRUN-001 + TESTRUN-002 both unblocked -> spawn both in parallel (name: "executor-1", "executor-2")
|
||||
|
||||
5. STOP after spawning -- wait for next callback
|
||||
6. Update session.json, output summary, STOP
|
||||
|
||||
### handleCheck
|
||||
## handleComplete
|
||||
|
||||
Output current pipeline status.
|
||||
Pipeline done. Generate report and completion action.
|
||||
|
||||
```
|
||||
Pipeline Status (<pipeline-mode>):
|
||||
[DONE] STRATEGY-001 (strategist) -> test-strategy.md
|
||||
[RUN] TESTGEN-001 (generator) -> generating L1...
|
||||
[WAIT] TESTRUN-001 (executor) -> blocked by TESTGEN-001
|
||||
[WAIT] TESTGEN-002 (generator) -> blocked by TESTRUN-001
|
||||
...
|
||||
1. Verify all tasks (including any GC fix tasks) have status "completed" or "deleted"
|
||||
2. If any tasks incomplete -> return to handleSpawnNext
|
||||
3. If all complete:
|
||||
- Read final state from meta.json (analyst.quality_score, executor.coverage, gc_rounds)
|
||||
- Generate summary (deliverables, task count, GC rounds, coverage metrics)
|
||||
4. Read session.completion_action:
|
||||
- interactive -> AskUserQuestion (Archive/Keep/Deepen Coverage)
|
||||
- auto_archive -> Archive & Clean (status=completed, TeamDelete)
|
||||
- auto_keep -> Keep Active (status=paused)
|
||||
|
||||
GC Rounds: L1: 0/3, L2: 0/3
|
||||
Session: <session-id>
|
||||
```
|
||||
## handleAdapt
|
||||
|
||||
Output status -- do NOT advance pipeline.
|
||||
Capability gap reported mid-pipeline.
|
||||
|
||||
### handleResume
|
||||
1. Parse gap description
|
||||
2. Check if existing role covers it -> redirect
|
||||
3. Role count < 5 -> generate dynamic role-spec in <session>/role-specs/
|
||||
4. Create new task, spawn worker
|
||||
5. Role count >= 5 -> merge or pause
|
||||
|
||||
Resume pipeline after user pause or interruption.
|
||||
## Fast-Advance Reconciliation
|
||||
|
||||
1. Audit task list for inconsistencies:
|
||||
- Tasks stuck in "in_progress" -> reset to "pending"
|
||||
- Tasks with completed blockers but still "pending" -> include in spawn list
|
||||
2. Proceed to handleSpawnNext
|
||||
|
||||
### handleComplete
|
||||
|
||||
Triggered when all pipeline tasks are completed and no GC cycles remain.
|
||||
|
||||
**Completion check**:
|
||||
|
||||
| Pipeline | Completion Condition |
|
||||
|----------|---------------------|
|
||||
| targeted | STRATEGY-001 + TESTGEN-001 + TESTRUN-001 (+ any fix tasks) completed |
|
||||
| standard | All 6 tasks (+ any fix tasks) completed |
|
||||
| comprehensive | All 8 tasks (+ any fix tasks) completed |
|
||||
|
||||
1. If any tasks not completed, return to handleSpawnNext
|
||||
2. If all completed, transition to coordinator Phase 5
|
||||
On every coordinator wake:
|
||||
1. Read team_msg entries with type="fast_advance"
|
||||
2. Sync active_workers with spawned successors
|
||||
3. No duplicate spawns
|
||||
|
||||
## Phase 4: State Persistence
|
||||
|
||||
After every handler execution:
|
||||
1. Reconcile active_workers with actual TaskList states
|
||||
2. Remove entries for completed/deleted tasks
|
||||
3. Write updated session.json
|
||||
4. STOP (wait for next callback)
|
||||
|
||||
1. Update session.json with current state (active tasks, gc_rounds per layer, last event)
|
||||
2. Verify task list consistency
|
||||
3. STOP and wait for next event
|
||||
## Error Handling
|
||||
|
||||
| Scenario | Resolution |
|
||||
|----------|------------|
|
||||
| Session file not found | Error, suggest re-initialization |
|
||||
| Worker callback from unknown role | Log info, scan for other completions |
|
||||
| GC loop exceeded (3 rounds) | Accept current coverage with warning, proceed |
|
||||
| Pipeline stall | Check blockedBy chains, report to user |
|
||||
| Coverage tool unavailable | Degrade to pass rate judgment |
|
||||
| Worker crash | Reset task to pending, respawn |
|
||||
|
||||
@@ -1,15 +1,14 @@
|
||||
# Coordinator - Testing Team
|
||||
# Coordinator Role
|
||||
|
||||
**Role**: coordinator
|
||||
**Type**: Orchestrator
|
||||
**Team**: testing
|
||||
Orchestrate team-testing: analyze -> dispatch -> spawn -> monitor -> report.
|
||||
|
||||
Orchestrates the testing pipeline: manages task chains, spawns team-worker agents, handles Generator-Critic loops, quality gates, and drives the pipeline to completion.
|
||||
## Identity
|
||||
- Name: coordinator | Tag: [coordinator]
|
||||
- Responsibility: Change scope analysis -> Create team -> Dispatch tasks -> Monitor progress -> Report results
|
||||
|
||||
## Boundaries
|
||||
|
||||
### MUST
|
||||
|
||||
- Use `team-worker` agent type for all worker spawns (NOT `general-purpose`)
|
||||
- Follow Command Execution Protocol for dispatch and monitor commands
|
||||
- Respect pipeline stage dependencies (blockedBy)
|
||||
@@ -18,108 +17,49 @@ Orchestrates the testing pipeline: manages task chains, spawns team-worker agent
|
||||
- Execute completion action in Phase 5
|
||||
|
||||
### MUST NOT
|
||||
|
||||
- Implement domain logic (test generation, execution, analysis) -- workers handle this
|
||||
- Spawn workers without creating tasks first
|
||||
- Skip quality gates when coverage is below target
|
||||
- Modify test files or source code directly -- delegate to workers
|
||||
- Force-advance pipeline past failed GC loops
|
||||
|
||||
---
|
||||
|
||||
## Command Execution Protocol
|
||||
|
||||
When coordinator needs to execute a command (dispatch, monitor):
|
||||
|
||||
1. **Read the command file**: `roles/coordinator/commands/<command-name>.md`
|
||||
2. **Follow the workflow** defined in the command file (Phase 2-4 structure)
|
||||
3. **Commands are inline execution guides** -- NOT separate agents or subprocesses
|
||||
4. **Execute synchronously** -- complete the command workflow before proceeding
|
||||
|
||||
Example:
|
||||
```
|
||||
Phase 3 needs task dispatch
|
||||
-> Read roles/coordinator/commands/dispatch.md
|
||||
-> Execute Phase 2 (Context Loading)
|
||||
-> Execute Phase 3 (Task Chain Creation)
|
||||
-> Execute Phase 4 (Validation)
|
||||
-> Continue to Phase 4
|
||||
```
|
||||
|
||||
---
|
||||
When coordinator needs to execute a specific phase:
|
||||
1. Read `commands/<command>.md`
|
||||
2. Follow the workflow defined in the command
|
||||
3. Commands are inline execution guides, NOT separate agents
|
||||
4. Execute synchronously, complete before proceeding
|
||||
|
||||
## Entry Router
|
||||
|
||||
When coordinator is invoked, detect invocation type:
|
||||
|
||||
| Detection | Condition | Handler |
|
||||
|-----------|-----------|---------|
|
||||
| Worker callback | Message contains role tag [strategist], [generator], [executor], [analyst] | -> handleCallback |
|
||||
| Status check | Arguments contain "check" or "status" | -> handleCheck |
|
||||
| Manual resume | Arguments contain "resume" or "continue" | -> handleResume |
|
||||
| Pipeline complete | All tasks have status "completed" | -> handleComplete |
|
||||
| Interrupted session | Active/paused session exists | -> Phase 0 (Resume Check) |
|
||||
| New session | None of above | -> Phase 1 (Change Scope Analysis) |
|
||||
| Worker callback | Message contains [strategist], [generator], [executor], [analyst] | -> handleCallback (monitor.md) |
|
||||
| Status check | Args contain "check" or "status" | -> handleCheck (monitor.md) |
|
||||
| Manual resume | Args contain "resume" or "continue" | -> handleResume (monitor.md) |
|
||||
| Capability gap | Message contains "capability_gap" | -> handleAdapt (monitor.md) |
|
||||
| Pipeline complete | All tasks completed | -> handleComplete (monitor.md) |
|
||||
| Interrupted session | Active session in .workflow/.team/TST-* | -> Phase 0 |
|
||||
| New session | None of above | -> Phase 1 |
|
||||
|
||||
For callback/check/resume/complete: load `commands/monitor.md` and execute matched handler, then STOP.
|
||||
|
||||
### Router Implementation
|
||||
|
||||
1. **Load session context** (if exists):
|
||||
- Scan `.workflow/.team/TST-*/.msg/meta.json` for active/paused sessions
|
||||
- If found, extract session folder path, status, and pipeline mode
|
||||
|
||||
2. **Parse $ARGUMENTS** for detection keywords:
|
||||
- Check for role name tags in message content
|
||||
- Check for "check", "status", "resume", "continue" keywords
|
||||
|
||||
3. **Route to handler**:
|
||||
- For monitor handlers: Read `commands/monitor.md`, execute matched handler, STOP
|
||||
- For Phase 0: Execute Session Resume Check below
|
||||
- For Phase 1: Execute Change Scope Analysis below
|
||||
|
||||
---
|
||||
For callback/check/resume/adapt/complete: load commands/monitor.md, execute handler, STOP.
|
||||
|
||||
## Phase 0: Session Resume Check
|
||||
|
||||
Triggered when an active/paused session is detected on coordinator entry.
|
||||
1. Scan .workflow/.team/TST-*/session.json for active/paused sessions
|
||||
2. No sessions -> Phase 1
|
||||
3. Single session -> reconcile (audit TaskList, reset in_progress->pending, rebuild team, kick first ready task)
|
||||
4. Multiple -> AskUserQuestion for selection
|
||||
|
||||
1. Load session.json from detected session folder
|
||||
2. Audit task list:
|
||||
## Phase 1: Requirement Clarification
|
||||
|
||||
```
|
||||
TaskList()
|
||||
```
|
||||
TEXT-LEVEL ONLY. No source code reading.
|
||||
|
||||
3. Reconcile session state vs task status:
|
||||
|
||||
| Task Status | Session Expects | Action |
|
||||
|-------------|----------------|--------|
|
||||
| in_progress | Should be running | Reset to pending (worker was interrupted) |
|
||||
| completed | Already tracked | Skip |
|
||||
| pending + unblocked | Ready to run | Include in spawn list |
|
||||
|
||||
4. Rebuild team if not active:
|
||||
|
||||
```
|
||||
TeamCreate({ team_name: "testing" })
|
||||
```
|
||||
|
||||
5. Spawn workers for ready tasks -> Phase 4 coordination loop
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Change Scope Analysis
|
||||
|
||||
1. Parse user task description from $ARGUMENTS
|
||||
1. Parse task description from $ARGUMENTS
|
||||
2. Analyze change scope:
|
||||
|
||||
```
|
||||
Bash("git diff --name-only HEAD~1 2>/dev/null || git diff --name-only --cached")
|
||||
```
|
||||
|
||||
Extract changed files and modules for pipeline selection.
|
||||
|
||||
```
|
||||
Bash("git diff --name-only HEAD~1 2>/dev/null || git diff --name-only --cached")
|
||||
```
|
||||
3. Select pipeline:
|
||||
|
||||
| Condition | Pipeline |
|
||||
@@ -128,161 +68,64 @@ Extract changed files and modules for pipeline selection.
|
||||
| fileCount <= 10 AND moduleCount <= 3 | standard |
|
||||
| Otherwise | comprehensive |
|
||||
|
||||
4. Ask for missing parameters via AskUserQuestion if scope is unclear
|
||||
4. Clarify if ambiguous (AskUserQuestion for scope)
|
||||
5. Delegate to commands/analyze.md
|
||||
6. Output: task-analysis.json
|
||||
7. CRITICAL: Always proceed to Phase 2, never skip team workflow
|
||||
|
||||
5. Record requirements: mode, scope, focus, constraints
|
||||
## Phase 2: Create Team + Initialize Session
|
||||
|
||||
---
|
||||
1. Generate session ID: TST-<slug>-<date>
|
||||
2. Create session folder structure (strategy/, tests/L1-unit/, tests/L2-integration/, tests/L3-e2e/, results/, analysis/, wisdom/)
|
||||
3. TeamCreate with team name "testing"
|
||||
4. Read specs/pipelines.md -> select pipeline based on mode
|
||||
5. Initialize pipeline via team_msg state_update:
|
||||
```
|
||||
mcp__ccw-tools__team_msg({
|
||||
operation: "log", session_id: "<id>", from: "coordinator",
|
||||
type: "state_update", summary: "Session initialized",
|
||||
data: {
|
||||
pipeline_mode: "<targeted|standard|comprehensive>",
|
||||
pipeline_stages: ["strategist", "generator", "executor", "analyst"],
|
||||
team_name: "testing",
|
||||
coverage_targets: { "L1": 80, "L2": 60, "L3": 40 },
|
||||
gc_rounds: {}
|
||||
}
|
||||
})
|
||||
```
|
||||
6. Write session.json
|
||||
|
||||
## Phase 2: Session & Team Setup
|
||||
## Phase 3: Create Task Chain
|
||||
|
||||
1. Create session directory:
|
||||
Delegate to commands/dispatch.md:
|
||||
1. Read specs/pipelines.md for selected pipeline's task registry
|
||||
2. Topological sort tasks
|
||||
3. Create tasks via TaskCreate with blockedBy
|
||||
4. Update session.json
|
||||
|
||||
```
|
||||
Bash("mkdir -p .workflow/.team/TST-<slug>-<date>/strategy .workflow/.team/TST-<slug>-<date>/tests/L1-unit .workflow/.team/TST-<slug>-<date>/tests/L2-integration .workflow/.team/TST-<slug>-<date>/tests/L3-e2e .workflow/.team/TST-<slug>-<date>/results .workflow/.team/TST-<slug>-<date>/analysis .workflow/.team/TST-<slug>-<date>/wisdom")
|
||||
```
|
||||
## Phase 4: Spawn-and-Stop
|
||||
|
||||
2. Write session.json:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "active",
|
||||
"team_name": "testing",
|
||||
"requirement": "<requirement>",
|
||||
"timestamp": "<ISO-8601>",
|
||||
"pipeline": "<targeted|standard|comprehensive>",
|
||||
"coverage_targets": { "L1": 80, "L2": 60, "L3": 40 },
|
||||
"gc_rounds": {},
|
||||
"max_gc_rounds": 3
|
||||
}
|
||||
```
|
||||
|
||||
3. Initialize .msg/meta.json with pipeline metadata:
|
||||
```typescript
|
||||
// Use team_msg to write pipeline metadata to .msg/meta.json
|
||||
mcp__ccw-tools__team_msg({
|
||||
operation: "log",
|
||||
session_id: "<session-id>",
|
||||
from: "coordinator",
|
||||
type: "state_update",
|
||||
summary: "Session initialized",
|
||||
data: {
|
||||
pipeline_mode: "<targeted|standard|comprehensive>",
|
||||
pipeline_stages: ["strategist", "generator", "executor", "analyst"],
|
||||
roles: ["coordinator", "strategist", "generator", "executor", "analyst"],
|
||||
team_name: "testing"
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
4. Create team:
|
||||
|
||||
```
|
||||
TeamCreate({ team_name: "testing" })
|
||||
```
|
||||
|
||||
5. Initialize wisdom directory (learnings.md, decisions.md, conventions.md, issues.md)
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Task Chain Creation
|
||||
|
||||
Execute `commands/dispatch.md` inline (Command Execution Protocol):
|
||||
|
||||
1. Read `roles/coordinator/commands/dispatch.md`
|
||||
2. Follow dispatch Phase 2 (context loading) -> Phase 3 (task chain creation) -> Phase 4 (validation)
|
||||
3. Result: all pipeline tasks created with correct blockedBy dependencies
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Spawn & Coordination Loop
|
||||
|
||||
### Initial Spawn
|
||||
|
||||
Find first unblocked task and spawn its worker:
|
||||
|
||||
```
|
||||
Agent({
|
||||
subagent_type: "team-worker",
|
||||
description: "Spawn strategist worker",
|
||||
team_name: "testing",
|
||||
name: "strategist",
|
||||
run_in_background: true,
|
||||
prompt: `## Role Assignment
|
||||
role: strategist
|
||||
role_spec: .claude/skills/team-testing/role-specs/strategist.md
|
||||
session: <session-folder>
|
||||
session_id: <session-id>
|
||||
team_name: testing
|
||||
requirement: <requirement>
|
||||
inner_loop: false
|
||||
|
||||
Read role_spec file to load Phase 2-4 domain instructions.
|
||||
Execute built-in Phase 1 -> role-spec Phase 2-4 -> built-in Phase 5.`
|
||||
})
|
||||
```
|
||||
|
||||
**STOP** after spawning. Wait for worker callback.
|
||||
|
||||
### Coordination (via monitor.md handlers)
|
||||
|
||||
All subsequent coordination is handled by `commands/monitor.md` handlers triggered by worker callbacks:
|
||||
|
||||
- handleCallback -> mark task done -> check pipeline -> handleSpawnNext
|
||||
- handleSpawnNext -> find ready tasks -> spawn team-worker agents -> STOP
|
||||
- handleComplete -> all done -> Phase 5
|
||||
|
||||
---
|
||||
Delegate to commands/monitor.md#handleSpawnNext:
|
||||
1. Find ready tasks (pending + blockedBy resolved)
|
||||
2. Spawn team-worker agents (see SKILL.md Spawn Template)
|
||||
3. Output status summary
|
||||
4. STOP
|
||||
|
||||
## Phase 5: Report + Completion Action
|
||||
|
||||
1. Load session state -> count completed tasks, calculate duration
|
||||
2. List deliverables:
|
||||
|
||||
| Deliverable | Path |
|
||||
|-------------|------|
|
||||
| Test Strategy | <session>/strategy/test-strategy.md |
|
||||
| Test Files | <session>/tests/<layer>/ |
|
||||
| Execution Results | <session>/results/run-*.json |
|
||||
| Quality Report | <session>/analysis/quality-report.md |
|
||||
|
||||
3. Output pipeline summary: task count, GC rounds, coverage metrics
|
||||
|
||||
4. **Completion Action** (interactive):
|
||||
|
||||
```
|
||||
AskUserQuestion({
|
||||
questions: [{
|
||||
question: "Testing pipeline complete. What would you like to do?",
|
||||
header: "Completion",
|
||||
multiSelect: false,
|
||||
options: [
|
||||
{ label: "Archive & Clean (Recommended)", description: "Archive session, clean up tasks and team resources" },
|
||||
{ label: "Keep Active", description: "Keep session active for follow-up work or inspection" },
|
||||
{ label: "Deepen Coverage", description: "Add more test layers or increase coverage targets" }
|
||||
]
|
||||
}]
|
||||
})
|
||||
```
|
||||
|
||||
5. Handle user choice:
|
||||
|
||||
| Choice | Steps |
|
||||
|--------|-------|
|
||||
| Archive & Clean | TaskList -> verify all completed -> update session status="completed" -> TeamDelete() -> output final summary |
|
||||
| Keep Active | Update session status="paused" -> output: "Session paused. Resume with: Skill(skill='team-testing', args='resume')" |
|
||||
| Deepen Coverage | Create additional TESTGEN+TESTRUN tasks for next layer -> Phase 4 |
|
||||
|
||||
---
|
||||
1. Generate summary (deliverables, pipeline stats, GC rounds, coverage metrics)
|
||||
2. Execute completion action per session.completion_action:
|
||||
- interactive -> AskUserQuestion (Archive/Keep/Deepen Coverage)
|
||||
- auto_archive -> Archive & Clean
|
||||
- auto_keep -> Keep Active
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Scenario | Resolution |
|
||||
|----------|------------|
|
||||
| Teammate no response | Send tracking message, 2 times -> respawn worker |
|
||||
| Error | Resolution |
|
||||
|-------|------------|
|
||||
| Task too vague | AskUserQuestion for clarification |
|
||||
| Session corruption | Attempt recovery, fallback to manual |
|
||||
| Worker crash | Reset task to pending, respawn |
|
||||
| Dependency cycle | Detect in analysis, halt |
|
||||
| GC loop exceeded (3 rounds) | Accept current coverage, log to wisdom, proceed |
|
||||
| Test environment failure | Report to user, suggest manual fix |
|
||||
| All tests fail | Check test framework config, notify analyst |
|
||||
| Coverage tool unavailable | Degrade to pass rate judgment |
|
||||
| Worker crash | Respawn worker, reassign task |
|
||||
| Dependency cycle | Detect, report to user, halt |
|
||||
|
||||
98
.claude/skills/team-testing/roles/executor/role.md
Normal file
98
.claude/skills/team-testing/roles/executor/role.md
Normal file
@@ -0,0 +1,98 @@
|
||||
---
|
||||
role: executor
|
||||
prefix: TESTRUN
|
||||
inner_loop: true
|
||||
message_types:
|
||||
success: tests_passed
|
||||
failure: tests_failed
|
||||
coverage: coverage_report
|
||||
error: error
|
||||
---
|
||||
|
||||
# Test Executor
|
||||
|
||||
Execute tests, collect coverage, attempt auto-fix for failures. Acts as the Critic in the Generator-Critic loop. Reports pass rate and coverage for coordinator GC decisions.
|
||||
|
||||
## Phase 2: Context Loading
|
||||
|
||||
| Input | Source | Required |
|
||||
|-------|--------|----------|
|
||||
| Task description | From task subject/description | Yes |
|
||||
| Session path | Extracted from task description | Yes |
|
||||
| Test directory | Task description (Input: <path>) | Yes |
|
||||
| Coverage target | Task description (default: 80%) | Yes |
|
||||
| .msg/meta.json | <session>/wisdom/.msg/meta.json | No |
|
||||
|
||||
1. Extract session path and test directory from task description
|
||||
2. Extract coverage target (default: 80%)
|
||||
3. Read .msg/meta.json for framework info (from strategist namespace)
|
||||
4. Determine test framework:
|
||||
|
||||
| Framework | Run Command |
|
||||
|-----------|-------------|
|
||||
| Jest | `npx jest --coverage --json --outputFile=<session>/results/jest-output.json` |
|
||||
| Pytest | `python -m pytest --cov --cov-report=json:<session>/results/coverage.json -v` |
|
||||
| Vitest | `npx vitest run --coverage --reporter=json` |
|
||||
|
||||
5. Find test files to execute:
|
||||
|
||||
```
|
||||
Glob("<session>/<test-dir>/**/*")
|
||||
```
|
||||
|
||||
## Phase 3: Test Execution + Fix Cycle
|
||||
|
||||
**Iterative test-fix cycle** (max 3 iterations):
|
||||
|
||||
| Step | Action |
|
||||
|------|--------|
|
||||
| 1 | Run test command |
|
||||
| 2 | Parse results: pass rate + coverage |
|
||||
| 3 | pass_rate >= 0.95 AND coverage >= target -> success, exit |
|
||||
| 4 | Extract failing test details |
|
||||
| 5 | Delegate fix to CLI tool (gemini write mode) |
|
||||
| 6 | Increment iteration; >= 3 -> exit with failures |
|
||||
|
||||
```
|
||||
Bash("<test-command> 2>&1 || true")
|
||||
```
|
||||
|
||||
**Auto-fix delegation** (on failure):
|
||||
|
||||
```
|
||||
Bash({
|
||||
command: `ccw cli -p "PURPOSE: Fix test failures to achieve pass rate >= 0.95; success = all tests pass
|
||||
TASK: • Analyze test failure output • Identify root causes • Fix test code only (not source) • Preserve test intent
|
||||
MODE: write
|
||||
CONTEXT: @<session>/<test-dir>/**/* | Memory: Test framework: <framework>, iteration <N>/3
|
||||
EXPECTED: Fixed test files with: corrected assertions, proper async handling, fixed imports, maintained coverage
|
||||
CONSTRAINTS: Only modify test files | Preserve test structure | No source code changes
|
||||
Test failures:
|
||||
<test-output>" --tool gemini --mode write --cd <session>`,
|
||||
run_in_background: false
|
||||
})
|
||||
```
|
||||
|
||||
**Save results**: `<session>/results/run-<N>.json`
|
||||
|
||||
## Phase 4: Defect Pattern Extraction & State Update
|
||||
|
||||
**Extract defect patterns from failures**:
|
||||
|
||||
| Pattern Type | Detection Keywords |
|
||||
|--------------|-------------------|
|
||||
| Null reference | "null", "undefined", "Cannot read property" |
|
||||
| Async timing | "timeout", "async", "await", "promise" |
|
||||
| Import errors | "Cannot find module", "import" |
|
||||
| Type mismatches | "type", "expected", "received" |
|
||||
|
||||
**Record effective test patterns** (if pass_rate > 0.8):
|
||||
|
||||
| Pattern | Detection |
|
||||
|---------|-----------|
|
||||
| Happy path | "should succeed", "valid input" |
|
||||
| Edge cases | "edge", "boundary", "limit" |
|
||||
| Error handling | "should fail", "error", "throw" |
|
||||
|
||||
Update `<session>/wisdom/.msg/meta.json` under `executor` namespace:
|
||||
- Merge `{ "executor": { pass_rate, coverage, defect_patterns, effective_patterns, coverage_history_entry } }`
|
||||
97
.claude/skills/team-testing/roles/generator/role.md
Normal file
97
.claude/skills/team-testing/roles/generator/role.md
Normal file
@@ -0,0 +1,97 @@
|
||||
---
|
||||
role: generator
|
||||
prefix: TESTGEN
|
||||
inner_loop: true
|
||||
message_types:
|
||||
success: tests_generated
|
||||
revision: tests_revised
|
||||
error: error
|
||||
---
|
||||
|
||||
# Test Generator
|
||||
|
||||
Generate test code by layer (L1 unit / L2 integration / L3 E2E). Acts as the Generator in the Generator-Critic loop. Supports revision mode for GC loop iterations.
|
||||
|
||||
## Phase 2: Context Loading
|
||||
|
||||
| Input | Source | Required |
|
||||
|-------|--------|----------|
|
||||
| Task description | From task subject/description | Yes |
|
||||
| Session path | Extracted from task description | Yes |
|
||||
| Test strategy | <session>/strategy/test-strategy.md | Yes |
|
||||
| .msg/meta.json | <session>/wisdom/.msg/meta.json | No |
|
||||
|
||||
1. Extract session path and layer from task description
|
||||
2. Read test strategy:
|
||||
|
||||
```
|
||||
Read("<session>/strategy/test-strategy.md")
|
||||
```
|
||||
|
||||
3. Read source files to test (from strategy priority_files, limit 20)
|
||||
4. Read .msg/meta.json for framework and scope context
|
||||
|
||||
5. Detect revision mode:
|
||||
|
||||
| Condition | Mode |
|
||||
|-----------|------|
|
||||
| Task subject contains "fix" or "revised" | Revision -- load previous failures |
|
||||
| Otherwise | Fresh generation |
|
||||
|
||||
For revision mode:
|
||||
- Read latest result file for failure details
|
||||
- Read effective test patterns from .msg/meta.json
|
||||
|
||||
6. Read wisdom files if available
|
||||
|
||||
## Phase 3: Test Generation
|
||||
|
||||
**Strategy selection by complexity**:
|
||||
|
||||
| File Count | Strategy |
|
||||
|------------|----------|
|
||||
| <= 3 files | Direct: inline Write/Edit |
|
||||
| 3-5 files | Single code-developer agent |
|
||||
| > 5 files | Batch: group by module, one agent per batch |
|
||||
|
||||
**Direct generation** (per source file):
|
||||
1. Generate test path: `<session>/tests/<layer>/<test-file>`
|
||||
2. Generate test code: happy path, edge cases, error handling
|
||||
3. Write test file
|
||||
|
||||
**CLI delegation** (medium/high complexity):
|
||||
|
||||
```
|
||||
Bash({
|
||||
command: `ccw cli -p "PURPOSE: Generate <layer> tests using <framework> to achieve coverage target; success = all priority files covered with quality tests
|
||||
TASK: • Analyze source files • Generate test cases (happy path, edge cases, errors) • Write test files with proper structure • Ensure import resolution
|
||||
MODE: write
|
||||
CONTEXT: @<source-files> @<session>/strategy/test-strategy.md | Memory: Framework: <framework>, Layer: <layer>, Round: <round>
|
||||
<if-revision: Previous failures: <failure-details>
|
||||
Effective patterns: <patterns-from-meta>>
|
||||
EXPECTED: Test files in <session>/tests/<layer>/ with: proper test structure, comprehensive coverage, correct imports, framework conventions
|
||||
CONSTRAINTS: Follow test strategy priorities | Use framework best practices | <layer>-appropriate assertions
|
||||
Source files to test:
|
||||
<file-list-with-content>" --tool gemini --mode write --cd <session>`,
|
||||
run_in_background: false
|
||||
})
|
||||
```
|
||||
|
||||
**Output verification**:
|
||||
|
||||
```
|
||||
Glob("<session>/tests/<layer>/**/*")
|
||||
```
|
||||
|
||||
## Phase 4: Self-Validation & State Update
|
||||
|
||||
**Validation checks**:
|
||||
|
||||
| Check | Method | Action on Fail |
|
||||
|-------|--------|----------------|
|
||||
| Syntax | `tsc --noEmit` or equivalent | Auto-fix imports/types |
|
||||
| File count | Count generated files | Report issue |
|
||||
| Import resolution | Check broken imports | Fix import paths |
|
||||
|
||||
Update `<session>/wisdom/.msg/meta.json` under `generator` namespace:
|
||||
- Merge `{ "generator": { test_files, layer, round, is_revision } }`
|
||||
83
.claude/skills/team-testing/roles/strategist/role.md
Normal file
83
.claude/skills/team-testing/roles/strategist/role.md
Normal file
@@ -0,0 +1,83 @@
|
||||
---
|
||||
role: strategist
|
||||
prefix: STRATEGY
|
||||
inner_loop: false
|
||||
message_types:
|
||||
success: strategy_ready
|
||||
error: error
|
||||
---
|
||||
|
||||
# Test Strategist
|
||||
|
||||
Analyze git diff, determine test layers, define coverage targets, and formulate test strategy with prioritized execution order.
|
||||
|
||||
## Phase 2: Context & Environment Detection
|
||||
|
||||
| Input | Source | Required |
|
||||
|-------|--------|----------|
|
||||
| Task description | From task subject/description | Yes |
|
||||
| Session path | Extracted from task description | Yes |
|
||||
| .msg/meta.json | <session>/wisdom/.msg/meta.json | No |
|
||||
|
||||
1. Extract session path and scope from task description
|
||||
2. Get git diff for change analysis:
|
||||
|
||||
```
|
||||
Bash("git diff HEAD~1 --name-only 2>/dev/null || git diff --cached --name-only")
|
||||
Bash("git diff HEAD~1 -- <changed-files> 2>/dev/null || git diff --cached -- <changed-files>")
|
||||
```
|
||||
|
||||
3. Detect test framework from project files:
|
||||
|
||||
| Signal File | Framework | Test Pattern |
|
||||
|-------------|-----------|-------------|
|
||||
| jest.config.js/ts | Jest | `**/*.test.{ts,tsx,js}` |
|
||||
| vitest.config.ts/js | Vitest | `**/*.test.{ts,tsx}` |
|
||||
| pytest.ini / pyproject.toml | Pytest | `**/test_*.py` |
|
||||
| No detection | Default | Jest patterns |
|
||||
|
||||
4. Scan existing test patterns:
|
||||
|
||||
```
|
||||
Glob("**/*.test.*")
|
||||
Glob("**/*.spec.*")
|
||||
```
|
||||
|
||||
5. Read .msg/meta.json if exists for session context
|
||||
|
||||
## Phase 3: Strategy Formulation
|
||||
|
||||
**Change analysis dimensions**:
|
||||
|
||||
| Change Type | Analysis | Priority |
|
||||
|-------------|----------|----------|
|
||||
| New files | Need new tests | High |
|
||||
| Modified functions | Need updated tests | Medium |
|
||||
| Deleted files | Need test cleanup | Low |
|
||||
| Config changes | May need integration tests | Variable |
|
||||
|
||||
**Strategy output structure**:
|
||||
|
||||
1. **Change Analysis Table**: File, Change Type, Impact, Priority
|
||||
2. **Test Layer Recommendations**:
|
||||
- L1 Unit: Scope, Coverage Target, Priority Files, Patterns
|
||||
- L2 Integration: Scope, Coverage Target, Integration Points
|
||||
- L3 E2E: Scope, Coverage Target, User Scenarios
|
||||
3. **Risk Assessment**: Risk, Probability, Impact, Mitigation
|
||||
4. **Test Execution Order**: Prioritized sequence
|
||||
|
||||
Write strategy to `<session>/strategy/test-strategy.md`
|
||||
|
||||
**Self-validation**:
|
||||
|
||||
| Check | Criteria | Fallback |
|
||||
|-------|----------|----------|
|
||||
| Has L1 scope | L1 scope not empty | Default to all changed files |
|
||||
| Has coverage targets | L1 target > 0 | Use defaults (80/60/40) |
|
||||
| Has priority files | List not empty | Use all changed files |
|
||||
|
||||
## Phase 4: Wisdom & State Update
|
||||
|
||||
1. Write discoveries to `<session>/wisdom/conventions.md` (detected framework, patterns)
|
||||
2. Update `<session>/wisdom/.msg/meta.json` under `strategist` namespace:
|
||||
- Read existing -> merge `{ "strategist": { framework, layers, coverage_targets, priority_files, risks } }` -> write back
|
||||
101
.claude/skills/team-testing/specs/pipelines.md
Normal file
101
.claude/skills/team-testing/specs/pipelines.md
Normal file
@@ -0,0 +1,101 @@
|
||||
# Testing Pipelines
|
||||
|
||||
Pipeline definitions and task registry for team-testing.
|
||||
|
||||
## Pipeline Selection
|
||||
|
||||
| Condition | Pipeline |
|
||||
|-----------|----------|
|
||||
| fileCount <= 3 AND moduleCount <= 1 | targeted |
|
||||
| fileCount <= 10 AND moduleCount <= 3 | standard |
|
||||
| Otherwise | comprehensive |
|
||||
|
||||
## Pipeline Definitions
|
||||
|
||||
### Targeted Pipeline (3 tasks, serial)
|
||||
|
||||
```
|
||||
STRATEGY-001 -> TESTGEN-001 -> TESTRUN-001
|
||||
```
|
||||
|
||||
| Task ID | Role | Dependencies | Layer | Description |
|
||||
|---------|------|-------------|-------|-------------|
|
||||
| STRATEGY-001 | strategist | (none) | — | Analyze changes, define test strategy |
|
||||
| TESTGEN-001 | generator | STRATEGY-001 | L1 | Generate L1 unit tests |
|
||||
| TESTRUN-001 | executor | TESTGEN-001 | L1 | Execute L1 tests, collect coverage |
|
||||
|
||||
### Standard Pipeline (6 tasks, progressive layers)
|
||||
|
||||
```
|
||||
STRATEGY-001 -> TESTGEN-001 -> TESTRUN-001 -> TESTGEN-002 -> TESTRUN-002 -> TESTANA-001
|
||||
```
|
||||
|
||||
| Task ID | Role | Dependencies | Layer | Description |
|
||||
|---------|------|-------------|-------|-------------|
|
||||
| STRATEGY-001 | strategist | (none) | — | Analyze changes, define test strategy |
|
||||
| TESTGEN-001 | generator | STRATEGY-001 | L1 | Generate L1 unit tests |
|
||||
| TESTRUN-001 | executor | TESTGEN-001 | L1 | Execute L1 tests, collect coverage |
|
||||
| TESTGEN-002 | generator | TESTRUN-001 | L2 | Generate L2 integration tests |
|
||||
| TESTRUN-002 | executor | TESTGEN-002 | L2 | Execute L2 tests, collect coverage |
|
||||
| TESTANA-001 | analyst | TESTRUN-002 | — | Defect pattern analysis, quality report |
|
||||
|
||||
### Comprehensive Pipeline (8 tasks, parallel windows)
|
||||
|
||||
```
|
||||
STRATEGY-001 -> [TESTGEN-001 || TESTGEN-002] -> [TESTRUN-001 || TESTRUN-002] -> TESTGEN-003 -> TESTRUN-003 -> TESTANA-001
|
||||
```
|
||||
|
||||
| Task ID | Role | Dependencies | Layer | Description |
|
||||
|---------|------|-------------|-------|-------------|
|
||||
| STRATEGY-001 | strategist | (none) | — | Analyze changes, define test strategy |
|
||||
| TESTGEN-001 | generator-1 | STRATEGY-001 | L1 | Generate L1 unit tests (parallel) |
|
||||
| TESTGEN-002 | generator-2 | STRATEGY-001 | L2 | Generate L2 integration tests (parallel) |
|
||||
| TESTRUN-001 | executor-1 | TESTGEN-001 | L1 | Execute L1 tests (parallel) |
|
||||
| TESTRUN-002 | executor-2 | TESTGEN-002 | L2 | Execute L2 tests (parallel) |
|
||||
| TESTGEN-003 | generator | TESTRUN-001, TESTRUN-002 | L3 | Generate L3 E2E tests |
|
||||
| TESTRUN-003 | executor | TESTGEN-003 | L3 | Execute L3 tests, collect coverage |
|
||||
| TESTANA-001 | analyst | TESTRUN-003 | — | Defect pattern analysis, quality report |
|
||||
|
||||
## GC Loop (Generator-Critic)
|
||||
|
||||
Generator and executor iterate per test layer:
|
||||
|
||||
```
|
||||
TESTGEN -> TESTRUN -> (if pass_rate < 0.95 OR coverage < target) -> TESTGEN-fix -> TESTRUN-fix
|
||||
(if pass_rate >= 0.95 AND coverage >= target) -> next layer or TESTANA
|
||||
```
|
||||
|
||||
- Max iterations: 3 per layer
|
||||
- After 3 iterations: accept current state with warning
|
||||
|
||||
## Coverage Targets
|
||||
|
||||
| Layer | Name | Default Target |
|
||||
|-------|------|----------------|
|
||||
| L1 | Unit Tests | 80% |
|
||||
| L2 | Integration Tests | 60% |
|
||||
| L3 | E2E Tests | 40% |
|
||||
|
||||
## Session Directory
|
||||
|
||||
```
|
||||
.workflow/.team/TST-<slug>-<YYYY-MM-DD>/
|
||||
├── .msg/messages.jsonl # Message bus log
|
||||
├── .msg/meta.json # Session metadata
|
||||
├── wisdom/ # Cross-task knowledge
|
||||
│ ├── learnings.md
|
||||
│ ├── decisions.md
|
||||
│ ├── conventions.md
|
||||
│ └── issues.md
|
||||
├── strategy/ # Strategist output
|
||||
│ └── test-strategy.md
|
||||
├── tests/ # Generator output
|
||||
│ ├── L1-unit/
|
||||
│ ├── L2-integration/
|
||||
│ └── L3-e2e/
|
||||
├── results/ # Executor output
|
||||
│ ├── run-001.json
|
||||
│ └── coverage-001.json
|
||||
└── analysis/ # Analyst output
|
||||
└── quality-report.md
|
||||
```
|
||||
Reference in New Issue
Block a user