Add quality gates and tuning strategies documentation

- Introduced quality gates specification for skill tuning, detailing quality dimensions, scoring, and gate definitions.
- Added comprehensive tuning strategies for various issue categories, including context explosion, long-tail forgetting, data flow, and agent coordination.
- Created templates for diagnosis reports and fix proposals to standardize documentation and reporting processes.
This commit is contained in:
catlog22
2026-01-14 12:59:13 +08:00
parent 6b4b9b0775
commit 633d918da1
20 changed files with 5755 additions and 0 deletions

View File

@@ -0,0 +1,153 @@
# Diagnosis Report Template
Template for individual diagnosis action reports.
## Template
```markdown
# {{diagnosis_type}} Diagnosis Report
**Target Skill**: {{skill_name}}
**Diagnosis Type**: {{diagnosis_type}}
**Executed At**: {{timestamp}}
**Duration**: {{duration_ms}}ms
---
## Summary
| Metric | Value |
|--------|-------|
| Issues Found | {{issues_found}} |
| Severity | {{severity}} |
| Patterns Checked | {{patterns_checked_count}} |
| Patterns Matched | {{patterns_matched_count}} |
---
## Patterns Analyzed
{{#each patterns_checked}}
### {{pattern_name}}
- **Status**: {{status}}
- **Matches**: {{match_count}}
- **Files Affected**: {{affected_files}}
{{/each}}
---
## Issues Identified
{{#if issues.length}}
{{#each issues}}
### {{id}}: {{description}}
| Field | Value |
|-------|-------|
| Type | {{type}} |
| Severity | {{severity}} |
| Location | {{location}} |
| Root Cause | {{root_cause}} |
| Impact | {{impact}} |
**Evidence**:
{{#each evidence}}
- `{{this}}`
{{/each}}
**Suggested Fix**: {{suggested_fix}}
---
{{/each}}
{{else}}
_No issues found in this diagnosis area._
{{/if}}
---
## Recommendations
{{#if recommendations.length}}
{{#each recommendations}}
{{@index}}. {{this}}
{{/each}}
{{else}}
No specific recommendations - area appears healthy.
{{/if}}
---
## Raw Data
Full diagnosis data available at:
`{{output_file}}`
```
## Variable Reference
| Variable | Type | Source |
|----------|------|--------|
| `diagnosis_type` | string | 'context' \| 'memory' \| 'dataflow' \| 'agent' |
| `skill_name` | string | state.target_skill.name |
| `timestamp` | string | ISO timestamp |
| `duration_ms` | number | Execution time |
| `issues_found` | number | issues.length |
| `severity` | string | Calculated severity |
| `patterns_checked` | array | Patterns analyzed |
| `patterns_matched` | array | Patterns with matches |
| `issues` | array | Issue objects |
| `recommendations` | array | String recommendations |
| `output_file` | string | Path to JSON file |
## Usage
```javascript
function renderDiagnosisReport(diagnosis, diagnosisType, skillName, outputFile) {
return `# ${diagnosisType} Diagnosis Report
**Target Skill**: ${skillName}
**Diagnosis Type**: ${diagnosisType}
**Executed At**: ${new Date().toISOString()}
**Duration**: ${diagnosis.execution_time_ms}ms
---
## Summary
| Metric | Value |
|--------|-------|
| Issues Found | ${diagnosis.issues_found} |
| Severity | ${diagnosis.severity} |
| Patterns Checked | ${diagnosis.details.patterns_checked.length} |
| Patterns Matched | ${diagnosis.details.patterns_matched.length} |
---
## Issues Identified
${diagnosis.details.evidence.map((e, i) => `
### Issue ${i + 1}
- **File**: ${e.file}
- **Pattern**: ${e.pattern}
- **Severity**: ${e.severity}
- **Context**: \`${e.context}\`
`).join('\n')}
---
## Recommendations
${diagnosis.details.recommendations.map((r, i) => `${i + 1}. ${r}`).join('\n')}
---
## Raw Data
Full diagnosis data available at:
\`${outputFile}\`
`;
}
```

View File

@@ -0,0 +1,204 @@
# Fix Proposal Template
Template for fix proposal documentation.
## Template
```markdown
# Fix Proposal: {{fix_id}}
**Strategy**: {{strategy}}
**Risk Level**: {{risk}}
**Issues Addressed**: {{issue_ids}}
---
## Description
{{description}}
## Rationale
{{rationale}}
---
## Affected Files
{{#each changes}}
### {{file}}
**Action**: {{action}}
```diff
{{diff}}
```
{{/each}}
---
## Implementation Steps
{{#each implementation_steps}}
{{@index}}. {{this}}
{{/each}}
---
## Risk Assessment
| Factor | Assessment |
|--------|------------|
| Complexity | {{complexity}} |
| Reversibility | {{reversible ? 'Yes' : 'No'}} |
| Breaking Changes | {{breaking_changes}} |
| Test Coverage | {{test_coverage}} |
**Overall Risk**: {{risk}}
---
## Verification Steps
{{#each verification_steps}}
- [ ] {{this}}
{{/each}}
---
## Rollback Plan
{{#if rollback_available}}
To rollback this fix:
```bash
{{rollback_command}}
```
{{else}}
_Rollback not available for this fix type._
{{/if}}
---
## Estimated Impact
{{estimated_impact}}
```
## Variable Reference
| Variable | Type | Source |
|----------|------|--------|
| `fix_id` | string | Generated ID (FIX-001) |
| `strategy` | string | Fix strategy name |
| `risk` | string | 'low' \| 'medium' \| 'high' |
| `issue_ids` | array | Related issue IDs |
| `description` | string | Human-readable description |
| `rationale` | string | Why this fix works |
| `changes` | array | File change objects |
| `implementation_steps` | array | Step-by-step guide |
| `verification_steps` | array | How to verify fix worked |
| `estimated_impact` | string | Expected improvement |
## Usage
```javascript
function renderFixProposal(fix) {
return `# Fix Proposal: ${fix.id}
**Strategy**: ${fix.strategy}
**Risk Level**: ${fix.risk}
**Issues Addressed**: ${fix.issue_ids.join(', ')}
---
## Description
${fix.description}
## Rationale
${fix.rationale}
---
## Affected Files
${fix.changes.map(change => `
### ${change.file}
**Action**: ${change.action}
\`\`\`diff
${change.diff || change.new_content?.slice(0, 200) || 'N/A'}
\`\`\`
`).join('\n')}
---
## Verification Steps
${fix.verification_steps.map(step => `- [ ] ${step}`).join('\n')}
---
## Estimated Impact
${fix.estimated_impact}
`;
}
```
## Fix Strategy Templates
### sliding_window
```markdown
## Description
Implement sliding window for conversation history to prevent unbounded growth.
## Changes
- Add MAX_HISTORY constant
- Modify history update logic to slice array
- Update state schema documentation
## Verification
- [ ] Run skill for 10+ iterations
- [ ] Verify history.length <= MAX_HISTORY
- [ ] Check no data loss for recent items
```
### constraint_injection
```markdown
## Description
Add explicit constraint section to each phase prompt.
## Changes
- Add [CONSTRAINTS] section template
- Reference state.original_requirements
- Add reminder before output section
## Verification
- [ ] Check constraints visible in all phases
- [ ] Test with specific constraint
- [ ] Verify output respects constraint
```
### error_wrapping
```markdown
## Description
Wrap all Task calls in try-catch with retry logic.
## Changes
- Create safeTask wrapper function
- Replace direct Task calls
- Add error logging to state
## Verification
- [ ] Simulate agent failure
- [ ] Verify graceful error handling
- [ ] Check retry logic
```