mirror of
https://github.com/catlog22/Claude-Code-Workflow.git
synced 2026-02-12 02:37:45 +08:00
feat: 添加令牌消耗诊断功能,优化输出和状态管理
This commit is contained in:
@@ -70,6 +70,7 @@ Based on comprehensive analysis, skill-tuning addresses **core skill issues** an
|
|||||||
| **P2** | Agent Coordination | Fragile call chains, merge complexity | error_wrapping, result_validation |
|
| **P2** | Agent Coordination | Fragile call chains, merge complexity | error_wrapping, result_validation |
|
||||||
| **P3** | Context Explosion | Token accumulation, multi-turn bloat | sliding_window, context_summarization |
|
| **P3** | Context Explosion | Token accumulation, multi-turn bloat | sliding_window, context_summarization |
|
||||||
| **P4** | Long-tail Forgetting | Early constraint loss | constraint_injection, checkpoint_restore |
|
| **P4** | Long-tail Forgetting | Early constraint loss | constraint_injection, checkpoint_restore |
|
||||||
|
| **P5** | Token Consumption | Verbose prompts, excessive state, redundant I/O | prompt_compression, lazy_loading, output_minimization |
|
||||||
|
|
||||||
### General Optimization Areas (按需分析 via Gemini CLI)
|
### General Optimization Areas (按需分析 via Gemini CLI)
|
||||||
|
|
||||||
@@ -202,47 +203,27 @@ RULES: $(cat ~/.claude/workflows/cli-templates/protocols/analysis-protocol.md) |
|
|||||||
│ → Initialize state.json with target skill info │
|
│ → Initialize state.json with target skill info │
|
||||||
│ → Create backup of target skill files │
|
│ → Create backup of target skill files │
|
||||||
├─────────────────────────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────────────────────────┤
|
||||||
│ action-analyze-requirements: Requirement Analysis (NEW) │
|
│ action-analyze-requirements: Requirement Analysis │
|
||||||
│ → Phase 1: 维度拆解 (Gemini CLI) - 单一描述 → 多个关注维度 │
|
│ → Phase 1: 维度拆解 (Gemini CLI) - 单一描述 → 多个关注维度 │
|
||||||
│ → Phase 2: Spec 匹配 - 每个维度 → taxonomy + strategy │
|
│ → Phase 2: Spec 匹配 - 每个维度 → taxonomy + strategy │
|
||||||
│ → Phase 3: 覆盖度评估 - 以"有修复策略"为满足标准 │
|
│ → Phase 3: 覆盖度评估 - 以"有修复策略"为满足标准 │
|
||||||
│ → Phase 4: 歧义检测 - 识别多义性描述,必要时请求澄清 │
|
│ → Phase 4: 歧义检测 - 识别多义性描述,必要时请求澄清 │
|
||||||
│ → Output: requirement-analysis.json, 自动优化 focus_areas │
|
│ → Output: state.json (requirement_analysis field) │
|
||||||
├─────────────────────────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────────────────────────┤
|
||||||
│ action-diagnose-context: Context Explosion Analysis │
|
│ action-diagnose-*: Diagnosis Actions (context/memory/dataflow/agent/docs/ │
|
||||||
│ → Scan for token accumulation patterns │
|
│ token_consumption) │
|
||||||
│ → Detect multi-turn dialogue growth │
|
│ → Execute pattern-based detection for each category │
|
||||||
│ → Output: context-diagnosis.json │
|
│ → Output: state.json (diagnosis.{category} field) │
|
||||||
├─────────────────────────────────────────────────────────────────────────────┤
|
|
||||||
│ action-diagnose-memory: Long-tail Forgetting Analysis │
|
|
||||||
│ → Trace constraint propagation through phases │
|
|
||||||
│ → Detect early instruction loss │
|
|
||||||
│ → Output: memory-diagnosis.json │
|
|
||||||
├─────────────────────────────────────────────────────────────────────────────┤
|
|
||||||
│ action-diagnose-dataflow: Data Flow Analysis │
|
|
||||||
│ → Map state transitions between phases │
|
|
||||||
│ → Detect format inconsistencies │
|
|
||||||
│ → Output: dataflow-diagnosis.json │
|
|
||||||
├─────────────────────────────────────────────────────────────────────────────┤
|
|
||||||
│ action-diagnose-agent: Agent Coordination Analysis │
|
|
||||||
│ → Analyze agent call patterns │
|
|
||||||
│ → Detect result passing issues │
|
|
||||||
│ → Output: agent-diagnosis.json │
|
|
||||||
├─────────────────────────────────────────────────────────────────────────────┤
|
|
||||||
│ action-diagnose-docs: Documentation Structure Analysis (Optional) │
|
|
||||||
│ → Detect definition duplicates across files │
|
|
||||||
│ → Detect conflicting definitions │
|
|
||||||
│ → Output: docs-diagnosis.json │
|
|
||||||
├─────────────────────────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────────────────────────┤
|
||||||
│ action-generate-report: Consolidated Report │
|
│ action-generate-report: Consolidated Report │
|
||||||
│ → Merge all diagnosis results │
|
│ → Generate markdown summary from state.diagnosis │
|
||||||
│ → Prioritize issues by severity │
|
│ → Prioritize issues by severity │
|
||||||
│ → Output: tuning-report.md │
|
│ → Output: state.json (final_report field) │
|
||||||
├─────────────────────────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────────────────────────┤
|
||||||
│ action-propose-fixes: Fix Proposal Generation │
|
│ action-propose-fixes: Fix Proposal Generation │
|
||||||
│ → Generate fix strategies for each issue │
|
│ → Generate fix strategies for each issue │
|
||||||
│ → Create implementation plan │
|
│ → Create implementation plan │
|
||||||
│ → Output: fix-proposals.json │
|
│ → Output: state.json (proposed_fixes field) │
|
||||||
├─────────────────────────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────────────────────────┤
|
||||||
│ action-apply-fix: Apply Selected Fix │
|
│ action-apply-fix: Apply Selected Fix │
|
||||||
│ → User selects fix to apply │
|
│ → User selects fix to apply │
|
||||||
@@ -255,9 +236,9 @@ RULES: $(cat ~/.claude/workflows/cli-templates/protocols/analysis-protocol.md) |
|
|||||||
│ → Update iteration count │
|
│ → Update iteration count │
|
||||||
├─────────────────────────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────────────────────────┤
|
||||||
│ action-complete: Finalization │
|
│ action-complete: Finalization │
|
||||||
│ → Generate final report │
|
│ → Set status='completed' │
|
||||||
│ → Cleanup temporary files │
|
│ → Final report already in state.json (final_report field) │
|
||||||
│ → Output: tuning-summary.md │
|
│ → Output: state.json (final) │
|
||||||
└─────────────────────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -267,31 +248,25 @@ RULES: $(cat ~/.claude/workflows/cli-templates/protocols/analysis-protocol.md) |
|
|||||||
const timestamp = new Date().toISOString().slice(0,19).replace(/[-:T]/g, '');
|
const timestamp = new Date().toISOString().slice(0,19).replace(/[-:T]/g, '');
|
||||||
const workDir = `.workflow/.scratchpad/skill-tuning-${timestamp}`;
|
const workDir = `.workflow/.scratchpad/skill-tuning-${timestamp}`;
|
||||||
|
|
||||||
Bash(`mkdir -p "${workDir}/diagnosis"`);
|
// Simplified: Only backups dir needed, diagnosis results go into state.json
|
||||||
Bash(`mkdir -p "${workDir}/backups"`);
|
Bash(`mkdir -p "${workDir}/backups"`);
|
||||||
Bash(`mkdir -p "${workDir}/fixes"`);
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Output Structure
|
## Output Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
.workflow/.scratchpad/skill-tuning-{timestamp}/
|
.workflow/.scratchpad/skill-tuning-{timestamp}/
|
||||||
├── state.json # Session state (orchestrator-managed)
|
├── state.json # Single source of truth (all results consolidated)
|
||||||
├── diagnosis/
|
│ ├── diagnosis.* # All diagnosis results embedded
|
||||||
│ ├── context-diagnosis.json # Context explosion analysis
|
│ ├── issues[] # Found issues
|
||||||
│ ├── memory-diagnosis.json # Long-tail forgetting analysis
|
│ ├── proposed_fixes[] # Fix proposals
|
||||||
│ ├── dataflow-diagnosis.json # Data flow analysis
|
│ └── final_report # Markdown summary (on completion)
|
||||||
│ ├── agent-diagnosis.json # Agent coordination analysis
|
└── backups/
|
||||||
│ └── docs-diagnosis.json # Documentation structure analysis (optional)
|
└── {skill-name}-backup/ # Original skill files backup
|
||||||
├── backups/
|
|
||||||
│ └── {skill-name}-backup/ # Original skill files backup
|
|
||||||
├── fixes/
|
|
||||||
│ ├── fix-proposals.json # Proposed fixes
|
|
||||||
│ └── applied-fixes.json # Applied fix history
|
|
||||||
├── tuning-report.md # Consolidated diagnosis report
|
|
||||||
└── tuning-summary.md # Final summary
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
> **Token Optimization**: All outputs consolidated into state.json. No separate diagnosis files or report files.
|
||||||
|
|
||||||
## State Schema
|
## State Schema
|
||||||
|
|
||||||
详细状态结构定义请参阅 [phases/state-schema.md](phases/state-schema.md)。
|
详细状态结构定义请参阅 [phases/state-schema.md](phases/state-schema.md)。
|
||||||
@@ -316,6 +291,7 @@ Bash(`mkdir -p "${workDir}/fixes"`);
|
|||||||
| [phases/actions/action-diagnose-dataflow.md](phases/actions/action-diagnose-dataflow.md) | Data flow diagnosis |
|
| [phases/actions/action-diagnose-dataflow.md](phases/actions/action-diagnose-dataflow.md) | Data flow diagnosis |
|
||||||
| [phases/actions/action-diagnose-agent.md](phases/actions/action-diagnose-agent.md) | Agent coordination diagnosis |
|
| [phases/actions/action-diagnose-agent.md](phases/actions/action-diagnose-agent.md) | Agent coordination diagnosis |
|
||||||
| [phases/actions/action-diagnose-docs.md](phases/actions/action-diagnose-docs.md) | Documentation structure diagnosis |
|
| [phases/actions/action-diagnose-docs.md](phases/actions/action-diagnose-docs.md) | Documentation structure diagnosis |
|
||||||
|
| [phases/actions/action-diagnose-token-consumption.md](phases/actions/action-diagnose-token-consumption.md) | Token consumption diagnosis |
|
||||||
| [phases/actions/action-generate-report.md](phases/actions/action-generate-report.md) | Report generation |
|
| [phases/actions/action-generate-report.md](phases/actions/action-generate-report.md) | Report generation |
|
||||||
| [phases/actions/action-propose-fixes.md](phases/actions/action-propose-fixes.md) | Fix proposal |
|
| [phases/actions/action-propose-fixes.md](phases/actions/action-propose-fixes.md) | Fix proposal |
|
||||||
| [phases/actions/action-apply-fix.md](phases/actions/action-apply-fix.md) | Fix application |
|
| [phases/actions/action-apply-fix.md](phases/actions/action-apply-fix.md) | Fix application |
|
||||||
|
|||||||
@@ -0,0 +1,200 @@
|
|||||||
|
# Action: Diagnose Token Consumption
|
||||||
|
|
||||||
|
Analyze target skill for token consumption inefficiencies and output optimization opportunities.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Detect patterns that cause excessive token usage:
|
||||||
|
- Verbose prompts without compression
|
||||||
|
- Large state objects with unnecessary fields
|
||||||
|
- Full content passing instead of references
|
||||||
|
- Unbounded arrays without sliding windows
|
||||||
|
- Redundant file I/O (write-then-read patterns)
|
||||||
|
|
||||||
|
## Detection Patterns
|
||||||
|
|
||||||
|
| Pattern ID | Name | Detection Logic | Severity |
|
||||||
|
|------------|------|-----------------|----------|
|
||||||
|
| TKN-001 | Verbose Prompts | Prompt files > 4KB or high static/variable ratio | medium |
|
||||||
|
| TKN-002 | Excessive State Fields | State schema > 15 top-level keys | medium |
|
||||||
|
| TKN-003 | Full Content Passing | `Read()` result embedded directly in prompt | high |
|
||||||
|
| TKN-004 | Unbounded Arrays | `.push`/`concat` without `.slice(-N)` | high |
|
||||||
|
| TKN-005 | Redundant Write→Read | `Write(file)` followed by `Read(file)` | medium |
|
||||||
|
|
||||||
|
## Execution Steps
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
async function diagnoseTokenConsumption(state, workDir) {
|
||||||
|
const issues = [];
|
||||||
|
const evidence = [];
|
||||||
|
const skillPath = state.target_skill.path;
|
||||||
|
|
||||||
|
// 1. Scan for verbose prompts (TKN-001)
|
||||||
|
const mdFiles = Glob(`${skillPath}/**/*.md`);
|
||||||
|
for (const file of mdFiles) {
|
||||||
|
const content = Read(file);
|
||||||
|
if (content.length > 4000) {
|
||||||
|
evidence.push({
|
||||||
|
file: file,
|
||||||
|
pattern: 'TKN-001',
|
||||||
|
severity: 'medium',
|
||||||
|
context: `File size: ${content.length} chars (threshold: 4000)`
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. Check state schema field count (TKN-002)
|
||||||
|
const stateSchema = Glob(`${skillPath}/**/state-schema.md`)[0];
|
||||||
|
if (stateSchema) {
|
||||||
|
const schemaContent = Read(stateSchema);
|
||||||
|
const fieldMatches = schemaContent.match(/^\s*\w+:/gm) || [];
|
||||||
|
if (fieldMatches.length > 15) {
|
||||||
|
evidence.push({
|
||||||
|
file: stateSchema,
|
||||||
|
pattern: 'TKN-002',
|
||||||
|
severity: 'medium',
|
||||||
|
context: `State has ${fieldMatches.length} fields (threshold: 15)`
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3. Detect full content passing (TKN-003)
|
||||||
|
const fullContentPattern = /Read\([^)]+\)\s*[\+,]|`\$\{.*Read\(/g;
|
||||||
|
for (const file of mdFiles) {
|
||||||
|
const content = Read(file);
|
||||||
|
const matches = content.match(fullContentPattern);
|
||||||
|
if (matches) {
|
||||||
|
evidence.push({
|
||||||
|
file: file,
|
||||||
|
pattern: 'TKN-003',
|
||||||
|
severity: 'high',
|
||||||
|
context: `Full content passing detected: ${matches[0]}`
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4. Detect unbounded arrays (TKN-004)
|
||||||
|
const unboundedPattern = /\.(push|concat)\([^)]+\)(?!.*\.slice)/g;
|
||||||
|
for (const file of mdFiles) {
|
||||||
|
const content = Read(file);
|
||||||
|
const matches = content.match(unboundedPattern);
|
||||||
|
if (matches) {
|
||||||
|
evidence.push({
|
||||||
|
file: file,
|
||||||
|
pattern: 'TKN-004',
|
||||||
|
severity: 'high',
|
||||||
|
context: `Unbounded array growth: ${matches[0]}`
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 5. Detect write-then-read patterns (TKN-005)
|
||||||
|
const writeReadPattern = /Write\([^)]+\)[\s\S]{0,100}Read\([^)]+\)/g;
|
||||||
|
for (const file of mdFiles) {
|
||||||
|
const content = Read(file);
|
||||||
|
const matches = content.match(writeReadPattern);
|
||||||
|
if (matches) {
|
||||||
|
evidence.push({
|
||||||
|
file: file,
|
||||||
|
pattern: 'TKN-005',
|
||||||
|
severity: 'medium',
|
||||||
|
context: `Write-then-read pattern detected`
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Calculate severity
|
||||||
|
const highCount = evidence.filter(e => e.severity === 'high').length;
|
||||||
|
const mediumCount = evidence.filter(e => e.severity === 'medium').length;
|
||||||
|
|
||||||
|
let severity = 'none';
|
||||||
|
if (highCount > 0) severity = 'high';
|
||||||
|
else if (mediumCount > 2) severity = 'medium';
|
||||||
|
else if (mediumCount > 0) severity = 'low';
|
||||||
|
|
||||||
|
return {
|
||||||
|
status: 'completed',
|
||||||
|
issues_found: evidence.length,
|
||||||
|
severity: severity,
|
||||||
|
execution_time_ms: Date.now() - startTime,
|
||||||
|
details: {
|
||||||
|
patterns_checked: ['TKN-001', 'TKN-002', 'TKN-003', 'TKN-004', 'TKN-005'],
|
||||||
|
patterns_matched: [...new Set(evidence.map(e => e.pattern))],
|
||||||
|
evidence: evidence,
|
||||||
|
recommendations: generateRecommendations(evidence)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function generateRecommendations(evidence) {
|
||||||
|
const recs = [];
|
||||||
|
const patterns = [...new Set(evidence.map(e => e.pattern))];
|
||||||
|
|
||||||
|
if (patterns.includes('TKN-001')) {
|
||||||
|
recs.push('Apply prompt_compression: Extract static instructions to templates, use placeholders');
|
||||||
|
}
|
||||||
|
if (patterns.includes('TKN-002')) {
|
||||||
|
recs.push('Apply state_field_reduction: Remove debug/cache fields, consolidate related fields');
|
||||||
|
}
|
||||||
|
if (patterns.includes('TKN-003')) {
|
||||||
|
recs.push('Apply lazy_loading: Pass file paths instead of content, let agents read if needed');
|
||||||
|
}
|
||||||
|
if (patterns.includes('TKN-004')) {
|
||||||
|
recs.push('Apply sliding_window: Add .slice(-N) to array operations to bound growth');
|
||||||
|
}
|
||||||
|
if (patterns.includes('TKN-005')) {
|
||||||
|
recs.push('Apply output_minimization: Use in-memory data passing, eliminate temporary files');
|
||||||
|
}
|
||||||
|
|
||||||
|
return recs;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Output
|
||||||
|
|
||||||
|
Write diagnosis result to `${workDir}/diagnosis/token-consumption-diagnosis.json`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "completed",
|
||||||
|
"issues_found": 3,
|
||||||
|
"severity": "medium",
|
||||||
|
"execution_time_ms": 1500,
|
||||||
|
"details": {
|
||||||
|
"patterns_checked": ["TKN-001", "TKN-002", "TKN-003", "TKN-004", "TKN-005"],
|
||||||
|
"patterns_matched": ["TKN-001", "TKN-003"],
|
||||||
|
"evidence": [
|
||||||
|
{
|
||||||
|
"file": "phases/orchestrator.md",
|
||||||
|
"pattern": "TKN-001",
|
||||||
|
"severity": "medium",
|
||||||
|
"context": "File size: 5200 chars (threshold: 4000)"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"recommendations": [
|
||||||
|
"Apply prompt_compression: Extract static instructions to templates"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## State Update
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
updateState({
|
||||||
|
diagnosis: {
|
||||||
|
...state.diagnosis,
|
||||||
|
token_consumption: diagnosisResult
|
||||||
|
}
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fix Strategies Mapping
|
||||||
|
|
||||||
|
| Pattern | Strategy | Implementation |
|
||||||
|
|---------|----------|----------------|
|
||||||
|
| TKN-001 | prompt_compression | Extract static text to variables, use template inheritance |
|
||||||
|
| TKN-002 | state_field_reduction | Audit and consolidate fields, remove non-essential data |
|
||||||
|
| TKN-003 | lazy_loading | Pass paths instead of content, agents load when needed |
|
||||||
|
| TKN-004 | sliding_window | Add `.slice(-N)` after push/concat operations |
|
||||||
|
| TKN-005 | output_minimization | Use return values instead of file relay |
|
||||||
@@ -93,7 +93,7 @@ function selectNextAction(state) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// 4. Run diagnosis in order (only if not completed)
|
// 4. Run diagnosis in order (only if not completed)
|
||||||
const diagnosisOrder = ['context', 'memory', 'dataflow', 'agent', 'docs'];
|
const diagnosisOrder = ['context', 'memory', 'dataflow', 'agent', 'docs', 'token_consumption'];
|
||||||
|
|
||||||
for (const diagType of diagnosisOrder) {
|
for (const diagType of diagnosisOrder) {
|
||||||
if (state.diagnosis[diagType] === null) {
|
if (state.diagnosis[diagType] === null) {
|
||||||
@@ -221,6 +221,7 @@ async function runOrchestrator(workDir) {
|
|||||||
console.log(`[Loop ${iteration}] Executing: ${actionId}`);
|
console.log(`[Loop ${iteration}] Executing: ${actionId}`);
|
||||||
|
|
||||||
// 3. Update state: current action
|
// 3. Update state: current action
|
||||||
|
// FIX CTX-001: sliding window for action_history (keep last 10)
|
||||||
updateState({
|
updateState({
|
||||||
current_action: actionId,
|
current_action: actionId,
|
||||||
action_history: [...state.action_history, {
|
action_history: [...state.action_history, {
|
||||||
@@ -229,13 +230,24 @@ async function runOrchestrator(workDir) {
|
|||||||
completed_at: null,
|
completed_at: null,
|
||||||
result: null,
|
result: null,
|
||||||
output_files: []
|
output_files: []
|
||||||
}]
|
}].slice(-10) // Sliding window: prevent unbounded growth
|
||||||
});
|
});
|
||||||
|
|
||||||
// 4. Execute action
|
// 4. Execute action
|
||||||
try {
|
try {
|
||||||
const actionPrompt = Read(`phases/actions/${actionId}.md`);
|
const actionPrompt = Read(`phases/actions/${actionId}.md`);
|
||||||
const stateJson = JSON.stringify(state, null, 2);
|
// FIX CTX-003: Pass state path + key fields only instead of full state
|
||||||
|
const stateKeyInfo = {
|
||||||
|
status: state.status,
|
||||||
|
iteration_count: state.iteration_count,
|
||||||
|
issues_by_severity: state.issues_by_severity,
|
||||||
|
quality_gate: state.quality_gate,
|
||||||
|
current_action: state.current_action,
|
||||||
|
completed_actions: state.completed_actions,
|
||||||
|
user_issue_description: state.user_issue_description,
|
||||||
|
target_skill: { name: state.target_skill.name, path: state.target_skill.path }
|
||||||
|
};
|
||||||
|
const stateKeyJson = JSON.stringify(stateKeyInfo, null, 2);
|
||||||
|
|
||||||
const result = await Task({
|
const result = await Task({
|
||||||
subagent_type: 'universal-executor',
|
subagent_type: 'universal-executor',
|
||||||
@@ -245,8 +257,12 @@ async function runOrchestrator(workDir) {
|
|||||||
You are executing action "${actionId}" for skill-tuning workflow.
|
You are executing action "${actionId}" for skill-tuning workflow.
|
||||||
Work directory: ${workDir}
|
Work directory: ${workDir}
|
||||||
|
|
||||||
[STATE]
|
[STATE KEY INFO]
|
||||||
${stateJson}
|
${stateKeyJson}
|
||||||
|
|
||||||
|
[FULL STATE PATH]
|
||||||
|
${workDir}/state.json
|
||||||
|
(Read full state from this file if you need additional fields)
|
||||||
|
|
||||||
[ACTION INSTRUCTIONS]
|
[ACTION INSTRUCTIONS]
|
||||||
${actionPrompt}
|
${actionPrompt}
|
||||||
@@ -295,6 +311,7 @@ After completing the action:
|
|||||||
console.log(`[Loop ${iteration}] Error in ${actionId}: ${error.message}`);
|
console.log(`[Loop ${iteration}] Error in ${actionId}: ${error.message}`);
|
||||||
|
|
||||||
// Error handling
|
// Error handling
|
||||||
|
// FIX CTX-002: sliding window for errors (keep last 5)
|
||||||
updateState({
|
updateState({
|
||||||
current_action: null,
|
current_action: null,
|
||||||
errors: [...state.errors, {
|
errors: [...state.errors, {
|
||||||
@@ -302,7 +319,7 @@ After completing the action:
|
|||||||
message: error.message,
|
message: error.message,
|
||||||
timestamp: new Date().toISOString(),
|
timestamp: new Date().toISOString(),
|
||||||
recoverable: true
|
recoverable: true
|
||||||
}],
|
}].slice(-5), // Sliding window: prevent unbounded growth
|
||||||
error_count: state.error_count + 1
|
error_count: state.error_count + 1
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -31,6 +31,7 @@ interface TuningState {
|
|||||||
dataflow: DiagnosisResult | null;
|
dataflow: DiagnosisResult | null;
|
||||||
agent: DiagnosisResult | null;
|
agent: DiagnosisResult | null;
|
||||||
docs: DocsDiagnosisResult | null; // 文档结构诊断
|
docs: DocsDiagnosisResult | null; // 文档结构诊断
|
||||||
|
token_consumption: DiagnosisResult | null; // Token消耗诊断
|
||||||
};
|
};
|
||||||
|
|
||||||
// === Issues Found ===
|
// === Issues Found ===
|
||||||
@@ -69,6 +70,9 @@ interface TuningState {
|
|||||||
work_dir: string;
|
work_dir: string;
|
||||||
backup_dir: string;
|
backup_dir: string;
|
||||||
|
|
||||||
|
// === Final Report (consolidated output) ===
|
||||||
|
final_report: string | null; // Markdown summary generated on completion
|
||||||
|
|
||||||
// === Requirement Analysis (新增) ===
|
// === Requirement Analysis (新增) ===
|
||||||
requirement_analysis: RequirementAnalysis | null;
|
requirement_analysis: RequirementAnalysis | null;
|
||||||
}
|
}
|
||||||
@@ -176,7 +180,7 @@ interface Evidence {
|
|||||||
|
|
||||||
interface Issue {
|
interface Issue {
|
||||||
id: string; // e.g., "ISS-001"
|
id: string; // e.g., "ISS-001"
|
||||||
type: 'context_explosion' | 'memory_loss' | 'dataflow_break' | 'agent_failure';
|
type: 'context_explosion' | 'memory_loss' | 'dataflow_break' | 'agent_failure' | 'token_consumption';
|
||||||
severity: 'critical' | 'high' | 'medium' | 'low';
|
severity: 'critical' | 'high' | 'medium' | 'low';
|
||||||
priority: number; // 1 = highest
|
priority: number; // 1 = highest
|
||||||
location: {
|
location: {
|
||||||
@@ -214,6 +218,10 @@ type FixStrategy =
|
|||||||
| 'schema_enforcement' // Add data contract validation
|
| 'schema_enforcement' // Add data contract validation
|
||||||
| 'orchestrator_refactor' // Refactor agent coordination
|
| 'orchestrator_refactor' // Refactor agent coordination
|
||||||
| 'state_centralization' // Centralize state management
|
| 'state_centralization' // Centralize state management
|
||||||
|
| 'prompt_compression' // Extract static text, use templates
|
||||||
|
| 'lazy_loading' // Pass paths instead of content
|
||||||
|
| 'output_minimization' // Return minimal structured JSON
|
||||||
|
| 'state_field_reduction' // Audit and consolidate state fields
|
||||||
| 'custom'; // Custom fix
|
| 'custom'; // Custom fix
|
||||||
|
|
||||||
interface FileChange {
|
interface FileChange {
|
||||||
@@ -270,7 +278,8 @@ interface ErrorEntry {
|
|||||||
"memory": null,
|
"memory": null,
|
||||||
"dataflow": null,
|
"dataflow": null,
|
||||||
"agent": null,
|
"agent": null,
|
||||||
"docs": null
|
"docs": null,
|
||||||
|
"token_consumption": null
|
||||||
},
|
},
|
||||||
"issues": [],
|
"issues": [],
|
||||||
"issues_by_severity": {
|
"issues_by_severity": {
|
||||||
@@ -294,6 +303,7 @@ interface ErrorEntry {
|
|||||||
"max_errors": 3,
|
"max_errors": 3,
|
||||||
"work_dir": null,
|
"work_dir": null,
|
||||||
"backup_dir": null,
|
"backup_dir": null,
|
||||||
|
"final_report": null,
|
||||||
"requirement_analysis": null
|
"requirement_analysis": null
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -93,6 +93,14 @@
|
|||||||
"detection_focus": null,
|
"detection_focus": null,
|
||||||
"needs_gemini_analysis": true,
|
"needs_gemini_analysis": true,
|
||||||
"priority_order": [1, 2, 3, 4]
|
"priority_order": [1, 2, 3, 4]
|
||||||
|
},
|
||||||
|
"token_consumption": {
|
||||||
|
"pattern_ids": ["TKN-001", "TKN-002", "TKN-003", "TKN-004", "TKN-005"],
|
||||||
|
"severity_hint": "medium",
|
||||||
|
"strategies": ["prompt_compression", "lazy_loading", "output_minimization", "state_field_reduction", "sliding_window"],
|
||||||
|
"risk_levels": ["low", "low", "low", "low", "low"],
|
||||||
|
"detection_focus": "Verbose prompts, excessive state fields, full content passing, unbounded arrays, redundant I/O",
|
||||||
|
"priority_order": [1, 2, 3, 4, 5]
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"keywords": {
|
"keywords": {
|
||||||
@@ -149,7 +157,12 @@
|
|||||||
"中间文件": "authoring_principles_violation",
|
"中间文件": "authoring_principles_violation",
|
||||||
"临时文件": "authoring_principles_violation",
|
"临时文件": "authoring_principles_violation",
|
||||||
"文件中转": "authoring_principles_violation",
|
"文件中转": "authoring_principles_violation",
|
||||||
"state膨胀": "authoring_principles_violation"
|
"state膨胀": "authoring_principles_violation",
|
||||||
|
"token消耗": "token_consumption",
|
||||||
|
"token优化": "token_consumption",
|
||||||
|
"产出简化": "token_consumption",
|
||||||
|
"冗长": "token_consumption",
|
||||||
|
"精简": "token_consumption"
|
||||||
},
|
},
|
||||||
"english": {
|
"english": {
|
||||||
"token": "context_explosion",
|
"token": "context_explosion",
|
||||||
@@ -198,7 +211,12 @@
|
|||||||
"feedback": "user_experience",
|
"feedback": "user_experience",
|
||||||
"intermediate": "authoring_principles_violation",
|
"intermediate": "authoring_principles_violation",
|
||||||
"temp": "authoring_principles_violation",
|
"temp": "authoring_principles_violation",
|
||||||
"relay": "authoring_principles_violation"
|
"relay": "authoring_principles_violation",
|
||||||
|
"verbose": "token_consumption",
|
||||||
|
"minimize": "token_consumption",
|
||||||
|
"compress": "token_consumption",
|
||||||
|
"simplify": "token_consumption",
|
||||||
|
"reduction": "token_consumption"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"category_labels": {
|
"category_labels": {
|
||||||
@@ -213,6 +231,7 @@
|
|||||||
"output_quality": "Output Quality",
|
"output_quality": "Output Quality",
|
||||||
"user_experience": "User Experience",
|
"user_experience": "User Experience",
|
||||||
"authoring_principles_violation": "Authoring Principles Violation",
|
"authoring_principles_violation": "Authoring Principles Violation",
|
||||||
|
"token_consumption": "Token Consumption",
|
||||||
"custom": "Custom"
|
"custom": "Custom"
|
||||||
},
|
},
|
||||||
"category_labels_chinese": {
|
"category_labels_chinese": {
|
||||||
@@ -227,6 +246,7 @@
|
|||||||
"output_quality": "Output Quality",
|
"output_quality": "Output Quality",
|
||||||
"user_experience": "User Experience",
|
"user_experience": "User Experience",
|
||||||
"authoring_principles_violation": "Authoring Principles Violation",
|
"authoring_principles_violation": "Authoring Principles Violation",
|
||||||
|
"token_consumption": "Token Consumption Optimization",
|
||||||
"custom": "Other Issues"
|
"custom": "Other Issues"
|
||||||
},
|
},
|
||||||
"category_descriptions": {
|
"category_descriptions": {
|
||||||
@@ -241,12 +261,13 @@
|
|||||||
"output_quality": "Output validation or completeness issues",
|
"output_quality": "Output validation or completeness issues",
|
||||||
"user_experience": "Interaction or feedback clarity issues",
|
"user_experience": "Interaction or feedback clarity issues",
|
||||||
"authoring_principles_violation": "Violation of skill authoring principles",
|
"authoring_principles_violation": "Violation of skill authoring principles",
|
||||||
|
"token_consumption": "Excessive token usage from verbose prompts, large state objects, or redundant I/O patterns",
|
||||||
"custom": "Requires custom analysis"
|
"custom": "Requires custom analysis"
|
||||||
},
|
},
|
||||||
"fix_priority_order": {
|
"fix_priority_order": {
|
||||||
"P0": ["dataflow_break", "authoring_principles_violation"],
|
"P0": ["dataflow_break", "authoring_principles_violation"],
|
||||||
"P1": ["agent_failure"],
|
"P1": ["agent_failure"],
|
||||||
"P2": ["context_explosion"],
|
"P2": ["context_explosion", "token_consumption"],
|
||||||
"P3": ["memory_loss"]
|
"P3": ["memory_loss"]
|
||||||
},
|
},
|
||||||
"cross_category_dependencies": {
|
"cross_category_dependencies": {
|
||||||
|
|||||||
@@ -180,7 +180,35 @@ Classification of skill execution issues with detection patterns and severity cr
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 6. Documentation Conflict (P6)
|
### 6. Token Consumption (P6)
|
||||||
|
|
||||||
|
**Definition**: Excessive token usage from verbose prompts, large state objects, or inefficient I/O patterns.
|
||||||
|
|
||||||
|
**Root Causes**:
|
||||||
|
- Long static prompts without compression
|
||||||
|
- State schema with too many fields
|
||||||
|
- Full content embedding instead of path references
|
||||||
|
- Arrays growing unbounded without sliding windows
|
||||||
|
- Write-then-read file relay patterns
|
||||||
|
|
||||||
|
**Detection Patterns**:
|
||||||
|
|
||||||
|
| Pattern ID | Regex/Check | Description |
|
||||||
|
|------------|-------------|-------------|
|
||||||
|
| TKN-001 | File size > 4KB | Verbose prompt files |
|
||||||
|
| TKN-002 | State fields > 15 | Excessive state schema |
|
||||||
|
| TKN-003 | `/Read\([^)]+\)\s*[\+,]/` | Full content passing |
|
||||||
|
| TKN-004 | `/.push\|concat(?!.*\.slice)/` | Unbounded array growth |
|
||||||
|
| TKN-005 | `/Write\([^)]+\)[\s\S]{0,100}Read\([^)]+\)/` | Write-then-read pattern |
|
||||||
|
|
||||||
|
**Impact Levels**:
|
||||||
|
- **High**: Multiple TKN-003/TKN-004 issues causing significant token waste
|
||||||
|
- **Medium**: Several verbose files or state bloat
|
||||||
|
- **Low**: Minor optimization opportunities
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 7. Documentation Conflict (P7)
|
||||||
|
|
||||||
**Definition**: 同一概念在不同文件中定义不一致,导致行为不可预测和文档误导。
|
**Definition**: 同一概念在不同文件中定义不一致,导致行为不可预测和文档误导。
|
||||||
|
|
||||||
@@ -262,6 +290,7 @@ function calculateIssueSeverity(issue) {
|
|||||||
| Long-tail Forgetting | constraint_injection, state_constraints_field, checkpoint | 1, 2, 3 |
|
| Long-tail Forgetting | constraint_injection, state_constraints_field, checkpoint | 1, 2, 3 |
|
||||||
| Data Flow Disruption | state_centralization, schema_enforcement, field_normalization | 1, 2, 3 |
|
| Data Flow Disruption | state_centralization, schema_enforcement, field_normalization | 1, 2, 3 |
|
||||||
| Agent Coordination | error_wrapping, result_validation, flatten_nesting | 1, 2, 3 |
|
| Agent Coordination | error_wrapping, result_validation, flatten_nesting | 1, 2, 3 |
|
||||||
|
| **Token Consumption** | prompt_compression, lazy_loading, output_minimization, state_field_reduction | 1, 2, 3, 4 |
|
||||||
| **Documentation Redundancy** | consolidate_to_ssot, centralize_mapping_config | 1, 2 |
|
| **Documentation Redundancy** | consolidate_to_ssot, centralize_mapping_config | 1, 2 |
|
||||||
| **Documentation Conflict** | reconcile_conflicting_definitions | 1 |
|
| **Documentation Conflict** | reconcile_conflicting_definitions | 1 |
|
||||||
|
|
||||||
|
|||||||
@@ -1308,3 +1308,230 @@ async function checkpoint(name, summary, options) {
|
|||||||
return response;
|
return response;
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Token Consumption Strategies
|
||||||
|
|
||||||
|
Strategies for reducing token usage and simplifying skill outputs.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Strategy: prompt_compression
|
||||||
|
|
||||||
|
**Purpose**: Reduce verbose prompts by extracting static text and using templates.
|
||||||
|
|
||||||
|
**Implementation**:
|
||||||
|
```javascript
|
||||||
|
// Before: Long inline prompt
|
||||||
|
const prompt = `
|
||||||
|
You are an expert code analyzer specializing in identifying patterns.
|
||||||
|
Your role is to examine the provided code and identify any issues.
|
||||||
|
Please follow these detailed instructions carefully:
|
||||||
|
1. Read the code thoroughly
|
||||||
|
2. Identify any anti-patterns
|
||||||
|
3. Check for security vulnerabilities
|
||||||
|
... (continues for many lines)
|
||||||
|
|
||||||
|
Code to analyze:
|
||||||
|
${fullCodeContent}
|
||||||
|
`;
|
||||||
|
|
||||||
|
// After: Compressed with template reference
|
||||||
|
const PROMPT_TEMPLATE_PATH = 'templates/analyzer-prompt.md';
|
||||||
|
|
||||||
|
const prompt = `
|
||||||
|
[TEMPLATE: ${PROMPT_TEMPLATE_PATH}]
|
||||||
|
[CODE_PATH: ${codePath}]
|
||||||
|
[FOCUS: patterns, security]
|
||||||
|
`;
|
||||||
|
|
||||||
|
// Or use key instructions only
|
||||||
|
const prompt = `
|
||||||
|
Analyze ${codePath} for: patterns, security, performance.
|
||||||
|
Return JSON: { issues: [], severity: string }
|
||||||
|
`;
|
||||||
|
```
|
||||||
|
|
||||||
|
**Risk**: Low
|
||||||
|
**Verification**: Compare token count before/after compression
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Strategy: lazy_loading
|
||||||
|
|
||||||
|
**Purpose**: Pass file paths instead of full content, let consumers load if needed.
|
||||||
|
|
||||||
|
**Implementation**:
|
||||||
|
```javascript
|
||||||
|
// Before: Full content in prompt
|
||||||
|
const content = Read(filePath);
|
||||||
|
const prompt = `Analyze this content:\n${content}`;
|
||||||
|
|
||||||
|
// After: Path reference with lazy loading instruction
|
||||||
|
const prompt = `
|
||||||
|
Analyze file at: ${filePath}
|
||||||
|
(Read the file content if you need to examine it)
|
||||||
|
Return: { summary: string, issues: [] }
|
||||||
|
`;
|
||||||
|
|
||||||
|
// For agent calls
|
||||||
|
const result = await Task({
|
||||||
|
subagent_type: 'universal-executor',
|
||||||
|
prompt: `
|
||||||
|
[FILE_PATH]: ${dataPath}
|
||||||
|
[TASK]: Analyze the file and extract key metrics.
|
||||||
|
[NOTE]: Read the file only if needed for your analysis.
|
||||||
|
`
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Risk**: Low
|
||||||
|
**Verification**: Verify agents can still access required data
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Strategy: output_minimization
|
||||||
|
|
||||||
|
**Purpose**: Configure agents to return minimal, structured output instead of verbose text.
|
||||||
|
|
||||||
|
**Implementation**:
|
||||||
|
```javascript
|
||||||
|
// Before: Verbose output expectation
|
||||||
|
const prompt = `
|
||||||
|
Analyze the code and provide a detailed report including:
|
||||||
|
- Executive summary
|
||||||
|
- Detailed findings with explanations
|
||||||
|
- Code examples for each issue
|
||||||
|
- Recommendations with rationale
|
||||||
|
...
|
||||||
|
`;
|
||||||
|
|
||||||
|
// After: Minimal structured output
|
||||||
|
const prompt = `
|
||||||
|
Analyze the code. Return ONLY this JSON:
|
||||||
|
{
|
||||||
|
"status": "pass|review|fail",
|
||||||
|
"issues": [{ "id": string, "severity": string, "file": string, "line": number }],
|
||||||
|
"summary": "one sentence"
|
||||||
|
}
|
||||||
|
Do not include explanations or code examples.
|
||||||
|
`;
|
||||||
|
|
||||||
|
// Validation
|
||||||
|
const result = JSON.parse(agentOutput);
|
||||||
|
if (!result.status || !Array.isArray(result.issues)) {
|
||||||
|
throw new Error('Invalid output format');
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Risk**: Low
|
||||||
|
**Verification**: JSON.parse succeeds, output size reduced
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Strategy: state_field_reduction
|
||||||
|
|
||||||
|
**Purpose**: Audit and consolidate state fields to minimize serialization overhead.
|
||||||
|
|
||||||
|
**Implementation**:
|
||||||
|
```typescript
|
||||||
|
// Before: Bloated state
|
||||||
|
interface State {
|
||||||
|
status: string;
|
||||||
|
target: TargetInfo;
|
||||||
|
user_input: string;
|
||||||
|
parsed_input: ParsedInput; // Remove - temporary
|
||||||
|
intermediate_result: any; // Remove - not persisted
|
||||||
|
debug_info: DebugInfo; // Remove - debugging only
|
||||||
|
analysis_cache: any; // Remove - session cache
|
||||||
|
full_history: HistoryEntry[]; // Remove - unbounded
|
||||||
|
step1_output: any; // Remove - intermediate
|
||||||
|
step2_output: any; // Remove - intermediate
|
||||||
|
step3_output: any; // Remove - intermediate
|
||||||
|
final_result: FinalResult;
|
||||||
|
error_log: string[]; // Remove - debugging
|
||||||
|
metrics: Metrics; // Remove - optional
|
||||||
|
}
|
||||||
|
|
||||||
|
// After: Minimal state (≤15 fields)
|
||||||
|
interface State {
|
||||||
|
status: 'pending' | 'running' | 'completed' | 'failed';
|
||||||
|
target: { name: string; path: string };
|
||||||
|
input_summary: string; // Summarized user input
|
||||||
|
result_path: string; // Path to final result
|
||||||
|
quality_gate: 'pass' | 'fail';
|
||||||
|
error?: string; // Only if failed
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Audit Checklist**:
|
||||||
|
```javascript
|
||||||
|
function auditStateFields(stateSchema) {
|
||||||
|
const removeCandidates = [];
|
||||||
|
|
||||||
|
for (const [key, type] of Object.entries(stateSchema)) {
|
||||||
|
// Identify removal candidates
|
||||||
|
if (key.startsWith('debug_')) removeCandidates.push(key);
|
||||||
|
if (key.endsWith('_cache')) removeCandidates.push(key);
|
||||||
|
if (key.endsWith('_temp')) removeCandidates.push(key);
|
||||||
|
if (key.includes('intermediate')) removeCandidates.push(key);
|
||||||
|
if (key.includes('step') && key.includes('output')) removeCandidates.push(key);
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
total_fields: Object.keys(stateSchema).length,
|
||||||
|
remove_candidates: removeCandidates,
|
||||||
|
estimated_reduction: removeCandidates.length
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Risk**: Medium (ensure no essential data removed)
|
||||||
|
**Verification**: State field count ≤ 15, all essential data preserved
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Strategy: in_memory_consolidation
|
||||||
|
|
||||||
|
**Purpose**: Consolidate outputs into single file, eliminate redundant report files.
|
||||||
|
|
||||||
|
**Implementation**:
|
||||||
|
```javascript
|
||||||
|
// Before: Multiple output files
|
||||||
|
Write(`${workDir}/diagnosis-report.md`, reportMarkdown);
|
||||||
|
Write(`${workDir}/diagnosis-summary.json`, summaryJson);
|
||||||
|
Write(`${workDir}/state.json`, JSON.stringify(state));
|
||||||
|
Write(`${workDir}/tuning-report.md`, tuningReport);
|
||||||
|
Write(`${workDir}/tuning-summary.md`, finalSummary);
|
||||||
|
|
||||||
|
// After: Single source of truth
|
||||||
|
const consolidatedState = {
|
||||||
|
...state,
|
||||||
|
final_report: {
|
||||||
|
summary: summaryJson,
|
||||||
|
details_available_in_state: true,
|
||||||
|
generated_at: new Date().toISOString()
|
||||||
|
}
|
||||||
|
};
|
||||||
|
Write(`${workDir}/state.json`, JSON.stringify(consolidatedState, null, 2));
|
||||||
|
|
||||||
|
// Report can be rendered from state on-demand
|
||||||
|
function renderReport(state) {
|
||||||
|
return `
|
||||||
|
# Tuning Report: ${state.target_skill.name}
|
||||||
|
Status: ${state.status}
|
||||||
|
Quality: ${state.quality_gate}
|
||||||
|
Issues: ${state.issues.length}
|
||||||
|
...
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Benefits**:
|
||||||
|
- Single file to read/write
|
||||||
|
- No data duplication
|
||||||
|
- On-demand rendering
|
||||||
|
|
||||||
|
**Risk**: Low
|
||||||
|
**Verification**: Only state.json exists as output, rendering works correctly
|
||||||
|
|||||||
Reference in New Issue
Block a user