feat: Add code analysis and LLM action templates with detailed configurations and examples

- Introduced a comprehensive code analysis action template for integrating code exploration and analysis capabilities.
- Added LLM action template for seamless integration of LLM calls with customizable prompts and tools.
- Implemented a benchmark search script to compare multiple search methods across various dimensions including speed, result quality, ranking stability, and coverage.
- Provided preset configurations for common analysis tasks and LLM actions, enhancing usability and flexibility.
This commit is contained in:
catlog22
2026-01-03 17:37:49 +08:00
parent 6a45035e3f
commit be498acf59
10 changed files with 3076 additions and 217 deletions

View File

@@ -0,0 +1,503 @@
# Code Analysis Action Template
代码分析动作模板,用于在 Skill 中集成代码探索和分析能力。
---
## 配置结构
```typescript
interface CodeAnalysisActionConfig {
id: string; // "analyze-structure", "explore-patterns"
name: string; // "Code Structure Analysis"
type: 'code-analysis'; // 动作类型标识
// 分析范围
scope: {
paths: string[]; // 目标路径
patterns: string[]; // Glob 模式
excludes?: string[]; // 排除模式
};
// 分析类型
analysis_type: 'structure' | 'patterns' | 'dependencies' | 'quality' | 'security';
// Agent 配置
agent: {
type: 'Explore' | 'cli-explore-agent' | 'universal-executor';
thoroughness: 'quick' | 'medium' | 'very thorough';
};
// 输出配置
output: {
format: 'json' | 'markdown';
file: string;
};
// MCP 工具增强
mcp_tools?: string[]; // ['mcp__ace-tool__search_context']
}
```
---
## 模板生成函数
```javascript
function generateCodeAnalysisAction(config) {
const { id, name, scope, analysis_type, agent, output, mcp_tools = [] } = config;
return `
# ${name}
## Action: ${id}
### 分析范围
- **路径**: ${scope.paths.join(', ')}
- **模式**: ${scope.patterns.join(', ')}
${scope.excludes ? `- **排除**: ${scope.excludes.join(', ')}` : ''}
### 执行逻辑
\`\`\`javascript
async function execute${toPascalCase(id)}(context) {
const workDir = context.workDir;
const results = [];
// 1. 文件发现
const files = await discoverFiles({
paths: ${JSON.stringify(scope.paths)},
patterns: ${JSON.stringify(scope.patterns)},
excludes: ${JSON.stringify(scope.excludes || [])}
});
console.log(\`Found \${files.length} files to analyze\`);
// 2. 使用 MCP 工具进行语义搜索(如果配置)
${mcp_tools.length > 0 ? `
const semanticResults = await mcp__ace_tool__search_context({
project_root_path: context.projectRoot,
query: '${getQueryForAnalysisType(analysis_type)}'
});
results.push({ type: 'semantic', data: semanticResults });
` : '// No MCP tools configured'}
// 3. 启动 Agent 进行深度分析
const agentResult = await Task({
subagent_type: '${agent.type}',
prompt: \`
${generateAgentPrompt(analysis_type, scope)}
\`,
run_in_background: false
});
results.push({ type: 'agent', data: agentResult });
// 4. 汇总结果
const summary = aggregateResults(results);
// 5. 输出结果
const outputPath = \`\${workDir}/${output.file}\`;
${output.format === 'json'
? `Write(outputPath, JSON.stringify(summary, null, 2));`
: `Write(outputPath, formatAsMarkdown(summary));`}
return {
success: true,
output: '${output.file}',
files_analyzed: files.length,
analysis_type: '${analysis_type}'
};
}
\`\`\`
`;
}
function getQueryForAnalysisType(type) {
const queries = {
structure: 'main entry points, module organization, exports',
patterns: 'design patterns, abstractions, reusable components',
dependencies: 'imports, external dependencies, coupling',
quality: 'code complexity, test coverage, documentation',
security: 'authentication, authorization, input validation, secrets'
};
return queries[type] || queries.structure;
}
function generateAgentPrompt(type, scope) {
const prompts = {
structure: `分析以下路径的代码结构:
${scope.paths.map(p => `- ${p}`).join('\\n')}
任务:
1. 识别主要模块和入口点
2. 分析目录组织结构
3. 提取模块间的导入导出关系
4. 生成结构概览图 (Mermaid)
输出格式: JSON
{
"modules": [...],
"entry_points": [...],
"structure_diagram": "mermaid code"
}`,
patterns: `分析以下路径的设计模式:
${scope.paths.map(p => `- ${p}`).join('\\n')}
任务:
1. 识别使用的设计模式 (Factory, Strategy, Observer 等)
2. 分析抽象层级
3. 评估模式使用的恰当性
4. 提取可复用的模式实例
输出格式: JSON
{
"patterns": [{ "name": "...", "location": "...", "usage": "..." }],
"abstractions": [...],
"reusable_components": [...]
}`,
dependencies: `分析以下路径的依赖关系:
${scope.paths.map(p => `- ${p}`).join('\\n')}
任务:
1. 提取内部模块依赖
2. 识别外部包依赖
3. 分析耦合度
4. 检测循环依赖
输出格式: JSON
{
"internal_deps": [...],
"external_deps": [...],
"coupling_score": 0-100,
"circular_deps": [...]
}`,
quality: `分析以下路径的代码质量:
${scope.paths.map(p => `- ${p}`).join('\\n')}
任务:
1. 评估代码复杂度
2. 检查测试覆盖率
3. 分析文档完整性
4. 识别技术债务
输出格式: JSON
{
"complexity": { "avg": 0, "max": 0, "hotspots": [...] },
"test_coverage": { "percentage": 0, "gaps": [...] },
"documentation": { "score": 0, "missing": [...] },
"tech_debt": [...]
}`,
security: `分析以下路径的安全性:
${scope.paths.map(p => `- ${p}`).join('\\n')}
任务:
1. 检查认证授权实现
2. 分析输入验证
3. 检测敏感数据处理
4. 识别常见漏洞模式
输出格式: JSON
{
"auth": { "methods": [...], "issues": [...] },
"input_validation": { "coverage": 0, "gaps": [...] },
"sensitive_data": { "found": [...], "protected": true/false },
"vulnerabilities": [{ "type": "...", "severity": "...", "location": "..." }]
}`
};
return prompts[type] || prompts.structure;
}
```
---
## 预置代码分析动作
### 1. 项目结构分析
```yaml
id: analyze-project-structure
name: Project Structure Analysis
type: code-analysis
scope:
paths:
- src/
patterns:
- "**/*.ts"
- "**/*.js"
excludes:
- "**/node_modules/**"
- "**/*.test.*"
analysis_type: structure
agent:
type: Explore
thoroughness: medium
output:
format: json
file: structure-analysis.json
mcp_tools:
- mcp__ace-tool__search_context
```
### 2. 设计模式提取
```yaml
id: extract-design-patterns
name: Design Pattern Extraction
type: code-analysis
scope:
paths:
- src/
patterns:
- "**/*.ts"
analysis_type: patterns
agent:
type: cli-explore-agent
thoroughness: very thorough
output:
format: markdown
file: patterns-report.md
```
### 3. 依赖关系分析
```yaml
id: analyze-dependencies
name: Dependency Analysis
type: code-analysis
scope:
paths:
- src/
- packages/
patterns:
- "**/package.json"
- "**/*.ts"
analysis_type: dependencies
agent:
type: Explore
thoroughness: medium
output:
format: json
file: dependency-graph.json
```
### 4. 安全审计
```yaml
id: security-audit
name: Security Audit
type: code-analysis
scope:
paths:
- src/auth/
- src/api/
patterns:
- "**/*.ts"
analysis_type: security
agent:
type: universal-executor
thoroughness: very thorough
output:
format: json
file: security-report.json
mcp_tools:
- mcp__ace-tool__search_context
```
---
## 使用示例
### 在 Phase 中使用
```javascript
// phases/01-code-exploration.md
const analysisConfig = {
id: 'explore-skill-structure',
name: 'Skill Structure Exploration',
type: 'code-analysis',
scope: {
paths: ['D:\\Claude_dms3\\.claude\\skills\\software-manual'],
patterns: ['**/*.md'],
excludes: ['**/node_modules/**']
},
analysis_type: 'structure',
agent: {
type: 'Explore',
thoroughness: 'medium'
},
output: {
format: 'json',
file: 'skill-structure.json'
}
};
// 执行
const result = await executeCodeAnalysis(analysisConfig, context);
```
### 组合多种分析
```javascript
// 串行执行多种分析
const analyses = [
{ type: 'structure', file: 'structure.json' },
{ type: 'patterns', file: 'patterns.json' },
{ type: 'dependencies', file: 'deps.json' }
];
for (const analysis of analyses) {
await executeCodeAnalysis({
...baseConfig,
analysis_type: analysis.type,
output: { format: 'json', file: analysis.file }
}, context);
}
// 并行执行(独立分析)
const parallelResults = await Promise.all(
analyses.map(a => executeCodeAnalysis({
...baseConfig,
analysis_type: a.type,
output: { format: 'json', file: a.file }
}, context))
);
```
---
## Agent 选择指南
| 分析类型 | 推荐 Agent | Thoroughness | 原因 |
|---------|-----------|--------------|------|
| structure | Explore | medium | 快速获取目录结构 |
| patterns | cli-explore-agent | very thorough | 需要深度代码理解 |
| dependencies | Explore | medium | 主要分析 import 语句 |
| quality | universal-executor | medium | 需要运行分析工具 |
| security | universal-executor | very thorough | 需要全面扫描 |
---
## MCP 工具集成
### 语义搜索增强
```javascript
// 使用 ACE 工具进行语义搜索
const semanticContext = await mcp__ace_tool__search_context({
project_root_path: projectRoot,
query: 'authentication logic, user session management'
});
// 将语义搜索结果作为 Agent 的输入上下文
const agentResult = await Task({
subagent_type: 'Explore',
prompt: `
基于以下语义搜索结果进行深度分析:
${semanticContext}
任务: 分析认证逻辑的实现细节...
`,
run_in_background: false
});
```
### smart_search 集成
```javascript
// 使用 smart_search 进行精确搜索
const exactMatches = await mcp__ccw_tools__smart_search({
action: 'search',
query: 'class.*Controller',
mode: 'ripgrep',
path: 'src/'
});
// 使用 find_files 发现文件
const configFiles = await mcp__ccw_tools__smart_search({
action: 'find_files',
pattern: '**/*.config.ts',
path: 'src/'
});
```
---
## 结果聚合
```javascript
function aggregateResults(results) {
const aggregated = {
timestamp: new Date().toISOString(),
sources: [],
summary: {},
details: []
};
for (const result of results) {
aggregated.sources.push(result.type);
if (result.type === 'semantic') {
aggregated.summary.semantic_matches = result.data.length;
aggregated.details.push({
source: 'semantic',
data: result.data.slice(0, 10) // Top 10
});
}
if (result.type === 'agent') {
aggregated.summary.agent_findings = extractKeyFindings(result.data);
aggregated.details.push({
source: 'agent',
data: result.data
});
}
}
return aggregated;
}
function extractKeyFindings(agentResult) {
// 从 Agent 结果中提取关键发现
// 实现取决于 Agent 的输出格式
return {
modules: agentResult.modules?.length || 0,
patterns: agentResult.patterns?.length || 0,
issues: agentResult.issues?.length || 0
};
}
```
---
## 最佳实践
1. **范围控制**
- 使用精确的 patterns 减少分析范围
- 配置 excludes 排除无关文件
2. **Agent 选择**
- 快速探索用 Explore
- 深度分析用 cli-explore-agent
- 需要执行操作用 universal-executor
3. **MCP 工具组合**
- 先用 mcp__ace-tool__search_context 获取语义上下文
- 再用 Agent 进行深度分析
- 最后用 smart_search 补充精确匹配
4. **结果缓存**
- 将分析结果持久化到 workDir
- 后续阶段可直接读取,避免重复分析
5. **Brief Returns**
- Agent 返回路径 + 摘要,而非完整内容
- 避免上下文溢出

View File

@@ -0,0 +1,355 @@
# LLM Action Template
LLM 动作模板,用于在 Skill 中集成 LLM 调用能力。
---
## 配置结构
```typescript
interface LLMActionConfig {
id: string; // "llm-analyze", "llm-generate"
name: string; // "LLM Analysis"
type: 'llm'; // 动作类型标识
// LLM 工具配置
tool: {
primary: 'gemini' | 'qwen' | 'codex';
fallback_chain: string[]; // ['gemini', 'qwen', 'codex']
};
// 执行模式
mode: 'analysis' | 'write';
// 提示词配置
prompt: {
template: string; // 提示词模板路径或内联
variables: string[]; // 需要替换的变量
};
// 输入输出
input: string[]; // 依赖的上下文文件
output: string; // 输出文件路径
// 超时配置
timeout?: number; // 毫秒,默认 600000 (10min)
}
```
---
## 模板生成函数
```javascript
function generateLLMAction(config) {
const { id, name, tool, mode, prompt, input, output, timeout = 600000 } = config;
return `
# ${name}
## Action: ${id}
### 执行逻辑
\`\`\`javascript
async function execute${toPascalCase(id)}(context) {
const workDir = context.workDir;
const state = context.state;
// 1. 收集输入上下文
const inputContext = ${JSON.stringify(input)}.map(f => {
const path = \`\${workDir}/\${f}\`;
return Read(path);
}).join('\\n\\n---\\n\\n');
// 2. 构建提示词
const promptTemplate = \`${prompt.template}\`;
const finalPrompt = promptTemplate
${prompt.variables.map(v => `.replace('{{${v}}}', context.${v} || '')`).join('\n ')};
// 3. 执行 LLM 调用 (带 fallback)
const tools = ['${tool.primary}', ${tool.fallback_chain.map(t => `'${t}'`).join(', ')}];
let result = null;
let usedTool = null;
for (const t of tools) {
try {
result = await callLLM(t, finalPrompt, '${mode}', ${timeout});
usedTool = t;
break;
} catch (error) {
console.log(\`\${t} failed: \${error.message}, trying next...\`);
}
}
if (!result) {
throw new Error('All LLM tools failed');
}
// 4. 保存结果
Write(\`\${workDir}/${output}\`, result);
// 5. 更新状态
state.llm_calls = (state.llm_calls || 0) + 1;
state.last_llm_tool = usedTool;
return {
success: true,
output: '${output}',
tool_used: usedTool
};
}
// LLM 调用封装
async function callLLM(tool, prompt, mode, timeout) {
const modeFlag = mode === 'write' ? '--mode write' : '--mode analysis';
// 使用 CCW CLI 统一接口
const command = \`ccw cli -p "\${escapePrompt(prompt)}" --tool \${tool} \${modeFlag}\`;
const result = Bash({
command,
timeout,
run_in_background: true // 异步执行
});
// 等待完成
return await waitForResult(result.task_id, timeout);
}
function escapePrompt(prompt) {
// 转义双引号和特殊字符
return prompt.replace(/"/g, '\\\\"').replace(/\$/g, '\\\\$');
}
\`\`\`
### Prompt 模板
\`\`\`
${prompt.template}
\`\`\`
### 变量说明
${prompt.variables.map(v => `- \`{{${v}}}\`: ${v} 变量`).join('\n')}
`;
}
function toPascalCase(str) {
return str.split('-').map(s => s.charAt(0).toUpperCase() + s.slice(1)).join('');
}
```
---
## 预置 LLM 动作模板
### 1. 代码分析动作
```yaml
id: llm-code-analysis
name: LLM Code Analysis
type: llm
tool:
primary: gemini
fallback_chain: [qwen]
mode: analysis
prompt:
template: |
PURPOSE: 分析代码结构和模式,提取关键设计特征
TASK:
• 识别主要模块和组件
• 分析依赖关系
• 提取设计模式
• 评估代码质量
MODE: analysis
CONTEXT: {{code_context}}
EXPECTED: JSON 格式的分析报告,包含 modules, dependencies, patterns, quality_score
RULES: $(cat ~/.claude/workflows/cli-templates/protocols/analysis-protocol.md)
variables:
- code_context
input:
- collected-code.md
output: analysis-report.json
timeout: 900000
```
### 2. 文档生成动作
```yaml
id: llm-doc-generation
name: LLM Documentation Generation
type: llm
tool:
primary: gemini
fallback_chain: [qwen, codex]
mode: write
prompt:
template: |
PURPOSE: 根据分析结果生成高质量文档
TASK:
• 基于分析报告生成文档大纲
• 填充各章节内容
• 添加代码示例和说明
• 生成 Mermaid 图表
MODE: write
CONTEXT: {{analysis_report}}
EXPECTED: 完整的 Markdown 文档,包含目录、章节、图表
RULES: $(cat ~/.claude/workflows/cli-templates/protocols/write-protocol.md)
variables:
- analysis_report
input:
- analysis-report.json
output: generated-doc.md
timeout: 1200000
```
### 3. 代码重构建议动作
```yaml
id: llm-refactor-suggest
name: LLM Refactoring Suggestions
type: llm
tool:
primary: codex
fallback_chain: [gemini]
mode: analysis
prompt:
template: |
PURPOSE: 分析代码并提供重构建议
TASK:
• 识别代码异味 (code smells)
• 评估复杂度热点
• 提出具体重构方案
• 估算重构影响范围
MODE: analysis
CONTEXT: {{source_code}}
EXPECTED: 重构建议列表,每项包含 location, issue, suggestion, impact
RULES: $(cat ~/.claude/workflows/cli-templates/protocols/analysis-protocol.md)
variables:
- source_code
input:
- source-files.md
output: refactor-suggestions.json
timeout: 600000
```
---
## 使用示例
### 在 Phase 中使用 LLM 动作
```javascript
// phases/02-llm-analysis.md
const llmConfig = {
id: 'llm-analyze-skill',
name: 'Skill Pattern Analysis',
type: 'llm',
tool: {
primary: 'gemini',
fallback_chain: ['qwen']
},
mode: 'analysis',
prompt: {
template: `
PURPOSE: 分析现有 Skill 的设计模式
TASK:
• 提取 Skill 结构规范
• 识别 Phase 组织模式
• 分析 Agent 调用模式
MODE: analysis
CONTEXT: {{skill_source}}
EXPECTED: 结构化的设计模式分析
`,
variables: ['skill_source']
},
input: ['collected-skills.md'],
output: 'skill-patterns.json'
};
// 执行
const result = await executeLLMAction(llmConfig, {
workDir: '.workflow/.scratchpad/skill-gen-xxx',
skill_source: Read('.workflow/.scratchpad/skill-gen-xxx/collected-skills.md')
});
```
### 在 Orchestrator 中调度 LLM 动作
```javascript
// autonomous-orchestrator 中的 LLM 动作调度
const actions = [
{ type: 'collect', priority: 100 },
{ type: 'llm', id: 'llm-analyze', priority: 90 }, // LLM 分析
{ type: 'process', priority: 80 },
{ type: 'llm', id: 'llm-generate', priority: 70 }, // LLM 生成
{ type: 'validate', priority: 60 }
];
for (const action of sortByPriority(actions)) {
if (action.type === 'llm') {
const llmResult = await executeLLMAction(
getLLMConfig(action.id),
context
);
context.state[action.id] = llmResult;
}
}
```
---
## 错误处理
```javascript
async function executeLLMActionWithRetry(config, context, maxRetries = 3) {
let lastError = null;
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
return await executeLLMAction(config, context);
} catch (error) {
lastError = error;
console.log(`Attempt ${attempt} failed: ${error.message}`);
// 指数退避
if (attempt < maxRetries) {
await sleep(Math.pow(2, attempt) * 1000);
}
}
}
// 所有重试失败
return {
success: false,
error: lastError.message,
fallback: 'manual_review_required'
};
}
```
---
## 最佳实践
1. **选择合适的工具**
- 分析任务Gemini大上下文> Qwen
- 生成任务Codex自主执行> Gemini > Qwen
- 代码修改Codex > Gemini
2. **配置 Fallback Chain**
- 总是配置至少一个 fallback
- 考虑工具特性选择 fallback 顺序
3. **超时设置**
- 分析任务10-15 分钟
- 生成任务15-20 分钟
- 复杂任务20-60 分钟
4. **提示词设计**
- 使用 PURPOSE/TASK/MODE/CONTEXT/EXPECTED/RULES 结构
- 引用标准协议模板
- 明确输出格式要求