mirror of
https://github.com/cexll/myclaude.git
synced 2026-02-05 02:30:26 +08:00
Compare commits
6 Commits
v5.2.2
...
fix/preven
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a09c103cfb | ||
|
|
b3f8fcfea6 | ||
|
|
806bb04a35 | ||
|
|
b1156038de | ||
|
|
0c93bbe574 | ||
|
|
6f4f4e701b |
257
CHANGELOG.md
257
CHANGELOG.md
@@ -1,129 +1,198 @@
|
||||
# Changelog
|
||||
|
||||
## 5.2.0 - 2025-12-13
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
### 🚀 Core Features
|
||||
## [5.2.3] - 2025-12-15
|
||||
|
||||
#### Skills System Enhancements
|
||||
- **New Skills**: Added `codeagent`, `product-requirements`, `prototype-prompt-generator` to `skill-rules.json`
|
||||
- **Auto-Activation**: Skills automatically trigger based on keyword/pattern matching via hooks
|
||||
- **Backward Compatibility**: Retained `skills/codex/SKILL.md` for existing workflows
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
#### Multi-Backend Support (codeagent-wrapper)
|
||||
- **Renamed**: `codex-wrapper` → `codeagent-wrapper` with pluggable backend architecture
|
||||
- **Three Backends**: Codex (default), Claude, Gemini via `--backend` flag
|
||||
- **Smart Parser**: Auto-detects backend JSON stream formats
|
||||
- **Session Resume**: All backends support `-r <session_id>` cross-session resume
|
||||
- **Parallel Execution**: DAG task scheduling with global and per-task backend configuration
|
||||
- **Concurrency Control**: `CODEAGENT_MAX_PARALLEL_WORKERS` env var limits concurrent tasks (max 100)
|
||||
- **Test Coverage**: 93.4% (backend.go 100%, config.go 97.8%, executor.go 96.4%)
|
||||
- *(parser)* 修复 bufio.Scanner token too long 错误 (#64)
|
||||
|
||||
#### Dev Workflow
|
||||
- **`/dev`**: 6-step minimal dev workflow with mandatory 90% test coverage
|
||||
### 🧪 Testing
|
||||
|
||||
#### Hooks System
|
||||
- **UserPromptSubmit**: Auto-activate skills based on context
|
||||
- **PostToolUse**: Auto-validation/formatting after tool execution
|
||||
- **Stop**: Cleanup and reporting on session end
|
||||
- **Examples**: Skill auto-activation, pre-commit checks
|
||||
- 同步测试中的版本号至 5.2.3
|
||||
|
||||
#### Skills System
|
||||
- **Auto-Activation**: `skill-rules.json` regex trigger rules
|
||||
- **codeagent skill**: Multi-backend wrapper integration
|
||||
- **Modular Design**: Easy to extend with custom skills
|
||||
## [5.2.2] - 2025-12-13
|
||||
|
||||
#### Installation System Enhancements
|
||||
- **`merge_json` operation**: Auto-merge `settings.json` configuration
|
||||
- **Modular Installation**: `python3 install.py --module dev`
|
||||
- **Verbose Logging**: `--verbose/-v` enables terminal real-time output
|
||||
- **Streaming Output**: `op_run_command` streams bash script execution
|
||||
- **Configuration Cleanup**: Removed deprecated `gh` module from `config.json`
|
||||
### 🧪 Testing
|
||||
|
||||
- Fix tests for ClaudeBackend default --dangerously-skip-permissions
|
||||
|
||||
### ⚙️ Miscellaneous Tasks
|
||||
|
||||
- *(v5.2.2)* Bump version and clean up documentation
|
||||
|
||||
## [5.2.0] - 2025-12-13
|
||||
|
||||
### 🚀 Features
|
||||
|
||||
- *(dev-workflow)* 替换 Codex 为 codeagent 并添加 UI 自动检测
|
||||
- *(codeagent-wrapper)* 完整多后端支持与安全优化
|
||||
- *(install)* 添加终端日志输出和 verbose 模式
|
||||
- *(v5.2.0)* Improve release notes and installation scripts
|
||||
- *(v5.2.0)* Complete skills system integration and config cleanup
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- *(merge)* 修复master合并后的编译和测试问题
|
||||
- *(parallel)* 修复并行执行启动横幅重复打印问题
|
||||
- *(ci)* 移除 .claude 配置文件验证步骤
|
||||
- *(codeagent-wrapper)* 重构信号处理逻辑避免重复 nil 检查
|
||||
- *(codeagent-wrapper)* 修复权限标志逻辑和版本号测试
|
||||
- *(install)* Op_run_command 实时流式输出
|
||||
- *(codeagent-wrapper)* 异常退出时显示最近错误信息
|
||||
- *(codeagent-wrapper)* Remove binary artifacts and improve error messages
|
||||
- *(codeagent-wrapper)* Use -r flag for claude backend resume
|
||||
- *(install)* Clarify module list shows default state not enabled
|
||||
- *(codeagent-wrapper)* Use -r flag for gemini backend resume
|
||||
- *(codeagent-wrapper)* Add worker limit cap and remove legacy alias
|
||||
- *(codeagent-wrapper)* Fix race condition in stdout parsing
|
||||
|
||||
### 🚜 Refactor
|
||||
|
||||
- *(pr-53)* 调整文件命名和技能定义
|
||||
|
||||
### 📚 Documentation
|
||||
|
||||
- `docs/architecture.md` (21KB): Architecture overview with ASCII diagrams
|
||||
- `docs/CODEAGENT-WRAPPER.md` (9KB): Complete usage guide
|
||||
- `docs/HOOKS.md` (4KB): Customization guide
|
||||
- `README.md`: Added documentation index, corrected default backend description
|
||||
- *(changelog)* Remove GitHub workflow related content
|
||||
|
||||
### 🔧 Important Fixes
|
||||
### 🧪 Testing
|
||||
|
||||
#### codeagent-wrapper
|
||||
- Fixed Claude/Gemini backend `-C` (workdir) and `-r` (resume) parameter support (codeagent-wrapper/backend.go:80-120)
|
||||
- Corrected Claude backend permission flag logic `if cfg.SkipPermissions` (codeagent-wrapper/backend.go:95)
|
||||
- Fixed parallel mode startup banner duplication (codeagent-wrapper/main.go:184-194 removed)
|
||||
- Extract and display recent errors on abnormal exit `Logger.ExtractRecentErrors()` (codeagent-wrapper/logger.go:156)
|
||||
- Added task block index to parallel config error messages (codeagent-wrapper/config.go:245)
|
||||
- Refactored signal handling logic to avoid duplicate nil checks (codeagent-wrapper/main.go:290-305)
|
||||
- Removed binary artifacts from tracking (codeagent-wrapper, *.test, coverage.out)
|
||||
- *(codeagent-wrapper)* 添加 ExtractRecentErrors 单元测试
|
||||
|
||||
#### Installation Scripts
|
||||
- Fixed issue #55: `op_run_command` uses Popen + selectors for real-time streaming output
|
||||
- Fixed issue #56: Display recent errors instead of entire log
|
||||
- Changed module list header from "Enabled" to "Default" to avoid ambiguity
|
||||
### ⚙️ Miscellaneous Tasks
|
||||
|
||||
#### CI/CD
|
||||
- Removed `.claude/` config file validation step (.github/workflows/ci.yml:45)
|
||||
- Updated version test case from 5.1.0 → 5.2.0 (codeagent-wrapper/main_test.go:23)
|
||||
- *(v5.2.0)* Update CHANGELOG and remove deprecated test files
|
||||
|
||||
#### Commands & Documentation
|
||||
- Reverted `skills/codex/SKILL.md` to `codex-wrapper` for backward compatibility
|
||||
## [5.1.4] - 2025-12-09
|
||||
|
||||
#### dev-workflow
|
||||
- Replaced Codex skill → codeagent skill throughout
|
||||
- Added UI auto-detection: backend tasks use codex, UI tasks use gemini
|
||||
- Corrected agent name: `develop-doc-generator` → `dev-plan-generator`
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
### ⚙️ Configuration & Environment Variables
|
||||
- *(parallel)* 任务启动时立即返回日志文件路径以支持实时调试
|
||||
|
||||
#### New Environment Variables
|
||||
- `CODEAGENT_SKIP_PERMISSIONS`: Control permission check behavior
|
||||
- Claude backend defaults to `--dangerously-skip-permissions` enabled, set to `true` to disable
|
||||
- Codex/Gemini backends default to permission checks enabled, set to `true` to skip
|
||||
- `CODEAGENT_MAX_PARALLEL_WORKERS`: Parallel task concurrency limit (default: unlimited, recommended: 8, max: 100)
|
||||
## [5.1.3] - 2025-12-08
|
||||
|
||||
#### Configuration Files
|
||||
- `config.schema.json`: Added `op_merge_json` schema validation
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
### ⚠️ Breaking Changes
|
||||
- *(test)* Resolve CI timing race in TestFakeCmdInfra
|
||||
|
||||
**codex-wrapper → codeagent-wrapper rename**
|
||||
## [5.1.2] - 2025-12-08
|
||||
|
||||
**Migration**:
|
||||
```bash
|
||||
python3 install.py --module dev --force
|
||||
```
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
**Backward Compatibility**: `codex-wrapper/main.go` provides compatibility entry point
|
||||
- 修复channel同步竞态条件和死锁问题
|
||||
|
||||
### 📦 Installation
|
||||
## [5.1.1] - 2025-12-08
|
||||
|
||||
```bash
|
||||
# Install dev module
|
||||
python3 install.py --module dev
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
# List all modules
|
||||
python3 install.py --list-modules
|
||||
- *(test)* Resolve data race on forceKillDelay with atomic operations
|
||||
- 增强日志清理的安全性和可靠性
|
||||
|
||||
# Verbose logging mode
|
||||
python3 install.py --module dev --verbose
|
||||
```
|
||||
### 💼 Other
|
||||
|
||||
### 🧪 Test Results
|
||||
- Resolve signal handling conflict preserving testability and Windows support
|
||||
|
||||
✅ **All tests passing**
|
||||
- Overall coverage: 93.4%
|
||||
- Security scan: 0 issues (gosec)
|
||||
- Linting: Pass
|
||||
### 🧪 Testing
|
||||
|
||||
### 📄 Related PRs & Issues
|
||||
- 补充测试覆盖提升至 89.3%
|
||||
|
||||
- PR #53: Enterprise Workflow with Multi-Backend Support
|
||||
- Issue #55: Installation script execution not visible
|
||||
- Issue #56: Unfriendly error logging on abnormal exit
|
||||
## [5.1.0] - 2025-12-07
|
||||
|
||||
### 👥 Contributors
|
||||
### 🚀 Features
|
||||
|
||||
- Claude Sonnet 4.5
|
||||
- Claude Opus 4.5
|
||||
- SWE-Agent-Bot
|
||||
- Implement enterprise workflow with multi-backend support
|
||||
- *(cleanup)* 添加启动时清理日志的功能和--cleanup标志支持
|
||||
|
||||
## [5.0.0] - 2025-12-05
|
||||
|
||||
### 🚀 Features
|
||||
|
||||
- Implement modular installation system
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- *(codex-wrapper)* Defer startup log until args parsed
|
||||
|
||||
### 🚜 Refactor
|
||||
|
||||
- Remove deprecated plugin modules
|
||||
|
||||
### 📚 Documentation
|
||||
|
||||
- Rewrite documentation for v5.0 modular architecture
|
||||
|
||||
### ⚙️ Miscellaneous Tasks
|
||||
|
||||
- Clarify unit-test coverage levels in requirement questions
|
||||
|
||||
## [4.8.2] - 2025-12-02
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- *(codex-wrapper)* Capture and include stderr in error messages
|
||||
- Correct Go version in go.mod from 1.25.3 to 1.21
|
||||
- Make forceKillDelay testable to prevent signal test timeout
|
||||
- Skip signal test in CI environment
|
||||
|
||||
## [4.8.1] - 2025-12-01
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- *(codex-wrapper)* Improve --parallel parameter validation and docs
|
||||
|
||||
### 🎨 Styling
|
||||
|
||||
- *(codex-skill)* Replace emoji with text labels
|
||||
|
||||
## [4.7.3] - 2025-11-29
|
||||
|
||||
### 🚀 Features
|
||||
|
||||
- Add async logging to temp file with lifecycle management
|
||||
- Add parallel execution support to codex-wrapper
|
||||
- Add session resume support and improve output format
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- *(logger)* 保留日志文件以便程序退出后调试并完善日志输出功能
|
||||
|
||||
### 📚 Documentation
|
||||
|
||||
- Improve codex skill parameter best practices
|
||||
|
||||
## [4.7.2] - 2025-11-28
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- *(main)* Improve buffer size and streamline message extraction
|
||||
|
||||
### 🧪 Testing
|
||||
|
||||
- *(ParseJSONStream)* 增加对超大单行文本和非字符串文本的处理测试
|
||||
|
||||
## [4.7] - 2025-11-27
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- Update repository URLs to cexll/myclaude
|
||||
|
||||
## [4.4] - 2025-11-22
|
||||
|
||||
### 🚀 Features
|
||||
|
||||
- 支持通过环境变量配置 skills 模型
|
||||
|
||||
## [4.1] - 2025-11-04
|
||||
|
||||
### 📚 Documentation
|
||||
|
||||
- 新增 /enhance-prompt 命令并更新所有 README 文档
|
||||
|
||||
## [3.1] - 2025-09-17
|
||||
|
||||
### 💼 Other
|
||||
|
||||
- Sync READMEs with actual commands/agents; remove nonexistent commands; enhance requirements-pilot with testing decision gate and options.
|
||||
|
||||
<!-- generated by git-cliff -->
|
||||
|
||||
@@ -36,6 +36,10 @@ func (ClaudeBackend) BuildArgs(cfg *Config, targetArg string) []string {
|
||||
// args = append(args, "--dangerously-skip-permissions")
|
||||
// }
|
||||
|
||||
// Prevent infinite recursion: disable all setting sources (user, project, local)
|
||||
// This ensures a clean execution environment without CLAUDE.md or skills that would trigger codeagent
|
||||
args = append(args, "--setting-sources", "")
|
||||
|
||||
if cfg.Mode == "resume" {
|
||||
if cfg.SessionID != "" {
|
||||
// Claude CLI uses -r <session_id> for resume.
|
||||
|
||||
@@ -11,7 +11,7 @@ func TestClaudeBuildArgs_ModesAndPermissions(t *testing.T) {
|
||||
t.Run("new mode uses workdir without skip by default", func(t *testing.T) {
|
||||
cfg := &Config{Mode: "new", WorkDir: "/repo"}
|
||||
got := backend.BuildArgs(cfg, "todo")
|
||||
want := []string{"-p", "--dangerously-skip-permissions", "--output-format", "stream-json", "--verbose", "todo"}
|
||||
want := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "--output-format", "stream-json", "--verbose", "todo"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("got %v, want %v", got, want)
|
||||
}
|
||||
@@ -20,7 +20,7 @@ func TestClaudeBuildArgs_ModesAndPermissions(t *testing.T) {
|
||||
t.Run("new mode opt-in skip permissions with default workdir", func(t *testing.T) {
|
||||
cfg := &Config{Mode: "new", SkipPermissions: true}
|
||||
got := backend.BuildArgs(cfg, "-")
|
||||
want := []string{"-p", "--dangerously-skip-permissions", "--output-format", "stream-json", "--verbose", "-"}
|
||||
want := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "--output-format", "stream-json", "--verbose", "-"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("got %v, want %v", got, want)
|
||||
}
|
||||
@@ -29,7 +29,7 @@ func TestClaudeBuildArgs_ModesAndPermissions(t *testing.T) {
|
||||
t.Run("resume mode uses session id and omits workdir", func(t *testing.T) {
|
||||
cfg := &Config{Mode: "resume", SessionID: "sid-123", WorkDir: "/ignored"}
|
||||
got := backend.BuildArgs(cfg, "resume-task")
|
||||
want := []string{"-p", "--dangerously-skip-permissions", "-r", "sid-123", "--output-format", "stream-json", "--verbose", "resume-task"}
|
||||
want := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "-r", "sid-123", "--output-format", "stream-json", "--verbose", "resume-task"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("got %v, want %v", got, want)
|
||||
}
|
||||
@@ -38,7 +38,7 @@ func TestClaudeBuildArgs_ModesAndPermissions(t *testing.T) {
|
||||
t.Run("resume mode without session still returns base flags", func(t *testing.T) {
|
||||
cfg := &Config{Mode: "resume", WorkDir: "/ignored"}
|
||||
got := backend.BuildArgs(cfg, "follow-up")
|
||||
want := []string{"-p", "--dangerously-skip-permissions", "--output-format", "stream-json", "--verbose", "follow-up"}
|
||||
want := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "--output-format", "stream-json", "--verbose", "follow-up"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("got %v, want %v", got, want)
|
||||
}
|
||||
|
||||
39
codeagent-wrapper/log_writer_limit_test.go
Normal file
39
codeagent-wrapper/log_writer_limit_test.go
Normal file
@@ -0,0 +1,39 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestLogWriterWriteLimitsBuffer(t *testing.T) {
|
||||
defer resetTestHooks()
|
||||
|
||||
logger, err := NewLogger()
|
||||
if err != nil {
|
||||
t.Fatalf("NewLogger error: %v", err)
|
||||
}
|
||||
setLogger(logger)
|
||||
defer closeLogger()
|
||||
|
||||
lw := newLogWriter("P:", 10)
|
||||
_, _ = lw.Write([]byte(strings.Repeat("a", 100)))
|
||||
|
||||
if lw.buf.Len() != 10 {
|
||||
t.Fatalf("logWriter buffer len=%d, want %d", lw.buf.Len(), 10)
|
||||
}
|
||||
if !lw.dropped {
|
||||
t.Fatalf("expected logWriter to drop overlong line bytes")
|
||||
}
|
||||
|
||||
lw.Flush()
|
||||
logger.Flush()
|
||||
data, err := os.ReadFile(logger.Path())
|
||||
if err != nil {
|
||||
t.Fatalf("ReadFile error: %v", err)
|
||||
}
|
||||
if !strings.Contains(string(data), "P:aaaaaaa...") {
|
||||
t.Fatalf("log output missing truncated entry, got %q", string(data))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -14,7 +14,7 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
version = "5.2.2"
|
||||
version = "5.2.3"
|
||||
defaultWorkdir = "."
|
||||
defaultTimeout = 7200 // seconds
|
||||
codexLogLineLimit = 1000
|
||||
|
||||
@@ -1377,7 +1377,7 @@ func TestBackendBuildArgs_ClaudeBackend(t *testing.T) {
|
||||
backend := ClaudeBackend{}
|
||||
cfg := &Config{Mode: "new", WorkDir: defaultWorkdir}
|
||||
got := backend.BuildArgs(cfg, "todo")
|
||||
want := []string{"-p", "--dangerously-skip-permissions", "--output-format", "stream-json", "--verbose", "todo"}
|
||||
want := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "--output-format", "stream-json", "--verbose", "todo"}
|
||||
if len(got) != len(want) {
|
||||
t.Fatalf("length mismatch")
|
||||
}
|
||||
@@ -1398,7 +1398,7 @@ func TestClaudeBackendBuildArgs_OutputValidation(t *testing.T) {
|
||||
target := "ensure-flags"
|
||||
|
||||
args := backend.BuildArgs(cfg, target)
|
||||
expectedPrefix := []string{"-p", "--dangerously-skip-permissions", "--output-format", "stream-json", "--verbose"}
|
||||
expectedPrefix := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "--output-format", "stream-json", "--verbose"}
|
||||
|
||||
if len(args) != len(expectedPrefix)+1 {
|
||||
t.Fatalf("args length=%d, want %d", len(args), len(expectedPrefix)+1)
|
||||
@@ -2690,7 +2690,7 @@ func TestVersionFlag(t *testing.T) {
|
||||
t.Errorf("exit = %d, want 0", code)
|
||||
}
|
||||
})
|
||||
want := "codeagent-wrapper version 5.2.2\n"
|
||||
want := "codeagent-wrapper version 5.2.3\n"
|
||||
if output != want {
|
||||
t.Fatalf("output = %q, want %q", output, want)
|
||||
}
|
||||
@@ -2704,7 +2704,7 @@ func TestVersionShortFlag(t *testing.T) {
|
||||
t.Errorf("exit = %d, want 0", code)
|
||||
}
|
||||
})
|
||||
want := "codeagent-wrapper version 5.2.2\n"
|
||||
want := "codeagent-wrapper version 5.2.3\n"
|
||||
if output != want {
|
||||
t.Fatalf("output = %q, want %q", output, want)
|
||||
}
|
||||
@@ -2718,7 +2718,7 @@ func TestVersionLegacyAlias(t *testing.T) {
|
||||
t.Errorf("exit = %d, want 0", code)
|
||||
}
|
||||
})
|
||||
want := "codex-wrapper version 5.2.2\n"
|
||||
want := "codex-wrapper version 5.2.3\n"
|
||||
if output != want {
|
||||
t.Fatalf("output = %q, want %q", output, want)
|
||||
}
|
||||
|
||||
@@ -53,9 +53,22 @@ func parseJSONStreamWithLog(r io.Reader, warnFn func(string), infoFn func(string
|
||||
return parseJSONStreamInternal(r, warnFn, infoFn, nil)
|
||||
}
|
||||
|
||||
const (
|
||||
jsonLineReaderSize = 64 * 1024
|
||||
jsonLineMaxBytes = 10 * 1024 * 1024
|
||||
jsonLinePreviewBytes = 256
|
||||
)
|
||||
|
||||
type codexHeader struct {
|
||||
Type string `json:"type"`
|
||||
ThreadID string `json:"thread_id,omitempty"`
|
||||
Item *struct {
|
||||
Type string `json:"type"`
|
||||
} `json:"item,omitempty"`
|
||||
}
|
||||
|
||||
func parseJSONStreamInternal(r io.Reader, warnFn func(string), infoFn func(string), onMessage func()) (message, threadID string) {
|
||||
scanner := bufio.NewScanner(r)
|
||||
scanner.Buffer(make([]byte, 64*1024), 10*1024*1024)
|
||||
reader := bufio.NewReaderSize(r, jsonLineReaderSize)
|
||||
|
||||
if warnFn == nil {
|
||||
warnFn = func(string) {}
|
||||
@@ -78,79 +91,89 @@ func parseJSONStreamInternal(r io.Reader, warnFn func(string), infoFn func(strin
|
||||
geminiBuffer strings.Builder
|
||||
)
|
||||
|
||||
for scanner.Scan() {
|
||||
line := strings.TrimSpace(scanner.Text())
|
||||
if line == "" {
|
||||
for {
|
||||
line, tooLong, err := readLineWithLimit(reader, jsonLineMaxBytes, jsonLinePreviewBytes)
|
||||
if err != nil {
|
||||
if errors.Is(err, io.EOF) {
|
||||
break
|
||||
}
|
||||
warnFn("Read stdout error: " + err.Error())
|
||||
break
|
||||
}
|
||||
|
||||
line = bytes.TrimSpace(line)
|
||||
if len(line) == 0 {
|
||||
continue
|
||||
}
|
||||
totalEvents++
|
||||
|
||||
var raw map[string]json.RawMessage
|
||||
if err := json.Unmarshal([]byte(line), &raw); err != nil {
|
||||
warnFn(fmt.Sprintf("Failed to parse line: %s", truncate(line, 100)))
|
||||
if tooLong {
|
||||
warnFn(fmt.Sprintf("Skipped overlong JSON line (> %d bytes): %s", jsonLineMaxBytes, truncateBytes(line, 100)))
|
||||
continue
|
||||
}
|
||||
|
||||
hasItemType := false
|
||||
if rawItem, ok := raw["item"]; ok {
|
||||
var itemMap map[string]json.RawMessage
|
||||
if err := json.Unmarshal(rawItem, &itemMap); err == nil {
|
||||
if _, ok := itemMap["type"]; ok {
|
||||
hasItemType = true
|
||||
var codex codexHeader
|
||||
if err := json.Unmarshal(line, &codex); err == nil {
|
||||
isCodex := codex.ThreadID != "" || (codex.Item != nil && codex.Item.Type != "")
|
||||
if isCodex {
|
||||
var details []string
|
||||
if codex.ThreadID != "" {
|
||||
details = append(details, fmt.Sprintf("thread_id=%s", codex.ThreadID))
|
||||
}
|
||||
if codex.Item != nil && codex.Item.Type != "" {
|
||||
details = append(details, fmt.Sprintf("item_type=%s", codex.Item.Type))
|
||||
}
|
||||
if len(details) > 0 {
|
||||
infoFn(fmt.Sprintf("Parsed event #%d type=%s (%s)", totalEvents, codex.Type, strings.Join(details, ", ")))
|
||||
} else {
|
||||
infoFn(fmt.Sprintf("Parsed event #%d type=%s", totalEvents, codex.Type))
|
||||
}
|
||||
|
||||
switch codex.Type {
|
||||
case "thread.started":
|
||||
threadID = codex.ThreadID
|
||||
infoFn(fmt.Sprintf("thread.started event thread_id=%s", threadID))
|
||||
case "item.completed":
|
||||
itemType := ""
|
||||
if codex.Item != nil {
|
||||
itemType = codex.Item.Type
|
||||
}
|
||||
|
||||
if itemType == "agent_message" {
|
||||
var event JSONEvent
|
||||
if err := json.Unmarshal(line, &event); err != nil {
|
||||
warnFn(fmt.Sprintf("Failed to parse Codex event: %s", truncateBytes(line, 100)))
|
||||
continue
|
||||
}
|
||||
|
||||
normalized := ""
|
||||
if event.Item != nil {
|
||||
normalized = normalizeText(event.Item.Text)
|
||||
}
|
||||
infoFn(fmt.Sprintf("item.completed event item_type=%s message_len=%d", itemType, len(normalized)))
|
||||
if normalized != "" {
|
||||
codexMessage = normalized
|
||||
notifyMessage()
|
||||
}
|
||||
} else {
|
||||
infoFn(fmt.Sprintf("item.completed event item_type=%s", itemType))
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
isCodex := hasItemType
|
||||
if !isCodex {
|
||||
if _, ok := raw["thread_id"]; ok {
|
||||
isCodex = true
|
||||
}
|
||||
var raw map[string]json.RawMessage
|
||||
if err := json.Unmarshal(line, &raw); err != nil {
|
||||
warnFn(fmt.Sprintf("Failed to parse line: %s", truncateBytes(line, 100)))
|
||||
continue
|
||||
}
|
||||
|
||||
switch {
|
||||
case isCodex:
|
||||
var event JSONEvent
|
||||
if err := json.Unmarshal([]byte(line), &event); err != nil {
|
||||
warnFn(fmt.Sprintf("Failed to parse Codex event: %s", truncate(line, 100)))
|
||||
continue
|
||||
}
|
||||
|
||||
var details []string
|
||||
if event.ThreadID != "" {
|
||||
details = append(details, fmt.Sprintf("thread_id=%s", event.ThreadID))
|
||||
}
|
||||
if event.Item != nil && event.Item.Type != "" {
|
||||
details = append(details, fmt.Sprintf("item_type=%s", event.Item.Type))
|
||||
}
|
||||
if len(details) > 0 {
|
||||
infoFn(fmt.Sprintf("Parsed event #%d type=%s (%s)", totalEvents, event.Type, strings.Join(details, ", ")))
|
||||
} else {
|
||||
infoFn(fmt.Sprintf("Parsed event #%d type=%s", totalEvents, event.Type))
|
||||
}
|
||||
|
||||
switch event.Type {
|
||||
case "thread.started":
|
||||
threadID = event.ThreadID
|
||||
infoFn(fmt.Sprintf("thread.started event thread_id=%s", threadID))
|
||||
case "item.completed":
|
||||
var itemType string
|
||||
var normalized string
|
||||
if event.Item != nil {
|
||||
itemType = event.Item.Type
|
||||
normalized = normalizeText(event.Item.Text)
|
||||
}
|
||||
infoFn(fmt.Sprintf("item.completed event item_type=%s message_len=%d", itemType, len(normalized)))
|
||||
if event.Item != nil && event.Item.Type == "agent_message" && normalized != "" {
|
||||
codexMessage = normalized
|
||||
notifyMessage()
|
||||
}
|
||||
}
|
||||
|
||||
case hasKey(raw, "subtype") || hasKey(raw, "result"):
|
||||
var event ClaudeEvent
|
||||
if err := json.Unmarshal([]byte(line), &event); err != nil {
|
||||
warnFn(fmt.Sprintf("Failed to parse Claude event: %s", truncate(line, 100)))
|
||||
if err := json.Unmarshal(line, &event); err != nil {
|
||||
warnFn(fmt.Sprintf("Failed to parse Claude event: %s", truncateBytes(line, 100)))
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -167,8 +190,8 @@ func parseJSONStreamInternal(r io.Reader, warnFn func(string), infoFn func(strin
|
||||
|
||||
case hasKey(raw, "role") || hasKey(raw, "delta"):
|
||||
var event GeminiEvent
|
||||
if err := json.Unmarshal([]byte(line), &event); err != nil {
|
||||
warnFn(fmt.Sprintf("Failed to parse Gemini event: %s", truncate(line, 100)))
|
||||
if err := json.Unmarshal(line, &event); err != nil {
|
||||
warnFn(fmt.Sprintf("Failed to parse Gemini event: %s", truncateBytes(line, 100)))
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -184,14 +207,10 @@ func parseJSONStreamInternal(r io.Reader, warnFn func(string), infoFn func(strin
|
||||
infoFn(fmt.Sprintf("Parsed Gemini event #%d type=%s role=%s delta=%t status=%s content_len=%d", totalEvents, event.Type, event.Role, event.Delta, event.Status, len(event.Content)))
|
||||
|
||||
default:
|
||||
warnFn(fmt.Sprintf("Unknown event format: %s", truncate(line, 100)))
|
||||
warnFn(fmt.Sprintf("Unknown event format: %s", truncateBytes(line, 100)))
|
||||
}
|
||||
}
|
||||
|
||||
if err := scanner.Err(); err != nil && !errors.Is(err, io.EOF) {
|
||||
warnFn("Read stdout error: " + err.Error())
|
||||
}
|
||||
|
||||
switch {
|
||||
case geminiBuffer.Len() > 0:
|
||||
message = geminiBuffer.String()
|
||||
@@ -236,6 +255,79 @@ func discardInvalidJSON(decoder *json.Decoder, reader *bufio.Reader) (*bufio.Rea
|
||||
return bufio.NewReader(io.MultiReader(bytes.NewReader(remaining), reader)), err
|
||||
}
|
||||
|
||||
func readLineWithLimit(r *bufio.Reader, maxBytes int, previewBytes int) (line []byte, tooLong bool, err error) {
|
||||
if r == nil {
|
||||
return nil, false, errors.New("reader is nil")
|
||||
}
|
||||
if maxBytes <= 0 {
|
||||
return nil, false, errors.New("maxBytes must be > 0")
|
||||
}
|
||||
if previewBytes < 0 {
|
||||
previewBytes = 0
|
||||
}
|
||||
|
||||
part, isPrefix, err := r.ReadLine()
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
|
||||
if !isPrefix {
|
||||
if len(part) > maxBytes {
|
||||
return part[:min(len(part), previewBytes)], true, nil
|
||||
}
|
||||
return part, false, nil
|
||||
}
|
||||
|
||||
preview := make([]byte, 0, min(previewBytes, len(part)))
|
||||
if previewBytes > 0 {
|
||||
preview = append(preview, part[:min(previewBytes, len(part))]...)
|
||||
}
|
||||
|
||||
buf := make([]byte, 0, min(maxBytes, len(part)*2))
|
||||
total := 0
|
||||
if len(part) > maxBytes {
|
||||
tooLong = true
|
||||
} else {
|
||||
buf = append(buf, part...)
|
||||
total = len(part)
|
||||
}
|
||||
|
||||
for isPrefix {
|
||||
part, isPrefix, err = r.ReadLine()
|
||||
if err != nil {
|
||||
return nil, tooLong, err
|
||||
}
|
||||
|
||||
if previewBytes > 0 && len(preview) < previewBytes {
|
||||
preview = append(preview, part[:min(previewBytes-len(preview), len(part))]...)
|
||||
}
|
||||
|
||||
if !tooLong {
|
||||
if total+len(part) > maxBytes {
|
||||
tooLong = true
|
||||
continue
|
||||
}
|
||||
buf = append(buf, part...)
|
||||
total += len(part)
|
||||
}
|
||||
}
|
||||
|
||||
if tooLong {
|
||||
return preview, true, nil
|
||||
}
|
||||
return buf, false, nil
|
||||
}
|
||||
|
||||
func truncateBytes(b []byte, maxLen int) string {
|
||||
if len(b) <= maxLen {
|
||||
return string(b)
|
||||
}
|
||||
if maxLen < 0 {
|
||||
return ""
|
||||
}
|
||||
return string(b[:maxLen]) + "..."
|
||||
}
|
||||
|
||||
func normalizeText(text interface{}) string {
|
||||
switch v := text.(type) {
|
||||
case string:
|
||||
|
||||
31
codeagent-wrapper/parser_token_too_long_test.go
Normal file
31
codeagent-wrapper/parser_token_too_long_test.go
Normal file
@@ -0,0 +1,31 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestParseJSONStream_SkipsOverlongLineAndContinues(t *testing.T) {
|
||||
// Exceed the 10MB bufio.Scanner limit in parseJSONStreamInternal.
|
||||
tooLong := strings.Repeat("a", 11*1024*1024)
|
||||
|
||||
input := strings.Join([]string{
|
||||
`{"type":"item.completed","item":{"type":"other_type","text":"` + tooLong + `"}}`,
|
||||
`{"type":"thread.started","thread_id":"t-1"}`,
|
||||
`{"type":"item.completed","item":{"type":"agent_message","text":"ok"}}`,
|
||||
}, "\n")
|
||||
|
||||
var warns []string
|
||||
warnFn := func(msg string) { warns = append(warns, msg) }
|
||||
|
||||
gotMessage, gotThreadID := parseJSONStreamInternal(strings.NewReader(input), warnFn, nil, nil)
|
||||
if gotMessage != "ok" {
|
||||
t.Fatalf("message=%q, want %q (warns=%v)", gotMessage, "ok", warns)
|
||||
}
|
||||
if gotThreadID != "t-1" {
|
||||
t.Fatalf("threadID=%q, want %q (warns=%v)", gotThreadID, "t-1", warns)
|
||||
}
|
||||
if len(warns) == 0 || !strings.Contains(warns[0], "Skipped overlong JSON line") {
|
||||
t.Fatalf("expected warning about overlong JSON line, got %v", warns)
|
||||
}
|
||||
}
|
||||
@@ -78,6 +78,7 @@ type logWriter struct {
|
||||
prefix string
|
||||
maxLen int
|
||||
buf bytes.Buffer
|
||||
dropped bool
|
||||
}
|
||||
|
||||
func newLogWriter(prefix string, maxLen int) *logWriter {
|
||||
@@ -94,12 +95,12 @@ func (lw *logWriter) Write(p []byte) (int, error) {
|
||||
total := len(p)
|
||||
for len(p) > 0 {
|
||||
if idx := bytes.IndexByte(p, '\n'); idx >= 0 {
|
||||
lw.buf.Write(p[:idx])
|
||||
lw.writeLimited(p[:idx])
|
||||
lw.logLine(true)
|
||||
p = p[idx+1:]
|
||||
continue
|
||||
}
|
||||
lw.buf.Write(p)
|
||||
lw.writeLimited(p)
|
||||
break
|
||||
}
|
||||
return total, nil
|
||||
@@ -117,21 +118,53 @@ func (lw *logWriter) logLine(force bool) {
|
||||
return
|
||||
}
|
||||
line := lw.buf.String()
|
||||
dropped := lw.dropped
|
||||
lw.dropped = false
|
||||
lw.buf.Reset()
|
||||
if line == "" && !force {
|
||||
return
|
||||
}
|
||||
if lw.maxLen > 0 && len(line) > lw.maxLen {
|
||||
cutoff := lw.maxLen
|
||||
if cutoff > 3 {
|
||||
line = line[:cutoff-3] + "..."
|
||||
} else {
|
||||
line = line[:cutoff]
|
||||
if lw.maxLen > 0 {
|
||||
if dropped {
|
||||
if lw.maxLen > 3 {
|
||||
line = line[:min(len(line), lw.maxLen-3)] + "..."
|
||||
} else {
|
||||
line = line[:min(len(line), lw.maxLen)]
|
||||
}
|
||||
} else if len(line) > lw.maxLen {
|
||||
cutoff := lw.maxLen
|
||||
if cutoff > 3 {
|
||||
line = line[:cutoff-3] + "..."
|
||||
} else {
|
||||
line = line[:cutoff]
|
||||
}
|
||||
}
|
||||
}
|
||||
logInfo(lw.prefix + line)
|
||||
}
|
||||
|
||||
func (lw *logWriter) writeLimited(p []byte) {
|
||||
if lw == nil || len(p) == 0 {
|
||||
return
|
||||
}
|
||||
if lw.maxLen <= 0 {
|
||||
lw.buf.Write(p)
|
||||
return
|
||||
}
|
||||
|
||||
remaining := lw.maxLen - lw.buf.Len()
|
||||
if remaining <= 0 {
|
||||
lw.dropped = true
|
||||
return
|
||||
}
|
||||
if len(p) <= remaining {
|
||||
lw.buf.Write(p)
|
||||
return
|
||||
}
|
||||
lw.buf.Write(p[:remaining])
|
||||
lw.dropped = true
|
||||
}
|
||||
|
||||
type tailBuffer struct {
|
||||
limit int
|
||||
data []byte
|
||||
|
||||
Reference in New Issue
Block a user