Compare commits

...

6 Commits

Author SHA1 Message Date
cexll
a09c103cfb fix(codeagent): 防止 Claude backend 无限递归调用
通过设置 --setting-sources="" 禁用所有配置源(user, project, local),
避免被调用的 Claude 实例加载 ~/.claude/CLAUDE.md 和 skills,
从而防止再次调用 codeagent 导致的循环超时问题。

修改内容:
- backend.go: ClaudeBackend.BuildArgs 添加 --setting-sources="" 参数
- backend_test.go: 更新 4 个测试用例以匹配新的参数列表
- main_test.go: 更新 2 个测试用例以匹配新的参数列表

Generated with swe-agent-bot

Co-Authored-By: swe-agent-bot <agent@swe-agent.ai>
2025-12-16 10:27:21 +08:00
swe-agent[bot]
b3f8fcfea6 update CHANGELOG.md 2025-12-15 22:23:34 +08:00
ben
806bb04a35 Merge pull request #65 from cexll/fix/issue-64-buffer-overflow
fix(parser): 修复 bufio.Scanner token too long 错误
2025-12-15 14:22:03 +08:00
swe-agent[bot]
b1156038de test: 同步测试中的版本号至 5.2.3
修复 CI 失败:将 main_test.go 中的版本期望值从 5.2.2 更新为 5.2.3,
与 main.go 中的实际版本号保持一致。

修改文件:
- codeagent-wrapper/main_test.go:2693 (TestVersionFlag)
- codeagent-wrapper/main_test.go:2707 (TestVersionShortFlag)
- codeagent-wrapper/main_test.go:2721 (TestVersionLegacyAlias)

Generated with swe-agent-bot

Co-Authored-By: swe-agent-bot <agent@swe-agent.ai>
2025-12-15 14:13:03 +08:00
swe-agent[bot]
0c93bbe574 change version 2025-12-15 13:23:26 +08:00
swe-agent[bot]
6f4f4e701b fix(parser): 修复 bufio.Scanner token too long 错误 (#64)
## 问题
- 执行 rg 等命令时,如果匹配到 minified 文件,单行输出可能超过 10MB
- 旧实现使用 bufio.Scanner,遇到超长行会报错并中止整个解析
- 导致后续的 agent_message 无法读取,任务失败

## 修复
1. **parser.go**:
   - 移除 bufio.Scanner,改用 bufio.Reader + readLineWithLimit
   - 超长行(>10MB)会被跳过但继续处理后续事件
   - 添加 codexHeader 轻量级解析,只在 agent_message 时完整解析

2. **utils.go**:
   - 修复 logWriter 内存膨胀问题
   - 添加 writeLimited 方法限制缓冲区大小

3. **测试**:
   - parser_token_too_long_test.go: 验证超长行处理
   - log_writer_limit_test.go: 验证日志缓冲限制

## 测试结果
-  TestParseJSONStream_SkipsOverlongLineAndContinues
-  TestLogWriterWriteLimitsBuffer
-  完整测试套件通过

Fixes #64

Generated with swe-agent-bot

Co-Authored-By: swe-agent-bot <agent@swe-agent.ai>
2025-12-15 13:19:51 +08:00
9 changed files with 446 additions and 178 deletions

View File

@@ -1,129 +1,198 @@
# Changelog
## 5.2.0 - 2025-12-13
All notable changes to this project will be documented in this file.
### 🚀 Core Features
## [5.2.3] - 2025-12-15
#### Skills System Enhancements
- **New Skills**: Added `codeagent`, `product-requirements`, `prototype-prompt-generator` to `skill-rules.json`
- **Auto-Activation**: Skills automatically trigger based on keyword/pattern matching via hooks
- **Backward Compatibility**: Retained `skills/codex/SKILL.md` for existing workflows
### 🐛 Bug Fixes
#### Multi-Backend Support (codeagent-wrapper)
- **Renamed**: `codex-wrapper``codeagent-wrapper` with pluggable backend architecture
- **Three Backends**: Codex (default), Claude, Gemini via `--backend` flag
- **Smart Parser**: Auto-detects backend JSON stream formats
- **Session Resume**: All backends support `-r <session_id>` cross-session resume
- **Parallel Execution**: DAG task scheduling with global and per-task backend configuration
- **Concurrency Control**: `CODEAGENT_MAX_PARALLEL_WORKERS` env var limits concurrent tasks (max 100)
- **Test Coverage**: 93.4% (backend.go 100%, config.go 97.8%, executor.go 96.4%)
- *(parser)* 修复 bufio.Scanner token too long 错误 (#64)
#### Dev Workflow
- **`/dev`**: 6-step minimal dev workflow with mandatory 90% test coverage
### 🧪 Testing
#### Hooks System
- **UserPromptSubmit**: Auto-activate skills based on context
- **PostToolUse**: Auto-validation/formatting after tool execution
- **Stop**: Cleanup and reporting on session end
- **Examples**: Skill auto-activation, pre-commit checks
- 同步测试中的版本号至 5.2.3
#### Skills System
- **Auto-Activation**: `skill-rules.json` regex trigger rules
- **codeagent skill**: Multi-backend wrapper integration
- **Modular Design**: Easy to extend with custom skills
## [5.2.2] - 2025-12-13
#### Installation System Enhancements
- **`merge_json` operation**: Auto-merge `settings.json` configuration
- **Modular Installation**: `python3 install.py --module dev`
- **Verbose Logging**: `--verbose/-v` enables terminal real-time output
- **Streaming Output**: `op_run_command` streams bash script execution
- **Configuration Cleanup**: Removed deprecated `gh` module from `config.json`
### 🧪 Testing
- Fix tests for ClaudeBackend default --dangerously-skip-permissions
### ⚙️ Miscellaneous Tasks
- *(v5.2.2)* Bump version and clean up documentation
## [5.2.0] - 2025-12-13
### 🚀 Features
- *(dev-workflow)* 替换 Codex 为 codeagent 并添加 UI 自动检测
- *(codeagent-wrapper)* 完整多后端支持与安全优化
- *(install)* 添加终端日志输出和 verbose 模式
- *(v5.2.0)* Improve release notes and installation scripts
- *(v5.2.0)* Complete skills system integration and config cleanup
### 🐛 Bug Fixes
- *(merge)* 修复master合并后的编译和测试问题
- *(parallel)* 修复并行执行启动横幅重复打印问题
- *(ci)* 移除 .claude 配置文件验证步骤
- *(codeagent-wrapper)* 重构信号处理逻辑避免重复 nil 检查
- *(codeagent-wrapper)* 修复权限标志逻辑和版本号测试
- *(install)* Op_run_command 实时流式输出
- *(codeagent-wrapper)* 异常退出时显示最近错误信息
- *(codeagent-wrapper)* Remove binary artifacts and improve error messages
- *(codeagent-wrapper)* Use -r flag for claude backend resume
- *(install)* Clarify module list shows default state not enabled
- *(codeagent-wrapper)* Use -r flag for gemini backend resume
- *(codeagent-wrapper)* Add worker limit cap and remove legacy alias
- *(codeagent-wrapper)* Fix race condition in stdout parsing
### 🚜 Refactor
- *(pr-53)* 调整文件命名和技能定义
### 📚 Documentation
- `docs/architecture.md` (21KB): Architecture overview with ASCII diagrams
- `docs/CODEAGENT-WRAPPER.md` (9KB): Complete usage guide
- `docs/HOOKS.md` (4KB): Customization guide
- `README.md`: Added documentation index, corrected default backend description
- *(changelog)* Remove GitHub workflow related content
### 🔧 Important Fixes
### 🧪 Testing
#### codeagent-wrapper
- Fixed Claude/Gemini backend `-C` (workdir) and `-r` (resume) parameter support (codeagent-wrapper/backend.go:80-120)
- Corrected Claude backend permission flag logic `if cfg.SkipPermissions` (codeagent-wrapper/backend.go:95)
- Fixed parallel mode startup banner duplication (codeagent-wrapper/main.go:184-194 removed)
- Extract and display recent errors on abnormal exit `Logger.ExtractRecentErrors()` (codeagent-wrapper/logger.go:156)
- Added task block index to parallel config error messages (codeagent-wrapper/config.go:245)
- Refactored signal handling logic to avoid duplicate nil checks (codeagent-wrapper/main.go:290-305)
- Removed binary artifacts from tracking (codeagent-wrapper, *.test, coverage.out)
- *(codeagent-wrapper)* 添加 ExtractRecentErrors 单元测试
#### Installation Scripts
- Fixed issue #55: `op_run_command` uses Popen + selectors for real-time streaming output
- Fixed issue #56: Display recent errors instead of entire log
- Changed module list header from "Enabled" to "Default" to avoid ambiguity
### ⚙️ Miscellaneous Tasks
#### CI/CD
- Removed `.claude/` config file validation step (.github/workflows/ci.yml:45)
- Updated version test case from 5.1.0 → 5.2.0 (codeagent-wrapper/main_test.go:23)
- *(v5.2.0)* Update CHANGELOG and remove deprecated test files
#### Commands & Documentation
- Reverted `skills/codex/SKILL.md` to `codex-wrapper` for backward compatibility
## [5.1.4] - 2025-12-09
#### dev-workflow
- Replaced Codex skill → codeagent skill throughout
- Added UI auto-detection: backend tasks use codex, UI tasks use gemini
- Corrected agent name: `develop-doc-generator``dev-plan-generator`
### 🐛 Bug Fixes
### ⚙️ Configuration & Environment Variables
- *(parallel)* 任务启动时立即返回日志文件路径以支持实时调试
#### New Environment Variables
- `CODEAGENT_SKIP_PERMISSIONS`: Control permission check behavior
- Claude backend defaults to `--dangerously-skip-permissions` enabled, set to `true` to disable
- Codex/Gemini backends default to permission checks enabled, set to `true` to skip
- `CODEAGENT_MAX_PARALLEL_WORKERS`: Parallel task concurrency limit (default: unlimited, recommended: 8, max: 100)
## [5.1.3] - 2025-12-08
#### Configuration Files
- `config.schema.json`: Added `op_merge_json` schema validation
### 🐛 Bug Fixes
### ⚠️ Breaking Changes
- *(test)* Resolve CI timing race in TestFakeCmdInfra
**codex-wrapper → codeagent-wrapper rename**
## [5.1.2] - 2025-12-08
**Migration**:
```bash
python3 install.py --module dev --force
```
### 🐛 Bug Fixes
**Backward Compatibility**: `codex-wrapper/main.go` provides compatibility entry point
- 修复channel同步竞态条件和死锁问题
### 📦 Installation
## [5.1.1] - 2025-12-08
```bash
# Install dev module
python3 install.py --module dev
### 🐛 Bug Fixes
# List all modules
python3 install.py --list-modules
- *(test)* Resolve data race on forceKillDelay with atomic operations
- 增强日志清理的安全性和可靠性
# Verbose logging mode
python3 install.py --module dev --verbose
```
### 💼 Other
### 🧪 Test Results
- Resolve signal handling conflict preserving testability and Windows support
**All tests passing**
- Overall coverage: 93.4%
- Security scan: 0 issues (gosec)
- Linting: Pass
### 🧪 Testing
### 📄 Related PRs & Issues
- 补充测试覆盖提升至 89.3%
- PR #53: Enterprise Workflow with Multi-Backend Support
- Issue #55: Installation script execution not visible
- Issue #56: Unfriendly error logging on abnormal exit
## [5.1.0] - 2025-12-07
### 👥 Contributors
### 🚀 Features
- Claude Sonnet 4.5
- Claude Opus 4.5
- SWE-Agent-Bot
- Implement enterprise workflow with multi-backend support
- *(cleanup)* 添加启动时清理日志的功能和--cleanup标志支持
## [5.0.0] - 2025-12-05
### 🚀 Features
- Implement modular installation system
### 🐛 Bug Fixes
- *(codex-wrapper)* Defer startup log until args parsed
### 🚜 Refactor
- Remove deprecated plugin modules
### 📚 Documentation
- Rewrite documentation for v5.0 modular architecture
### ⚙️ Miscellaneous Tasks
- Clarify unit-test coverage levels in requirement questions
## [4.8.2] - 2025-12-02
### 🐛 Bug Fixes
- *(codex-wrapper)* Capture and include stderr in error messages
- Correct Go version in go.mod from 1.25.3 to 1.21
- Make forceKillDelay testable to prevent signal test timeout
- Skip signal test in CI environment
## [4.8.1] - 2025-12-01
### 🐛 Bug Fixes
- *(codex-wrapper)* Improve --parallel parameter validation and docs
### 🎨 Styling
- *(codex-skill)* Replace emoji with text labels
## [4.7.3] - 2025-11-29
### 🚀 Features
- Add async logging to temp file with lifecycle management
- Add parallel execution support to codex-wrapper
- Add session resume support and improve output format
### 🐛 Bug Fixes
- *(logger)* 保留日志文件以便程序退出后调试并完善日志输出功能
### 📚 Documentation
- Improve codex skill parameter best practices
## [4.7.2] - 2025-11-28
### 🐛 Bug Fixes
- *(main)* Improve buffer size and streamline message extraction
### 🧪 Testing
- *(ParseJSONStream)* 增加对超大单行文本和非字符串文本的处理测试
## [4.7] - 2025-11-27
### 🐛 Bug Fixes
- Update repository URLs to cexll/myclaude
## [4.4] - 2025-11-22
### 🚀 Features
- 支持通过环境变量配置 skills 模型
## [4.1] - 2025-11-04
### 📚 Documentation
- 新增 /enhance-prompt 命令并更新所有 README 文档
## [3.1] - 2025-09-17
### 💼 Other
- Sync READMEs with actual commands/agents; remove nonexistent commands; enhance requirements-pilot with testing decision gate and options.
<!-- generated by git-cliff -->

View File

@@ -36,6 +36,10 @@ func (ClaudeBackend) BuildArgs(cfg *Config, targetArg string) []string {
// args = append(args, "--dangerously-skip-permissions")
// }
// Prevent infinite recursion: disable all setting sources (user, project, local)
// This ensures a clean execution environment without CLAUDE.md or skills that would trigger codeagent
args = append(args, "--setting-sources", "")
if cfg.Mode == "resume" {
if cfg.SessionID != "" {
// Claude CLI uses -r <session_id> for resume.

View File

@@ -11,7 +11,7 @@ func TestClaudeBuildArgs_ModesAndPermissions(t *testing.T) {
t.Run("new mode uses workdir without skip by default", func(t *testing.T) {
cfg := &Config{Mode: "new", WorkDir: "/repo"}
got := backend.BuildArgs(cfg, "todo")
want := []string{"-p", "--dangerously-skip-permissions", "--output-format", "stream-json", "--verbose", "todo"}
want := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "--output-format", "stream-json", "--verbose", "todo"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("got %v, want %v", got, want)
}
@@ -20,7 +20,7 @@ func TestClaudeBuildArgs_ModesAndPermissions(t *testing.T) {
t.Run("new mode opt-in skip permissions with default workdir", func(t *testing.T) {
cfg := &Config{Mode: "new", SkipPermissions: true}
got := backend.BuildArgs(cfg, "-")
want := []string{"-p", "--dangerously-skip-permissions", "--output-format", "stream-json", "--verbose", "-"}
want := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "--output-format", "stream-json", "--verbose", "-"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("got %v, want %v", got, want)
}
@@ -29,7 +29,7 @@ func TestClaudeBuildArgs_ModesAndPermissions(t *testing.T) {
t.Run("resume mode uses session id and omits workdir", func(t *testing.T) {
cfg := &Config{Mode: "resume", SessionID: "sid-123", WorkDir: "/ignored"}
got := backend.BuildArgs(cfg, "resume-task")
want := []string{"-p", "--dangerously-skip-permissions", "-r", "sid-123", "--output-format", "stream-json", "--verbose", "resume-task"}
want := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "-r", "sid-123", "--output-format", "stream-json", "--verbose", "resume-task"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("got %v, want %v", got, want)
}
@@ -38,7 +38,7 @@ func TestClaudeBuildArgs_ModesAndPermissions(t *testing.T) {
t.Run("resume mode without session still returns base flags", func(t *testing.T) {
cfg := &Config{Mode: "resume", WorkDir: "/ignored"}
got := backend.BuildArgs(cfg, "follow-up")
want := []string{"-p", "--dangerously-skip-permissions", "--output-format", "stream-json", "--verbose", "follow-up"}
want := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "--output-format", "stream-json", "--verbose", "follow-up"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("got %v, want %v", got, want)
}

View File

@@ -0,0 +1,39 @@
package main
import (
"os"
"strings"
"testing"
)
func TestLogWriterWriteLimitsBuffer(t *testing.T) {
defer resetTestHooks()
logger, err := NewLogger()
if err != nil {
t.Fatalf("NewLogger error: %v", err)
}
setLogger(logger)
defer closeLogger()
lw := newLogWriter("P:", 10)
_, _ = lw.Write([]byte(strings.Repeat("a", 100)))
if lw.buf.Len() != 10 {
t.Fatalf("logWriter buffer len=%d, want %d", lw.buf.Len(), 10)
}
if !lw.dropped {
t.Fatalf("expected logWriter to drop overlong line bytes")
}
lw.Flush()
logger.Flush()
data, err := os.ReadFile(logger.Path())
if err != nil {
t.Fatalf("ReadFile error: %v", err)
}
if !strings.Contains(string(data), "P:aaaaaaa...") {
t.Fatalf("log output missing truncated entry, got %q", string(data))
}
}

View File

@@ -14,7 +14,7 @@ import (
)
const (
version = "5.2.2"
version = "5.2.3"
defaultWorkdir = "."
defaultTimeout = 7200 // seconds
codexLogLineLimit = 1000

View File

@@ -1377,7 +1377,7 @@ func TestBackendBuildArgs_ClaudeBackend(t *testing.T) {
backend := ClaudeBackend{}
cfg := &Config{Mode: "new", WorkDir: defaultWorkdir}
got := backend.BuildArgs(cfg, "todo")
want := []string{"-p", "--dangerously-skip-permissions", "--output-format", "stream-json", "--verbose", "todo"}
want := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "--output-format", "stream-json", "--verbose", "todo"}
if len(got) != len(want) {
t.Fatalf("length mismatch")
}
@@ -1398,7 +1398,7 @@ func TestClaudeBackendBuildArgs_OutputValidation(t *testing.T) {
target := "ensure-flags"
args := backend.BuildArgs(cfg, target)
expectedPrefix := []string{"-p", "--dangerously-skip-permissions", "--output-format", "stream-json", "--verbose"}
expectedPrefix := []string{"-p", "--dangerously-skip-permissions", "--setting-sources", "", "--output-format", "stream-json", "--verbose"}
if len(args) != len(expectedPrefix)+1 {
t.Fatalf("args length=%d, want %d", len(args), len(expectedPrefix)+1)
@@ -2690,7 +2690,7 @@ func TestVersionFlag(t *testing.T) {
t.Errorf("exit = %d, want 0", code)
}
})
want := "codeagent-wrapper version 5.2.2\n"
want := "codeagent-wrapper version 5.2.3\n"
if output != want {
t.Fatalf("output = %q, want %q", output, want)
}
@@ -2704,7 +2704,7 @@ func TestVersionShortFlag(t *testing.T) {
t.Errorf("exit = %d, want 0", code)
}
})
want := "codeagent-wrapper version 5.2.2\n"
want := "codeagent-wrapper version 5.2.3\n"
if output != want {
t.Fatalf("output = %q, want %q", output, want)
}
@@ -2718,7 +2718,7 @@ func TestVersionLegacyAlias(t *testing.T) {
t.Errorf("exit = %d, want 0", code)
}
})
want := "codex-wrapper version 5.2.2\n"
want := "codex-wrapper version 5.2.3\n"
if output != want {
t.Fatalf("output = %q, want %q", output, want)
}

View File

@@ -53,9 +53,22 @@ func parseJSONStreamWithLog(r io.Reader, warnFn func(string), infoFn func(string
return parseJSONStreamInternal(r, warnFn, infoFn, nil)
}
const (
jsonLineReaderSize = 64 * 1024
jsonLineMaxBytes = 10 * 1024 * 1024
jsonLinePreviewBytes = 256
)
type codexHeader struct {
Type string `json:"type"`
ThreadID string `json:"thread_id,omitempty"`
Item *struct {
Type string `json:"type"`
} `json:"item,omitempty"`
}
func parseJSONStreamInternal(r io.Reader, warnFn func(string), infoFn func(string), onMessage func()) (message, threadID string) {
scanner := bufio.NewScanner(r)
scanner.Buffer(make([]byte, 64*1024), 10*1024*1024)
reader := bufio.NewReaderSize(r, jsonLineReaderSize)
if warnFn == nil {
warnFn = func(string) {}
@@ -78,79 +91,89 @@ func parseJSONStreamInternal(r io.Reader, warnFn func(string), infoFn func(strin
geminiBuffer strings.Builder
)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
for {
line, tooLong, err := readLineWithLimit(reader, jsonLineMaxBytes, jsonLinePreviewBytes)
if err != nil {
if errors.Is(err, io.EOF) {
break
}
warnFn("Read stdout error: " + err.Error())
break
}
line = bytes.TrimSpace(line)
if len(line) == 0 {
continue
}
totalEvents++
var raw map[string]json.RawMessage
if err := json.Unmarshal([]byte(line), &raw); err != nil {
warnFn(fmt.Sprintf("Failed to parse line: %s", truncate(line, 100)))
if tooLong {
warnFn(fmt.Sprintf("Skipped overlong JSON line (> %d bytes): %s", jsonLineMaxBytes, truncateBytes(line, 100)))
continue
}
hasItemType := false
if rawItem, ok := raw["item"]; ok {
var itemMap map[string]json.RawMessage
if err := json.Unmarshal(rawItem, &itemMap); err == nil {
if _, ok := itemMap["type"]; ok {
hasItemType = true
var codex codexHeader
if err := json.Unmarshal(line, &codex); err == nil {
isCodex := codex.ThreadID != "" || (codex.Item != nil && codex.Item.Type != "")
if isCodex {
var details []string
if codex.ThreadID != "" {
details = append(details, fmt.Sprintf("thread_id=%s", codex.ThreadID))
}
if codex.Item != nil && codex.Item.Type != "" {
details = append(details, fmt.Sprintf("item_type=%s", codex.Item.Type))
}
if len(details) > 0 {
infoFn(fmt.Sprintf("Parsed event #%d type=%s (%s)", totalEvents, codex.Type, strings.Join(details, ", ")))
} else {
infoFn(fmt.Sprintf("Parsed event #%d type=%s", totalEvents, codex.Type))
}
switch codex.Type {
case "thread.started":
threadID = codex.ThreadID
infoFn(fmt.Sprintf("thread.started event thread_id=%s", threadID))
case "item.completed":
itemType := ""
if codex.Item != nil {
itemType = codex.Item.Type
}
if itemType == "agent_message" {
var event JSONEvent
if err := json.Unmarshal(line, &event); err != nil {
warnFn(fmt.Sprintf("Failed to parse Codex event: %s", truncateBytes(line, 100)))
continue
}
normalized := ""
if event.Item != nil {
normalized = normalizeText(event.Item.Text)
}
infoFn(fmt.Sprintf("item.completed event item_type=%s message_len=%d", itemType, len(normalized)))
if normalized != "" {
codexMessage = normalized
notifyMessage()
}
} else {
infoFn(fmt.Sprintf("item.completed event item_type=%s", itemType))
}
}
continue
}
}
isCodex := hasItemType
if !isCodex {
if _, ok := raw["thread_id"]; ok {
isCodex = true
}
var raw map[string]json.RawMessage
if err := json.Unmarshal(line, &raw); err != nil {
warnFn(fmt.Sprintf("Failed to parse line: %s", truncateBytes(line, 100)))
continue
}
switch {
case isCodex:
var event JSONEvent
if err := json.Unmarshal([]byte(line), &event); err != nil {
warnFn(fmt.Sprintf("Failed to parse Codex event: %s", truncate(line, 100)))
continue
}
var details []string
if event.ThreadID != "" {
details = append(details, fmt.Sprintf("thread_id=%s", event.ThreadID))
}
if event.Item != nil && event.Item.Type != "" {
details = append(details, fmt.Sprintf("item_type=%s", event.Item.Type))
}
if len(details) > 0 {
infoFn(fmt.Sprintf("Parsed event #%d type=%s (%s)", totalEvents, event.Type, strings.Join(details, ", ")))
} else {
infoFn(fmt.Sprintf("Parsed event #%d type=%s", totalEvents, event.Type))
}
switch event.Type {
case "thread.started":
threadID = event.ThreadID
infoFn(fmt.Sprintf("thread.started event thread_id=%s", threadID))
case "item.completed":
var itemType string
var normalized string
if event.Item != nil {
itemType = event.Item.Type
normalized = normalizeText(event.Item.Text)
}
infoFn(fmt.Sprintf("item.completed event item_type=%s message_len=%d", itemType, len(normalized)))
if event.Item != nil && event.Item.Type == "agent_message" && normalized != "" {
codexMessage = normalized
notifyMessage()
}
}
case hasKey(raw, "subtype") || hasKey(raw, "result"):
var event ClaudeEvent
if err := json.Unmarshal([]byte(line), &event); err != nil {
warnFn(fmt.Sprintf("Failed to parse Claude event: %s", truncate(line, 100)))
if err := json.Unmarshal(line, &event); err != nil {
warnFn(fmt.Sprintf("Failed to parse Claude event: %s", truncateBytes(line, 100)))
continue
}
@@ -167,8 +190,8 @@ func parseJSONStreamInternal(r io.Reader, warnFn func(string), infoFn func(strin
case hasKey(raw, "role") || hasKey(raw, "delta"):
var event GeminiEvent
if err := json.Unmarshal([]byte(line), &event); err != nil {
warnFn(fmt.Sprintf("Failed to parse Gemini event: %s", truncate(line, 100)))
if err := json.Unmarshal(line, &event); err != nil {
warnFn(fmt.Sprintf("Failed to parse Gemini event: %s", truncateBytes(line, 100)))
continue
}
@@ -184,14 +207,10 @@ func parseJSONStreamInternal(r io.Reader, warnFn func(string), infoFn func(strin
infoFn(fmt.Sprintf("Parsed Gemini event #%d type=%s role=%s delta=%t status=%s content_len=%d", totalEvents, event.Type, event.Role, event.Delta, event.Status, len(event.Content)))
default:
warnFn(fmt.Sprintf("Unknown event format: %s", truncate(line, 100)))
warnFn(fmt.Sprintf("Unknown event format: %s", truncateBytes(line, 100)))
}
}
if err := scanner.Err(); err != nil && !errors.Is(err, io.EOF) {
warnFn("Read stdout error: " + err.Error())
}
switch {
case geminiBuffer.Len() > 0:
message = geminiBuffer.String()
@@ -236,6 +255,79 @@ func discardInvalidJSON(decoder *json.Decoder, reader *bufio.Reader) (*bufio.Rea
return bufio.NewReader(io.MultiReader(bytes.NewReader(remaining), reader)), err
}
func readLineWithLimit(r *bufio.Reader, maxBytes int, previewBytes int) (line []byte, tooLong bool, err error) {
if r == nil {
return nil, false, errors.New("reader is nil")
}
if maxBytes <= 0 {
return nil, false, errors.New("maxBytes must be > 0")
}
if previewBytes < 0 {
previewBytes = 0
}
part, isPrefix, err := r.ReadLine()
if err != nil {
return nil, false, err
}
if !isPrefix {
if len(part) > maxBytes {
return part[:min(len(part), previewBytes)], true, nil
}
return part, false, nil
}
preview := make([]byte, 0, min(previewBytes, len(part)))
if previewBytes > 0 {
preview = append(preview, part[:min(previewBytes, len(part))]...)
}
buf := make([]byte, 0, min(maxBytes, len(part)*2))
total := 0
if len(part) > maxBytes {
tooLong = true
} else {
buf = append(buf, part...)
total = len(part)
}
for isPrefix {
part, isPrefix, err = r.ReadLine()
if err != nil {
return nil, tooLong, err
}
if previewBytes > 0 && len(preview) < previewBytes {
preview = append(preview, part[:min(previewBytes-len(preview), len(part))]...)
}
if !tooLong {
if total+len(part) > maxBytes {
tooLong = true
continue
}
buf = append(buf, part...)
total += len(part)
}
}
if tooLong {
return preview, true, nil
}
return buf, false, nil
}
func truncateBytes(b []byte, maxLen int) string {
if len(b) <= maxLen {
return string(b)
}
if maxLen < 0 {
return ""
}
return string(b[:maxLen]) + "..."
}
func normalizeText(text interface{}) string {
switch v := text.(type) {
case string:

View File

@@ -0,0 +1,31 @@
package main
import (
"strings"
"testing"
)
func TestParseJSONStream_SkipsOverlongLineAndContinues(t *testing.T) {
// Exceed the 10MB bufio.Scanner limit in parseJSONStreamInternal.
tooLong := strings.Repeat("a", 11*1024*1024)
input := strings.Join([]string{
`{"type":"item.completed","item":{"type":"other_type","text":"` + tooLong + `"}}`,
`{"type":"thread.started","thread_id":"t-1"}`,
`{"type":"item.completed","item":{"type":"agent_message","text":"ok"}}`,
}, "\n")
var warns []string
warnFn := func(msg string) { warns = append(warns, msg) }
gotMessage, gotThreadID := parseJSONStreamInternal(strings.NewReader(input), warnFn, nil, nil)
if gotMessage != "ok" {
t.Fatalf("message=%q, want %q (warns=%v)", gotMessage, "ok", warns)
}
if gotThreadID != "t-1" {
t.Fatalf("threadID=%q, want %q (warns=%v)", gotThreadID, "t-1", warns)
}
if len(warns) == 0 || !strings.Contains(warns[0], "Skipped overlong JSON line") {
t.Fatalf("expected warning about overlong JSON line, got %v", warns)
}
}

View File

@@ -78,6 +78,7 @@ type logWriter struct {
prefix string
maxLen int
buf bytes.Buffer
dropped bool
}
func newLogWriter(prefix string, maxLen int) *logWriter {
@@ -94,12 +95,12 @@ func (lw *logWriter) Write(p []byte) (int, error) {
total := len(p)
for len(p) > 0 {
if idx := bytes.IndexByte(p, '\n'); idx >= 0 {
lw.buf.Write(p[:idx])
lw.writeLimited(p[:idx])
lw.logLine(true)
p = p[idx+1:]
continue
}
lw.buf.Write(p)
lw.writeLimited(p)
break
}
return total, nil
@@ -117,21 +118,53 @@ func (lw *logWriter) logLine(force bool) {
return
}
line := lw.buf.String()
dropped := lw.dropped
lw.dropped = false
lw.buf.Reset()
if line == "" && !force {
return
}
if lw.maxLen > 0 && len(line) > lw.maxLen {
cutoff := lw.maxLen
if cutoff > 3 {
line = line[:cutoff-3] + "..."
} else {
line = line[:cutoff]
if lw.maxLen > 0 {
if dropped {
if lw.maxLen > 3 {
line = line[:min(len(line), lw.maxLen-3)] + "..."
} else {
line = line[:min(len(line), lw.maxLen)]
}
} else if len(line) > lw.maxLen {
cutoff := lw.maxLen
if cutoff > 3 {
line = line[:cutoff-3] + "..."
} else {
line = line[:cutoff]
}
}
}
logInfo(lw.prefix + line)
}
func (lw *logWriter) writeLimited(p []byte) {
if lw == nil || len(p) == 0 {
return
}
if lw.maxLen <= 0 {
lw.buf.Write(p)
return
}
remaining := lw.maxLen - lw.buf.Len()
if remaining <= 0 {
lw.dropped = true
return
}
if len(p) <= remaining {
lw.buf.Write(p)
return
}
lw.buf.Write(p[:remaining])
lw.dropped = true
}
type tailBuffer struct {
limit int
data []byte