Commit Graph

1396 Commits

Author SHA1 Message Date
catlog22
92ed2524b7 feat: Enhance SPLADE indexing command to support multiple index databases and add chunk ID management 2026-01-02 13:25:23 +08:00
catlog22
56c03c847a feat: Add method to retrieve all semantic chunks from the vector store
- Implemented `get_all_chunks` method in `VectorStore` class to fetch all semantic chunks from the database.
- Added a new benchmark script `analyze_methods.py` for analyzing hybrid search methods and storage architecture.
- Included detailed analysis of method contributions, storage conflicts, and FTS + Rerank fusion experiments.
- Updated results JSON structure to reflect new analysis outputs and method performance metrics.
2026-01-02 12:32:43 +08:00
catlog22
9129c981a4 feat: Enhance BinaryANNIndex with vectorized search and performance benchmarking 2026-01-02 11:49:54 +08:00
catlog22
da68ba0b82 feat: Implement cascade indexing command and benchmark script for performance evaluation 2026-01-02 11:24:06 +08:00
catlog22
e21d801523 feat: Add multi-type embedding backends for cascade retrieval
- Implemented BinaryEmbeddingBackend for fast coarse filtering using 256-dimensional binary vectors.
- Developed DenseEmbeddingBackend for high-precision dense vectors (2048 dimensions) for reranking.
- Created CascadeEmbeddingBackend to combine binary and dense embeddings for two-stage retrieval.
- Introduced utility functions for embedding conversion and distance computation.

chore: Migration 010 - Add multi-vector storage support

- Added 'chunks' table to support multi-vector embeddings for cascade retrieval.
- Included new columns: embedding_binary (256-dim) and embedding_dense (2048-dim) for efficient storage.
- Implemented upgrade and downgrade functions to manage schema changes and data migration.
2026-01-02 10:52:43 +08:00
catlog22
195438d26a feat(splade): add cache directory support for ONNX models and improve thread-local database connection handling 2026-01-01 22:40:00 +08:00
catlog22
5bb01755bc Implement SPLADE sparse encoder and associated database migrations
- Added `splade_encoder.py` for ONNX-optimized SPLADE encoding, including methods for encoding text and batch processing.
- Created `SPLADE_IMPLEMENTATION.md` to document the SPLADE encoder's functionality, design patterns, and integration points.
- Introduced migration script `migration_009_add_splade.py` to add SPLADE metadata and posting list tables to the database.
- Developed `splade_index.py` for managing the SPLADE inverted index, supporting efficient sparse vector retrieval.
- Added verification script `verify_watcher.py` to test FileWatcher event filtering and debouncing functionality.
2026-01-01 17:41:22 +08:00
catlog22
520f2d26f2 feat(codex-lens): add unified reranker architecture and file watcher
Unified Reranker Architecture:
- Add BaseReranker ABC with factory pattern
- Implement 4 backends: ONNX (default), API, LiteLLM, Legacy
- Add .env configuration parsing for API credentials
- Migrate from sentence-transformers to optimum+onnxruntime

File Watcher Module:
- Add real-time file system monitoring with watchdog
- Implement IncrementalIndexer for single-file updates
- Add WatcherManager with signal handling and graceful shutdown
- Add 'codexlens watch' CLI command
- Event filtering, debouncing, and deduplication
- Thread-safe design with proper resource cleanup

Tests: 16 watcher tests + 5 reranker test files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 13:23:52 +08:00
catlog22
8ac27548ad refactor(lite-execute): unify task prompt builder with scope+modification_points
- Consolidate Agent and CLI prompt builders into single buildExecutionPrompt()
- Replace file-based task format with scope + modification_points structure
- Simplify to 4-part task template: Modification Points → How → Reference → Done
- Remove duplicate formatTaskForCLI and buildCLIPrompt functions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 12:46:17 +08:00
catlog22
adc0dd23e4 feat(docs): add swagger-docs command for RESTful API documentation
Add new /memory:swagger-docs command to generate complete Swagger/OpenAPI
documentation following RESTful standards with:
- Global security configuration (Bearer Token auth)
- Complete endpoint documentation with parameters and responses
- Unified error code specification (AUTH/PARAM/BIZ/SYS)
- Multi-language support (--lang zh|en, default: zh)
- Validation test reports

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 11:41:57 +08:00
catlog22
31a45f1f30 Add graph expansion and cross-encoder reranking features
- Implemented GraphExpander to enhance search results with related symbols using precomputed neighbors.
- Added CrossEncoderReranker for second-stage search ranking, allowing for improved result scoring.
- Created migrations to establish necessary database tables for relationships and graph neighbors.
- Developed tests for graph expansion functionality, ensuring related results are populated correctly.
- Enhanced performance benchmarks for cross-encoder reranking latency and graph expansion overhead.
- Updated schema cleanup tests to reflect changes in versioning and deprecated fields.
- Added new test cases for Treesitter parser to validate relationship extraction with alias resolution.
2025-12-31 16:58:59 +08:00
catlog22
4bde13e83a chore: bump version to 6.3.18
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-30 22:16:52 +08:00
catlog22
a5ab3f8b26 feat(docs): enhance issue planning and queue documentation with new guidelines and commands 2025-12-30 22:15:16 +08:00
catlog22
d183a647dd chore: bump version to 6.3.17
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-30 20:02:59 +08:00
catlog22
e5797bff8f feat(docs): add guidelines for Git operations to ensure parallel task safety 2025-12-30 19:56:31 +08:00
catlog22
4d73a3c9a9 feat(cli): add streaming output to Dashboard and child process termination
- Add broadcastStreamEvent() for real-time CLI output to Dashboard
- Send CLI_EXECUTION_STARTED/OUTPUT/COMPLETED events via /api/hook
- Add killCurrentCliProcess() to terminate child CLI on SIGINT/SIGTERM
- Track child process reference for cleanup on interruption
- Make CLI stream viewer panel full-height (calc(100vh - 32px))
- Fix completion status not updating by using consistent executionId

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-30 15:56:18 +08:00
catlog22
754cddd4ad fix(python): improve Python detection and pip command reliability
- Add shared python-utils.ts module for consistent Python detection
- Use `python -m pip` instead of direct pip command (fixes "pip not found")
- Support CCW_PYTHON env var for custom Python path
- Use Windows py launcher to find compatible versions (3.9-3.12)
- Warn users when Python version may not be compatible with onnxruntime

Fixes issues where users couldn't install ccw-litellm due to:
- pip not in PATH
- Only pip3 available (not pip)
- Python 3.13+ without onnxruntime support

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-30 15:23:44 +08:00
catlog22
f6cc3736b2 feat(cli): support stdin input for issue creation and solution commands 2025-12-30 14:51:25 +08:00
catlog22
6e99cd97ca feat(docs): update Bash tool usage to enforce foreground execution for CLI calls 2025-12-30 14:28:32 +08:00
catlog22
f566b8aabc feat(cli): add --debug/-d flag for CLI debugging
- Add -d/--debug option to ccw cli command
- Enable debug logging at runtime when flag is set
- Change DEBUG check from const to function for runtime evaluation
- Support debug mode for both exec and status subcommands
- Update help text to include --debug option

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 23:26:48 +08:00
catlog22
6efc499c77 feat(cli): add detailed debug logging for CLI execution
- Add debugLog and errorLog utility functions with DEBUG env control
- Add logging for tool availability check (TOOL_CHECK)
- Add logging for command building (BUILD_CMD)
- Add logging for process spawn (SPAWN, STDIN)
- Add logging for process completion (CLOSE, STATUS)
- Enhance error output with command details, exit code, and stderr
- Add troubleshooting hints for common failures
- Support DEBUG=true or CCW_DEBUG=true environment variables

Closes #46

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 23:17:39 +08:00
catlog22
2c675ee4db chore: bump version to 6.3.14
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
v6.3.14
2025-12-29 22:57:02 +08:00
catlog22
f6dfe28e08 refactor(issue): rename 'labels' to 'tags' in issue schemas and related code 2025-12-29 22:55:33 +08:00
catlog22
e8e8746cc6 refactor(issue): update brief mode to use 'labels' instead of 'tags' in issue listing 2025-12-29 22:47:55 +08:00
catlog22
603bc00bca refactor(issue): enhance documentation and add core guidelines for data access 2025-12-29 22:41:10 +08:00
catlog22
ae76926d5a feat(queue): support solution-based queues and update metadata handling 2025-12-29 22:14:06 +08:00
catlog22
fd48045fe3 Refactor issue management and history tracking
- Removed the interactive issue management command and its associated documentation.
- Enhanced the issue planning command to streamline project context reading and solution creation.
- Improved queue management with conflict clarification and status syncing from queues.
- Added functionality to track completed issues, moving them to a history file upon completion.
- Updated CLI options to support syncing issue statuses from queues.
- Introduced new API endpoint for retrieving completed issues from history.
- Enhanced error handling and validation for issue updates and queue management.
2025-12-29 21:42:06 +08:00
catlog22
6ec6643448 refactor(issue-plan-agent): enhance conflict detection and execution flow 2025-12-29 20:36:56 +08:00
catlog22
945fda2d14 docs(issue): update agent/prompts to use CLI endpoints
- issue-plan-agent.md: Use `ccw issue solution` instead of Bash echo
- issue-plan.md (codex): Use `ccw issue solution` endpoint
- issue-queue.md (codex): Remove assigned_executor from schema

All solution creation now uses the dedicated CLI endpoint for:
- Auto-increment ID: SOL-{issue-id}-{seq}
- Proper JSONL format with trailing newline
- Multi-solution support per issue

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 20:28:39 +08:00
catlog22
7d71f603fe feat(issue): add solution endpoint with auto-increment ID
- Add `ccw issue solution <id> --data '{...}'` for solution creation
- Add createSolution() with proper JSONL handling (trailing newline)
- Fix writeSolutions() to always add trailing newline
- Update plan.md to use CLI endpoint

Supports multiple solutions per issue with sequential IDs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 20:24:31 +08:00
catlog22
bd11a538a7 docs(issue/new): update Phase 5 to use ccw issue create endpoint
Replace manual Bash echo with CLI endpoint documentation:
- Auto-increment ID: ISS-YYYYMMDD-NNN
- Proper JSONL format with trailing newline
- JSON output with created issue

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 20:16:24 +08:00
catlog22
b9b4da6d8c feat(issue): add create endpoint with auto-increment ID
- Add `ccw issue create --data '{...}'` for programmatic issue creation
- Add generateIssueId() for auto-increment: ISS-YYYYMMDD-NNN
- Add createIssue() with proper JSONL handling (trailing newline)
- Fix writeIssues() to always add trailing newline

Fixes JSONL format corruption when appending via shell echo.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 20:12:57 +08:00
catlog22
70f8b14eaa refactor(vector_store): use safer SQL query construction pattern
Replaces f-string interpolation with safer string formatting.
Adds documentation on SQL injection prevention.
No functional changes - parameterized queries still used.

Fixes: ISS-1766921318981-9

Solution-ID: SOL-1735386000-9
Issue-ID: ISS-1766921318981-9
Task-ID: T1
2025-12-29 20:09:49 +08:00
catlog22
0c8b2f2ec9 fix(vector_store): add bounds checking for chunk ID generation
Prevents potential integer overflow when start_id is near sys.maxsize.
Adds validation before range() calculation in batch insert methods.

Fixes: ISS-1766921318981-6

Solution-ID: SOL-1735386000-6
Issue-ID: ISS-1766921318981-6
Task-ID: T1
2025-12-29 20:02:19 +08:00
catlog22
d532b3fd02 refactor(queue): streamline output requirements and storage structure documentation 2025-12-29 19:54:10 +08:00
catlog22
c56104c082 fix(vector_store): add null check for ANN search results before filtering
Prevents errors when HNSW search returns null/empty results due to race conditions.
Adds validation for ids and distances before zip operation.

Fixes: ISS-1766921318981-5

Solution-ID: SOL-1735386000-5
Issue-ID: ISS-1766921318981-5
Task-ID: T1
2025-12-29 19:53:32 +08:00
catlog22
66ae1972ae refactor(issue): remove assigned_executor from QueueItem interface
Queue is an execution plan without executor assignment.
Executor is determined at runtime by /issue:execute command.

Changes:
- Remove assigned_executor from QueueItem interface
- Remove from dag nodes, list display, execution_hints
- Simplify queue list output (remove Executor column)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 19:49:25 +08:00
catlog22
7f4433e449 fix(vector_store): add parameter validation for min_score range
Validates min_score is within [0.0, 1.0] for cosine similarity.
Raises ValueError for out-of-range values to prevent unexpected filtering.

Fixes: ISS-1766921318981-14

Solution-ID: SOL-1735386000-14
Issue-ID: ISS-1766921318981-14
Task-ID: T1
2025-12-29 19:46:26 +08:00
catlog22
e1f2fc72d9 refactor(schema): remove assigned_executor from queue-schema
Queue produces execution plan without executor assignment.
Executor is determined at runtime by /issue:execute command.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 19:45:43 +08:00
catlog22
aa093f9468 refactor(issue/queue): remove assigned_executor field from schema
Executor assignment is not needed in solution schema - execution
context is determined at runtime.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 19:42:27 +08:00
catlog22
a27f76abcb refactor(issue/queue): simplify Phase 1 and Phase 5 with semantic descriptions
- Replace complex JavaScript code in Phase 1 with semantic bullet points
- Simplify Phase 5 validation code to semantic description
- Add Quality Checklist for completion verification
- Consistent style with plan.md optimizations

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 19:41:32 +08:00
catlog22
df34ef38d9 refactor(issue/plan): update semantic grouping limit to 4 issues per group and enhance conflict handling process 2025-12-29 19:38:45 +08:00
catlog22
60fbb4177c fix(config): add specific exception handling for path operations
Replaces generic Exception handling with specific PermissionError and OSError
handling in __post_init__ and ensure_runtime_dirs(). Provides clear diagnostic
messages to distinguish permission issues from other filesystem errors.

Solution-ID: SOL-1735385400008
Issue-ID: ISS-1766921318981-8
Task-ID: T1
2025-12-29 19:34:27 +08:00
catlog22
3289562be7 docs(issue/plan): add quality checklist for completion verification
Checklist items:
- Solution files exist for all issues
- Auto-binding for single solutions
- User selection for multi-solutions
- Tasks have modification_points
- Acceptance criteria quantified
- Conflicts detected
- Status updated to planned

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 19:32:19 +08:00
catlog22
73fc68a187 refactor(issue/plan): remove redundant file specs from command doc
Agent (issue-plan-agent.md) handles file generation.
Command doc should only describe input/output behavior.

Removed: Generate Files, Storage Structure, Completion Criteria
Kept: Command Output JSON format, Behavior summary

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 19:31:00 +08:00
catlog22
bce6fa7a91 refactor(issue/new): simplify clarification to open-ended prompt
Remove prescriptive options (Bug fix/New feature/Improvement)
that may interfere with natural issue description.

Now: single open-ended prompt for user to describe freely.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 19:28:03 +08:00
catlog22
88724a4df9 refactor(issue): simplify Issue schema and remove lifecycle_requirements
Schema changes:
- Remove lifecycle_requirements (deferred to plan agent)
- Remove solution_count (computed dynamically from solutions/*.jsonl)
- Keep context as single source of truth (remove problem_statement)
- Add feedback[] for failure history + human clarifications
- Add source, source_url, labels, expected/actual_behavior

Fields removed (unused downstream):
- lifecycle_requirements.test_strategy
- lifecycle_requirements.regression_scope
- lifecycle_requirements.acceptance_type
- lifecycle_requirements.commit_strategy
- solution_count (denormalized)

Fields added:
- feedback[]: { type, stage, content, created_at }
- source, source_url, labels
- expected_behavior, actual_behavior, affected_components

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 19:25:58 +08:00
catlog22
5914b1c5fc fix(vector-store): protect bulk insert mode transitions with lock
Ensure begin_bulk_insert() and end_bulk_insert() are fully
lock-protected to prevent TOCTOU race conditions.

Solution-ID: SOL-1735392000003
Issue-ID: ISS-1766921318981-12
Task-ID: T2
2025-12-29 19:20:02 +08:00
catlog22
d8be23fa83 fix(vector-store): add lock protection for bulk insert mode flag
Protect _bulk_insert_mode flag and accumulation lists with
_ann_write_lock to prevent corruption during concurrent access.

Solution-ID: SOL-1735392000003
Issue-ID: ISS-1766921318981-12
Task-ID: T1
2025-12-29 19:16:30 +08:00
catlog22
ffbc4a4b76 style(issue/new): use hybrid Phase + table flow diagram
Combines sequential Phase structure with inline table for branching:
- Phase 1-4 for overall flow clarity
- Table for Score 0/1-2/3 decision branching
- More compact and scannable than pure ASCII boxes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 19:16:25 +08:00