- Introduced best practices requirements specification covering code quality, performance, maintainability, error handling, and documentation standards.
- Established quality standards with overall quality metrics and mandatory checks for security, code quality, performance, and maintainability.
- Created security requirements specification aligned with OWASP Top 10 and CWE Top 25, detailing checks and patterns for common vulnerabilities.
- Developed templates for documenting best practice findings, security findings, and generating reports, including structured markdown and JSON formats.
- Updated dependencies in the project, ensuring compatibility and stability.
- Added test files and README documentation for vector indexing tests.
- Implemented CLI commands for listing, downloading, deleting, and retrieving information about reranker models.
- Enhanced the dashboard UI to support embedding and reranker configurations with internationalization.
- Updated environment variable management for embedding and reranker settings.
- Added functionality to dynamically update model options based on selected backend.
- Improved user experience with status indicators and action buttons for model management.
- Integrated new reranker models with detailed metadata and recommendations.
- Added `rerankerModels` property to the `ProviderCredential` interface in `litellm-api-config.ts` to support additional reranker configurations.
- Introduced a numerically stable sigmoid function in `FastEmbedReranker` for score normalization.
- Updated the scoring logic in `FastEmbedReranker` to use raw float scores from the encoder and normalize them using the new sigmoid function.
- Adjusted the result mapping to maintain original document order while applying normalization.
- Introduced a comprehensive code analysis action template for integrating code exploration and analysis capabilities.
- Added LLM action template for seamless integration of LLM calls with customizable prompts and tools.
- Implemented a benchmark search script to compare multiple search methods across various dimensions including speed, result quality, ranking stability, and coverage.
- Provided preset configurations for common analysis tasks and LLM actions, enhancing usability and flexibility.
- Introduced a comprehensive template for autonomous actions, detailing structure, execution, and error handling.
- Added an orchestrator template to manage state and decision logic for autonomous actions.
- Created a sequential phase template to outline execution steps and objectives for structured workflows.
- Developed a skill documentation template to standardize the generation of skill entry files.
- Implemented a Python script to compare search results between hybrid and cascade methods, analyzing ranking changes.
1. GlobalSymbolIndex constructor: Add project_id parameter lookup
- Get project_id from registry using source_root
- Pass project_id to GlobalSymbolIndex constructor
2. Binary cascade search path handling:
- Add VectorMetadataStore import for centralized search
- Fix _build_results_from_candidates to handle centralized mode
- Use VectorMetadataStore for metadata, source_index_db for embeddings
- Properly distinguish between index_root and index_path
3. Dense reranking for centralized search:
- Get chunk metadata from _vectors_meta.db
- Group chunks by source_index_db
- Retrieve dense embeddings from respective _index.db files
- Added pre-calculation of estimated chunk count for HNSW capacity in `generate_dense_embeddings_centralized` to optimize indexing performance.
- Implemented binary vector generation with memory-mapped storage for efficient cascade search, including metadata saving.
- Introduced SPLADE sparse index generation with improved handling and metadata storage.
- Updated `ChainSearchEngine` to prefer centralized binary searcher for improved performance and added fallback to legacy binary index.
- Deprecated `BinaryANNIndex` in favor of `BinarySearcher` for better memory management and performance.
- Enhanced `SpladeEncoder` with warmup functionality to reduce latency spikes during first-time inference.
- Improved `SpladeIndex` with cache size adjustments for better query performance.
- Added methods for managing binary vectors in `VectorMetadataStore`, including batch insertion and retrieval.
- Created a new `BinarySearcher` class for efficient binary vector search using Hamming distance, supporting both memory-mapped and database loading modes.
- Added centralized SPLADE database and vector storage configuration in config.py.
- Updated embedding_manager.py to support centralized SPLADE database path.
- Enhanced generate_embeddings and generate_embeddings_recursive functions for centralized storage.
- Introduced centralized ANN index creation in ann_index.py.
- Modified hybrid_search.py to utilize centralized vector index for searches.
- Implemented methods to discover and manage centralized SPLADE and HNSW files.
- Added category support for programming and documentation languages in Config.
- Implemented category-based filtering in HybridSearchEngine to improve search relevance based on query intent.
- Introduced functions for filtering results by category and determining file categories based on extensions.
- Updated VectorStore to include a category column in the database schema and modified chunk addition methods to support category tagging.
- Enhanced the WatcherConfig to ignore additional common directories and files.
- Created a benchmark script to compare performance between Binary Cascade, SPLADE, and Vector semantic search methods, including detailed result analysis and overlap comparison.
- Implemented `get_all_chunks` method in `VectorStore` class to fetch all semantic chunks from the database.
- Added a new benchmark script `analyze_methods.py` for analyzing hybrid search methods and storage architecture.
- Included detailed analysis of method contributions, storage conflicts, and FTS + Rerank fusion experiments.
- Updated results JSON structure to reflect new analysis outputs and method performance metrics.
- Implemented BinaryEmbeddingBackend for fast coarse filtering using 256-dimensional binary vectors.
- Developed DenseEmbeddingBackend for high-precision dense vectors (2048 dimensions) for reranking.
- Created CascadeEmbeddingBackend to combine binary and dense embeddings for two-stage retrieval.
- Introduced utility functions for embedding conversion and distance computation.
chore: Migration 010 - Add multi-vector storage support
- Added 'chunks' table to support multi-vector embeddings for cascade retrieval.
- Included new columns: embedding_binary (256-dim) and embedding_dense (2048-dim) for efficient storage.
- Implemented upgrade and downgrade functions to manage schema changes and data migration.
- Added `splade_encoder.py` for ONNX-optimized SPLADE encoding, including methods for encoding text and batch processing.
- Created `SPLADE_IMPLEMENTATION.md` to document the SPLADE encoder's functionality, design patterns, and integration points.
- Introduced migration script `migration_009_add_splade.py` to add SPLADE metadata and posting list tables to the database.
- Developed `splade_index.py` for managing the SPLADE inverted index, supporting efficient sparse vector retrieval.
- Added verification script `verify_watcher.py` to test FileWatcher event filtering and debouncing functionality.
- Implemented GraphExpander to enhance search results with related symbols using precomputed neighbors.
- Added CrossEncoderReranker for second-stage search ranking, allowing for improved result scoring.
- Created migrations to establish necessary database tables for relationships and graph neighbors.
- Developed tests for graph expansion functionality, ensuring related results are populated correctly.
- Enhanced performance benchmarks for cross-encoder reranking latency and graph expansion overhead.
- Updated schema cleanup tests to reflect changes in versioning and deprecated fields.
- Added new test cases for Treesitter parser to validate relationship extraction with alias resolution.
Prevents errors when HNSW search returns null/empty results due to race conditions.
Adds validation for ids and distances before zip operation.
Fixes: ISS-1766921318981-5
Solution-ID: SOL-1735386000-5
Issue-ID: ISS-1766921318981-5
Task-ID: T1
Replaces generic Exception handling with specific PermissionError and OSError
handling in __post_init__ and ensure_runtime_dirs(). Provides clear diagnostic
messages to distinguish permission issues from other filesystem errors.
Solution-ID: SOL-1735385400008
Issue-ID: ISS-1766921318981-8
Task-ID: T1
Protect _bulk_insert_mode flag and accumulation lists with
_ann_write_lock to prevent corruption during concurrent access.
Solution-ID: SOL-1735392000003
Issue-ID: ISS-1766921318981-12
Task-ID: T1
Adds nested exception handling in add_files() and _migrate_fts_to_external()
to catch and log rollback failures. Uses exception chaining to preserve both
transaction and rollback errors, preventing silent database inconsistency.
Solution-ID: SOL-1735385400010
Issue-ID: ISS-1766921318981-10
Task-ID: T1
Add comprehensive test coverage for zero and near-zero vector
detection in SemanticChunk embedding validation.
Solution-ID: SOL-20251228113612
Issue-ID: ISS-1766921318981-7
Task-ID: T2
Add L2 norm check to SemanticChunk.validate_embedding to reject
zero vectors. Prevents division by zero in cosine similarity
calculations downstream in vector search.
Solution-ID: SOL-20251228113612
Issue-ID: ISS-1766921318981-7
Task-ID: T1
Adds robust exception handling for os.path.commonpath() in search_symbols()
to prevent crashes on malformed paths and Windows cross-drive scenarios.
Invalid symbols are skipped with debug logging, search continues.
Solution-ID: SOL-1735385400004
Issue-ID: ISS-1766921318981-4
Task-ID: T1
Add fallback validation to detect dead threads missed by
threading.enumerate(), ensuring all stale connections are cleaned.
Solution-ID: SOL-1735392000002
Issue-ID: ISS-1766921318981-3
Task-ID: T2
