Files
Claude-Code-Workflow/ccw/docs/GRAPH_EXPLORER_FIX.md
catlog22 df23975a0b Add comprehensive tests for schema cleanup migration and search comparison
- Implement tests for migration 005 to verify removal of deprecated fields in the database schema.
- Ensure that new databases are created with a clean schema.
- Validate that keywords are correctly extracted from the normalized file_keywords table.
- Test symbol insertion without deprecated fields and subdir operations without direct_files.
- Create a detailed search comparison test to evaluate vector search vs hybrid search performance.
- Add a script for reindexing projects to extract code relationships and verify GraphAnalyzer functionality.
- Include a test script to check TreeSitter parser availability and relationship extraction from sample files.
2025-12-16 19:27:05 +08:00

6.0 KiB

Graph Explorer Fix - Migration 005 Compatibility

Issue Description

The CCW Dashboard's Graph Explorer view was broken after codex-lens migration 005, which cleaned up unused database fields.

Root Cause

Migration 005 removed unused/redundant columns from the codex-lens database:

  • symbols.token_count (unused, always NULL)
  • symbols.symbol_type (redundant duplicate of kind)

However, ccw/src/core/routes/graph-routes.ts was still querying these removed columns, causing SQL errors:

// BEFORE (broken):
SELECT
  s.id,
  s.name,
  s.kind,
  s.start_line,
  s.token_count,      // ❌ Column removed in migration 005
  s.symbol_type,      // ❌ Column removed in migration 005
  f.path as file
FROM symbols s

This resulted in database query failures when trying to load the graph visualization.

Fix Applied

Updated graph-routes.ts to match the new database schema (v5):

1. Updated GraphNode Interface

Before:

interface GraphNode {
  id: string;
  name: string;
  type: string;
  file: string;
  line: number;
  docstring?: string;    // ❌ Removed (no longer available)
  tokenCount?: number;   // ❌ Removed (no longer available)
}

After:

interface GraphNode {
  id: string;
  name: string;
  type: string;
  file: string;
  line: number;
}

2. Updated SQL Query

Before:

SELECT
  s.id,
  s.name,
  s.kind,
  s.start_line,
  s.token_count,    // ❌ Removed
  s.symbol_type,    // ❌ Removed
  f.path as file
FROM symbols s

After:

SELECT
  s.id,
  s.name,
  s.kind,
  s.start_line,
  f.path as file
FROM symbols s

3. Updated Row Mapping

Before:

return rows.map((row: any) => ({
  id: `${row.file}:${row.name}:${row.start_line}`,
  name: row.name,
  type: mapSymbolKind(row.kind),
  file: row.file,
  line: row.start_line,
  docstring: row.symbol_type || undefined,   // ❌ Removed
  tokenCount: row.token_count || undefined,  // ❌ Removed
}));

After:

return rows.map((row: any) => ({
  id: `${row.file}:${row.name}:${row.start_line}`,
  name: row.name,
  type: mapSymbolKind(row.kind),
  file: row.file,
  line: row.start_line,
}));

4. Updated API Documentation

Updated graph-routes.md to reflect the simplified schema without the removed fields.

How to Use Graph Explorer

Prerequisites

  1. CodexLens must be installed and initialized:

    pip install -e codex-lens/
    
  2. Project must be indexed:

    # Via CLI
    codex init <project-path>
    
    # Or via CCW Dashboard
    # Navigate to "CodexLens" view → Click "Initialize" → Select project
    

    This creates the _index.db database at ~/.codexlens/indexes/<normalized-path>/_index.db

  3. Symbols and relationships must be extracted:

    • CodexLens automatically indexes symbols during init
    • Requires TreeSitter parsers for your programming language
    • Relationships are extracted via migration 003 (code_relationships table)

Accessing Graph Explorer

  1. Start CCW Dashboard:

    ccw view
    
  2. Navigate to Graph Explorer:

    • Click the "Graph" icon in the left sidebar (git-branch icon)
    • Or use keyboard shortcut if configured
  3. View Code Structure:

    • Code Relations Tab: Interactive graph visualization of symbols and their relationships
    • Search Process Tab: Visualizes search pipeline steps (experimental)

Graph Controls

Toolbar (top-right):

  • Fit View: Zoom to fit all nodes in viewport
  • Center: Center the graph
  • Reset Filters: Clear all node/edge type filters
  • Refresh: Reload data from database

Sidebar Filters:

  • Node Types: Filter by MODULE, CLASS, FUNCTION, METHOD, VARIABLE
  • Edge Types: Filter by CALLS, IMPORTS, INHERITS
  • Legend: Color-coded guide for node/edge types

Interaction:

  • Click node: Show details panel with symbol information
  • Drag nodes: Rearrange graph layout
  • Scroll: Zoom in/out
  • Pan: Click and drag on empty space

API Endpoints

The Graph Explorer uses these REST endpoints:

  1. GET /api/graph/nodes

    • Returns all symbols as graph nodes
    • Query param: path (optional, defaults to current project)
  2. GET /api/graph/edges

    • Returns all code relationships as graph edges
    • Query param: path (optional)
  3. GET /api/graph/impact

    • Returns impact analysis for a symbol
    • Query params: path, symbol (required, format: file:name:line)

Verification

To verify the fix works:

  1. Ensure project is indexed:

    ls ~/.codexlens/indexes/
    # Should show your project path
    
  2. Check database has symbols:

    sqlite3 ~/.codexlens/indexes/<your-project>/_index.db "SELECT COUNT(*) FROM symbols"
    # Should return > 0
    
  3. Check schema version:

    sqlite3 ~/.codexlens/indexes/<your-project>/_index.db "PRAGMA user_version"
    # Should return: 5 (after migration 005)
    
  4. Test Graph Explorer:

    • Open CCW dashboard: ccw view
    • Navigate to Graph view
    • Should see nodes/edges displayed without errors
  • Implementation: ccw/src/core/routes/graph-routes.ts
  • Frontend: ccw/src/templates/dashboard-js/views/graph-explorer.js
  • Styles: ccw/src/templates/dashboard-css/14-graph-explorer.css
  • API Docs: ccw/src/core/routes/graph-routes.md
  • Migration: codex-lens/src/codexlens/storage/migrations/migration_005_cleanup_unused_fields.py

Impact

  • Breaking Change: Graph Explorer required codex-lens database schema v5
  • Data Loss: None (removed fields were unused or redundant)
  • Compatibility: Graph Explorer now works correctly with migration 005+
  • Future: All CCW features requiring codex-lens database access must respect schema version 5

References

  • Migration 005 Documentation: codex-lens/docs/MIGRATION_005_SUMMARY.md
  • Graph Routes API: ccw/src/core/routes/graph-routes.md
  • CodexLens Schema: codex-lens/src/codexlens/storage/dir_index.py