Fix path handling for git repos and improve init command - #15
Merged
mrsimpson merged 3 commits intoNov 29, 2025
Merged
Conversation
This commit addresses the issues with how paths are handled during docset initialization, following TDD principles. ## Changes Made: ### 1. Separate --force and --discover-paths flags - `--force`: Clears and re-initializes docset directory - `--discover-paths`: Discovers and updates config with directory patterns - These can be used independently or together ### 2. Directory cleanup on force re-init - When using `--force`, the docset directory is now completely cleared before re-initialization - Fixes issue where old files accumulated when paths configuration changed - Applies to both git repos and local folder symlinks ### 3. Path discovery function - Created `discoverDirectoryPatterns()` to convert file lists to directory patterns - Instead of storing 50+ individual file paths, stores directory patterns like "docs/", "examples/", etc. - Reduces config file bloat and makes paths configuration cleaner ### 4. Export removeSymlinks utility - Made `removeSymlinks()` available for cleanup operations - Directory clearing handles both regular files and symlinks ### Test Coverage: - Added comprehensive tests for path discovery function (9 tests) - Added tests for symlink cleanup behavior (6 tests) - Added tests for init command behavior (documentation tests) - All existing tests still pass (115 core tests, 31 CLI tests) ## Usage: # Initialize with current paths in config: agentic-knowledge-mcp init <docset-id> # Force re-init (clears directory, re-extracts): agentic-knowledge-mcp init <docset-id> --force # Discover and update paths in config: agentic-knowledge-mcp init <docset-id> --discover-paths # Both together (clear, re-extract, update config): agentic-knowledge-mcp init <docset-id> --force --discover-paths Fixes: Path configuration issues with git repo docsets
CRITICAL: Ensure source files are NEVER deleted when clearing docset
directories that contain symlinks to local_folder sources.
## Safety Features Added:
### 1. Comprehensive Safety Tests
- Added 5 tests verifying that fs.rm() doesn't follow symlinks
- Tests confirm source files are preserved when clearing target directory
- Tests cover single files, nested directories, and mixed content
### 2. Safe Cleanup Utilities
Created `packages/core/src/paths/cleanup.ts` with:
- `safelyClearDirectory()`: Explicitly documented safe cleanup
- `containsSymlinks()`: Detect symlinks in directory
- `getDirectoryInfo()`: Get counts of files/dirs/symlinks for logging
### 3. Enhanced Init Command
- Uses `safelyClearDirectory()` instead of raw `fs.rm()`
- Logs what's being removed (files, dirs, symlinks)
- Warns user when symlinks are present that source files are preserved
- Makes safety guarantees explicit in user-facing messages
## Test Results:
- ✅ 5 local folder safety tests
- ✅ 9 cleanup utility tests
- ✅ All existing 115 core tests still pass
## Node.js Behavior Documented:
Node.js fs.rm() does NOT follow symlinks by default - it only removes
the symlink itself, not the target. This is the correct and safe
behavior, but we now make this explicit in our code and logging.
Example output when force re-initializing:
```
🗑️ Clearing existing directory...
Removing: 10 files, 2 dirs, 1 symlinks
⚠️ Note: Symlinks will be removed, but source files are preserved
```
Safety is paramount when dealing with user data!
Updated USER_GUIDE.md with comprehensive documentation for the new init command features and safety guarantees. ## Documentation Updates: ### 1. Enhanced init Command Section - Documented `--force` flag behavior (clears directory before re-init) - Documented new `--discover-paths` flag (auto-discovers optimal paths) - Explained how flags can be used independently or together - Added safety guarantees for local folder sources ### 2. Command Comparison Table Added clear comparison between init, refresh, and their flag variants: - When to use each command - What each command does - Which commands modify configuration - Decision guide for choosing the right command ### 3. Path Configuration Strategies Documented three approaches: - Explicit paths (manual, recommended for control) - Auto-discovery (good for initial setup) - Smart filtering (when no paths specified) ### 4. New Workflow Examples **Workflow 4: Auto-Discovering Optimal Paths** - How to use --discover-paths to avoid config bloat - Shows how 100+ file paths become clean directory patterns **Workflow 5: Managing Path Configuration Changes** - How to use --force when changing path configuration - Example output users will see - Step-by-step process ### 5. Enhanced Troubleshooting **Changed Paths But Still Seeing Old Files** - Solution: Use `init --force` **Too Many Individual File Paths in Config** - Solution: Use `init --force --discover-paths` **Worried About Deleting Source Files** - Explicit safety guarantee - Example of safe operation with output - Explanation of symlink behavior ### 6. Safety Messaging Throughout - Emphasized that source files are NEVER deleted - Node.js does not follow symlinks when removing directories - Clear examples of safety messages users will see All documentation now accurately reflects the new behavior while emphasizing safety and providing clear guidance.
mrsimpson
deleted the
claude/review-path-handling-01FVNWT2mogG2KAVsT91d2Jq
branch
November 29, 2025 19:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit addresses the issues with how paths are handled during
docset initialization, following TDD principles.
Changes Made:
1. Separate --force and --discover-paths flags
--force: Clears and re-initializes docset directory--discover-paths: Discovers and updates config with directory patterns2. Directory cleanup on force re-init
--force, the docset directory is now completely clearedbefore re-initialization
3. Path discovery function
discoverDirectoryPatterns()to convert file lists todirectory patterns
like "docs/", "examples/", etc.
4. Export removeSymlinks utility
removeSymlinks()available for cleanup operationsTest Coverage:
Usage:
Initialize with current paths in config:
agentic-knowledge-mcp init
Force re-init (clears directory, re-extracts):
agentic-knowledge-mcp init --force
Discover and update paths in config:
agentic-knowledge-mcp init --discover-paths
Both together (clear, re-extract, update config):
agentic-knowledge-mcp init --force --discover-paths
Fixes: Path configuration issues with git repo docsets