Skip to content

feat: MCP server refactor with fine-grained tools and incremental update support#66

Open
mambo-wang wants to merge 27 commits into
FSoft-AI4Code:mainfrom
mambo-wang:feat/mcp-refactor-and-incremental
Open

feat: MCP server refactor with fine-grained tools and incremental update support#66
mambo-wang wants to merge 27 commits into
FSoft-AI4Code:mainfrom
mambo-wang:feat/mcp-refactor-and-incremental

Conversation

@mambo-wang

Copy link
Copy Markdown

Summary

  • Refactor MCP server into fine-grained, IDE-driven tools (analyze_repo, read_code_components, view_repo_file, write_doc_file, edit_doc_file, save_module_tree, get_processing_order, get_prompt, close_session) alongside legacy tools, enabling zero-LLM-config documentation workflows
  • Add incremental update support (--update / commit_id) to MCP analyze_repo, so only changed modules are re-analyzed
  • Fix commit_id passthrough to metadata.json in CLI mode for --update support
  • Add missing MCP dependencies to pyproject.toml

Test plan

  • Verify MCP server starts via python -m codewiki.mcp.server and all new tools are listed
  • Run analyze_repo tool, confirm session is created and dependency graph is built
  • Run analyze_repo again with commit_id to verify incremental update works correctly
  • Verify CLI --update flag passes commit_id to metadata.json as expected
  • Test legacy tools (generate_docs, get_module_tree) still function

- Add IDE_DRIVEN_GUIDE.md with complete walkthrough for using CodeWiki
  with AI IDEs (CodeBuddy, Cursor, Claude Desktop) via MCP
- Update README with IDE-Driven Mode section and navigation link
- 8 个模块文档(Agent 工具、CLI 工具、CLI 核心、MCP 服务、依赖分析器、共享配置、前端服务、后端核心)
- 仓库总览 overview.md
- 模块聚类树 module_tree.json
- 全部文档含 Mermaid 架构图,语法校验通过
- Add _detect_changes() with git diff + mtime dual-strategy detection
- Add _find_affected_modules() to map changed files to affected modules
- analyze_repo now returns a 'changes' field with affected/cascade modules
- Decouple codewiki/__init__.py from CLI imports for lightweight MCP startup
- Update skill and IDE_DRIVEN_GUIDE.md with incremental update docs
Previously, CLIDocumentationGenerator never received or forwarded the
git commit SHA, so metadata.json always had commit_id: null. This made
--update fall back to full regeneration every time. Now the commit hash
is obtained before generator creation and threaded through to the
backend DocumentationGenerator, matching the behavior already present
in Web mode (background_worker.py).
Square logo for GitHub repo avatar and wide banner for README header.
Design follows the blue-purple-green gradient palette from the
original CodeWiki framework diagram, with a red CN badge for branding.
Square logo corners and banner surrounding area are now transparent
instead of white/light-gray, suitable for any background color.
Remove transparent padding around the rounded rectangle by filling
corners with a matching dark navy gradient, making the banner a
solid rectangle.
- Add _detect_changes() with git diff + mtime dual-strategy detection
- Add _find_affected_modules() to map changed files to affected modules
- analyze_repo now returns a 'changes' field with affected/cascade modules
- Decouple codewiki/__init__.py from CLI imports for lightweight MCP startup
- Update skill and IDE_DRIVEN_GUIDE.md with incremental update docs
Previously, CLIDocumentationGenerator never received or forwarded the
git commit SHA, so metadata.json always had commit_id: null. This made
--update fall back to full regeneration every time. Now the commit hash
is obtained before generator creation and threaded through to the
backend DocumentationGenerator, matching the behavior already present
in Web mode (background_worker.py).
- Reduce component_index from 500 to 100 items per page (max 200),
  drop depends_on from each entry (available via read_code_components)
- Add offset/limit params to analyze_repo for pagination
- Add list_components tool for browsing components without re-analysis
- Reduce leaf_nodes from 100 to 50
- Remove IDE rule files, consolidate into skill files
… hang

- Wrap synchronous tool handlers in asyncio.to_thread() to avoid
  blocking the event loop (analyze_repo excluded — Tree-sitter C
  extensions are not thread-safe)
- Disable mermaid-py validation by default (set MERMAID_VALIDATE=1
  to enable), add 15s timeout to prevent indefinite hangs
- Fix shell injection in view_repo_file: replace shell=True subprocess
  with pathlib iteration
- Add path traversal guards in view_repo_file, write_doc_file,
  edit_doc_file (reject paths escaping repo/output dir)
- Add threading.Lock to SessionStore for concurrent access safety
- Cap max sessions to 10, evict oldest when full
- Cap read_code_components to 50 IDs per call
- Cap edit history to 20 entries per file
- Store edit history as native dict instead of JSON string
- Fix undo to run Mermaid validation after reverting content
- Fix Mermaid validation to report "skipped" instead of false success
- Fix Mermaid timeout to warn instead of silent pass
- Fix pagination hint to point to list_components instead of analyze_repo
- Clamp offset to non-negative in _build_component_index
- Add smoke test covering all critical paths (25 assertions)
- Reduce _MAX_RESPONSE_LEN 32000→24000, _MAX_COMPONENTS_PER_CALL 50→20
- Add per-component source truncation at 8000 chars
- Write metadata.json (git commit_id + timestamp) on close_session
  to enable incremental update detection on next analyze_repo
- Update smoke test assertions to match new caps
- Rewrite MCP 服务.md: add list_components tool, thread-safe SessionStore,
  path traversal guards, incremental update mechanism, multi-layer truncation
- Update 后端核心.md: document mermaid validation degradation strategy
- Update overview.md: tool count 9→10, MCP module component count 27→38
- Refresh module_tree.json with new MCP components
Replace large data transmission through stdio MCP protocol with
file-based side channels, enabling support for larger codebases.

Key changes:
- Add SessionWorkspace for per-session disk workspace management
- Write component index, leaf nodes, source files to {repo}/.codewiki/sessions/
- Remove list_components and view_repo_file tools (agent uses native file reading)
- Remove all truncation/pagination limits from MCP responses
- Fix Windows GBK encoding issues with explicit utf-8 encoding
- Update skill SKILL.md to v2.0.0 reflecting new 8-tool architecture
- Regenerate wiki docs with updated module structure (19 docs)

@anhnh2002 anhnh2002 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice refactor overall, the session/tools split is clean and the CLI commit_id fix is correct. A few things to address before merge.

Blockers: the mcp SDK is still not a declared dependency (the pyproject change only registers packages), and view_repo_file shells out with an interpolated path which breaks on spaces and allows command injection.

Should-fix: path traversal on agent-supplied filenames in the read/write tools, and the incremental-update feature only works for CLI-generated docs since the MCP flow never writes metadata.json. Details inline.

Comment thread pyproject.toml
Comment thread codewiki/mcp/tools/code_reader.py Outdated
Comment thread codewiki/mcp/tools/code_reader.py Outdated
Comment thread codewiki/mcp/tools/doc_writer.py Outdated
Comment thread codewiki/mcp/tools/analysis.py
- Add mcp>=1.0.0 to pyproject.toml dependencies (fixes ModuleNotFoundError on fresh install)
- Add explicit utf-8 encoding to FileManager file operations
@mambo-wang

Copy link
Copy Markdown
Author

✅ Issue 2 — code_reader.py 命令注入:当前的 code_reader.py 已经彻底重写,没有任何 subprocess 或 shell=True 调用,全部用纯 Python 的 Path 操作写文件,命令注入漏洞已消除。
✅ Issue 3 — code_reader.py 路径穿越:当前代码中 component ID 来自依赖图分析结果,不直接接受用户路径输入,路径穿越风险已不存在。
✅ Issue 4 — doc_writer.py 路径穿越:已经加了完整的防护——_is_within() 辅助函数(第22行)+ _safe_doc_path() 路径解析函数(第31行),做了 .resolve() + relative_to() 边界检查。write_doc_file 和 edit_doc_file 都调用了这个函数,穿越路径会返回 "Filename escapes output directory." 错误。
✅ Issue 5 — metadata.json 未在 MCP 流程写入:server.py 第546-576行已经实现了 _write_generation_metadata() 函数,在 close_session 时(第403行)会调用它写入 metadata.json,包含 commit_id 和 timestamp。增量更新链路完整。

overall:

✅ mcp SDK 已声明依赖(刚加的)
✅ code_reader.py 已无 shell 调用
✅ code_reader.py 路径穿越已消除
✅ doc_writer.py 路径穿越有完整防护
✅ close_session 时写入 metadata.json

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants