Skip to content

fix(skill-and-tool-validator): harden three non-blocking review leftovers - #970

Merged
justinmclean merged 2 commits into
apache:mainfrom
Shaurya2k06:fix/validator-robustness-defects
Jul 27, 2026
Merged

fix(skill-and-tool-validator): harden three non-blocking review leftovers#970
justinmclean merged 2 commits into
apache:mainfrom
Shaurya2k06:fix/validator-robustness-defects

Conversation

@Shaurya2k06

Copy link
Copy Markdown
Contributor

Summary

  • Fix the collect_tool_python_files docstring (and size-threshold comment) so it says bytes, matching path.stat().st_size.
  • Guard the live license-header integration with a skipif when the checkout is not a full magpie tree; add a unit test that proves the skip condition fires out-of-tree.
  • Catch OSError from validate_eval_coverage's iterdir() calls (skills/ and evals/) the same way collect_tool_python_files does, so a restrictive runner gets a quiet skip instead of an unhandled PermissionError.
  • Re-lock root uv.lock mypy requires-dev specifiers to >=2.3.0 so prek stops rewriting the lock when this tool is touched (same correction as build(deps-dev): re-lock uv.lock against the mypy>=2.3.0 dev pins #965 — drop that hunk if build(deps-dev): re-lock uv.lock against the mypy>=2.3.0 dev pins #965 lands first).

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • uv run --directory tools/skill-and-tool-validator --group dev pytest — all passed
  • Targeted: live-header skip proving test, live-header integration, unreadable skills/evals PermissionError cases
  • prek commit hooks pass on changed files (ruff / mypy / workspace pytest / skill-and-tool-validate)
  • Commit is GPG-signed and DCO-signed (Signed-off-by)
  • For Groovy bridges touched: command-line invocation tested end-to-end
  • For skill changes: eval suite passes for the affected skill
  • For skill behaviour changes: a new or updated eval fixture is included in this PR

RFC-AI-0004 compliance

  • HITL — any new mutation is gated on explicit user confirmation
  • Sandbox — no new unrestricted host access; network reach declared in the adapter
  • Vendor neutrality — placeholders used in skill / tool prose (N/A for this robustness fix)
  • Conversational + correctable — agentic-override path documented if behaviour is adopter-tunable
  • Write-access discipline — no autonomous outbound messages; drafts only, sent on confirmation
  • Privacy LLM — private content does not reach a non-approved LLM; redactor invoked where needed

Linked issues

Closes #941

Shaurya2k06 and others added 2 commits July 27, 2026 18:34
…vers

Correct the license-size docstring to bytes, skip the live header
integration out-of-tree instead of aborting, and catch OSError from
validate_eval_coverage directory listing the same way
collect_tool_python_files does.

Also re-lock root uv.lock mypy requires-dev specifiers to >=2.3.0 so
prek stops rewriting the lock when this tool is touched (same
correction as apache#965).

Signed-off-by: shaurya2k06 <shaurya2k06@gmail.com>

@justinmclean justinmclean left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid. I checked the skipif first, since evaluating a helper in a decorator can break collection rather than skip if it raises: find_repo_root returns CWD when it finds no skills/, so out-of-tree the helper returns None and the test skips cleanly. Good.

@justinmclean
justinmclean merged commit 4712e27 into apache:main Jul 27, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix three robustness defects in skill-and-tool-validator

2 participants