Repository navigation
feat(skills): external mounts get an 'external' provenance tier (#108032) - #128387
Conversation
|
🤖 Hermes Agent automated review: PR reviewed. Diff analyzed (424 lines changed). Check CI status and manual review recommended. |
|
Independent verification on the PR head (2785b50): desktop provenance suite 6/6 green. Python provenance suite 7/8: test_used_local_skill_still_produces_learning_node fails under scripts/run_tests.sh but passes under direct pytest. Root cause is test isolation, not product code: the runner execs pytest under env -i, and the test only sees its fixture skill when ambient environment leaks in. If CI runs through run_tests.sh this test fails there too. Worth making the test hermetic before merge. |
arkheioncorp
left a comment
There was a problem hiding this comment.
Verdict: APPROVE
Clean implementation of the 'external' provenance tier. Key points:
- Priority chain in
provenance()is correct: hub > bundled > external > agent — local copy wins on name clashes viais_external_only() - External skills correctly excluded from learning graph (
_has_learning_signal), while keeping in-place edit rights (isEditableProvenance) - Defensive import in
learning_graph.py(try/except with fallback) is appropriate - Set-based twin in web router matches the per-skill
provenance()semantics - Tests cover classification, learning graph exclusion, router twin, and the local-copies-external regression
૮ >ﻌ< ა ci reviewran on 76dea6e — fix(desktop): fix lint in skill provenance files (hermes-age debug info |
2785b50 to
e29848a
Compare
|
@whyyagswhy investigated on both the head you reviewed (2785b50) and the current one — reproduction says the test is not env-dependent, and the suite is green under the scrubbed runner on both:
To keep that order-dependence from ever hiding again, de01606 pins the hostile order directly: the new
|
) Skills mounted from skills.external_dirs are externally authored, but every provenance surface flattened them into 'agent': the usage report, the dashboard's set-based twin, the desktop catalog/detail UI, and the learning journey graph (where a used external mount rendered as a learned skill). - tools/skill_usage: provenance() gains an 'external' tier (external-only: a local copy of the same name keeps winning, mirroring is_agent_created's external exclusion and _external_skill_names() scans external dirs once for the router's set-based twin. - web_routers/skills: /api/skills classifies hub > bundled > external > agent. - learning_graph: skills whose path resolves under an external dir are tagged source='external' and are never learning milestones — a mount is configured, not learned, even when used. - desktop: SkillInfo.provenance gains 'external'; the catalog labels mounts 'external' and the detail pane keeps them editable in place. Origin only, never mutability: read-only enforcement for external mounts was deliberately rejected by maintainers (commit 8c8fc6c / PR #17512 — a refusal just breeds silent local duplicates). Provenance decisions extracted into skill-provenance.ts pure helpers (testable without source-reading tests). - curator: usage report/filter knows the 'external' origin. The desktop edit logic in this commit is the natural reading of 8c8fc6c (external dirs exist so the user can point Hermes at a vault they curate elsewhere); if maintainers prefer the read-only half as well, it slots in behind isEditableProvenance() in one place. EOF )
test_used_local_skill_still_produces_learning_node was reported as failing under scripts/run_tests.sh (env -i) but passing under bare pytest, which read as ambient-env leakage. Reproduction on both the reviewed head and the current one shows the suite green under the scrubbed runner; the flake the reviewer hit was the 'external' source loop-variable leak (fixed in 71485f5), whose manifestation depends on rglob scan order — env -i changes TMPDIR, which changes the tmpdir layout, which changes the order the mount and the local skill are yielded in. That env-dependence was coincidental, not causal. Pin the hostile order directly: patch Path.rglob so the external mount's SKILL.md always scans before the local skill, and assert the local skill still classifies 'agent' (red at the reviewed head, green now). Also drop the no-op monkeypatch of hermes_constants._hermes_home_cache — no such attribute exists, so it reset nothing and only implied a cache reset that never happened.
c907835 to
76dea6e
Compare
What
Skills mounted from
skills.external_dirsare externally authored, but every provenance surface flattened them intoagent— the usage report, the dashboard's/api/skillsset-based twin, the desktop catalog/detail UI, and the learning journey graph (where a used external mount rendered as a learned skill milestone).This adds an
externalprovenance tier at every site, without touching mutability.Addresses #108032
Changes
tools/skill_usage.py—provenance()gains anexternaltier: an external-only skill (present underskills.external_dirs, absent from the local store — the local copy keeps winning on name clashes, mirroringis_agent_created's existing external exclusion) classifies asexternal. New_external_skill_names()scans external dirs once for the router's set-based twin.hermes_cli/web_routers/skills.py—/api/skillsclassifieshub > bundled > external > agent.agent/learning_graph.py— skills whose path resolves under an external dir are taggedsource='external'(this is the shape the graph actually sees: the common symlinked SKILL.md into~/.hermes/skills/, which the profile-root scan yields) and_has_learning_signalexcludes them: a mount is configured, not learned, even when used.SkillInfo.provenancegains'external'; the catalog labels mountsexternal(notlocal), and the detail pane shows an external skill as externally mounted while keeping it editable in place. The two provenance decisions are extracted into pure helpers (skill-provenance.ts) so they're testable directly.hermes_cli/curator.py—curator usagereports and filters theexternalorigin.What this deliberately does NOT do
The issue also asks for
skills.external_dirsto be read-only by default for foreground skill management. That half is not included because maintainers rejected it: commit8c8fc6c1ec(PR #17512, which settled the then-open #4759 and #4381) removed exactly that default read-only gate — the refusal made agents create silent local duplicates, and the standing regression test attests/tools/test_skill_manager_tool.py:784+pins in-place external mutation succeeding. Re-imposing default-deny would resurrect a closed direction; the opt-in variant (external_dirs_readonly) is a separate live design question (see open PR #51412). This PR is the origin-labeling half only: the label says where a skill came from, and edit rights stay keyed to the standing April decision.Prior art
Two earlier PRs addressed this issue and are both gone — #108040 (deleted) and #108767 (closed 2026-09-12 as its duplicate) — so this is a fresh implementation, not a salvage; no code was reused from either.
Testing
tests/tools/test_skill_provenance_external.py(new, 8 tests): tier classification (external-only →external, local staysagent, local copy wins,is_agent_createdexclusion, names helper), the router twin through a realTestClientagainst the mounted app, and the journey graph (a used external skill produces no learning node via the symlinked-into-profile shape; a used local skill still does). 4 of 8 fail onorigin/mainsources, all pass with the change.apps/desktop/src/app/capabilities/skills/skill-provenance.test.ts(new, 6 vitest cases): both helpers, including that bundled/hub stay non-editable and absent provenance keeps its legacy meaning.test_skill_provenance.py,test_skill_usage.py,test_learning_graph.py,test_web_server_skills_profiles.py(38 tests).