Skip to content

fix(verilog): emit referenced modules and packages as stubs (#1402) - #3332

Open
gpwns3717 wants to merge 1 commit into
Graphify-Labs:v8from
gpwns3717:verilog-reference-node-stubs
Open

fix(verilog): emit referenced modules and packages as stubs (#1402)#3332
gpwns3717 wants to merge 1 commit into
Graphify-Labs:v8from
gpwns3717:verilog-reference-node-stubs

Conversation

@gpwns3717

Copy link
Copy Markdown

Problem

graphify/extractors/verilog.py stamps source_file on the target of
instantiates and of package_import_declaration — modules and packages the
file merely references. A sourced node is a definition to the corpus
resolver, so it never enters the stub pool _rewire_unique_stub_nodes()
consumes, and _disambiguate_colliding_node_ids bakes the referencing file's
path into its id.

go.py already documents this exact failure:

# stub — like the inheritance-base path in the other extractors — so the
# corpus-level rewire can collapse it onto the real definition. A sourced
# stub here makes _disambiguate_colliding_node_ids bake the referencing
# file's path (with extension) into the id and blocks the rewire, which is
# the phantom-duplicate-node bug (#1402).

graphify/extractors/go.py:152-156

The Verilog extractor appears never to have been converted.

It is SystemVerilog-specific

Same fixture in six languages — one file defines Widget, another uses it:

lang nodes edges components cross-file edges duplicate labels
python 6 9 1 5 0
go 6 5 1 1 0
rust 6 6 1 2 0
typescript 6 9 1 5 0
java 6 7 1 3 0
systemverilog 5 3 2 0 1

The reference node each extractor emits, with _rewire_unique_stub_nodes
stubbed out so this is raw extractor output:

go       id=widget            source_file=''            stub  <- rewirable
rust     id=widget            source_file=''            stub  <- rewirable
python   id=widget            source_file=''            stub  <- rewirable
java     id=widget            source_file=''            stub  <- rewirable
sv       id=gadget_sv_widget  source_file='gadget.sv'   REAL  <- widget is not defined in gadget.sv

Ablating _rewire_unique_stub_nodes() to a no-op drops go to 0 cross-file edges
and rust to 0 — they get all of their cross-file structure from that pass.
Verilog is at 0 either way; it never participates.

Impact

A 289-file SystemVerilog RTL tree, measured on v8 before and after this
change, with the shipped tree-sitter-verilog:

before after
nodes 1157 750
connected components 289 (= the file count) 83
largest component 18 518
cross-file edges 0 175
instantiations bound to a definition 0 / 308 138 / 224
labels bound to more than one node 183 (605 nodes) 45 (321 nodes)

Every shared primitive — reset synchronisers, CDC FIFOs, pipeline stages —
existed once per referencing file, and the most widely imported package was
57 separate nodes.
GRAPH_REPORT.md read Surprising Connections: None detected - all connections are within the same source files, which was not a property of the corpus but
the only possible outcome. Community detection returned one community per file.

For hardware the module hierarchy is the architecture, so the graph answered
none of the questions it is built for. After the change a top-level module's
instantiation of a sub-block declared in another file appears as an edge, and
the report's surprising-connections section is no longer empty.

Two details beyond the straight port

type: "module" is load-bearing, not cosmetic.
_node_disambiguation_source_key() falls back to origin_file when
source_file is empty, so blanking the source alone still salted the stubs per
referencing file and the most widely imported package stayed 57 nodes. The
module/namespace
anchor exemption is what collapses them — the #1327 case
_disambiguate_colliding_node_ids describes as "import CoreKit from three
files ... those are the same module"
. A SystemVerilog package import, and a
primitive with no in-corpus definition, are exactly that. Measured:

variant nodes components
blank source_file only 1138 54
+ anchor on imports 955 39
+ anchor on both 835 37

(that table is from a run with a newer grammar, so the absolute numbers differ
from the table above; the relative effect is the point.)

Case-distinct names. _make_id lowercases, but SystemVerilog is
case-sensitive. Widget and widget already collapse onto one id on v8 — but
into a name-only node. Once the target is rewirable, the merged node binds to
whichever definition matched first, so widget u_lower would claim to
instantiate Widget in a different file. That is a regression this change would
otherwise introduce, so _sv_reference_id salts the id with a hash of the
exact-case name when the name is not already lowercase:

before        both instantiations -> id=widget          label='Widget'  src=top.sv
naive stub    both instantiations -> id=w_upper_widget  label='Widget'  src=w_upper.sv
this PR       Widget u_upper -> w_upper_widget_adf611   label='Widget'  src=w_upper.sv
              widget u_lower -> w_lower_widget          label='widget'  src=w_lower.sv

An all-lowercase name keeps a byte-identical id, so a corpus with no case
collision is unaffected — the impact table above is unchanged by the salt.

Notes

  • pytest tests/test_languages.py is green (407 passed) on this branch with the
    dependencies v8 pins.
  • SystemVerilog classes are untouched: _augment_systemverilog_semantics
    works on source text, not on the tree, and still emits per-file nodes for
    referenced types.
  • Independent of fix(verilog): parse SystemVerilog with tree-sitter-systemverilog #3324 (the grammar): verified by applying this change with both
    tree-sitter-verilog and tree-sitter-systemverilog, which produce the same
    cross-file edges. The two can merge in either order — the grammar decides what
    is read out of one file, this decides whether those facts join across files.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EszWxScCETGBjTc8yvCNuf

…-Labs#1402)

The Verilog extractor stamps `source_file` on the target of `instantiates` and
of `package_import_declaration` — modules and packages the file merely
references. A sourced node is a definition to the corpus resolver, so it never
enters the stub pool `_rewire_unique_stub_nodes()` consumes, and
`_disambiguate_colliding_node_ids` bakes the referencing file's path into its
id. `go.py` names this the phantom-duplicate-node bug and emits such targets as
sourceless stubs; the Verilog extractor was never converted.

The result is a graph with no cross-file edge at all. On a 289-file
SystemVerilog RTL tree:

  connected components                     289  ->   83   (289 = the file count)
  largest component                         18  ->  518
  cross-file edges                           0  ->  175
  instantiations bound to a definition   0/308  -> 138/224
  labels bound to more than one node    183     ->   45

Every shared primitive existed once per referencing file, and the most widely
imported package was 57 separate nodes. For hardware the module hierarchy is
the architecture, so none of it survived.

Two details beyond the straight port:

`type: "module"` is load-bearing. `_node_disambiguation_source_key()` falls back
to `origin_file` when `source_file` is empty, so the stubs were still salted per
referencing file; the module/namespace anchor exemption (Graphify-Labs#1327 — the same module
imported from three files is one module) is what collapses them.

`_make_id` lowercases, but SystemVerilog is case-sensitive. `Widget` and
`widget` already collapsed onto one id before this change; once the target is
rewirable, the merged node binds to whichever definition matched first, so
`widget u_lower` would claim to instantiate `Widget` in another file.
`_sv_reference_id` salts the id with the exact-case name when it is not already
lowercase, leaving all-lowercase names byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EszWxScCETGBjTc8yvCNuf

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.

Formal verification. No changes could be formally verified in this run.


Graphify review — findings

Emits referenced modules and packages (imports, instantiations) as sourceless stub nodes via add_reference_node instead of sourced add_node, so the corpus-level rewire can collapse them onto real definitions rather than leaving phantom duplicates. Computes their ids through _sv_reference_id, which keeps the historical lowercase id for all-lowercase names but appends a hash suffix for mixed-case names so case-sensitive SystemVerilog identifiers like Widget and widget don't share a stub.

Worth a look

  • Mixed-case instantiation edges target a new hashed stub instead of the defined module nodegraphify/extractors/verilog.py:342 · Escalate · high
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 29 functions depend on the 21 functions this change touches.

Health — this change adds coupling hotspots:

  • new: extract_verilog() — 10 callers, 6 callees
  • new: _augment_systemverilog_semantics() — 1 callers, 7 callees
  • new: walk() — 1 callers, 6 callees

Verification — 29 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 29 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify extract\_verilog.

The verifier did not have enough to check extract\_verilog, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

· 3 more finding(s) on lines outside this diff (see the check run).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant