Skip to content

perf(native): query retained export table facts - #6907

Merged
gensokyo-dev merged 5 commits into
mainfrom
perf/6900-native-export-views
Oct 23, 2026
Merged

gensokyo-dev merged 5 commits into
mainfrom
perf/6900-native-export-views

Conversation

@gensokyo-dev

@gensokyo-dev gensokyo-dev commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixes #6900. Native export questions can request a summary and select retained PE/ELF export facts by exact names, original indices, fields and pages. Complete output remains the default and the complete parent Evidence stays accessible.

Problem and expected behavior

libLLVM has 44,459 exports: asking for two names previously delivered the whole table, exceeding the default 10 MiB MCP budget. The new summary plus retained pages succeeds at that budget, preserves every matching version/alias and distinguishes page coverage from incomplete source coverage.

Change and scope

One domain projector serves CLI and MCP. It validates an exact immutable retained report once with a weak lifetime; it does not cache filesystem results. Unfiltered pages slice existing order; filtered queries scan retained rows. Source count validation belongs to the report contract. Projection schemas preserve omitted fields rather than filling historical source defaults; a failing regression and actual MCP checks cover this. Scope decision: not applicable; this extends static file inspection within existing authority.

Contract and boundary impact

  • Semantic owner: native export contracts and shared analysis-view projector; adapters only deliver Evidence.

  • CLI/catalog: additive detail: summary and native-exports exports/issues views; CLI summary requires --evidence_output.

  • Compatibility: existing PE/ELF parsing and complete defaults remain; source indices identify rows within their complete parent, including ordinal-only and forwarded PE exports.

  • Evidence/provenance: stable complete parent ID, exact name encoding facts, source coverage, partial/unknown explanations and complete bundle access.

  • Authority/cleanup: static reads, existing session retention and CLI Evidence writes; no target execution or new provider authority.

  • Generated metadata: contracts and authored skill reference updated; ignored catalogs/packaged skills regenerated and checked locally, not committed.

  • Read existing files or captured data.

  • Launch or attach to a process. Validation launches CLI/MCP helper processes only.

  • Inject instrumentation or run scripts/code.

  • Change process memory, registers, files, or configuration. Explicit Evidence output files only.

  • Contact network services.

  • Recover protected code or work around protections.

  • Documentation or usability only; no target interaction.

  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Independent GNU readelf plus directly mapped ELF version bytes verified real LLVM/libc answers, full counts and unchanged artifact bytes. Three alternating complete/selected task pairs per artifact had identical answers, parent IDs and complete source report digests. Reproduction, full ranges and receipts.

Completed task median Complete → selected
LLVM wall / family CPU / peak MiB 97.516 → 37.094 s / 88.030 → 32.645 s / 1833.8 → 1142.6
LLVM reply bytes / token estimate 28,986,227 → 19,695 / 4,355,448 → 57,346
libc wall / family CPU / peak MiB 18.848 → 18.858 s / 17.260 → 17.476 s / 993.1 → 928.6
libc reply bytes / token estimate 1,653,415 → 25,952 / 289,912 → 58,253

Token estimates include all 220 input tool definitions, arguments and text replies once (cl100k_base); no actual model ran. LLVM comparison used 64 MiB for both lanes; separate default 10 MiB probes observed baseline refusal and selected success. Fresh servers, OS caches unflushed, 100% aggregate CPU quota, 3 GiB early stop/4 GiB hard/no swap. All twelve tasks passed with verified payload cleanup and no resource stops/OOM. No parser speedup or universal latency improvement is claimed; libc pays extra calls with no established latency gain. Real PE performance is unmeasured.

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npm run test:focused -- src/native/exports/NativeExportsView.test.ts src/native/exports/NativeExportParser.test.ts tests/acceptance/applications/nativeExportsViews.test.ts tests/boundary/mcp/binaryDiagnosticsMcp.test.ts — 48 passed, including actual CLI/stdio transport, advertised schema validation, exact sparse columns and recomputed returned view digests.
  • npm run check — passed, repeated after adding measurement reports.
  • Documentation generation and npm run docs:check — passed, repeated after reporting changes; nine README conformance tests passed.
  • npm run verify:test-discovery — all 2,180 tests discovered exactly once.
  • Real LLVM CLI summary → complete Evidence bundle → offline selected view — passed, all 44,459 exports retained.
  • Twelve real stdio task comparisons and two 10 MiB boundary probes — outcomes above; measured implementation c7a90b742a29328e5b0069c6f8a34381e0c11e57, baseline main f713087132021d235682f908a1c0f3bc6ab8762d. Later reporting commit changes no production code.

Initial CI run 40015745440: static/build/docs and selected real-format qualifications passed. All eight shard logs were inspected, including cancelled jobs: 11 source + 46 built failures were actually observed. Controls against that exact main base and current implementation reproduced all 57 with identical case identities and full failure messages, normalizing only the two checkout paths (including their shared dependency realpath). The broader selection executed 11 source cases (11 failed) and 52 built cases (48 failed, four passed) on both; extra selected cases are not claimed as CI observations. The report preserves scope and receipt references. Later CI runs are evaluated separately.

Compatibility, safety, and release

Additive API, no migration or dependency changes. PE/ELF fixture coverage includes aliases, ordinal-only/forwarded exports, versions, encoding, partial/unsupported empty pages, malformed imports and historical omissions. Real Linux ELF workflows qualified; no real Hopper/Ghidra/IDA or Windows/PE performance qualification. No target execution, byte mutation or installation changes.

Author readiness

Human owner: requesting user; GitHub identity not separately asserted.

The requesting user explicitly authorized continuing to ready/merge and waiving per-PR human confirmation in this maintenance session. The declarations below remain unchecked because this record does not attest that a human reviewed the final diff.

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

@gensokyo-dev

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-23T20:35:28.294283Z dfacdbe Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 5e9a17bea7

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@morluto
morluto marked this pull request as ready for review October 23, 2026 20:13
@greptile-apps

greptile-apps Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The reviewed changes appear safe to merge.

Summary

Adds summary delivery and retained PE/ELF export queries shared by CLI and MCP.

  • Native export questions can return summaries and selected pages.
  • The CLI and MCP can deliver summaries while keeping full export evidence available.
  • Experiment runs compare complete export replies with selected queries.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[PE or ELF file] --> B[Complete Evidence]
  B --> C[Retained parent]
  C --> D[Summary]
  C --> E[Selected export or issue page]
  C --> F[Complete Evidence bundle]
Loading

Reviews (3) · Last reviewed commit: "test(native): verify sparse export view ..." · Reviewed by Greptile

Comment thread src/domain/analysisView/nativeExportsView.ts
@gensokyo-dev

Copy link
Copy Markdown
Collaborator Author

@codex review

The follow-up fixes a field-selection bug found during local review: Zod .partial() retained the complete ELF row's historical defaults, so selecting only value added unrequested name-encoding fields. The projection schema now removes those defaults; complete-source normalization retains them. A regression failed on the prior implementation and now passes, along with exact-field checks through actual stdio MCP. All 48 selected tests, static checks, docs checks and discovery verification passed.

The performance report was regenerated from twelve fresh alternating tasks against main f713087132021d235682f908a1c0f3bc6ab8762d and implementation c7a90b742a29328e5b0069c6f8a34381e0c11e57. Current 10 MiB boundary probes and the real LLVM CLI → complete Evidence → offline view also passed their expected outcomes. The report preserves measured revisions, ranges, tokenizer assumptions and unavailable coverage.

Initial CI run 40015745440 had 57 actual failed cases across source and all eight built shard logs, including cancelled shards. All 57 reproduced with identical full failure messages on that exact main base and the corrected implementation after checkout-path normalization only. The wider control selection and its four passing cases are reported separately. This does not classify any later CI failure without inspection.

The requesting user has explicitly authorized proceeding to ready and merge without repeated human confirmation, including waiving those local confirmation requirements. Human-review attestations remain unchecked because no human final-diff review is asserted here.

@gensokyo-dev

Copy link
Copy Markdown
Collaborator Author

@codex review

@gensokyo-dev

Copy link
Copy Markdown
Collaborator Author

Final-head integration qualification for dfacdbe2ffc6868e4b6cbbceca9836e122c949f5:

  • CI actually checked out merge 6026528f948dcc72e0cb4ded99577b51633ec2ab, whose parents are main 9e6c6241d7e595c6f3e90d23f30f3d43bc8c9527 and this PR head. The workflow's BASE_SHA=f713087… is only its change-selection base.
  • On that exact merge, all 48 focused parser/view/CLI/MCP tests passed with one worker, including exact sparse fields and recomputed returned view digests. Real LLVM/libc selected stdio tasks passed at 10 MiB; real LLVM CLI summary → complete bundle → offline selected view passed. Their complete parent IDs, source report digests and answers match the measured implementation. Current integration advertises 221 tools; the pinned performance cohort measured 220 tools before unrelated main changes. These extra checks qualify integration behavior, not a new timing comparison.
  • All eight built shard records were inspected. Four never started (runner_id=0, no steps); their absent logs are not treated as passing tests. Logs from every executed/cancelled shard contain two actual built failures, and source tests contain eleven failures. Per-file reported failure counts match the identified cases.
  • Those exact thirteen failed cases were run against the actual main parent and CI merge. Both produced 11 source failures and 2 built failures, with zero selected passes. Full case identities and failure messages are identical after replacing only the two checkout paths. All resource-guarded controls verified payload cleanup and had no OOM/resource stop. This attributes the observed portable failures to the baseline; it does not claim unexecuted tests passed.

The measured workflow, twelve task receipts, oracle and resource/token limitations remain in the committed report. Exact current bindings and local raw receipts are retained under /tmp/rea-perf-native-exports-20261023/final-ci-attribution.json and final-integration-index.json.

@gensokyo-dev

Copy link
Copy Markdown
Collaborator Author

Integrating tested head dfacdbe2ffc6868e4b6cbbceca9836e122c949f5 under the requesting user's explicit standing authorization for baseline-only CI bypass and waiver of repeated human confirmation.

Greptile completed successfully on this exact head and all review threads are resolved. The valid sparse-default/digest finding is fixed and covered through actual MCP. CI's build/static/docs/reverse-tool quality, native package baseline and selected real-format lanes passed. Every observed portable failure in run 40020366436 is reproduced on its actual main parent and CI merge with identical full failure messages; four never-started shards are explicitly unexecuted. The aggregate failures are downstream of those same source/shard failures.

The author-declaration gate remains a documented, user-waived confirmation requirement; unchecked human-review attestations are not represented as completed human review. The additional Codex refresh is still pending and is not counted as a completed review. Its eventual actionable findings will be followed up.

Post-merge verification will exercise the actual merged main with the real LLVM/libc stdio tasks and CLI/offline Evidence workflow.

@gensokyo-dev
gensokyo-dev merged commit 57b1f82 into main Oct 23, 2026
29 of 44 checks passed
@gensokyo-dev
gensokyo-dev deleted the perf/6900-native-export-views branch October 23, 2026 21:00
@gensokyo-dev

Copy link
Copy Markdown
Collaborator Author

Post-merge verification passed on actual main 57b1f8236717185815f21e0bf421fbeb380cc8e7: clean build, all 48 focused parser/view/CLI/MCP tests, real LLVM/libc selected stdio tasks at the default 10 MiB budget, and real LLVM CLI summary → full Evidence bundle → offline selected view. Both real artifacts retain the same complete parent IDs, source report digests and answers as the measured implementation; LLVM retains all 44,459 exports and libc all 3,025. The current integrated catalog has 221 tools. All five guarded checks verified payload cleanup with no OOM or resource stop. These are correctness/integration checks, not replacement performance measurements. Local bindings and resource receipt hashes are in postmerge-qualification.json under the previously documented receipt directory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(native): select retained PE and ELF export facts

1 participant