dev: include generated config and function docs in the local lint suite - #25525
kumarUjjawal merged 2 commits into
Conversation
Add `ci/scripts/check_config_function_docs.sh`, which regenerates `docs/source/user-guide/configs.md` and the aggregate, scalar, and window function pages and fails if a committed page differs, and run it from `dev/rust_lint.sh` as a write step. Contributors see a stale generated page before pushing, and `--write` replaces the four pages under the same clean-tree rule as the other formatter scripts. The check generates into per-run scratch directories beneath the two documentation directories and compares directly against the working files, so it never rewrites them in check mode, does not depend on a repository-wide `git diff`, and leaves staged and unstaged edits and the index alone. Scratch directories are removed on success, failure, and interruption. A `diff` error is reported as a comparison failure, not as stale documentation. `dev/update_config_docs.sh` and `dev/update_function_docs.sh` gain an `--output-dir DIR` option so the checker reuses the complete existing generators, headers, and Prettier invocation without copying them. Without the option both scripts behave as before, and their output is byte-identical in either mode. The `config-docs-check` job now runs the shared checker followed by the existing `git diff --exit-code` guard. Its identity, condition, runner, container, setup actions, and Node version are unchanged. Partial progress on apache#21048.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #25525 +/- ##
==========================================
+ Coverage 82.38% 82.40% +0.01%
==========================================
Files 1138 1138
Lines 434328 434951 +623
Branches 434328 434951 +623
==========================================
+ Hits 357824 358406 +582
+ Misses 54872 54844 -28
- Partials 21632 21701 +69 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
alamb
left a comment
There was a problem hiding this comment.
Thank you @kumarUjjawal -- this is a nice change in my mind.
I had a suggestion for some improvements but I don't think they are needed
There was a problem hiding this comment.
Minor: I suggest calling this check_generated_docs.sh as it is both shorter and more general
|
|
||
| ## Config and Function Docs Check | ||
|
|
||
| `docs/source/user-guide/configs.md` and the three function pages under |
There was a problem hiding this comment.
Can we also please make file references github url links so it is easier to follow what is going on?
ALso I think a bunch of details of the implementation are irrelevant for people running this check (e.g. that needs cargo and npx ,etc) is irrelevant here -- we could just say somehting like
To check they are up to date, run the
ci/scripts/check_config_function_docs.shscript, which is also run as part of./dev/rust_lint.sh
|
Thank you @alamb for suggestions. |
Which issue does this PR close?
Rationale for this change
dev/rust_lint.shdoes not check the generatedconfigs.mdand function pages. A contributor finds a stale page only in theconfig-docs-checkjob. That job also regenerates the tracked pages in place and then runsgit diff, so the same check cannot run in a working tree with other changes.What changes are included in this PR?
ci/scripts/check_config_function_docs.shregenerates the four pages into per-run scratch directories, compares each with its working file, and prints a diff for each stale page. Check mode never rewrites a page.--writereplaces the four pages, with the same clean-tree rule asrust_fmt.sh.dev/update_config_docs.shanddev/update_function_docs.shgain--output-dir DIR, so the checker reuses the existing generators and headers. Without the option, both scripts behave as before.dev/rust_lint.shruns the checker as a write step.config-docs-checkjob runs the checker and keeps itsgit diff --exit-codeguard. The job identity and setup do not change.docs/source/contributor-guide/testing.mddocuments the check and the update command.What is the testing strategy for this PR?
--output-dirproduce byte-identical pages, the four committed pages pass, a committed stale page fails with its diff,--writerestores it byte for byte, the dirty-tree gate refuses, and staged edits survive check mode. No lockfile changed.cargoandnpxcovered each page stale on its own, generator and formatter failures in both modes, cleanup after failure and interruption, concurrent runs, missingnpx, comparison errors, and argument errors. No page changed in any failure case.mainexcept the replaced steps. The full./dev/rust_lint.shpasses.Are there any user-facing changes?
No.
🤖 Generated with Claude Code