Repository navigation
Description
Describe the bug
scripts/check_translations_utest.py and scripts/check_translations_itest.py never run in CI. All the workflows discover tests with unittest discover -s "tests/scripts" (run-tests.yaml:35,37,40, pre-release.yml:38,40,43, release.yml:39,41), and these two files live in scripts/. None of the 685 tests CI currently runs cover check_translations.py, including the ones in check_translations_utest.py that #3701 changed.
They ended up in scripts/ through #1997, following your review comment there ("The name of the file should be: scripts/check_translations_utest.py for unit tests and scripts/check_translations_itest.py for integration tests"). The integration test started as tests/scripts/test_translation_tags.py (86c47eaf) and still finds the repository root with Path(__file__).parent.parent.parent. That was right in tests/scripts/. From scripts/ it points at the parent of the repository, so the test never sees source/.
To Reproduce
- On a clean checkout of
master, from the repository root:
pipenv run python -m unittest discover -s scripts -p "check_translations_*test.py" - Output:
FAIL: test_english_files_exist … AssertionError: 0 not greater than 0 : No English card files found in source directory FAIL: test_source_directory_exists … AssertionError: False is not true : Source directory not found: <parent of the repository>/source Ran 32 tests in … FAILED (failures=2) - To see that CI never runs them:
pipenv run python -m unittest discover -v -s tests/scripts -p "*_*test.py" 2>&1 | grep -c check_translationsprints0(the run isRan 685 tests … OK).
Expected behavior
CI runs the checker's unit and integration tests, and the integration test reads the mock files in tests/test_files/source/.
Screenshots
The Error: Source directory not found: …/source line near the top comes from a unit test that passes, since it checks that error message on purpose. The two failures are the FAIL: blocks below it.
Additional context
I'd move both files to tests/scripts/, since that touches the fewest files. They also need a few path fixes (the repository root and the scripts/ import path). I tried it on a copy of master: the CI-style run then has 717 tests (685 + 32) and only the two failures below remain. If you'd rather keep the files in scripts/, the other option is adding that folder to the discovery lines in the three workflows. Tell me which you prefer.
With the root corrected, 2 of the 6 integration tests fail on today's data, and I'd like your call on both.
-
test_translations_completenessfinds 1,047 issues across 31 files (11 missing, 1,036 untranslated, 0 empty). You said in Feature: Add automated validation tests for all language translations #2443 that a test on this doesn't make sense because not all tags are translated. I'd make it fail only on empty tags (0 today) and keep reporting the rest. Or should it stay report-only? -
test_no_duplicate_tags_in_englishfails on repeated ids in the English files.T00005appears twice inwebapp-cards-3.0-en.yamlandwebapp-cards-2.2-en.yaml, both times as "Index".T00090appears twice inwebapp-cards-2.2-en.yamlwith different text: "© 2012-2026 OWASP Foundation" (line 473) and "Acknowledgements" (line 476). The es, fr and nl 2.2 files only have the © one. That changes card text, so I'd leave it to you. Can I open a separate issue for these, and skip this one test with a link to it until then, so CI stays green?
The PR would only move the files and fix the paths. No changes to check_translations.py or source/, and no coverage gate.
@sydseter, I'd like to work on this. Could you assign me?
Activity
rajaysingh commentedon Nov 8, 2026
@sydseter, a correction to my plan above. I went back through your review on #1997, where you asked for the integration test to "read from this dir: tests/test_files" and use mock files. It still reads the real
source/, and that's what causes both failures I asked you about. So I'd do it your way:tests/scripts/, keeping the_utest/_itestnames, so CI runs them.tests/test_files/source/and have it assert that the checker finds the missing, untranslated and empty tags already intest-cards-1.0-es.yaml.I tried this on master. The CI test run goes from 754 to 786 tests, all passing with none skipped, and
check_translations.pyis at 98%. Nothing changes incheck_translations.py,source/or the workflows. If you'd rather keep the files inscripts/, I can add that folder to the test discovery instead. Could you assign me?sydseter commentedon Nov 8, 2026
@rajaysingh Thank you for the report. I have assigned you.
test: run the translation checker tests in CI (#3726)
Merge pull request #3764 from rajaysingh/fix/3726-run-translation-che…
rajaysingh commentedon Nov 10, 2026
Fixed by #3764 (merged in 25dcad1). Closing.