Run the full export tests with the v4 API by diemol · Pull Request #2002 · SeleniumHQ/selenium-ide

diemol · 2026-10-29T23:46:34Z

Description

Ported index.spec.js in the six language packages from the v3 API (emitTest, emitSuite) to languageExport.emit.test and languageExport.emit.suite. Same cases as before: a test, grid execution, a suite, reused test methods, control flow, and new windows.
For Python, Ruby and JavaScript, each exported file is also compiled (py_compile, ruby -c, node --check). The check is skipped when the tool isn't installed.
Dropped the v3 normalizeTestsInSuite step: v4 suites list test IDs.
Rewrote the snapshots: the old ones had v3 output.

Motivation and Context

These tests were skipped, so nothing checked a full exported file. The recent export fixes (indentation, variables in strings, times) would have been caught by these tests.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist

I have read the contributing document.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.
All new and existing tests passed.

🤖 Generated with Claude Code

The index tests used the v3 export API and were skipped. They now export whole tests and suites with languageExport.emit.test and .suite, and check that the Python, Ruby and JavaScript output compiles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

diemol · 2026-10-29T23:46:36Z

/agentic_review

qodo-code-review · 2026-10-29T23:47:10Z

PR Summary by Qodo

Re-enable full-file export tests using the v4 API

🧪 Tests 🕐 20-40 Minutes

AI Description

• Re-enable full-file export tests across six languages using the v4 test and suite APIs.
• Refresh snapshots for current output, including reused methods, control flow, grid execution, and
 windows.
• Check generated Python, Ruby, and JavaScript files for syntax errors when their tools are
 available.

Diagram

graph TD
  F["SIDE fixtures"] --> T["Export tests"] --> V["v4 emit API"] --> O["Generated files"] --> S["Snapshots"]
  O --> C["Syntax checkers"]

High-Level Assessment

The following are alternative approaches to this PR:

1. Share the syntax-check helper

➕ Avoids duplicating temporary-file creation and process handling across three packages.
➖ Adds a cross-package test dependency and configuration for a small helper.

Recommendation: Keep the package-local tests for this PR: they restore coverage without changing production exporters or test infrastructure. A shared syntax-check helper becomes worthwhile if more language packages adopt these checks.

Files changed (12) +564 / -505

Tests (12) +564 / -505

index.spec.js.snapRefresh C# NUnit full-export snapshots +23/-0

Refresh C# NUnit full-export snapshots
• Records the v4 output for complete tests and suites, including spacing around fixtures and reused methods.
packages/code-export-csharp-nunit/test/src/snapshots/index.spec.js.snap

index.spec.jsRe-enable C# NUnit export scenarios +41/-67

Re-enable C# NUnit export scenarios
• Moves the skipped scenarios to the v4 test and suite emitters. Passes the project and its tests directly instead of normalizing suite test IDs.
packages/code-export-csharp-nunit/test/src/index.spec.js

index.spec.js.snapRefresh C# xUnit full-export snapshots +28/-15

Refresh C# xUnit full-export snapshots
• Records v4 output for complete tests and suites, including changed namespace layout and spacing.
packages/code-export-csharp-xunit/test/src/snapshots/index.spec.js.snap

index.spec.jsRe-enable C# xUnit export scenarios +41/-67

Re-enable C# xUnit export scenarios
• Uses the v4 emitters for the existing test, suite, grid, reuse, control-flow, and window scenarios. Removes obsolete suite normalization.
packages/code-export-csharp-xunit/test/src/index.spec.js

index.spec.js.snapRefresh Java JUnit full-export snapshots +75/-52

Refresh Java JUnit full-export snapshots
• Updates expected v4 output and renames the grid-execution snapshot to match its test.
packages/code-export-java-junit/test/src/snapshots/index.spec.js.snap

index.spec.jsRe-enable Java JUnit export scenarios +42/-68

Re-enable Java JUnit export scenarios
• Ports the skipped full-file scenarios to the v4 test and suite emitters. Uses suite test IDs without the former name-normalization step.
packages/code-export-java-junit/test/src/index.spec.js

index.spec.js.snapRefresh JavaScript Mocha full-export snapshots +31/-8

Refresh JavaScript Mocha full-export snapshots
• Records current v4 output, including assertion and variable-to-string changes alongside updated spacing.
packages/code-export-javascript-mocha/test/src/snapshots/index.spec.js.snap

index.spec.jsRe-enable Mocha exports and check JavaScript syntax +64/-62

Re-enable Mocha exports and check JavaScript syntax
• Ports full-file scenarios to the v4 emitters and checks each exported body with Node's syntax checker. Removes obsolete suite normalization.
packages/code-export-javascript-mocha/test/src/index.spec.js

index.spec.js.snapRefresh Python pytest full-export snapshots +27/-4

Refresh Python pytest full-export snapshots
• Records v4 output for full tests and suites, including string conversion for reused variables and updated spacing.
packages/code-export-python-pytest/test/src/snapshots/index.spec.js.snap

index.spec.jsRe-enable pytest exports and check Python syntax +67/-62

Re-enable pytest exports and check Python syntax
• Ports full-file scenarios to the v4 emitters and checks each exported body with an available Python compiler. Removes obsolete suite normalization.
packages/code-export-python-pytest/test/src/index.spec.js

index.spec.js.snapRefresh Ruby RSpec full-export snapshots +60/-37

Refresh Ruby RSpec full-export snapshots
• Records v4 output, including URL quoting, variable-to-string conversion, spacing, and the renamed grid-execution case.
packages/code-export-ruby-rspec/test/src/snapshots/index.spec.js.snap

index.spec.jsRe-enable RSpec exports and check Ruby syntax +65/-63

Re-enable RSpec exports and check Ruby syntax
• Ports full-file scenarios to the v4 emitters and checks each exported body with ruby -c when available. Removes obsolete suite normalization.
packages/code-export-ruby-rspec/test/src/index.spec.js

qodo-code-review · 2026-10-29T23:48:14Z

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

1. ~~Syntax checks pass when processes fail~~ ✓ Resolved

Description

expectValidSyntax treats every spawnSync error as a reason to skip the checker, rather than
skipping only an unavailable optional executable. If a checker cannot start because of a permission
or resource error, the loop can finish without an assertion and the export test passes without
checking the generated file.

Code

packages/code-export-python-pytest/test/src/index.spec.js[63]
+    if (result.error) continue

Evidence
All three new helpers continue when result.error is set and have no assertion after the loop.
Python and Ruby may intentionally skip an absent compiler, but the same branch also suppresses other
spawn failures; JavaScript uses the current Node executable and has no unavailable-tool case to
accommodate.
packages/code-export-python-pytest/test/src/index.spec.js[50-68]
packages/code-export-ruby-rspec/test/src/index.spec.js[50-65]
packages/code-export-javascript-mocha/test/src/index.spec.js[50-65]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new syntax-check helpers silently skip any subprocess spawn error, so a compiler that fails to start for reasons other than being absent can leave generated output unchecked.

## Fix Focus Areas
- packages/code-export-python-pytest/__test__/src/index.spec.js[61-68]
- packages/code-export-ruby-rspec/__test__/src/index.spec.js[58-65]
- packages/code-export-javascript-mocha/__test__/src/index.spec.js[58-65]

## Recommended Fix
For optional Python and Ruby checkers, continue only when the error indicates the executable is absent; throw other spawn errors. For the JavaScript checker, fail on any spawn error because `process.execPath` is the executable running the tests.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Context sources

Review mode: Auto: 🚀 Fast: Localized test-checker changes across three analogous files; low behavioral risk.

✅ Fix-loop round: fix loop

Tip of the day

💡 Did you know, you can route each severity your way: inline, summary, both, or drop

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Previous reviews

Review updated until commit 69e4a73

Results up to commit b2ac2f9 ⚖️ Balanced

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. ~~Syntax checks pass when processes fail~~ ✓ Resolved

Description

expectValidSyntax treats every spawnSync error as a reason to skip the checker, rather than
skipping only an unavailable optional executable. If a checker cannot start because of a permission
or resource error, the loop can finish without an assertion and the export test passes without
checking the generated file.

Code

packages/code-export-python-pytest/test/src/index.spec.js[63]
+    if (result.error) continue

Evidence
All three new helpers continue when result.error is set and have no assertion after the loop.
Python and Ruby may intentionally skip an absent compiler, but the same branch also suppresses other
spawn failures; JavaScript uses the current Node executable and has no unavailable-tool case to
accommodate.
packages/code-export-python-pytest/test/src/index.spec.js[50-68]
packages/code-export-ruby-rspec/test/src/index.spec.js[50-65]
packages/code-export-javascript-mocha/test/src/index.spec.js[50-65]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new syntax-check helpers silently skip any subprocess spawn error, so a compiler that fails to start for reasons other than being absent can leave generated output unchecked.

## Fix Focus Areas
- packages/code-export-python-pytest/__test__/src/index.spec.js[61-68]
- packages/code-export-ruby-rspec/__test__/src/index.spec.js[58-65]
- packages/code-export-javascript-mocha/__test__/src/index.spec.js[58-65]

## Recommended Fix
For optional Python and Ruby checkers, continue only when the error indicates the executable is absent; throw other spawn errors. For the JavaScript checker, fail on any spawn error because `process.execPath` is the executable running the tests.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

diemol · 2026-10-29T23:54:37Z

/agentic_review

qodo-code-review · 2026-10-29T23:55:33Z

Code review by qodo was updated up to the latest commit 69e4a73

qodo-code-review Bot reviewed Oct 29, 2026

View reviewed changes

Comment thread packages/code-export-python-pytest/__test__/src/index.spec.js Resolved

Only skip a syntax checker that isn't installed

69e4a73

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

diemol merged commit 4b2332d into trunk Oct 30, 2026
5 checks passed

diemol deleted the port-export-index-tests branch October 30, 2026 00:01

Navigation Menu

Repository navigation

Run the full export tests with the v4 API - #2002

Run the full export tests with the v4 API#2002
diemol merged 2 commits into
trunkfrom
port-export-index-tests

diemol commented Oct 29, 2026

Uh oh!

diemol commented Oct 29, 2026

Uh oh!

qodo-code-review Bot commented Oct 29, 2026

Uh oh!

qodo-code-review Bot commented Oct 29, 2026 •

edited

Loading

Uh oh!

Uh oh!

diemol commented Oct 29, 2026

Uh oh!

qodo-code-review Bot commented Oct 29, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

diemol commented Oct 29, 2026

Description

Motivation and Context

Types of changes

Checklist

Uh oh!

diemol commented Oct 29, 2026

Uh oh!

qodo-code-review Bot commented Oct 29, 2026

PR Summary by Qodo

Uh oh!

qodo-code-review Bot commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

Uh oh!

diemol commented Oct 29, 2026

Uh oh!

qodo-code-review Bot commented Oct 29, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

qodo-code-review Bot commented Oct 29, 2026 •

edited

Loading