Skip to content

Reject unsupported three-phase transformer groups and repair CI warning paths - #3145

Open
RanaPriyansh wants to merge 12 commits into
e2nIEE:developfrom
RanaPriyansh:fix-3122-reject-unsupported-vector-groups
Open

RanaPriyansh wants to merge 12 commits into
e2nIEE:developfrom
RanaPriyansh:fix-3122-reject-unsupported-vector-groups

Conversation

@RanaPriyansh

@RanaPriyansh RanaPriyansh commented Sep 22, 2026 •

Copy link
Copy Markdown

runpp_3ph accepted unsupported two-winding transformer vector groups yy, yd, dy, and dd. An early continue skipped validation. Validation now runs before that branch and still applies only to three-phase power flow. This addresses #3122.

The change also corrects transformer data and JSON loading:

  • Thirty-seven bundled network and active test JSON assets now store tap_dependency_table=False with Boolean dtype for 11,714 transformer rows. All other serialized values remain unchanged.
  • LogSplineCharacteristic JSON loading now supplies positive raw-domain values to its constructor, then restores the stored logarithmic arrays. A round-trip test checks interpolation.
  • Nullable Boolean masks now treat missing values as False for short-circuit power-station flags and temperature-dependent line calculations.

The CI repairs address numerical edge cases and preserve established warning contracts:

  • The relying matrix keeps pandapipes on Python 3.10. It installs and tests SimBench only on Python 3.11 and newer, as required by SimBench develop.
  • CIM conversion preserves the boundary-table schema when joins or append targets are empty. The transformer characteristic table starts with numeric column types. Estimation paths use explicit dtypes or copies where pandas changed warning behavior.
  • Grid-equivalent and comparison paths preserve numerical values and null equality without deprecated NumPy matrix operations or implicit pandas conversions.
  • Three-phase power flow now treats zero delta power at zero delta voltage as zero current. It initializes an out-of-service line-end auxiliary bus from the connected in-service bus. Tests check convergence and finite results for the supplied component, including networks with out-of-service buses.
  • Short-circuit branch-current reductions preserve all-NaN results. Existing result paths retain their NaN-to-zero conversion. A focused test checks finite, infinite, and undefined voltage-unbalance results.
  • Tests assert intended warnings for legacy network data, the default line temperature coefficient, and SciPy 1.17+ ill-conditioned inverses. A state-estimation test checks zero-injection measurements rather than assuming distinct voltage estimates. Numerical reference checks remain.

The Python 3.14 strict-warning group 1 and Python 3.12 full suite ran on source tree 158f264904208afff75ffcf26718482c550d270b at pre-rewrite commit 08dfc7630f90c7dea701bd98d69d3f0f6c954dcf. The identity rewrite represents that same source tree at f4ea606677e09e07e83698679bed6e616031ac47. Group 1 reports 778 passed, 39 skipped, and 7 xfailed. The Python 3.12 full suite reports 1,678 passed, 168 skipped, 22 xfailed, and 12 xpassed. Python 3.14 strict-warning group 2 ran before the final signed-zero fixture correction and reports 902 passed, 129 skipped, 16 xfailed, and 11 xpassed. A focused strict Python 3.14 signed-zero check passed after that correction. Focused tests for three-phase convergence, isolated buses, short-circuit currents, and voltage unbalance pass. Before this repair, Codecov reported 71.18% project coverage on pre-rewrite commit 7ecc152, whose source tree 84c49a4ed8b034dcb9b3fd71dd9e2dacbf93ad42 is now represented by d727ed3302ef5d8dc4aaee94bc4e277e8a893f0a, versus 71.19% on the base.

Transmission-network expansion (TNEP) now initializes res_ne_line.built with Boolean dtype. A regression test checks initial False values and checks that read_tnep_results maps 0.25 to False and 0.75 to True while preserving Boolean dtype.

SQLite connections use a shared context manager that completes the transaction before closing the connection. The Python 3.12 SQLite checks report one passed test and one existing expected failure. The full suite was not repeated for the SQLite change. Codacy and SonarCloud passed on 3276fc0e3dc039f75fe6c8dedfe016f5f6c12c17, before the TNEP change.

The OPF setpoint tests now write the 0.99 and 0.96 targets through single-step .loc[row, column] assignments. Julia is unavailable locally. The hosted Python 3.14 OPF job passed on the current head. The storage OPF fixture now initializes load_ts and sgen_ts with float dtype. It writes sgen_ts values through direct DataFrame .loc assignments. A focused strict-warning data-transfer check passed without Julia.

PandaModels storage results now read scalar ps, qs, and se values for each storage index. A strict non-Julia regression checks power, reactive power, and state of charge across two storages and two timesteps. The qflex test now writes transformer pm_param/setpoint_q and pm_param/side through direct DataFrame .loc assignments. A strict non-Julia semantic and convert_pp_to_pm check passed.

The runpm_ploss and runpm_loading paths now write the branch-side metadata with direct DataFrame .loc assignments for branches marked True. A strict non-Julia semantic check passed for both functions. The current-head workflow on c8ef952e07b75dfe024b52e5b693986ca9297aa4 completed with 25 successful jobs, one failed job, and five cancelled jobs. The hosted OPF job and strict Python 3.14 warning group 2 passed. Strict Python 3.14 warning group 1 logged a 60-second pytest timeout in unchanged test_contingency_parallel[case14]. The 30-minute job limit then cancelled group 1. Focused local Python 3.14.5 checks passed both parallel contingency cases with -W error, including an explicit forkserver run. These macOS checks do not establish a Linux CI pass. The failed relying Python 3.10 job repeated the pandapipes test_convert_format[0.1.0-True] KeyError: 'component_list' seen on the base. Group 1 did not complete strict-warning coverage. The PR remains open and requires review.

The strict Python 3.14 warning group 1 on b1a834f4 stopped when LightSim warned that the contingency test had no tagged slack generator. The test now expects this warning around its run_contingency_ls2g call. Warning group 2 was cancelled on that head, so it did not validate the TNEP path. That workflow ended with 24 passed, two failed, and five cancelled jobs.

The strict Python 3.14 warning group 2 on d38f5b8 passed the TNEP path, then stopped at an unchanged chained assignment in test_runpm_vstab. The new setpoint change replaces both chained assignments in those tests. Warning group 1 timed out in an unchanged parallel contingency test before it reached the revised LightSim warning assertion. The d38f5b8 workflow ended with 24 passed, two failed, and five cancelled jobs. The other failure is the same downstream pandapipes loader case described below.

The strict Python 3.14 warning group 2 on a8a0d9b9 passed test_runpm_vstab, then stopped at test_storage_opt. Its time-series fixture produced an object-dtype array for a float64 target, which raised a pandas FutureWarning under strict warnings. The fixture change initializes float tables and uses direct generator assignments. Warning group 1 again timed out in the unchanged parallel contingency test before it reached the LightSim assertion. The relying Python 3.10 job repeated the downstream pandapipes loader failure.

The strict Python 3.14 warning group 2 on 7a061a8 passed the storage fixture transfer, then stopped in read_pm_storage_results. It assigned a single-element Series to a scalar result cell, which triggered a pandas warning and ValueError. The scalar read change addresses that assignment for each storage index. Warning group 1 again timed out in an unchanged parallel contingency test before reaching the LightSim assertion. The relying Python 3.10 job repeated the downstream pandapipes loader failure.

The strict Python 3.14 warning group 2 on a6faeea passed the storage result test, then stopped at a chained transformer setpoint assignment in test_runpm_qflex_and_multi_qflex. The qflex change replaces that assignment and the adjacent transformer-side assignment. Warning group 1 again timed out in an unchanged parallel contingency test before reaching the LightSim assertion. The relying Python 3.10 job repeated the downstream pandapipes loader failure.

The strict Python 3.14 warning group 2 on 1d66799 passed the qflex test, then stopped at a chained assignment inside runpm_ploss. The branch-side change replaces that write in runpm_ploss and the equivalent write in runpm_loading. Warning group 1 again timed out in an unchanged parallel contingency test before reaching the LightSim assertion. The relying Python 3.10 job repeated the downstream pandapipes loader failure.

The earlier relying job stopped while installing SimBench on Python 3.10. On earlier head 3276fc0, 28 checks passed, two failed, and five were cancelled. The strict Python 3.14 warning failure occurred when TNEP wrote a Boolean result to an integer column. The TNEP change initializes that column with Boolean dtype. The relying job on 3276fc0 skips SimBench as intended, then fails in pandapipes test_convert_format[0.1.0-True] with KeyError: 'component_list'. The base job shows the same failure. Pandapipes #818 and #817 cover that downstream loader contract. This PR does not change its decoder allowlist. Older-format schema behavior remains under pandapower #3044. The separate Yyn documentation question remains under #2507.

Codex used.

@codecov

codecov Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.75630% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.21%. Comparing base (4c4d862) to head (c8ef952).

Files with missing lines Patch % Lines
pandapower/converter/jao/from_jao.py 33.33% 4 Missing ⚠️
pandapower/opf/pm_storage.py 0.00% 4 Missing ⚠️
pandapower/runpm.py 0.00% 2 Missing ⚠️
pandapower/converter/pandamodels/to_pm.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3145      +/-   ##
===========================================
+ Coverage    71.19%   71.21%   +0.02%     
===========================================
  Files          360      360              
  Lines        40545    40599      +54     
===========================================
+ Hits         28864    28912      +48     
- Misses       11681    11687       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@RanaPriyansh

Copy link
Copy Markdown
Author

CI at 798e1381 has two failures whose diagnostics also occur on current develop (4c4d8620):

  • relying (3.11): test_convert_format[0.1.0-True] raises KeyError: 'component_list', then a network-loading warning. PR job, base job.
  • warnings (3.14, 2): collecting test_geo.py raises DeprecationWarning: tap_dependency_table is missing in net. PR job, base job.

The Python 3.10–3.14 build matrix and linting jobs passed.

vogt31337
vogt31337 previously approved these changes Sep 23, 2026
@RanaPriyansh RanaPriyansh changed the title Reject unsupported transformer groups in three-phase power flow Reject unsupported three-phase transformer groups and repair CI warning paths Sep 24, 2026
@RanaPriyansh
RanaPriyansh force-pushed the fix-3122-reject-unsupported-vector-groups branch from 0f4299d to 3276fc0 Compare September 24, 2026 09:44
@sonarqubecloud

Copy link
Copy Markdown

@vogt31337 vogt31337 added this to the pandapower 4.0 milestone Sep 25, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants