Fix data pipeline bugs and write GOOD 2.x format

[headisbagent]

Summary

Fixes the data problems found in the September 2026 GOOD model review, fixes several pipeline bugs, and makes the pipeline write GOOD 2.x format (ucdavis/good_model#3). The notebooks are replaced by one command, python build.py, which validates every output graph with GOOD. CI checks that the committed data can be rebuilt from Data/US/Raw (numbers must match to their last written digit).

Data fixes (from the model review)

• Wind profiles had 25 values per day (D1). table_4-39_onshore.csv has an index column, so long_wide()'s iloc[:, 6:] started each day at "Day Of Month". Hour columns are now selected by name everywhere. The resulting wind profiles are identical to those in Grant 2/20/2026: Fixed onshore wind data #1, which fixed the same problem by editing the CSV; this change works with either version of the file.
• Hydro profiles were never used and were one hour short (D2). Assets used "REGION:hydro:" and profiles "REGION:hydro", and iloc[:, 3:] dropped Hour_0. One profile_key() now names every profile, and hydro uses its monthly capacity factors as a daily energy budget.
• New wind and solar capex was 1,000x too low (D5): $/kW / 1e6. It is now $/MW, with Table 4-16 fixed O&M and EPA's 9.78% capital charge rate. Range: $929–2,193/kW.
• Storage had no duration or efficiency (D3). Existing batteries use plant-level durations from EIA-860 2021 Schedule 3-4 (added to Raw/; median 2.0 h, 63 of 102 matched). New batteries use EPA Table 4-35.
• Transmission was lossless. Lines now carry EPA's 2.8% (WECC) and 2.4% losses, and the two directions of a path share a corridor.

Pipeline bugs found along the way

• Missouri's RPS applied to nothing: 'Missouri': 'MO ' had a trailing space. 495 assets now match rps_MO.
• Results were not reproducible: fill-in fuel and VOM costs used unseeded np.random.choice. The build now uses a seeded generator.
• Offshore wind, biomass and landfill gas were not flagged renewable.
• All existing wind and solar in a region used the first resource class's profile. They now use a capacity-weighted average of the region's classes.
• Nuclear dispatch cost included fixed O&M and was rescaled to $21.2/MWh. It is now fuel plus variable O&M: mean $6.8/MWh. This moves nuclear down the dispatch order; please review.
• pandas 3 breaks the old code. It crashes in renewable_transmission_cost, ffill(inplace=True) on a column becomes a no-op, and groupby().apply() offsets shift. The new code gives the same output on pandas 2.3 and 3.0.

Structure

• build.py replaces Raw_Data.ipynb, Graph.ipynb, Aggregation.ipynb and process_data.py. It writes Data/US/Processed/ and GOOD graphs to Outputs/: the US, nine interconnect regions, and an aggregated California example.
• good.graph and good.aggregate replace src/graph.py and the old aggregator.
• plants.json and combined_assets.json are removed; nothing produced or read them.
• metadata.json records units and every assumed parameter with its source.

Unchanged

Existing capacity is identical (1,135,501 MW, the same 18,993 units). Coal, gas and oil mean dispatch costs are unchanged. 22 zero-MW wind cost classes are no longer written as candidate sites.

Needs team review

• Nuclear dispatch cost ($21.2 → $6.8/MWh).
• Placeholders in PARAMETERS:
  • wind and solar capacity credits (0.15 / 0.10);
  • pumped hydro duration and efficiency (10 h / 80%);
  • new-battery lifetime (15 years);
  • geothermal as must-run at full capacity.
• Coal and oil calibration targets ($23 and $32/MWh) are carried over; their source isn't recorded.
• Unrecorded sources: capacity_factor.csv (hydro) and rps_fraction.csv (its layout matches NREL ReEDS inputs). Please add them to the README.
• Relationship to Grant 2/20/2026: Fixed onshore wind data #1: Grant 2/20/2026: Fixed onshore wind data #1 edits the raw CSV; this PR fixes the code, and both give identical wind profiles. Grant 2/20/2026: Fixed onshore wind data #1 now conflicts with main. Keeping the cleaner CSV from Grant 2/20/2026: Fixed onshore wind data #1 is fine with this code.
• Dependency: requirements.txt pins good to the v2.0-review-fixes branch of GOOD 2.0: fix review findings, move to linopy and MW units, add docs good_model#3. Switch it to a release tag once that PR merges.

Test plan

• pytest -q: 21 tests pass on pandas 2.3 and 3.0. They cover the parsing fixes on synthetic tables and checks on the committed data, including GOOD validation of the full US graph.
• python build.py --check: a fresh build matches the committed files on pandas 2.3 and 3.0.
• python build.py: all 11 graphs pass GOOD validation.
• Team review of the items above.

🤖 Generated with Claude Code