Transport improvements from 2025-W09 #300

khaeru · 2025-02-24T13:16:51Z

Handle flow=AVBUNK in IEA EWEB data #229
Adjust efficiencies of LDV PHEV technologies using duty cycle data (carried from Transport improvements from 2025-W05 #289 and Transport improvements from 2024-W49 #259).
Use/test LDV parametrization for non-demand parameters.
- Load factor (output)
- Confirm all mentioned at iiasa/message_data#580.
~~Debug possible double-counting via summation in IAMC-structured reporting of final energy (discussed 2025-03-06)~~ Next PR.
Add $y$ dimension to pdt-elasticity.csv, expand current data (discussed 2025-03-06).
~~Allow to distinguish LED-SSP1 from LED-SSP2 in pdt-cap.csv input data (discussed 2025-03-11)~~ Next PR.
Introduce a GDP pseudo-elasticity for freight transport activity #311
Many other items; see doc/whatsnew.rst.

The branch contained about 120 commits with extensive iteration on/calibration of input values @r-aneeque, but has been rebased to drop all but the last set of changes to:

.github/workflow/transport.yaml — 27 commits
message_ix_models/data/transport/R12/
- speed.csv — 18 commits
- pdt-elasticity.csv — 15 commits
- config.yaml (particularly, share weight convergence) — 4 commits
- ldv-inv_cost.csv — 2 commits

How to review

Read the diff and note that the CI checks all pass.
Read the changes to doc/whatsnew.

PR checklist

Continuous integration checks all ✅
Add or expand tests; coverage checks both ✅
Add, expand, or update documentation.
- Input and output data flows for MESSAGEix-Transport are now completely documented.
Update doc/whatsnew.

codecov · 2025-02-24T13:31:05Z

Codecov Report

Attention: Patch coverage is 97.44346% with 26 lines in your changes missing coverage. Please review.

Project coverage is 77.7%. Comparing base (5ca9f9f) to head (79b5417).
Report is 78 commits behind head on main.

Files with missing lines	Patch %	Lines
message_ix_models/model/transport/data.py	97.7%	4 Missing ⚠️
message_ix_models/testing/check.py	81.8%	4 Missing ⚠️
...sage_ix_models/tests/model/transport/test_build.py	91.1%	4 Missing ⚠️
message_ix_models/model/transport/plot.py	70.0%	3 Missing ⚠️
message_ix_models/model/config.py	85.7%	2 Missing ⚠️
message_ix_models/model/transport/ldv.py	97.1%	2 Missing ⚠️
message_ix_models/model/transport/build.py	95.4%	1 Missing ⚠️
message_ix_models/report/__init__.py	80.0%	1 Missing ⚠️
message_ix_models/report/operator.py	95.2%	1 Missing ⚠️
message_ix_models/tools/iea/web.py	98.1%	1 Missing ⚠️
... and 3 more

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #300     +/-   ##
=======================================
- Coverage   78.3%   77.7%   -0.7%     
=======================================
  Files        217     215      -2     
  Lines      16925   17148    +223     
=======================================
+ Hits       13264   13325     +61     
- Misses      3661    3823    +162

Files with missing lines	Coverage Δ
message_ix_models/cli.py	`93.4% <100.0%> (ø)`
message_ix_models/model/transport/config.py	`98.5% <100.0%> (+1.2%)`	⬆️
message_ix_models/model/transport/demand.py	`100.0% <100.0%> (ø)`
message_ix_models/model/transport/disutility.py	`100.0% <100.0%> (ø)`
message_ix_models/model/transport/freight.py	`82.4% <100.0%> (-17.6%)`	⬇️
message_ix_models/model/transport/groups.py	`97.0% <100.0%> (-0.1%)`	⬇️
message_ix_models/model/transport/ikarus.py	`86.2% <100.0%> (+0.6%)`	⬆️
message_ix_models/model/transport/key.py	`100.0% <100.0%> (ø)`
message_ix_models/model/transport/operator.py	`86.2% <100.0%> (ø)`
message_ix_models/model/transport/other.py	`100.0% <100.0%> (ø)`
... and 36 more

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Fix LDV speed in scenario=SSP1. For narrative consistency, this should stay constant (or maybe even decrease) rather than increasing. - Use 70 for scenario=SSP1, t=RAIL. - Adjust values for scenario=SSP1, t=AIR in some regions. - Add (commented) block of values used to derive PDT per capita in scenario=LED-SSP1.

- Adjust values for SSP1, SSP2, and SSP3 scenarios - cf. discussion and alternate methods at https://iiasa-ece.slack.com/archives/CCFHDNA6P/p1742279792200949?thread_ts=1742205408.282339&cid=CCFHDNA6P

- Fix LDV speed in scenario=SSP1. For narrative consistency, this should stay constant (or maybe even decrease) rather than increasing. - Use 70 for scenario=SSP1, t=RAIL. - Adjust values for scenario=SSP1, t=AIR in some regions. - Add (commented) block of values used to derive PDT per capita in scenario=LED-SSP1.

- Reduce cost of EVs in scenario=SSP1; adjust EV/ICE cost ratio for SSP1. - Use lower BEV and PHEV costs in some regions.

- Adjust values for SSP1, SSP2, and SSP3 scenarios - cf. discussion and alternate methods at https://iiasa-ece.slack.com/archives/CCFHDNA6P/p1742279792200949?thread_ts=1742205408.282339&cid=CCFHDNA6P

- Change values for scenario=LED, SSP1.

- Fix LDV speed in scenario=SSP1. For narrative consistency, this should stay constant (or maybe even decrease) rather than increasing. - Use 70 for scenario=SSP1, t=RAIL. - Adjust values for scenario=SSP1, t=AIR in some regions. - Add (commented) block of values used to derive PDT per capita in scenario=LED-SSP1.

- Reduce cost of EVs in scenario=SSP1; adjust EV/ICE cost ratio for SSP1. - Use lower BEV and PHEV costs in some regions.

- Adjust values for SSP1, SSP2, and SSP3 scenarios - cf. discussion and alternate methods at https://iiasa-ece.slack.com/archives/CCFHDNA6P/p1742279792200949?thread_ts=1742205408.282339&cid=CCFHDNA6P

- In order to address scale-1 values > 1, limit the following technologies to a low share in all regions in y=2020: - t="f road electr". - Two F ROAD technologies using c="gas". - Two P BUS technologies using c="gas". - Delete corresponding entries from R12/config.yaml "minimum-activity" key.

- Change values for scenario=LED, SSP1.

Previously the value was pop()'d and did not reach handle_cli_args().

- for c=gas -using road technologies. - Divide placeholder 0.01 by scale-1 values from trial run.

- Fix LDV speed in scenario=SSP1. For narrative consistency, this should stay constant (or maybe even decrease) rather than increasing. - Use 70 for scenario=SSP1, t=RAIL. - Adjust values for scenario=SSP1, t=AIR in some regions. - Add (commented) block of values used to derive PDT per capita in scenario=LED-SSP1.

- ContainsParameterData: sort items in log message for easier comparison. - HasCoords: - Handle xr.DataArray coords. - Use consistent language in log messages. - Give a positive/confirmation log message for a passing check. - HasUnits: handle ParameterData recursively.

This allows to reuse the same data in other tasks.

- Move COMMON, DIMS constants to globals. - Add a TARGET global. - Reference .keys instead of hard-coding. - Use genno.Key generation. - Move structural items to .build.STRUCTURE_STATIC.

- Separate .transport.other for non-passenger data. - Adjust usage, imports.

- Improve and expand .test_build.test_debug() checks. - Generate some expectations dynamically based on config. - Use .testing.check.verbose_check(). - Roll in assertions from 3 other tests. - test_data.test_get_{freight,non_ldv}_data(): drop; see above. - test_disutility.test_disutility(): drop; see above.

- This is necessary when the input data has interior gaps, as it does in modifications for #300. - Drop use of genno.KeySeq; use Key generation features.

Store the annotation as unquoted string.

The string repr() of a unit cannot be eval()'d.

- Add .testing.MARK. - Chain .transport.testing.MARK from .testing.MARK. - Sort globals in .testing.

khaeru added enh New features or functionality transport MESSAGEix-Transport variant or transport in the base model labels Feb 24, 2025

khaeru self-assigned this Feb 24, 2025

khaeru requested a review from glatterf42 as a code owner February 24, 2025 13:16

khaeru marked this pull request as draft February 24, 2025 13:23

khaeru force-pushed the transport/2025-w09 branch 2 times, most recently from 93819ec to 15d2a67 Compare February 28, 2025 15:26

khaeru force-pushed the transport/2025-w09 branch 4 times, most recently from ef606e7 to be40603 Compare March 12, 2025 11:20

khaeru force-pushed the transport/2025-w09 branch 3 times, most recently from 2783070 to e93396a Compare March 24, 2025 16:06

khaeru force-pushed the transport/2025-w09 branch 2 times, most recently from 1f3b723 to b2373cb Compare March 26, 2025 10:37

khaeru force-pushed the transport/2025-w09 branch from b2373cb to bc386f7 Compare March 26, 2025 11:26

khaeru force-pushed the transport/2025-w09 branch from bc386f7 to 5d4eea1 Compare March 26, 2025 11:42

khaeru pushed a commit that referenced this pull request Mar 26, 2025

Update transport/R12/ldv-inv_cost.csv for #300

6b685cb

- Reduce cost of EVs in scenario=SSP1; adjust EV/ICE cost ratio for SSP1. - Use lower BEV and PHEV costs in some regions.

khaeru force-pushed the transport/2025-w09 branch from 5d4eea1 to 458ac65 Compare March 26, 2025 11:50

khaeru pushed a commit that referenced this pull request Mar 26, 2025

Update transport/R12/elasticity-f.csv for #300

3a2da18

- Change values for scenario=LED, SSP1.

khaeru pushed a commit that referenced this pull request Mar 27, 2025

Update transport/R12/ldv-inv_cost.csv for #300

5eeb132

- Reduce cost of EVs in scenario=SSP1; adjust EV/ICE cost ratio for SSP1. - Use lower BEV and PHEV costs in some regions.

khaeru and others added 27 commits March 29, 2025 15:36

Update transport/R12/elasticity-f.csv for #300

1f552e1

- Change values for scenario=LED, SSP1.

Fix storage of --verbose CLI option on context

69fdb4e

Previously the value was pop()'d and did not reach handle_cli_args().

Show entire Quantity for verbose report

4fe1ff5

Recompute transport/R12/act-non_ldv.csv

43b84c9

- for c=gas -using road technologies. - Divide placeholder 0.01 by scale-1 values from trial run.

Update transport/R12/pdt-cap.csv for scenario=LED

d77b31d

Update test marks for #300

2c9983e

Format transport/R12/disutility.csv

0bcf314

Add .testing.check.verbose_check()

b9f54a0

Add assertion text in .transport.testing.assert_units()

6f7ceb0

Mark interpolated data from activity-ldv.csv

4a7ff11

This allows to reuse the same data in other tasks.

Modernize .transport.disutility

3f29525

- Move COMMON, DIMS constants to globals. - Add a TARGET global. - Reference .keys instead of hard-coding. - Use genno.Key generation. - Move structural items to .build.STRUCTURE_STATIC.

Add .ikarus.TARGET global

af70353

Rename .transport.non_ldv → .passenger

4dfd402

- Separate .transport.other for non-passenger data. - Adjust usage, imports.

Simplify .test_ikarus using insert_checks()

0eeed86

Adjust imports of Annotation to quiet warnings

4c95003

Interpolate inv_cost on y in .transport.ikarus

fabcce3

- This is necessary when the input data has interior gaps, as it does in modifications for #300. - Drop use of genno.KeySeq; use Key generation features.

Use preferred units for LDV historical_new_capacity

5219603

Add Dataflow.module attribute

cdad186

Store the annotation as unquoted string.

Use get_annotation() in Dataflow.units

383e1ee

The string repr() of a unit cannot be eval()'d.

Filter other module data flows in iter_files()

0485f47

Improve test coverage of .sdmx.Dataflow

e6e873c

Xfail one test for khaeru/sdmx#230

998111b

- Add .testing.MARK. - Chain .transport.testing.MARK from .testing.MARK. - Sort globals in .testing.

Add #300 to doc/whatsnew

79b5417

khaeru force-pushed the transport/2025-w09 branch from 17f90a1 to 79b5417 Compare March 29, 2025 14:36

khaeru merged commit ae96255 into main Mar 29, 2025
25 checks passed

khaeru deleted the transport/2025-w09 branch March 29, 2025 17:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Transport improvements from 2025-W09 #300

Transport improvements from 2025-W09 #300

khaeru commented Feb 24, 2025 •

edited

Loading

codecov bot commented Feb 24, 2025 •

edited

Loading

Transport improvements from 2025-W09 #300

Transport improvements from 2025-W09 #300

Conversation

khaeru commented Feb 24, 2025 • edited Loading

How to review

PR checklist

codecov bot commented Feb 24, 2025 • edited Loading

Codecov Report

khaeru commented Feb 24, 2025 •

edited

Loading

codecov bot commented Feb 24, 2025 •

edited

Loading