Add BMT workflows and update functions to build Buildings module

[yiyi1991]

This is a draft PR to track bmt development (issue #431) for the milestone "build the Buildings sector on a baseline scenario".

(Changes in B module, BMT module, (legacy report on message-data branch), tests, and docs, listing this as TODOs to be deleted later.)

How to review

• Read the diff and note that the CI checks all pass.
• Run mix-models bmt run --from="M solved" "B built" to see if it works on an R12 scenario.
• Run mix-models bmt run --from="B solved" "B reported" to see if the report works.
• (the build function in the Buildings module should still work with the previous NAVIGATE scenarios but not sure how to test)
• The bmt documentation makes sense.

PR checklist

• Add or expand tests; coverage checks both ✅
• Add, expand, or update documentation.
• Update doc/whatsnew.

[codecov]

Codecov Report

❌ Patch coverage is 14.81481% with 230 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.2%. Comparing base (b51d8f0) to head (cb5cabf).

Files with missing lines	Patch %	Lines
message_ix_models/model/bmt/utils.py	12.8%	61 Missing ⚠️
message_ix_models/model/buildings/build.py	12.3%	57 Missing ⚠️
message_ix_models/model/bmt/workflow.py	0.0%	37 Missing ⚠️
message_ix_models/model/material/build.py	3.2%	30 Missing ⚠️
message_ix_models/model/bmt/cli.py	39.4%	23 Missing ⚠️
message_ix_models/model/material/config.py	0.0%	13 Missing ⚠️
message_ix_models/tools/add_dac.py	0.0%	6 Missing ⚠️
message_ix_models/util/context.py	25.0%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #433     +/-   ##
=======================================
- Coverage   75.5%   74.2%   -1.4%     
=======================================
  Files        270     277      +7     
  Lines      22153   22407    +254     
=======================================
- Hits       16740   16639    -101     
- Misses      5413    5768    +355     

Files with missing lines	Coverage Δ
message_ix_models/cli.py	93.4% <ø> (ø)
message_ix_models/model/bmt/__init__.py	100.0% <100.0%> (ø)
message_ix_models/model/buildings/config.py	100.0% <100.0%> (ø)
message_ix_models/model/material/cli.py	23.9% <ø> (ø)
...sage_ix_models/model/material/data_power_sector.py	7.2% <ø> (+7.2%)	⬆️
message_ix_models/util/context.py	95.2% <25.0%> (-1.6%)	⬇️
message_ix_models/tools/add_dac.py	0.0% <0.0%> (ø)
message_ix_models/model/material/config.py	0.0% <0.0%> (ø)
message_ix_models/model/bmt/cli.py	39.4% <39.4%> (ø)
message_ix_models/model/material/build.py	52.5% <3.2%> (-14.2%)	⬇️
... and 3 more

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[khaeru]

Here and in .buildings.build, I think it would be preferable to adjust the build() or main() function to take a Context object, and then decide what to do/not do based on settings attached to it, like a .material.Config or .buildings.Config instance.

This way we could hopefully avoid duplicating methods, and reduce the risk that 2+ different 'build' methods diverge from one another.

[khaeru]

(As always, happy to try to help make this change.)

[yiyi1991]

Thanks, yes, that is what is supposed to be here.
(Duplicating the build() or main() for now simply to avoid touching something that so far works well.)

Let me list the requirements I can think of for now.

• The build() or main() are also called in CLI (e.g., mix-models material-ix build --), so it should still be able to collect CLI options that users put in.
• When they are called in other workflows, they collect what is in the config (config appending to the original context) and build.

That is how I organize the duplicated build functions. It would be highly appreciated if you could help take a look at the duplicated builds when you have time (...it works but i feel my way is not the most smart/clean way...), or just point out the lines showing how transport build cover these.

[khaeru]

The key transport lines are:

message-ix-models/message_ix_models/model/transport/build.py

Lines 647 to 652 in b51d8f0

	# Set up a Computer for input data calculations. This also:
	# - Creates a Config instance
	# - Generates and stores context.transport.spec, i.e the specification of the
	# MESSAGEix-Transport structure: required, added, and removed set items
	# - Prepares the "add transport data" key used below
	c = get_computer(context, scenario=scenario, options=options)

and:

message-ix-models/message_ix_models/model/transport/build.py

Lines 529 to 545 in b51d8f0

	# Update .model.Config with the regions of a given scenario
	context.model.regions_from_scenario(scenario)
	# Ensure an instance of .transport.Config
	if options is not None and "config" in options:
	# Use an instance passed as a keyword argument
	config = context.transport = options.pop("config")
	if len(options):
	raise ValueError(
	"Both config=.transport.Config(...) and additional options={...}"
	)
	elif "transport" in context:
	# Retrieve the current .transport.Config. AttributeError if no instance exists.
	config = context.transport
	else:
	# Create a new instance using `kwargs`
	config = Config.from_context(context, options=options)

The latter seems complicated, but it is basically trying to cover for different CLI, testing, and other uses of the code, and also allow to selectively override one or a few Config settings. For a minimal implementation, you could try something like:

if "material" in context:
    # Existing config instance, don't do anything
    log.info(f"Use existing materials config: {context.material!r}")
else:
    # There's no materials Config instance, create a new one
    context["material"] = material.Config(**kwargs)

[khaeru]

Probably equivalent to:

print(repr(context.asdict()))

[yiyi1991]

ah, thx