Repository navigation
Commodities with levies double-counted in reduced cost calculation #877
Description
For commodities with levies, such as CO2 in the example models, these levies are included in the reduced cost calculation for assets twice: once via Asset::get_operating_cost() and once when we multiply flows by prices (for CO2 in the examples, the price will be the same as the levy).
This is wrong. We need to figure out a principled way to exclude these commodities from one or other of these calculation steps.
@ahawkes has suggested excluding OTH-type commodities from the flow-related calculation. This needs a bit of thought though, as there may be implications to doing this (for example, SED and SVD commodities can also have levies which can inform their prices too).
Marking as "on hold" until we can figure it out.
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
ahawkes commentedon Oct 1, 2025
alexdewar commentedon Oct 1, 2025
Yep! I've opened a separate issue for that: #878
alexdewar commentedon Oct 2, 2025
Overview
I'll expand on this issue description a bit, because I think we need to have a discussion before we start trying to fix this. If anyone else has thoughts too, that would be appreciated (@dalonsoa @tsmbland @Aurashk).
Note that I've opened some more issues re the reduced costs calculation (see #876). We'll tackle these one by one. The results will therefore be wrong until they're all fixed, but it does make working on the problem easier and it makes it easier to review.
The problem is that we are double-counting levies for commodities which have them -- such as CO2 in the example models -- when calculating reduced costs for existing assets (though going forward we will use this method to calculate them for candidates too: #879). There are two places they crop up in the calculation: once when we calculate operating cost and then again when we calculate what we call "revenues from flows" ($price \times flow$ -- for commodities with levies, the price is usually the same as the levy). I'll explain these in more detail below, but note that there are two places we could exclude CO2 from the calculation. The thing is that while we could do either with some justification and fix the results for our example models, I want to make sure that we come up with a solution that works for future models too. For example, it is totally possible to have a levy on an SED commodity, in which case the price may be the same as the levy, so a solution that just sets price to zero for OTH commodities, such as CO2 in the examples, won't work in this case.
If we have to just hack it for now to get the results, then we can do that for the sake of having something working for the MVP, but we should have an issue open to fix it in future. I'd rather not do this if possible, though. My concern is that if we pick a bad heuristic for e.g. excluding certain commodities from the flow revenues calculation then it will work for all the toy examples we use for the next year or so, but will silently break models further down the line and will give broken results without users noticing.
We could also constrain the system for now to prevent users from creating models with configurations which will pose problems for whatever hack we come up with. For example, we could prohibit adding levies to SED and SVD commodities for now, which would then mean we could set the prices of OTH commodities to zero for the calculation. Better to prohibit potentially problematic models than to give bad results that someone might publish! We could always lift these restrictions later if we come up with a better solution.
Reduced cost calculation
This is described in detail in the documentation, but we've written the code in a slightly different way to the equation in the docs (though they are supposed to be equivalent!). Note that there is an option to adjust prices for scarcity, which we are not doing by default, because it causes problems (#677), so we're using the non-adjusted versions of the equations.
I'll spell out what we're doing to make sure we're on the same page and that it sounds right to everyone.
We break it down into what we call "operating cost" and "revenues from flows".
Note that the method for calculating operating cost is also used when calculating cost coefficients for the dispatch optimisation, so if we decide we want to change this part of the code, we should make sure we're not breaking the cost coeff calculation at the same time.
Possible solutions
It's worth remembering that, in the current implementation, commodity prices are calculated as either commodity shadow prices or levies, but if there are both present, then the larger of the two will be used. So option 3 will give a different result for commodities that have both. That feels more right to me, because it actually incorporates both levies and shadow prices in some way (but my intuitions about economics are probably off 😉).
Any thoughts welcome!
Footnotes
== the
costcolumn inprocess_flows.csv. I think they're zero for all the flows in the example models. ↩Although we should exclude the primary output commodity for LCOX, but aren't doing so currently (The price for the commodity of interest should be zero when calculating reduced costs for assets with LCOX #878) ↩
alexdewar commentedon Oct 2, 2025
PS -- I'm wondering now whether this is also a problem for the cost coeff calculation, which is similar to the reduced cost calculation, except that only input flows are considered for the "flow revenues" part. If there was an input commodity with an associated levy, I think it would be double-counted in that calculation too (once for the operating cost bit and once for the flow revenues).
ahawkes commentedon Oct 2, 2025
alexdewar commentedon Oct 2, 2025
By activity cost, I guess you mean reduced cost? We're currently calculating it as$operating\_ cost - revenue\_ from\_ flows$ . It sounds like you're saying we should be adding it instead (the signs might be different to in your formula)?
The cost coeffs for the dispatch optimisation, as in the docs. We use the same operating cost routine here as for the reduced costs.
alexdewar commentedon Oct 2, 2025
Sorry, I think I know what you meant about the sign of the price now... I think in general we're doing the right thing, but there might be something wrong with the way we're handling levies here.
In the case of CO2 for our examples:
There's something wrong here. Would it make sense to give CO2 a negative price instead? I'm wondering whether this should be the case for all levies that are applied to output flows, but not input flows, because a levy on an input flow should also reduce the flow revenue.
That might work, but it smells wrong to me for a couple of reasons:
balance_typeisnet)? We don't want to have to conditionally multiply the price by -1 when calculating flow revenue! That's technically possible (by tracking prices' provenance), but a horrid designIn relation to point 1, it's never been entirely clear to me when it would actually make sense to have a
balance_typeofnet. When would you want to tax a commodity flow by the same amount, regardless if it is being consumed or produced?Given these problems, I'm wondering if it actually makes sense for us to give CO2 a "price" within MUSE2 at all. If it didn't have a price at all, then the double-counting problem would also go away (it'd just be skipped in the flow revenue calculation). If we went down this route, then we'd probably have to prohibit giving levies to SVD and SED commodities as having shadow prices and levies together would complicate things.
Footnotes
I've also noticed that the
balance_typeof CO2 isnetfor all the examples, but this doesn't seem right; as it's a tax on CO2 production, surely it should beprod? In practice it doesn't matter as CO2 isn't actually consumed by any processes in the examples, but if it were (e.g. with some kind of carbon capture gizmo) then presumably we wouldn't want this CO2 consumption to be taxed. ↩ahawkes commentedon Oct 2, 2025
18 remaining items
tsmbland commentedon Oct 7, 2025
I suspect this may not be fully resolved yet so I'm reopening
tsmbland commentedon Oct 8, 2025
Having another read of this and I don't quite agree. Or, rather, I think what we're currently doing is correct (at least when using LCOX). The approach will depend on whether the levy is on coal production or consumption, and whether we're appraising the consuming process or the producing process:
If the levy is on coal consumption, the levy won't be factored into the price of coal*. Therefore:
(*I think this if correct, but if I'm horribly wrong about this then all my other point will be nonsense, so please let me know!)
If the levy is on coal production, the levy will be factored into the price of coal. Therefore:
Overall, the approach (which we are currently doing) is simple: operating cost calculations for all processes should explicitly include consumption levies for commodities they consume, and production levies for commodities they produce.
Since #889, OTH type commodities will never have a commodity price, so the same rule applies.
A few discussion points:
A lot of this breaks down if you have processes producing more than one SED/SVD commodity. To be honest, this scenario complicates a lot of things and we should really go through the code with a fine toothcomb if we want to allow this (which presumably we will in the long run). For now, however, I'd say we should ban this.
Currently there's no way to explicitly give prices to OTH commodities. Let's say one of your processes consumes limestone, and you want to give limestone a price without having to model the processes the produce/extract limestone. This was allowed in MUSE1 via the input files. In MUSE2, you'd have to model limestone as an OTH commodity, and "hack" a price for it either through a levy on consumption or a flow cost. Maybe this is fine, but I don't think it's very intuitive. Maybe we should have another input file (
commodity_prices.csv) that allows users to explicitly set prices for commodities, but this should be limited to OTH type commodities that won't have prices calculated internally.I think our implementation of NPV is flawed (or, at least, I don't understand it). Variable operating costs, flow costs and levies will already be factored into the price of the primary commodity, so if we include the primary commodity in the revenue calculation (which we currently do), then we're going to be cancelling out all of these things. I think this implies the following:
@ahawkes @alexdewar Thoughts/comments welcome, but overall I think we're in a good place
ahawkes commentedon Oct 8, 2025
Sorry yes just re-thinking this through:
We have two separate concepts here - market prices and direct costs (varom and levies). We should not conflate these (and I am guilty of this above - apologies!).
So, total variable operating cost, TVOC, (for LCOX here, so setting the price of the COI to zero)
TVOC = Market element (Impact of commodities output + Impact of commodities input)
For the market element we have prices for all the SED and SVD and possibly some OTH commodities from the commodity balance constraints (the case of OTH, shadow price present only if it is constrained). Anything we don't have a commodity balance constraint for, market commodity price is assumed to be zero. So the calculation is easy - e.g. for outputs, sum of production of each commodity multiplied by the shadow price (for each time slice), and then summed over time slices.
For the direct element, VAROM is easy, and same logic as above applies to levies. A positive levy in input data is always a cost (regardless of whether its production or consumption related), which needs to be handled by appropriate +ve and -ve signs in the equation. And needs to be documented so users know how the number they enter is applied.
For our simple CO2 case, there is no shadow price. But the cost of CO2 emissions to the asset is captured in the direct element.
For our coal consumption levy case, the asset needs to pay this levy so it's included in the direct cost. It does not matter if that levy has had an impact on coal price or not, as it's still the market price of coal, and the asset has to pay that to consume the coal.
tsmbland commentedon Oct 8, 2025
Thanks, I believe we're on the same page. Market element and direct element is a good way of putting it
There's currently no way for OTH commodities to have a shadow price as we don't apply any constraints on these, but I guess this is a plan for the future so we should keep this in mind.
Just so I understand, could you give me an example of a scenario where you might want to apply a constraint to an OTH commodity?
ahawkes commentedon Oct 8, 2025
tsmbland commentedon Oct 8, 2025
Ok interesting.
This is very different from MUSE1 where we iteratively change the CO2 "price" to bring emissions within the budget.
I'm not sure about this to be honest as it seems like you're conflating prices and levies. I guess the shadow price for CO2 in this case (almost certainly negative) is equivalent to the (positive) levy that you'd have to apply on CO2 production to get the same CO2 emissions...
I was enjoying how prices and levies were distinct concepts in MUSE2 (unlike MUSE1), but it seems like giving CO2 a market value in this context throws that distinction out of the window a bit...
Anyway, a future problem
alexdewar commentedon Oct 9, 2025
It seems like we're happy with the way MUSE2 does things for now. There are some interesting discussion points in here though, so I'll convert this to a discussion page so we can revisit.
Given that NPV is broken, I'm wondering whether it might be an idea to gate it behind another option, along with other things that are also known to be broken (i.e. scarcity-adjusted prices: #677). We could have an option in
model.tomlcalledenable_broken_optionsor even something that more strongly discourages users from enabling it, such asplease_give_me_broken_results. What do we think?tsmbland commentedon Oct 9, 2025
Yeah I think this is sensible. We have warnings, but in my experience people tend to ignore these
ahawkes commentedon Oct 9, 2025
Sounds good.