"PV in use" cumulative mass in facility inventories is not consistent between scenarios 44 and 95 #326
Description
Background
Running dev-circfutures_liaison at commit a84340e on national circularity futures data branch at commit 3281d8d
Reviewing the mass_cumulative_histories_raw files from scenarios 95 (baseline), 44 (closed loop) and 39 (open loop) (link to scenario 95, link to scenario 44, link to scenario 39) and comparing only the inventories for "pv in use" facility_types. To pinpoint the issue, these inventories were pulled out into another file (here) to see in which years and for which facilities the discrepancy occurs. The expected behavior is that "pv in use" inventories over time remain identical across all scenarios, as these flows are expected to be completely dependent on the exogenous demand signals and unaffected by supply chain configuration.
NOTE: We have not exhaustively checked all other scenarios and facility inventories to see if this is an isolated bug. Depending on the cause, we may want to do that before considering this issue fully resolved.
After comparing facility-level inventories:
- There are no discrepancies between scenario 39 and scenario 95.
- 92 out of 4888 facility inventory values differ between scenario 44 and scenario 95.
- Where discrepancies occur between scenarios 44 and 95, the inventories in scenario 44 are always greater than the analogous inventories in scenario 95.
- The discrepancy is showing up only in timesteps 38 and later
- The discrepancy is showing up only for PV power plants with node IDs in the format P10000XYZ
- The majority of discrepancies are below ~15% and the magnitude of discrepancy ranges from <0.00% up to ~61%, with the following distribution:
As we only see this bug in scenario 44, that would indicate there is faulty logic in how closed loop materials re-enter the supply chain for their second lifetime. It's not clear right now why this only increases the inventory of in use materials - possibly some components are being "double manufactured"? It's also puzzling that only some facilities have inventory discrepancies.
Next steps
- Get more clarity on which type of power plants have this discrepancy, and what happens in or around timestep 38 to cause this discrepancy.
Outcome: 29 facilities show this discrepancy:
<style> </style>facilities
P10000358
P10000359
P10000362
P10000363
P10000364
P10000367
P10000368
P10000369
P10000371
P10000374
P10000375
P10000376
P10000377
P10000378
P10000381
P10000383
P10000384
P10000385
P10000388
P10000389
P10000390
P10000392
P10000393
P10000395
P10000396
P10000400
P10000401
P10000403
P10000404
- Make 100% certain that the source data and exogenous demand is identical between 44 and 39.
Outcome: Source data is identical. See this file
- Run the model until timestep 37, then step through to examine the logic of components entering use. Cross-check with what logic gets activated in scenario 39. (In progress, notes below)
Testing procedure
Attempt 1
In component.py, bol_process, immediately following the "Component waits to transition to in use" timeout, I inserted:
_discrepancies = ['P10000358',
'P10000359',
'P10000362',
'P10000363',
'P10000364',
'P10000367',
'P10000368',
'P10000369',
'P10000371',
'P10000374',
'P10000375',
'P10000376',
'P10000377',
'P10000378',
'P10000381',
'P10000383',
'P10000384',
'P10000385',
'P10000388',
'P10000389',
'P10000390',
'P10000392',
'P10000393',
'P10000395',
'P10000396',
'P10000400',
'P10000401',
'P10000403',
'P10000404']
if self.in_use_facility in _discrepancies and begin_timestep >= 38: pdb.set_trace()
Additionally, in inventory.py, increment_quantity, immediately following the print statement about "Inventory cannot go negative" and the quantity correction, I inserted:
_discrepancies = ['P10000358',
'P10000359',
'P10000362',
'P10000363',
'P10000364',
'P10000367',
'P10000368',
'P10000369',
'P10000371',
'P10000374',
'P10000375',
'P10000376',
'P10000377',
'P10000378',
'P10000381',
'P10000383',
'P10000384',
'P10000385',
'P10000388',
'P10000389',
'P10000390',
'P10000392',
'P10000393',
'P10000395',
'P10000396',
'P10000400',
'P10000401',
'P10000403',
'P10000404']
if self.facility_id in _discrepancies and timestep >= 38:
print(f'Facility {self.facility_id} at timestep {timestep}: Quantity {quantity}')
elif self.facility_id in _discrepancies and timestep > 38:
pdb.set_trace()
Running on scenario 44 on pre-computed locations and routes, fresh instantiation of CostGraph.
Attempt 2
- Get more details on the inventory.increment_quantity logic by accessing quantities by item_name
- Update debug statement in inventory.increment_quantity to also print out item name - not all the quantities printed out were for glass
- In component.bol_process: Double check the format of self.in_use_facility because that debugging statement wasn't triggered as expected.
- Fixed: prepend "pv in use_" to the codes listed above. In inventory.bol_process, the codes are fine as is (that statement worked as expected the first time).
Re-running on scenario 44, pre-computed locations and routes, fresh instantiation of CostGraph. Started 9:06a Dec 11
Post-Test Summary
Conclusion: While I haven't identified a cause for why mass flows in scenario 44 occur in different timesteps relative to scenario 95, the DES does appear to be behaving as expected. One minor change to the data may improve the results (see last point).
- I added several print statements to track the incrementing/decrementing of individual facility inventories at timesteps >= 36. After performing spot checks on the facilities involved in PV panel beginning-of-life (on multiple runs of scenario 44), I didn't find any erroneous panel mass or count flows entering use. All flows matched with the scaled number of technology units in magnitude, although there were 2-3 timesteps of lag between the "install date" in the technology units file and the actual install date of the DES. This is due to components being manufactured in the same timestep as they should be installed; the multiple facilities in between manufacture and install cause the delay.
Sample of files used for spot checks:
- Following the same procedure (spot checks on facility inventory print statements), I checked whether additional mass flows were being "module manufactured" or "module installed" (i.e. if additional flows were being simulated without being manufactured). I found none: all the mass flows I checked were as expected, and the incrementing/decrementing of facilities along the supply chain proceeded correctly.
- I expanded the comparison between scenarios 44 and 95 in this file (path: Walzberg, Julien - CircularityFutures\Task 3\CircularityFuturesTask3_CELAVI\Results_exploration_Scorpion\Results\Check_duplicates_runs\mass_cumulative_histories_raw_44_95.xlsx). Looking at the mass inventories from scenario 44 and 95 for one facility at a time, the mass discrepancies are largely due to flows of the same overall magnitude occurring in different timesteps. (See sheets "module manuf compare", "module install compare", and "pv in use compare".
- Where the code, particularly the data sent for LCA calculations, can be improved is in the solar glass manufacturing steps, both virgin and from cullet.
- Essential data change: The "solar glass manufacturing from cullet" facility types are meant to accumulate solar glass in inventory until it is required to manufacture new components. However, this has not been specified correctly in the scenario[xy].yaml files, leading to these facility inventories being decremented automatically. This was the cause of the recurring "Inventory cannot go negative" warning statements seen in previous runs. The fix required is adding "solar glass manufacturing from cullet" and "window glass recovery" to the pass list under path_split in the scenario YAML file (below). I confirmed with several runs that adding these lines eliminates the "Inventory cannot go negative" warnings.
