Do not unload subassets or the root asset unless all subassets and root asset handles have been dropped.

[andriyDev]

The problem

If you do not use subassets, assets work quite intuitively. You call AssetServer::load, you get a handle, and as long as that handle is held, the asset system will keep your asset loaded, and will not reload the asset even if you call load again (barring hot reloading).

In contrast, if you do use subassets, it's chaos. The problem is that the lifetimes of subassets are all separate. This means one subasset may have been dropped, while others are still being used. This results in weird situations like an asset being reloaded just because another of its (expired) subassets got loaded. These distinct lifetimes makes solving problems like #12756 extremely difficult - we can't know whether the subasset hasn't loaded yet, or its handle has just expired since no-one was holding it.

A solution

Instead of dropping subassets as soon as that subasset's handle has been dropped, only drop a subasset or the root asset if all subassets and root asset handles are gone. In other words, extend make the lifetime of all subassets and root assets for a path be the same.

Example 1: glTF

glTF is a format specifically designed for easy transmission. Unfortunately, Bevy struggles with it quite a bit. One nasty issue is that you can easily incur multiple loads. For example, you have a character with mesh asset_server.load("character.gltf#Mesh0/Primitive0"). You make it play animation asset_server.load("character.gltf#Animation0") - an idle animation, then you set its animation to asset_server.load("character.gltf#Animation1") - a walk animation, and finally back to asset_server.load("character.gltf#Animation0"). You've incurred up to 4 loads of all the glTF data here, since you didn't store all those handles.

In this particular case, the answer is somewhat obvious in hindsight - just hold the handles for all the assets you want to use ahead of time. But this is not intuitive behavior to say the least - the user is using an asset that they've loaded, yet the asset keeps reloading in the background because they keep missing one of the subassets.

By applying the proposed solution, this would incur one load, and then the glTF data would just remain in memory. Since you're using some parts of the asset, the whole asset stays alive.

Example 2: #21222

This issue is just about our documentation, but the real problem is the fact that this logic is confusing to begin with! We shouldn't need to guess which asset loader an asset came from. On one hand, maybe we should be storing

An immediate benefit we get is we can avoid loads for subassets that definitely don't exist. If the root asset is loaded, but the subasset isn't, then that subasset label is definitely not a subasset - we can skip the load here entirely!

Risks/Concerns

1. Since a file is either loaded or not, some parts of an asset that you aren't using may continue to be loaded. This incurs a memory cost.
   1. This is certainly a concern, but how often are users taking significant advantage of this? Probably not that often. The common use case is likely loading all or nothing - loading an entire glTF scene, or an entire sound file. The "easy answer" to this is to encourage users to make more granular assets. So if you only want a small part of your glTF file, make that its own file. We already encourage this by the nature that we load the whole asset and then just discard everything we don't use - users likely don't want to pay the cost of loading all the unused data anyway, so the recommendation remains the same. For glTF specifically, we should invest in one-to-many asset processing (mentioned here Bevy Asset V2 Tracking Issue #9714), and provide a processor for splitting glTF files into nicer units.
2. Users may be using the current behavior to only load small parts of their files, like a ZIP file loader for example.
   1. This is a bad solution, but it works. Users pay the cost of loading the whole ZIP, but they get out their one file in the end. Instead, we should resolve Improve support for "random-access" assets (e.g., ZIP files) #21641. I think at least solving that issue partially is necessary before we commit to this change, since otherwise we leave users with no recourse (though maybe we can just tell users to use nested immediate asset loading and then take the assets they want - this will leave them in the same state as before, loading the whole asset just for one subasset, but at least they won't keep the rest of the asset in memory).

---

Two discussions on this:
https://discord.com/channels/691052431525675048/749332104487108618/1430629996438749184
https://discord.com/channels/691052431525675048/749332104487108618/1439999328566644746

[andriyDev]

The implementation here is also non-trivial. Currently when users call add_labeled_asset we return a Handle back which users will often store (and that is somewhat expected). With the above proposal, this would result in a reference cycle, since the root asset would hold the subasset handle, and we only drop the root asset if all handles to its subassets and root assets are dropped - which will never be the case now.

One solution could be to return a Weak<StrongHandle> from add_labeled_asset or some such wrapper. This is a type I end up needing frequently, so I think it's worthwhile to add regardless. It allows the holder to upgrade it into a regular Handle if the handle is still alive, and doesn't require accessing the ECS in order to get a handle (great for any async tasks). This is error prone though, since users in their loaders can just upgrade their handles and then be left with a leaked object. In addition, nested loaded assets need handles, not just Weak<StrongHandle>, since those nested assets will have a different lifetime from the root asset. So now we've essentially partitioned these kinds of handles, making it hard to work with both!

Another solution: what if add_labeled_asset returned nothing? Instead of assets storing handles to their subassets, we just require you to call load again. So for example, the Gltf asset would no longer store scenes: Vec<Handle<Scene>>. Instead, maybe we create a method Gltf::get_scene(asset_server: &AssetServer, scene_index: usize) -> Handle<Scene>. Internally, this method would just call AssetServer::load with the correct path. This solution would cause some pain though, as many users may like just having handles they can work with rather than having to go through the AssetServer to fetch them. But it's quite straight forward, and is the common solution to avoid cycles in reference counting - don't use pointers, just use the key and do a lookup!

[andriyDev]

Here's what I've been trying to do towards this.

1. Solve Allow adding and removing asset sources at runtime. #21758. This means we can actually load archive assets at runtime.
2. Implement a ZIP asset source for Improve support for "random-access" assets (e.g., ZIP files) #21641. This should prove out that it is possible to load parts of a file without loading the whole thing - and without keeping all its subassets in memory. Using the previous step, we can add any zip files we want to partially load at runtime.
3. Implement this proposal.
4. Use that to solve Loading an asset and one of its subassets causes two loads of the same asset to occur. #12756. It should be much easier once we know that loads are per-file instead of per-handle.

[andriyDev]

I realized a rather big blocker though: nested immediately loaded assets. These assets are in an awful situation. When you do a nested immediate load, you get back a LoadedAsset. This contains the nested root asset, plus the nested subassets, and their handles. Note however that this doesn't include the nested root asset's handle. That's because the nested root handle is only created when you call LoadContext::add_loaded_labeled_asset in the root asset's loader. So you now have an asset (the nested asset) whose subassets are loaded, but not its root asset.

This is very tricky to resolve. In the past I've advocated for making nested loads just get sent to the ECS. See #18295 and some discussion here https://discord.com/channels/691052431525675048/749332104487108618/1354530035570180227. I still think that's the right move, but either way we need to resolve this. We either need to make the whole nested asset be sent to the ECS, or make the whole nested asset be "owned" by the root loader (and somehow register the nested handles to be relative to the root asset path).

[greeble-dev]

Adding a few notes here following some Discord chat. I think I'm broadly in agreement with @andriyDev but differ in the details.

Motivation

On the proposed behaviour, I want to make an argument from user explainability. The current behaviour makes sense to someone familiar with the asset system internals - but it's hard to explain to anyone else why loading two things from the same file loads the entire thing twice. The proposed behaviour might be a little surprising, but I think it's much easier to explain that everything from the same file stays together.

Reference Counting Cycles

To solve reference counting cycles between assets and their sub-assets, I'm tending towards:

• Add a struct InternalHandle(AssetId) newtype.
• LoadContext::add_labeled_asset returns a InternalHandle.
• Internal references between super-assets and sub-assets use an InternalHandle.

This quietly guides asset authors towards the right solution - creating a sub-asset gives them type they should store. And in some cases it's invisible to users of assets (Assets::get can take a InternalHandle).

But it's still annoyingly fiddly. Turning InternalHandle into a Handle requires AssetServer::get_id_handle. And users could accidentally copy the InternalHandle and not realise it's a weak reference.

An alternative path is to use AssetId but rename it to WeakHandle. Doesn't directly solve anything, but does provides a clearer hint.

Immediate Nested Loads

I think the immediate nested load issue is an inevitable outcome of how loaders work:

1. Loaders return an asset and sub-assets, which may contain a Handle to another asset.
2. The loader must ensure that any Handles are loading or loaded. The user shouldn't get a Handle and find it never loads.
3. LoadedAsset::dependencies should contain the exact set of handles stored in the asset.

For deferred nested loads this is fairly straightforward, since LoadContext::load will both kick off the load and remember the dependency. There is a small risk that the loader never actually stores the handle, so it's dropped and the asset is left with an unloaded dependency.

For immediate nested loads in current main, 2) and 3) are easy to break - the loader can grab whatever handles it likes and LoadContext can't track them. #21863 fixes 2), but at the cost of loading more than might be needed. 3) is trickier.

I think all paths lead to making VisitAssetDependencies required, so the system can find the exact set of handles and kick off any that aren't loading. This sucks for all kinds of reasons, but I can't see another reliable solution.

(The Asset V2 PR talks about this too - search for "Implied Dependencies vs Dependency Enumeration". Just gonna note that I disagree with the last point - parallel loads can be started, and this is automatic for most cases since the loader needs to call LoadContext::load to get a handle.)

There's some silver lining if VisitAssetDependencies is reliable and can be expanded a bit. It could do validation, like "is this asset correctly use InternalHandle versus Handle?". And it could give finer-grained debugging info, like "exactly which member is referencing this particular asset?".