Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2024.11 compatibility hacks #2013

Draft
wants to merge 29 commits into
base: underanalyzer
Choose a base branch
from

Conversation

Dobby233Liu
Copy link
Contributor

@Dobby233Liu Dobby233Liu commented Jan 9, 2025

Description

  • GM 2024.11 introduced "Automatically remove unused assets when compiling". If this is enabled (but as I observed, SDF shaders and filter assets will receive the same treatment even if it isn't enabled), the spots for unused assets in the UTObject pointer lists are written with null, which makes UMT freak out.
  • In addition, a currently always zero property was added to UTFont in GM 2024.11.

This horribly hacks around ModLib code not written with the null entires in mind to carry on if it ever hits them, and the ModTool is touched up to represent/hide the null entries. The unknown property in UTFont is now also accounted for. It should now be possible to load and save unless if there are edge cases.

Depends on #1937.

Fixes #1996, fixes #2006, fixes #2011

Caveats

---

Notes

@Dobby233Liu Dobby233Liu changed the base branch from master to underanalyzer January 9, 2025 06:47
Copy link

github-actions bot commented Jan 9, 2025

Download the artifacts for this pull request here:

GUI:

CLI:

@Dobby233Liu Dobby233Liu changed the title 2024.11 compatibility: Horrible hack to ignore null entries in list chunks 2024.11 compatibility fixes Jan 15, 2025
@Dobby233Liu Dobby233Liu changed the title 2024.11 compatibility fixes 2024.11 compatibility hacks Jan 15, 2025
@Dobby233Liu
Copy link
Contributor Author

Dobby233Liu commented Jan 18, 2025

Note that there's currently a bug, where if you leave null entries exposed on the asset tree for too long and move around too much, maybe click into one of the nulls (the true conditions are uncertain), it could cause a hang or even crash. I'm consistently getting the crash as descripted by the incomplete stacktrace below:

System.ArgumentNullException: Value cannot be null. (Parameter 'key')
   at System.ArgumentNullException.Throw(String paramName)
   at System.Collections.Hashtable.ContainsKey(Object key)
   at MS.Internal.WeakDictionary`2.TryGetValue(TKey key, TValue& value)
   at MS.Internal.Helper.SetItemValuesOnContainer(DependencyObject owner, DependencyObject container, Object item)
   at System.Windows.Controls.VirtualizingStackPanel.InsertContainer(Int32 childIndex, UIElement container, Boolean isRecycled)

Might be a WPF bug, if inquired I will try to grab a full crash dump.

@Dobby233Liu
Copy link
Contributor Author

Dobby233Liu commented Jan 19, 2025

Note that there's currently a bug [...] (the true conditions are uncertain) [...]

Ok now I see, you have to very specifically click into the null entries first

What's even funnier is, if I run ModTool through VSCode, it hangs instead of crashing

@CoolModder
Copy link

Sorry to bother you, but it appears there are two issues:

  1. Shaders are improperly imported creating null entries that cause shaders not to be visible in editor and create errors when saving code (Seen in the Mindwave Demo)
  2. Structs and/or variable set to functions are improperly handled, and not being properly recompiled. This can been seen by trying to import modified minigames from Mindwave Demo into their data.win.

@Bloxxel64
Copy link

oh thank goodness

@MichaelEpicA
Copy link

Code does not work for me, Decompiling just gives me this. /* EXCEPTION!
Underanalyzer.Decompiler.DecompilerException: Decompiler error during AST building: Data left over on VM stack at end of fragment.

@colinator27
Copy link
Member

colinator27 commented Feb 4, 2025

Code does not work for me, Decompiling just gives me this. /* EXCEPTION! Underanalyzer.Decompiler.DecompilerException: Decompiler error during AST building: Data left over on VM stack at end of fragment.

@MichaelEpicA Unsure if this is fixed in upstream Underanalyzer (this PR may need to be updated), but does this happen on ALL code entries, or just some? If it's just some, if you could send a small code entry's disassembly, that would be helpful for debugging (but report it over here: https://github.com/UnderminersTeam/Underanalyzer/issues). Also, you may be able to turn that specific error into a warning in the UTMT settings.

@MichaelEpicA
Copy link

Can confirm this can be turned into a warning in the UTMT settings, and saving appears to work, however I'm doubtful it will load correctly due to UMT reporting a object miscount. It is every single code entry.

@colinator27
Copy link
Member

Can confirm this can be turned into a warning in the UTMT settings, and saving appears to work, however I'm doubtful it will load correctly due to UMT reporting a object miscount. It is every single code entry.

@MichaelEpicA Hmm... if there's a small code entry sample, I guess I'm still curious to look at its disassembly to see if it has to do with actual decompilation, or if there's some other problem at play here.

@MichaelEpicA
Copy link

If so, should I just move this over to https://github.com/UnderminersTeam/Underanalyzer/issues then?

@colinator27
Copy link
Member

If so, should I just move this over to https://github.com/UnderminersTeam/Underanalyzer/issues then?

Sure, I'll probably just close it if it ends up being irrelevant.

@Dobby233Liu Dobby233Liu force-pushed the 2024_11-null-entries-in-chunk-lists-hack branch from e8ad45f to 640d7ae Compare February 4, 2025 13:00
@CoolModder
Copy link

CoolModder commented Feb 4, 2025

Note: the editor (on Mindwave's Demo) shows shaders now
However, when AddAssetsFromList(Data.Shaders, RefType.Shader); is called, it still crashes when compiling.
Also, when attempting to compile code like this:
define_boop("boop", { init: init, fleas: fleas })
It causes an error at 1499 of AssemblyWriter.CS.

@CoolModder
Copy link

While the editor can now save the define_boop("boop", { init: init, fleas: fleas }), it appears there are other issues that appear upon runtime after recompilation including SetStatic not being readded and global scopes being changed to static scopes. This is visible in the Mindwave Demo when attempting to recompile a microgame.
Recompiled is on the left, Original is on the right
image
image

image

Error:

___________________________________________
############################################################################################
ERROR in action number 1
of  Step Event0 for object transition_controller:
Variable <unknown_object>.ms0scr_player_tick(103885, -2147483648) not set before reading it.
at gml_Script_mg_sGrizz
############################################################################################
gml_Script_mg_sGrizz (line -1)
gml_Script_anon@6949@gml_Object_obj_minigame_controller_Create_0 (line -1)
gml_Script_anon@5610@charselect_character_constructor@charselect_character_constructor
gml_Script_anon@4882@anon@4418@charselect_character_constructor@charselect_character_constructor
gml_Object_transition_controller_Step_0

@Dobby233Liu
Copy link
Contributor Author

Dobby233Liu commented Feb 26, 2025

While the editor can now save the define_boop("boop", { init: init, fleas: fleas }), it appears there are other issues that appear upon runtime after recompilation [...]

For now I'm waiting for the new UA-provided compiler to be integrated into the upstream underanalyzer branch, which may or may not solve this


NTS: Two things to check & fix after I come back to this

  • Bogus "if asset list is null" check removals
  • Whether some hacks in CheckForGMxxxx functions I added here are necessary

I'm not going to guess 2024.11 in AlignUpdatedListChunk because it's more likely to be bogus there
Because:
- It's not too useful by default
- The nasty bug can make the tool nearly unusable

Will add UI later
Import/ExportAllStrings and find all references are too scary.
Bytecode conversion scripts are ignored.
@Dobby233Liu Dobby233Liu force-pushed the 2024_11-null-entries-in-chunk-lists-hack branch from afcbff0 to c057ef9 Compare March 7, 2025 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants