Custom level generation #750

thicco-catto · 2025-11-03T11:51:27Z

Added a new callback and a couple functions that allow for mods to generate custom floors.

MC_PRE_GENERATE_DUNGEON
Called before level::generate_dungeon is called, which places all the rooms in a floor. Passes an RNG.
Return true to cancel vanilla floor generation.

Level:ResetRoomList(bool)
Needs to be called before placing any rooms in the custom level.
The bool is the same as the one in level::init. Testing it so far, setting it to true makes it work all the time.

Level:SetLastBossRoomListIndex(int)
Also needs to be called after creating the custom floor, since not setting it crashes the game when continuing.
Doesn't actually need to be set to a boss room for the game to not crash, but I haven't tested any other side effects apart from boss rooms not spawning the trapdoor if they are not the last.

Sample

local TestMod = RegisterMod("Test Mod", 1)

local function place_at(room, col, row, seed)
    local level = Game():GetLevel()

    local entry = Isaac.LevelGeneratorEntry()
    entry:SetAllowedDoors(15)
    entry:SetColIdx(col)
    entry:SetLineIdx(row)

    level:PlaceRoom(entry, room, seed)
end

local function get_normal_room(seed)
    return RoomConfigHolder.GetRandomRoom(seed, false, StbType.BASEMENT, RoomType.ROOM_DEFAULT, RoomShape.ROOMSHAPE_1x1, -1, -1, 0, 10, 15)
end

TestMod:AddCallback(ModCallbacks.MC_PRE_GENERATE_DUNGEON, function (_, rng)
    local level = Game():GetLevel()
    level:ResetRoomList(true)

    print(rng:GetSeed())

    local start_room = RoomConfigHolder.GetRandomRoom(rng:Next(), false, StbType.SPECIAL_ROOMS, RoomType.ROOM_DEFAULT, RoomShape.ROOMSHAPE_1x1, 2, 2)
    local big_room = RoomConfigHolder.GetRandomRoom(rng:Next(), false, StbType.BASEMENT, RoomType.ROOM_DEFAULT, RoomShape.ROOMSHAPE_2x2, -1, -1, 0, 10, 255)
    local boss_room = RoomConfigHolder.GetRandomRoom(rng:Next(), false, 0, RoomType.ROOM_BOSS, RoomShape.ROOMSHAPE_1x1)

    place_at(start_room, 6, 6, rng:Next())
    place_at(get_normal_room(rng:Next()), 7, 6, rng:Next())
    place_at(boss_room, 8, 6, rng:Next())
    place_at(get_normal_room(rng:Next()), 6, 7, rng:Next())
    place_at(get_normal_room(rng:Next()), 6, 8, rng:Next())
    place_at(get_normal_room(rng:Next()), 7, 8, rng:Next())
    place_at(get_normal_room(rng:Next()), 8, 7, rng:Next())
    place_at(big_room, 8, 8, rng:Next())

    level:SetLastBossRoomListIndex(2)

    return true
end)

EliteMasterEric · 2025-11-04T22:07:20Z

Was going to test this out but it appears to be under heavy development still (I downloaded d313242 and after figuring out the dungeon generator controller, invoking it caused a 0xc0000005 (access violation) error).

Very excited for it to be stable enough to do more stress testing with.

ConnorForan · 2025-11-05T01:33:56Z

repentogon/Patches/ASMPatches/ASMLevel.cpp

The code added here is actually in the wrong place, technically. This directory is primarily for Assembly (ASM) Patches

Since this is just function hooks, should probably move this stuff to repentogon/LuaInterfaces/CustomCallbacks.cpp

ConnorForan · 2025-11-05T01:38:08Z

repentogon/Patches/ASMPatches/ASMLevel.cpp

+		return false;
+	}
+
+	level->reset_room_list(false);


I feel like it might make sense for the logic to reset & build out the floor be a DungeonGenerator function (like generator->Generate())

May also be good for it to return a success boolean, in case we decide it failed somehow, we could maybe fall back to allowing the vanilla generation to happen again.

ConnorForan · 2025-11-05T01:43:36Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+}
+
+LUA_FUNCTION(place_room) {
+	DungeonGenerator* generator = GetDungeonGenerator(L);


I think it would be good for DungeonGenerator itself to contain all its functions such as PlaceRoom (ie, generator->PlaceRoom(config, row, col, seed), and the lua bindings just parse the inputs from lua and redirect to the generator's functions. This will separate the DungeonGenerator logic from the lua bindings, and allow DungeonGenerator code to call its own functions.

ConnorForan · 2025-11-05T01:44:41Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+	return *lua::GetRawUserdata<DungeonGenerator**>(L, 1, lua::metatables::DungeonGeneratorMT);
+}
+
+LUA_FUNCTION(place_room) {


nitpick: Use names like Lua_PlaceRoom for the lua bindings, mostly just a consistency thing (not that our consistency is fantastic overall)

ConnorForan · 2025-11-05T01:47:41Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+#include "Log.h"
+#include "LuaDungeonGenerator.h"
+
+DungeonGenerator* GetDungeonGenerator(lua_State* L) {


LUALIB_API DungeonGenerator* GetDungeonGenerator(lua_State* L)

I don't know exactly how important the LUALIB_API bit is, but its typically used for such functions. Might impact properly surfacing lua errors.

ConnorForan · 2025-11-05T02:08:17Z

repentogon/LuaInterfaces/LuaDungeonGenerator.h

+};
+
+struct DungeonGenerator {
+	int a;


What is a?

ConnorForan · 2025-11-05T02:08:50Z

repentogon/LuaInterfaces/LuaDungeonGenerator.h

+	bool is_final_boss = false;
+
+	DungeonGeneratorRoom() {
+		this->room = NULL;


nit: Prefer nullptr instead of NULL

ConnorForan · 2025-11-05T02:12:21Z

repentogon/Patches/ASMPatches/ASMLevel.cpp

+		DungeonGeneratorRoom generator_room = generator->rooms[i];
+
+		if (generator_room.room != NULL) {
+			LevelGenerator_Room* level_generator_room = new LevelGenerator_Room();


As pointed out by Guantol, this is a memory leak atm

You can initialize a local variable class/struct as just LevelGenerator_Room level_generator_room; and it will be cleaned up when it leaves scope as you'd expect. This also calls the default constructor (for classes at least) automatically where applicable.

Initializing classes with new requires explicit freeing later, though there's usually better options than new

ConnorForan · 2025-11-05T02:16:40Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+	uint32_t col = (uint32_t)luaL_checkinteger(L, 4);
+	uint32_t seed = (uint32_t)luaL_checkinteger(L, 5);
+
+	DungeonGeneratorRoom* generatorRoom = new DungeonGeneratorRoom(config, row, col, seed);


This initializes a DungeonGeneratorRoom, and then saves a COPY of it into the array on the next line. The original is also a memory leak.

I think you can initialize it like this:

generator->rooms[generator->num_rooms] = DungeonGeneratorRoom(config, row, col seed); generator->num_rooms++; DungeonGeneratorRoom* generatorRoom = &generator->rooms[generator->num_rooms];

This should initialize a DungeonGeneratorRoom inline and assigns it into the array (no copying and no memory management required).

The three example line above would probably be good as a small helper function (ie, DungeonGeneratorRoom* generatorRoom = generator->CreateRoom();?

ConnorForan · 2025-11-05T02:19:39Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+	DungeonGenerator* generator = GetDungeonGenerator(L);
+	RoomConfig_Room* config = lua::GetLuabridgeUserdata<RoomConfig_Room*>(L, 2, lua::Metatables::CONST_ROOM_CONFIG_ROOM, "RoomConfig");
+	uint32_t row = (uint32_t)luaL_checkinteger(L, 3);
+	uint32_t col = (uint32_t)luaL_checkinteger(L, 4);


Validate the row/col (both bounds checking and verifying that there isn't already a room in that slot)

In theory we'll also need to do other stuff like handling overlapping large rooms and checking door slots, but you don't HAVE to add that right now, unless you want to

Guantol-Lemat · 2025-11-07T23:18:13Z

repentogon/LuaInterfaces/CustomCallbacks.cpp

+
+HOOK_METHOD(Level, generate_dungeon, (RNG* rng) -> void)
+{
+	bool skip = ProcessGenerateDungeonCallback(this, *rng, 0);


Minor nitpick, but I would suggest replacing the magic numbers for the DungeonType into either an enum or a constexpr, to make the code more readable.

Guantol-Lemat · 2025-11-07T23:23:27Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+
+	std::vector<RoomCoords> forbidden_coords = GetForbiddenNeighbors(base_coords, room_shape, doors);
+
+	for (int i = 0; i < this->num_rooms; i++) {


I would suggest having the generator hold a map that ties gridIdx to roomIdx, in order to quickly find the neighboring rooms, and see if a room already occupies this room's slot, as going through the whole room list is not really efficient.

Guantol-Lemat · 2025-11-07T23:27:00Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+	return generatorRoom;
+}
+
+void DungeonGenerator::SetFinalBossRoom(DungeonGeneratorRoom* boss_room) {


I would have the generator hold a single value that defines the index of the final boss room. This would trivially guarantee the constraint that only one room can be the FinalBossRoom, as well as being faster to set and check.

Guantol-Lemat · 2025-11-07T23:30:20Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+
+#pragma endregion
+
+#pragma region Helpers


I would suggest placing these functions inside of their own Utils file, since they can also reused internally and are not strictly for the LuaDungeonGenerator.

Guantol-Lemat · 2025-11-07T23:34:42Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+LUA_FUNCTION(Lua_PlaceDefaultStartingRoom) {
+	DungeonGenerator* generator = GetDungeonGenerator(L);
+
+	int doors = (int)luaL_optinteger(L, 2, 15);


Since the start room has essentially a predefined config, and GetRandomRoom can only ever pick one room, due to the variant range being set to just the value 2, there is no need to pass the doors parameter.

The doors parameter limits the allowed doors for the room. It could be useful to mimick the behaviour of the mega satan door/polaroid door

It would be better to use a system similar to the blocked grid indices, rather than having the user having to deal with doors directly. In the original level gen the doors parameter is meant as a way to specify which doors are necessary, not those that are allowed. Of course, we should still allow control over the doors parameter, when doing very specific things, like how secret rooms and ultra secret room generation has to mark certain doors as allowed, post layout generation.

I was basing the doors parameter on the LevelGeneratorEntry.doors parameter, since I'm later setting the LevelGeneratorEntry doors field to that. There it does specify the doors argument refers to the allowed doors.
https://repentogon.com/LevelGeneratorEntry.html

LevelGeneratorEntry is a wrapper for a LevelGenerator_Room and LevelGenerator_Room only specifies the necessary rooms, not the allowed ones. Most likely the parameter was named like that because the necessary rooms end up being used as the allowed rooms when placing the room.

Guantol-Lemat · 2025-11-07T23:37:55Z

repentogon/LuaInterfaces/CustomCallbacks.cpp

+}
+
+HOOK_METHOD(Level, generate_blue_womb, () -> void) {
+	bool skip = ProcessGenerateDungeonCallback(this, g_Game->_generationRNG, 1);


For now I would suggest forcibly making it impossible to generate layouts on dungeon types other than NORMAL, perhaps with an error message explaining that they are currently not supported, since we need to properly evaluate the constraints needed to make the other dungeon types work properly first.

Would it be better to just skip the callback call and write a note in the docs explaining that the callback only works for default generation?

Guantol-Lemat · 2025-11-07T23:39:44Z

libzhl/functions/Level.zhl

 __thiscall void Level::Update();

+"558bec6aff68????????64a1????????5081ecc0000000a1????????33c58945??535657508d45??64a3????????8bc1":
+__thiscall void Level::reset_room_list(bool unkinitStartRoom);


minor nitpick, change the name of the bool parameter to 'resetLilPortalRoom'

Guantol-Lemat · 2025-11-07T23:43:06Z

repentogon/LuaInterfaces/CustomCallbacks.cpp

+		.push(dungeonType)
+		.call(1);
+
+	if (results || !lua_isboolean(L, -1) || !lua_toboolean(L, -1))


We might want to change this, as right now I expect that if someone returns an invalid level layout, we don't give a chance for other callbacks to run and simply return to default generation.

The logic here seems like it might be fine, but yeah this callback will likely require custom handling in main_ex.lua. We could potentially detect that a mod left things in an invalid state, possibly print an error to the console, then reset it before we run the next callback.

Not mandatory for an initial push, but is something we should sort out before this gets included in a release.

Guantol-Lemat · 2025-11-07T23:44:46Z

repentogon/Patches/ASMPatches/ASMLevel.cpp

 #include "ASMLevel.h"
 #include "../../LuaInterfaces/Level.h"
 #include "../../LuaInterfaces/Room/Room.h"
+#include "../../LuaInterfaces/LuaDungeonGenerator.h"


From my understanding this include is unused, so it should be removed.

Guantol-Lemat · 2025-11-10T18:15:59Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

 	this->shape = room->Shape;
 }

+RoomConfig_Room* DungeonGeneratorRoom::GetRoomConfig(uint32_t seed, int required_doors) {


I presume this function was originally meant to do something else due to the unused parameters and unnecessary check for nullptr, what was this supposed to do originally?

ConnorForan · 2025-11-10T23:55:38Z

repentogon/LuaInterfaces/CustomCallbacks.cpp

+		.push(dungeonType)
+		.call(1);
+
+	if (results || !lua_isboolean(L, -1) || !lua_toboolean(L, -1))


The logic here seems like it might be fine, but yeah this callback will likely require custom handling in main_ex.lua. We could potentially detect that a mod left things in an invalid state, possibly print an error to the console, then reset it before we run the next callback.

Not mandatory for an initial push, but is something we should sort out before this gets included in a release.

ConnorForan · 2025-11-11T00:09:02Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+	this->level_generator._isXL = false;
+}
+
+bool DungeonGenerator::CanRoomBePlaced(XY& base_coords, int shape, int allowed_doors, bool allow_unconnected) {


When placing rooms with RoomConfig_Rooms, we may need to validate somewhere that the two rooms are actually able to mutually connect allowed doorslots to each other. I dont think vanilla really does this since iirc it specifically queries rooms with specific doors available when filling the layout with configs

This is already handled by the allowed_doors parameter. When a room is placed in the layout it blocks all inaccessible grid indexes

ConnorForan · 2025-11-11T00:11:44Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+			2,
+			0,
+			10,
+			(unsigned int*)&required_doors, // If I don't do it like this it shits itself


Just make the int required_doors variable an unsigned int to begin with so you dont have to do the weird casting

ConnorForan · 2025-11-11T00:13:26Z

repentogon/LuaInterfaces/LuaDungeonGenerator.cpp

+		return 1;
+	}
+	else {
+		return 0;


In cases like this, instead of returning 0:

lua_pushnil(L); return 1;

Prefer to only return 0 for functions that never return anything.

Dunno how severely this matters, but it seems like the consistent thing to do

thicco-catto and others added 5 commits November 2, 2025 21:23

Reset room list function

8b17521

Added bool parameter

7c047e1

Added callback enum

ea8320c

Merge branch 'TeamREPENTOGON:main' into level-gen

d2bf067

Added RNG parameter to callback

2490b78

epfly6 requested a review from ConnorForan November 3, 2025 11:57

thicco-catto added 4 commits November 4, 2025 18:43

Added dungeon generator controller

8f8c157

Removed test functions and fixed final boss not set

f9fb019

Callback triggers for more dungeon types

f65b5f5

Filter by generation type

d313242

ConnorForan requested a review from Guantol-Lemat November 5, 2025 00:24

ConnorForan reviewed Nov 5, 2025

View reviewed changes

thicco-catto added 15 commits November 5, 2025 18:30

Moved custom callback to appropiate file

e2f77b4

Fixed return values for lua functions

32e508d

Renamed lua functions

1feba7f

PlaceRoom returns the room

bfab985

Generator returns bool

845072c

Dungeon generator has own RNG

c2cc975

Refactored floor generation

47dbdd9

Moved implementations to source file

7d5ce08

Added room shape check and swapped col/row

57a718d

Added door validation

05c839a

Added shape field to DungeonGeneratorRoom

9a7e37e

Added PlaceDefaultStartingRoom

40605fd

Fixed constructors

797a52d

Added dungeon type param to callback

8ad7897

Changed some pointers to references

2e6f9f4

Guantol-Lemat reviewed Nov 7, 2025

View reviewed changes

Changed resetLilPortalRoom param

e1b7d98

thicco-catto added 8 commits November 8, 2025 11:08

Removed unnecesary include

bd2415e

Use bitset to check room placement

52c50b5

Created utils file and generation type enum

c308a3e

Enum is passed as number to callback

0bbb155

DungeonGenerator now holds index for final boss

d89ff98

Removed some debug logs

fc5098b

Refactored DungeonGenerator to use LevelGenerator

6edd184

Removed ununsed fields

2c37191

Guantol-Lemat reviewed Nov 10, 2025

View reviewed changes

ConnorForan reviewed Nov 11, 2025

View reviewed changes

thicco-catto added 3 commits November 11, 2025 17:04

Added BlockIndex function

86ff2bb

Changed optional return

062d157

Added custom callback logic

4732779


		std::vector<RoomCoords> forbidden_coords = GetForbiddenNeighbors(base_coords, room_shape, doors);

		for (int i = 0; i < this->num_rooms; i++) {

Custom level generation #750

Are you sure you want to change the base?

Custom level generation #750

Conversation

thicco-catto commented Nov 3, 2025

Uh oh!

EliteMasterEric commented Nov 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Guantol-Lemat Nov 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Guantol-Lemat Nov 8, 2025 •

edited

Loading