Skip to content

RenderDoc Integration - #662

Open
TheLostInPlace wants to merge 4 commits into
themrdemonized:all-in-one-vs2022-wpo-mtfrom
TheLostInPlace:renderdoc-integration
Open

RenderDoc Integration#662
TheLostInPlace wants to merge 4 commits into
themrdemonized:all-in-one-vs2022-wpo-mtfrom
TheLostInPlace:renderdoc-integration

Conversation

@TheLostInPlace

@TheLostInPlace TheLostInPlace commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Why

Shader and render work on this engine gets debugged by guessing until a capture exists. RenderDoc captures answer those questions directly, but taking one meant launching the game through RenderDoc's UI by first hooking it into MO2 or the global hook (which rarely works) and pressing a hotkey at the right moment (before APi initialization). This makes captures an engine tool: triggered from the console or a script, at the exact frame you want, from a normal launch with an argument (-renderdoc).

How

The engine loads RenderDoc's in-application API when renderdoc.dll is already in the process, or loads it itself from the bin folder when launched with -renderdoc. It negotiates the newest API the host offers (1.7.0 down to 1.1.0) and keeps the engine's own crash handler. Three console commands drive it:

  • rdoc_capture [1..60] queues frames
  • rdoc_open raises or launches the replay UI on the last capture
  • rdoc_overlay 0/1 toggles the overlay (bare prints the state)
  • -rdoc_refall and -rdoc_cmdlists set the matching capture options

Captures are stored in appdata\renderdoc\captures\<exe>_frameN.rdc and are logged when they finish. On 1.7.0 hosts each capture carries frame, time, camera, fov, render size, level, the second-viewport flag (not PiP, the default engine's) and the eight shader_param vectors as annotations.

R4's PIX markers get colours through the d3d9 PERF entry points RenderDoc hooks, with ID3DUserDefinedAnnotation as the fallback, and render targets get debug names so they appear under their engine names.

Benefits over RenderDoc injection

  • Launch the game normally: the UI can attach to the running process afterwards
  • Capture from a script or console at a chosen moment: including a run of frames, instead of a hotkey
  • Every capture is self-describing: annotated with where and when it was taken
  • Passes are coloured and named: render targets are named, so the event browser reads like the engine's own frame
  • Crash handling stays the engine's own. RenderDoc's default F12 capture hotkey still overlaps the engine screenshot key here, there will be a follow-up PR later disables the host hotkeys and adds more tools

Overhead

With no RenderDoc present: one flag read per frame in FrameMove and one in CRender::Render. Marker sites read a per-frame cached listener state instead of the BeginEvent/EndEvent pair each site issued before, so the no-debugger path is cheaper than it was. d3d9.dll is loaded only once something is already listening. Render target naming is a one-off SetPrivateData at creation. R1, R2 and R3 are untouched apart from two launch-flag enumerators, R3 compiles only the naming helper.

The engine loads renderdoc.dll when it is injected or when -renderdoc is
passed, negotiates the newest api the host implements, and points captures at
the engine data root. Three console commands drive it, rdoc_capture queues
frames, rdoc_open raises the replay ui, and rdoc_overlay shows, hides or
reports the capture overlay.

renderdoc_app.h is renderdoc/api/app/renderdoc_app.h from baldurk/renderdoc v1.x at c26a5405, API 1.7.0, MIT, unchanged apart from LF to CRLF
The r4 passes now open tinted scopes so the Event Browser groups a frame by
stage. Colour only reaches the browser through the d3d9 marker exports, so the
wrapper resolves those by name and falls back to the d3d11 annotation
interface when they are absent. Nothing is emitted until a debugger listens
and the answer is cached per frame, so the marker sites stay cheap.
Render targets carry their engine name into a capture through SetPrivateData,
so a debugger lists them by name instead of by pointer. The guid is spelled
out in the header so nothing has to link against dxguid.
A captured frame now carries the frame number and time, the camera, the render
size, whether the pass is the second viewport, the level name and the eight
shader parameter vectors. The keys need api 1.7.0 and a capture that is
already recording, so they cost nothing otherwise, and a call that the host
refuses reports once instead of every frame.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant