Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposed fix for issue #282: search and remove ImDrawCallback_ImCanvas after m_DrawListFirstCommandIndex #285

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

pthom
Copy link

@pthom pthom commented Mar 20, 2024

Hi,

This is a draft for a fix for #282, with lengthy explanations inside the code, because it needs to be further analyzed.

Analysis of the bug #282
when using this repro: https://github.com/pthom/node_window_clipping_issue/tree/suspend_resume_issue2

At the 4th call of LeaveLocalSpace(), we have:

this = {ImGuiEx::Canvas *}
     m_DrawList = {ImDrawList *}                      mDrawList is of size 4
         CmdBuffer = {ImVector<ImDrawCmd>}            its elements at index 2
             Size = {int} 4                           has UserCallback == {ImDrawCallback_ImCanvas}
             Capacity = {int} 8
             Data = {ImDrawCmd *}
     m_ExpectedChannel = {int} 0
     m_DrawListFirstCommandIndex = {int} 2
     m_DrawListCommadBufferSize = {int} 1
     m_DrawListStartVertexIndex = {int} 8

We have m_DrawListFirstCommandIndex = 2, however, the tests in the original code will test only at index 0 and 1:

if (m_DrawList->CmdBuffer.size() > m_DrawListCommadBufferSize && m_DrawList->CmdBuffer[m_DrawListCommadBufferSize].UserCallback == ImDrawCallback_ImCanvas).   
    m_DrawList->CmdBuffer.erase(m_DrawList->CmdBuffer.Data + m_DrawListCommadBufferSize);
else if (m_DrawList->CmdBuffer.size() >= m_DrawListCommadBufferSize && m_DrawList->CmdBuffer[m_DrawListCommadBufferSize - 1].UserCallback == ImDrawCallback_ImCanvas)
    m_DrawList->CmdBuffer.erase(m_DrawList->CmdBuffer.Data + m_DrawListCommadBufferSize - 1);

Proposed solution: test all commands from index >= m_DrawListFirstCommandIndex
and remove the one with UserCallback == ImDrawCallback_ImCanvas
(based on the original code, it seems there can be only one)

        int idxCommand_ImDrawCallback_ImCanvas = -1;
        for (int i = m_DrawListFirstCommandIndex; i < m_DrawList->CmdBuffer.size(); ++i)
        {
            auto & command = m_DrawList->CmdBuffer[i];
            if (command.UserCallback == ImDrawCallback_ImCanvas)
            {
                idxCommand_ImDrawCallback_ImCanvas = i;
                break;
            }
        }
        if (idxCommand_ImDrawCallback_ImCanvas >= 0)
            m_DrawList->CmdBuffer.erase(m_DrawList->CmdBuffer.Data + idxCommand_ImDrawCallback_ImCanvas);

…tFirstCommandIndex (proposed fix for thedmd#282)

        Analysis of the bug thedmd#282
        when using this repro: https://github.com/pthom/node_window_clipping_issue/tree/suspend_resume_issue2

        At the 4th call of LeaveLocalSpace(), we have:

            this = {ImGuiEx::Canvas *}
                 m_DrawList = {ImDrawList *}                      mDrawList is of size 4
                     CmdBuffer = {ImVector<ImDrawCmd>}            its elements at index 2
                         Size = {int} 4                           has UserCallback == {ImDrawCallback_ImCanvas}
                         Capacity = {int} 8
                         Data = {ImDrawCmd *}
                 m_ExpectedChannel = {int} 0
                 m_DrawListFirstCommandIndex = {int} 2
                 m_DrawListCommadBufferSize = {int} 1
                 m_DrawListStartVertexIndex = {int} 8

         We have m_DrawListFirstCommandIndex = 2, however, the tests in the original code will test only at index 0 and 1!
                if (m_DrawList->CmdBuffer.size() > m_DrawListCommadBufferSize && m_DrawList->CmdBuffer[m_DrawListCommadBufferSize].UserCallback == ImDrawCallback_ImCanvas)
                    m_DrawList->CmdBuffer.erase(m_DrawList->CmdBuffer.Data + m_DrawListCommadBufferSize);
                else if (m_DrawList->CmdBuffer.size() >= m_DrawListCommadBufferSize && m_DrawList->CmdBuffer[m_DrawListCommadBufferSize - 1].UserCallback == ImDrawCallback_ImCanvas)
                    m_DrawList->CmdBuffer.erase(m_DrawList->CmdBuffer.Data + m_DrawListCommadBufferSize - 1);

        // Proposed solution: test all commands from index >= m_DrawListFirstCommandIndex
        // and remove the one with UserCallback == ImDrawCallback_ImCanvas
        // (based on the original code, it seems there can be only one)
        int idxCommand_ImDrawCallback_ImCanvas = -1;
        for (int i = m_DrawListFirstCommandIndex; i < m_DrawList->CmdBuffer.size(); ++i)
        {
            auto & command = m_DrawList->CmdBuffer[i];
            if (command.UserCallback == ImDrawCallback_ImCanvas)
            {
                idxCommand_ImDrawCallback_ImCanvas = i;
                break;
            }
        }
        if (idxCommand_ImDrawCallback_ImCanvas >= 0)
            m_DrawList->CmdBuffer.erase(m_DrawList->CmdBuffer.Data + idxCommand_ImDrawCallback_ImCanvas);
@IceLuna
Copy link

IceLuna commented May 5, 2024

I had the same crash and the PR seems to fix it

IceLuna pushed a commit to IceLuna/imgui-node-editor that referenced this pull request May 5, 2024
@thedmd thedmd force-pushed the develop branch 3 times, most recently from bd1924c to b302971 Compare July 21, 2024 23:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants