Extreme slowdown withing Love2d / breakpoints ignored

[freds72]

Symptoms

Setting any breakpoints makes the application (Love2d game) extremely slow (but breakpoints are working).
Removing/disabling breakpoints before starting a debug session fixes the slowdown, but breakpoints are ignored.

Deleted workspaceCache did not fix the issue

Note: debug used to work for that project - might be application code related

Environment

Windows 10
Love2d 11.4

VSCode

Version: 1.65.2 (system setup)
Commit: c722ca6c7eed3d7987c0d5c3df5c45f6b15e77d1
Date: 2022-03-10T14:34:03.833Z (3 wks ago)
Electron: 13.5.2
Chromium: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Windows_NT ia32 10.0.19043

[tomblind]

Unfortunately, debugging with breakpoints on will always be slower due to the need for maintaining a line-based hook (basically, every line that's executed needs to be checked if it's a breakpoint). Some work has gone into optimizing this in recent versions, but if it's still too slow there may not be much more that can be done.

[freds72]

that I can understand- thing is, it is something like a 120x slowdown - I cannot understand how my code (?) would trigger that (it works at full speed without debug attached)

[tomblind]

That does seem excessive. If you have anything you'd be willing to share I can take a look at it. It's possible some bug is being triggered by a combination of things.

[sewbacca]

My application also runs much, MUCH slower while debugging. As it turns out, half of the time, it is running the debugger and not my application. As op I am using löve2d.
I used today to learn about flame graphs, and to find out why application is slow. Then I tried profiling with debugging enabled and I may found a few interesting things how the debugger could be speed up,
Consider this flamegraph (sampled using LuaJIT Löve2D 11,4):

The magenta parts are when the debugger runs. It tells me, that it runs 55,2% of the time, that is excessive!
I have only set one conditional breakpoint.
When we zoom into this level:

Look at this! There are not enough samples to even show what my application is doing.
So I looked into the source file:

It seems to me, that you are compiling conditional breakpoints on the fly, which really hurts badly, if you hit this line each frame.
That being said, I understand that this is quite hard to factor out. I would propose to create a precompiled function for each breakpoint, that maps input parameters:

local value1 = ...
return value1 == "foo"

So that input parameters can be mapped by lua, which is much, much faster.
Let's remove any breakpoints and see how it affects performance:

That is what I would expect!
Now the most common case: One pure line breakpoint:

penalty of 48%! Better than a conditional breakpoint, but still slow.
This is by 7% better, so I think the precompiling would save at most 7% time. Unfortunatly in the end it really seems to be a limitation of Lua, but maybe there could be stuff factored out of the debughook function, so that time intensive tasks are done not as often.
If I have enough time on my hand, I will look into it and maybe make a pull request, but I thought untill then, it may be beneficial to share my findings.

EDIT:
Demos:


Both times, profiler enabled.

[stuta]

What tools are you using to create these LuaJIT flame graphs?

[tomblind]

Yeah - conditional breakpoints were one area I haven't put serious effort into optimizing yet (wasn't sure how frequently they were actually used by folks). For regular breakpoints, I'm not sure there's much room for significant gains unless a new strategy is used altogether. Simply having the line-based hook is a huge slowdown, so if there's any chance for a notable speedup, it's going to involve somehow not using that.

A theoretical improvement could be using a function-based hook and switching to a line-based hook only when inside of a function that contains a breakpoint. This increases complexity quite a bit and the possible performance gains are dependent on the structure of the code being debugged, but it might be worth experimenting with.

[sewbacca]

Conditional breakpoints are quite usefull, especially if code is executed frequently. They are much easier to setup than change the code (e.g. an if) and restart the debugger with a line breakpoint.
This seems like a great idea. Maybe setting and unsetting hooks could add a performance penalty, but i don't really know the debug api very well.
I replaced a performance heavy function with another retain the same logic.
Heavy in the sense, that the debugger slowed it down quite significantly.

@stuta basicly I dump stacktraces using luajits profiler, which is now bundled in löve2d 11.4 if you are on Linux or Windows. Mac I am not sure, last time i checked, mac only supported 5.1. Once dumped (and in the correct format), you can feed it into flamegraph, which will output svgs. You can view these in any modern browser (e.g. firefox/chrome). To dump the stacktraces, I modified this gist, to suit my needs. The readme of flamegraph doesn't really explain the format though, so look into flamegraph.pl , which has a nice header comment explaining everything, including the fomat (line based stacktraces, seperated by semicolons and a sample count in the end).
The guy who maintains the repo, has done a neat presentation, i recommend checking out. At least the parts were he explains flamegraphs, icicles as well as differential flame graphs.

[stuta]

Thanks for the profilier info!

[sewbacca]

For anyone interested: I created a sample based profiler for lua over here.