-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problem with the g_max_inflight_buffers design? #2
Comments
It may be that I answered my own question, in the course of writing up the above message. But since I'm not sure, I figured I'd still send the message to you and see whether you can clarify my thinking on this. One thing I found while writing up the above is in the doc for For my code, the other thing I found is in the doc for Am I thinking about all this right? It's a confusing new world! :-> |
Wow, I didn't realize Satori had been ported to the macOS screen saver engine! That screen saver, as well as Lunatic Fringe and You Bet Your Head, were my favorite After Dark screen savers back in the classic macOS days. (You must get that a lot.) I just installed it. Thanks for working on that! It's been a while since I worked on this, but from what I recall, CVDisplayLink will always fire on the same thread, so it's okay to pause the thread if the CPU gets ahead of the GPU; no more threads will be created. Is there a reason why the drawing needs to happen on the main thread? Metal really wants you to set up your drawables on a background thread. At the time I wrote this screen saver, I used CVDisplayLink and CAMetalLayer because I wanted to figure out how to make a Metal-powered animation at a low level. If I didn't care about figuring out how this all worked at a low level, I would've just used MTKView, which automates media timing to be synchronized with the display. Maybe that would work better for you? |
Hey, that's nice that you still remember my old stuff. Not as many people do any more. :->
OK, good. I think I understand how it's all working, then.
There was indeed a reason, but I forget what it was. I don't think it really matters, though; the drawable setup for Satori is utterly trivial: two triangles filled with a pre-existing texture, basically. The texture gets calculated and set up in the background, so the main thread is pretty free.
Yeah; when I started doing this my thinking was "ScreenSaverView doesn't inherit from MTKView, so I have to use CAMetalLayer", but of course I could have just created an MTKView subview of my ScreenSaverView. Oh duh. But the CAMetalLayer design seems to be working very well, thanks to your example code. Thanks for the reply! I think I get all this stuff now, so feel free to close this issue unless you want to keep chatting. :-> Nice to meet you! |
Hi again Nick. Actually: while I've got you on the horn, let me ask you something unrelated to the above issues. I'm setting up a display link associated with the current screen in pretty much the way you do it, and everything seems to work well, EXCEPT that the frame rate is inconsistent; it speeds up and slows down by a factor of two for no apparent reason. Because of your comment above, I have now moved my Metal rendering onto the CVDisplayLink's background thread, rather than the main thread, so that isn't the issue. With simple code similar to yours that just samples the clock at the start of each frame and does (current time - previous time), I get a log of time-between-frames like this:
As you can see, at the beginning and the end of these logs the time spent rendering is around 0.008-0.009 seconds, which is more than fast enough to sustain a frame rate of 1/60th of a second, and so the "time since previous" for each frame is around 0.0166. But then there's a period in the middle where the time spent rendering jumps to 0.0270, and so I start missing frames and drop to a frame rate of 1/30th of a second. This is quite visible in Satori's animation. You don't really see anything in between; the time spent rendering is never really in between the two values, when it stalls it stalls for about a full frame. As far as I can tell from Instruments, the main time-sinks in the rendering are [metalLayer_ nextDrawable] and [commandBuffer renderCommandEncoderWithDescriptor:]. When things stall, I think it is probably [metalLayer_ nextDrawable], and I'm suspicious of a function shown in Instruments under nextDrawable named CAImageQueueCollect_ but it's hard to know. I've done a bunch of googling, and as far as I can tell this is out of my control – I'm being pretty careful to return drawables quickly, not request them until needed, etc., I think, and I don't start rendering the next frame until the previous frame has finished rendering so I should never run out of drawables (I have a frames-in-flight semaphore, and it never stalls). To cope with this problem, I've introduced variable-speed logic: I look at the time that elapsed from the start of the previous render to the start of this render, and I advance my animation state proportionally to that time. So if the previous render stalled and we missed a frame, the animation state gets advanced by double the usual amount, making the speed of the animation smooth even if frames get dropped. It seems like the best I can do. I'm wondering if you've confronted this issue, though. I'm surprised that it's so hard for me to get a steady 60fps, since my rendering is so trivial (two triangles per frame, four textures that don't change from one frame to the next, a couple very small constant buffers). I'm wondering if I'm just doing something very basic wrong – failing to set some flag, and thus putting Metal into a less efficient mode than it could be in. Anyway, this is probably all gibberish; sorry to waste your time, if you're not interested. If you are interested, though, I'd be happy to share the code with you as long as you promise to keep it to yourself (I don't own the copyright on it, as it happens; I'm able to release Satori on OS X because of a licensing deal). Thanks! |
Update: it is indeed -nextDrawable. It occurred to me that I could time that call specifically to see how it performs. Usually it returns in about 0.008 s; but now and then it takes about 0.025 s, which kills the next frame. Nothing in between; the method clearly has two different "modes" that it falls into, although it may be that I'm doing something wrong that pushes it into the bad "mode". Anyway, there it is. At this point I think I'll log a bug on Apple and move on, unless you have any ideas. :-> Nice chatting with/at you, enjoy your weekend. :-> |
How complicated is your shader program? Since macOS isn't guaranteed to run on the same GPU across all installations, Metal has to recompile the shader under certain circumstances, which causes drawing to chug a little until the MTLCompilerService task is done recompiling the shader. Maybe that might be why you're seeing a little chugging? Do you see MTLCompilerService doing something while the screen saver runs? |
I've never seen MTLCompilerService in a backtrace. My shaders are pretty simple. I've got less than ten of them, all pretty similar to this:
Shouldn't take it more than a second to compile them all, I wouldn't think, but the missed frames persist for at least ten seconds; I think they just keep happening no matter how long I run for, actually, but I haven't actually tried. I really think it's -nextDrawable; I logged a bug on Apple about it. :-> I just released Satori 2.1.1, a minor bugfix release on top of what you recently downloaded. It starts up faster (the initial image is now calculated in a compute shader in Metal, even though all later images are done on the CPU so as to keep the GPU free for animation). It's also smoother, as a result of smoothing over the missed frames with better animation management – the frames still get dropped but the user can't see it. :-> I'm really happy with it now; I think it's the best Satori has ever been. Metal is a truly amazing technology, it has been really fun learning how to use it. Thanks for your help! Stay well! |
Hi Nick. Your code has been useful for me to understand how to make a screensaver that uses Metal, thanks. I think there might be an issue with it that needs fixing, though. I'm not sure whether this has user-visible consequences or not.
Here's the problem. The
CVDisplayLink
timer set up in-startAnimation
ticks along, firingDisplayLinkCallback
periodically, which calls-animateOneFrame
. That keeps going unless/until-stopAnimation
is called, or the screensaver wakes up. So far, so good.On the other hand,
-animateOneFrame
basically makes an autorelease pool and calls-re_render
, and-rc_render
does the following:dispatch_semaphore_wait(_inflight_semaphore, DISPATCH_TIME_FOREVER);
This blocks the thread until
_inflight_semaphore
says we're clear to proceed with rendering the next frame. And that makes sense; we don't want to get ahead of the GPU too much, and we have a limited number of vector buffers to work with. So far, so good.The issue, it seems to me – if there is an issue! – lies in the way these two pieces of code interact. Suppose the GPU is chronically behind in handling the rendering; it can't keep up with the frame rate we'd like to feed it. Once it gets sufficiently far behind, the
dispatch_semaphore_wait()
call will block waiting for the GPU to catch up; but theCVDisplayLink
will keep firing, and so further calls will be made to-animateOneFrame
, and onward to-rc_render
, which will also block. So if the GPU is slow enough, and just can't keep up, you'll start burning through threads, all blocked indispatch_semaphore_wait()
, adding a new one every 60th of a second or whatever, and releasing one whenever the GPU finishes rendering a frame and_inflight_semaphore
allows one thread to unblock.Is this not a problem for some reason? I guess I'm assuming that
CVDisplayLink
fires on whatever free thread it is given by GCD, and that if that thread blocks, it will then fire next time on whatever different thread it is given by GCD the next time around, or something. But maybe if the thread that it fires on blocks, it just doesn't fire any more until that thread returns?I'm wondering about all this in the context of my own screensaver, Satori, which I just recently ported to Metal (http://www.sticksoftware.com/software/Satori.html). Based on some Apple sample code, I set up my
CVDisplayLink
timer a bit differently:Even if your code does not suffer from the problem I described above, I'm wondering whether maybe mine does, since I shift the rendering work onto the main thread. The main thread is then the one that blocks in
dispatch_semaphore_wait()
, in my code; so then perhaps further firings ofCVDisplayLink
can occur, and clog up background threads, while the main thread is blocked? I'm still trying to get used to how GCD and blocks and such manage threads; this is a new world for me, I've usedNSThread
for multithreading until this recent work moving Satori to Metal.If you're willing to take a few minutes to clarify whether this is an issue, for either your code or mine, and if not, why not, I'd greatly appreciate the guidance. Again, thanks for the very useful example code!
The text was updated successfully, but these errors were encountered: