[BUG] The whole screen is refreshed on each key press

[vvaltchev]

Describe the bug
The whole screen is refreshed on each key press and that is super-ugly on some terminals like the Windows Terminal, plus it will cause a serious lag on slow network connections.

To Reproduce
Steps to reproduce the behavior:

1. Run the demo code in README on WSL using the Windows Terminal:

# -- demo.py --
import pytermgui as ptg

with ptg.WindowManager() as manager:
   demo = ptg.Window(
      ptg.Label("[210 bold]Hello world!"),
      ptg.Label(),
      ptg.InputField(prompt="Who are you?"),
      ptg.Label(),
      ptg.Button("Submit!")
   )
   
   manager.add(demo)
   manager.run()

2. Type something.

Note: this problem does not occur on smarter terminals like Gnome Terminal simply because they redraw the screen in a async way. But, the problem will we visible across network connections, because the whole screen data must be re-sent on each key press.

Expected behavior
When writing text, only the single chars must be written to the screen, without redrawing everything. That is harder to achieve, I know, but it will make a big difference.

System information
I installed the latest version from pip, ptg.__version__ is 6.4.0.

Possible solution
I haven't studied this project enough to propose a reasonable solution, but more or less it's Flutter's problem: UI = Func(data) does not work well in the real world without a lot of hacks under the hood. For example, Flutter compares the new UI tree with the old one of the fly and redraws only the diff.

[bczsalba]

Welcome to the project!

This is something I've working on a fix for on-and-off for a couple of months now. You can see it mentioned in the Compositor class:

pytermgui/pytermgui/window_manager/compositor.py

Lines 230 to 239 in b2bb571

	def draw(self, force: bool = False) -> None:
	"""Writes composited screen to the terminal.
	At the moment this uses full-screen rewrites. There is a compositing
	implementation in `composite`, but it is currently not performant enough to use.
	Args:
	force: When set, new composited lines will not be checked against the
	previous ones, and everything will be redrawn.
	"""

I have a decently performant Canvas implementation locally that I hope to introduce in the near future. The idea there would be to pass the entire screen buffer to the compositor, which can then get all the changes needed from its internal canvas.

This issue is actually much more widespread than it may appear; the sandbox file inputfield.py lags and stutters a lot when doing scrolling while running with the --highlighted flag. This on the surface might seem pretty much completely down to slow Python code, but is in fact mostly a result of the terminal choking under the amount of updates we are sending to it. The reason I know this is the case is that the new TIM engine is way more performant than the current one in the latest release (to the point where warm-cache parses take <200 ns -- 0.0002 milliseconds, making them practically instant), and yet there is 0 discernible difference in performance of the aforementioned sandbox file.

There is also a case to be made to only "render" each widget when it requests it, not on every frame. As mentioned above, this should be a secondary concern at the moment, since Python speed is very rarely going to be the bottleneck. Of course, it will likely still be improved in the future.

Might have gotten a bit too rant-y there, so:

TL;DR: A really cool fix is in works and on the way.

[vvaltchev]

Thank you for the update and I'm happy to hear that you're working on the problem.
And yeah, I totally agree that Python won't be the bottleneck here : even an optimized C implementation of a ncurses app lags (flickers) if the screen is re-drawn of each key-press. The reason is that terminals are slow and complicated by themselves (escape seq. to parse, colors, many layers etc.) and too much data has to be processed by the CPU, typically single-threaded before rendering on the screen. It's kind of the opposite of what happens for video games.

I'm not sure I've understood well enough your first plan with the canvas, but I believe that the 2nd part about just rendering each widget only when it's modified will improve the performance substantially. Also, try it on slow SSH connections, e.g. across the ocean back and forth, and see how a simple app compares to typical ncurses app.

[ajh123]

Seems like this flicker bug is prevalent when moving the mouse in the Windows Terminal as well. With xterm in WSL the flicker is non-existent.

[bczsalba]

While we have made some progress towards helping with this, unfortunately the ideal solution (at least from what I found, implemented in shade40's slate) requires PTG's core to be uprooted and reimplemented, which isn't likely to happen. I'll keep this issue open, but I'll remove the in-progress label.

[vvaltchev]

@bczsalba Thank you for the update, I appreciate it.