Key Up/Down/Press events don't work as expected. (at least on MacOS)

[Albator11]

Describe the bug
KeyUp triggers the same time KeyDown is triggered, and KeyPress never get triggered at all.

To Reproduce
Simply add event handlers to a view's Key* events, or override the view's OnKey* methods.
The "Keys" scenario in the UICatalog behaves this way.
Or here's a working test I made:

public static class Program {
    private static Window SetupWindow() {
        List<string> keyEvents = new();
        ListView listView = new(keyEvents) {
            Height = Dim.Fill(),
            Width = Dim.Fill(),
        };

        Window window = new();
        window.Add(listView);

        window.KeyDown += args => {
            keyEvents.Add($"KeyDown: {args.KeyEvent}");
            listView.SelectedItem = listView.Source.Count - 1;
        };
        window.KeyUp += args => {
            keyEvents.Add($"KeyUp: {args.KeyEvent}");
            listView.SelectedItem = listView.Source.Count - 1;
        };
        window.KeyPress += args => {
            keyEvents.Add($"KeyPress: {args.KeyEvent}");
            listView.SelectedItem = listView.Source.Count - 1;
        };

        return window;
    }

    public static void Main(string[] args) {
        Application.Init();
        try {
            Window window = SetupWindow();
            Application.Run(window);
        } finally {
            Application.Shutdown();
        }
    }
}

Expected behavior
KeyUp should run when the key is released rather than key is pressed down.
KeyPress should run after both KeyDown and KeyUp events are run.

Desktop

• OS: MacOS
• Version 12.6.8 (Monterey)

Additional context
I think the key events worked properly when I tested it on a Windows computer about a year ago... so I assume this has to do with the CursesDriver?

This has been a problem in Terminal.Gui for a long time, and I've not yet seen any bug report mentioning this... so I finally got around to writing it.

This hasn't bothered me too much for "normal" applications, but I'm trying to make a "real-time" game with movement &c. based on when a key is down/up.

[BDisp]

You are right, it's a limitation with CursesDriver, because ncurses don't recognize a key up event. It only recognize a key pressed, unlike the WindowsDriver. The NetDriver also have that limitation. What we did in both driver is raising sequentially a KeyDown, KeyPress and KeyUp.
I think you are confusing with a mouse click event that have to be fired after button pressed/released/clicked.
With keyboard the key press must be fired before key up and after key down. But remember Terminal.Gui also have another keys events with this order (ProcessHotKey, ProcessKey and ProcessColdKey). If some of this is consumed by a view by returning true will not propagate to another subscribed events.

[Albator11]

Oh, ok.
Well that sucks... and I don't suppose there'd be any way to "patch" ncurses to support a key release, otherwise someone would've already done it probably.

Any ideas on how I could get/simulate a key released event then?
Could Terminal.Gui possibly use a more low-level keyboard IO hook like SharpHook or something? (or is that out of the scope of Terminal.Gui's purpose?)
I did try using SharpHook, but I couldn't seem to get it to do A-Z keys... only special keys like command, option, shift, &c. (maybe I should create a bug report for that, because I'm pretty sure it's suppose to support A-Z)

I suppose it's my fault for wanting to make a real-time game that runs in a terminal. :(

[BDisp]

Oh, ok. Well that sucks... and I don't suppose there'd be any way to "patch" ncurses to support a key release, otherwise someone would've already done it probably.

I think ncurses only support that on X11 and not on console.

Any ideas on how I could get/simulate a key released event then? Could Terminal.Gui possibly use a more low-level keyboard IO hook like SharpHook or something? (or is that out of the scope of Terminal.Gui's purpose?) I did try using SharpHook, but I couldn't seem to get it to do A-Z keys... only special keys like command, option, shift, &c. (maybe I should create a bug report for that, because I'm pretty sure it's suppose to support A-Z)

That's because all the keys aren't processed at once and while a key isn't released they don't save the old state of the key and that why it only processes the specials keys.

I suppose it's my fault for wanting to make a real-time game that runs in a terminal. :(

For the drivers that not supported it, while the key is always been pressed then is the key down. When they stopped echoing then is key up. May be a timer that check for a flag if some key was pressed or not.
For WindowsDriver if you set the listView focus to false, you can achieve what you want with this change:

public static class Program {
		private static Window SetupWindow ()
		{
			List<string> keyEvents = new ();
			ListView listView = new (keyEvents) {
				Height = Dim.Fill (),
				Width = Dim.Fill (),
				CanFocus = false
			};

			Window window = new ();
			window.Add (listView);

			window.KeyDown += (s, e) => {
				keyEvents.Add ($"KeyDown: {e.KeyEvent}");
				listView.SelectedItem = listView.Source.Count - 1;
				listView.SetNeedsDisplay ();
			};
			window.KeyUp += (s, e) => {
				keyEvents.Add ($"KeyUp: {e.KeyEvent}");
				listView.SelectedItem = listView.Source.Count - 1;
				listView.SetNeedsDisplay ();
			};
			window.KeyPress += (s, e) => {
				keyEvents.Add ($"KeyPress: {e.KeyEvent}");
				listView.SelectedItem = listView.Source.Count - 1;
				listView.SetNeedsDisplay ();
				e.Handled = true;
			};

			return window;
		}

    public static void Main(string[] args) {
        Application.Init();
        try {
            Window window = SetupWindow();
            Application.Run(window);
        } finally {
            Application.Shutdown();
        }
    }
}

[Albator11]

That's because all the keys aren't processed at once and while a key isn't released they don't save the old state of the key and that why it only processes the specials keys.

I guess I'm not quite following what you mean as to why only the special keys are processed... SharpHook has an example that I tried and doesn't work. (as far as I can tell anyway)
I'm planning on doing more testing to see if I can get something working, but life is starting to get busy, so I may not get it done soon.

For the drivers that not supported it, while the key is always been pressed then is the key down. When they stopped echoing then is key up. May be a timer that check for a flag if some key was pressed or not. For WindowsDriver if you set the listView focus to false, you can achieve what you want with this change:

I think I know what you're saying, but I don't see how that's possible in Terminal.Gui though. If I'm understanding correctly, you're talking about checking to see if a key is pressed every (for example) 100ms or something?
And why'd you mention the Windows driver? I thought it worked as expected with KeyUp events? (because I tried it on Mac, and it appeared to act the same as what I had...)

[BDisp]

I guess I'm not quite following what you mean as to why only the special keys are processed... SharpHook has an example that I tried and doesn't work. (as far as I can tell anyway)

I'll take a look on that example.

I'm planning on doing more testing to see if I can get something working, but life is starting to get busy, so I may not get it done soon.

All the help is welcome when you have time.

I think I know what you're saying, but I don't see how that's possible in Terminal.Gui though. If I'm understanding correctly, you're talking about checking to see if a key is pressed every (for example) 100ms or something?

Yes I already try that but it isn't very accurate. I have to think on something.

And why'd you mention the Windows driver? I thought it worked as expected with KeyUp events? (because I tried it on Mac, and it appeared to act the same as what I had...)

The KeyPress sometimes is not print because listview search an item that start with the key pressed and return true if found something, avoiding the window receiving the event. So, if the listview isn't focused the window will get all events. Try yo press some key that you already pressed before and the KeyPress isn't handle by the window if the listview is focused.

[Albator11]

Alright, I figured out what was wrong with SharpHook... apparently you need to run the app as sudo as I describe here.

[BDisp]

Alright, I figured out what was wrong with SharpHook... apparently you need to run the app as sudo as I describe here.

I didn't try it because I'm working on a workaround without introduce more third libraries.

[Albator11]

That would be awesome if you can get something working! Until then I think I'll probably be happy enough with using SharpHook. :)

I'm curious what your workaround might be though? And if you'd like any help with it, I wouldn't mind trying to help. (although I wouldn't call myself an expert by any means)

[BDisp]

I already having this working with NetDriver and CursesDriver but I need a little more time to be sure that all is working well.

[tig]

• Fixes #2926 - Refactor KeyEvent and KeyEventEventArgs to simplify #2927

The above PR should have addressed this. Can we confirm and close?

[Albator11]

I just pulled the latest v2_develop branch and tried the UICatalog > Mouse and Keyboard > Keys scenario, and KeyUp is still getting triggered the same time KeyDown is for me...
I also tried a slightly modified (due to API changes) version of my first example. It also triggers the first KeyDown and KeyUp at the same time, but then seems to stop triggering KeyDown completely after that.

[tig]

It is not possible (as far as we know at this time) to detect distinct KeyDown/KeyUp events on Linux/MacOS.

The CursesDriver, which is the default on Linux/Mac does this when a key is pressed/released:

			OnKeyDown (key);
			OnKeyUp (key);

Same is true of NetDriver, BTW.

OnKeyPress no longer exists in v2_develop.

It MAY be possible for terminal apps on Mac (or even Linux) to see keydown distinctly from keyup, but we have seen no evidence of that. If we find another TUI library that supports Linux/Mac that DOES, we'd be quite excited to dive in and figure out how to make it work.'

Until then, the correct resolution to this Issue is: Closing, By Design.

Sorry!

[Albator11]

Am I confused as to what BDisp meant by this then? It sounded like there was some workaround possibly...

I didn't try it because I'm working on a workaround without introduce more third libraries.

I already having this working with NetDriver and CursesDriver but I need a little more time to be sure that all is working well.

Would it be possible to take a more low-level approach like SharpHook or make it easy to integrate it? (or even just use SharpHook? although I get not wanting more 3rd-party dependencies &c.)
I suppose in most cases it's alright though, and if I really need it I can just roll out my own input system. 😕

[BDisp]

At the time, due to the profound changes to KeyDown, KeyPress and KeyUp, I think that KeyUp wasn't working but later I verified that this was gradually corrected by @tig. But really NetDriver and CursesDriver don't have any feature to recognize whether the key was released or not. Therefore, the KeyDown and KeyUp events (KeyPress no longer exists) are both handled when the key is pressed and even with the key pressed, the KeyUp event is fired after the KeyDown event is processed.
I don't know SharpHook and therefore I can't say if it would be an alternative. In any case, a system native to the OS itself would be more appropriate. In the case of NetDriver, it would be great if the System.Console could evolve to provide this functionality and in the case of CursesDriver, it would be great if the ncurses library or some native library provided it.

[tig]

SharpHook looks quite cool.

But I can't tell if it works in a .NET console app, or if it gives distinct key down/up events on MacOS in a console

@Albator11, could you test that?

[Albator11]

I have tried it in a "real-time" game I'm casually working on, so can confirm it works in .NET console apps!
And yes, it does give distinct key down/up events!

It does however IIRC, report key events even when the console window is not active, so that's something to be aware about.

Is there more you'd like me to test?

[BDisp]

It does however IIRC, report key events even when the console window is not active, so that's something to be aware about.

That's what I'm afraid of.

[Albator11]

That's what I'm afraid of.

I had the thought you could probably listen to only the key up events from SharpHook, and handle key down the way they currently are handled? I'm not sure, but yeah, it's for sure not ideal.

[BDisp]

I had the thought you could probably listen to only the key up events from SharpHook, and handle key down the way they currently are handled? I'm not sure, but yeah, it's for sure not ideal.

Regarding just managing the released keys, it should be enough, but what worries me is what you said about it being active after the window is not active and continuing to report important events. Well, it seems that it continues to operate even when we don't need it, appearing to be spying on what we do. I may be exaggerating but I like to be very careful.

[tznind]

Many use cases for Terminal.Gui involve running over SSH or in docker containers etc. So I think it is important that the main library handle the 'lowest common denominator' in terms of environments.

Maybe when Driver has been fully simplified it will be easier to subclass and do narrow scope activities e.g. swap out key handling for alternatives (e.g. SharpHook).

[Albator11]

Regarding just managing the released keys, it should be enough, but what worries me is what you said about it being active after the window is not active and continuing to report important events. Well, it seems that it continues to operate even when we don't need it, appearing to be spying on what we do. I may be exaggerating but I like to be very careful.

Ah, yes... that does make sense! I hadn't thought about that...