Skip to contentSkip to conversation

cocoa: keyboard quality of life

PATCH In review Akihiko Odaki 2 participants 5 messages Mar 6, 2022

lore
Share & tools
lore
#0 lore raw

v2 cover cocoa: keyboard quality of life

The patch series was originally posted by Gustavo Noronha Silva, and rebased to commit 2acf5e1d0e0f15be1b0ad85cf05b3a6e6307680c by Akihiko Odaki.

This series adds two new options to the cocoa display:

- full-grab causes it to use a global tap to steal system combos away from Mac OS X, so they can be handled by the VM

- swap-opt-cmd does what it says on the tin; while that is something you can do at the Mac OS X level or even supported by some keyboards, it is much more convenient to have qemu put Meta/Super and Alt where they belong if you are running a non-Mac VM

Both are off by default. For full-grab in particular, it is off also because unfortunately it needs accessibility permissions for input grabbing, so it requires more deliberate action by the user anyway.

v2:
- Removed extra cast operator (BALATON Zoltan)
- Renamed swap-option-command to swap-opt-cmd (BALATON Zoltan)
- Simplified swap-opt-cmd flag setting
- Updated qapi/ui.json for 7.0

Gustavo Noronha Silva (2):
ui/cocoa: capture all keys and combos when mouse is grabbed ui/cocoa: add option to swap Option and Command

 qapi/ui.json    |  21 +++++++++
 qemu-options.hx |   3 ++
 ui/cocoa.m      | 111 ++++++++++++++++++++++++++++++++++++++++++++----
 3 files changed, 126 insertions(+), 9 deletions(-)
Signature · 1 line
2.32.0 (Apple Git-132)
#1 lore raw

PATCH v2 1/2 ui/cocoa: capture all keys and combos when mouse is grabbed

From: Gustavo Noronha Silva <[email protected]>

Applications such as Gnome may use Alt-Tab and Super-Tab for different purposes, some use Ctrl-arrows so we want to allow qemu to handle everything when it captures the mouse/keyboard.

However, Mac OS handles some combos like Command-Tab and Ctrl-arrows at an earlier part of the event handling chain, not letting qemu see it.

We add a global Event Tap that allows qemu to see all events when the mouse is grabbed. Note that this requires additional permissions.

See:

https://developer.apple.com/documentation/coregraphics/1454426-cgeventtapcreate?language=objc#discussion
https://support.apple.com/en-in/guide/mac-help/mh32356/mac

Message-Id: <[email protected]>

---
 qapi/ui.json    | 16 ++++++++++++
 qemu-options.hx |  3 +++
 ui/cocoa.m      | 65 ++++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 83 insertions(+), 1 deletion(-)
qapi/ui.json +16 -0
index 4a13f883a30..ff0a04da792 100644
@@ -1260,6 +1260,21 @@
 { 'struct'  : 'DisplayCurses',
   'data'    : { '*charset'       : 'str' } }
 
+##
+# @DisplayCocoa:
+#
+# Cocoa display options.
+#
+# @full-grab: Capture all key presses, including system combos. This
+#             requires accessibility permissions, since it performs
+#             a global grab on key events. (default: off)
+#             See https://support.apple.com/en-in/guide/mac-help/mh32356/mac
+#
+# Since: 7.0
+##
+{ 'struct'  : 'DisplayCocoa',
+  'data'    : { '*full-grab'     : 'bool' } }
+
 ##
 # @DisplayType:
 #
@@ -1338,6 +1353,7 @@
   'discriminator' : 'type',
   'data'    : {
       'gtk': { 'type': 'DisplayGTK', 'if': 'CONFIG_GTK' },
+      'cocoa': { 'type': 'DisplayCocoa', 'if': 'CONFIG_COCOA' },
       'curses': { 'type': 'DisplayCurses', 'if': 'CONFIG_CURSES' },
       'egl-headless': { 'type': 'DisplayEGLHeadless',
                         'if': { 'all': ['CONFIG_OPENGL', 'CONFIG_GBM'] } },
qemu-options.hx +3 -0
index 094a6c1d7c2..4df9ccc3446 100644
@@ -1916,6 +1916,9 @@ DEF("display", HAS_ARG, QEMU_OPTION_display,
 #if defined(CONFIG_CURSES)
     "-display curses[,charset=<encoding>]\n"
 #endif
+#if defined(CONFIG_COCOA)
+    "-display cocoa[,full_grab=on|off]\n"
+#endif
 #if defined(CONFIG_OPENGL)
     "-display egl-headless[,rendernode=<file>]\n"
 #endif
ui/cocoa.m +64 -1

110 lines not on this page · raw message

Signature · 1 line
2.32.0 (Apple Git-132)
#2 lore raw

Re: [PATCH v2 1/2] ui/cocoa: capture all keys and combos when mouse is grabbed

Quoted 94 lines · Akihiko Odaki

From: Gustavo Noronha Silva <[email protected]>

Applications such as Gnome may use Alt-Tab and Super-Tab for different purposes, some use Ctrl-arrows so we want to allow qemu to handle everything when it captures the mouse/keyboard.

However, Mac OS handles some combos like Command-Tab and Ctrl-arrows at an earlier part of the event handling chain, not letting qemu see it.

We add a global Event Tap that allows qemu to see all events when the mouse is grabbed. Note that this requires additional permissions.

See:

https://developer.apple.com/documentation/coregraphics/1454426-cgeventtapcreate?language=objc#discussion
https://support.apple.com/en-in/guide/mac-help/mh32356/mac

Message-Id: <[email protected]>

---

qapi/ui.json    | 16 ++++++++++++
qemu-options.hx |  3 +++
ui/cocoa.m      | 65 ++++++++++++++++++++++++++++++++++++++++++++++++-
3 files changed, 83 insertions(+), 1 deletion(-)
qapi/ui.json +16 -0
index 4a13f883a30..ff0a04da792 100644
@@ -1260,6 +1260,21 @@
 { 'struct'  : 'DisplayCurses',
  'data'    : { '*charset'       : 'str' } }
 
+##
+# @DisplayCocoa:
+#
+# Cocoa display options.
+#
+# @full-grab: Capture all key presses, including system combos. This
+#             requires accessibility permissions, since it performs
+#             a global grab on key events. (default: off)
+#             See https://support.apple.com/en-in/guide/mac-help/mh32356/mac
+#
+# Since: 7.0
+##
+{ 'struct'  : 'DisplayCocoa',
+  'data'    : { '*full-grab'     : 'bool' } }
+
 ##
 # @DisplayType:
 #
@@ -1338,6 +1353,7 @@
  'discriminator' : 'type',
  'data'    : {
      'gtk': { 'type': 'DisplayGTK', 'if': 'CONFIG_GTK' },
+      'cocoa': { 'type': 'DisplayCocoa', 'if': 'CONFIG_COCOA' },
      'curses': { 'type': 'DisplayCurses', 'if': 'CONFIG_CURSES' },
      'egl-headless': { 'type': 'DisplayEGLHeadless',
                        'if': { 'all': ['CONFIG_OPENGL', 'CONFIG_GBM'] } },
qemu-options.hx +3 -0
index 094a6c1d7c2..4df9ccc3446 100644
@@ -1916,6 +1916,9 @@ DEF("display", HAS_ARG, QEMU_OPTION_display,
 #if defined(CONFIG_CURSES)
    "-display curses[,charset=<encoding>]\n"
 #endif
+#if defined(CONFIG_COCOA)
+    "-display cocoa[,full_grab=on|off]\n"
+#endif
 #if defined(CONFIG_OPENGL)
    "-display egl-headless[,rendernode=<file>]\n"
 #endif
ui/cocoa.m +2 -7
index 8ab9ab5e84d..bfd602a96b9 100644
@@ -308,11 +308,13 @@ @interface QemuCocoaView : NSView
    BOOL isMouseGrabbed;
    BOOL isFullscreen;
    BOOL isAbsoluteEnabled;
+    CFMachPortRef eventsTap;
 }
- (void) switchSurface:(pixman_image_t *)image;
- (void) grabMouse;
- (void) ungrabMouse;
- (void) toggleFullScreen:(id)sender;
+- (void) setFullGrab:(id)sender;
- (void) handleMonitorInput:(NSEvent *)event;
- (bool) handleEvent:(NSEvent *)event;
- (bool) handleEventLocked:(NSEvent *)event;

ui/cocoa.m - (void) raiseAllKeys; · line 343

Quoted 7 lines
ui/cocoa.m +4 -0
@@ -335,6 +337,19 @@ - (void) raiseAllKeys;
 
 QemuCocoaView *cocoaView;
 
+static CGEventRef handleTapEvent(CGEventTapProxy proxy, CGEventType type, CGEventRef cgEvent, void *userInfo)
+{
+    QemuCocoaView *cocoaView = userInfo;
+    NSEvent* event = [NSEvent eventWithCGEvent:cgEvent];

Sorry, I've missed this before but the space in NSEvent* event is also on the wrong side of the * it should be NSEvent *event. Does checkpatch miss these or it can't handle Objective-C? Another one below...

Quoted 76 lines
ui/cocoa.m +9 -0
+    if ([cocoaView isMouseGrabbed] && [cocoaView handleEvent:event]) {
+        COCOA_DEBUG("Global events tap: qemu handled the event, capturing!\n");
+        return NULL;
+    }
+    COCOA_DEBUG("Global events tap: qemu did not handle the event, letting it through...\n");
+
+    return cgEvent;
+}
+

@implementation QemuCocoaView
- (id)initWithFrame:(NSRect)frameRect
{

ui/cocoa.m +42 -2
@@ -360,6 +375,11 @@ - (void) dealloc
    }
 
    qkbd_state_free(kbd);
+
+    if (eventsTap) {
+        CFRelease(eventsTap);
+    }
+
    [super dealloc];
 }
 
@@ -654,6 +674,36 @@ - (void) toggleFullScreen:(id)sender
    }
 }
 
+- (void) setFullGrab:(id)sender
+{
+    COCOA_DEBUG("QemuCocoaView: setFullGrab\n");
+
+    CGEventMask mask = CGEventMaskBit(kCGEventKeyDown) | CGEventMaskBit(kCGEventKeyUp) | CGEventMaskBit(kCGEventFlagsChanged);
+    eventsTap = CGEventTapCreate(kCGHIDEventTap, kCGHeadInsertEventTap, kCGEventTapOptionDefault,
+                                 mask, handleTapEvent, self);
+    if (!eventsTap) {
+        warn_report("Could not create event tap, system key combos will not be captured.\n");
+        return;
+    } else {
+        COCOA_DEBUG("Global events tap created! Will capture system key combos.\n");
+    }
+
+    CFRunLoopRef runLoop = CFRunLoopGetCurrent();
+    if (!runLoop) {
+        warn_report("Could not obtain current CF RunLoop, system key combos will not be captured.\n");
+        return;
+    }
+
+    CFRunLoopSourceRef tapEventsSrc = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, eventsTap, 0);
+    if (!tapEventsSrc ) {
+        warn_report("Could not obtain current CF RunLoop, system key combos will not be captured.\n");
+        return;
+    }
+
+    CFRunLoopAddSource(runLoop, tapEventsSrc, kCFRunLoopDefaultMode);
+    CFRelease(tapEventsSrc);
+}
+
- (void) toggleKey: (int)keycode {
    qkbd_state_key_event(kbd, keycode, !qkbd_state_key_get(kbd, keycode));
 }
@@ -1281,6 +1331,13 @@ - (void)toggleFullScreen:(id)sender
    [cocoaView toggleFullScreen:sender];
 }
 
+- (void) setFullGrab:(id)sender
+{
+    COCOA_DEBUG("QemuCocoaAppController: setFullGrab\n");
+
+    [cocoaView setFullGrab:sender];
+}
+
 /* Tries to find then open the specified filename */
- (void) openDocumentation: (NSString *) filename
 {

ui/cocoa.m static void cocoa_display_init(DisplayState *ds, DisplayOptions *opts) · line 2117

Quoted 4 lines
ui/cocoa.m +1 -0
@@ -2057,11 +2114,17 @@ static void cocoa_display_init(DisplayState *ds, DisplayOptions *opts)
    qemu_sem_wait(&app_started_sem);
    COCOA_DEBUG("cocoa_display_init: app start completed\n");
 
+    QemuCocoaAppController* controller = (QemuCocoaAppController*)[[NSApplication sharedApplication] delegate];

Both in the type declaration and in the cast on the right hand side space is missing between type and *.

Otherwise I'd give an R-b but it does not worth much as I've never used CGEventTap and have not checked the docs so maybe someone else who actually knows this could better review it.

Regards,
BALATON Zoltan

Quoted 15 lines
ui/cocoa.m +6 -1
    /* if fullscreen mode is to be used */
    if (opts->has_full_screen && opts->full_screen) {
        dispatch_async(dispatch_get_main_queue(), ^{
            [NSApp activateIgnoringOtherApps: YES];
-            [(QemuCocoaAppController *)[[NSApplication sharedApplication] delegate] toggleFullScreen: nil];
+            [controller toggleFullScreen: nil];
+        });
+    }
+    if (opts->u.cocoa.has_full_grab && opts->u.cocoa.full_grab) {
+        dispatch_async(dispatch_get_main_queue(), ^{
+            [controller setFullGrab: nil];
        });
    }
    if (opts->has_show_cursor && opts->show_cursor) {
#3 lore raw

PATCH v2 2/2 ui/cocoa: add option to swap Option and Command

In reply to Akihiko Odaki

From: Gustavo Noronha Silva <[email protected]>

On Mac OS X the Option key maps to Alt and Command to Super/Meta. This change swaps them around so that Alt is the key closer to the space bar and Meta/Super is between Control and Alt, like on non-Mac keyboards.

It is a cocoa display option, disabled by default.

Message-Id: <[email protected]>

---
 qapi/ui.json    |  7 ++++++-
 qemu-options.hx |  2 +-
 ui/cocoa.m      | 46 ++++++++++++++++++++++++++++++++++++++--------
 3 files changed, 45 insertions(+), 10 deletions(-)
qapi/ui.json +6 -1
index ff0a04da792..e28339887e0 100644
@@ -1270,10 +1270,15 @@
 #             a global grab on key events. (default: off)
 #             See https://support.apple.com/en-in/guide/mac-help/mh32356/mac
 #
+# @swap-opt-cmd: Swap the Option and Command keys so that their key codes match
+#                their position on non-Mac keyboards and you can use Meta/Super
+#                and Alt where you expect them. (default: off)
+#
 # Since: 7.0
 ##
 { 'struct'  : 'DisplayCocoa',
-  'data'    : { '*full-grab'     : 'bool' } }
+  'data'    : { '*full-grab'           : 'bool',
+                '*swap-opt-cmd' : 'bool' } }
 
 ##
 # @DisplayType:
qemu-options.hx +1 -1
index 4df9ccc3446..b57cbcb2c9e 100644
@@ -1917,7 +1917,7 @@ DEF("display", HAS_ARG, QEMU_OPTION_display,
     "-display curses[,charset=<encoding>]\n"
 #endif
 #if defined(CONFIG_COCOA)
-    "-display cocoa[,full_grab=on|off]\n"
+    "-display cocoa[,full-grab=on|off][,swap-opt-cmd=on|off]\n"
 #endif
 #if defined(CONFIG_OPENGL)
     "-display egl-headless[,rendernode=<file>]\n"
ui/cocoa.m +38 -8

89 lines not on this page · raw message

Signature · 1 line
2.32.0 (Apple Git-132)
#4 lore raw

Re: [PATCH v2 2/2] ui/cocoa: add option to swap Option and Command

Quoted 25 lines · Akihiko Odaki

From: Gustavo Noronha Silva <[email protected]>

On Mac OS X the Option key maps to Alt and Command to Super/Meta. This change swaps them around so that Alt is the key closer to the space bar and Meta/Super is between Control and Alt, like on non-Mac keyboards.

It is a cocoa display option, disabled by default.

Message-Id: <[email protected]>

---

qapi/ui.json    |  7 ++++++-
qemu-options.hx |  2 +-
ui/cocoa.m      | 46 ++++++++++++++++++++++++++++++++++++++--------
3 files changed, 45 insertions(+), 10 deletions(-)
qapi/ui.json +0 -0
index ff0a04da792..e28339887e0 100644
@@ -1270,10 +1270,15 @@
 #             a global grab on key events. (default: off)
 #             See https://support.apple.com/en-in/guide/mac-help/mh32356/mac

qapi/ui.json line 1280

Quoted 10 lines
qapi/ui.json +5 -1
 #
+# @swap-opt-cmd: Swap the Option and Command keys so that their key codes match
+#                their position on non-Mac keyboards and you can use Meta/Super
+#                and Alt where you expect them. (default: off)
+#
 # Since: 7.0
 ##
 { 'struct'  : 'DisplayCocoa',
-  'data'    : { '*full-grab'     : 'bool' } }
+  'data'    : { '*full-grab'           : 'bool',

Indentation off now that next line became shorter.

Regards,
BALATON Zoltan

Quoted 116 lines
qapi/ui.json +1 -0
+                '*swap-opt-cmd' : 'bool' } }
 
 ##
 # @DisplayType:
qemu-options.hx +1 -1
index 4df9ccc3446..b57cbcb2c9e 100644
@@ -1917,7 +1917,7 @@ DEF("display", HAS_ARG, QEMU_OPTION_display,
    "-display curses[,charset=<encoding>]\n"
 #endif
 #if defined(CONFIG_COCOA)
-    "-display cocoa[,full_grab=on|off]\n"
+    "-display cocoa[,full-grab=on|off][,swap-opt-cmd=on|off]\n"
 #endif
 #if defined(CONFIG_OPENGL)
    "-display egl-headless[,rendernode=<file>]\n"
ui/cocoa.m +38 -8
index bfd602a96b9..ab9de404687 100644
@@ -95,6 +95,7 @@ static void cocoa_switch(DisplayChangeListener *dcl,
 };
 static int last_buttons;
 static int cursor_hide = 1;
+static bool swap_opt_cmd;
 
 static int gArgc;
 static char **gArgv;
@@ -853,12 +854,22 @@ - (bool) handleEventLocked:(NSEvent *)event
        qkbd_state_key_event(kbd, Q_KEY_CODE_CTRL_R, false);
    }
    if (!(modifiers & NSEventModifierFlagOption)) {
-        qkbd_state_key_event(kbd, Q_KEY_CODE_ALT, false);
-        qkbd_state_key_event(kbd, Q_KEY_CODE_ALT_R, false);
+        if (swap_opt_cmd) {
+            qkbd_state_key_event(kbd, Q_KEY_CODE_META_L, false);
+            qkbd_state_key_event(kbd, Q_KEY_CODE_META_R, false);
+        } else {
+            qkbd_state_key_event(kbd, Q_KEY_CODE_ALT, false);
+            qkbd_state_key_event(kbd, Q_KEY_CODE_ALT_R, false);
+        }
    }
    if (!(modifiers & NSEventModifierFlagCommand)) {
-        qkbd_state_key_event(kbd, Q_KEY_CODE_META_L, false);
-        qkbd_state_key_event(kbd, Q_KEY_CODE_META_R, false);
+        if (swap_opt_cmd) {
+            qkbd_state_key_event(kbd, Q_KEY_CODE_ALT, false);
+            qkbd_state_key_event(kbd, Q_KEY_CODE_ALT_R, false);
+        } else {
+            qkbd_state_key_event(kbd, Q_KEY_CODE_META_L, false);
+            qkbd_state_key_event(kbd, Q_KEY_CODE_META_R, false);
+        }
    }
 
    switch ([event type]) {
@@ -890,13 +901,21 @@ - (bool) handleEventLocked:(NSEvent *)event
 
                case kVK_Option:
                    if (!!(modifiers & NSEventModifierFlagOption)) {
-                        [self toggleKey:Q_KEY_CODE_ALT];
+                        if (swap_opt_cmd) {
+                            [self toggleKey:Q_KEY_CODE_META_L];
+                        } else {
+                            [self toggleKey:Q_KEY_CODE_ALT];
+                        }
                    }
                    break;
 
                case kVK_RightOption:
                    if (!!(modifiers & NSEventModifierFlagOption)) {
-                        [self toggleKey:Q_KEY_CODE_ALT_R];
+                        if (swap_opt_cmd) {
+                            [self toggleKey:Q_KEY_CODE_META_R];
+                        } else {
+                            [self toggleKey:Q_KEY_CODE_ALT_R];
+                        }
                    }
                    break;
 
@@ -904,14 +923,22 @@ - (bool) handleEventLocked:(NSEvent *)event
                case kVK_Command:
                    if (isMouseGrabbed &&
                        !!(modifiers & NSEventModifierFlagCommand)) {
-                        [self toggleKey:Q_KEY_CODE_META_L];
+                        if (swap_opt_cmd) {
+                            [self toggleKey:Q_KEY_CODE_ALT];
+                        } else {
+                            [self toggleKey:Q_KEY_CODE_META_L];
+                        }
                    }
                    break;
 
                case kVK_RightCommand:
                    if (isMouseGrabbed &&
                        !!(modifiers & NSEventModifierFlagCommand)) {
-                        [self toggleKey:Q_KEY_CODE_META_R];
+                        if (swap_opt_cmd) {
+                            [self toggleKey:Q_KEY_CODE_ALT_R];
+                        } else {
+                            [self toggleKey:Q_KEY_CODE_META_R];
+                        }
                    }
                    break;
            }
@@ -2130,6 +2157,9 @@ static void cocoa_display_init(DisplayState *ds, DisplayOptions *opts)
    if (opts->has_show_cursor && opts->show_cursor) {
        cursor_hide = 0;
    }
+    if (opts->u.cocoa.has_swap_opt_cmd) {
+        swap_opt_cmd = opts->u.cocoa.swap_opt_cmd;
+    }
 
    // register vga output callbacks
    register_displaychangelistener(&dcl);