Skip to contentSkip to conversation

Register virtual GPIO keys for VNC

PATCH In review [email protected] 2 participants 6 messages Jul 29, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Register virtual GPIO keys for VNC

From: Menglong Wu <[email protected]>

Test platform
-------------
Kernel: 4.14.98
Rootfs: BusyBox 1.29
QT: 4.7
x11vnc: 0.9.16

Purpose
-------------

To emulate a full ANSI keyboard on embedded devices without physical keyboards.
The server responds to key events via x11vnc.
This resolves the limitation where embedded systems only register a few GPIO keys, making them unable to process full keyboard inputs.

documented in Documentation/devicetree/bindings/input/vnc-virtual-input.yaml

Scenario
-------------

- /dev/input/event1: Physical GPIO buttons

- /dev/input/event2: Physical touchscreen

Assume my-pcb.dts defines two GPIO-based hardware keys (A and B). When the VNC server x11vnc is started, a remote PC connects via a VNC client and attempts to send key inputs.
Since the Linux kernel only recognizes two keys, the UI on the embedded device only responds to ‘A’ and ‘B’.

Start x11vnc server:

```
    x11vnc -noipv6 -rawfb /dev/fb0 -forever -clip 640x480+0+0 \
      -pipeinput UINPUT:touch,tslib_cal=/etc/pointercal,\
      direct_key=/dev/input/event1,\
      direct_abs=/dev/input/event2
```

Monitor key events:

```
    hexdump /dev/input/event1
```

Original my-pcb.dts:

```
    gpio_keys {
        compatible = "gpio-keys";
        pinctrl-0 = <&pinctrl_gpio_keys>;

        key-a {
            label = "btn-key-a";
            gpios = <&gpio1 10 GPIO_ACTIVE_HIGH>;
            linux,code = <KEY_A>;
        };
        key-b {
            label = "btn-key-b";
            gpios = <&gpio1 11 GPIO_ACTIVE_HIGH>;
            linux,code = <KEY_A>;
        };
    };
```

Solution
-------------

Modify my-pcb.dts to support a virtual ANSI keyboard:

```
    #include "vnc-virtual-input.dtsi"

    // Originally, the DTS relied on physical GPIO keys
    gpio_keys {
        // Disable the original physical key node
        status = "disable";
    };

    // The virtual GPIO DTS node now integrates physical GPIO
    vnc_key {
        compatible = "gpio-keys";
        pinctrl-0 = <&pinctrl_gpio_keys>;
        
        key-a {
            label = "btn-key-a";
            gpios = <&gpio1 10 GPIO_ACTIVE_HIGH>;
            linux,code = <KEY_A>;
        };
        key-b {
            label = "btn-key-b";
            gpios = <&gpio1 11 GPIO_ACTIVE_HIGH>;
            linux,code = <KEY_A>;
        };
    }
```

Menglong Wu (2):
dt-bindings vnc virtual input
ARM: dts: Add virtual GPIO input for VNC keyboard

 .../bindings/input/vnc-virtual-input.txt      | 153 +++++
 .../bindings/input/vnc-virtual-input.yaml     |  86 +++
 MAINTAINERS                                   |   8 +
 arch/arm/boot/dts/vnc-virtual-input.dtsi      | 569 ++++++++++++++++++
 4 files changed, 816 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/input/vnc-virtual-input.txt
 create mode 100644 Documentation/devicetree/bindings/input/vnc-virtual-input.yaml
 create mode 100644 arch/arm/boot/dts/vnc-virtual-input.dtsi
Signature · 1 line
2.39.5
#1 lore raw

PATCH 1/2 dt-bindings vnc virtual input

From: Menglong Wu <[email protected]>

Add documentation for virtual GPIO keys to enable full ANSI keyboard emulation via VNC on embedded devices without physical keyboards.

- vnc-virtual-input.txt: Usage scenario and implementation details
- vnc-virtual-input.yaml: Formal DT binding specification
- MAINTAINERS: Add entry for VNC virtual input support

The solution registers virtual GPIOs (starting from 400) to avoid conflicts with physical GPIOs while supporting standard Linux input codes.

---
 .../bindings/input/vnc-virtual-input.txt      | 153 ++++++++++++++++++
 .../bindings/input/vnc-virtual-input.yaml     |  86 ++++++++++
 MAINTAINERS                                   |   7 +
 3 files changed, 246 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/input/vnc-virtual-input.txt
 create mode 100644 Documentation/devicetree/bindings/input/vnc-virtual-input.yaml
Documentation/devicetree/bindings/input/vnc-virtual-input.txt new file +153 -0
new file mode 100644
index 000000000000..77d12308c553
@@ -0,0 +1,153 @@
+Register virtual GPIO keys for VNC
+==========================
+
+# Purpose
+
+To emulate a full ANSI keyboard on embedded devices without physical
+keyboards.The server responds to key events via x11vnc.
+This resolves the limitation where embedded systems only register a few
+GPIO keys, making them unable to process full keyboard inputs.
+
+documented in Documentation/devicetree/bindings/input/vnc-virtual-input.yaml
+
+# Scenario
+
+- /dev/input/event1: Physical GPIO buttons
+
+- /dev/input/event2: Physical touchscreen
+
+Assume my-pcb.dts defines two GPIO-based hardware keys (‘A’ and ‘B’).
+When the VNC server x11vnc is started, a remote PC connects via a VNC
+client and attempts to send key inputs.
+Since the Linux kernel only recognizes two keys, the UI on the embedded
+device only responds to ‘A’ and ‘B’.
+
+Start x11vnc server:
+
+```
+    x11vnc -noipv6 -rawfb /dev/fb0 -forever -clip 640x480+0+0 \
+      -pipeinput UINPUT:touch,tslib_cal=/etc/pointercal,\
+      direct_key=/dev/input/event1,\
+      direct_abs=/dev/input/event2
+```
+
+Monitor key events:
+
+```
+    hexdump /dev/input/event1
+```
+
+Check registered virtual GPIOs:
+
+```
+    mount -t debugfs none /sys/kernel/debug/
+    cat /sys/kernel/debug/gpio
+
+    gpiochip5: GPIOs 400-503, parent: platform/gpio_mockup_hw, gpio-mockup-A:
+    gpio-400 (                    |vk_key_0            ) in  lo
+    gpio-401 (                    |vk_key_1            ) in  lo
+```
+
+Original my-pcb.dts:
+
+
+```
+    gpio_keys {
+        compatible = "gpio-keys";
+        pinctrl-0 = <&pinctrl_gpio_keys>;
+
+        key-a {
+            label = "btn-key-a";
+            gpios = <&gpio1 10 GPIO_ACTIVE_HIGH>;
+            linux,code = <KEY_A>;
+        };
+        key-b {
+            label = "btn-key-b";
+            gpios = <&gpio1 11 GPIO_ACTIVE_HIGH>;
+            linux,code = <KEY_A>;
+        };
+    };
+```
+
+
+# Solution
+
+Modify my-pcb.dts to support a virtual ANSI keyboard:
+
+```
+    #include "vnc-virtual-input.dtsi"
+
+    // Originally, the DTS relied on physical GPIO keys
+    gpio_keys {
+        // Disable the original physical key node
+        status = "disable";
+    };
+
+    // The virtual GPIO DTS node now integrates physical GPIO
+    vnc_key {
+        compatible = "gpio-keys";
+        pinctrl-0 = <&pinctrl_gpio_keys>;
+
+        key-a {
+            label = "btn-key-a";
+            gpios = <&gpio1 10 GPIO_ACTIVE_HIGH>;
+            linux,code = <KEY_A>;
+        };
+        key-b {
+            label = "btn-key-b";
+            gpios = <&gpio1 11 GPIO_ACTIVE_HIGH>;
+            linux,code = <KEY_A>;
+        };
+    }
+```
+
+# Explanation
+
+A virtual GPIO driver gpio_mockup_hw is registered, providing 104 GPIOs,
+starting from number 400.
+
+The starting number (400) avoids conflicts with physical GPIOs on
+commercialSoCs, which typically do not exceed 400. Most SoC vendors also
+start GPIO numbering from 0 in their drivers.
+
+The gpio_key_mockup node defines 100 ANSI standard keyboard keys.
+
+Key codes (KEY_xx) follow the definitions in
+include/uapi/linux/input-event-codes.h.
+
+
+
+ANSI Keyboard Layout (Symbolic Representation):
+
+```
+L FFFFFFFFFFFFFFF   MMM
+L NNNNNNNNNNNN  R   CCC   PPPP
+LL  AAAAAAAAA RRR   CCC   PPPP
+LLL  AAAAAAA RRRR         PPPP
+LLL   AAAAA RRRRR    C    PPPP
+LLLL  LLLLL RRRRR   CCC   PPPP
+```
+
+
+Key Layout Categories
+
+L: Left-side keys (ESC, TAB, CAPS LOCK, etc.)
+F: Function keys (F1–F12)
+N: Number row (1–0)
+A: Alphabet keys (A–Z)
+R: Right-side keys (BACKSPACE, [, ], , ENTER, etc.)
+M: Miscellaneous (Print Screen, Scroll Lock, Pause/Break)
+C: Cursor controls (Insert, Home, Arrow keys, etc.)
+P: Numpad keys (0–9, -, +, etc.)
+
+
+Example from vnc-virtual-input.dtsi:
+
+```
+    /* Left of a-z */
+    vk_78 {
+        label = "vk_key_78";
+        gpios = <&gpio_mockup_hw 78 GPIO_ACTIVE_HIGH>;
+        linux,code = <KEY_ESC>;
+    };
+```
Documentation/devicetree/bindings/input/vnc-virtual-input.yaml new file +86 -0

86 lines not on this page · raw message

MAINTAINERS +7 -0
index c0b444e5fd5a..658ef6619542 100644
@@ -27476,6 +27476,13 @@ S:	Maintained
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound.git
 F:	sound/pci/hda/patch_senarytech.c
 
+VNC VIRTUAL INPUT SUPPORT
+M:	Menglong Wu <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	Documentation/devicetree/bindings/input/vnc-virtual-input.txt
+F:	Documentation/devicetree/bindings/input/vnc-virtual-input.yaml
+
 THE REST
 M:	Linus Torvalds <[email protected]>
 L:	[email protected]
Signature · 1 line
2.39.5
#2 lore raw

Re: [PATCH 1/2] dt-bindings vnc virtual input

Quoted 1 line · On 29/07/2025 08:43, [email protected] wrote:

From: Menglong Wu <[email protected]>

Please use subject prefixes matching the subsystem. You can get them for example with git log --oneline -- DIRECTORY_OR_FILE on the directory your patch is touching. For bindings, the preferred subjects are explained here:
https://www.kernel.org/doc/html/latest/devicetree/bindings/submitting-patches.html#i-for-patch-submitters

Quoted 11 lines

Add documentation for virtual GPIO keys to enable full ANSI keyboard emulation via VNC on embedded devices without physical keyboards.

- vnc-virtual-input.txt: Usage scenario and implementation details
- vnc-virtual-input.yaml: Formal DT binding specification
- MAINTAINERS: Add entry for VNC virtual input support

The solution registers virtual GPIOs (starting from 400) to avoid conflicts with physical GPIOs while supporting standard Linux input codes.

Quoted 7 lines
---
 .../bindings/input/vnc-virtual-input.txt      | 153 ++++++++++++++++++
 .../bindings/input/vnc-virtual-input.yaml     |  86 ++++++++++
 MAINTAINERS                                   |   7 +
 3 files changed, 246 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/input/vnc-virtual-input.txt

We do not take txt bindings anymore, sorry.

Quoted 9 lines
 create mode 100644 Documentation/devicetree/bindings/input/vnc-virtual-input.yaml
Documentation/devicetree/bindings/input/vnc-virtual-input.txt new file +1 -0
new file mode 100644
index 000000000000..77d12308c553
@@ -0,0 +1,153 @@
+Register virtual GPIO keys for VNC

Documentation/devicetree/bindings/input/vnc-virtual-input.txt line 11

Quoted 10 lines
Documentation/devicetree/bindings/input/vnc-virtual-input.txt new file +10 -0
+==========================
+
+# Purpose
+
+To emulate a full ANSI keyboard on embedded devices without physical
+keyboards.The server responds to key events via x11vnc.
+This resolves the limitation where embedded systems only register a few
+GPIO keys, making them unable to process full keyboard inputs.
+
+documented in Documentation/devicetree/bindings/input/vnc-virtual-input.yaml

I don't understand the placement of this file in bindings. What are you writing bindings for here?

Quoted 10 lines
Documentation/devicetree/bindings/input/vnc-virtual-input.yaml new file +4 -0
new file mode 100644
index 000000000000..8b5414cb2bea
@@ -0,0 +1,86 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/input/a.yaml#

Documentation/devicetree/bindings/input/vnc-virtual-input.yaml line 14

Quoted 10 lines
Documentation/devicetree/bindings/input/vnc-virtual-input.yaml new file +10 -0
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: VNC Virtual GPIO Keys
+
+maintainers:
+  - Menglong Wu <[email protected]>
+
+description: |
+  This binding describes a virtual GPIO key input node for VNC remote
+  input.

Describe the hardware, not the binding...

Documentation/devicetree/bindings/input/vnc-virtual-input.yaml line 24

Quoted 10 lines
Documentation/devicetree/bindings/input/vnc-virtual-input.yaml new file +10 -0
+  It is based on the standard gpio-keys binding.
+
+select: false
+
+allOf:
+  - $ref: gpio-keys.yaml#
+
+properties:
+  compatible:
+    const: gpio-keys

So this is all completely redundant and not needed. Drop the bindings.

Documentation/devicetree/bindings/input/vnc-virtual-input.yaml line 29

Quoted 5 lines
Documentation/devicetree/bindings/input/vnc-virtual-input.yaml new file +5 -0
+
+  status:
+    enum:
+      - okay
+      - disabled

Do you see any bindings with this?

Srsly, it's second bindings today. I think you send some AI slops to us.

Please confirm:

Did you use any AI tool to generate this file?

Best regards,
Krzysztof

#3 lore raw

PATCH 2/2 ARM: dts: Add virtual GPIO input for VNC keyboard

In reply to [email protected]

From: Menglong Wu <[email protected]>

Implements a complete ANSI keyboard solution for VNC remote input on ARM devices by:

1. Adding a device tree include file (vnc-virtual-input.dtsi) that:
- Defines a gpio-mockup controller with 104 virtual GPIOs (startingfrom 400)
- Maps all standard keyboard keys
(F1-F12, alphanumeric, modifiers, etc.)
- Follows Linux input event codes for key mappings

2. Updating MAINTAINERS to track the new dtsi file

The virtual GPIO approach allows embedded systems without physical keyboards to receive full keyboard input via VNC while avoiding conflicts with realGPIO numbering.

---
 MAINTAINERS                              |   1 +
 arch/arm/boot/dts/vnc-virtual-input.dtsi | 569 +++++++++++++++++++++++
 2 files changed, 570 insertions(+)
 create mode 100644 arch/arm/boot/dts/vnc-virtual-input.dtsi
MAINTAINERS +1 -0
index 658ef6619542..da69f516388c 100644
@@ -27482,6 +27482,7 @@ L:	[email protected]
 S:	Maintained
 F:	Documentation/devicetree/bindings/input/vnc-virtual-input.txt
 F:	Documentation/devicetree/bindings/input/vnc-virtual-input.yaml
+F:	arch/arm/boot/dts/vnc-virtual-input.da9052_tsi
 
 THE REST
 M:	Linus Torvalds <[email protected]>
arch/arm/boot/dts/vnc-virtual-input.dtsi new file +569 -0

569 lines not on this page · raw message

Signature · 1 line
2.39.5
#4 lore raw

Re: [PATCH 2/2] ARM: dts: Add virtual GPIO input for VNC keyboard

Quoted 14 lines · On 29/07/2025 08:43, [email protected] wrote:

From: Menglong Wu <[email protected]>

Implements a complete ANSI keyboard solution for VNC remote input on ARM devices by:

1. Adding a device tree include file (vnc-virtual-input.dtsi) that:
- Defines a gpio-mockup controller with 104 virtual GPIOs (startingfrom 400)
- Maps all standard keyboard keys
(F1-F12, alphanumeric, modifiers, etc.)
- Follows Linux input event codes for key mappings

2. Updating MAINTAINERS to track the new dtsi file

Quoted 8 lines

The virtual GPIO approach allows embedded systems without physical keyboards to receive full keyboard input via VNC while avoiding conflicts with realGPIO numbering.

---
 MAINTAINERS                              |   1 +
 arch/arm/boot/dts/vnc-virtual-input.dtsi | 569 +++++++++++++++++++++++

This is a dead, no-op, impossible to test code.

Also, does not follow any upstream coding style.

Best regards,
Krzysztof