input: Add a detailed multi-touch finger data report protocol (rev2)
Subsystems (2)
PATCH 2/2 input: Add a detailed multi-touch finger data report protocol (rev2)
In order to utilize the full power of the new multi-touch devices, a way to report detailed finger data to user space is needed. This patch adds a multi-touch (MT) protocol which allows drivers to report details for an arbitrary number of fingers.
The driver sends a SYN_MT_REPORT event via the input_mt_sync() function when a complete finger has been reported.
In order to stay compatible with existing applications, the data reported in a finger packet must not be recognized as single-touch events. In addition, all finger data must bypass input filtering, since subsequent events of the same type refer to different fingers.
A set of ABS_MT events with the desired properties are defined. The events are divided into categories, to allow for partial implementation. The minimum set consists of ABS_MT_TOUCH_MAJOR, ABS_MT_POSITION_X and ABS_MT_POSITION_Y, which allows for multiple fingers to be tracked. If the device supports it, the ABS_MT_WIDTH_MAJOR may be used to provide the size of the approaching finger. Anisotropy and direction may be specified with ABS_MT_TOUCH_MINOR, ABS_MT_WIDTH_MINOR and ABS_MT_ORIENTATION. Devices with more granular information may specify general shapes as blobs, i.e., as a sequence of rectangular shapes grouped together by a ABS_MT_BLOB_ID. Finally, the ABS_MT_TOOL_TYPE may be used to specify whether the touching tool is a finger or a pen.
--- drivers/input/input.c | 13 +++++++++++++ include/linux/input.h | 27 +++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 0 deletions(-)
drivers/input/input.c +13 -0
@@ -33,6 +33,15 @@ MODULE_LICENSE("GPL");* EV_ABS events which should not be cached are listed here. */ static __initdata unsigned int input_abs_bypass_init_data[] = { +ABS_MT_TOUCH_MAJOR , +ABS_MT_TOUCH_MINOR , +ABS_MT_WIDTH_MAJOR , +ABS_MT_WIDTH_MINOR , +ABS_MT_ORIENTATION , +ABS_MT_POSITION_X , +ABS_MT_POSITION_Y , +ABS_MT_TOOL_TYPE , +ABS_MT_BLOB_ID ,0 };
@@ -165,6 +174,10 @@ static void input_handle_event(struct input_dev *dev, disposition =INPUT_PASS_TO_HANDLERS ; }break ; +case SYN_MT_REPORT : + dev->sync =0 ; + disposition =INPUT_PASS_TO_HANDLERS ; +break ; }break ;
include/linux/input.h +27 -0
@@ -106,6 +106,7 @@ struct input_absinfo {#define SYN_REPORT 0#define SYN_CONFIG 1 +#define SYN_MT_REPORT 2/* * Keys and buttons
@@ -646,6 +647,19 @@ struct input_absinfo {#define ABS_TOOL_WIDTH 0x1c#define ABS_VOLUME 0x20#define ABS_MISC 0x28 + +#define ABS_MT_TOUCH 0x30/* Diameter of touching circle */ +#define ABS_MT_TOUCH_MAJOR 0x30/* Major axis of touching ellipse */ +#define ABS_MT_TOUCH_MINOR 0x31/* Minor axis of touching ellipse */ +#define ABS_MT_WIDTH 0x32/* Diameter of approaching circle */ +#define ABS_MT_WIDTH_MAJOR 0x32/* Major axis of approaching ellipse */ +#define ABS_MT_WIDTH_MINOR 0x33/* Minor axis of approaching ellipse */ +#define ABS_MT_ORIENTATION 0x34/* Ellipse orientation */ +#define ABS_MT_POSITION_X 0x35/* Center X ellipse position */ +#define ABS_MT_POSITION_Y 0x36/* Center Y ellipse position */ +#define ABS_MT_TOOL_TYPE 0x37/* Type of touching device */ +#define ABS_MT_BLOB_ID 0x38/* Group a set of packets as a blob */ +#define ABS_MAX 0x3f#define ABS_CNT (ABS_MAX+1)
@@ -742,6 +756,14 @@ struct input_absinfo {#define BUS_ATARI 0x1B/* +* MT_TOOL types +*/ +#define MT_TOOL_FINGER 0 +#define MT_TOOL_PEN 1 +#define MT_TOOL_MAX 9 +#define MT_TOOL_CNT (MT_TOOL_MAX + 1) + +/* * Values describing the status of a force-feedback effect */ #define FF_STATUS_STOPPED 0x00
@@ -1310,6 +1332,11 @@ static inline void input_sync(struct input_dev *dev)input_event (dev,EV_SYN ,SYN_REPORT ,0 ); } +static inline void input_mt_sync (struct input_dev *dev) +{ +input_event (dev,EV_SYN ,SYN_MT_REPORT ,0 ); +} +void input_set_capability (struct input_dev *dev,unsigned int type,unsigned int code);static inline void input_set_abs_params (struct input_dev *dev,int axis,int min,int max,int fuzz,int flat)
Quoted 13 lines · Henrik Rydberg
In order to utilize the full power of the new multi-touch devices, a way to report detailed finger data to user space is needed. This patch adds a multi-touch (MT) protocol which allows drivers to report details for an arbitrary number of fingers.
The driver sends a
SYN_MT_REPORTevent via theinput_mt_sync()function when a complete finger has been reported.In order to stay compatible with existing applications, the data reported in a finger packet must not be recognized as single-touch events. In addition, all finger data must bypass input filtering, since subsequent events of the same type refer to different fingers.
Quoted 10 lines
A set of
ABS_MTevents with the desired properties are defined. The events are divided into categories, to allow for partial implementation. The minimum set consists ofABS_MT_TOUCH_MAJOR,ABS_MT_POSITION_XandABS_MT_POSITION_Y, which allows for multiple fingers to be tracked. If the device supports it, theABS_MT_WIDTH_MAJORmay be used to provide the size of the approaching finger. Anisotropy and direction may be specified withABS_MT_TOUCH_MINOR,ABS_MT_WIDTH_MINORandABS_MT_ORIENTATION. Devices with more granular information may specify general shapes as blobs, i.e., as a sequence of rectangular shapes grouped together by aABS_MT_BLOB_ID.
To clarify: the MT_TOUCH_MAJOR is to track a touchpoint over time, and the BLOB_ID to compile a arbitrarily shaped touchpoint within the same event?
Would it make sense to allow specification of a blob as bit/bytemask? There are a few devices that can do detection of multi-color non-rectangular touchpoints, especially camera-based systems. Limiting these to a sequence of rectangles means dropping information that may be useful for certain tasks (e.g. fingerprint detection).
OTOH, a rectangle as bounding box with an accompanying (optional) bytemask can pass this data on to prospective clients.
Cheers,
Peter
Quoted 28 lines · Peter Hutterer
Quoted 13 lines · Henrik Rydberg
In order to utilize the full power of the new multi-touch devices, a way to report detailed finger data to user space is needed. This patch adds a multi-touch (MT) protocol which allows drivers to report details for an arbitrary number of fingers.
The driver sends a
SYN_MT_REPORTevent via theinput_mt_sync()function when a complete finger has been reported.In order to stay compatible with existing applications, the data reported in a finger packet must not be recognized as single-touch events. In addition, all finger data must bypass input filtering, since subsequent events of the same type refer to different fingers.
Quoted 10 lines
A set of
ABS_MTevents with the desired properties are defined. The events are divided into categories, to allow for partial implementation. The minimum set consists ofABS_MT_TOUCH_MAJOR,ABS_MT_POSITION_XandABS_MT_POSITION_Y, which allows for multiple fingers to be tracked. If the device supports it, theABS_MT_WIDTH_MAJORmay be used to provide the size of the approaching finger. Anisotropy and direction may be specified withABS_MT_TOUCH_MINOR,ABS_MT_WIDTH_MINORandABS_MT_ORIENTATION. Devices with more granular information may specify general shapes as blobs, i.e., as a sequence of rectangular shapes grouped together by aABS_MT_BLOB_ID.To clarify: the
MT_TOUCH_MAJORis to track a touchpoint over time, and theBLOB_IDto compile a arbitrarily shaped touchpoint within the same event?
Quoted 7 lines
Would it make sense to allow specification of a blob as bit/bytemask? There are a few devices that can do detection of multi-color non-rectangular touchpoints, especially camera-based systems. Limiting these to a sequence of rectangles means dropping information that may be useful for certain tasks (e.g. fingerprint detection).
OTOH, a rectangle as bounding box with an accompanying (optional) bytemask can pass this data on to prospective clients.
Urk. I'd forgotten entirely about color. Some people are certainly using tokens on surfaces, distinguishable either by shape or color. But maybe that is better left outside of this until we have a more concrete non-research system to grok before defining a kernel interface; many of these systems may be doing advanced image processing in user space processes and talking directly to X with the result rather than via /dev/input.... Or do you feel you have good enough experience with the MERL table and other systems to take a stab at it now?
- Jim
Signature · 2 lines
Jim Gettys <[email protected]> One Laptop Per Child
Peter Hutterer wrote:
[...]
Quoted 3 lines
To clarify: the
MT_TOUCH_MAJORis to track a touchpoint over time, and theBLOB_IDto compile a arbitrarily shaped touchpoint within the same event?
MT_TOUCH_MAJOR is the major size of the touching object, or "pressure" in case the size of the approaching object (MT_WIDTH_MAJOR) is not available to compare the size to. The touch and position packets are sent to the multitouch X driver, where the actual finger tracking takes place. Thanks for the chance to clarify this.
Quoted 7 lines
Would it make sense to allow specification of a blob as bit/bytemask? There are a few devices that can do detection of multi-color non-rectangular touchpoints, especially camera-based systems. Limiting these to a sequence of rectangles means dropping information that may be useful for certain tasks (e.g. fingerprint detection).
OTOH, a rectangle as bounding box with an accompanying (optional) bytemask can pass this data on to prospective clients.
Very interesting idea, although I believe it is beyond the scope of the current patch.
There is a remaining question: how/whether to implement this interface in USB HID.....
On page 9 of the document you'll find at:
http://www.microsoft.com/whdc/device/input/DigitizerDrvs_touch.mspx
Microsoft is claiming in that document of December 8 they are proposing the interfaces in the document to the USB IF organization, and that hardware vendors for Windows 7 should follow that spec.
Jiri, Greg:
1) Does anyone know if Microsoft has actually made the proposal to the USB IF? Is there anyone in the community watching/commenting on USB IF proposals?
2) do you want generic USB HID patches or do you want to, for now, confine these to USB device specific drivers as much as possible?
- Jim
Signature · 2 lines
Jim Gettys <[email protected]> One Laptop Per Child
Quoted 10 lines · Jim Gettys
There is a remaining question: how/whether to implement this interface in USB HID.....
On page 9 of the document you'll find at:
http://www.microsoft.com/whdc/device/input/DigitizerDrvs_touch.mspxMicrosoft is claiming in that document of December 8 they are proposing the interfaces in the document to the USB IF organization, and that hardware vendors for Windows 7 should follow that spec.
Quoted 5 lines
Jiri, Greg:
1) Does anyone know if Microsoft has actually made the proposal to the USB IF? Is there anyone in the community watching/commenting on USB IF proposals?
Yes, I'm watching them, but nothing is public at this time (if it was currently under review privately within the USB-IF, I wouldn't be allowed to tell anyone, sorry.)
Quoted 2 lines
2) do you want generic USB HID patches or do you want to, for now, confine these to USB device specific drivers as much as possible?
As more devices should be showing up with this kind of support, it would seem to make sense to add it to the HID core, right?
thanks,
greg k-h
Quoted 7 lines · Greg KH
Quoted 3 lines
2) do you want generic USB HID patches or do you want to, for now, confine these to USB device specific drivers as much as possible?
As more devices should be showing up with this kind of support, it would seem to make sense to add it to the HID core, right?
Yes, the HID core makes sense to me if we had more than one (partially) implemented example in captivity and/or more than the unconfirmed pious claims of one Microsoft document. Right now, it is less than clear to me which way to jump.
I'll try to see what may be involved....
- Jim
Signature · 2 lines
Jim Gettys <[email protected]> One Laptop Per Child
Quoted 3 lines · Jim Gettys
1) Does anyone know if Microsoft has actually made the proposal to the USB IF? Is there anyone in the community watching/commenting on USB IF proposals?
This would be question for Greg.
Quoted 2 lines
2) do you want generic USB HID patches or do you want to, for now, confine these to USB device specific drivers as much as possible?
We already have a lot of drivers for special HID devices that emit usages that have not been defined by HID standard. These drivers are independent on the underlying transport (USB, Bluetooth). Therefore all you ideally need to do is to add just another driver which hooks itself to HID bus and performs all that is needed in addition to what generic HID code does. There are lot of such drivers in drivers/hid/hid-*.c which you can use as examples. Please let me know if you have any further questions.
Signature · 2 lines
Jiri Kosina SUSE Labs