LE: Low Latency GATT Write-Sign-Cmd
Hi All,
One of the useful features of Low Energy is the ability to rapidly establish a connection, send a responseless Signed Write Command, and tear the connection down. The obvious use case for this would be a standard remote control, where extremely low latency is expected.
The difference between this use case, and something like one of the Health profiles, is that a Health device (Heart rate Monitor, or Thermometer) is that where the Health device would have a persistent connection with high latency (less frequent Beacons), the Remote Control use case would rarely be connected, but when it is it will be as low latency as possible.
Currently we do not support Write Signed Commands at all, although I hope to have that addressed in the not too distant future. However the nature of it's usage model is not very well established here (BlueZ/Linux) so I would like to float a few ideas.
This is what should happen when the user hits a button on his remote control (or in his remote control linux APP):
1. Request LE Connection with Low Latency Parameters
2. Compose & sign the GATT Write Cmd (in parallel with Connect Req)
3. When Successful Connect Cmplt Evt occurs, send the Write Cmd Pkt
4. WAIT for the Num Completed Pkts Evt (VERY IMPORTANT)
5. As soon as you have it, Disconnect
This use case is only valid for LE (GATT connections must be Secure on BR/EDR, and therefore will never be as fast, and will never in fact even need to be signed). If done properly, the entire sequence can be completed in the 10's of milliseconds, from button press to Disconnect.
This calls for a different thread of operation from current GATT operation for a few reasons:
1. Standard GATT connections will have different Connect Parameters
2. Standard GATT connection establishment requires on a round trip from
User space--> Kernel --> Baseband and back before the ACL packet is even written to the Socket in User Space.
3. The signing crypto code is in the Kernel, and must be applied to each signed write command in a way that suggests some LE-GATT-Write-Cmd knowledge must exist in the kernel, as opposed to being a simple ACL pipe.
4. There is no current mechanism for the Kernel to inform the User space code that the ACL packet has finished transmission, and that it is OK to disconnect.
5. We already special case the handling of the LE-L2Cap CIDs for Signaling and SMP.
So I would like to suggest the following straw-man:
A new LE only MGMT command, which accepts as parameters the Remote Address, a length and a buffer. The buffer would be reasonable sized (I would recommend 27 bytes, which is the default LE MTU+HDR). The byte stream itself could either be composed by the User space GATT code (L2CAP header, Attribute Handle, Write Data) or they could be structure fields in the MGMT packet. On the Kernel side, the MGMT code would shepherd the packet through the connection, signing, and send process. If the connection to the remote device was already established, the existing connection would be used, the signing would only be done if the link was not encrypted, and the connection would be left up. When the Num-Completed-Pkts evt arrives, send the MGMT cmd_status response.
I think this would be way cleaner than trying to wedge this use case into our current GATT usage model.
Thoughts?
Signature · 4 lines
Brian Gix [email protected] Employee of Qualcomm Innovation Center, Inc. Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum
Marcel Holtmann
maintainerHi Brian,
Quoted 37 lines
One of the useful features of Low Energy is the ability to rapidly establish a connection, send a responseless Signed Write Command, and tear the connection down. The obvious use case for this would be a standard remote control, where extremely low latency is expected.
The difference between this use case, and something like one of the Health profiles, is that a Health device (Heart rate Monitor, or Thermometer) is that where the Health device would have a persistent connection with high latency (less frequent Beacons), the Remote Control use case would rarely be connected, but when it is it will be as low latency as possible.
Currently we do not support Write Signed Commands at all, although I hope to have that addressed in the not too distant future. However the nature of it's usage model is not very well established here (BlueZ/Linux) so I would like to float a few ideas.
This is what should happen when the user hits a button on his remote control (or in his remote control linux APP):
1. Request LE Connection with Low Latency Parameters
2. Compose & sign the GATT Write Cmd (in parallel with Connect Req)
3. When Successful Connect Cmplt Evt occurs, send the Write Cmd Pkt
4. WAIT for the Num Completed Pkts Evt (VERY IMPORTANT)
5. As soon as you have it, DisconnectThis use case is only valid for LE (GATT connections must be Secure on BR/EDR, and therefore will never be as fast, and will never in fact even need to be signed). If done properly, the entire sequence can be completed in the 10's of milliseconds, from button press to Disconnect.
This calls for a different thread of operation from current GATT operation for a few reasons:
1. Standard GATT connections will have different Connect Parameters
Quoted 7 lines
2. Standard GATT connection establishment requires on a round trip from User space--> Kernel --> Baseband and back before the ACL packet is even written to the Socket in User Space.
3. The signing crypto code is in the Kernel, and must be applied to each signed write command in a way that suggests some LE-GATT-Write-Cmd knowledge must exist in the kernel, as opposed to being a simple ACL pipe.
the kernel crypto can be easily used from userspace. See PF_ALG.
So the real question is what's the price of a context switch for us?
Quoted 22 lines
4. There is no current mechanism for the Kernel to inform the User space code that the ACL packet has finished transmission, and that it is OK to disconnect.
5. We already special case the handling of the LE-L2Cap CIDs for Signaling and SMP.
So I would like to suggest the following straw-man:
A new LE only MGMT command, which accepts as parameters the Remote Address, a length and a buffer. The buffer would be reasonable sized (I would recommend 27 bytes, which is the default LE MTU+HDR). The byte stream itself could either be composed by the User space GATT code (L2CAP header, Attribute Handle, Write Data) or they could be structure fields in the MGMT packet. On the Kernel side, the MGMT code would shepherd the packet through the connection, signing, and send process. If the connection to the remote device was already established, the existing connection would be used, the signing would only be done if the link was not encrypted, and the connection would be left up. When the Num-Completed-Pkts evt arrives, send the MGMT
cmd_statusresponse.
Quoted 2 lines
I think this would be way cleaner than trying to wedge this use case into our current GATT usage model.
I think one of the most important questions that we have to ask ourselves at some point is if we wanna put ATT into the kernel.
The potential candidate that forces us to think about this is HID over Low Energy. However I like to see numbers on how the context switches with keeping ATT in userspace will effect our latency.
Regards
Marcel
Hi Marcel,
Quoted 4 lines · Marcel Holtmann
the kernel crypto can be easily used from userspace. See
PF_ALG.So the real question is what's the price of a context switch for us?
The timing to hit the earliest slot possible is what most concerns me. Although as Master we control the initial connection parameters, we really want the ACL packet ready to go as soon as we know the connection is up.
There is also the question about depending on positive verification from the Baseband that the packet has been sent (Num Cmplt Pkts Evt) before issuing the Disconnect HCI command. An early problem that I had with gatttool (before the interactive enhancement) was that even if the ACL pkt had been properly sent over the HCI link, the Disconnect cmd was preempting it, and the remote device would never receive the ACL pkt.
[....]
Quoted 9 lines
Quoted 2 lines
I think this would be way cleaner than trying to wedge this use case into our current GATT usage model.
I think one of the most important questions that we have to ask ourselves at some point is if we wanna put ATT into the kernel.
The potential candidate that forces us to think about this is HID over Low Energy. However I like to see numbers on how the context switches with keeping ATT in userspace will effect our latency.
I would support migrating ATT to the kernel. In fact another issue I have dealt with is ensuring during Pairing, that we are able to at least respond with minimal Error responses if someone tries to read our name, and there is no GATT client socket to user space to respond.
Signature · 4 lines
Brian Gix [email protected] Employee of Qualcomm Innovation Center, Inc. Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum
Hi Brian,
Quoted 1 line · Brian Gix
I would support migrating ATT to the kernel.
Same here.
Quoted 4 lines
In fact another issue I have dealt with is ensuring during Pairing, that we are able to at least respond with minimal Error responses if someone tries to read our name, and there is no GATT client socket to user space to respond.
This issue was already encountered during the last UPF and INdT has a fix for it. For whatever reason that fix doesn't seem to have showed up here on the mailing list for upstreaming. In general the capability of doing ATT signaling should be available as soon as we have a connection handle, and it should be independent of any ongoing SMP procedure. Only if the characteristic accessed requires higher security than is currently provided by the LE link should the ATT procedures fail with an error. In the use case you describe, since the Name characteristic doesn't have any special security requirements a request for it should be always responded to with a positive reply regardless of the current level of security or if there's an ongoing SMP operation.
Johan
Hi Johan,
Quoted 12 lines · Johan Hedberg
Hi Brian,
Quoted 1 line · Brian Gix
I would support migrating ATT to the kernel.
Same here.
Quoted 4 lines
In fact another issue I have dealt with is ensuring during Pairing, that we are able to at least respond with minimal Error responses if someone tries to read our name, and there is no GATT client socket to user space to respond.
Quoted 11 lines
This issue was already encountered during the last UPF and INdT has a fix for it. For whatever reason that fix doesn't seem to have showed up here on the mailing list for upstreaming. In general the capability of doing ATT signaling should be available as soon as we have a connection handle, and it should be independent of any ongoing SMP procedure. Only if the characteristic accessed requires higher security than is currently provided by the LE link should the ATT procedures fail with an error. In the use case you describe, since the Name characteristic doesn't have any special security requirements a request for it should be always responded to with a positive reply regardless of the current level of security or if there's an ongoing SMP operation.
I have the solution to return an error that I could upstream if needed.
It worked sufficiently well to get past the UPF testing scenario's
where it was an issue, but as you point out, it should never return an error in the first place. However, as a stop-gap temporary solution, it works quite well.
Signature · 4 lines
Brian Gix [email protected] Employee of Qualcomm Innovation Center, Inc. Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum
Hi Johan,
Quoted 7 lines · Johan Hedberg
Hi Brian,
Quoted 1 line · Brian Gix
I would support migrating ATT to the kernel.
Same here.
Quoted 8 lines
Quoted 4 lines
In fact another issue I have dealt with is ensuring during Pairing, that we are able to at least respond with minimal Error responses if someone tries to read our name, and there is no GATT client socket to user space to respond.
This issue was already encountered during the last UPF and INdT has a fix for it. For whatever reason that fix doesn't seem to have showed up here on the mailing list for upstreaming.
For that to work, and have a matching userspace and kernel, it needs proper support for pairing, i.e. "re-using" a LE link that was used for SMP for GATT requests. Those patches are being tested right now, they should appear on the list soon.
Quoted 10 lines
In general the capability of
doing ATT signaling should be available as soon as we have a connection handle, and it should be independent of any ongoing SMP procedure. Only if the characteristic accessed requires higher security than is currently provided by the LE link should the ATT procedures fail with an error. In the use case you describe, since the Name characteristic doesn't have any special security requirements a request for it should be always responded to with a positive reply regardless of the current level of security or if there's an ongoing SMP operation.
Quoted 5 lines
Johan
--
To unsubscribe from this list: send the line "unsubscribe linux-bluetooth" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Cheers,
Signature · 1 line
Vinicius
Hi Marcel,
Quoted 6 lines · Marcel Holtmann
I think one of the most important questions that we have to ask ourselves at some point is if we wanna put ATT into the kernel.
The potential candidate that forces us to think about this is HID over Low Energy. However I like to see numbers on how the context switches with keeping ATT in userspace will effect our latency.
I still fail to see how ATT handling in kernel would reduce context switches. A GATT operation is composed of one or more (possibly many, see e.g. discovery procedures) ATT PDUs.
Unless you are proposing GATT on kernel as well?
Regards,
Signature · 3 lines
Anderson Lizardo Instituto Nokia de Tecnologia - INdT Manaus - Brazil
Hi Anderson,
Quoted 10 lines · Anderson Lizardo
Hi Marcel,
Quoted 6 lines · Marcel Holtmann
I think one of the most important questions that we have to ask ourselves at some point is if we wanna put ATT into the kernel.
The potential candidate that forces us to think about this is HID over Low Energy. However I like to see numbers on how the context switches with keeping ATT in userspace will effect our latency.
Quoted 5 lines
I still fail to see how ATT handling in kernel would reduce context switches. A GATT operation is composed of one or more (possibly many, see e.g. discovery procedures) ATT PDUs.
Unless you are proposing GATT on kernel as well?
I don't think having GATT in the kernel is necessarily the logical conclusion to an ATT-to-kernel migration.
Remember that the focus of LE is *not* the discovery process. Discovery (of Devices, Services and Characteristics) is important, but what makes LE "Low Energy" is what happens *after* pairing and discovery have been completed, which ideally would happen One Time Only.
The LE profiles and services are being spec'd so that they can be highly efficient over the lifetime of the Low Energy device. The Linux side of the equation will rarely be an actual Low Energy device: It will just know how to talk to an LE device in such a way as to make that Button-cell battery driven device last a couple years if possible.
For that reason MOST day-to-day communication between a Linux based "Central" device will in fact be single-ATT procedures. If it takes more energy and context switches to do discovery using many compound procedures, then we pay that price, because it should only happens Once.
HID devices in particular, will almost certainly NOT use compound GATT procedures to communicate after the pairing and discovery point in time has past.
Signature · 4 lines
Brian Gix [email protected] Employee of Qualcomm Innovation Center, Inc. Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum
Hi Brian,
Quoted 16 lines · Brian Gix
Hi Anderson,
Quoted 13 lines · Anderson Lizardo
Hi Marcel,
Quoted 7 lines · Marcel Holtmann
I think one of the most important questions that we have to ask ourselves at some point is if we wanna put ATT into the kernel.
The potential candidate that forces us to think about this is HID over Low Energy. However I like to see numbers on how the context switches with keeping ATT in userspace will effect our latency.
Quoted 8 lines
Quoted 5 lines
I still fail to see how ATT handling in kernel would reduce context switches. A GATT operation is composed of one or more (possibly many, see e.g. discovery procedures) ATT PDUs.
Unless you are proposing GATT on kernel as well?
I don't think having GATT in the kernel is necessarily the logical conclusion to an ATT-to-kernel migration.
I was actually referring only to the context switch topic. You still need to send the ATT request parameters from userspace, which is not different from building the PDU on userspace (vs. building on kernel).
Unless I'm not understanding what Marcel meant with "ATT on kernel" (which is most probably the case).
Quoted 4 lines
Remember that the focus of LE is *not* the discovery process. Discovery (of Devices, Services and Characteristics) is important, but what makes LE "Low Energy" is what happens *after* pairing and discovery have been completed, which ideally would happen One Time Only.
The discovery procedure was only an example of multiple PDU operation. Take e.g. "read long". If only ATT is on kernel, it still need multiple context switches for the multiple PDUs involved for the GATT operation.
Quoted 10 lines
The LE profiles and services are being spec'd so that they can be highly efficient over the lifetime of the Low Energy device. The Linux side of the equation will rarely be an actual Low Energy device: It will just know how to talk to an LE device in such a way as to make that Button-cell battery driven device last a couple years if possible.
For that reason MOST day-to-day communication between a Linux based "Central" device will in fact be single-ATT procedures. If it takes more energy and context switches to do discovery using many compound procedures, then we pay that price, because it should only happens Once.
... and for this reason I don't see much gain in reducing context switches if *only* ATT is on kernel. (note: I'm not defending GATT on kernel).
Quoted 3 lines
HID devices in particular, will almost certainly NOT use compound GATT procedures to communicate after the pairing and discovery point in time has past.
Yes, and also I don't see why LE HID is an example of "many context switches". For sure, you will need a bunch of them for setting up the HID device (discovery + pairing), but after that it will "talk" HID with the kernel input subsystem (If I understood the basics of this profile).
Regards,
Signature · 3 lines
Anderson Lizardo Instituto Nokia de Tecnologia - INdT Manaus - Brazil
Hi Anderson,
Quoted 57 lines · Anderson Lizardo
Quoted 11 lines · Brian Gix
Quoted 10 lines · Anderson Lizardo
Quoted 7 lines · Marcel Holtmann
I think one of the most important questions that we have to ask ourselves at some point is if we wanna put ATT into the kernel.
The potential candidate that forces us to think about this is HID over Low Energy. However I like to see numbers on how the context switches with keeping ATT in userspace will effect our latency.
Quoted 8 lines
Quoted 5 lines
I still fail to see how ATT handling in kernel would reduce context switches. A GATT operation is composed of one or more (possibly many, see e.g. discovery procedures) ATT PDUs.
Unless you are proposing GATT on kernel as well?
I don't think having GATT in the kernel is necessarily the logical conclusion to an ATT-to-kernel migration.
I was actually referring only to the context switch topic. You still need to send the ATT request parameters from userspace, which is not different from building the PDU on userspace (vs. building on kernel).
Unless I'm not understanding what Marcel meant with "ATT on kernel" (which is most probably the case).
Quoted 4 lines
Remember that the focus of LE is *not* the discovery process. Discovery (of Devices, Services and Characteristics) is important, but what makes LE "Low Energy" is what happens *after* pairing and discovery have been completed, which ideally would happen One Time Only.
The discovery procedure was only an example of multiple PDU operation. Take e.g. "read long". If only ATT is on kernel, it still need multiple context switches for the multiple PDUs involved for the GATT operation.
Quoted 10 lines
The LE profiles and services are being spec'd so that they can be highly efficient over the lifetime of the Low Energy device. The Linux side of the equation will rarely be an actual Low Energy device: It will just know how to talk to an LE device in such a way as to make that Button-cell battery driven device last a couple years if possible.
For that reason MOST day-to-day communication between a Linux based "Central" device will in fact be single-ATT procedures. If it takes more energy and context switches to do discovery using many compound procedures, then we pay that price, because it should only happens Once.
... and for this reason I don't see much gain in reducing context switches if *only* ATT is on kernel. (note: I'm not defending GATT on kernel).
Quoted 3 lines
HID devices in particular, will almost certainly NOT use compound GATT procedures to communicate after the pairing and discovery point in time has past.
Quoted 5 lines
Yes, and also I don't see why LE HID is an example of "many context switches". For sure, you will need a bunch of them for setting up the HID device (discovery + pairing), but after that it will "talk" HID with the kernel input subsystem (If I understood the basics of this profile).
I don't know about everyone else, but most of the context switching that I would like to see avoided is for single ATT-procedures that:
Connect-DoSomething-Disconnect
That is the case for any Write-Signed-Command type of operations.
Perhaps HID is different, although I think the idea for HID is to not require the round trip of:
Baseband --> Kernel --> UserSpace --> Kernel --> HIDQueue
Would the Context switching make significant difference? I suspect that on some platforms it would be hardly noticeable, but I would rather design something that we *know* requires low latency, to assume it is operating in an environment that might be hostile to latency concerns.
I personally think a fully feature rich GATT is too heavy to include in the kernel, but that perhaps a specialized HID server (recognized by a kernel resident ATT driver as being in the proper ATT Handle range) might make some sense.
Signature · 4 lines
Brian Gix [email protected] Employee of Qualcomm Innovation Center, Inc. Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum