EVT_NUM_COMP_PKTS and LE, GATT and SM
Hi All,
I've have encountered a problem while using the gatttool, where my write commands get clobbered by the LE ACL being disconnected prior to the ATT (fixed channel 4) WRITE_CMD being sent over the LE based ACL link.
I believe this is fundamentally due to there being no dependency on the EVT_NUM_COMP_PKTS event when gatttool decides that the WRITE_CMD has been successfully sent. There is multiple parts to this.
1. In User space, the WRITE_CMD pkt is written to the socket. Gatttool erroneously considers a successful socket write as completion, disconnects the socket and exits.
2. In Kernel space, the ACL packet is added to an ACL queue, which is separate from the CMD queue, which can allow either the Disconnect request, or the ACL packet to be sent over the H4 link to the baseband.
3. In the baseband, due to LE clocking (and possible other baseband activity) the ACL packet could be received first, and the Disconnect CMD second, and still result in the connection being detached prior to Tx of the ACL packet containing the ATT WRITE_CMD.
This is not an issue with any of the ATT READ/FIND/MTU or WRITE_REQ transactions, because they require a response from the server. I believe for ATT, this problem is restricted to the WRITE_CMD only, due to it's unacknowledged nature.
However, this will also be an issue with the LE Security Manager, because as stated in the Core Spec v4.0, Vol 3, in the last paragraph of 3.6.1 Key Distribution on page 630 (of 656):
> Key distribution is complete in the device sending the final key
when it receives
> the baseband acknowledgement for that key and is complete in the
receiving
> device when it receives the final key being distributed.
This is intended to prevent exactly the kind of problem I am experiencing with the ATT WRITE_CMD, and the acknowledgement from the baseband can only be the EVT_NUM_COMP_PKTS event.
While talking to my colleagues here, we were thinking that the cleanest method to get this accomplished would be by using the "select" method with the ATT socket, where the socket could be marked as non-writable by the kernel driver until the EVT_NUM_COMP_PKTS is received. The User space code could then wait for the socket to become writeable before issuing the socket disconnect.
If the Security manager is totally within the kernel, it probably does not have to do as much work, however it does still need to wait on the EVT_NUM_COMP_PKTS before disconnecting the remote device.
Has anybody else observed this issue with ATT WRITE_CMD? It could be getting exacerbated by slow (115Kbps) H4 links that I am using, however the hcidump tool confirms that the disconnect happens prior to the EVT_NUM_COMP_PKTS.
Brian Gix
[email protected]
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum
Additional point--
Quoted 62 lines
-----Original Message-----
From: [email protected] [mailto:linux-bluetooth- [email protected]] On Behalf Of Brian Gix
Sent: 06 December, 2010 4:36 PM
To: [email protected]
Subject:EVT_NUM_COMP_PKTSand LE, GATT and SMHi All,
I've have encountered a problem while using the gatttool, where my write
commands get clobbered by the LE ACL being disconnected prior to the ATT
(fixed channel 4)WRITE_CMDbeing sent over the LE based ACL link.I believe this is fundamentally due to there being no dependency on the
EVT_NUM_COMP_PKTSevent when gatttool decides that theWRITE_CMDhas been
successfully sent. There is multiple parts to this.1. In User space, the
WRITE_CMDpkt is written to the socket. Gatttool erroneously considers a successful socket write as completion, disconnects
the socket and exits.2. In Kernel space, the ACL packet is added to an ACL queue, which is separate from the CMD queue, which can allow either the Disconnect request,
or the ACL packet to be sent over the H4 link to the baseband.3. In the baseband, due to LE clocking (and possible other baseband activity) the ACL packet could be received first, and the Disconnect CMD
second, and still result in the connection being detached prior to Tx of the
ACL packet containing the ATTWRITE_CMD.This is not an issue with any of the ATT READ/FIND/MTU or
WRITE_REQtransactions, because they require a response from the server. I believe
for ATT, this problem is restricted to theWRITE_CMDonly, due to it's unacknowledged nature.However, this will also be an issue with the LE Security Manager, because as
stated in the Core Spec v4.0, Vol 3, in the last paragraph of 3.6.1 Key Distribution on page 630 (of 656):> Key distribution is complete in the device sending the final
key
when it receives
> the baseband acknowledgement for that key and is complete in
the
receiving
> device when it receives the final key being distributed.This is intended to prevent exactly the kind of problem I am experiencing
with the ATTWRITE_CMD, and the acknowledgement from the baseband can only
be theEVT_NUM_COMP_PKTSevent.
Quoted 6 lines
While talking to my colleagues here, we were thinking that the cleanest method to get this accomplished would be by using the "select" method with the ATT socket, where the socket could be marked as non-writable by the kernel driver until the
EVT_NUM_COMP_PKTSis received. The User space code could then wait for the socket to become writeable before issuing the socket disconnect.
This solution could be isolated to LE links (and perhaps even fixed channel 4) because of the strict cmd-resp nature of ATT transactions. This would cause the inability to queue multiple WRITE_CMDS before disconnecting however.
Quoted 26 lines
If the Security manager is totally within the kernel, it probably does not
have to do as much work, however it does still need to wait on theEVT_NUM_COMP_PKTSbefore disconnecting the remote device.Has anybody else observed this issue with ATT
WRITE_CMD? It could be getting
exacerbated by slow (115Kbps) H4 links that I am using, however the hcidump
tool confirms that the disconnect happens prior to theEVT_NUM_COMP_PKTS.Brian Gix
[email protected]
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum--
To unsubscribe from this list: send the line "unsubscribe linux- bluetooth" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Brian,
Quoted 10 lines · ext Brian Gix
Hi All,
I've have encountered a problem while using the gatttool, where my write commands get clobbered by the LE ACL being disconnected prior to the ATT (fixed channel 4)
WRITE_CMDbeing sent over the LE based ACL link.I believe this is fundamentally due to there being no dependency on the
EVT_NUM_COMP_PKTSevent when gatttool decides that theWRITE_CMDhas been successfully sent. There is multiple parts to this.
Quoted 8 lines
1. In User space, the
WRITE_CMDpkt is written to the socket. Gatttool erroneously considers a successful socket write as completion, disconnects the socket and exits.2. In Kernel space, the ACL packet is added to an ACL queue, which is separate from the CMD queue, which can allow either the Disconnect request, or the ACL packet to be sent over the H4 link to the baseband.
And in usb case CMD and ACL are even using different endpoints.
Quoted 9 lines
3. In the baseband, due to LE clocking (and possible other baseband activity) the ACL packet could be received first, and the Disconnect CMD second, and still result in the connection being detached prior to Tx of the ACL packet containing the ATT
WRITE_CMD.This is not an issue with any of the ATT READ/FIND/MTU or
WRITE_REQtransactions, because they require a response from the server. I believe for ATT, this problem is restricted to theWRITE_CMDonly, due to it's unacknowledged nature.
True.
Quoted 14 lines
However, this will also be an issue with the LE Security Manager, because as stated in the Core Spec v4.0, Vol 3, in the last paragraph of 3.6.1 Key Distribution on page 630 (of 656):
> Key distribution is complete in the device sending the final key
when it receives
> the baseband acknowledgement for that key and is complete in the
receiving
> device when it receives the final key being distributed.This is intended to prevent exactly the kind of problem I am experiencing with the ATT
WRITE_CMD, and the acknowledgement from the baseband can only be theEVT_NUM_COMP_PKTSevent.
Quoted 7 lines
While talking to my colleagues here, we were thinking that the cleanest method to get this accomplished would be by using the "select" method with the ATT socket, where the socket could be marked as non-writable by the kernel driver until the
EVT_NUM_COMP_PKTSis received. The User space code could then wait for the socket to become writeable before issuing the socket disconnect.
How about implement waiting on l2cap_sock_shutdown() like for ERTM. User space could then call shutdown() before closing the socket so make sure the data is sent or timeout occurred.
Would this be enough to solve the problem?
Quoted 7 lines
If the Security manager is totally within the kernel, it probably does not have to do as much work, however it does still need to wait on the
EVT_NUM_COMP_PKTSbefore disconnecting the remote device.Has anybody else observed this issue with ATT
WRITE_CMD? It could be getting exacerbated by slow (115Kbps) H4 links that I am using, however the hcidump tool confirms that the disconnect happens prior to theEVT_NUM_COMP_PKTS.
I have seen this problem but didn't dig it deeper.
Signature · 1 line
Ville
Hi Ville, Vinicius,
Quoted 13 lines · Ville Tervo
Hi Brian,
Quoted 3 lines · ext Brian Gix
Hi All,
I've have encountered a problem while using the gatttool, where my
write
Quoted 1 line
commands get clobbered by the LE ACL being disconnected prior to the
ATT
Quoted 3 lines
(fixed channel 4)
WRITE_CMDbeing sent over the LE based ACL link.I believe this is fundamentally due to there being no dependency on
the
Quoted 9 lines
Quoted 1 line
EVT_NUM_COMP_PKTSevent when gatttool decides that theWRITE_CMDhasbeen
Quoted 3 lines
successfully sent. There is multiple parts to this.
1. In User space, the
WRITE_CMDpkt is written to the socket.Gatttool
Quoted 1 line
erroneously considers a successful socket write as completion,
disconnects
Quoted 1 line
the socket and exits.
I would like to try Vinicius' patch that (may) address this, but again I do not have access to infradead, although I think gitorious, which you used before, was not a problem.
Quoted 55 lines
Quoted 3 lines
2. In Kernel space, the ACL packet is added to an ACL queue, which is separate from the CMD queue, which can allow either the Disconnect
request,
Quoted 2 lines
or the ACL packet to be sent over the H4 link to the baseband.
And in usb case CMD and ACL are even using different endpoints.
Quoted 2 lines
3. In the baseband, due to LE clocking (and possible other baseband activity) the ACL packet could be received first, and the Disconnect
CMD
Quoted 1 line
second, and still result in the connection being detached prior to Tx
of the
Quoted 4 lines
ACL packet containing the ATT
WRITE_CMD.This is not an issue with any of the ATT READ/FIND/MTU or
WRITE_REQtransactions, because they require a response from the server. Ibelieve
Quoted 1 line
for ATT, this problem is restricted to the
WRITE_CMDonly, due toit's
Quoted 1 line
unacknowledged nature.
True.
Quoted 1 line
However, this will also be an issue with the LE Security Manager,
because as
Quoted 1 line
stated in the Core Spec v4.0, Vol 3, in the last paragraph of 3.6.1
Key
Quoted 3 lines
Distribution on page 630 (of 656):
> Key distribution is complete in the device sending the final
key
Quoted 2 lines
when it receives
> the baseband acknowledgement for that key and is complete inthe
Quoted 4 lines
receiving
> device when it receives the final key being distributed.This is intended to prevent exactly the kind of problem I am
experiencing
Quoted 1 line
with the ATT
WRITE_CMD, and the acknowledgement from the baseband canonly
Quoted 3 lines
be the
EVT_NUM_COMP_PKTSevent.While talking to my colleagues here, we were thinking that the
cleanest
Quoted 1 line
method to get this accomplished would be by using the "select" method
with
Quoted 1 line
the ATT socket, where the socket could be marked as non-writable by
the
Quoted 1 line
kernel driver until the
EVT_NUM_COMP_PKTSis received. The User spacecode
Quoted 1 line
could then wait for the socket to become writeable before issuing the
socket
Quoted 8 lines
Quoted 2 lines
disconnect.
How about implement waiting on
l2cap_sock_shutdown()like for ERTM. User space
could then callshutdown()before closing the socket so make sure the data is
sent or timeout occurred.
If I understand l2cap_sock_shutdown() correctly, then I think this is the correct solution. The socket mode would be BASIC, and the channel (dcid) would be fixed-4 (or perhaps fixed-5 or fixed-6 as well), but instead of waiting for an ack, it would be waiting on the EVT_NUM_COMP_PKTS, that indicates the number of outstanding ACL packets on the connection is Zero, with the rest of the logic unchanged.
Quoted 19 lines
Would this be enough to solve the problem?
Quoted 1 line
If the Security manager is totally within the kernel, it probably
does not
Quoted 4 lines
have to do as much work, however it does still need to wait on the
EVT_NUM_COMP_PKTSbefore disconnecting the remote device.Has anybody else observed this issue with ATT
WRITE_CMD? It could begetting
Quoted 1 line
exacerbated by slow (115Kbps) H4 links that I am using, however the
hcidump
Quoted 1 line
tool confirms that the disconnect happens prior to the
EVT_NUM_COMP_PKTS.I have seen this problem but didn't dig it deeper.
--
Ville
Hi Brian,
Quoted 21 lines · Brian Gix
Hi Ville, Vinicius,
Quoted 18 lines · Ville Tervo
Hi Brian,
Quoted 3 lines · ext Brian Gix
Hi All,
I've have encountered a problem while using the gatttool, where my
write
Quoted 1 line
commands get clobbered by the LE ACL being disconnected prior to the
ATT
Quoted 3 lines
(fixed channel 4)
WRITE_CMDbeing sent over the LE based ACL link.I believe this is fundamentally due to there being no dependency on
the
Quoted 1 line
EVT_NUM_COMP_PKTSevent when gatttool decides that theWRITE_CMDhasbeen
Quoted 3 lines
successfully sent. There is multiple parts to this.
1. In User space, the
WRITE_CMDpkt is written to the socket.
Quoted 9 lines
Quoted 4 lines
Gatttool
Quoted 1 line
erroneously considers a successful socket write as completion,
disconnects
Quoted 1 line
the socket and exits.
I would like to try Vinicius' patch that (may) address this, but again I do not have access to infradead, although I think gitorious, which you used before, was not a problem.
Sorry, I forgot about this problem, please take a look here:
http://gitorious.org/~vcgomes/bluez/vcgomes-bluez
Quoted 89 lines
Quoted 55 lines
Quoted 3 lines
2. In Kernel space, the ACL packet is added to an ACL queue, which is separate from the CMD queue, which can allow either the Disconnect
request,
Quoted 2 lines
or the ACL packet to be sent over the H4 link to the baseband.
And in usb case CMD and ACL are even using different endpoints.
Quoted 2 lines
3. In the baseband, due to LE clocking (and possible other baseband activity) the ACL packet could be received first, and the Disconnect
CMD
Quoted 1 line
second, and still result in the connection being detached prior to Tx
of the
Quoted 4 lines
ACL packet containing the ATT
WRITE_CMD.This is not an issue with any of the ATT READ/FIND/MTU or
WRITE_REQtransactions, because they require a response from the server. Ibelieve
Quoted 1 line
for ATT, this problem is restricted to the
WRITE_CMDonly, due toit's
Quoted 1 line
unacknowledged nature.
True.
Quoted 1 line
However, this will also be an issue with the LE Security Manager,
because as
Quoted 1 line
stated in the Core Spec v4.0, Vol 3, in the last paragraph of 3.6.1
Key
Quoted 3 lines
Distribution on page 630 (of 656):
> Key distribution is complete in the device sending the final
key
Quoted 2 lines
when it receives
> the baseband acknowledgement for that key and is complete inthe
Quoted 4 lines
receiving
> device when it receives the final key being distributed.This is intended to prevent exactly the kind of problem I am
experiencing
Quoted 1 line
with the ATT
WRITE_CMD, and the acknowledgement from the baseband canonly
Quoted 3 lines
be the
EVT_NUM_COMP_PKTSevent.While talking to my colleagues here, we were thinking that the
cleanest
Quoted 1 line
method to get this accomplished would be by using the "select" method
with
Quoted 1 line
the ATT socket, where the socket could be marked as non-writable by
the
Quoted 1 line
kernel driver until the
EVT_NUM_COMP_PKTSis received. The User spacecode
Quoted 1 line
could then wait for the socket to become writeable before issuing the
socket
Quoted 8 lines
Quoted 2 lines
disconnect.
How about implement waiting on
l2cap_sock_shutdown()like for ERTM. User space
could then callshutdown()before closing the socket so make sure the data is
sent or timeout occurred.If I understand
l2cap_sock_shutdown()correctly, then I think this is the correct solution. The socket mode would be BASIC, and the channel (dcid) would be fixed-4 (or perhaps fixed-5 or fixed-6 as well), but instead of waiting for an ack, it would be waiting on theEVT_NUM_COMP_PKTS, that indicates the number of outstanding ACL packets on the connection is Zero, with the rest of the logic unchanged.Quoted 17 lines
Would this be enough to solve the problem?
Quoted 1 line
If the Security manager is totally within the kernel, it probably
does not
Quoted 4 lines
have to do as much work, however it does still need to wait on the
EVT_NUM_COMP_PKTSbefore disconnecting the remote device.Has anybody else observed this issue with ATT
WRITE_CMD? It could begetting
Quoted 1 line
exacerbated by slow (115Kbps) H4 links that I am using, however the
hcidump
Quoted 1 line
tool confirms that the disconnect happens prior to the
EVT_NUM_COMP_PKTS.I have seen this problem but didn't dig it deeper.
Quoted 7 lines
Quoted 2 lines
--
Ville--
To unsubscribe from this list: send the line "unsubscribe linux-bluetooth" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Cheers,
Signature · 1 line
Vinicius