xen: xenbus_dev_frontend: Fix XS_TRANSACTION_END handling
Subsystems (2)
PATCH 1/2 xen: xenbus_dev_frontend: Fix XS_TRANSACTION_END handling
Commit fd8aa9095a95 ("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic of xenbus_dev_request_and_reply() and xenbus_transaction_end().
Before on an error response to XS_TRANSACTION_END xenbus_dev_request_and_reply() would not decrement the active transaction counter. But xenbus_transaction_end() has always counted the transaction as finished regardless of the response.
The new behavior is that xenbus_dev_request_and_reply() and xenbus_transaction_end() will always count the transaction as finished regardless the response code (handled in xs_request_exit()).
But xenbus_dev_frontend tries to end a transaction on closing of the device if the XS_TRANSACTION_END failed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sent XS_TRANSACTION_END once regardless of the return code.
--- drivers/xen/xenbus/xenbus_dev_frontend.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/xen/xenbus/xenbus_dev_frontend.c +1 -1
@@ -365,7 +365,7 @@ void xenbus_dev_queue_reply(struct xb_req_data *req)if (WARN_ON (rc)) goto out; } - }else if (req->msg.type ==XS_TRANSACTION_END ) { + }else if (req->type ==XS_TRANSACTION_END ) { trans =xenbus_get_transaction (u, req->msg.tx_id);if (WARN_ON (!trans)) goto out;
Signature · 1 line
2.15.1
PATCH 2/2 xen: xenbus: WARN_ON XS_TRANSACTION_{START,END} misuse
As the previous commit shows it's quite easy to confuse the transaction reference counting by ending a transaction twice. So at least try to detect and report it.
--- drivers/xen/xenbus/xenbus_xs.c | 9 +++++++++ 1 file changed, 9 insertions(+)
drivers/xen/xenbus/xenbus_xs.c +9 -0
@@ -137,11 +137,20 @@ static uint32_t xs_request_enter(struct xb_req_data *req)void xs_request_exit (struct xb_req_data *req) { +unsigned int users_old; +spin_lock (&xs_state_lock); + users_old = xs_state_users; xs_state_users--;if ((req->type ==XS_TRANSACTION_START && req->msg.type ==XS_ERROR ) || req->type ==XS_TRANSACTION_END ) xs_state_users--; +if (WARN_ON (xs_state_users > users_old)) +/* +* Someone misused XS_TRANSACTION_{START,END}. Reset the +* reference counter so we might survive. +*/ + xs_state_users =0 ;spin_unlock (&xs_state_lock);if (xs_suspend_active && !xs_state_users)
Signature · 1 line
2.15.1
Re: [PATCH 2/2] xen: xenbus: WARN_ON XS_TRANSACTION_{START,END} misuse
Quoted 16 lines · Simon Gaiser
As the previous commit shows it's quite easy to confuse the transaction reference counting by ending a transaction twice. So at least try to detect and report it.
Signed-off-by Simon Gaiser <[email protected]>--- drivers/xen/xenbus/xenbus_xs.c | 9 +++++++++ 1 file changed, 9 insertions(+)drivers/xen/xenbus/xenbus_xs.c +0
-0@@ -137,11 +137,20 @@ static uint32_t xs_request_enter(struct xb_req_data *req)void xs_request_exit (struct xb_req_data *req)
drivers/xen/xenbus/xenbus_xs.c static uint32_t xs_request_enter(struct xb_req_data *req) · line 148
Quoted 10 lines
drivers/xen/xenbus/xenbus_xs.c +4
-0{ +unsigned int users_old; +spin_lock (&xs_state_lock); + users_old = xs_state_users; xs_state_users--;if ((req->type ==XS_TRANSACTION_START && req->msg.type ==XS_ERROR ) || req->type ==XS_TRANSACTION_END ) xs_state_users--; +if (WARN_ON (xs_state_users > users_old))
WARN_ON_ONCE()?
-boris
Quoted 9 lines
drivers/xen/xenbus/xenbus_xs.c +5
-0+/* +* Someone misused XS_TRANSACTION_{START,END}. Reset the +* reference counter so we might survive. +*/ + xs_state_users =0 ;spin_unlock (&xs_state_lock);if (xs_suspend_active && !xs_state_users)
Re: [PATCH 2/2] xen: xenbus: WARN_ON XS_TRANSACTION_{START,END} misuse
Boris Ostrovsky:
Quoted 10 lines · Simon Gaiser
drivers/xen/xenbus/xenbus_xs.c static uint32_t xs_request_enter(struct xb_req_data *req) · line 148
Quoted 6 lines
drivers/xen/xenbus/xenbus_xs.c +2
-0+ users_old = xs_state_users; xs_state_users--;if ((req->type ==XS_TRANSACTION_START && req->msg.type ==XS_ERROR ) || req->type ==XS_TRANSACTION_END ) xs_state_users--; +if (WARN_ON (xs_state_users > users_old))
WARN_ON_ONCE()?
Since we "fix" the wrong decrement by clamping at zero it should not happen immediately again. But if you prefer _ONCE I can change it.
Re: [PATCH 2/2] xen: xenbus: WARN_ON XS_TRANSACTION_{START, END} misuse
Quoted 10 lines · Simon Gaiser
Boris Ostrovsky:
Quoted 9 lines · Simon Gaiser
Quoted 6 lines
drivers/xen/xenbus/xenbus_xs.c +2
-0+ users_old = xs_state_users; xs_state_users--;if ((req->type ==XS_TRANSACTION_START && req->msg.type ==XS_ERROR ) || req->type ==XS_TRANSACTION_END ) xs_state_users--; +if (WARN_ON (xs_state_users > users_old))
Quoted 5 lines
Quoted 1 line
WARN_ON_ONCE()?Since we "fix" the wrong decrement by clamping at zero it should not happen immediately again. But if you prefer
_ONCEI can change it.
If this error can happen once then someone at least theoretically can construct a case when it is repeated. So let's switch to _ONCE() variant.
Thanks.
-boris
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH 2/2] xen: xenbus: WARN_ON XS_TRANSACTION_{START,END} misuse
In reply to Simon GaiserQuoted 10 lines · Simon Gaiser
Boris Ostrovsky:
Quoted 9 lines · Simon Gaiser
Quoted 6 lines
drivers/xen/xenbus/xenbus_xs.c +2
-0+ users_old = xs_state_users; xs_state_users--;if ((req->type ==XS_TRANSACTION_START && req->msg.type ==XS_ERROR ) || req->type ==XS_TRANSACTION_END ) xs_state_users--; +if (WARN_ON (xs_state_users > users_old))
Quoted 5 lines
Quoted 1 line
WARN_ON_ONCE()?Since we "fix" the wrong decrement by clamping at zero it should not happen immediately again. But if you prefer
_ONCEI can change it.
If this error can happen once then someone at least theoretically can construct a case when it is repeated. So let's switch to _ONCE() variant.
Thanks.
-boris
Re: [PATCH 2/2] xen: xenbus: WARN_ON XS_TRANSACTION_{START, END} misuse
In reply to Boris OstrovskyBoris Ostrovsky:
Quoted 10 lines · Simon Gaiser
drivers/xen/xenbus/xenbus_xs.c static uint32_t xs_request_enter(struct xb_req_data *req) · line 148
Quoted 6 lines
drivers/xen/xenbus/xenbus_xs.c +2
-0+ users_old = xs_state_users; xs_state_users--;if ((req->type ==XS_TRANSACTION_START && req->msg.type ==XS_ERROR ) || req->type ==XS_TRANSACTION_END ) xs_state_users--; +if (WARN_ON (xs_state_users > users_old))
WARN_ON_ONCE()?
Since we "fix" the wrong decrement by clamping at zero it should not happen immediately again. But if you prefer _ONCE I can change it.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH 2/2] xen: xenbus: WARN_ON XS_TRANSACTION_{START, END} misuse
In reply to Simon GaiserQuoted 16 lines · Simon Gaiser
As the previous commit shows it's quite easy to confuse the transaction reference counting by ending a transaction twice. So at least try to detect and report it.
Signed-off-by Simon Gaiser <[email protected]>--- drivers/xen/xenbus/xenbus_xs.c | 9 +++++++++ 1 file changed, 9 insertions(+)drivers/xen/xenbus/xenbus_xs.c +0
-0@@ -137,11 +137,20 @@ static uint32_t xs_request_enter(struct xb_req_data *req)void xs_request_exit (struct xb_req_data *req)
drivers/xen/xenbus/xenbus_xs.c static uint32_t xs_request_enter(struct xb_req_data *req) · line 148
Quoted 10 lines
drivers/xen/xenbus/xenbus_xs.c +4
-0{ +unsigned int users_old; +spin_lock (&xs_state_lock); + users_old = xs_state_users; xs_state_users--;if ((req->type ==XS_TRANSACTION_START && req->msg.type ==XS_ERROR ) || req->type ==XS_TRANSACTION_END ) xs_state_users--; +if (WARN_ON (xs_state_users > users_old))
WARN_ON_ONCE()?
-boris
drivers/xen/xenbus/xenbus_xs.c static uint32_t xs_request_enter(struct xb_req_data *req) · line 156
Quoted 9 lines
drivers/xen/xenbus/xenbus_xs.c +5
-0+/* +* Someone misused XS_TRANSACTION_{START,END}. Reset the +* reference counter so we might survive. +*/ + xs_state_users =0 ;spin_unlock (&xs_state_lock);if (xs_suspend_active && !xs_state_users)
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 2/2 xen: xenbus: WARN_ON XS_TRANSACTION_{START, END} misuse
In reply to Simon GaiserAs the previous commit shows it's quite easy to confuse the transaction reference counting by ending a transaction twice. So at least try to detect and report it.
--- drivers/xen/xenbus/xenbus_xs.c | 9 +++++++++ 1 file changed, 9 insertions(+)
drivers/xen/xenbus/xenbus_xs.c +9 -0
@@ -137,11 +137,20 @@ static uint32_t xs_request_enter(struct xb_req_data *req)void xs_request_exit (struct xb_req_data *req) { +unsigned int users_old; +spin_lock (&xs_state_lock); + users_old = xs_state_users; xs_state_users--;if ((req->type ==XS_TRANSACTION_START && req->msg.type ==XS_ERROR ) || req->type ==XS_TRANSACTION_END ) xs_state_users--; +if (WARN_ON (xs_state_users > users_old)) +/* +* Someone misused XS_TRANSACTION_{START,END}. Reset the +* reference counter so we might survive. +*/ + xs_state_users =0 ;spin_unlock (&xs_state_lock);if (xs_suspend_active && !xs_state_users)
Signature · 7 lines
2.15.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 19 lines · Simon Gaiser
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.
Quoted 3 lines
Cc <[email protected]> # 4.11Fixes fd8aa9095a95 ("xen: optimize xenbus driver for multiple concurrent xenstore accesses")Signed-off-by Simon Gaiser <[email protected]>
(although I'd prefer Juergen to also take a look at this)
Quoted 18 lines
--- drivers/xen/xenbus/xenbus_dev_frontend.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/xen/xenbus/xenbus_dev_frontend.c +1
-1@@ -365,7 +365,7 @@ void xenbus_dev_queue_reply(struct xb_req_data *req)if (WARN_ON (rc)) goto out; } - }else if (req->msg.type ==XS_TRANSACTION_END ) { + }else if (req->type ==XS_TRANSACTION_END ) { trans =xenbus_get_transaction (u, req->msg.tx_id);if (WARN_ON (!trans)) goto out;
Quoted 19 lines · Simon Gaiser
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.
Quoted 3 lines
Cc <[email protected]> # 4.11Fixes fd8aa9095a95 ("xen: optimize xenbus driver for multiple concurrent xenstore accesses")Signed-off-by Simon Gaiser <[email protected]>
(although I'd prefer Juergen to also take a look at this)
Quoted 10 lines
--- drivers/xen/xenbus/xenbus_dev_frontend.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/xen/xenbus/xenbus_dev_frontend.c +0
-0
drivers/xen/xenbus/xenbus_dev_frontend.c void xenbus_dev_queue_reply(struct xb_req_data *req) · line 371
Quoted 8 lines
drivers/xen/xenbus/xenbus_dev_frontend.c +1
-1@@ -365,7 +365,7 @@ void xenbus_dev_queue_reply(struct xb_req_data *req)if (WARN_ON (rc)) goto out; } - }else if (req->msg.type ==XS_TRANSACTION_END ) { + }else if (req->type ==XS_TRANSACTION_END ) { trans =xenbus_get_transaction (u, req->msg.tx_id);if (WARN_ON (!trans)) goto out;
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 8 lines · Simon Gaiser
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.
Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
The real problem is decrementing the counter when XS_TRANSACTION_END for a non-existing transaction is being sent.
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).
ENOENT should not decrement the transaction counter, while all other responses to XS_TRANSACTION_END should still do so.
Quoted 5 lines
But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.
A transaction in the list of transactions should not considered to be finished. Either it is not on the list or it is still pending.
Quoted 4 lines
Cc <[email protected]> # 4.11Fixes fd8aa9095a95 ("xen: optimize xenbus driver for multiple concurrent xenstore accesses")Signed-off-by Simon Gaiser <[email protected]>
So: your patch is a band-aid trying to cure the symptoms, but not the real problem. Please do it properly.
NAK.
Juergen
Juergen Gross
maintainerQuoted 17 lines · Juergen Gross
Quoted 8 lines · Simon Gaiser
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
The real problem is decrementing the counter when
XS_TRANSACTION_ENDfor a non-existing transaction is being sent.
Quoted 6 lines
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).ENOENT should not decrement the transaction counter, while all other responses to
XS_TRANSACTION_ENDshould still do so.
Sorry, I stand corrected: the ENOENT case should never happen, as this case is tested in xenbus_write_transaction(). It doesn't hurt to test for ENOENT, though.
What should be handled is EINVAL: this would happen if a user specified a string different from "T" and "F".
Juergen
Juergen Gross:
Quoted 5 lines · Simon Gaiser
Quoted 4 lines
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().
Quoted 8 lines
Quoted 4 lines
Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
So if xenstore frees the transaction why should we keep it in the list with pending transaction in xenbus_dev_frontend? That's exactly what this patch fixes by always removing it from the list, not only on a successful response (See below for the EINVAL case).
[...]
Quoted 8 lines
Quoted 5 lines
But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.A transaction in the list of transactions should not considered to be finished. Either it is not on the list or it is still pending.
With "considering a transaction closed" I mean "take the code path which removes the transaction from the list with pending transactions".
From the follow-up mail:
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).ENOENT should not decrement the transaction counter, while all other responses to
XS_TRANSACTION_ENDshould still do so.
Quoted 6 lines
Sorry, I stand corrected: the ENOENT case should never happen, as this case is tested in
xenbus_write_transaction(). It doesn't hurt to test for ENOENT, though.What should be handled is EINVAL: this would happen if a user specified a string different from "T" and "F".
Ok, I will handle those cases in xs_request_exit(). Although I don't like that this depends on the internals of xenstore (At least to me it's not obvious why it should only return ENOENT or EINVAL in this case).
In the xenbus_write_transaction() case checking the string before sending the transaction (like the transaction itself is verified) would avoid this problem.
Juergen Gross
maintainerQuoted 11 lines · Simon Gaiser
Juergen Gross:
Quoted 10 lines · Simon Gaiser
Quoted 8 lines
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.
Quoted 8 lines
Quoted 3 lines
Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
So if xenstore frees the transaction why should we keep it in the list with pending transaction in
xenbus_dev_frontend? That's exactly what this patch fixes by always removing it from the list, not only on a successful response (See below for the EINVAL case).
Aah, sorry, I seem to have misread my own coding. :-(
Yes, you are right. Sorry for not seeing it before.
Quoted 29 lines
[...]
Quoted 8 lines
Quoted 5 lines
But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.A transaction in the list of transactions should not considered to be finished. Either it is not on the list or it is still pending.
With "considering a transaction closed" I mean "take the code path which removes the transaction from the list with pending transactions".
From the follow-up mail:
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).ENOENT should not decrement the transaction counter, while all other responses to
XS_TRANSACTION_ENDshould still do so.Quoted 6 lines
Sorry, I stand corrected: the ENOENT case should never happen, as this case is tested in
xenbus_write_transaction(). It doesn't hurt to test for ENOENT, though.What should be handled is EINVAL: this would happen if a user specified a string different from "T" and "F".
Quoted 7 lines
Ok, I will handle those cases in
xs_request_exit(). Although I don't like that this depends on the internals of xenstore (At least to me it's not obvious why it should only return ENOENT or EINVAL in this case).In the
xenbus_write_transaction()case checking the string before sending the transaction (like the transaction itself is verified) would avoid this problem.
Right. I'd prefer this solution.
Remains the only problem you tried to tackle with your second patch: a kernel driver going crazy and ending transactions it never started (or ending them multiple times). The EINVAL case can't happen here, but ENOENT can. Either ENOENT has to be handled in xs_request_exit() or you need to keep track of the transactions like in the user interface and refuse ending an unknown transaction. Or you trust the kernel users. Trying to fix the usage counter seems to be the wrong approach IMO.
Juergen
Juergen Gross:
Quoted 64 lines · Simon Gaiser
Quoted 11 lines
Juergen Gross:
Quoted 10 lines · Simon Gaiser
Quoted 8 lines
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.Quoted 8 lines
Quoted 3 lines
Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
So if xenstore frees the transaction why should we keep it in the list with pending transaction in
xenbus_dev_frontend? That's exactly what this patch fixes by always removing it from the list, not only on a successful response (See below for the EINVAL case).Aah, sorry, I seem to have misread my own coding. :-(
Yes, you are right. Sorry for not seeing it before.
Quoted 29 lines
[...]
Quoted 8 lines
Quoted 5 lines
But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.A transaction in the list of transactions should not considered to be finished. Either it is not on the list or it is still pending.
With "considering a transaction closed" I mean "take the code path which removes the transaction from the list with pending transactions".
From the follow-up mail:
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).ENOENT should not decrement the transaction counter, while all other responses to
XS_TRANSACTION_ENDshould still do so.Quoted 6 lines
Sorry, I stand corrected: the ENOENT case should never happen, as this case is tested in
xenbus_write_transaction(). It doesn't hurt to test for ENOENT, though.What should be handled is EINVAL: this would happen if a user specified a string different from "T" and "F".
Quoted 7 lines
Ok, I will handle those cases in
xs_request_exit(). Although I don't like that this depends on the internals of xenstore (At least to me it's not obvious why it should only return ENOENT or EINVAL in this case).In the
xenbus_write_transaction()case checking the string before sending the transaction (like the transaction itself is verified) would avoid this problem.Right. I'd prefer this solution.
Quoted 7 lines
Remains the only problem you tried to tackle with your second patch: a kernel driver going crazy and ending transactions it never started (or ending them multiple times). The EINVAL case can't happen here, but ENOENT can. Either ENOENT has to be handled in
xs_request_exit()or you need to keep track of the transactions like in the user interface and refuse ending an unknown transaction. Or you trust the kernel users. Trying to fix the usage counter seems to be the wrong approach IMO.
The point of the second patch was to detect such bugs. This would have saved quite some time to find this bug. I added the "fix" of the counter I just because it was trivial after having the if there.
Adding tracking seems to be a quite complex solution for a _potential_ problem.
So I would go with checking ENOENT in xs_request_exit(). Should this be WARN_ON_ONCE()? Since this normally should not happen I would say yes.
Should I keep the reference counter sanity check? And if yes, with the "fix" to the counter?
Juergen Gross
maintainerQuoted 73 lines · Simon Gaiser
Juergen Gross:
Quoted 64 lines · Simon Gaiser
Quoted 11 lines
Juergen Gross:
Quoted 10 lines · Simon Gaiser
Quoted 8 lines
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.Quoted 8 lines
Quoted 3 lines
Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
So if xenstore frees the transaction why should we keep it in the list with pending transaction in
xenbus_dev_frontend? That's exactly what this patch fixes by always removing it from the list, not only on a successful response (See below for the EINVAL case).Aah, sorry, I seem to have misread my own coding. :-(
Yes, you are right. Sorry for not seeing it before.
Quoted 29 lines
[...]
Quoted 8 lines
Quoted 5 lines
But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.A transaction in the list of transactions should not considered to be finished. Either it is not on the list or it is still pending.
With "considering a transaction closed" I mean "take the code path which removes the transaction from the list with pending transactions".
From the follow-up mail:
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).ENOENT should not decrement the transaction counter, while all other responses to
XS_TRANSACTION_ENDshould still do so.Quoted 6 lines
Sorry, I stand corrected: the ENOENT case should never happen, as this case is tested in
xenbus_write_transaction(). It doesn't hurt to test for ENOENT, though.What should be handled is EINVAL: this would happen if a user specified a string different from "T" and "F".
Quoted 7 lines
Ok, I will handle those cases in
xs_request_exit(). Although I don't like that this depends on the internals of xenstore (At least to me it's not obvious why it should only return ENOENT or EINVAL in this case).In the
xenbus_write_transaction()case checking the string before sending the transaction (like the transaction itself is verified) would avoid this problem.Right. I'd prefer this solution.
Quoted 7 lines
Remains the only problem you tried to tackle with your second patch: a kernel driver going crazy and ending transactions it never started (or ending them multiple times). The EINVAL case can't happen here, but ENOENT can. Either ENOENT has to be handled in
xs_request_exit()or you need to keep track of the transactions like in the user interface and refuse ending an unknown transaction. Or you trust the kernel users. Trying to fix the usage counter seems to be the wrong approach IMO.
Quoted 6 lines
The point of the second patch was to detect such bugs. This would have saved quite some time to find this bug. I added the "fix" of the counter I just because it was trivial after having the if there.
Adding tracking seems to be a quite complex solution for a _potential_ problem.
I agree.
Quoted 2 lines
So I would go with checking ENOENT in
xs_request_exit(). Should this beWARN_ON_ONCE()? Since this normally should not happen I would say yes.
Yes, having a WARN_ON_ONCE here will help.
Quoted 2 lines
Should I keep the reference counter sanity check? And if yes, with the "fix" to the counter?
I'd drop it. This really should not happen and blowing up kernel size with checks of impossible situations isn't the way to go.
In case you really want to do something here you can add something like ASSERT(xs_state_users) before decrementing the counter.
Juergen
Juergen Gross
maintainerQuoted 73 lines · Simon Gaiser
Juergen Gross:
Quoted 64 lines · Simon Gaiser
Quoted 11 lines
Juergen Gross:
Quoted 10 lines · Simon Gaiser
Quoted 8 lines
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.Quoted 8 lines
Quoted 3 lines
Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
So if xenstore frees the transaction why should we keep it in the list with pending transaction in
xenbus_dev_frontend? That's exactly what this patch fixes by always removing it from the list, not only on a successful response (See below for the EINVAL case).Aah, sorry, I seem to have misread my own coding. :-(
Yes, you are right. Sorry for not seeing it before.
Quoted 29 lines
[...]
Quoted 8 lines
Quoted 5 lines
But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.A transaction in the list of transactions should not considered to be finished. Either it is not on the list or it is still pending.
With "considering a transaction closed" I mean "take the code path which removes the transaction from the list with pending transactions".
From the follow-up mail:
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).ENOENT should not decrement the transaction counter, while all other responses to
XS_TRANSACTION_ENDshould still do so.Quoted 6 lines
Sorry, I stand corrected: the ENOENT case should never happen, as this case is tested in
xenbus_write_transaction(). It doesn't hurt to test for ENOENT, though.What should be handled is EINVAL: this would happen if a user specified a string different from "T" and "F".
Quoted 7 lines
Ok, I will handle those cases in
xs_request_exit(). Although I don't like that this depends on the internals of xenstore (At least to me it's not obvious why it should only return ENOENT or EINVAL in this case).In the
xenbus_write_transaction()case checking the string before sending the transaction (like the transaction itself is verified) would avoid this problem.Right. I'd prefer this solution.
Quoted 7 lines
Remains the only problem you tried to tackle with your second patch: a kernel driver going crazy and ending transactions it never started (or ending them multiple times). The EINVAL case can't happen here, but ENOENT can. Either ENOENT has to be handled in
xs_request_exit()or you need to keep track of the transactions like in the user interface and refuse ending an unknown transaction. Or you trust the kernel users. Trying to fix the usage counter seems to be the wrong approach IMO.
Quoted 6 lines
The point of the second patch was to detect such bugs. This would have saved quite some time to find this bug. I added the "fix" of the counter I just because it was trivial after having the if there.
Adding tracking seems to be a quite complex solution for a _potential_ problem.
I agree.
Quoted 2 lines
So I would go with checking ENOENT in
xs_request_exit(). Should this beWARN_ON_ONCE()? Since this normally should not happen I would say yes.
Yes, having a WARN_ON_ONCE here will help.
Quoted 2 lines
Should I keep the reference counter sanity check? And if yes, with the "fix" to the counter?
I'd drop it. This really should not happen and blowing up kernel size with checks of impossible situations isn't the way to go.
In case you really want to do something here you can add something like ASSERT(xs_state_users) before decrementing the counter.
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross:
Quoted 64 lines · Simon Gaiser
Quoted 11 lines
Juergen Gross:
Quoted 10 lines · Simon Gaiser
Quoted 8 lines
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.Quoted 8 lines
Quoted 3 lines
Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
So if xenstore frees the transaction why should we keep it in the list with pending transaction in
xenbus_dev_frontend? That's exactly what this patch fixes by always removing it from the list, not only on a successful response (See below for the EINVAL case).Aah, sorry, I seem to have misread my own coding. :-(
Yes, you are right. Sorry for not seeing it before.
Quoted 29 lines
[...]
Quoted 8 lines
Quoted 5 lines
But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.A transaction in the list of transactions should not considered to be finished. Either it is not on the list or it is still pending.
With "considering a transaction closed" I mean "take the code path which removes the transaction from the list with pending transactions".
From the follow-up mail:
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).ENOENT should not decrement the transaction counter, while all other responses to
XS_TRANSACTION_ENDshould still do so.Quoted 6 lines
Sorry, I stand corrected: the ENOENT case should never happen, as this case is tested in
xenbus_write_transaction(). It doesn't hurt to test for ENOENT, though.What should be handled is EINVAL: this would happen if a user specified a string different from "T" and "F".
Quoted 7 lines
Ok, I will handle those cases in
xs_request_exit(). Although I don't like that this depends on the internals of xenstore (At least to me it's not obvious why it should only return ENOENT or EINVAL in this case).In the
xenbus_write_transaction()case checking the string before sending the transaction (like the transaction itself is verified) would avoid this problem.Right. I'd prefer this solution.
Quoted 7 lines
Remains the only problem you tried to tackle with your second patch: a kernel driver going crazy and ending transactions it never started (or ending them multiple times). The EINVAL case can't happen here, but ENOENT can. Either ENOENT has to be handled in
xs_request_exit()or you need to keep track of the transactions like in the user interface and refuse ending an unknown transaction. Or you trust the kernel users. Trying to fix the usage counter seems to be the wrong approach IMO.
The point of the second patch was to detect such bugs. This would have saved quite some time to find this bug. I added the "fix" of the counter I just because it was trivial after having the if there.
Adding tracking seems to be a quite complex solution for a _potential_ problem.
So I would go with checking ENOENT in xs_request_exit(). Should this be WARN_ON_ONCE()? Since this normally should not happen I would say yes.
Should I keep the reference counter sanity check? And if yes, with the "fix" to the counter?
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross
maintainerQuoted 11 lines · Simon Gaiser
Juergen Gross:
Quoted 10 lines · Simon Gaiser
Quoted 8 lines
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.
Quoted 8 lines
Quoted 3 lines
Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
So if xenstore frees the transaction why should we keep it in the list with pending transaction in
xenbus_dev_frontend? That's exactly what this patch fixes by always removing it from the list, not only on a successful response (See below for the EINVAL case).
Aah, sorry, I seem to have misread my own coding. :-(
Yes, you are right. Sorry for not seeing it before.
Quoted 29 lines
[...]
Quoted 8 lines
Quoted 5 lines
But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.A transaction in the list of transactions should not considered to be finished. Either it is not on the list or it is still pending.
With "considering a transaction closed" I mean "take the code path which removes the transaction from the list with pending transactions".
From the follow-up mail:
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).ENOENT should not decrement the transaction counter, while all other responses to
XS_TRANSACTION_ENDshould still do so.Quoted 6 lines
Sorry, I stand corrected: the ENOENT case should never happen, as this case is tested in
xenbus_write_transaction(). It doesn't hurt to test for ENOENT, though.What should be handled is EINVAL: this would happen if a user specified a string different from "T" and "F".
Quoted 7 lines
Ok, I will handle those cases in
xs_request_exit(). Although I don't like that this depends on the internals of xenstore (At least to me it's not obvious why it should only return ENOENT or EINVAL in this case).In the
xenbus_write_transaction()case checking the string before sending the transaction (like the transaction itself is verified) would avoid this problem.
Right. I'd prefer this solution.
Remains the only problem you tried to tackle with your second patch: a kernel driver going crazy and ending transactions it never started (or ending them multiple times). The EINVAL case can't happen here, but ENOENT can. Either ENOENT has to be handled in xs_request_exit() or you need to keep track of the transactions like in the user interface and refuse ending an unknown transaction. Or you trust the kernel users. Trying to fix the usage counter seems to be the wrong approach IMO.
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross
maintainerQuoted 17 lines · Juergen Gross
Quoted 8 lines · Simon Gaiser
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
The real problem is decrementing the counter when
XS_TRANSACTION_ENDfor a non-existing transaction is being sent.
Quoted 6 lines
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).ENOENT should not decrement the transaction counter, while all other responses to
XS_TRANSACTION_ENDshould still do so.
Sorry, I stand corrected: the ENOENT case should never happen, as this case is tested in xenbus_write_transaction(). It doesn't hurt to test for ENOENT, though.
What should be handled is EINVAL: this would happen if a user specified a string different from "T" and "F".
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 8 lines · Simon Gaiser
Commit
fd8aa9095a95("xen: optimize xenbus driver for multiple concurrent xenstore accesses") made a subtle change to the semantic ofxenbus_dev_request_and_reply()andxenbus_transaction_end().Before on an error response to
XS_TRANSACTION_ENDxenbus_dev_request_and_reply()would not decrement the active transaction counter. Butxenbus_transaction_end()has always counted the transaction as finished regardless of the response.
Which is correct now. Xenstore will free all transaction related data regardless of the response. A once failed transaction can't be repaired, it has to be repeated completely.
The real problem is decrementing the counter when XS_TRANSACTION_END for a non-existing transaction is being sent.
Quoted 3 lines
The new behavior is that
xenbus_dev_request_and_reply()andxenbus_transaction_end()will always count the transaction as finished regardless the response code (handled inxs_request_exit()).
ENOENT should not decrement the transaction counter, while all other responses to XS_TRANSACTION_END should still do so.
Quoted 5 lines
But
xenbus_dev_frontendtries to end a transaction on closing of the device if theXS_TRANSACTION_ENDfailed before. Trying to close the transaction twice corrupts the reference count. So fix this by also considering a transaction closed if we have sentXS_TRANSACTION_ENDonce regardless of the return code.
A transaction in the list of transactions should not considered to be finished. Either it is not on the list or it is still pending.
Quoted 4 lines
Cc <[email protected]> # 4.11Fixes fd8aa9095a95 ("xen: optimize xenbus driver for multiple concurrent xenstore accesses")Signed-off-by Simon Gaiser <[email protected]>
So: your patch is a band-aid trying to cure the symptoms, but not the real problem. Please do it properly.
NAK.
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel