usb: dwc3: gadget: Use list_replace_init() before traversing lists
Subsystems (2)
PATCH 1/1 usb: dwc3: gadget: Use list_replace_init() before traversing lists
The list_for_each_entry_safe() macro saves the current item (n) and the item after (n+1), so that n can be safely removed without corrupting the list. However, when traversing the list and removing items using gadget giveback, the DWC3 lock is briefly released, allowing other routines to execute. There is a situation where, while items are being removed from the cancelled_list using dwc3_gadget_ep_cleanup_cancelled_requests(), the pullup disable routine is running in parallel (due to UDC unbind). As the cleanup routine removes n, and the pullup disable removes n+1, once the cleanup retakes the DWC3 lock, it references a request who was already removed/handled. With list debug enabled, this leads to a panic. Ensure all instances of the macro are replaced where gadget giveback is used.
Example call stack:
Thread#1:__dwc3_gadget_ep_set_halt() - CLEAR HALT
-> dwc3_gadget_ep_cleanup_cancelled_requests()
->list_for_each_entry_safe()
->dwc3_gadget_giveback(n)
->dwc3_gadget_del_and_unmap_request()- n deleted[cancelled_list]
->spin_unlock
->Thread#2 executes
...
->dwc3_gadget_giveback(n+1)
->Already removed!Thread#2:dwc3_gadget_pullup()
->waiting for dwc3 spin_lock
...
->Thread#1 released lock
->dwc3_stop_active_transfers()
->dwc3_remove_requests()
->fetches n+1 item from cancelled_list (n removed by Thread#1)
->dwc3_gadget_giveback()
->dwc3_gadget_del_and_unmap_request()- n+1
deleted[cancelled_list]
->spin_unlockFix this condition by utilizing list_replace_init(), and traversing through a local copy of the current elements in the endpoint lists. This will also set the parent list as empty, so if another thread is also looping through the list, it will be empty on the next iteration.
---
Previous patchset:
https://lore.kernel.org/linux-usb/[email protected]/
--- drivers/usb/dwc3/gadget.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)
drivers/usb/dwc3/gadget.c +16 -2
@@ -1926,9 +1926,13 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) {struct dwc3_request *req;struct dwc3_request *tmp; +struct list_head local;struct dwc3 *dwc = dep->dwc; -list_for_each_entry_safe (req, tmp, &dep->cancelled_list, list) { +restart: +list_replace_init (&dep->cancelled_list, &local); + +list_for_each_entry_safe (req, tmp, &local, list) {dwc3_gadget_ep_skip_trbs (dep, req);switch (req->status) {case DWC3_REQUEST_STATUS_DISCONNECTED :
@@ -1946,6 +1950,9 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep)break ; } } + +if (!list_empty (&dep->cancelled_list)) + goto restart; } static int dwc3_gadget_ep_dequeue(struct usb_ep *ep,
@@ -3190,8 +3197,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep, {struct dwc3_request *req;struct dwc3_request *tmp; +struct list_head local; -list_for_each_entry_safe (req, tmp, &dep->started_list, list) { +restart: +list_replace_init (&dep->started_list, &local); + +list_for_each_entry_safe (req, tmp, &local, list) {int ret; ret = dwc3_gadget_ep_cleanup_completed_request(dep, event,
@@ -3199,6 +3210,9 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,if (ret)break ; } + +if (!list_empty (&dep->started_list)) + goto restart; }static bool dwc3_gadget_ep_should_continue (struct dwc3_ep *dep)
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
Hi,
Quoted 61 lines · Wesley Cheng
The
list_for_each_entry_safe()macro saves the current item (n) and the item after (n+1), so that n can be safely removed without corrupting the list. However, when traversing the list and removing items using gadget giveback, the DWC3 lock is briefly released, allowing other routines to execute. There is a situation where, while items are being removed from thecancelled_listusingdwc3_gadget_ep_cleanup_cancelled_requests(), the pullup disable routine is running in parallel (due to UDC unbind). As the cleanup routine removes n, and the pullup disable removes n+1, once the cleanup retakes the DWC3 lock, it references a request who was already removed/handled. With list debug enabled, this leads to a panic. Ensure all instances of the macro are replaced where gadget giveback is used.Example call stack:
Thread#1:
__dwc3_gadget_ep_set_halt()- CLEAR HALT
->dwc3_gadget_ep_cleanup_cancelled_requests()->list_for_each_entry_safe() ->dwc3_gadget_giveback(n) ->dwc3_gadget_del_and_unmap_request()- n deleted[cancelled_list] ->spin_unlock ->Thread#2 executes ... ->dwc3_gadget_giveback(n+1) ->Already removed!Thread#2:
dwc3_gadget_pullup()
->waiting for dwc3spin_lock
...
->Thread#1 released lock
->dwc3_stop_active_transfers()->dwc3_remove_requests() ->fetches n+1 item from cancelled_list (n removed by Thread#1) ->dwc3_gadget_giveback() ->dwc3_gadget_del_and_unmap_request()- n+1 deleted[cancelled_list] ->spin_unlockFix this condition by utilizing
list_replace_init(), and traversing through a local copy of the current elements in the endpoint lists. This will also set the parent list as empty, so if another thread is also looping through the list, it will be empty on the next iteration.Fixes d4f1afe5e896 ("usb: dwc3: gadget: move requests to cancelled_list")Signed-off-by Wesley Cheng <[email protected]>---
Previous patchset:
https://lore.kernel.org/linux-usb/[email protected]/--- drivers/usb/dwc3/gadget.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)drivers/usb/dwc3/gadget.c +0
-0
drivers/usb/dwc3/gadget.c static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) · line 1933
Quoted 9 lines
drivers/usb/dwc3/gadget.c +3
-1@@ -1926,9 +1926,13 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) {struct dwc3_request *req;struct dwc3_request *tmp; +struct list_head local;struct dwc3 *dwc = dep->dwc; -list_for_each_entry_safe (req, tmp, &dep->cancelled_list, list) { +restart: +list_replace_init (&dep->cancelled_list, &local);
hmm, if the lock is held and IRQs disabled when this runs, then no other threads will be able to append requests to the list which makes the "restart" label unnecessary, no?
I wonder if we should release the lock and reenable interrupts after replacing the head. The problem is that dwc3_gadget_ep_cleanup_cancelled_requests() can run from the IRQ handler.
Alan, could you provide your insight here? Do you think we should defer this to a low priority tasklet or something along those lines?
Quoted 4 lines
drivers/usb/dwc3/gadget.c +1
-0+list_for_each_entry_safe (req, tmp, &local, list) {dwc3_gadget_ep_skip_trbs (dep, req);switch (req->status) {case DWC3_REQUEST_STATUS_DISCONNECTED :
Signature · 1 line
balbi
Hi Felipe,
Quoted 72 lines · Felipe Balbi
Hi,
Quoted 67 lines · Wesley Cheng
The
list_for_each_entry_safe()macro saves the current item (n) and the item after (n+1), so that n can be safely removed without corrupting the list. However, when traversing the list and removing items using gadget giveback, the DWC3 lock is briefly released, allowing other routines to execute. There is a situation where, while items are being removed from thecancelled_listusingdwc3_gadget_ep_cleanup_cancelled_requests(), the pullup disable routine is running in parallel (due to UDC unbind). As the cleanup routine removes n, and the pullup disable removes n+1, once the cleanup retakes the DWC3 lock, it references a request who was already removed/handled. With list debug enabled, this leads to a panic. Ensure all instances of the macro are replaced where gadget giveback is used.Example call stack:
Thread#1:
__dwc3_gadget_ep_set_halt()- CLEAR HALT
->dwc3_gadget_ep_cleanup_cancelled_requests()->list_for_each_entry_safe() ->dwc3_gadget_giveback(n) ->dwc3_gadget_del_and_unmap_request()- n deleted[cancelled_list] ->spin_unlock ->Thread#2 executes ... ->dwc3_gadget_giveback(n+1) ->Already removed!Thread#2:
dwc3_gadget_pullup()
->waiting for dwc3spin_lock
...
->Thread#1 released lock
->dwc3_stop_active_transfers()->dwc3_remove_requests() ->fetches n+1 item from cancelled_list (n removed by Thread#1) ->dwc3_gadget_giveback() ->dwc3_gadget_del_and_unmap_request()- n+1 deleted[cancelled_list] ->spin_unlockFix this condition by utilizing
list_replace_init(), and traversing through a local copy of the current elements in the endpoint lists. This will also set the parent list as empty, so if another thread is also looping through the list, it will be empty on the next iteration.Fixes d4f1afe5e896 ("usb: dwc3: gadget: move requests to cancelled_list")Signed-off-by Wesley Cheng <[email protected]>---
Previous patchset:
https://lore.kernel.org/linux-usb/[email protected]/--- drivers/usb/dwc3/gadget.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)drivers/usb/dwc3/gadget.c +1
-0@@ -1926,9 +1926,13 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) {struct dwc3_request *req;struct dwc3_request *tmp; +struct list_head local;struct dwc3 *dwc = dep->dwc;
Quoted 7 lines
drivers/usb/dwc3/gadget.c static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) · line 1933
Quoted 3 lines
drivers/usb/dwc3/gadget.c +2
-1-list_for_each_entry_safe (req, tmp, &dep->cancelled_list, list) { +restart: +list_replace_init (&dep->cancelled_list, &local);hmm, if the lock is held and IRQs disabled when this runs, then no other threads will be able to append requests to the list which makes the "restart" label unnecessary, no?
We do still call dwc3_gadget_giveback() which would release the lock briefly, so if there was another thread waiting on dwc->lock, it would be able to add additional items to that list.
Quoted 6 lines
I wonder if we should release the lock and reenable interrupts after replacing the head. The problem is that
dwc3_gadget_ep_cleanup_cancelled_requests()can run from the IRQ handler.
We would also need to consider that some of the APIs being called in these situations would also have the assumption that the dwc->lock is held, ie dwc3_gadget_giveback()
Thanks
Wesley Cheng
Quoted 9 lines
Alan, could you provide your insight here? Do you think we should defer this to a low priority tasklet or something along those lines?
Quoted 4 lines
drivers/usb/dwc3/gadget.c +1
-0+list_for_each_entry_safe (req, tmp, &local, list) {dwc3_gadget_ep_skip_trbs (dep, req);switch (req->status) {case DWC3_REQUEST_STATUS_DISCONNECTED :
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
Felipe Balbi
Acked-byHi,
Quoted 86 lines · Wesley Cheng
Quoted 67 lines
Quoted 67 lines
The
list_for_each_entry_safe()macro saves the current item (n) and the item after (n+1), so that n can be safely removed without corrupting the list. However, when traversing the list and removing items using gadget giveback, the DWC3 lock is briefly released, allowing other routines to execute. There is a situation where, while items are being removed from thecancelled_listusingdwc3_gadget_ep_cleanup_cancelled_requests(), the pullup disable routine is running in parallel (due to UDC unbind). As the cleanup routine removes n, and the pullup disable removes n+1, once the cleanup retakes the DWC3 lock, it references a request who was already removed/handled. With list debug enabled, this leads to a panic. Ensure all instances of the macro are replaced where gadget giveback is used.Example call stack:
Thread#1:
__dwc3_gadget_ep_set_halt()- CLEAR HALT
->dwc3_gadget_ep_cleanup_cancelled_requests()->list_for_each_entry_safe() ->dwc3_gadget_giveback(n) ->dwc3_gadget_del_and_unmap_request()- n deleted[cancelled_list] ->spin_unlock ->Thread#2 executes ... ->dwc3_gadget_giveback(n+1) ->Already removed!Thread#2:
dwc3_gadget_pullup()
->waiting for dwc3spin_lock
...
->Thread#1 released lock
->dwc3_stop_active_transfers()->dwc3_remove_requests() ->fetches n+1 item from cancelled_list (n removed by Thread#1) ->dwc3_gadget_giveback() ->dwc3_gadget_del_and_unmap_request()- n+1 deleted[cancelled_list] ->spin_unlockFix this condition by utilizing
list_replace_init(), and traversing through a local copy of the current elements in the endpoint lists. This will also set the parent list as empty, so if another thread is also looping through the list, it will be empty on the next iteration.Fixes d4f1afe5e896 ("usb: dwc3: gadget: move requests to cancelled_list")Signed-off-by Wesley Cheng <[email protected]>---
Previous patchset:
https://lore.kernel.org/linux-usb/[email protected]/--- drivers/usb/dwc3/gadget.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)drivers/usb/dwc3/gadget.c +1
-0@@ -1926,9 +1926,13 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) {struct dwc3_request *req;struct dwc3_request *tmp; +struct list_head local;struct dwc3 *dwc = dep->dwc;Quoted 7 lines
drivers/usb/dwc3/gadget.c static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) · line 1933
Quoted 3 lines
drivers/usb/dwc3/gadget.c +2
-1-list_for_each_entry_safe (req, tmp, &dep->cancelled_list, list) { +restart: +list_replace_init (&dep->cancelled_list, &local);hmm, if the lock is held and IRQs disabled when this runs, then no other threads will be able to append requests to the list which makes the "restart" label unnecessary, no?
We do still call
dwc3_gadget_giveback()which would release the lock briefly, so if there was another thread waiting ondwc->lock, it would be able to add additional items to that list.Quoted 6 lines
I wonder if we should release the lock and reenable interrupts after replacing the head. The problem is that
dwc3_gadget_ep_cleanup_cancelled_requests()can run from the IRQ handler.
Quoted 3 lines
We would also need to consider that some of the APIs being called in these situations would also have the assumption that the
dwc->lockis held, iedwc3_gadget_giveback()
yeah, good point. I think we're good to integrate this, unless Alan can shed some light on some particular possible race scenario we may have missed.
In any case:
Signature · 1 line
balbi
Quoted 72 lines · Felipe Balbi
Hi,
Quoted 67 lines · Wesley Cheng
The
list_for_each_entry_safe()macro saves the current item (n) and the item after (n+1), so that n can be safely removed without corrupting the list. However, when traversing the list and removing items using gadget giveback, the DWC3 lock is briefly released, allowing other routines to execute. There is a situation where, while items are being removed from thecancelled_listusingdwc3_gadget_ep_cleanup_cancelled_requests(), the pullup disable routine is running in parallel (due to UDC unbind). As the cleanup routine removes n, and the pullup disable removes n+1, once the cleanup retakes the DWC3 lock, it references a request who was already removed/handled. With list debug enabled, this leads to a panic. Ensure all instances of the macro are replaced where gadget giveback is used.Example call stack:
Thread#1:
__dwc3_gadget_ep_set_halt()- CLEAR HALT
->dwc3_gadget_ep_cleanup_cancelled_requests()->list_for_each_entry_safe() ->dwc3_gadget_giveback(n) ->dwc3_gadget_del_and_unmap_request()- n deleted[cancelled_list] ->spin_unlock ->Thread#2 executes ... ->dwc3_gadget_giveback(n+1) ->Already removed!Thread#2:
dwc3_gadget_pullup()
->waiting for dwc3spin_lock
...
->Thread#1 released lock
->dwc3_stop_active_transfers()->dwc3_remove_requests() ->fetches n+1 item from cancelled_list (n removed by Thread#1) ->dwc3_gadget_giveback() ->dwc3_gadget_del_and_unmap_request()- n+1 deleted[cancelled_list] ->spin_unlockFix this condition by utilizing
list_replace_init(), and traversing through a local copy of the current elements in the endpoint lists. This will also set the parent list as empty, so if another thread is also looping through the list, it will be empty on the next iteration.Fixes d4f1afe5e896 ("usb: dwc3: gadget: move requests to cancelled_list")Signed-off-by Wesley Cheng <[email protected]>---
Previous patchset:
https://lore.kernel.org/linux-usb/[email protected]/--- drivers/usb/dwc3/gadget.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)drivers/usb/dwc3/gadget.c +1
-0@@ -1926,9 +1926,13 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) {struct dwc3_request *req;struct dwc3_request *tmp; +struct list_head local;struct dwc3 *dwc = dep->dwc;
Quoted 7 lines
drivers/usb/dwc3/gadget.c static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) · line 1933
Quoted 3 lines
drivers/usb/dwc3/gadget.c +2
-1-list_for_each_entry_safe (req, tmp, &dep->cancelled_list, list) { +restart: +list_replace_init (&dep->cancelled_list, &local);hmm, if the lock is held and IRQs disabled when this runs, then no other threads will be able to append requests to the list which makes the "restart" label unnecessary, no?
As Wesley pointed out, the lock can be released during giveback and requests can be added to the cancelled_list at that time.
On the other hand, if that happens, do you need to process those requests in this function call? Will another cleanup iteration take care of them later? (I don't know the driver well enough to answer this.) If it will, you may not need to restart anything.
Quoted 7 lines
I wonder if we should release the lock and reenable interrupts after replacing the head. The problem is that
dwc3_gadget_ep_cleanup_cancelled_requests()can run from the IRQ handler.Alan, could you provide your insight here? Do you think we should defer this to a low priority tasklet or something along those lines?
I don't see why anything like that would be necessary. Giving back cancelled requests isn't important enough to warrant special treatment.
An alternative approach, used by some other drivers, is to stick with list_for_each_entry_safe as in the existing code, but go back to the restart label immediately each time the lock is released and reacquired.
Also, if this loop always removes the entry it is processing from the list (I don't know whether it does this), you don't have to use list_for_each_entry_safe. You can simply use list_first_entry.
Alan Stern
Quoted 8 lines
Quoted 4 lines
drivers/usb/dwc3/gadget.c +1
-0+list_for_each_entry_safe (req, tmp, &local, list) {dwc3_gadget_ep_skip_trbs (dep, req);switch (req->status) {case DWC3_REQUEST_STATUS_DISCONNECTED :Signature · 1 line
balbi
Quoted 42 lines · Wesley Cheng
The
list_for_each_entry_safe()macro saves the current item (n) and the item after (n+1), so that n can be safely removed without corrupting the list. However, when traversing the list and removing items using gadget giveback, the DWC3 lock is briefly released, allowing other routines to execute. There is a situation where, while items are being removed from thecancelled_listusingdwc3_gadget_ep_cleanup_cancelled_requests(), the pullup disable routine is running in parallel (due to UDC unbind). As the cleanup routine removes n, and the pullup disable removes n+1, once the cleanup retakes the DWC3 lock, it references a request who was already removed/handled. With list debug enabled, this leads to a panic. Ensure all instances of the macro are replaced where gadget giveback is used.Example call stack:
Thread#1:
__dwc3_gadget_ep_set_halt()- CLEAR HALT
->dwc3_gadget_ep_cleanup_cancelled_requests()->list_for_each_entry_safe() ->dwc3_gadget_giveback(n) ->dwc3_gadget_del_and_unmap_request()- n deleted[cancelled_list] ->spin_unlock ->Thread#2 executes ... ->dwc3_gadget_giveback(n+1) ->Already removed!Thread#2:
dwc3_gadget_pullup()
->waiting for dwc3spin_lock
...
->Thread#1 released lock
->dwc3_stop_active_transfers()->dwc3_remove_requests() ->fetches n+1 item from cancelled_list (n removed by Thread#1) ->dwc3_gadget_giveback() ->dwc3_gadget_del_and_unmap_request()- n+1 deleted[cancelled_list] ->spin_unlock
Quoted 7 lines
Fix this condition by utilizing
list_replace_init(), and traversing through a local copy of the current elements in the endpoint lists. This will also set the parent list as empty, so if another thread is also looping through the list, it will be empty on the next iteration.Fixes d4f1afe5e896 ("usb: dwc3: gadget: move requests to cancelled_list")Signed-off-by Wesley Cheng <[email protected]>
Hey Wesley,
Just as a heads up, since this patch just landed upstream, I've
bisected it down as causing a regression on the db845c/RB3 board.
After booting with mainline, I'm seeing attempts to connect via adb fail with:
error: device offline
Running "adb devices" provides:
List of devices attached
c4e1189c offline
After reverting this patch, I can properly connect via adb again, and "adb devices" shows the expected output:
List of devices attached
c4e1189c device
I've not been able to isolate what might be going on, as there's no obvious errors in dmesg. Any suggestions to further debug this?
thanks
-john
Quoted 77 lines · Wesley Cheng
The
list_for_each_entry_safe()macro saves the current item (n) and the item after (n+1), so that n can be safely removed without corrupting the list. However, when traversing the list and removing items using gadget giveback, the DWC3 lock is briefly released, allowing other routines to execute. There is a situation where, while items are being removed from thecancelled_listusingdwc3_gadget_ep_cleanup_cancelled_requests(), the pullup disable routine is running in parallel (due to UDC unbind). As the cleanup routine removes n, and the pullup disable removes n+1, once the cleanup retakes the DWC3 lock, it references a request who was already removed/handled. With list debug enabled, this leads to a panic. Ensure all instances of the macro are replaced where gadget giveback is used.Example call stack:
Thread#1:
__dwc3_gadget_ep_set_halt()- CLEAR HALT
->dwc3_gadget_ep_cleanup_cancelled_requests()->list_for_each_entry_safe() ->dwc3_gadget_giveback(n) ->dwc3_gadget_del_and_unmap_request()- n deleted[cancelled_list] ->spin_unlock ->Thread#2 executes ... ->dwc3_gadget_giveback(n+1) ->Already removed!Thread#2:
dwc3_gadget_pullup()
->waiting for dwc3spin_lock
...
->Thread#1 released lock
->dwc3_stop_active_transfers()->dwc3_remove_requests() ->fetches n+1 item from cancelled_list (n removed by Thread#1) ->dwc3_gadget_giveback() ->dwc3_gadget_del_and_unmap_request()- n+1 deleted[cancelled_list] ->spin_unlockFix this condition by utilizing
list_replace_init(), and traversing through a local copy of the current elements in the endpoint lists. This will also set the parent list as empty, so if another thread is also looping through the list, it will be empty on the next iteration.Fixes d4f1afe5e896 ("usb: dwc3: gadget: move requests to cancelled_list")Signed-off-by Wesley Cheng <[email protected]>---
Previous patchset:
https://lore.kernel.org/linux-usb/[email protected]/--- drivers/usb/dwc3/gadget.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)drivers/usb/dwc3/gadget.c +5
-1@@ -1926,9 +1926,13 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) {struct dwc3_request *req;struct dwc3_request *tmp; +struct list_head local;struct dwc3 *dwc = dep->dwc; -list_for_each_entry_safe (req, tmp, &dep->cancelled_list, list) { +restart: +list_replace_init (&dep->cancelled_list, &local); + +list_for_each_entry_safe (req, tmp, &local, list) {dwc3_gadget_ep_skip_trbs (dep, req);switch (req->status) {case DWC3_REQUEST_STATUS_DISCONNECTED :
drivers/usb/dwc3/gadget.c static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) · line 1956
Quoted 7 lines
drivers/usb/dwc3/gadget.c +3
-0@@ -1946,6 +1950,9 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep)break ; } } + +if (!list_empty (&dep->cancelled_list)) + goto restart; }
So, I'm not sure yet, but the "break" cases in the list_for_each_entry_safe seem suspicious to me.
It seems we've move the list to the local listhead, then as we process the local listhead, we may hit the "break" case, which will stop processing the list, and then we end up returning, losing the unprocessed items on the local listhead.
I suspect we need to move them back to the started/cancelled_list, or rework things so we don't hit the "break" cases and fully process the local list before returning.
thanks
-john
Thinh Nguyen
maintainer+ John Stultz
Quoted 100 lines · Wesley Cheng
The
list_for_each_entry_safe()macro saves the current item (n) and the item after (n+1), so that n can be safely removed without corrupting the list. However, when traversing the list and removing items using gadget giveback, the DWC3 lock is briefly released, allowing other routines to execute. There is a situation where, while items are being removed from thecancelled_listusingdwc3_gadget_ep_cleanup_cancelled_requests(), the pullup disable routine is running in parallel (due to UDC unbind). As the cleanup routine removes n, and the pullup disable removes n+1, once the cleanup retakes the DWC3 lock, it references a request who was already removed/handled. With list debug enabled, this leads to a panic. Ensure all instances of the macro are replaced where gadget giveback is used.Example call stack:
Thread#1:
__dwc3_gadget_ep_set_halt()- CLEAR HALT
->dwc3_gadget_ep_cleanup_cancelled_requests()->list_for_each_entry_safe() ->dwc3_gadget_giveback(n) ->dwc3_gadget_del_and_unmap_request()- n deleted[cancelled_list] ->spin_unlock ->Thread#2 executes ... ->dwc3_gadget_giveback(n+1) ->Already removed!Thread#2:
dwc3_gadget_pullup()
->waiting for dwc3spin_lock
...
->Thread#1 released lock
->dwc3_stop_active_transfers()->dwc3_remove_requests() ->fetches n+1 item from cancelled_list (n removed by Thread#1) ->dwc3_gadget_giveback() ->dwc3_gadget_del_and_unmap_request()- n+1 deleted[cancelled_list] ->spin_unlockFix this condition by utilizing
list_replace_init(), and traversing through a local copy of the current elements in the endpoint lists. This will also set the parent list as empty, so if another thread is also looping through the list, it will be empty on the next iteration.Fixes d4f1afe5e896 ("usb: dwc3: gadget: move requests to cancelled_list")Signed-off-by Wesley Cheng <[email protected]>---
Previous patchset:
https://urldefense.com/v3/__https://lore.kernel.org/linux-usb/[email protected]/__;!!A4F2R9G_pg!Ngid3pREhM1FWiRmEnCGrN6FhBvSxDTkPbZ4RzAEO5Ubs0aGSxtikFT1APzTWhgw42As$--- drivers/usb/dwc3/gadget.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)drivers/usb/dwc3/gadget.c +13
-2@@ -1926,9 +1926,13 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep) {struct dwc3_request *req;struct dwc3_request *tmp; +struct list_head local;struct dwc3 *dwc = dep->dwc; -list_for_each_entry_safe (req, tmp, &dep->cancelled_list, list) { +restart: +list_replace_init (&dep->cancelled_list, &local); + +list_for_each_entry_safe (req, tmp, &local, list) {dwc3_gadget_ep_skip_trbs (dep, req);switch (req->status) {case DWC3_REQUEST_STATUS_DISCONNECTED :@@ -1946,6 +1950,9 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep)break ; } } + +if (!list_empty (&dep->cancelled_list)) + goto restart; } static int dwc3_gadget_ep_dequeue(struct usb_ep *ep,@@ -3190,8 +3197,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep, {struct dwc3_request *req;struct dwc3_request *tmp; +struct list_head local; -list_for_each_entry_safe (req, tmp, &dep->started_list, list) { +restart: +list_replace_init (&dep->started_list, &local); + +list_for_each_entry_safe (req, tmp, &local, list) {int ret; ret = dwc3_gadget_ep_cleanup_completed_request(dep, event,
drivers/usb/dwc3/gadget.c static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep, · line 3215
Quoted 6 lines
drivers/usb/dwc3/gadget.c +3
-0@@ -3199,6 +3210,9 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,if (ret)break ; } + +if (!list_empty (&dep->started_list)) + goto restart;
This is not right. We don't cleanup the entire started list here. Sometime we end early because some TRBs are completed but not all.
BR,
Thinh
RFC PATCH 1/1 dwc3: gadget: Fix losing list items in dwc3_gadget_ep_cleanup_completed_requests()
In reply to John StultzIn commit d25d85061bd8 ("usb: dwc3: gadget: Use list_replace_init() before traversing lists"), a local list_head was introduced to process the started_list items to avoid races.
However, in dwc3_gadget_ep_cleanup_completed_requests() if dwc3_gadget_ep_cleanup_completed_request() fails, we break early, causing the items on the local list_head to be lost.
This issue showed up as problems on the db845c/RB3 board, where adb connetions would fail, showing the device as "offline".
This patch tries to fix the issue by if we are returning early we splice in the local list head back into the started_list and return (avoiding an infinite loop, as the started_list is now non-null).
Not sure if this is fully correct, but seems to work for me so I wanted to share for feedback.
--- drivers/usb/dwc3/gadget.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/usb/dwc3/gadget.c +6 -0
@@ -2990,6 +2990,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,break ; } +if (!list_empty (&local)) { +list_splice_tail (&local, &dep->started_list); +/* Return so we don't hit the restart case and loop forever */ +return ; + } +if (!list_empty (&dep->started_list)) goto restart; }
Signature · 1 line
2.25.1
Thinh Nguyen
maintainerRe: [RFC][PATCH] dwc3: gadget: Fix losing list items in dwc3_gadget_ep_cleanup_completed_requests()
Quoted 45 lines · John Stultz
In commit
d25d85061bd8("usb: dwc3: gadget: Uselist_replace_init()before traversing lists"), a locallist_headwas introduced to process thestarted_listitems to avoid races.However, in
dwc3_gadget_ep_cleanup_completed_requests()ifdwc3_gadget_ep_cleanup_completed_request()fails, we break early, causing the items on the locallist_headto be lost.This issue showed up as problems on the db845c/RB3 board, where adb connetions would fail, showing the device as "offline".
This patch tries to fix the issue by if we are returning early we splice in the local list head back into the
started_listand return (avoiding an infinite loop, as thestarted_listis now non-null).Not sure if this is fully correct, but seems to work for me so I wanted to share for feedback.
Cc Wesley Cheng <[email protected]>Cc Felipe Balbi <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Alan Stern <[email protected]>Cc Jack Pham <[email protected]>Cc Thinh Nguyen <[email protected]>Cc Todd Kjos <[email protected]>Cc Amit Pundir <[email protected]>Cc YongQin Liu <[email protected]>Cc Sumit Semwal <[email protected]>Cc Petri Gynther <[email protected]>Fixes d25d85061bd8 ("usb: dwc3: gadget: Use list_replace_init() before traversing lists")Signed-off-by John Stultz <[email protected]>--- drivers/usb/dwc3/gadget.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/usb/dwc3/gadget.c +0
-0@@ -2990,6 +2990,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,break ; }
drivers/usb/dwc3/gadget.c static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep, · line 3001
Quoted 10 lines
drivers/usb/dwc3/gadget.c +6
-0+if (!list_empty (&local)) { +list_splice_tail (&local, &dep->started_list); +/* Return so we don't hit the restart case and loop forever */ +return ; + } +if (!list_empty (&dep->started_list)) goto restart; }
No, we should revert the change for dwc3_gadget_ep_cleaup_completed_requests(). As I mentioned previously, we don't cleanup the entire started_list. If the original problem is due to disconnection in the middle of request completion, then we can just check for pullup_connected and exit the loop and let the dwc3_remove_requests() do the cleanup.
BR,
Thinh
Re: [RFC][PATCH] dwc3: gadget: Fix losing list items in dwc3_gadget_ep_cleanup_completed_requests()
Quoted 56 lines · Thinh Nguyen
Quoted 54 lines · John Stultz
In commit
d25d85061bd8("usb: dwc3: gadget: Uselist_replace_init()before traversing lists"), a locallist_headwas introduced to process thestarted_listitems to avoid races.However, in
dwc3_gadget_ep_cleanup_completed_requests()ifdwc3_gadget_ep_cleanup_completed_request()fails, we break early, causing the items on the locallist_headto be lost.This issue showed up as problems on the db845c/RB3 board, where adb connetions would fail, showing the device as "offline".
This patch tries to fix the issue by if we are returning early we splice in the local list head back into the
started_listand return (avoiding an infinite loop, as thestarted_listis now non-null).Not sure if this is fully correct, but seems to work for me so I wanted to share for feedback.
Cc Wesley Cheng <[email protected]>Cc Felipe Balbi <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Alan Stern <[email protected]>Cc Jack Pham <[email protected]>Cc Thinh Nguyen <[email protected]>Cc Todd Kjos <[email protected]>Cc Amit Pundir <[email protected]>Cc YongQin Liu <[email protected]>Cc Sumit Semwal <[email protected]>Cc Petri Gynther <[email protected]>Fixes d25d85061bd8 ("usb: dwc3: gadget: Use list_replace_init() before traversing lists")Signed-off-by John Stultz <[email protected]>--- drivers/usb/dwc3/gadget.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/usb/dwc3/gadget.c +6
-0@@ -2990,6 +2990,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,break ; } +if (!list_empty (&local)) { +list_splice_tail (&local, &dep->started_list); +/* Return so we don't hit the restart case and loop forever */ +return ; + } +if (!list_empty (&dep->started_list)) goto restart; }
Quoted 8 lines
Quoted 1 line
No, we should revert the change for
dwc3_gadget_ep_cleaup_completed_requests(). As I mentioned previously, we don't cleanup the entirestarted_list. If the original problem is due to disconnection in the middle of request completion, then we can just check forpullup_connectedand exit the loop and let thedwc3_remove_requests()do the cleanup.
Ok, sorry, I didn't read your mail in depth until I had this patch sent out. If a revert of d25d85061bd8 is the better fix, I'm fine with that too.
thanks
-john
Thinh Nguyen
maintainerRe: [RFC][PATCH] dwc3: gadget: Fix losing list items in dwc3_gadget_ep_cleanup_completed_requests()
Quoted 66 lines · John Stultz
Quoted 56 lines · Thinh Nguyen
Quoted 54 lines · John Stultz
In commit
d25d85061bd8("usb: dwc3: gadget: Uselist_replace_init()before traversing lists"), a locallist_headwas introduced to process thestarted_listitems to avoid races.However, in
dwc3_gadget_ep_cleanup_completed_requests()ifdwc3_gadget_ep_cleanup_completed_request()fails, we break early, causing the items on the locallist_headto be lost.This issue showed up as problems on the db845c/RB3 board, where adb connetions would fail, showing the device as "offline".
This patch tries to fix the issue by if we are returning early we splice in the local list head back into the
started_listand return (avoiding an infinite loop, as thestarted_listis now non-null).Not sure if this is fully correct, but seems to work for me so I wanted to share for feedback.
Cc Wesley Cheng <[email protected]>Cc Felipe Balbi <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Alan Stern <[email protected]>Cc Jack Pham <[email protected]>Cc Thinh Nguyen <[email protected]>Cc Todd Kjos <[email protected]>Cc Amit Pundir <[email protected]>Cc YongQin Liu <[email protected]>Cc Sumit Semwal <[email protected]>Cc Petri Gynther <[email protected]>Fixes d25d85061bd8 ("usb: dwc3: gadget: Use list_replace_init() before traversing lists")Signed-off-by John Stultz <[email protected]>--- drivers/usb/dwc3/gadget.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/usb/dwc3/gadget.c +6
-0@@ -2990,6 +2990,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,break ; } +if (!list_empty (&local)) { +list_splice_tail (&local, &dep->started_list); +/* Return so we don't hit the restart case and loop forever */ +return ; + } +if (!list_empty (&dep->started_list)) goto restart; }Quoted 8 lines
Quoted 1 line
No, we should revert the change for
dwc3_gadget_ep_cleaup_completed_requests(). As I mentioned previously, we don't cleanup the entirestarted_list. If the original problem is due to disconnection in the middle of request completion, then we can just check forpullup_connectedand exit the loop and let thedwc3_remove_requests()do the cleanup.
Quoted 7 lines
Ok, sorry, I didn't read your mail in depth until I had this patch sent out. If a revert of
d25d85061bd8is the better fix, I'm fine with that too.thanks
-john
IMO, we should revert this patch for now since it will cause regression. We can review and test a proper fix at a later time.
Thanks,
Thinh
Greg Kroah-Hartman
maintainerRe: [RFC][PATCH] dwc3: gadget: Fix losing list items in dwc3_gadget_ep_cleanup_completed_requests()
Quoted 71 lines · Thinh Nguyen
Quoted 70 lines · John Stultz
Quoted 56 lines · Thinh Nguyen
Quoted 54 lines · John Stultz
In commit
d25d85061bd8("usb: dwc3: gadget: Uselist_replace_init()before traversing lists"), a locallist_headwas introduced to process thestarted_listitems to avoid races.However, in
dwc3_gadget_ep_cleanup_completed_requests()ifdwc3_gadget_ep_cleanup_completed_request()fails, we break early, causing the items on the locallist_headto be lost.This issue showed up as problems on the db845c/RB3 board, where adb connetions would fail, showing the device as "offline".
This patch tries to fix the issue by if we are returning early we splice in the local list head back into the
started_listand return (avoiding an infinite loop, as thestarted_listis now non-null).Not sure if this is fully correct, but seems to work for me so I wanted to share for feedback.
Cc Wesley Cheng <[email protected]>Cc Felipe Balbi <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Alan Stern <[email protected]>Cc Jack Pham <[email protected]>Cc Thinh Nguyen <[email protected]>Cc Todd Kjos <[email protected]>Cc Amit Pundir <[email protected]>Cc YongQin Liu <[email protected]>Cc Sumit Semwal <[email protected]>Cc Petri Gynther <[email protected]>Fixes d25d85061bd8 ("usb: dwc3: gadget: Use list_replace_init() before traversing lists")Signed-off-by John Stultz <[email protected]>--- drivers/usb/dwc3/gadget.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/usb/dwc3/gadget.c +6
-0@@ -2990,6 +2990,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,break ; } +if (!list_empty (&local)) { +list_splice_tail (&local, &dep->started_list); +/* Return so we don't hit the restart case and loop forever */ +return ; + } +if (!list_empty (&dep->started_list)) goto restart; }Quoted 8 lines
Quoted 1 line
No, we should revert the change for
dwc3_gadget_ep_cleaup_completed_requests(). As I mentioned previously, we don't cleanup the entirestarted_list. If the original problem is due to disconnection in the middle of request completion, then we can just check forpullup_connectedand exit the loop and let thedwc3_remove_requests()do the cleanup.Ok, sorry, I didn't read your mail in depth until I had this patch sent out. If a revert of
d25d85061bd8is the better fix, I'm fine with that too.
Quoted 6 lines
Quoted 3 lines
thanks
-johnIMO, we should revert this patch for now since it will cause regression. We can review and test a proper fix at a later time.
Ok, can someone send me a revert please? That will go faster than me having to create it myself...
thanks,
greg k-h
Greg Kroah-Hartman
maintainerRe: [RFC][PATCH] dwc3: gadget: Fix losing list items in dwc3_gadget_ep_cleanup_completed_requests()
Quoted 76 lines · Greg Kroah-Hartman
Quoted 75 lines · Thinh Nguyen
Quoted 73 lines · John Stultz
Quoted 56 lines · Thinh Nguyen
Quoted 54 lines · John Stultz
In commit
d25d85061bd8("usb: dwc3: gadget: Uselist_replace_init()before traversing lists"), a locallist_headwas introduced to process thestarted_listitems to avoid races.However, in
dwc3_gadget_ep_cleanup_completed_requests()ifdwc3_gadget_ep_cleanup_completed_request()fails, we break early, causing the items on the locallist_headto be lost.This issue showed up as problems on the db845c/RB3 board, where adb connetions would fail, showing the device as "offline".
This patch tries to fix the issue by if we are returning early we splice in the local list head back into the
started_listand return (avoiding an infinite loop, as thestarted_listis now non-null).Not sure if this is fully correct, but seems to work for me so I wanted to share for feedback.
Cc Wesley Cheng <[email protected]>Cc Felipe Balbi <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Alan Stern <[email protected]>Cc Jack Pham <[email protected]>Cc Thinh Nguyen <[email protected]>Cc Todd Kjos <[email protected]>Cc Amit Pundir <[email protected]>Cc YongQin Liu <[email protected]>Cc Sumit Semwal <[email protected]>Cc Petri Gynther <[email protected]>Fixes d25d85061bd8 ("usb: dwc3: gadget: Use list_replace_init() before traversing lists")Signed-off-by John Stultz <[email protected]>--- drivers/usb/dwc3/gadget.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/usb/dwc3/gadget.c +6
-0@@ -2990,6 +2990,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,break ; } +if (!list_empty (&local)) { +list_splice_tail (&local, &dep->started_list); +/* Return so we don't hit the restart case and loop forever */ +return ; + } +if (!list_empty (&dep->started_list)) goto restart; }Quoted 8 lines
Quoted 1 line
No, we should revert the change for
dwc3_gadget_ep_cleaup_completed_requests(). As I mentioned previously, we don't cleanup the entirestarted_list. If the original problem is due to disconnection in the middle of request completion, then we can just check forpullup_connectedand exit the loop and let thedwc3_remove_requests()do the cleanup.Ok, sorry, I didn't read your mail in depth until I had this patch sent out. If a revert of
d25d85061bd8is the better fix, I'm fine with that too.thanks
-john
Quoted 5 lines
Quoted 2 lines
IMO, we should revert this patch for now since it will cause regression. We can review and test a proper fix at a later time.
Ok, can someone send me a revert please? That will go faster than me having to create it myself...
I'll go do this now...
Re: [RFC][PATCH] dwc3: gadget: Fix losing list items in dwc3_gadget_ep_cleanup_completed_requests()
In reply to Thinh NguyenHi Thinh,
Quoted 75 lines · Thinh Nguyen
Quoted 66 lines · John Stultz
Quoted 56 lines · Thinh Nguyen
Quoted 54 lines · John Stultz
In commit
d25d85061bd8("usb: dwc3: gadget: Uselist_replace_init()before traversing lists"), a locallist_headwas introduced to process thestarted_listitems to avoid races.However, in
dwc3_gadget_ep_cleanup_completed_requests()ifdwc3_gadget_ep_cleanup_completed_request()fails, we break early, causing the items on the locallist_headto be lost.This issue showed up as problems on the db845c/RB3 board, where adb connetions would fail, showing the device as "offline".
This patch tries to fix the issue by if we are returning early we splice in the local list head back into the
started_listand return (avoiding an infinite loop, as thestarted_listis now non-null).Not sure if this is fully correct, but seems to work for me so I wanted to share for feedback.
Cc Wesley Cheng <[email protected]>Cc Felipe Balbi <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Alan Stern <[email protected]>Cc Jack Pham <[email protected]>Cc Thinh Nguyen <[email protected]>Cc Todd Kjos <[email protected]>Cc Amit Pundir <[email protected]>Cc YongQin Liu <[email protected]>Cc Sumit Semwal <[email protected]>Cc Petri Gynther <[email protected]>Fixes d25d85061bd8 ("usb: dwc3: gadget: Use list_replace_init() before traversing lists")Signed-off-by John Stultz <[email protected]>--- drivers/usb/dwc3/gadget.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/usb/dwc3/gadget.c +6
-0@@ -2990,6 +2990,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,break ; } +if (!list_empty (&local)) { +list_splice_tail (&local, &dep->started_list); +/* Return so we don't hit the restart case and loop forever */ +return ; + } +if (!list_empty (&dep->started_list)) goto restart; }Quoted 8 lines
Quoted 1 line
No, we should revert the change for
dwc3_gadget_ep_cleaup_completed_requests(). As I mentioned previously, we don't cleanup the entirestarted_list. If the original problem is due to disconnection in the middle of request completion, then we can just check forpullup_connectedand exit the loop and let thedwc3_remove_requests()do the cleanup.Quoted 7 lines
Ok, sorry, I didn't read your mail in depth until I had this patch sent out. If a revert of
d25d85061bd8is the better fix, I'm fine with that too.thanks
-john
Quoted 6 lines
IMO, we should revert this patch for now since it will cause regression. We can review and test a proper fix at a later time.
Thanks,
Thinh
Another suggestion would just be to replace the loop with a while() loop and using list_entry() instead. That was what was discussed in the earlier patch series which also addresses the problem as well. Issue here is the tmp variable still carries a stale request after the dwc3 giveback is called. We can avoid that by always fetching the list_entry() instead of relying on list_for_each_safe()
https://lore.kernel.org/linux-usb/[email protected]/
Thanks
Wesley Cheng
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
Thinh Nguyen
maintainerRe: [RFC][PATCH] dwc3: gadget: Fix losing list items in dwc3_gadget_ep_cleanup_completed_requests()
Quoted 85 lines · Wesley Cheng
Hi Thinh,
Quoted 75 lines · Thinh Nguyen
Quoted 66 lines · John Stultz
Quoted 56 lines · Thinh Nguyen
Quoted 54 lines · John Stultz
In commit
d25d85061bd8("usb: dwc3: gadget: Uselist_replace_init()before traversing lists"), a locallist_headwas introduced to process thestarted_listitems to avoid races.However, in
dwc3_gadget_ep_cleanup_completed_requests()ifdwc3_gadget_ep_cleanup_completed_request()fails, we break early, causing the items on the locallist_headto be lost.This issue showed up as problems on the db845c/RB3 board, where adb connetions would fail, showing the device as "offline".
This patch tries to fix the issue by if we are returning early we splice in the local list head back into the
started_listand return (avoiding an infinite loop, as thestarted_listis now non-null).Not sure if this is fully correct, but seems to work for me so I wanted to share for feedback.
Cc Wesley Cheng <[email protected]>Cc Felipe Balbi <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Alan Stern <[email protected]>Cc Jack Pham <[email protected]>Cc Thinh Nguyen <[email protected]>Cc Todd Kjos <[email protected]>Cc Amit Pundir <[email protected]>Cc YongQin Liu <[email protected]>Cc Sumit Semwal <[email protected]>Cc Petri Gynther <[email protected]>Fixes d25d85061bd8 ("usb: dwc3: gadget: Use list_replace_init() before traversing lists")Signed-off-by John Stultz <[email protected]>--- drivers/usb/dwc3/gadget.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/usb/dwc3/gadget.c +6
-0@@ -2990,6 +2990,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,break ; } +if (!list_empty (&local)) { +list_splice_tail (&local, &dep->started_list); +/* Return so we don't hit the restart case and loop forever */ +return ; + } +if (!list_empty (&dep->started_list)) goto restart; }Quoted 8 lines
Quoted 1 line
No, we should revert the change for
dwc3_gadget_ep_cleaup_completed_requests(). As I mentioned previously, we don't cleanup the entirestarted_list. If the original problem is due to disconnection in the middle of request completion, then we can just check forpullup_connectedand exit the loop and let thedwc3_remove_requests()do the cleanup.Quoted 7 lines
Ok, sorry, I didn't read your mail in depth until I had this patch sent out. If a revert of
d25d85061bd8is the better fix, I'm fine with that too.thanks
-johnQuoted 6 lines
IMO, we should revert this patch for now since it will cause regression. We can review and test a proper fix at a later time.
Thanks,
Thinh
Quoted 9 lines
Another suggestion would just be to replace the loop with a
while()loop and usinglist_entry()instead. That was what was discussed in the earlier patch series which also addresses the problem as well. Issue here is the tmp variable still carries a stale request after the dwc3 giveback is called. We can avoid that by always fetching thelist_entry()instead of relying onlist_for_each_safe()
This should work, but the awkward thing is 2 loops from 2 separate threads competing to remove/giveback the requests and may report mix status.
BR,
Thinh
Thinh Nguyen
maintainerRe: [RFC][PATCH] dwc3: gadget: Fix losing list items in dwc3_gadget_ep_cleanup_completed_requests()
Quoted 93 lines · Thinh Nguyen
Quoted 92 lines · Wesley Cheng
Hi Thinh,
Quoted 75 lines · Thinh Nguyen
Quoted 66 lines · John Stultz
Quoted 56 lines · Thinh Nguyen
Quoted 54 lines · John Stultz
In commit
d25d85061bd8("usb: dwc3: gadget: Uselist_replace_init()before traversing lists"), a locallist_headwas introduced to process thestarted_listitems to avoid races.However, in
dwc3_gadget_ep_cleanup_completed_requests()ifdwc3_gadget_ep_cleanup_completed_request()fails, we break early, causing the items on the locallist_headto be lost.This issue showed up as problems on the db845c/RB3 board, where adb connetions would fail, showing the device as "offline".
This patch tries to fix the issue by if we are returning early we splice in the local list head back into the
started_listand return (avoiding an infinite loop, as thestarted_listis now non-null).Not sure if this is fully correct, but seems to work for me so I wanted to share for feedback.
Cc Wesley Cheng <[email protected]>Cc Felipe Balbi <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Alan Stern <[email protected]>Cc Jack Pham <[email protected]>Cc Thinh Nguyen <[email protected]>Cc Todd Kjos <[email protected]>Cc Amit Pundir <[email protected]>Cc YongQin Liu <[email protected]>Cc Sumit Semwal <[email protected]>Cc Petri Gynther <[email protected]>Fixes d25d85061bd8 ("usb: dwc3: gadget: Use list_replace_init() before traversing lists")Signed-off-by John Stultz <[email protected]>--- drivers/usb/dwc3/gadget.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/usb/dwc3/gadget.c +6
-0@@ -2990,6 +2990,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,break ; } +if (!list_empty (&local)) { +list_splice_tail (&local, &dep->started_list); +/* Return so we don't hit the restart case and loop forever */ +return ; + } +if (!list_empty (&dep->started_list)) goto restart; }Quoted 8 lines
Quoted 1 line
No, we should revert the change for
dwc3_gadget_ep_cleaup_completed_requests(). As I mentioned previously, we don't cleanup the entirestarted_list. If the original problem is due to disconnection in the middle of request completion, then we can just check forpullup_connectedand exit the loop and let thedwc3_remove_requests()do the cleanup.Quoted 7 lines
Ok, sorry, I didn't read your mail in depth until I had this patch sent out. If a revert of
d25d85061bd8is the better fix, I'm fine with that too.thanks
-johnQuoted 6 lines
IMO, we should revert this patch for now since it will cause regression. We can review and test a proper fix at a later time.
Thanks,
ThinhAnother suggestion would just be to replace the loop with a
while()loop and usinglist_entry()instead. That was what was discussed in the earlier patch series which also addresses the problem as well. Issue here is the tmp variable still carries a stale request after the dwc3 giveback is called. We can avoid that by always fetching thelist_entry()instead of relying onlist_for_each_safe()
Quoted 6 lines
This should work, but the awkward thing is 2 loops from 2 separate threads competing to remove/giveback the requests and may report mix status.
It's fine with me.
BR,
Thinh
Thinh Nguyen
maintainerRe: [RFC][PATCH] dwc3: gadget: Fix losing list items in dwc3_gadget_ep_cleanup_completed_requests()
Hi Wesley,
Quoted 94 lines · Thinh Nguyen
Quoted 93 lines · Thinh Nguyen
Quoted 92 lines · Wesley Cheng
Hi Thinh,
Quoted 75 lines · Thinh Nguyen
Quoted 66 lines · John Stultz
Quoted 56 lines · Thinh Nguyen
Quoted 54 lines · John Stultz
In commit
d25d85061bd8("usb: dwc3: gadget: Uselist_replace_init()before traversing lists"), a locallist_headwas introduced to process thestarted_listitems to avoid races.However, in
dwc3_gadget_ep_cleanup_completed_requests()ifdwc3_gadget_ep_cleanup_completed_request()fails, we break early, causing the items on the locallist_headto be lost.This issue showed up as problems on the db845c/RB3 board, where adb connetions would fail, showing the device as "offline".
This patch tries to fix the issue by if we are returning early we splice in the local list head back into the
started_listand return (avoiding an infinite loop, as thestarted_listis now non-null).Not sure if this is fully correct, but seems to work for me so I wanted to share for feedback.
Cc Wesley Cheng <[email protected]>Cc Felipe Balbi <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Alan Stern <[email protected]>Cc Jack Pham <[email protected]>Cc Thinh Nguyen <[email protected]>Cc Todd Kjos <[email protected]>Cc Amit Pundir <[email protected]>Cc YongQin Liu <[email protected]>Cc Sumit Semwal <[email protected]>Cc Petri Gynther <[email protected]>Fixes d25d85061bd8 ("usb: dwc3: gadget: Use list_replace_init() before traversing lists")Signed-off-by John Stultz <[email protected]>--- drivers/usb/dwc3/gadget.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/usb/dwc3/gadget.c +6
-0@@ -2990,6 +2990,12 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep,break ; } +if (!list_empty (&local)) { +list_splice_tail (&local, &dep->started_list); +/* Return so we don't hit the restart case and loop forever */ +return ; + } +if (!list_empty (&dep->started_list)) goto restart; }Quoted 8 lines
Quoted 1 line
No, we should revert the change for
dwc3_gadget_ep_cleaup_completed_requests(). As I mentioned previously, we don't cleanup the entirestarted_list. If the original problem is due to disconnection in the middle of request completion, then we can just check forpullup_connectedand exit the loop and let thedwc3_remove_requests()do the cleanup.Quoted 7 lines
Ok, sorry, I didn't read your mail in depth until I had this patch sent out. If a revert of
d25d85061bd8is the better fix, I'm fine with that too.thanks
-johnQuoted 6 lines
IMO, we should revert this patch for now since it will cause regression. We can review and test a proper fix at a later time.
Thanks,
ThinhAnother suggestion would just be to replace the loop with a
while()loop and usinglist_entry()instead. That was what was discussed in the earlier patch series which also addresses the problem as well. Issue here is the tmp variable still carries a stale request after the dwc3 giveback is called. We can avoid that by always fetching thelist_entry()instead of relying onlist_for_each_safe()
Quoted 6 lines
Quoted 6 lines
This should work, but the awkward thing is 2 loops from 2 separate threads competing to remove/giveback the requests and may report mix status.
Can you try this?
drivers/usb/dwc3/gadget.c +14 -0
@@ -2029,6 +2029,13 @@ static void dwc3_gadget_ep_cleanup_cancelled_requests(struct dwc3_ep *dep)dwc3_gadget_giveback (dep, req, -ECONNRESET );break ; } + +/* +* The endpoint is disabled, let the dwc3_remove_requests() +* handle the cleanup. +*/ +if (!dep->endpoint.desc) +break ; } }
@@ -3402,6 +3409,13 @@ static void dwc3_gadget_ep_cleanup_completed_requests(struct dwc3_ep *dep, req, status);if (ret)break ; + +/* +* The endpoint is disabled, let the dwc3_remove_requests() +* handle the cleanup. +*/ +if (!dep->endpoint.desc) +break ; } }
If needed, you can also use your change while(!list_empty(started_list)) along with this for future proof.
BR,
Thinh