Skip to contentSkip to conversation

staging: rtl8712: Fix memory leak in r8712_init_recv_priv

PATCH In review Dongliang Mu 2 participants 4 messages May 21, 2021 – May 22, 2021 Staging

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 staging: rtl8712: Fix memory leak in r8712_init_recv_priv

r871xu_dev_remove failed to call r8712_free_drv_sw() and free the resource (e.g., struct urb) due to the failure of firmware loading.

Fix this by invoking r8712_free_drv_sw at the failure site.

Reported-by [email protected]
Fixes b4383c971bc5 ("staging: rtl8712: handle firmware load failure")

---
v1->v2: fix the initialization of pnetdev

 drivers/staging/rtl8712/usb_intf.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/staging/rtl8712/usb_intf.c +7 -2
index dc21e7743349..57e773464e18 100644
@@ -593,13 +593,14 @@ static void r871xu_dev_remove(struct usb_interface *pusb_intf)
 	struct usb_device *udev = interface_to_usbdev(pusb_intf);
 
 	if (pnetdev) {
+		struct net_device *newpnetdev = NULL;
 		struct _adapter *padapter = netdev_priv(pnetdev);
 
 		/* never exit with a firmware callback pending */
 		wait_for_completion(&padapter->rtl8712_fw_ready);
-		pnetdev = usb_get_intfdata(pusb_intf);
+		newpnetdev = usb_get_intfdata(pusb_intf);
 		usb_set_intfdata(pusb_intf, NULL);
-		if (!pnetdev)
+		if (!newpnetdev)
 			goto firmware_load_fail;
 		release_firmware(padapter->fw);
 		if (drvpriv.drv_registered)
@@ -625,6 +626,10 @@ static void r871xu_dev_remove(struct usb_interface *pusb_intf)
 	 */
 	if (udev->state != USB_STATE_NOTATTACHED)
 		usb_reset_device(udev);
+	if (pnetdev) {
+		struct _adapter *padapter = netdev_priv(pnetdev);
+		r8712_free_drv_sw(padapter);
+	}
 }
 
 static int __init r8712u_drv_entry(void)
Signature · 1 line
2.25.1
author
#1 lore raw
Quoted 48 lines · Dongliang Mu

r871xu_dev_remove failed to call r8712_free_drv_sw() and free the resource (e.g., struct urb) due to the failure of firmware loading.

Fix this by invoking r8712_free_drv_sw at the failure site.

Reported-by [email protected]
Fixes b4383c971bc5 ("staging: rtl8712: handle firmware load failure")

---
v1->v2: fix the initialization of pnetdev

 drivers/staging/rtl8712/usb_intf.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/staging/rtl8712/usb_intf.c +7 -3
index dc21e7743349..57e773464e18 100644
@@ -593,13 +593,14 @@ static void r871xu_dev_remove(struct usb_interface *pusb_intf)
        struct usb_device *udev = interface_to_usbdev(pusb_intf);
 
        if (pnetdev) {
+               struct net_device *newpnetdev = NULL;
                struct _adapter *padapter = netdev_priv(pnetdev);
 
                /* never exit with a firmware callback pending */
                wait_for_completion(&padapter->rtl8712_fw_ready);
-               pnetdev = usb_get_intfdata(pusb_intf);
+               newpnetdev = usb_get_intfdata(pusb_intf);
                usb_set_intfdata(pusb_intf, NULL);
-               if (!pnetdev)
+               if (!newpnetdev)
                        goto firmware_load_fail;
                release_firmware(padapter->fw);
                if (drvpriv.drv_registered)
@@ -625,6 +626,10 @@ static void r871xu_dev_remove(struct usb_interface *pusb_intf)
         */
        if (udev->state != USB_STATE_NOTATTACHED)
                usb_reset_device(udev);
+       if (pnetdev) {
+               struct _adapter *padapter = netdev_priv(pnetdev);
+               r8712_free_drv_sw(padapter);
+       }
 }
 
 static int __init r8712u_drv_entry(void)
--
Quoted 2 lines

2.25.1

I tested my patch in my local workspace. No matter for the latest upstream or f40ddce88593482919761f74910f42f4b84c004b (the kernel version for the first crash in the syzbot), my local workspace both show the memory leak disappears once the kernel is patched. However, syzbot testing shows it still triggers the memory leak. @Dmitry Vyukov Can you please help take a look? Maybe I incorrectly use this feature. Thanks in advance.

maintainer
#2 lore raw
In reply to Dongliang Mu
Quoted 19 lines · Dongliang Mu

r871xu_dev_remove failed to call r8712_free_drv_sw() and free the resource (e.g., struct urb) due to the failure of firmware loading.

Fix this by invoking r8712_free_drv_sw at the failure site.

Reported-by [email protected]
Fixes b4383c971bc5 ("staging: rtl8712: handle firmware load failure")

---
v1->v2: fix the initialization of pnetdev

 drivers/staging/rtl8712/usb_intf.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/staging/rtl8712/usb_intf.c +0 -0
index dc21e7743349..57e773464e18 100644

drivers/staging/rtl8712/usb_intf.c static void r871xu_dev_remove(struct usb_interface *pusb_intf) · line 601

Quoted 10 lines
drivers/staging/rtl8712/usb_intf.c +2 -1
@@ -593,13 +593,14 @@ static void r871xu_dev_remove(struct usb_interface *pusb_intf)
 	struct usb_device *udev = interface_to_usbdev(pusb_intf);
 
 	if (pnetdev) {
+		struct net_device *newpnetdev = NULL;
 		struct _adapter *padapter = netdev_priv(pnetdev);
 
 		/* never exit with a firmware callback pending */
 		wait_for_completion(&padapter->rtl8712_fw_ready);
-		pnetdev = usb_get_intfdata(pusb_intf);
+		newpnetdev = usb_get_intfdata(pusb_intf);

Please learn a bit more C knowledge before messing around in the kernel tree. Between the mistake in this chunk you added, and the mistake in the previous submission, I think you need more experience here as there are some things you still need to learn.

Best of luck!

greg k-h

author
#3 lore raw
Quoted 27 lines · Greg KH
Quoted 25 lines · Dongliang Mu

r871xu_dev_remove failed to call r8712_free_drv_sw() and free the resource (e.g., struct urb) due to the failure of firmware loading.

Fix this by invoking r8712_free_drv_sw at the failure site.

Reported-by [email protected]
Fixes b4383c971bc5 ("staging: rtl8712: handle firmware load failure")

---
v1->v2: fix the initialization of pnetdev

 drivers/staging/rtl8712/usb_intf.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/staging/rtl8712/usb_intf.c +1 -0
index dc21e7743349..57e773464e18 100644
@@ -593,13 +593,14 @@ static void r871xu_dev_remove(struct usb_interface *pusb_intf)
      struct usb_device *udev = interface_to_usbdev(pusb_intf);
 
      if (pnetdev) {
+             struct net_device *newpnetdev = NULL;
              struct _adapter *padapter = netdev_priv(pnetdev);
 
Quoted 9 lines

drivers/staging/rtl8712/usb_intf.c static void r871xu_dev_remove(struct usb_interface *pusb_intf) · line 601

Quoted 4 lines
drivers/staging/rtl8712/usb_intf.c +1 -1
              /* never exit with a firmware callback pending */
              wait_for_completion(&padapter->rtl8712_fw_ready);
-             pnetdev = usb_get_intfdata(pusb_intf);
+             newpnetdev = usb_get_intfdata(pusb_intf);

Please learn a bit more C knowledge before messing around in the kernel tree. Between the mistake in this chunk you added, and the mistake in the previous submission, I think you need more experience here as there are some things you still need to learn.

I am a kernel newbie in developing patches for Linux kernel. Sorry for my naive mistakes. I will learn more and then submit some "good" patches.

BTW, I think I have pointed out the underlying bug here, you guys can fix it yourself.

Quoted 4 lines

Best of luck!

greg k-h