Skip to contentSkip to conversation

rtw88: BUG: scheduling while atomic: kworker/u16:0/33416/0x00000002

DISCUSSION Tobias S. Predel 4 participants 9 messages Apr 21, 2020 – Apr 23, 2020 Realtek wireless driver (rtw88)

lore
Share & tools
lore
#0 lore raw

Hi,

I'm using linux-next 20200421 on Arch Linux
with rtw88 for

$ lspci -k

Subsystem: Hewlett-Packard Company Realtek RTL8822BE 802.11ac 2 × 2 Wi-Fi + Bluetooth 4.2 Combo Adapter (MU-MIMO supported)
Kernel driver in use: rtw_pci
Kernel modules: rtwpci

and

$ ethtool -i wlp2s0

driver: rtw_pci
version: 5.7.0-rc2-next-20200421-1-next-
firmware-version: N/A
expansion-rom-version:
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

and I'm experiencing this

$ dmesg
[26628.497854] rtw_pci 0000:02:00.0: firmware failed to restore hardware setting
[26663.488394] rtw_pci 0000:02:00.0: firmware failed to restore hardware setting
[26973.646044] rtw_pci 0000:02:00.0: firmware failed to restore hardware setting
[26981.542221] rtw_pci 0000:02:00.0: firmware failed to restore hardware setting
[27047.668256] rtw_pci 0000:02:00.0: firmware failed to restore hardware setting
[27095.665971] rtw_pci 0000:02:00.0: firmware failed to restore hardware setting
[27131.504263] rtw_pci 0000:02:00.0: firmware failed to restore hardware setting

[and a lot more of this messages]

and following non-fatal bug

[28125.482259] BUG: scheduling while atomic: kworker/u16:0/33416/0x00000002
[28125.482266] Modules linked in: nfnetlink xt_conntrack xt_tcpudp wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 poly1305_x86_64 libblake2s blake2s_x86_64 ip6_udp_tunnel udp_tunnel libcurve25519_generic libchacha libblake2s_generic iptable_security iptable_raw iptable_mangle iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c dm_crypt dm_mod rfcomm fuse 8021q garp mrp stp llc ccm ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cmac algif_hash algif_skcipher af_alg snd_hda_codec_hdmi snd_hda_codec_conexant snd_hda_codec_generic ledtrig_audio bnep x86_pkg_temp_thermal btusb intel_powerclamp joydev btrtl btbcm btintel coretemp mousedev kvm_intel bluetooth uas kvm hid_multitouch usb_storage ecdh_generic iTCO_wdt uvcvideo ecc mei_hdcp hid_generic iTCO_vendor_support hp_wmi videobuf2_vmalloc irqbypass intel_rapl_msr usbhid sparse_keymap wmi_bmof snd_soc_skl videobuf2_memops videobuf2_v4l2 videobuf2_common crct10dif_pclmul videodev
[28125.482336]  snd_soc_sst_ipc crc32_pclmul snd_soc_sst_dsp ghash_clmulni_intel snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi mc aesni_intel snd_soc_core crypto_simd snd_compress ac97_bus rtwpci cryptd snd_pcm_dmaengine glue_helper nls_iso8859_1 rtw88 snd_hda_intel intel_cstate nls_cp437 snd_intel_dspcfg intel_uncore vfat mac80211 fat i915 snd_hda_codec intel_rapl_perf snd_hda_core snd_hwdep snd_pcm snd_timer pcspkr psmouse snd input_leds i2c_i801 cfg80211 soundcore r8169 realtek i2c_algo_bit rfkill libarc4 libphy drm_kms_helper mei_me cec mei rc_core intel_lpss_pci intel_lpss intel_gtt idma64 syscopyarea sysfillrect processor_thermal_device intel_xhci_usb_role_switch intel_pch_thermal sysimgblt roles intel_rapl_common fb_sys_fops intel_soc_dts_iosf i2c_hid tpm_crb wmi battery hid tpm_tis tpm_tis_core tpm int3400_thermal hp_accel int3403_thermal int340x_thermal_zone acpi_thermal_rel lis3lv02d hp_wireless evdev rng_core ac mac_hid drm crypto_user agpgart ip_tables x_tables ext4
[28125.482418]  crc32c_generic crc16 mbcache jbd2 rtsx_pci_sdmmc mmc_core serio_raw atkbd libps2 xhci_pci crc32c_intel xhci_hcd rtsx_pci i8042 serio
[28125.482436] Preemption disabled at:
[28125.482443] [<0000000000000000>] 0x0
[28125.482452] CPU: 5 PID: 33416 Comm: kworker/u16:0 Tainted: G        W         5.7.0-rc2-next-20200421-1-next-git #1
[28125.482456] Hardware name: HP HP ProBook 430 G5/8377, BIOS Q85 Ver. 01.09.01 10/15/2019
[28125.482477] Workqueue: phy0 rtw_watch_dog_work [rtw88]
[28125.482481] Call Trace:
[28125.482495]  dump_stack+0x66/0x90
[28125.482505]  __schedule_bug.cold+0x8e/0x9b
[28125.482512]  __schedule+0x686/0x7b0
[28125.482520]  ? _raw_spin_unlock_irqrestore+0x20/0x40
[28125.482525]  schedule+0x46/0xf0
[28125.482531]  schedule_hrtimeout_range_clock+0xa5/0x120
[28125.482540]  ? hrtimer_init_sleeper+0xa0/0xa0
[28125.482546]  usleep_range+0x67/0x90
[28125.482568]  rtw_fw_send_h2c_command+0xe0/0x1a0 [rtw88]
[28125.482590]  rtw_fw_set_pwr_mode+0x95/0xb0 [rtw88]
[28125.482610]  rtw_enter_lps+0xa1/0x100 [rtw88]
[28125.482625]  rtw_watch_dog_work+0x21c/0x230 [rtw88]
[28125.482635]  process_one_work+0x1da/0x3d0
[28125.482643]  worker_thread+0x4a/0x3d0
[28125.482651]  kthread+0x122/0x160
[28125.482658]  ? process_one_work+0x3d0/0x3d0
[28125.482663]  ? kthread_park+0x90/0x90
[28125.482670]  ret_from_fork+0x1f/0x40
[28169.561821] rtw_pci 0000:02:00.0: firmware failed to restore hardware setting
[28231.427898] rtw_pci 0000:02:00.0: firmware failed to restore hardware setting
[28299.480530] rtw_pci 0000:02:00.0: firmware failed to restore hardware setting

Maybe that's interesting for you.

Kind regards,
Tobias Predel

Signature · 1 line
Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail ausdrucken.
#1 lore raw

I'm not sure about the first half your problem, but for the scheduling-while-atomic:

Quoted 1 line · Tobias S. Predel
[28125.482259] BUG: scheduling while atomic: kworker/u16:0/33416/0x00000002

...

Quoted 2 lines
[28125.482436] Preemption disabled at:
[28125.482443] [<0000000000000000>] 0x0

^^ This line is a bit weird -- shouldn't this have a real PC?

Quoted 14 lines
[28125.482452] CPU: 5 PID: 33416 Comm: kworker/u16:0 Tainted: G        W         5.7.0-rc2-next-20200421-1-next-git #1
[28125.482456] Hardware name: HP HP ProBook 430 G5/8377, BIOS Q85 Ver. 01.09.01 10/15/2019
[28125.482477] Workqueue: phy0 rtw_watch_dog_work [rtw88]
[28125.482481] Call Trace:
[28125.482495]  dump_stack+0x66/0x90
[28125.482505]  __schedule_bug.cold+0x8e/0x9b
[28125.482512]  __schedule+0x686/0x7b0
[28125.482520]  ? _raw_spin_unlock_irqrestore+0x20/0x40
[28125.482525]  schedule+0x46/0xf0
[28125.482531]  schedule_hrtimeout_range_clock+0xa5/0x120
[28125.482540]  ? hrtimer_init_sleeper+0xa0/0xa0
[28125.482546]  usleep_range+0x67/0x90
[28125.482568]  rtw_fw_send_h2c_command+0xe0/0x1a0 [rtw88]
[28125.482590]  rtw_fw_set_pwr_mode+0x95/0xb0 [rtw88]
Quoted 8 lines
[28125.482610]  rtw_enter_lps+0xa1/0x100 [rtw88]
[28125.482625]  rtw_watch_dog_work+0x21c/0x230 [rtw88]
[28125.482635]  process_one_work+0x1da/0x3d0
[28125.482643]  worker_thread+0x4a/0x3d0
[28125.482651]  kthread+0x122/0x160
[28125.482658]  ? process_one_work+0x3d0/0x3d0
[28125.482663]  ? kthread_park+0x90/0x90
[28125.482670]  ret_from_fork+0x1f/0x40

This looks like it might be a regression here:

commit 6343a6d4b2130be9323f347d60af8a7ba8f7242c

Author: Kai-Heng Feng <[email protected]>
Date:   Tue Apr 7 15:33:31 2020 +0800

rtw88: Add delay on polling h2c command status bit

That poll macros is using usleep, which obviously can sleep. We need to be using a udelay-variant instead.

Brian

#2 lore raw

Brian Norris <[email protected]> :

Quoted 41 lines

I'm not sure about the first half your problem, but for the scheduling-while-atomic:

Quoted 1 line · Tobias S. Predel
[28125.482259] BUG: scheduling while atomic:

kworker/u16:0/33416/0x00000002
...

Quoted 2 lines
[28125.482436] Preemption disabled at:
[28125.482443] [<0000000000000000>] 0x0

^^ This line is a bit weird -- shouldn't this have a real PC?

Quoted 1 line
[28125.482452] CPU: 5 PID: 33416 Comm: kworker/u16:0 Tainted: G

W 5.7.0-rc2-next-20200421-1-next-git #1

Quoted 1 line
[28125.482456] Hardware name: HP HP ProBook 430 G5/8377, BIOS Q85

Ver. 01.09.01 10/15/2019

Quoted 12 lines
[28125.482477] Workqueue: phy0 rtw_watch_dog_work [rtw88]
[28125.482481] Call Trace:
[28125.482495]  dump_stack+0x66/0x90
[28125.482505]  __schedule_bug.cold+0x8e/0x9b
[28125.482512]  __schedule+0x686/0x7b0
[28125.482520]  ? _raw_spin_unlock_irqrestore+0x20/0x40
[28125.482525]  schedule+0x46/0xf0
[28125.482531]  schedule_hrtimeout_range_clock+0xa5/0x120
[28125.482540]  ? hrtimer_init_sleeper+0xa0/0xa0
[28125.482546]  usleep_range+0x67/0x90
[28125.482568]  rtw_fw_send_h2c_command+0xe0/0x1a0 [rtw88]
[28125.482590]  rtw_fw_set_pwr_mode+0x95/0xb0 [rtw88]
Quoted 8 lines
[28125.482610]  rtw_enter_lps+0xa1/0x100 [rtw88]
[28125.482625]  rtw_watch_dog_work+0x21c/0x230 [rtw88]
[28125.482635]  process_one_work+0x1da/0x3d0
[28125.482643]  worker_thread+0x4a/0x3d0
[28125.482651]  kthread+0x122/0x160
[28125.482658]  ? process_one_work+0x3d0/0x3d0
[28125.482663]  ? kthread_park+0x90/0x90
[28125.482670]  ret_from_fork+0x1f/0x40

This looks like it might be a regression here:

Quoted 9 lines

commit 6343a6d4b2130be9323f347d60af8a7ba8f7242c

Author: Kai-Heng Feng <[email protected]>
Date:   Tue Apr 7 15:33:31 2020 +0800

rtw88: Add delay on polling h2c command status bit

That poll macros is using usleep, which obviously can sleep. We need to be using a udelay-variant instead.

Maybe we need an atomic version of read_poll_timeout() ?
I am not sure if this is required, but seems like it is useful for me. Noticed much of them have its atomic version, but not for this new added one.

Yen-Hsuan

#3 lore raw

Hi Tobias,

Quoted 50 lines · Tony Chuang

Brian Norris <[email protected]> :

Quoted 47 lines

I'm not sure about the first half your problem, but for the scheduling-while-atomic:

Quoted 1 line · Tobias S. Predel
[28125.482259] BUG: scheduling while atomic:

kworker/u16:0/33416/0x00000002
...

Quoted 2 lines
[28125.482436] Preemption disabled at:
[28125.482443] [<0000000000000000>] 0x0

^^ This line is a bit weird -- shouldn't this have a real PC?

Quoted 1 line
[28125.482452] CPU: 5 PID: 33416 Comm: kworker/u16:0 Tainted: G

W 5.7.0-rc2-next-20200421-1-next-git #1

Quoted 1 line
[28125.482456] Hardware name: HP HP ProBook 430 G5/8377, BIOS Q85

Ver. 01.09.01 10/15/2019

Quoted 12 lines
[28125.482477] Workqueue: phy0 rtw_watch_dog_work [rtw88]
[28125.482481] Call Trace:
[28125.482495]  dump_stack+0x66/0x90
[28125.482505]  __schedule_bug.cold+0x8e/0x9b
[28125.482512]  __schedule+0x686/0x7b0
[28125.482520]  ? _raw_spin_unlock_irqrestore+0x20/0x40
[28125.482525]  schedule+0x46/0xf0
[28125.482531]  schedule_hrtimeout_range_clock+0xa5/0x120
[28125.482540]  ? hrtimer_init_sleeper+0xa0/0xa0
[28125.482546]  usleep_range+0x67/0x90
[28125.482568]  rtw_fw_send_h2c_command+0xe0/0x1a0 [rtw88]
[28125.482590]  rtw_fw_set_pwr_mode+0x95/0xb0 [rtw88]
Quoted 8 lines
[28125.482610]  rtw_enter_lps+0xa1/0x100 [rtw88]
[28125.482625]  rtw_watch_dog_work+0x21c/0x230 [rtw88]
[28125.482635]  process_one_work+0x1da/0x3d0
[28125.482643]  worker_thread+0x4a/0x3d0
[28125.482651]  kthread+0x122/0x160
[28125.482658]  ? process_one_work+0x3d0/0x3d0
[28125.482663]  ? kthread_park+0x90/0x90
[28125.482670]  ret_from_fork+0x1f/0x40

This looks like it might be a regression here:

commit 6343a6d4b2130be9323f347d60af8a7ba8f7242c

Author: Kai-Heng Feng <[email protected]>
Date:   Tue Apr 7 15:33:31 2020 +0800

rtw88: Add delay on polling h2c command status bit

Quoted 7 lines
Quoted 3 lines

That poll macros is using usleep, which obviously can sleep. We need to be using a udelay-variant instead.

Maybe we need an atomic version of read_poll_timeout() ?
I am not sure if this is required, but seems like it is useful for me. Noticed much of them have its atomic version, but not for this new added one.

Tony and Brian are right.

Tobias, can you please test the following patch:

drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
                goto out;
        }
 
-       ret = read_poll_timeout(rtw_read8, box_state,
+       ret = read_poll_timeout_atomic(rtw_read8, box_state,
                                !((box_state >> box) & 0x1), 100, 3000, false,
                                rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +42 -19

73 lines not on this page · raw message

Quoted 2 lines

Yen-Hsuan

#4 lore raw

Hi Kai-Heng,

Quoted 147 lines · Kai-Heng Feng

Hi Tobias,

Quoted 50 lines · Tony Chuang

Brian Norris <[email protected]> :

Quoted 47 lines

I'm not sure about the first half your problem, but for the scheduling-while-atomic:

Quoted 1 line · Tobias S. Predel
[28125.482259] BUG: scheduling while atomic:

kworker/u16:0/33416/0x00000002
...

Quoted 2 lines
[28125.482436] Preemption disabled at:
[28125.482443] [<0000000000000000>] 0x0

^^ This line is a bit weird -- shouldn't this have a real PC?

Quoted 1 line
[28125.482452] CPU: 5 PID: 33416 Comm: kworker/u16:0 Tainted: G

W 5.7.0-rc2-next-20200421-1-next-git #1

Quoted 1 line
[28125.482456] Hardware name: HP HP ProBook 430 G5/8377, BIOS Q85

Ver. 01.09.01 10/15/2019

Quoted 12 lines
[28125.482477] Workqueue: phy0 rtw_watch_dog_work [rtw88]
[28125.482481] Call Trace:
[28125.482495]  dump_stack+0x66/0x90
[28125.482505]  __schedule_bug.cold+0x8e/0x9b
[28125.482512]  __schedule+0x686/0x7b0
[28125.482520]  ? _raw_spin_unlock_irqrestore+0x20/0x40
[28125.482525]  schedule+0x46/0xf0
[28125.482531]  schedule_hrtimeout_range_clock+0xa5/0x120
[28125.482540]  ? hrtimer_init_sleeper+0xa0/0xa0
[28125.482546]  usleep_range+0x67/0x90
[28125.482568]  rtw_fw_send_h2c_command+0xe0/0x1a0 [rtw88]
[28125.482590]  rtw_fw_set_pwr_mode+0x95/0xb0 [rtw88]
Quoted 8 lines
[28125.482610]  rtw_enter_lps+0xa1/0x100 [rtw88]
[28125.482625]  rtw_watch_dog_work+0x21c/0x230 [rtw88]
[28125.482635]  process_one_work+0x1da/0x3d0
[28125.482643]  worker_thread+0x4a/0x3d0
[28125.482651]  kthread+0x122/0x160
[28125.482658]  ? process_one_work+0x3d0/0x3d0
[28125.482663]  ? kthread_park+0x90/0x90
[28125.482670]  ret_from_fork+0x1f/0x40

This looks like it might be a regression here:

commit 6343a6d4b2130be9323f347d60af8a7ba8f7242c

Author: Kai-Heng Feng <[email protected]>
Date:   Tue Apr 7 15:33:31 2020 +0800

rtw88: Add delay on polling h2c command status bit

Quoted 7 lines
Quoted 3 lines

That poll macros is using usleep, which obviously can sleep. We need to be using a udelay-variant instead.

Maybe we need an atomic version of read_poll_timeout() ?
I am not sure if this is required, but seems like it is useful for me. Noticed much of them have its atomic version, but not for this new added one.

Tony and Brian are right.

Tobias, can you please test the following patch:

drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
                goto out;
        }
 
-       ret = read_poll_timeout(rtw_read8, box_state,
+       ret = read_poll_timeout_atomic(rtw_read8, box_state,
                                !((box_state >> box) & 0x1), 100, 3000, false,
                                rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +41 -13
index cb20c733b15a..bc89ac625f26 100644
@@ -57,6 +57,48 @@
        (cond) ? 0 : -ETIMEDOUT; \
 })
 
+/**
+ * read_poll_timeout_atomic - Periodically poll an address until a condition is
+ *                             met or a timeout occurs
+ * @op: accessor function (takes @addr as its only argument)
+ * @addr: Address to poll
+ * @val: Variable to read the value into
+ * @cond: Break condition (usually involving @val)
+ * @delay_us: Time to udelay between reads in us (0 tight-loops).  Should
+ *            be less than ~10us since udelay is used (see
+ *            Documentation/timers/timers-howto.rst).
+ * @timeout_us: Timeout in us, 0 means never timeout
+ * @delay_before_read: if it is true, delay @delay_us before read.
+ *
+ * Returns 0 on success and -ETIMEDOUT upon a timeout. In either
+ * case, the last read value at @args is stored in @val.
+ *
+ * When available, you'll probably want to use one of the specialized
+ * macros defined below rather than this macro directly.
+ */
+#define read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, \
+                                       delay_before_read, args...) \
+({ \
+       u64 __timeout_us = (timeout_us); \
+       unsigned long __delay_us = (delay_us); \
+       ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
+       if (delay_before_read && __delay_us) \
+               udelay(__delay_us); \
+       for (;;) { \
+               (val) = op(args); \
+               if (cond) \
+                       break; \
+               if (__timeout_us && \
+                   ktime_compare(ktime_get(), __timeout) > 0) { \
+                       (val) = op(args); \
+                       break; \
+               } \
+               if (__delay_us) \
+       } \
+       (cond) ? 0 : -ETIMEDOUT; \
+})
+
 /**
  * readx_poll_timeout - Periodically poll an address until a condition is met or a timeout occurs
  * @op: accessor function (takes @addr as its only argument)
@@ -96,25 +138,7 @@
  * macros defined below rather than this macro directly.
  */
 #define readx_poll_timeout_atomic(op, addr, val, cond, delay_us, timeout_us) \
-({ \
-       u64 __timeout_us = (timeout_us); \
-       unsigned long __delay_us = (delay_us); \
-       ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
-       for (;;) { \
-               (val) = op(addr); \
-               if (cond) \
-                       break; \
-               if (__timeout_us && \
-                   ktime_compare(ktime_get(), __timeout) > 0) { \
-                       (val) = op(addr); \
-                       break; \
-               } \

include/linux/iopoll.h line 144

Quoted 10 lines
include/linux/iopoll.h +1 -6
-               if (__delay_us) \
-                       udelay(__delay_us);     \
-       } \
-       (cond) ? 0 : -ETIMEDOUT; \
-})
-
+       read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, false, addr)
 
 #define readb_poll_timeout(addr, val, cond, delay_us, timeout_us) \
        readx_poll_timeout(readb, addr, val, cond, delay_us, timeout_us)

thanks for your reply!

I tried to patch it against

$ git describe --long --tags
next-20200422-0-ga5840f9618a9

with

$ git am < ~/atomic-patch.mbox

but I get

Applying: rtw88: BUG: scheduling while atomic: kworker/u16:0/33416/0x00000002
error: corrupt patch at line 66
Patch failed at 0001 rtw88: BUG: scheduling while atomic: kworker/u16:0/33416/0x00000002
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

and

in am mode, then

$ git am --show-current-patch=diff
drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
                goto out;
        }
 
-       ret = read_poll_timeout(rtw_read8, box_state,
+       ret = read_poll_timeout_atomic(rtw_read8, box_state,
                                !((box_state >> box) & 0x1), 100, 3000, false,
                                rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +42 -19

73 lines not on this page · raw message

Quoted 2 lines

Yen-Hsuan

I'll try to patch manually and send back when there are news.

Kind regards,
Tobias

Signature · 1 line
Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail ausdrucken.
#5 lore raw

Hi Kai-Heng,

Quoted 117 lines · Tobias S. Predel

Hi Kai-Heng,

Quoted 114 lines · Kai-Heng Feng

Hi Tobias,

Quoted 50 lines · Tony Chuang

Brian Norris <[email protected]> :

Quoted 47 lines

I'm not sure about the first half your problem, but for the scheduling-while-atomic:

Quoted 1 line · Tobias S. Predel
[28125.482259] BUG: scheduling while atomic:

kworker/u16:0/33416/0x00000002
...

Quoted 2 lines
[28125.482436] Preemption disabled at:
[28125.482443] [<0000000000000000>] 0x0

^^ This line is a bit weird -- shouldn't this have a real PC?

Quoted 1 line
[28125.482452] CPU: 5 PID: 33416 Comm: kworker/u16:0 Tainted: G

W 5.7.0-rc2-next-20200421-1-next-git #1

Quoted 1 line
[28125.482456] Hardware name: HP HP ProBook 430 G5/8377, BIOS Q85

Ver. 01.09.01 10/15/2019

Quoted 12 lines
[28125.482477] Workqueue: phy0 rtw_watch_dog_work [rtw88]
[28125.482481] Call Trace:
[28125.482495]  dump_stack+0x66/0x90
[28125.482505]  __schedule_bug.cold+0x8e/0x9b
[28125.482512]  __schedule+0x686/0x7b0
[28125.482520]  ? _raw_spin_unlock_irqrestore+0x20/0x40
[28125.482525]  schedule+0x46/0xf0
[28125.482531]  schedule_hrtimeout_range_clock+0xa5/0x120
[28125.482540]  ? hrtimer_init_sleeper+0xa0/0xa0
[28125.482546]  usleep_range+0x67/0x90
[28125.482568]  rtw_fw_send_h2c_command+0xe0/0x1a0 [rtw88]
[28125.482590]  rtw_fw_set_pwr_mode+0x95/0xb0 [rtw88]
Quoted 8 lines
[28125.482610]  rtw_enter_lps+0xa1/0x100 [rtw88]
[28125.482625]  rtw_watch_dog_work+0x21c/0x230 [rtw88]
[28125.482635]  process_one_work+0x1da/0x3d0
[28125.482643]  worker_thread+0x4a/0x3d0
[28125.482651]  kthread+0x122/0x160
[28125.482658]  ? process_one_work+0x3d0/0x3d0
[28125.482663]  ? kthread_park+0x90/0x90
[28125.482670]  ret_from_fork+0x1f/0x40

This looks like it might be a regression here:

commit 6343a6d4b2130be9323f347d60af8a7ba8f7242c

Author: Kai-Heng Feng <[email protected]>
Date:   Tue Apr 7 15:33:31 2020 +0800

rtw88: Add delay on polling h2c command status bit

Quoted 7 lines
Quoted 3 lines

That poll macros is using usleep, which obviously can sleep. We need to be using a udelay-variant instead.

Maybe we need an atomic version of read_poll_timeout() ?
I am not sure if this is required, but seems like it is useful for me. Noticed much of them have its atomic version, but not for this new added one.

Tony and Brian are right.

Tobias, can you please test the following patch:

drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
                goto out;
        }
 
-       ret = read_poll_timeout(rtw_read8, box_state,
+       ret = read_poll_timeout_atomic(rtw_read8, box_state,
                                !((box_state >> box) & 0x1), 100, 3000, false,
                                rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +29 -0
index cb20c733b15a..bc89ac625f26 100644
@@ -57,6 +57,48 @@
        (cond) ? 0 : -ETIMEDOUT; \
 })
 
+/**
+ * read_poll_timeout_atomic - Periodically poll an address until a condition is
+ *                             met or a timeout occurs
+ * @op: accessor function (takes @addr as its only argument)
+ * @addr: Address to poll
+ * @val: Variable to read the value into
+ * @cond: Break condition (usually involving @val)
+ * @delay_us: Time to udelay between reads in us (0 tight-loops).  Should
+ *            be less than ~10us since udelay is used (see
+ *            Documentation/timers/timers-howto.rst).
+ * @timeout_us: Timeout in us, 0 means never timeout
+ * @delay_before_read: if it is true, delay @delay_us before read.
+ *
+ * Returns 0 on success and -ETIMEDOUT upon a timeout. In either
+ * case, the last read value at @args is stored in @val.
+ *
+ * When available, you'll probably want to use one of the specialized
+ * macros defined below rather than this macro directly.
+ */
+#define read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, \
+                                       delay_before_read, args...) \
+({ \
+       u64 __timeout_us = (timeout_us); \
+       unsigned long __delay_us = (delay_us); \
+       ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
+       if (delay_before_read && __delay_us) \
+               udelay(__delay_us); \
+       for (;;) { \
+               (val) = op(args); \
Quoted 8 lines

include/linux/iopoll.h line 96

Quoted 8 lines
include/linux/iopoll.h +8 -0
+               if (cond) \
+                       break; \
+               if (__timeout_us && \
+                   ktime_compare(ktime_get(), __timeout) > 0) { \
+                       (val) = op(args); \
+                       break; \
+               } \
+               if (__delay_us) \

Isn't there something missing here after __delay_us? I got compiler error, misses ;.

Quoted 36 lines
Quoted 35 lines
include/linux/iopoll.h +5 -19
+       } \
+       (cond) ? 0 : -ETIMEDOUT; \
+})
+
 /**
  * readx_poll_timeout - Periodically poll an address until a condition is met or a timeout occurs
  * @op: accessor function (takes @addr as its only argument)
@@ -96,25 +138,7 @@
  * macros defined below rather than this macro directly.
  */
 #define readx_poll_timeout_atomic(op, addr, val, cond, delay_us, timeout_us) \
-({ \
-       u64 __timeout_us = (timeout_us); \
-       unsigned long __delay_us = (delay_us); \
-       ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
-       for (;;) { \
-               (val) = op(addr); \
-               if (cond) \
-                       break; \
-               if (__timeout_us && \
-                   ktime_compare(ktime_get(), __timeout) > 0) { \
-                       (val) = op(addr); \
-                       break; \
-               } \
-               if (__delay_us) \
-                       udelay(__delay_us);     \
-       } \
-       (cond) ? 0 : -ETIMEDOUT; \
-})
-
+       read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, false, addr)
 
 #define readb_poll_timeout(addr, val, cond, delay_us, timeout_us) \
        readx_poll_timeout(readb, addr, val, cond, delay_us, timeout_us)
Signature · 1 line
Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail ausdrucken.
#6 lore raw

Hello,

I guessed from the deleted lines that
udelay(__delay_us) was missing.

So I compiled successfully and will observe how it will work out. But for the first start on the patched kernel it's working and I will report.

Thanks for your patch! Patch against commit a5840f9618a90ecbe1617f7632482563c0ee307e is attached.

Kind regards,
Tobias

drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
 		goto out;
 	}
 
-	ret = read_poll_timeout(rtw_read8, box_state,
+	ret = read_poll_timeout_atomic(rtw_read8, box_state,
 				!((box_state >> box) & 0x1), 100, 3000, false,
 				rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +44 -19

75 lines not on this page · raw message

Quoted 171 lines · Tobias S. Predel

Hi Kai-Heng,

Quoted 117 lines · Tobias S. Predel

Hi Kai-Heng,

Quoted 114 lines · Kai-Heng Feng

Hi Tobias,

Quoted 50 lines · Tony Chuang

Brian Norris <[email protected]> :

Quoted 47 lines

I'm not sure about the first half your problem, but for the scheduling-while-atomic:

Quoted 1 line · Tobias S. Predel
[28125.482259] BUG: scheduling while atomic:

kworker/u16:0/33416/0x00000002
...

Quoted 2 lines
[28125.482436] Preemption disabled at:
[28125.482443] [<0000000000000000>] 0x0

^^ This line is a bit weird -- shouldn't this have a real PC?

Quoted 1 line
[28125.482452] CPU: 5 PID: 33416 Comm: kworker/u16:0 Tainted: G

W 5.7.0-rc2-next-20200421-1-next-git #1

Quoted 1 line
[28125.482456] Hardware name: HP HP ProBook 430 G5/8377, BIOS Q85

Ver. 01.09.01 10/15/2019

Quoted 12 lines
[28125.482477] Workqueue: phy0 rtw_watch_dog_work [rtw88]
[28125.482481] Call Trace:
[28125.482495]  dump_stack+0x66/0x90
[28125.482505]  __schedule_bug.cold+0x8e/0x9b
[28125.482512]  __schedule+0x686/0x7b0
[28125.482520]  ? _raw_spin_unlock_irqrestore+0x20/0x40
[28125.482525]  schedule+0x46/0xf0
[28125.482531]  schedule_hrtimeout_range_clock+0xa5/0x120
[28125.482540]  ? hrtimer_init_sleeper+0xa0/0xa0
[28125.482546]  usleep_range+0x67/0x90
[28125.482568]  rtw_fw_send_h2c_command+0xe0/0x1a0 [rtw88]
[28125.482590]  rtw_fw_set_pwr_mode+0x95/0xb0 [rtw88]
Quoted 8 lines
[28125.482610]  rtw_enter_lps+0xa1/0x100 [rtw88]
[28125.482625]  rtw_watch_dog_work+0x21c/0x230 [rtw88]
[28125.482635]  process_one_work+0x1da/0x3d0
[28125.482643]  worker_thread+0x4a/0x3d0
[28125.482651]  kthread+0x122/0x160
[28125.482658]  ? process_one_work+0x3d0/0x3d0
[28125.482663]  ? kthread_park+0x90/0x90
[28125.482670]  ret_from_fork+0x1f/0x40

This looks like it might be a regression here:

commit 6343a6d4b2130be9323f347d60af8a7ba8f7242c

Author: Kai-Heng Feng <[email protected]>
Date:   Tue Apr 7 15:33:31 2020 +0800

rtw88: Add delay on polling h2c command status bit

Quoted 7 lines
Quoted 3 lines

That poll macros is using usleep, which obviously can sleep. We need to be using a udelay-variant instead.

Maybe we need an atomic version of read_poll_timeout() ?
I am not sure if this is required, but seems like it is useful for me. Noticed much of them have its atomic version, but not for this new added one.

Tony and Brian are right.

Tobias, can you please test the following patch:

drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
                goto out;
        }
 
-       ret = read_poll_timeout(rtw_read8, box_state,
+       ret = read_poll_timeout_atomic(rtw_read8, box_state,
                                !((box_state >> box) & 0x1), 100, 3000, false,
                                rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +29 -0
index cb20c733b15a..bc89ac625f26 100644
@@ -57,6 +57,48 @@
        (cond) ? 0 : -ETIMEDOUT; \
 })
 
+/**
+ * read_poll_timeout_atomic - Periodically poll an address until a condition is
+ *                             met or a timeout occurs
+ * @op: accessor function (takes @addr as its only argument)
+ * @addr: Address to poll
+ * @val: Variable to read the value into
+ * @cond: Break condition (usually involving @val)
+ * @delay_us: Time to udelay between reads in us (0 tight-loops).  Should
+ *            be less than ~10us since udelay is used (see
+ *            Documentation/timers/timers-howto.rst).
+ * @timeout_us: Timeout in us, 0 means never timeout
+ * @delay_before_read: if it is true, delay @delay_us before read.
+ *
+ * Returns 0 on success and -ETIMEDOUT upon a timeout. In either
+ * case, the last read value at @args is stored in @val.
+ *
+ * When available, you'll probably want to use one of the specialized
+ * macros defined below rather than this macro directly.
+ */
+#define read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, \
+                                       delay_before_read, args...) \
+({ \
+       u64 __timeout_us = (timeout_us); \
+       unsigned long __delay_us = (delay_us); \
+       ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
+       if (delay_before_read && __delay_us) \
+               udelay(__delay_us); \
+       for (;;) { \
+               (val) = op(args); \
Quoted 8 lines

include/linux/iopoll.h line 96

Quoted 8 lines
include/linux/iopoll.h +8 -0
+               if (cond) \
+                       break; \
+               if (__timeout_us && \
+                   ktime_compare(ktime_get(), __timeout) > 0) { \
+                       (val) = op(args); \
+                       break; \
+               } \
+               if (__delay_us) \

Isn't there something missing here after __delay_us? I got compiler error, misses ;.

Quoted 36 lines
Quoted 35 lines
include/linux/iopoll.h +5 -19
+       } \
+       (cond) ? 0 : -ETIMEDOUT; \
+})
+
 /**
  * readx_poll_timeout - Periodically poll an address until a condition is met or a timeout occurs
  * @op: accessor function (takes @addr as its only argument)
@@ -96,25 +138,7 @@
  * macros defined below rather than this macro directly.
  */
 #define readx_poll_timeout_atomic(op, addr, val, cond, delay_us, timeout_us) \
-({ \
-       u64 __timeout_us = (timeout_us); \
-       unsigned long __delay_us = (delay_us); \
-       ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
-       for (;;) { \
-               (val) = op(addr); \
-               if (cond) \
-                       break; \
-               if (__timeout_us && \
-                   ktime_compare(ktime_get(), __timeout) > 0) { \
-                       (val) = op(addr); \
-                       break; \
-               } \
-               if (__delay_us) \
-                       udelay(__delay_us);     \
-       } \
-       (cond) ? 0 : -ETIMEDOUT; \
-})
-
+       read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, false, addr)
 
 #define readb_poll_timeout(addr, val, cond, delay_us, timeout_us) \
        readx_poll_timeout(readb, addr, val, cond, delay_us, timeout_us)
Signature · 1 line
Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail ausdrucken.
Signature · 1 line
Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail ausdrucken.

From 2652ea8d9120b67c79dc85a9073cc5e85d6fc2dc Mon Sep 17 00:00:00 2001
From: Tobias Predel <[email protected]>
Date: Thu, 23 Apr 2020 00:44:15 +0200
Subject: [PATCH] rtw88: Make read_poll_timeout atomic

---
 drivers/net/wireless/realtek/rtw88/fw.c |  2 +-
 include/linux/iopoll.h                  | 63 +++++++++++++++++--------
 2 files changed, 45 insertions(+), 20 deletions(-)
drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
 		goto out;
 	}
 
-	ret = read_poll_timeout(rtw_read8, box_state,
+	ret = read_poll_timeout_atomic(rtw_read8, box_state,
 				!((box_state >> box) & 0x1), 100, 3000, false,
 				rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +44 -19

75 lines not on this page · raw message

Signature · 1 line
2.26.2
#7 lore raw

Hi,

Quoted 6 lines · Tobias S. Predel

Hello,

I guessed from the deleted lines that
udelay(__delay_us) was missing.

Yea I fail to copy that line :(

Quoted 3 lines

So I compiled successfully and will observe how it will work out. But for the first start on the patched kernel it's working and I will report.

Thanks. I'll send the patch.

Kai-Heng

Quoted 280 lines

Thanks for your patch! Patch against commit a5840f9618a90ecbe1617f7632482563c0ee307e is attached.

Kind regards,
Tobias

drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
 		goto out;
 	}
 
-	ret = read_poll_timeout(rtw_read8, box_state,
+	ret = read_poll_timeout_atomic(rtw_read8, box_state,
 				!((box_state >> box) & 0x1), 100, 3000, false,
 				rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +44 -19
index cb20c733b15a..e4462712b541 100644
@@ -57,6 +57,49 @@
 	(cond) ? 0 : -ETIMEDOUT; \
 })
 
+/**
+ * read_poll_timeout_atomic - Periodically poll an address until a condition is
+ *                             met or a timeout occurs
+ * @op: accessor function (takes @addr as its only argument)
+ * @addr: Address to poll
+ * @val: Variable to read the value into
+ * @cond: Break condition (usually involving @val)
+ * @delay_us: Time to udelay between reads in us (0 tight-loops).  Should
+ *            be less than ~10us since udelay is used (see
+ *            Documentation/timers/timers-howto.rst).
+ * @timeout_us: Timeout in us, 0 means never timeout
+ * @delay_before_read: if it is true, delay @delay_us before read.
+ *
+ * Returns 0 on success and -ETIMEDOUT upon a timeout. In either
+ * case, the last read value at @args is stored in @val.
+ *
+ * When available, you'll probably want to use one of the specialized
+ * macros defined below rather than this macro directly.
+ */
+#define read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, \
+				delay_before_read, args...) \
+({ \
+	u64 __timeout_us = (timeout_us); \
+	unsigned long __delay_us = (delay_us); \
+	ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
+	if (delay_before_read && __delay_us) \
+		udelay(__delay_us); \
+	for (;;) { \
+		(val) = op(args); \
+		if (cond) \
+			break; \
+		if (__timeout_us && \
+		    ktime_compare(ktime_get(), __timeout) > 0) { \
+			(val) = op(args); \
+			 break; \
+		 } \
+		 if (__delay_us) \
+			udelay(__delay_us); \
+	} \
+	(cond) ? 0 : -ETIMEDOUT; \
+})
+
+
 /**
 * readx_poll_timeout - Periodically poll an address until a condition is met or a timeout occurs
 * @op: accessor function (takes @addr as its only argument)
@@ -96,25 +139,7 @@
 * macros defined below rather than this macro directly.
 */
 #define readx_poll_timeout_atomic(op, addr, val, cond, delay_us, timeout_us) \
-({ \
-	u64 __timeout_us = (timeout_us); \
-	unsigned long __delay_us = (delay_us); \
-	ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
-	for (;;) { \
-		(val) = op(addr); \
-		if (cond) \
-			break; \
-		if (__timeout_us && \
-		    ktime_compare(ktime_get(), __timeout) > 0) { \
-			(val) = op(addr); \
-			break; \
-		} \
-		if (__delay_us) \
-			udelay(__delay_us);	\
-	} \
-	(cond) ? 0 : -ETIMEDOUT; \
-})
-
+	read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, false, addr)
 
 #define readb_poll_timeout(addr, val, cond, delay_us, timeout_us) \
 	readx_poll_timeout(readb, addr, val, cond, delay_us, timeout_us)
Quoted 171 lines · Tobias S. Predel

Hi Kai-Heng,

Quoted 117 lines · Tobias S. Predel

Hi Kai-Heng,

Quoted 114 lines · Kai-Heng Feng

Hi Tobias,

Quoted 50 lines · Tony Chuang

Brian Norris <[email protected]> :

Quoted 47 lines

I'm not sure about the first half your problem, but for the scheduling-while-atomic:

Quoted 1 line · Tobias S. Predel
[28125.482259] BUG: scheduling while atomic:

kworker/u16:0/33416/0x00000002
...

Quoted 2 lines
[28125.482436] Preemption disabled at:
[28125.482443] [<0000000000000000>] 0x0

^^ This line is a bit weird -- shouldn't this have a real PC?

Quoted 1 line
[28125.482452] CPU: 5 PID: 33416 Comm: kworker/u16:0 Tainted: G

W 5.7.0-rc2-next-20200421-1-next-git #1

Quoted 1 line
[28125.482456] Hardware name: HP HP ProBook 430 G5/8377, BIOS Q85

Ver. 01.09.01 10/15/2019

Quoted 12 lines
[28125.482477] Workqueue: phy0 rtw_watch_dog_work [rtw88]
[28125.482481] Call Trace:
[28125.482495]  dump_stack+0x66/0x90
[28125.482505]  __schedule_bug.cold+0x8e/0x9b
[28125.482512]  __schedule+0x686/0x7b0
[28125.482520]  ? _raw_spin_unlock_irqrestore+0x20/0x40
[28125.482525]  schedule+0x46/0xf0
[28125.482531]  schedule_hrtimeout_range_clock+0xa5/0x120
[28125.482540]  ? hrtimer_init_sleeper+0xa0/0xa0
[28125.482546]  usleep_range+0x67/0x90
[28125.482568]  rtw_fw_send_h2c_command+0xe0/0x1a0 [rtw88]
[28125.482590]  rtw_fw_set_pwr_mode+0x95/0xb0 [rtw88]
Quoted 8 lines
[28125.482610]  rtw_enter_lps+0xa1/0x100 [rtw88]
[28125.482625]  rtw_watch_dog_work+0x21c/0x230 [rtw88]
[28125.482635]  process_one_work+0x1da/0x3d0
[28125.482643]  worker_thread+0x4a/0x3d0
[28125.482651]  kthread+0x122/0x160
[28125.482658]  ? process_one_work+0x3d0/0x3d0
[28125.482663]  ? kthread_park+0x90/0x90
[28125.482670]  ret_from_fork+0x1f/0x40

This looks like it might be a regression here:

commit 6343a6d4b2130be9323f347d60af8a7ba8f7242c

Author: Kai-Heng Feng <[email protected]>
Date:   Tue Apr 7 15:33:31 2020 +0800

rtw88: Add delay on polling h2c command status bit

Quoted 7 lines
Quoted 3 lines

That poll macros is using usleep, which obviously can sleep. We need to be using a udelay-variant instead.

Maybe we need an atomic version of read_poll_timeout() ?
I am not sure if this is required, but seems like it is useful for me. Noticed much of them have its atomic version, but not for this new added one.

Tony and Brian are right.

Tobias, can you please test the following patch:

drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
               goto out;
       }
 
-       ret = read_poll_timeout(rtw_read8, box_state,
+       ret = read_poll_timeout_atomic(rtw_read8, box_state,
                               !((box_state >> box) & 0x1), 100, 3000, false,
                               rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +29 -0
index cb20c733b15a..bc89ac625f26 100644
@@ -57,6 +57,48 @@
       (cond) ? 0 : -ETIMEDOUT; \
 })
 
+/**
+ * read_poll_timeout_atomic - Periodically poll an address until a condition is
+ *                             met or a timeout occurs
+ * @op: accessor function (takes @addr as its only argument)
+ * @addr: Address to poll
+ * @val: Variable to read the value into
+ * @cond: Break condition (usually involving @val)
+ * @delay_us: Time to udelay between reads in us (0 tight-loops).  Should
+ *            be less than ~10us since udelay is used (see
+ *            Documentation/timers/timers-howto.rst).
+ * @timeout_us: Timeout in us, 0 means never timeout
+ * @delay_before_read: if it is true, delay @delay_us before read.
+ *
+ * Returns 0 on success and -ETIMEDOUT upon a timeout. In either
+ * case, the last read value at @args is stored in @val.
+ *
+ * When available, you'll probably want to use one of the specialized
+ * macros defined below rather than this macro directly.
+ */
+#define read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, \
+                                       delay_before_read, args...) \
+({ \
+       u64 __timeout_us = (timeout_us); \
+       unsigned long __delay_us = (delay_us); \
+       ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
+       if (delay_before_read && __delay_us) \
+               udelay(__delay_us); \
+       for (;;) { \
+               (val) = op(args); \
Quoted 8 lines

include/linux/iopoll.h line 96

Quoted 8 lines
include/linux/iopoll.h +8 -0
+               if (cond) \
+                       break; \
+               if (__timeout_us && \
+                   ktime_compare(ktime_get(), __timeout) > 0) { \
+                       (val) = op(args); \
+                       break; \
+               } \
+               if (__delay_us) \

Isn't there something missing here after __delay_us? I got compiler error, misses ;.

Quoted 36 lines
Quoted 35 lines
include/linux/iopoll.h +5 -19
+       } \
+       (cond) ? 0 : -ETIMEDOUT; \
+})
+
 /**
 * readx_poll_timeout - Periodically poll an address until a condition is met or a timeout occurs
 * @op: accessor function (takes @addr as its only argument)
@@ -96,25 +138,7 @@
 * macros defined below rather than this macro directly.
 */
 #define readx_poll_timeout_atomic(op, addr, val, cond, delay_us, timeout_us) \
-({ \
-       u64 __timeout_us = (timeout_us); \
-       unsigned long __delay_us = (delay_us); \
-       ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
-       for (;;) { \
-               (val) = op(addr); \
-               if (cond) \
-                       break; \
-               if (__timeout_us && \
-                   ktime_compare(ktime_get(), __timeout) > 0) { \
-                       (val) = op(addr); \
-                       break; \
-               } \
-               if (__delay_us) \
-                       udelay(__delay_us);     \
-       } \
-       (cond) ? 0 : -ETIMEDOUT; \
-})
-
+       read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, false, addr)
 
 #define readb_poll_timeout(addr, val, cond, delay_us, timeout_us) \
       readx_poll_timeout(readb, addr, val, cond, delay_us, timeout_us)
Signature · 1 line
Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail ausdrucken.
Signature · 2 lines
Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail ausdrucken.
<0001-rtw88-Make-read_poll_timeout-atomic.patch>
#8 lore raw

Hi,

Quoted 9 lines · Kai-Heng Feng

Hi,

Quoted 6 lines · Tobias S. Predel

Hello,

I guessed from the deleted lines that
udelay(__delay_us) was missing.

Quoted 7 lines

Yea I fail to copy that line :(

Quoted 3 lines

So I compiled successfully and will observe how it will work out. But for the first start on the patched kernel it's working and I will report.

Thanks. I'll send the patch.

You're welcome. Thanks for your work!
Have a nice day and stay healthy!

Quoted 3 lines

Kai-Heng

Kind regards,
Tobias

Quoted 281 lines
Quoted 280 lines

Thanks for your patch! Patch against commit a5840f9618a90ecbe1617f7632482563c0ee307e is attached.

Kind regards,
Tobias

drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
 		goto out;
 	}
 
-	ret = read_poll_timeout(rtw_read8, box_state,
+	ret = read_poll_timeout_atomic(rtw_read8, box_state,
 				!((box_state >> box) & 0x1), 100, 3000, false,
 				rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +44 -19
index cb20c733b15a..e4462712b541 100644
@@ -57,6 +57,49 @@
 	(cond) ? 0 : -ETIMEDOUT; \
 })
 
+/**
+ * read_poll_timeout_atomic - Periodically poll an address until a condition is
+ *                             met or a timeout occurs
+ * @op: accessor function (takes @addr as its only argument)
+ * @addr: Address to poll
+ * @val: Variable to read the value into
+ * @cond: Break condition (usually involving @val)
+ * @delay_us: Time to udelay between reads in us (0 tight-loops).  Should
+ *            be less than ~10us since udelay is used (see
+ *            Documentation/timers/timers-howto.rst).
+ * @timeout_us: Timeout in us, 0 means never timeout
+ * @delay_before_read: if it is true, delay @delay_us before read.
+ *
+ * Returns 0 on success and -ETIMEDOUT upon a timeout. In either
+ * case, the last read value at @args is stored in @val.
+ *
+ * When available, you'll probably want to use one of the specialized
+ * macros defined below rather than this macro directly.
+ */
+#define read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, \
+				delay_before_read, args...) \
+({ \
+	u64 __timeout_us = (timeout_us); \
+	unsigned long __delay_us = (delay_us); \
+	ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
+	if (delay_before_read && __delay_us) \
+		udelay(__delay_us); \
+	for (;;) { \
+		(val) = op(args); \
+		if (cond) \
+			break; \
+		if (__timeout_us && \
+		    ktime_compare(ktime_get(), __timeout) > 0) { \
+			(val) = op(args); \
+			 break; \
+		 } \
+		 if (__delay_us) \
+			udelay(__delay_us); \
+	} \
+	(cond) ? 0 : -ETIMEDOUT; \
+})
+
+
 /**
 * readx_poll_timeout - Periodically poll an address until a condition is met or a timeout occurs
 * @op: accessor function (takes @addr as its only argument)
@@ -96,25 +139,7 @@
 * macros defined below rather than this macro directly.
 */
 #define readx_poll_timeout_atomic(op, addr, val, cond, delay_us, timeout_us) \
-({ \
-	u64 __timeout_us = (timeout_us); \
-	unsigned long __delay_us = (delay_us); \
-	ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
-	for (;;) { \
-		(val) = op(addr); \
-		if (cond) \
-			break; \
-		if (__timeout_us && \
-		    ktime_compare(ktime_get(), __timeout) > 0) { \
-			(val) = op(addr); \
-			break; \
-		} \
-		if (__delay_us) \
-			udelay(__delay_us);	\
-	} \
-	(cond) ? 0 : -ETIMEDOUT; \
-})
-
+	read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, false, addr)
 
 #define readb_poll_timeout(addr, val, cond, delay_us, timeout_us) \
 	readx_poll_timeout(readb, addr, val, cond, delay_us, timeout_us)
Quoted 171 lines · Tobias S. Predel

Hi Kai-Heng,

Quoted 117 lines · Tobias S. Predel

Hi Kai-Heng,

Quoted 114 lines · Kai-Heng Feng

Hi Tobias,

Quoted 50 lines · Tony Chuang

Brian Norris <[email protected]> :

Quoted 47 lines

I'm not sure about the first half your problem, but for the scheduling-while-atomic:

Quoted 1 line · Tobias S. Predel
[28125.482259] BUG: scheduling while atomic:

kworker/u16:0/33416/0x00000002
...

Quoted 2 lines
[28125.482436] Preemption disabled at:
[28125.482443] [<0000000000000000>] 0x0

^^ This line is a bit weird -- shouldn't this have a real PC?

Quoted 1 line
[28125.482452] CPU: 5 PID: 33416 Comm: kworker/u16:0 Tainted: G

W 5.7.0-rc2-next-20200421-1-next-git #1

Quoted 1 line
[28125.482456] Hardware name: HP HP ProBook 430 G5/8377, BIOS Q85

Ver. 01.09.01 10/15/2019

Quoted 12 lines
[28125.482477] Workqueue: phy0 rtw_watch_dog_work [rtw88]
[28125.482481] Call Trace:
[28125.482495]  dump_stack+0x66/0x90
[28125.482505]  __schedule_bug.cold+0x8e/0x9b
[28125.482512]  __schedule+0x686/0x7b0
[28125.482520]  ? _raw_spin_unlock_irqrestore+0x20/0x40
[28125.482525]  schedule+0x46/0xf0
[28125.482531]  schedule_hrtimeout_range_clock+0xa5/0x120
[28125.482540]  ? hrtimer_init_sleeper+0xa0/0xa0
[28125.482546]  usleep_range+0x67/0x90
[28125.482568]  rtw_fw_send_h2c_command+0xe0/0x1a0 [rtw88]
[28125.482590]  rtw_fw_set_pwr_mode+0x95/0xb0 [rtw88]
Quoted 8 lines
[28125.482610]  rtw_enter_lps+0xa1/0x100 [rtw88]
[28125.482625]  rtw_watch_dog_work+0x21c/0x230 [rtw88]
[28125.482635]  process_one_work+0x1da/0x3d0
[28125.482643]  worker_thread+0x4a/0x3d0
[28125.482651]  kthread+0x122/0x160
[28125.482658]  ? process_one_work+0x3d0/0x3d0
[28125.482663]  ? kthread_park+0x90/0x90
[28125.482670]  ret_from_fork+0x1f/0x40

This looks like it might be a regression here:

commit 6343a6d4b2130be9323f347d60af8a7ba8f7242c

Author: Kai-Heng Feng <[email protected]>
Date:   Tue Apr 7 15:33:31 2020 +0800

rtw88: Add delay on polling h2c command status bit

Quoted 7 lines
Quoted 3 lines

That poll macros is using usleep, which obviously can sleep. We need to be using a udelay-variant instead.

Maybe we need an atomic version of read_poll_timeout() ?
I am not sure if this is required, but seems like it is useful for me. Noticed much of them have its atomic version, but not for this new added one.

Tony and Brian are right.

Tobias, can you please test the following patch:

drivers/net/wireless/realtek/rtw88/fw.c +1 -1
index 245da96dfddc..e44767ec0532 100644
@@ -228,7 +228,7 @@ static void rtw_fw_send_h2c_command(struct rtw_dev *rtwdev,
               goto out;
       }
 
-       ret = read_poll_timeout(rtw_read8, box_state,
+       ret = read_poll_timeout_atomic(rtw_read8, box_state,
                               !((box_state >> box) & 0x1), 100, 3000, false,
                               rtwdev, REG_HMETFR);
 
include/linux/iopoll.h +29 -0
index cb20c733b15a..bc89ac625f26 100644
@@ -57,6 +57,48 @@
       (cond) ? 0 : -ETIMEDOUT; \
 })
 
+/**
+ * read_poll_timeout_atomic - Periodically poll an address until a condition is
+ *                             met or a timeout occurs
+ * @op: accessor function (takes @addr as its only argument)
+ * @addr: Address to poll
+ * @val: Variable to read the value into
+ * @cond: Break condition (usually involving @val)
+ * @delay_us: Time to udelay between reads in us (0 tight-loops).  Should
+ *            be less than ~10us since udelay is used (see
+ *            Documentation/timers/timers-howto.rst).
+ * @timeout_us: Timeout in us, 0 means never timeout
+ * @delay_before_read: if it is true, delay @delay_us before read.
+ *
+ * Returns 0 on success and -ETIMEDOUT upon a timeout. In either
+ * case, the last read value at @args is stored in @val.
+ *
+ * When available, you'll probably want to use one of the specialized
+ * macros defined below rather than this macro directly.
+ */
+#define read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, \
+                                       delay_before_read, args...) \
+({ \
+       u64 __timeout_us = (timeout_us); \
+       unsigned long __delay_us = (delay_us); \
+       ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
+       if (delay_before_read && __delay_us) \
+               udelay(__delay_us); \
+       for (;;) { \
+               (val) = op(args); \
Quoted 8 lines

include/linux/iopoll.h line 96

Quoted 8 lines
include/linux/iopoll.h +8 -0
+               if (cond) \
+                       break; \
+               if (__timeout_us && \
+                   ktime_compare(ktime_get(), __timeout) > 0) { \
+                       (val) = op(args); \
+                       break; \
+               } \
+               if (__delay_us) \

Isn't there something missing here after __delay_us? I got compiler error, misses ;.

Quoted 36 lines
Quoted 35 lines
include/linux/iopoll.h +5 -19
+       } \
+       (cond) ? 0 : -ETIMEDOUT; \
+})
+
 /**
 * readx_poll_timeout - Periodically poll an address until a condition is met or a timeout occurs
 * @op: accessor function (takes @addr as its only argument)
@@ -96,25 +138,7 @@
 * macros defined below rather than this macro directly.
 */
 #define readx_poll_timeout_atomic(op, addr, val, cond, delay_us, timeout_us) \
-({ \
-       u64 __timeout_us = (timeout_us); \
-       unsigned long __delay_us = (delay_us); \
-       ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
-       for (;;) { \
-               (val) = op(addr); \
-               if (cond) \
-                       break; \
-               if (__timeout_us && \
-                   ktime_compare(ktime_get(), __timeout) > 0) { \
-                       (val) = op(addr); \
-                       break; \
-               } \
-               if (__delay_us) \
-                       udelay(__delay_us);     \
-       } \
-       (cond) ? 0 : -ETIMEDOUT; \
-})
-
+       read_poll_timeout_atomic(op, val, cond, delay_us, timeout_us, false, addr)
 
 #define readb_poll_timeout(addr, val, cond, delay_us, timeout_us) \
       readx_poll_timeout(readb, addr, val, cond, delay_us, timeout_us)
Signature · 1 line
Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail ausdrucken.
Signature · 2 lines
Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail ausdrucken.
<0001-rtw88-Make-read_poll_timeout-atomic.patch>
Signature · 1 line
Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail ausdrucken.