bisected: iwlwifi: use already mapped data when TXing an AMSDU

I hit the following bug in 6.11.0-rc2. Reverting the offending commit fixes the issue. This happens immediately when I run iperf3 -bidir.

[   42.873868] ------------[ cut here ]------------
[   42.873989] WARNING: CPU: 1 PID: 529 at drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836 iwl_pcie_get_sgt_tb_phys (drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836) iwlwifi

Quoted 2 lines · Chris Bainbridge

I hit the following bug in 6.11.0-rc2. Reverting the offending commit fixes the issue. This happens immediately when I run iperf3 -bidir.

[...]

Quoted 2 lines

[   42.873868] ------------[ cut here ]------------
[   42.873989] WARNING: CPU: 1 PID: 529 at drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836 iwl_pcie_get_sgt_tb_phys (drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836) iwlwifi

Does the connection still work after the warning?

Quoted 14 lines · Kalle Valo

Quoted 2 lines · Chris Bainbridge

I hit the following bug in 6.11.0-rc2. Reverting the offending commit fixes the issue. This happens immediately when I run iperf3 -bidir.

[...]

Quoted 5 lines

[   42.873868] ------------[ cut here ]------------
[ 42.873989] WARNING: CPU: 1 PID: 529 at

drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836 iwl_pcie_get_sgt_tb_phys (drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836) iwlwifi

Quoted 1 line

Does the connection still work after the warning?

Not really. It is functional but tx throughput is severely degraded, iperf shows ~1mbit/sec instead of the usual ~600mbit. Rx throughput (measured with iperf -c ... -R) doesn't seem to be affected.

Quoted 6 lines · Chris Bainbridge

Quoted 1 line · Kalle Valo

[SNIP]

Not really. It is functional but tx throughput is severely degraded,
iperf shows ~1mbit/sec instead of the usual ~600mbit. Rx throughput (measured with iperf -c ... -R) doesn't seem to be affected.

Ok, so a quite severe issue. Hopefully Intel can find a fix, otherwise
we need to consider should we revert commit 90db50755228 ("wifi:
iwlwifi: use already mapped data when TXing an AMSDU") for v6.11.

Hi,

Quoted 9 lines · Kalle Valo

Quoted 6 lines · Chris Bainbridge

Quoted 1 line · Kalle Valo

[SNIP]

Not really. It is functional but tx throughput is severely degraded,
iperf shows ~1mbit/sec instead of the usual ~600mbit. Rx throughput (measured with iperf -c ... -R) doesn't seem to be affected.

Quoted 4 lines

Ok, so a quite severe issue. Hopefully Intel can find a fix, otherwise
we need to consider should we revert commit 90db50755228 ("wifi:
iwlwifi: use already mapped data when TXing an AMSDU") for v6.11.

Yes, if we do not have a fix soon we'll need to revert. If so, then we should also revert:

wifi: iwlwifi: keep the TSO and workaround pages mapped
wifi: iwlwifi: map entire SKB when sending AMSDUs

as well as the related bugfix:
wifi: iwlwifi: correctly reference TSO page information

Otherwise there is a performance impact as DMA mapping is effectively done twice.

Hello,

I hit the following bug in 6.11.0-rc2. Reverting the offending commit fixes the issue. This happens immediately when I run iperf3 -bidir.

#regzbot introduced: 90db50755228252f94b143bbf2380242688b1104

90db50755228252f94b143bbf2380242688b1104 is the first bad commit commit 90db50755228252f94b143bbf2380242688b1104
Author: Benjamin Berg <[email protected]>
Date:   Wed Jul 3 12:58:55 2024 +0300

    wifi: iwlwifi: use already mapped data when TXing an AMSDU

    The previous commits added mappings for the SKB and TSO page. This
    switches the code to use these mappings instead of creating new ones.

Signed-off-by Benjamin Berg <[email protected]>

Signed-off-by Miri Korenblit <[email protected]>

Reviewed-by Johannes Berg <[email protected]>

    Link:
https://patch.msgid.link/20240703125541.35d89c5e4ae8.I4feb8d34e7b30768d21365ec22c944bacc274d0b@changeid

Signed-off-by Johannes Berg <[email protected]>

 drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c | 64
+++++++++++++++++------
 drivers/net/wireless/intel/iwlwifi/pcie/tx.c      | 21 +++++---

 2 files changed, 63 insertions(+), 22 deletions(-)

[   42.873868] ------------[ cut here ]------------
[   42.873989] WARNING: CPU: 1 PID: 529 at drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836 iwl_pcie_get_sgt_tb_phys (drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836) iwlwifi [   42.874019] Modules linked in: nvme_fabrics ccm qrtr overlay cmac algif_hash algif_skcipher af_alg bnep binfmt_misc iwlmvm snd_acp3x_rn snd_acp3x_pdm_dma snd_soc_dmic mac80211 snd_sof_amd_rembrandt snd_sof_amd_renoir snd_sof_amd_acp snd_sof_pci snd_sof_xtensa_dsp libarc4 snd_sof snd_ctl_led snd_hda_codec_realtek snd_sof_utils snd_soc_core uvcvideo snd_hda_codec_generic videobuf2_vmalloc videobuf2_memops snd_hda_codec_hdmi snd_hda_scodec_component uvc snd_compress snd_pci_ps snd_hda_intel videobuf2_v4l2 snd_rpl_pci_acp6x videodev snd_intel_dspcfg btusb snd_acp_pci snd_acp_legacy_common snd_hda_codec btrtl btintel snd_hwdep snd_hda_core videobuf2_common snd_pci_acp6x btbcm snd_pcm intel_rapl_msr btmtk iwlwifi mc intel_rapl_common snd_pci_acp5x snd_timer snd_rn_pci_acp3x snd_acp_config snd snd_soc_acpi kvm_amd cfg80211 bluetooth snd_pci_acp3x soundcore ee1004 kvm ccp k10temp sp5100_tco input_leds joydev serio_raw hp_wmi sparse_keymap ucsi_acpi rapl mac_hid typec_ucsi pcspkr amd_pmc platform_profile wmi_bmof typec
[   42.874165]  hid_multitouch acpi_tad msr parport_pc ppdev lp parport efi_pstore dmi_sysfs ip_tables x_tables autofs4 btrfs blake2b_generic libcrc32c xor raid6_pq uas usb_storage amdgpu i2c_algo_bit drm_ttm_helper ttm drm_exec nvme drm_suballoc_helper amdxcp drm_buddy hid_generic nvme_core video i2c_piix4 xhci_pci gpu_sched crct10dif_pclmul crc32_pclmul polyval_clmulni polyval_generic ghash_clmulni_intel drm_display_helper amd_sfh i2c_smbus xhci_pci_renesas nvme_auth i2c_hid_acpi i2c_hid wmi hid aesni_intel crypto_simd cryptd
[   42.874309] CPU: 1 UID: 0 PID: 529 Comm: irq/77-iwlwifi: Not tainted 6.11.0-rc2 #50
[   42.874319] Hardware name: HP HP Pavilion Aero Laptop 13- be0xxx/8916, BIOS F.14 10/25/2023
[   42.874326] RIP: 0010:iwl_pcie_get_sgt_tb_phys (drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836) iwlwifi

[ 42.874352] Code: 48 8d 34 0a 48 39 f3 72 0e 8b 47 18 48 01 c8 48 01

c2 48 39 d3 72 2b e8 41 7c 1e c2 41 83 c5 01 48 89 c7 45 3b 6c 24 08 72 b6 <0f> 0b 48 c7 c0 ff ff ff ff 5b 41 5c 41 5d 5d 31 d2 31 c9 31 f6 31
All code
========
   0: 48 8d 34 0a           lea    (%rdx,%rcx,1),%rsi
   4: 48 39 f3              cmp    %rsi,%rbx
   7: 72 0e                 jb     0x17
   9: 8b 47 18              mov    0x18(%rdi),%eax
   c: 48 01 c8              add    %rcx,%rax
   f: 48 01 c2              add    %rax,%rdx
  12: 48 39 d3              cmp    %rdx,%rbx
  15: 72 2b                 jb     0x42
  17: e8 41 7c 1e c2        call   0xffffffffc21e7c5d
  1c: 41 83 c5 01           add    $0x1,%r13d
  20: 48 89 c7              mov    %rax,%rdi
  23: 45 3b 6c 24 08        cmp    0x8(%r12),%r13d
  28: 72 b6                 jb     0xffffffffffffffe0
  2a:* 0f 0b                 ud2 <-- trapping
instruction
  2c: 48 c7 c0 ff ff ff ff  mov    $0xffffffffffffffff,%rax
  33: 5b                    pop    %rbx
  34: 41 5c                 pop    %r12
  36: 41 5d                 pop    %r13
  38: 5d                    pop    %rbp
  39: 31 d2                 xor    %edx,%edx
  3b: 31 c9                 xor    %ecx,%ecx
  3d: 31 f6                 xor    %esi,%esi
  3f: 31                    .byte 0x31

Code starting with the faulting instruction
===========================================
   0: 0f 0b                 ud2
   2: 48 c7 c0 ff ff ff ff  mov    $0xffffffffffffffff,%rax
   9: 5b                    pop    %rbx
   a: 41 5c                 pop    %r12
   c: 41 5d                 pop    %r13
   e: 5d                    pop    %rbp
   f: 31 d2                 xor    %edx,%edx
  11: 31 c9                 xor    %ecx,%ecx
  13: 31 f6                 xor    %esi,%esi
  15: 31                    .byte 0x31
[   42.874357] RSP: 0018:ffffb263803dff00 EFLAGS: 00010246 [   42.874364] RAX: ffff97e55199c4e0 RBX: ffff97e550670000 RCX:
0000000000005e50
[   42.874368] RDX: 0000000000000000 RSI: ffff97e549c8de50 RDI:
ffff97e55199c4e0
[   42.874372] RBP: ffffb263803dff18 R08: 0000000000000000 R09:
0000000000000000
[   42.874375] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff97e55199c490
[   42.874379] R13: 0000000000000002 R14: ffff97e54a8b8028 R15:
ffff97e55199c490
[   42.874382] FS:  0000000000000000(0000) GS:ffff97e84d080000(0000) knlGS:0000000000000000
[   42.874387] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [   42.874391] CR2: 00005c2f48bdc078 CR3: 000000019d658000 CR4:
0000000000f50ef0
[   42.874395] PKRU: 55555554
[   42.874399] Call Trace:
[   42.874402]  <IRQ>
[   42.874408] ? show_regs (arch/x86/kernel/dumpstack.c:479) [   42.874417] ? __warn (kernel/panic.c:735)
[   42.874428] ? iwl_pcie_get_sgt_tb_phys (drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836) iwlwifi [   42.874451] ? report_bug (lib/bug.c:180 lib/bug.c:219) [   42.874463] ? handle_bug (arch/x86/kernel/traps.c:218) [   42.874469] ? exc_invalid_op (arch/x86/kernel/traps.c:260 (discriminator 1))
[   42.874475] ? asm_exc_invalid_op (./arch/x86/include/asm/idtentry.h:621)
[   42.874491] ? iwl_pcie_get_sgt_tb_phys (drivers/net/wireless/intel/iwlwifi/pcie/tx.c:1836) iwlwifi [   42.874514] iwl_txq_gen2_tx (drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c:263 drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c:338 drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c:517 drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c:777) iwlwifi [   42.874551] iwl_trans_tx (drivers/net/wireless/intel/iwlwifi/iwl- trans.c:417) iwlwifi
[   42.874575] iwl_mvm_tx_mpdu (drivers/net/wireless/intel/iwlwifi/mvm/tx.c:1320) iwlmvm [   42.874610] ? iwl_mvm_tx_tso_segment (drivers/net/wireless/intel/iwlwifi/mvm/tx.c:973) iwlmvm
[   42.874641] iwl_mvm_tx_skb_sta (drivers/net/wireless/intel/iwlwifi/mvm/tx.c:1417 (discriminator 4)) iwlmvm
[   42.874677] iwl_mvm_tx_skb (drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c:801) iwlmvm [   42.874702] iwl_mvm_mac_itxq_xmit (./arch/x86/include/asm/bitops.h:206 ./arch/x86/include/asm/bitops.h:238 ./include/asm- generic/bitops/instrumented-non-atomic.h:142 drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c:911) iwlmvm [   42.874728] iwl_mvm_mac_wake_tx_queue (drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c:960) iwlmvm [   42.874751] ieee80211_queue_skb (net/mac80211/tx.c:1666) mac80211 [   42.874836] __ieee80211_xmit_fast (net/mac80211/tx.c:3732) mac80211
[   42.874905] ? find_held_lock (kernel/locking/lockdep.c:5249) [   42.874913] ? skb_mac_gso_segment (./include/linux/rcupdate.h:336 ./include/linux/rcupdate.h:869 net/core/gso.c:57)
[   42.874936] __ieee80211_subif_start_xmit (net/mac80211/tx.c:3796 net/mac80211/tx.c:4308) mac80211
[   42.874994] ieee80211_subif_start_xmit (net/mac80211/tx.c:4537) mac80211
[   42.875047] ? lock_acquire (kernel/locking/lockdep.c:466 kernel/locking/lockdep.c:5761 kernel/locking/lockdep.c:5724) [   42.875053] ? __dev_queue_xmit (./include/linux/netdevice.h:4358 net/core/dev.c:4419)
[   42.875064] dev_hard_start_xmit (./include/linux/netdevice.h:4913 ./include/linux/netdevice.h:4922 net/core/dev.c:3580 net/core/dev.c:3596)
[   42.875070] ? dev_hard_start_xmit (./include/linux/netdevice.h:4913 ./include/linux/netdevice.h:4922 net/core/dev.c:3580 net/core/dev.c:3596)
[   42.875081] __dev_queue_xmit (net/core/dev.h:168 net/core/dev.c:4424)
[   42.875087] ? select_task_rq_fair (./include/linux/rcupdate.h:336 ./include/linux/rcupdate.h:869 kernel/sched/fair.c:8251)
[   42.875095] ? lock_acquire (kernel/locking/lockdep.c:466 kernel/locking/lockdep.c:5761 kernel/locking/lockdep.c:5724) [   42.875100] ? mark_held_locks (kernel/locking/lockdep.c:4273) [   42.875104] ip_finish_output2 (./include/linux/netdevice.h:3105 ./include/net/neighbour.h:526 ./include/net/neighbour.h:540 net/ipv4/ip_output.c:235)
[   42.875107] ? rcu_read_lock_held (kernel/rcu/update.c:352) [   42.875112] __ip_finish_output (net/ipv4/ip_output.c:313 net/ipv4/ip_output.c:295)
[   42.875116] ip_finish_output (net/ipv4/ip_output.c:330) [   42.875119] ip_output (net/ipv4/ip_output.c:437)
[   42.875122] __ip_queue_xmit (./include/net/dst.h:450 net/ipv4/ip_output.c:129 net/ipv4/ip_output.c:535)
[   42.875126] ip_queue_xmit (net/ipv4/ip_output.c:550)
[   42.875128] __tcp_transmit_skb (net/ipv4/tcp_output.c:1466) [   42.875137] tcp_write_xmit (net/ipv4/tcp_output.c:2829) [   42.875142] __tcp_push_pending_frames (net/ipv4/tcp_output.c:3014) [   42.875145] tcp_rcv_established (net/ipv4/tcp_input.c:5712 net/ipv4/tcp_input.c:6145)
[   42.875148] ? lock_is_held_type (kernel/locking/lockdep.c:5500 kernel/locking/lockdep.c:5831)
[   42.875152] tcp_v4_do_rcv (net/ipv4/tcp_ipv4.c:1956)
[   42.875155] tcp_v4_rcv (net/ipv4/tcp_ipv4.c:2344)
[   42.875158] ? lock_release (kernel/locking/lockdep.c:5435 kernel/locking/lockdep.c:5780)
[   42.875162] ? lock_is_held_type (kernel/locking/lockdep.c:5500 kernel/locking/lockdep.c:5831)
[   42.875167] ip_protocol_deliver_rcu (net/ipv4/ip_input.c:205) [   42.875171] ip_local_deliver_finish (./include/linux/rcupdate.h:867 net/ipv4/ip_input.c:234)
[   42.875175] ip_local_deliver (net/ipv4/ip_input.c:257) [   42.875178] ip_rcv_finish (net/ipv4/ip_input.c:451)
[   42.875181] ip_rcv (net/ipv4/ip_input.c:572)
[   42.875185] __netif_receive_skb_core.constprop.0 (net/core/dev.c:2237 net/core/dev.c:2252 net/core/dev.c:5617) [   42.875189] ? __lock_acquire (kernel/locking/lockdep.c:4598 kernel/locking/lockdep.c:5096)
[   42.875197] __netif_receive_skb_list_core (net/core/dev.c:5738) [   42.875200] ? trace_hardirqs_on (kernel/trace/trace_preemptirq.c:63)
[   42.875205] netif_receive_skb_list_internal (net/core/dev.c:5806 net/core/dev.c:5896)
[   42.875210] napi_complete_done (./include/linux/list.h:37 ./include/net/gro.h:516 ./include/net/gro.h:511 net/core/dev.c:6247) [   42.875214] iwl_pcie_napi_poll_msix (drivers/net/wireless/intel/iwlwifi/pcie/rx.c:1065) iwlwifi [   42.875225] __napi_poll.constprop.0 (net/core/dev.c:6773) [   42.875229] net_rx_action (net/core/dev.c:6843 net/core/dev.c:6963)
[   42.875238] handle_softirqs (./arch/x86/include/asm/jump_label.h:27 ./include/linux/jump_label.h:207 ./include/trace/events/irq.h:142 kernel/softirq.c:555)
[   42.875242] ? iwl_pcie_irq_rx_msix_handler (./include/linux/bottom_half.h:33 drivers/net/wireless/intel/iwlwifi/pcie/rx.c:1669) iwlwifi [   42.875252] ? iwl_pcie_irq_rx_msix_handler (drivers/net/wireless/intel/iwlwifi/pcie/rx.c:1663) iwlwifi [   42.875260] __do_softirq (kernel/softirq.c:589)
[   42.875264] do_softirq.part.0 (kernel/softirq.c:455)
[   42.875266]  </IRQ>
[   42.875268]  <TASK>
[   42.875269] __local_bh_enable_ip (kernel/softirq.c:457 kernel/softirq.c:382)
[   42.875272] ? iwl_pcie_irq_rx_msix_handler (./include/linux/bottom_half.h:33 drivers/net/wireless/intel/iwlwifi/pcie/rx.c:1669) iwlwifi [   42.875281] iwl_pcie_irq_rx_msix_handler (drivers/net/wireless/intel/iwlwifi/pcie/rx.c:1671) iwlwifi [   42.875290] ? irq_thread (kernel/irq/manage.c:1322)
[   42.875293] irq_thread_fn (kernel/irq/manage.c:1222)
[   42.875296] ? irq_thread (kernel/irq/manage.c:1322)
[   42.875298] irq_thread (kernel/irq/manage.c:1328)
[   42.875300] ? disable_irq_nosync (kernel/irq/manage.c:1219) [   42.875303] ? irq_thread (kernel/irq/manage.c:1237)
[   42.875306] ? irq_set_affinity_notifier (kernel/irq/manage.c:1304) [   42.875308] kthread (kernel/kthread.c:389)
[   42.875312] ? kthread_insert_work_sanity_check (kernel/kthread.c:342)
[   42.875315] ret_from_fork (arch/x86/kernel/process.c:153) [   42.875318] ? kthread_insert_work_sanity_check (kernel/kthread.c:342)
[   42.875321] ret_from_fork_asm (arch/x86/entry/entry_64.S:257) [   42.875328]  </TASK>
[   42.875330] irq event stamp: 29672
[   42.875331] hardirqs last enabled at (29680): console_unlock (./arch/x86/include/asm/paravirt.h:698 ./arch/x86/include/asm/irqflags.h:155 kernel/printk/printk.c:341 kernel/printk/printk.c:2801 kernel/printk/printk.c:3120)
[   42.875334] hardirqs last disabled at (29687): console_unlock

(kernel/printk/printk.c:339 kernel/printk/printk.c:2801 kernel/printk/printk.c:3120)
[   42.875337] softirqs last enabled at (27924):
iwl_pcie_irq_rx_msix_handler (./include/linux/bottom_half.h:33 drivers/net/wireless/intel/iwlwifi/pcie/rx.c:1669) iwlwifi [   42.875346] softirqs last disabled at (27925): __do_softirq (kernel/softirq.c:589)
[   42.875348] ---[ end trace 0000000000000000 ]---