skb_warn_bad_offload warnings with FreeBSD guests
Subsystems (2)
We have a number of machines running qemu with bridged networking. We have noticed that *sometimes* FreeBSD guests cause this warning to flood the host "WARNING: CPU: 5 PID: 3705 at net/core/dev.c:2238 skb_warn_bad_offload+0xc3/0xd0()". I haven't been able to come up with any sort of reproduction steps, it just seems to happen to some FreeBSD guests, but not others.
A full stack trace looks like this:
------------[ cut here ]------------ WARNING: CPU: 1 PID: 7147 at net/core/dev.c:2233 skb_warn_bad_offload+0xc3/0xd0() igb: caps=(0x0000000190114bb3, 0x0000000000000000) len=2962 data_len=0 gso_size=1448 gso_type=5 ip_summed=0 Modules linked in: dm_snapshot dm_bufio ipmi_devintf xt_physdev ebt_arp ebt_ip ebtable_nat ebtables cls_fw sch_sfq sch_htb tun kvm_intel kvm 8021q garp nfnetlink_queue nfnetlink_log nfnetlink bluetooth rfkill bridge stp llc xt_CHECKSUM iptable_mangle ipt_REJECT iptable_filter ip _tables ip6t_REJECT ip6table_filter ip6_tables ipv6 iTCO_wdt iTCO_vendor_support ipmi_si ipmi_msghandler microcode pcspkr i2c_i801 joydev sg lpc_ich shpchp igb dca ptp pps_core hwmon ext4 jbd2 mbcache sd_mod crc_t10dif crct10dif_common video ahci libahci xhci_hcd ast ttm drm_kms _helper sysimgblt sysfillrect syscopyarea dm_mirror dm_region_hash dm_log dm_mod CPU: 1 PID: 7147 Comm: qemu-kvm Tainted: G W 3.15.5-1.el6.elrepo.x86_64 #1 Hardware name: Supermicro X10SLE-F/HF/X10SLE, BIOS 1.1 07/19/2013 00000000000008b9 ffff88081fc435d8 ffffffff8163ba90 00000000000008b9 ffff88081fc43628 ffff88081fc43618 ffffffff8106c30c ffffc90007a06e30 0000000000000000 ffff8807f2b64000 ffff8807f2b64000 0000000000000000 Call Trace: <IRQ> [<ffffffff8163ba90>] dump_stack+0x49/0x61 [<ffffffff8106c30c>] warn_slowpath_common+0x8c/0xc0 [<ffffffff8106c3f6>] warn_slowpath_fmt+0x46/0x50 [<ffffffff8156ce93>] skb_warn_bad_offload+0xc3/0xd0 [<ffffffff81574a29>] ? dev_hard_start_xmit+0x339/0x640 [<ffffffff81574699>] __skb_gso_segment+0x89/0xe0 [<ffffffff81574876>] dev_hard_start_xmit+0x186/0x640 [<ffffffff81594f5a>] sch_direct_xmit+0xfa/0x1d0 [<ffffffff81574f2f>] __dev_queue_xmit+0x1ff/0x4f0 [<ffffffff81575240>] dev_queue_xmit+0x10/0x20 [<ffffffffa02e6612>] br_dev_queue_push_xmit+0x82/0xb0 [bridge] [<ffffffffa02ee680>] br_nf_dev_queue_xmit+0x20/0x90 [bridge] [<ffffffffa02ef4b8>] br_nf_post_routing+0x2d8/0x300 [bridge] [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffffa02ee6f0>] ? br_nf_dev_queue_xmit+0x90/0x90 [bridge] [<ffffffffa02e6b43>] br_forward_finish+0x43/0x60 [bridge] [<ffffffffa02ee8a8>] br_nf_forward_finish+0x1b8/0x1d0 [bridge] [<ffffffffa02ef178>] br_nf_forward_ip+0x3a8/0x410 [bridge] [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffffa02e66e4>] __br_forward+0xa4/0x100 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e67d6>] br_forward+0x96/0xb0 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e7997>] br_handle_frame_finish+0x197/0x3f0 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02ef790>] br_nf_pre_routing_finish+0x2b0/0x370 [bridge] [<ffffffffa02ef4e0>] ? br_nf_post_routing+0x300/0x300 [bridge] [<ffffffffa02ed986>] NF_HOOK_THRESH+0x56/0x60 [bridge] [<ffffffffa02eed2b>] br_nf_pre_routing+0x2fb/0x3a0 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e7d8c>] br_handle_frame+0x19c/0x240 [bridge] [<ffffffffa02e7bf0>] ? br_handle_frame_finish+0x3f0/0x3f0 [bridge] [<ffffffff81572fa5>] __netif_receive_skb_core+0x1e5/0x620 [<ffffffff81573407>] __netif_receive_skb+0x27/0x70 [<ffffffff81573553>] process_backlog+0x103/0x200 [<ffffffff81573d62>] net_rx_action+0x112/0x2a0 [<ffffffff8107111c>] __do_softirq+0xfc/0x2b0 [<ffffffff810713cd>] ? irq_exit+0xad/0xd0 [<ffffffff8164a81c>] do_softirq_own_stack+0x1c/0x30 <EOI> [<ffffffff81070e75>] do_softirq+0x55/0x60 [<ffffffff81571e19>] netif_rx_ni+0x39/0x70 [<ffffffffa03e84e0>] tun_get_user+0x310/0x6c0 [tun] [<ffffffffa03e8995>] tun_chr_aio_write+0x85/0xa0 [tun] [<ffffffff811beb9d>] do_sync_readv_writev+0x4d/0x80 [<ffffffff811c0128>] do_readv_writev+0xc8/0x2c0 [<ffffffff811bebd0>] ? do_sync_readv_writev+0x80/0x80 [<ffffffff811d2c45>] ? poll_select_set_timeout+0x95/0xb0 [<ffffffff811c0357>] vfs_writev+0x37/0x50 [<ffffffff811c0496>] SyS_writev+0x56/0xf0 [<ffffffff81648ee9>] system_call_fastpath+0x16/0x1b ---[ end trace d26e70ba037ab631 ]---
gso_type=5 and ip_summed=0 are always the same (though len, data_len, and gso_size vary).
What is causing this? I've tried kernels as new as 3.15.5-1, which do not appear to help.
Quoted 91 lines · Brian Rak
We have a number of machines running qemu with bridged networking. We have noticed that *sometimes* FreeBSD guests cause this warning to flood the host "WARNING: CPU: 5 PID: 3705 at
net/core/dev.c:2238skb_warn_bad_offload+0xc3/0xd0()". I haven't been able to come up with any sort of reproduction steps, it just seems to happen to some FreeBSD guests, but not others.A full stack trace looks like this:
------------[ cut here ]------------ WARNING: CPU: 1 PID: 7147 at net/core/dev.c:2233 skb_warn_bad_offload+0xc3/0xd0() igb: caps=(0x0000000190114bb3, 0x0000000000000000) len=2962 data_len=0 gso_size=1448 gso_type=5 ip_summed=0 Modules linked in: dm_snapshot dm_bufio ipmi_devintf xt_physdev ebt_arp ebt_ip ebtable_natebtables
cls_fwsch_sfqsch_htbtunkvm_intelkvm 8021q garpnfnetlink_queuenfnetlink_lognfnetlink bluetooth rfkill bridge stp llcxt_CHECKSUMiptable_mangleipt_REJECTiptable_filterip_tablesip6t_REJECTip6table_filterip6_tablesipv6iTCO_wdtiTCO_vendor_supportipmi_siipmi_msghandlermicrocode pcspkri2c_i801joydev sglpc_ichshpchp igb dca ptppps_corehwmon ext4 jbd2 mbcachesd_modcrc_t10difcrct10dif_commonvideo ahci libahcixhci_hcdast ttmdrm_kms_helpersysimgblt sysfillrect syscopyareadm_mirrordm_region_hashdm_logdm_modCPU: 1 PID: 7147 Comm: qemu-kvm Tainted: G W 3.15.5-1.el6.elrepo.x86_64 #1 Hardware name: Supermicro X10SLE-F/HF/X10SLE, BIOS 1.1 07/19/2013 00000000000008b9 ffff88081fc435d8 ffffffff8163ba90 00000000000008b9 ffff88081fc43628 ffff88081fc43618 ffffffff8106c30c ffffc90007a06e30 0000000000000000 ffff8807f2b64000 ffff8807f2b64000 0000000000000000 Call Trace: <IRQ> [<ffffffff8163ba90>] dump_stack+0x49/0x61 [<ffffffff8106c30c>] warn_slowpath_common+0x8c/0xc0 [<ffffffff8106c3f6>] warn_slowpath_fmt+0x46/0x50 [<ffffffff8156ce93>] skb_warn_bad_offload+0xc3/0xd0 [<ffffffff81574a29>] ? dev_hard_start_xmit+0x339/0x640 [<ffffffff81574699>] __skb_gso_segment+0x89/0xe0 [<ffffffff81574876>] dev_hard_start_xmit+0x186/0x640 [<ffffffff81594f5a>] sch_direct_xmit+0xfa/0x1d0 [<ffffffff81574f2f>] __dev_queue_xmit+0x1ff/0x4f0 [<ffffffff81575240>] dev_queue_xmit+0x10/0x20 [<ffffffffa02e6612>] br_dev_queue_push_xmit+0x82/0xb0 [bridge] [<ffffffffa02ee680>] br_nf_dev_queue_xmit+0x20/0x90 [bridge] [<ffffffffa02ef4b8>] br_nf_post_routing+0x2d8/0x300 [bridge] [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffffa02ee6f0>] ? br_nf_dev_queue_xmit+0x90/0x90 [bridge] [<ffffffffa02e6b43>] br_forward_finish+0x43/0x60 [bridge] [<ffffffffa02ee8a8>] br_nf_forward_finish+0x1b8/0x1d0 [bridge] [<ffffffffa02ef178>] br_nf_forward_ip+0x3a8/0x410 [bridge] [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffffa02e66e4>] __br_forward+0xa4/0x100 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e67d6>] br_forward+0x96/0xb0 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e7997>] br_handle_frame_finish+0x197/0x3f0 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02ef790>] br_nf_pre_routing_finish+0x2b0/0x370 [bridge] [<ffffffffa02ef4e0>] ? br_nf_post_routing+0x300/0x300 [bridge] [<ffffffffa02ed986>] NF_HOOK_THRESH+0x56/0x60 [bridge] [<ffffffffa02eed2b>] br_nf_pre_routing+0x2fb/0x3a0 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e7d8c>] br_handle_frame+0x19c/0x240 [bridge] [<ffffffffa02e7bf0>] ? br_handle_frame_finish+0x3f0/0x3f0 [bridge] [<ffffffff81572fa5>] __netif_receive_skb_core+0x1e5/0x620 [<ffffffff81573407>] __netif_receive_skb+0x27/0x70 [<ffffffff81573553>] process_backlog+0x103/0x200 [<ffffffff81573d62>] net_rx_action+0x112/0x2a0 [<ffffffff8107111c>] __do_softirq+0xfc/0x2b0 [<ffffffff810713cd>] ? irq_exit+0xad/0xd0 [<ffffffff8164a81c>] do_softirq_own_stack+0x1c/0x30 <EOI> [<ffffffff81070e75>] do_softirq+0x55/0x60 [<ffffffff81571e19>] netif_rx_ni+0x39/0x70 [<ffffffffa03e84e0>] tun_get_user+0x310/0x6c0 [tun] [<ffffffffa03e8995>] tun_chr_aio_write+0x85/0xa0 [tun] [<ffffffff811beb9d>] do_sync_readv_writev+0x4d/0x80 [<ffffffff811c0128>] do_readv_writev+0xc8/0x2c0 [<ffffffff811bebd0>] ? do_sync_readv_writev+0x80/0x80 [<ffffffff811d2c45>] ? poll_select_set_timeout+0x95/0xb0 [<ffffffff811c0357>] vfs_writev+0x37/0x50 [<ffffffff811c0496>] SyS_writev+0x56/0xf0 [<ffffffff81648ee9>] system_call_fastpath+0x16/0x1b ---[ end trace d26e70ba037ab631 ]---
Quoted 3 lines
gso_type=5 and ip_summed=0 are always the same (though len,
data_len, andgso_sizevary).What is causing this?
The reason that the warning is triggered is ip_summed = 0 which means there is not checksum already in the packet and it needs to be calculated. If the packet is GSO, then it needs to have partial checksum set (ip_summed == 3).
You might try using systemtap or instrumenting tun and bridge to see what the ip_summed value is when this happens.
-vlad
Quoted 6 lines
I've tried kernels as new as 3.15.5-1, which do not appear to help.
Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 94 lines · Vlad Yasevich
Quoted 93 lines · Brian Rak
We have a number of machines running qemu with bridged networking. We have noticed that *sometimes* FreeBSD guests cause this warning to flood the host "WARNING: CPU: 5 PID: 3705 at
net/core/dev.c:2238skb_warn_bad_offload+0xc3/0xd0()". I haven't been able to come up with any sort of reproduction steps, it just seems to happen to some FreeBSD guests, but not others.A full stack trace looks like this:
------------[ cut here ]------------ WARNING: CPU: 1 PID: 7147 at net/core/dev.c:2233 skb_warn_bad_offload+0xc3/0xd0() igb: caps=(0x0000000190114bb3, 0x0000000000000000) len=2962 data_len=0 gso_size=1448 gso_type=5 ip_summed=0 Modules linked in: dm_snapshot dm_bufio ipmi_devintf xt_physdev ebt_arp ebt_ip ebtable_natebtables
cls_fwsch_sfqsch_htbtunkvm_intelkvm 8021q garpnfnetlink_queuenfnetlink_lognfnetlink bluetooth rfkill bridge stp llcxt_CHECKSUMiptable_mangleipt_REJECTiptable_filterip_tablesip6t_REJECTip6table_filterip6_tablesipv6iTCO_wdtiTCO_vendor_supportipmi_siipmi_msghandlermicrocode pcspkri2c_i801joydev sglpc_ichshpchp igb dca ptppps_corehwmon ext4 jbd2 mbcachesd_modcrc_t10difcrct10dif_commonvideo ahci libahcixhci_hcdast ttmdrm_kms_helpersysimgblt sysfillrect syscopyareadm_mirrordm_region_hashdm_logdm_modCPU: 1 PID: 7147 Comm: qemu-kvm Tainted: G W 3.15.5-1.el6.elrepo.x86_64 #1 Hardware name: Supermicro X10SLE-F/HF/X10SLE, BIOS 1.1 07/19/2013 00000000000008b9 ffff88081fc435d8 ffffffff8163ba90 00000000000008b9 ffff88081fc43628 ffff88081fc43618 ffffffff8106c30c ffffc90007a06e30 0000000000000000 ffff8807f2b64000 ffff8807f2b64000 0000000000000000 Call Trace: <IRQ> [<ffffffff8163ba90>] dump_stack+0x49/0x61 [<ffffffff8106c30c>] warn_slowpath_common+0x8c/0xc0 [<ffffffff8106c3f6>] warn_slowpath_fmt+0x46/0x50 [<ffffffff8156ce93>] skb_warn_bad_offload+0xc3/0xd0 [<ffffffff81574a29>] ? dev_hard_start_xmit+0x339/0x640 [<ffffffff81574699>] __skb_gso_segment+0x89/0xe0 [<ffffffff81574876>] dev_hard_start_xmit+0x186/0x640 [<ffffffff81594f5a>] sch_direct_xmit+0xfa/0x1d0 [<ffffffff81574f2f>] __dev_queue_xmit+0x1ff/0x4f0 [<ffffffff81575240>] dev_queue_xmit+0x10/0x20 [<ffffffffa02e6612>] br_dev_queue_push_xmit+0x82/0xb0 [bridge] [<ffffffffa02ee680>] br_nf_dev_queue_xmit+0x20/0x90 [bridge] [<ffffffffa02ef4b8>] br_nf_post_routing+0x2d8/0x300 [bridge] [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffffa02ee6f0>] ? br_nf_dev_queue_xmit+0x90/0x90 [bridge] [<ffffffffa02e6b43>] br_forward_finish+0x43/0x60 [bridge] [<ffffffffa02ee8a8>] br_nf_forward_finish+0x1b8/0x1d0 [bridge] [<ffffffffa02ef178>] br_nf_forward_ip+0x3a8/0x410 [bridge] [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffffa02e66e4>] __br_forward+0xa4/0x100 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e67d6>] br_forward+0x96/0xb0 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e7997>] br_handle_frame_finish+0x197/0x3f0 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02ef790>] br_nf_pre_routing_finish+0x2b0/0x370 [bridge] [<ffffffffa02ef4e0>] ? br_nf_post_routing+0x300/0x300 [bridge] [<ffffffffa02ed986>] NF_HOOK_THRESH+0x56/0x60 [bridge] [<ffffffffa02eed2b>] br_nf_pre_routing+0x2fb/0x3a0 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e7d8c>] br_handle_frame+0x19c/0x240 [bridge] [<ffffffffa02e7bf0>] ? br_handle_frame_finish+0x3f0/0x3f0 [bridge] [<ffffffff81572fa5>] __netif_receive_skb_core+0x1e5/0x620 [<ffffffff81573407>] __netif_receive_skb+0x27/0x70 [<ffffffff81573553>] process_backlog+0x103/0x200 [<ffffffff81573d62>] net_rx_action+0x112/0x2a0 [<ffffffff8107111c>] __do_softirq+0xfc/0x2b0 [<ffffffff810713cd>] ? irq_exit+0xad/0xd0 [<ffffffff8164a81c>] do_softirq_own_stack+0x1c/0x30 <EOI> [<ffffffff81070e75>] do_softirq+0x55/0x60 [<ffffffff81571e19>] netif_rx_ni+0x39/0x70 [<ffffffffa03e84e0>] tun_get_user+0x310/0x6c0 [tun] [<ffffffffa03e8995>] tun_chr_aio_write+0x85/0xa0 [tun] [<ffffffff811beb9d>] do_sync_readv_writev+0x4d/0x80 [<ffffffff811c0128>] do_readv_writev+0xc8/0x2c0 [<ffffffff811bebd0>] ? do_sync_readv_writev+0x80/0x80 [<ffffffff811d2c45>] ? poll_select_set_timeout+0x95/0xb0 [<ffffffff811c0357>] vfs_writev+0x37/0x50 [<ffffffff811c0496>] SyS_writev+0x56/0xf0 [<ffffffff81648ee9>] system_call_fastpath+0x16/0x1b ---[ end trace d26e70ba037ab631 ]---gso_type=5 and ip_summed=0 are always the same (though len,
data_len, andgso_sizevary).
Quoted 7 lines
Quoted 1 line
What is causing this?
The reason that the warning is triggered is
ip_summed= 0 which means there is not checksum already in the packet and it needs to be calculated. If the packet is GSO, then it needs to have partial checksum set (ip_summed== 3).You might try using systemtap or instrumenting tun and bridge to see what the
ip_summedvalue is when this happens.
Who needs systemtap when you have strace ;)
I managed to intercept the raw packet + headers being delivered to the tun device, though I'm having some trouble making sense of it. I've got this call:
writev(33, [{"\x00\x01\x42\x00\xa0\x05\x00\x00\x00\x00\x00\x00", 12}, .... ], 4) = 4258
If I ignore the first 12 bytes that were written, I end up with a 4246 byte packet, which matches the warning message:
kernel: igb: caps=(0x0000000390114bb3, 0x0000000000000000) len=4246 data_len=4180 gso_size=1440 gso_type=5 ip_summed=0
Looking at the code ( https://github.com/torvalds/linux/blob/68e370289c29e3beac99d59c6d840d470af9dfcf/drivers/net/tun.c#L1037 ) it seems that the tun device is expecting a virtio_net_hdr, but that structure is only 10 bytes long ( http://lxr.free-electrons.com/source/include/uapi/linux/virtio_net.h#L73 ). I'm assuming the last two bytes are padding, because then the rest of the structure decodes okay:
flags = 0gso_type = VIRTIO_NET_HDR_GSO_TCPV4hdr_len = 66gso_size = 1440csum_start = 0csum_offset = 0
This matches what the warning message says, so I'm fairly confident in it. If I decode the remainder of the write call (ignoring the 2 bytes after the header), I'm left with a perfectly normal looking TCP packet (with a 4180 byte payload).
Looking at the packet itself, I see a valid IP checksum, and a valid TCP checksum. So, it seems like FreeBSD is calculating the packet checksums correctly, but I'm unsure of why Linux isn't noticing that. I thought it might be related to VIRTIO_NET_HDR_F_DATA_VALID, but I can't seem to find any uses of this that seem relevant (not that FreeBSD sets it anyway).
Shouldn't the tun code be setting ip_summed after receiving a packet with a valid checksum? It's not clear to me where ip_summed should be getting set.
Quoted 40 lines · Brian Rak
I managed to intercept the raw packet + headers being delivered to the tun device, though I'm having some trouble making sense of it. I've got this call:
writev(33, [{"\x00\x01\x42\x00\xa0\x05\x00\x00\x00\x00\x00\x00", 12}, .... ], 4) = 4258
If I ignore the first 12 bytes that were written, I end up with a 4246 byte packet, which matches the warning message:
kernel: igb: caps=(0x0000000390114bb3,
0x0000000000000000) len=4246 data_len=4180 gso_size=1440 gso_type=5 ip_summed=0Looking at the code (
https://github.com/torvalds/linux/blob/68e370289c29e3beac99d59c6d840d470af9dfcf/drivers/net/tun.c#L1037
) it seems that the tun device is expecting avirtio_net_hdr, but that structure is only 10 bytes long (
http://lxr.free-electrons.com/source/include/uapi/linux/virtio_net.h#L73
). I'm assuming the last two bytes are padding, because then the rest of the structure decodes okay:flags = 0
gso_type=VIRTIO_NET_HDR_GSO_TCPV4hdr_len= 66gso_size= 1440csum_start= 0csum_offset= 0This matches what the warning message says, so I'm fairly confident in it. If I decode the remainder of the write call (ignoring the 2 bytes after the header), I'm left with a perfectly normal looking TCP packet (with a 4180 byte payload).
Looking at the packet itself, I see a valid IP checksum, and a valid TCP checksum. So, it seems like FreeBSD is calculating the packet checksums correctly, but I'm unsure of why Linux isn't noticing that. I thought it might be related to
VIRTIO_NET_HDR_F_DATA_VALID, but I can't seem to find any uses of this that seem relevant (not that FreeBSD sets it anyway).
Quoted 3 lines
Shouldn't the tun code be setting
ip_summedafter receiving a packet with a valid checksum? It's not clear to me whereip_summedshould be getting set.
You need VIRTIO_NET_HDR_F_NEEDS_CSUM, and to provide proper csum_start & csum_offset
Quoted 41 lines · Eric Dumazet
Quoted 39 lines · Brian Rak
I managed to intercept the raw packet + headers being delivered to the tun device, though I'm having some trouble making sense of it. I've got this call:
writev(33, [{"\x00\x01\x42\x00\xa0\x05\x00\x00\x00\x00\x00\x00", 12}, .... ], 4) = 4258
If I ignore the first 12 bytes that were written, I end up with a 4246 byte packet, which matches the warning message:
kernel: igb: caps=(0x0000000390114bb3,
0x0000000000000000) len=4246 data_len=4180 gso_size=1440 gso_type=5 ip_summed=0Looking at the code (
https://github.com/torvalds/linux/blob/68e370289c29e3beac99d59c6d840d470af9dfcf/drivers/net/tun.c#L1037
) it seems that the tun device is expecting avirtio_net_hdr, but that structure is only 10 bytes long (
http://lxr.free-electrons.com/source/include/uapi/linux/virtio_net.h#L73
). I'm assuming the last two bytes are padding, because then the rest of the structure decodes okay:flags = 0
gso_type=VIRTIO_NET_HDR_GSO_TCPV4hdr_len= 66gso_size= 1440csum_start= 0csum_offset= 0This matches what the warning message says, so I'm fairly confident in it. If I decode the remainder of the write call (ignoring the 2 bytes after the header), I'm left with a perfectly normal looking TCP packet (with a 4180 byte payload).
Looking at the packet itself, I see a valid IP checksum, and a valid TCP checksum. So, it seems like FreeBSD is calculating the packet checksums correctly, but I'm unsure of why Linux isn't noticing that. I thought it might be related to
VIRTIO_NET_HDR_F_DATA_VALID, but I can't seem to find any uses of this that seem relevant (not that FreeBSD sets it anyway).
Quoted 6 lines
Quoted 3 lines
Shouldn't the tun code be setting
ip_summedafter receiving a packet with a valid checksum? It's not clear to me whereip_summedshould be getting set.You need
VIRTIO_NET_HDR_F_NEEDS_CSUM, and to provide propercsum_start&csum_offset
Why? The packet doesn't need a checksum (it already has a valid one). Does 'VIRTIO_NET_HDR_F_NEEDS_CSUM' not mean 'this packet needs a checksum calculated'?
Quoted 123 lines · Brian Rak
Quoted 94 lines · Vlad Yasevich
Quoted 93 lines · Brian Rak
We have a number of machines running qemu with bridged networking. We have noticed that *sometimes* FreeBSD guests cause this warning to flood the host "WARNING: CPU: 5 PID: 3705 at
net/core/dev.c:2238skb_warn_bad_offload+0xc3/0xd0()". I haven't been able to come up with any sort of reproduction steps, it just seems to happen to some FreeBSD guests, but not others.A full stack trace looks like this:
------------[ cut here ]------------ WARNING: CPU: 1 PID: 7147 at net/core/dev.c:2233 skb_warn_bad_offload+0xc3/0xd0() igb: caps=(0x0000000190114bb3, 0x0000000000000000) len=2962 data_len=0 gso_size=1448 gso_type=5 ip_summed=0 Modules linked in: dm_snapshot dm_bufio ipmi_devintf xt_physdev ebt_arp ebt_ip ebtable_natebtables
cls_fwsch_sfqsch_htbtunkvm_intelkvm 8021q garpnfnetlink_queuenfnetlink_lognfnetlink bluetooth rfkill bridge stp llcxt_CHECKSUMiptable_mangleipt_REJECTiptable_filterip_tablesip6t_REJECTip6table_filterip6_tablesipv6iTCO_wdtiTCO_vendor_supportipmi_siipmi_msghandlermicrocode pcspkri2c_i801joydev sglpc_ichshpchp igb dca ptppps_corehwmon ext4 jbd2 mbcachesd_modcrc_t10difcrct10dif_commonvideo ahci libahcixhci_hcdast ttmdrm_kms_helpersysimgblt sysfillrect syscopyareadm_mirrordm_region_hashdm_logdm_modCPU: 1 PID: 7147 Comm: qemu-kvm Tainted: G W 3.15.5-1.el6.elrepo.x86_64 #1 Hardware name: Supermicro X10SLE-F/HF/X10SLE, BIOS 1.1 07/19/2013 00000000000008b9 ffff88081fc435d8 ffffffff8163ba90 00000000000008b9 ffff88081fc43628 ffff88081fc43618 ffffffff8106c30c ffffc90007a06e30 0000000000000000 ffff8807f2b64000 ffff8807f2b64000 0000000000000000 Call Trace: <IRQ> [<ffffffff8163ba90>] dump_stack+0x49/0x61 [<ffffffff8106c30c>] warn_slowpath_common+0x8c/0xc0 [<ffffffff8106c3f6>] warn_slowpath_fmt+0x46/0x50 [<ffffffff8156ce93>] skb_warn_bad_offload+0xc3/0xd0 [<ffffffff81574a29>] ? dev_hard_start_xmit+0x339/0x640 [<ffffffff81574699>] __skb_gso_segment+0x89/0xe0 [<ffffffff81574876>] dev_hard_start_xmit+0x186/0x640 [<ffffffff81594f5a>] sch_direct_xmit+0xfa/0x1d0 [<ffffffff81574f2f>] __dev_queue_xmit+0x1ff/0x4f0 [<ffffffff81575240>] dev_queue_xmit+0x10/0x20 [<ffffffffa02e6612>] br_dev_queue_push_xmit+0x82/0xb0 [bridge] [<ffffffffa02ee680>] br_nf_dev_queue_xmit+0x20/0x90 [bridge] [<ffffffffa02ef4b8>] br_nf_post_routing+0x2d8/0x300 [bridge] [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffffa02ee6f0>] ? br_nf_dev_queue_xmit+0x90/0x90 [bridge] [<ffffffffa02e6b43>] br_forward_finish+0x43/0x60 [bridge] [<ffffffffa02ee8a8>] br_nf_forward_finish+0x1b8/0x1d0 [bridge] [<ffffffffa02ef178>] br_nf_forward_ip+0x3a8/0x410 [bridge] [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffffa02e66e4>] __br_forward+0xa4/0x100 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e67d6>] br_forward+0x96/0xb0 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e7997>] br_handle_frame_finish+0x197/0x3f0 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02ef790>] br_nf_pre_routing_finish+0x2b0/0x370 [bridge] [<ffffffffa02ef4e0>] ? br_nf_post_routing+0x300/0x300 [bridge] [<ffffffffa02ed986>] NF_HOOK_THRESH+0x56/0x60 [bridge] [<ffffffffa02eed2b>] br_nf_pre_routing+0x2fb/0x3a0 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e7d8c>] br_handle_frame+0x19c/0x240 [bridge] [<ffffffffa02e7bf0>] ? br_handle_frame_finish+0x3f0/0x3f0 [bridge] [<ffffffff81572fa5>] __netif_receive_skb_core+0x1e5/0x620 [<ffffffff81573407>] __netif_receive_skb+0x27/0x70 [<ffffffff81573553>] process_backlog+0x103/0x200 [<ffffffff81573d62>] net_rx_action+0x112/0x2a0 [<ffffffff8107111c>] __do_softirq+0xfc/0x2b0 [<ffffffff810713cd>] ? irq_exit+0xad/0xd0 [<ffffffff8164a81c>] do_softirq_own_stack+0x1c/0x30 <EOI> [<ffffffff81070e75>] do_softirq+0x55/0x60 [<ffffffff81571e19>] netif_rx_ni+0x39/0x70 [<ffffffffa03e84e0>] tun_get_user+0x310/0x6c0 [tun] [<ffffffffa03e8995>] tun_chr_aio_write+0x85/0xa0 [tun] [<ffffffff811beb9d>] do_sync_readv_writev+0x4d/0x80 [<ffffffff811c0128>] do_readv_writev+0xc8/0x2c0 [<ffffffff811bebd0>] ? do_sync_readv_writev+0x80/0x80 [<ffffffff811d2c45>] ? poll_select_set_timeout+0x95/0xb0 [<ffffffff811c0357>] vfs_writev+0x37/0x50 [<ffffffff811c0496>] SyS_writev+0x56/0xf0 [<ffffffff81648ee9>] system_call_fastpath+0x16/0x1b ---[ end trace d26e70ba037ab631 ]---gso_type=5 and ip_summed=0 are always the same (though len,
data_len, andgso_sizevary).Quoted 7 lines
Quoted 1 line
What is causing this?
The reason that the warning is triggered is
ip_summed= 0 which means there is not checksum already in the packet and it needs to be calculated. If the packet is GSO, then it needs to have partial checksum set (ip_summed== 3).You might try using systemtap or instrumenting tun and bridge to see what the
ip_summedvalue is when this happens.Who needs systemtap when you have strace ;)
I managed to intercept the raw packet + headers being delivered to the tun device, though I'm having some trouble making sense of it. I've got this call:
writev(33, [{"\x00\x01\x42\x00\xa0\x05\x00\x00\x00\x00\x00\x00", 12}, .... ], 4) = 4258
If I ignore the first 12 bytes that were written, I end up with a 4246 byte packet, which matches the warning message:
kernel: igb: caps=(0x0000000390114bb3,
0x0000000000000000) len=4246 data_len=4180 gso_size=1440 gso_type=5 ip_summed=0Looking at the code (
https://github.com/torvalds/linux/blob/68e370289c29e3beac99d59c6d840d470af9dfcf/drivers/net/tun.c#L1037
) it seems that the tun device is expecting avirtio_net_hdr, but that structure is only 10 bytes long ( http://lxr.free-electrons.com/source/include/uapi/linux/virtio_net.h#L73
). I'm assuming the last two bytes are padding, because then the rest of the structure decodes okay:
Quoted 6 lines
flags = 0
gso_type=VIRTIO_NET_HDR_GSO_TCPV4hdr_len= 66gso_size= 1440csum_start= 0csum_offset= 0
This isn't right. Like Eric said, the flags should be set VIRTIO_NET_HDR_F_NEEDS_CSUM (1), and the csum_start and csum_offset should be set.
Quoted 10 lines
This matches what the warning message says, so I'm fairly confident in it. If I decode the remainder of the write call (ignoring the 2 bytes after the header), I'm left with a perfectly normal looking TCP packet (with a 4180 byte payload).
Looking at the packet itself, I see a valid IP checksum, and a valid TCP checksum. So, it seems like FreeBSD is calculating the packet checksums correctly, but I'm unsure of why Linux isn't noticing that. I thought it might be related to
VIRTIO_NET_HDR_F_DATA_VALID, but I can't seem to find any uses of this that seem relevant (not that FreeBSD sets it anyway).
Linux is looking at the flags to see what it needs to do. With flags = 0, it means Linux will have to compute the whole checksum all by itself.
When the code hits the linux segmentation to break the 4K packet into MSS chunks, it seem that there is no partial checksum computed and thus throws the warning you see.
It is rather pointless for BSD to compute the TCP checksum for the whole 4K packet, only to have linux host recompute it for every segment.
Looks like these are some bugs in the BSD virio-net implementation.
Quoted 3 lines
Shouldn't the tun code be setting
ip_summedafter receiving a packet with a valid checksum? It's not clear to me whereip_summedshould be getting set.
tun code with set the value of ip_summed based on the flags passed it.
-vlad
Quoted 149 lines · Vlad Yasevich
Quoted 122 lines · Brian Rak
Quoted 94 lines · Vlad Yasevich
Quoted 93 lines · Brian Rak
We have a number of machines running qemu with bridged networking. We have noticed that *sometimes* FreeBSD guests cause this warning to flood the host "WARNING: CPU: 5 PID: 3705 at
net/core/dev.c:2238skb_warn_bad_offload+0xc3/0xd0()". I haven't been able to come up with any sort of reproduction steps, it just seems to happen to some FreeBSD guests, but not others.A full stack trace looks like this:
------------[ cut here ]------------ WARNING: CPU: 1 PID: 7147 at net/core/dev.c:2233 skb_warn_bad_offload+0xc3/0xd0() igb: caps=(0x0000000190114bb3, 0x0000000000000000) len=2962 data_len=0 gso_size=1448 gso_type=5 ip_summed=0 Modules linked in: dm_snapshot dm_bufio ipmi_devintf xt_physdev ebt_arp ebt_ip ebtable_natebtables
cls_fwsch_sfqsch_htbtunkvm_intelkvm 8021q garpnfnetlink_queuenfnetlink_lognfnetlink bluetooth rfkill bridge stp llcxt_CHECKSUMiptable_mangleipt_REJECTiptable_filterip_tablesip6t_REJECTip6table_filterip6_tablesipv6iTCO_wdtiTCO_vendor_supportipmi_siipmi_msghandlermicrocode pcspkri2c_i801joydev sglpc_ichshpchp igb dca ptppps_corehwmon ext4 jbd2 mbcachesd_modcrc_t10difcrct10dif_commonvideo ahci libahcixhci_hcdast ttmdrm_kms_helpersysimgblt sysfillrect syscopyareadm_mirrordm_region_hashdm_logdm_modCPU: 1 PID: 7147 Comm: qemu-kvm Tainted: G W 3.15.5-1.el6.elrepo.x86_64 #1 Hardware name: Supermicro X10SLE-F/HF/X10SLE, BIOS 1.1 07/19/2013 00000000000008b9 ffff88081fc435d8 ffffffff8163ba90 00000000000008b9 ffff88081fc43628 ffff88081fc43618 ffffffff8106c30c ffffc90007a06e30 0000000000000000 ffff8807f2b64000 ffff8807f2b64000 0000000000000000 Call Trace: <IRQ> [<ffffffff8163ba90>] dump_stack+0x49/0x61 [<ffffffff8106c30c>] warn_slowpath_common+0x8c/0xc0 [<ffffffff8106c3f6>] warn_slowpath_fmt+0x46/0x50 [<ffffffff8156ce93>] skb_warn_bad_offload+0xc3/0xd0 [<ffffffff81574a29>] ? dev_hard_start_xmit+0x339/0x640 [<ffffffff81574699>] __skb_gso_segment+0x89/0xe0 [<ffffffff81574876>] dev_hard_start_xmit+0x186/0x640 [<ffffffff81594f5a>] sch_direct_xmit+0xfa/0x1d0 [<ffffffff81574f2f>] __dev_queue_xmit+0x1ff/0x4f0 [<ffffffff81575240>] dev_queue_xmit+0x10/0x20 [<ffffffffa02e6612>] br_dev_queue_push_xmit+0x82/0xb0 [bridge] [<ffffffffa02ee680>] br_nf_dev_queue_xmit+0x20/0x90 [bridge] [<ffffffffa02ef4b8>] br_nf_post_routing+0x2d8/0x300 [bridge] [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e6590>] ? deliver_clone+0x60/0x60 [bridge] [<ffffffffa02ee6f0>] ? br_nf_dev_queue_xmit+0x90/0x90 [bridge] [<ffffffffa02e6b43>] br_forward_finish+0x43/0x60 [bridge] [<ffffffffa02ee8a8>] br_nf_forward_finish+0x1b8/0x1d0 [bridge] [<ffffffffa02ef178>] br_nf_forward_ip+0x3a8/0x410 [bridge] [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e6b00>] ? br_flood_deliver+0x20/0x20 [bridge] [<ffffffffa02e66e4>] __br_forward+0xa4/0x100 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e67d6>] br_forward+0x96/0xb0 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e7997>] br_handle_frame_finish+0x197/0x3f0 [bridge] [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02ef790>] br_nf_pre_routing_finish+0x2b0/0x370 [bridge] [<ffffffffa02ef4e0>] ? br_nf_post_routing+0x300/0x300 [bridge] [<ffffffffa02ed986>] NF_HOOK_THRESH+0x56/0x60 [bridge] [<ffffffffa02eed2b>] br_nf_pre_routing+0x2fb/0x3a0 [bridge] [<ffffffff815a357e>] nf_iterate+0x8e/0xc0 [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffff815a37ad>] nf_hook_slow+0x7d/0x150 [<ffffffffa02e7800>] ? NF_HOOK.clone.0+0x70/0x70 [bridge] [<ffffffffa02e7d8c>] br_handle_frame+0x19c/0x240 [bridge] [<ffffffffa02e7bf0>] ? br_handle_frame_finish+0x3f0/0x3f0 [bridge] [<ffffffff81572fa5>] __netif_receive_skb_core+0x1e5/0x620 [<ffffffff81573407>] __netif_receive_skb+0x27/0x70 [<ffffffff81573553>] process_backlog+0x103/0x200 [<ffffffff81573d62>] net_rx_action+0x112/0x2a0 [<ffffffff8107111c>] __do_softirq+0xfc/0x2b0 [<ffffffff810713cd>] ? irq_exit+0xad/0xd0 [<ffffffff8164a81c>] do_softirq_own_stack+0x1c/0x30 <EOI> [<ffffffff81070e75>] do_softirq+0x55/0x60 [<ffffffff81571e19>] netif_rx_ni+0x39/0x70 [<ffffffffa03e84e0>] tun_get_user+0x310/0x6c0 [tun] [<ffffffffa03e8995>] tun_chr_aio_write+0x85/0xa0 [tun] [<ffffffff811beb9d>] do_sync_readv_writev+0x4d/0x80 [<ffffffff811c0128>] do_readv_writev+0xc8/0x2c0 [<ffffffff811bebd0>] ? do_sync_readv_writev+0x80/0x80 [<ffffffff811d2c45>] ? poll_select_set_timeout+0x95/0xb0 [<ffffffff811c0357>] vfs_writev+0x37/0x50 [<ffffffff811c0496>] SyS_writev+0x56/0xf0 [<ffffffff81648ee9>] system_call_fastpath+0x16/0x1b ---[ end trace d26e70ba037ab631 ]---gso_type=5 and ip_summed=0 are always the same (though len,
data_len, andgso_sizevary).Quoted 7 lines
Quoted 1 line
What is causing this?
The reason that the warning is triggered is
ip_summed= 0 which means there is not checksum already in the packet and it needs to be calculated. If the packet is GSO, then it needs to have partial checksum set (ip_summed== 3).You might try using systemtap or instrumenting tun and bridge to see what the
ip_summedvalue is when this happens.Who needs systemtap when you have strace ;)
I managed to intercept the raw packet + headers being delivered to the tun device, though I'm having some trouble making sense of it. I've got this call:
writev(33, [{"\x00\x01\x42\x00\xa0\x05\x00\x00\x00\x00\x00\x00", 12}, .... ], 4) = 4258
If I ignore the first 12 bytes that were written, I end up with a 4246 byte packet, which matches the warning message:
kernel: igb: caps=(0x0000000390114bb3,
0x0000000000000000) len=4246 data_len=4180 gso_size=1440 gso_type=5 ip_summed=0Looking at the code (
https://github.com/torvalds/linux/blob/68e370289c29e3beac99d59c6d840d470af9dfcf/drivers/net/tun.c#L1037
) it seems that the tun device is expecting avirtio_net_hdr, but that structure is only 10 bytes long ( http://lxr.free-electrons.com/source/include/uapi/linux/virtio_net.h#L73
). I'm assuming the last two bytes are padding, because then the rest of the structure decodes okay:Quoted 6 lines
flags = 0
gso_type=VIRTIO_NET_HDR_GSO_TCPV4hdr_len= 66gso_size= 1440csum_start= 0csum_offset= 0This isn't right. Like Eric said, the flags should be set
VIRTIO_NET_HDR_F_NEEDS_CSUM(1), and thecsum_startandcsum_offsetshould be set.Quoted 9 lines
This matches what the warning message says, so I'm fairly confident in it. If I decode the remainder of the write call (ignoring the 2 bytes after the header), I'm left with a perfectly normal looking TCP packet (with a 4180 byte payload).
Looking at the packet itself, I see a valid IP checksum, and a valid TCP checksum. So, it seems like FreeBSD is calculating the packet checksums correctly, but I'm unsure of why Linux isn't noticing that. I thought it might be related to
VIRTIO_NET_HDR_F_DATA_VALID, but I can't seem to find any uses of this that seem relevant (not that FreeBSD sets it anyway).Linux is looking at the flags to see what it needs to do. With flags = 0, it means Linux will have to compute the whole checksum all by itself.
When the code hits the linux segmentation to break the 4K packet into MSS chunks, it seem that there is no partial checksum computed and thus throws the warning you see.
It is rather pointless for BSD to compute the TCP checksum for the whole 4K packet, only to have linux host recompute it for every segment.
Quoted 7 lines
Looks like these are some bugs in the BSD virio-net implementation.
Quoted 2 lines
Shouldn't the tun code be setting
ip_summedafter receiving a packet with a valid checksum? It's not clear to me whereip_summedshould be getting set.tun code with set the value of
ip_summedbased on the flags passed it.-vlad
Thanks, that explination makes sense to me. I'll contact the FreeBSD developers and see if they can correct the issue.