6.6.15-rc1 review
Subsystems (7)
Greg Kroah-Hartman
maintainerPATCH 114/331 arm64: entry: fix ARM64_WORKAROUND_SPECULATIVE_UNPRIV_LOAD
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Mark Rutland <[email protected]>
commit 832dd634bd1b4e3bbe9f10b9c9ba5db6f6f2b97f upstream.
Currently the ARM64_WORKAROUND_SPECULATIVE_UNPRIV_LOAD workaround isn't quite right, as it is supposed to be applied after the last explicit memory access, but is immediately followed by an LDR.
The ARM64_WORKAROUND_SPECULATIVE_UNPRIV_LOAD workaround is used to handle Cortex-A520 erratum 2966298 and Cortex-A510 erratum 3117295, which are described in:
* https://developer.arm.com/documentation/SDEN2444153/0600/?lang=en
* https://developer.arm.com/documentation/SDEN1873361/1600/?lang=en
In both cases the workaround is described as:
| If pagetable isolation is disabled, the context switch logic in the | kernel can be updated to execute the following sequence on affected | cores before exiting to EL0, and after all explicit memory accesses: | | 1. A non-shareable TLBI to any context and/or address, including | unused contexts or addresses, such as a `TLBI VALE1 Xzr`. | | 2. A DSB NSH to guarantee completion of the TLBI.
The important part being that the TLBI+DSB must be placed "after all explicit memory accesses".
Unfortunately, as-implemented, the TLBI+DSB is immediately followed by an LDR, as we have:
| alternative_if ARM64_WORKAROUND_SPECULATIVE_UNPRIV_LOAD | tlbi vale1, xzr | dsb nsh | alternative_else_nop_endif | alternative_if_not ARM64_UNMAP_KERNEL_AT_EL0 | ldr lr, [sp, #S_LR] | add sp, sp, #PT_REGS_SIZE // restore sp | eret | alternative_else_nop_endif | | [ ... KPTI exception return path ... ]
This patch fixes this by reworking the logic to place the TLBI+DSB immediately before the ERET, after all explicit memory accesses.
The ERET is currently in a separate alternative block, and alternatives cannot be nested. To account for this, the alternative block for ARM64_UNMAP_KERNEL_AT_EL0 is replaced with a single alternative branch to skip the KPTI logic, with the new shape of the logic being:
| alternative_insn "b .L_skip_tramp_exit_\@", nop, ARM64_UNMAP_KERNEL_AT_EL0 | [ ... KPTI exception return path ... ] | .L_skip_tramp_exit_\@: | | ldr lr, [sp, #S_LR] | add sp, sp, #PT_REGS_SIZE // restore sp | | alternative_if ARM64_WORKAROUND_SPECULATIVE_UNPRIV_LOAD | tlbi vale1, xzr | dsb nsh | alternative_else_nop_endif | eret
The new structure means that the workaround is only applied when KPTI is not in use; this is fine as noted in the documented implications of the erratum:
| Pagetable isolation between EL0 and higher level ELs prevents the | issue from occurring.
... and as per the workaround description quoted above, the workaround is only necessary "If pagetable isolation is disabled".
--- arch/arm64/kernel/entry.S | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)
arch/arm64/kernel/entry.S +13 -9
@@ -428,16 +428,9 @@ alternative_else_nop_endif ldp x28, x29, [sp, #16 * 14] .if \el == 0 -alternative_if ARM64_WORKAROUND_SPECULATIVE_UNPRIV_LOAD - tlbi vale1, xzr - dsb nsh -alternative_else_nop_endif -alternative_if_not ARM64_UNMAP_KERNEL_AT_EL0 - ldr lr, [sp, #S_LR] - add sp, sp, #PT_REGS_SIZE // restore sp - eret -alternative_else_nop_endif #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + alternative_insn "b .L_skip_tramp_exit_\@", nop, ARM64_UNMAP_KERNEL_AT_EL0 + msr far_el1, x29 ldr_this_cpu x30, this_cpu_vector, x29
@@ -446,7 +439,18 @@ alternative_else_nop_endif ldr lr, [sp, #S_LR] // restore x30 add sp, sp, #PT_REGS_SIZE // restore sp br x29 + +.L_skip_tramp_exit_\@: #endif + ldr lr, [sp, #S_LR] + add sp, sp, #PT_REGS_SIZE // restore sp + + /* This must be after the last explicit memory access */ +alternative_if ARM64_WORKAROUND_SPECULATIVE_UNPRIV_LOAD + tlbi vale1, xzr + dsb nsh +alternative_else_nop_endif + eret .else ldr lr, [sp, #S_LR] add sp, sp, #PT_REGS_SIZE // restore sp
Greg Kroah-Hartman
maintainerPATCH 115/331 rtc: cmos: Use ACPI alarm for non-Intel x86 systems too
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Mario Limonciello <[email protected]>
commit 3d762e21d56370a43478b55e604b4a83dd85aafc upstream.
Intel systems > 2015 have been configured to use ACPI alarm instead of HPET to avoid s2idle issues.
Having HPET programmed for wakeup causes problems on AMD systems with s2idle as well.
One particular case is that the systemd "SuspendThenHibernate" feature doesn't work properly on the Framework 13" AMD model. Switching to using ACPI alarm fixes the issue.
Adjust the quirk to apply to AMD/Hygon systems from 2021 onwards. This matches what has been tested and is specifically to avoid potential risk to older systems.
--- drivers/rtc/rtc-cmos.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
drivers/rtc/rtc-cmos.c +12 -6
@@ -818,18 +818,24 @@ static void rtc_wake_off(struct device * }#ifdef CONFIG_X86 -/* Enable use_acpi_alarm mode for Intel platforms no earlier than 2015 */ static void use_acpi_alarm_quirks (void ) { -if (boot_cpu_data.x86_vendor !=X86_VENDOR_INTEL ) +switch (boot_cpu_data.x86_vendor) { +case X86_VENDOR_INTEL : +if (dmi_get_bios_year () <2015 ) +return ; +break ; +case X86_VENDOR_AMD : +case X86_VENDOR_HYGON : +if (dmi_get_bios_year () <2021 ) +return ; +break ; +default :return ; - + }if (!is_hpet_enabled ())return ; -if (dmi_get_bios_year () <2015 ) -return ; - use_acpi_alarm = true; }#else
Greg Kroah-Hartman
maintainerPATCH 116/331 rtc: Adjust failure return code for cmos_set_alarm()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Mario Limonciello <[email protected]>
commit 1311a8f0d4b23f58bbababa13623aa40b8ad4e0c upstream.
When mc146818_avoid_UIP() fails to return a valid value, this is because UIP didn't clear in the timeout period. Adjust the return code in this case to -ETIMEDOUT.
--- drivers/rtc/rtc-cmos.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/rtc/rtc-cmos.c +2 -2
@@ -292,7 +292,7 @@ static int cmos_read_alarm(struct device/* This not only a rtc_op, but also called directly */ if (!is_valid_irq (cmos->irq)) -return -EIO ; +return -ETIMEDOUT ;/* Basic alarms only support hour, minute, and seconds fields. * Some also support day and month, for alarms up to a year in
@@ -557,7 +557,7 @@ static int cmos_set_alarm(struct device* Use mc146818_avoid_UIP() to avoid this. */ if (!mc146818_avoid_UIP (cmos_set_alarm_callback, &p)) -return -EIO ; +return -ETIMEDOUT ; cmos->alarm_expires =rtc_tm_to_time64 (&t->time);
Greg Kroah-Hartman
maintainerPATCH 117/331 rtc: mc146818-lib: Adjust failure return code for mc146818_get_time()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Mario Limonciello <[email protected]>
commit af838635a3eb9b1bc0d98599c101ebca98f31311 upstream.
mc146818_get_time() calls mc146818_avoid_UIP() to avoid fetching the time while RTC update is in progress (UIP). When this fails, the return code is -EIO, but actually there was no IO failure.
The reason for the return from mc146818_avoid_UIP() is that the UIP wasn't cleared in the time period. Adjust the return code to -ETIMEDOUT to match the behavior.
--- drivers/rtc/rtc-mc146818-lib.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/rtc/rtc-mc146818-lib.c +1 -1
@@ -138,7 +138,7 @@ int mc146818_get_time(struct rtc_time *tif (!mc146818_avoid_UIP (mc146818_get_time_callback, &p)) {memset (time,0 ,sizeof (*time)); -return -EIO ; +return -ETIMEDOUT ; }if (!(p.ctrl &RTC_DM_BINARY ) ||RTC_ALWAYS_BCD )
Greg Kroah-Hartman
maintainerPATCH 118/331 rtc: Add support for configuring the UIP timeout for RTC reads
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Mario Limonciello <[email protected]>
commit 120931db07b49252aba2073096b595482d71857c upstream.
The UIP timeout is hardcoded to 10ms for all RTC reads, but in some contexts this might not be enough time. Add a timeout parameter to mc146818_get_time() and mc146818_get_time_callback().
If UIP timeout is configured by caller to be >=100 ms and a call takes this long, log a warning.
Make all callers use 10ms to ensure no functional changes.
--- arch/alpha/kernel/rtc.c | 2 +- arch/x86/kernel/hpet.c | 2 +- arch/x86/kernel/rtc.c | 2 +- drivers/base/power/trace.c | 2 +- drivers/rtc/rtc-cmos.c | 6 +++--- drivers/rtc/rtc-mc146818-lib.c | 37 +++++++++++++++++++++++++++++-------- include/linux/mc146818rtc.h | 3 ++- 7 files changed, 38 insertions(+), 16 deletions(-)
arch/alpha/kernel/rtc.c +1 -1
@@ -80,7 +80,7 @@ init_rtc_epoch(void)static int alpha_rtc_read_time (struct device *dev,struct rtc_time *tm) { -int ret =mc146818_get_time (tm); +int ret =mc146818_get_time (tm,10 );if (ret <0 ) {dev_err_ratelimited (dev,"unable to read current time\n" );
arch/x86/kernel/hpet.c +1 -1
@@ -1438,7 +1438,7 @@ irqreturn_t hpet_rtc_interrupt(int irq,memset (&curr_time,0 ,sizeof (struct rtc_time ));if (hpet_rtc_flags & (RTC_UIE |RTC_AIE )) { -if (unlikely (mc146818_get_time (&curr_time) <0 )) { +if (unlikely (mc146818_get_time (&curr_time,10 ) <0 )) {pr_err_ratelimited ("unable to read current time from RTC\n" );return IRQ_HANDLED ; }
arch/x86/kernel/rtc.c +1 -1
@@ -67,7 +67,7 @@ void mach_get_cmos_time(struct timespec6return ; } -if (mc146818_get_time (&tm)) { +if (mc146818_get_time (&tm,10 )) {pr_err ("Unable to read current time from RTC\n" ); now->tv_sec = now->tv_nsec =0 ;return ;
drivers/base/power/trace.c +1 -1
@@ -120,7 +120,7 @@ static unsigned int read_magic_time(voidstruct rtc_time time;unsigned int val; -if (mc146818_get_time (&time) <0 ) { +if (mc146818_get_time (&time,10 ) <0 ) {pr_err ("Unable to read current time from RTC\n" );return 0 ; }
drivers/rtc/rtc-cmos.c +3 -3
@@ -231,7 +231,7 @@ static int cmos_read_time(struct deviceif (!pm_trace_rtc_valid ())return -EIO ; - ret =mc146818_get_time (t); + ret =mc146818_get_time (t,10 );if (ret <0 ) {dev_err_ratelimited (dev,"unable to read current time\n" );return ret;
@@ -307,7 +307,7 @@ static int cmos_read_alarm(struct device* * Use the mc146818_avoid_UIP() function to avoid this. */ -if (!mc146818_avoid_UIP (cmos_read_alarm_callback, &p)) +if (!mc146818_avoid_UIP (cmos_read_alarm_callback,10 , &p))return -EIO ;if (!(p.rtc_control &RTC_DM_BINARY ) ||RTC_ALWAYS_BCD ) {
@@ -556,7 +556,7 @@ static int cmos_set_alarm(struct device* * Use mc146818_avoid_UIP() to avoid this. */ -if (!mc146818_avoid_UIP (cmos_set_alarm_callback, &p)) +if (!mc146818_avoid_UIP (cmos_set_alarm_callback,10 , &p))return -ETIMEDOUT ; cmos->alarm_expires =rtc_tm_to_time64 (&t->time);
drivers/rtc/rtc-mc146818-lib.c +29 -8
@@ -8,26 +8,31 @@#include <linux/acpi.h> #endif +#define UIP_RECHECK_DELAY 100/* usec */ +#define UIP_RECHECK_DELAY_MS (USEC_PER_MSEC / UIP_RECHECK_DELAY) +#define UIP_RECHECK_LOOPS_MS (x) (x / UIP_RECHECK_DELAY_MS) +/* * Execute a function while the UIP (Update-in-progress) bit of the RTC is -* unset. +* unset. The timeout is configurable by the caller in ms. * * Warning: callback may be executed more then once. */ bool mc146818_avoid_UIP (void (*callback)(unsigned char seconds,void *param), +int timeout,void *param) {int i;unsigned long flags;unsigned char seconds; -for (i =0 ; i <100 ; i++) { +for (i =0 ;UIP_RECHECK_LOOPS_MS (i) < timeout; i++) {spin_lock_irqsave (&rtc_lock, flags);/* * Check whether there is an update in progress during which the * readout is unspecified. The maximum update time is ~2ms. Poll -* every 100 usec for completion. +* for completion. * * Store the second value before checking UIP so a long lasting * NMI which happens to hit after the UIP check cannot make
@@ -37,7 +42,7 @@ bool mc146818_avoid_UIP(void (*callback)if (CMOS_READ (RTC_FREQ_SELECT ) &RTC_UIP ) {spin_unlock_irqrestore (&rtc_lock, flags); -udelay (100 ); +udelay (UIP_RECHECK_DELAY );continue ; }
@@ -56,7 +61,7 @@ bool mc146818_avoid_UIP(void (*callback)*/ if (CMOS_READ (RTC_FREQ_SELECT ) &RTC_UIP ) {spin_unlock_irqrestore (&rtc_lock, flags); -udelay (100 ); +udelay (UIP_RECHECK_DELAY );continue ; }
@@ -72,6 +77,10 @@ bool mc146818_avoid_UIP(void (*callback) }spin_unlock_irqrestore (&rtc_lock, flags); +if (UIP_RECHECK_LOOPS_MS (i) >=100 ) +pr_warn ("Reading current time from RTC took around %li ms\n" , +UIP_RECHECK_LOOPS_MS (i)); +return true; }return false;
@@ -84,7 +93,7 @@ EXPORT_SYMBOL_GPL(mc146818_avoid_UIP);*/ bool mc146818_does_rtc_work (void ) { -return mc146818_avoid_UIP (NULL ,NULL ); +return mc146818_avoid_UIP (NULL ,10 ,NULL ); }EXPORT_SYMBOL_GPL (mc146818_does_rtc_work);
@@ -130,13 +139,25 @@ static void mc146818_get_time_callback(u p->ctrl =CMOS_READ (RTC_CONTROL ); } -int mc146818_get_time (struct rtc_time *time) +/** +* mc146818_get_time - Get the current time from the RTC +* @time: pointer to struct rtc_time to store the current time +* @timeout: timeout value in ms +* +* This function reads the current time from the RTC and stores it in the +* provided struct rtc_time. The timeout parameter specifies the maximum +* time to wait for the RTC to become ready. +* +* Return: 0 on success, -ETIMEDOUT if the RTC did not become ready within +* the specified timeout, or another error code if an error occurred. +*/ +int mc146818_get_time (struct rtc_time *time,int timeout) {struct mc146818_get_time_callback_param p = { .time = time }; -if (!mc146818_avoid_UIP (mc146818_get_time_callback, &p)) { +if (!mc146818_avoid_UIP (mc146818_get_time_callback, timeout, &p)) {memset (time,0 ,sizeof (*time));return -ETIMEDOUT ; }
include/linux/mc146818rtc.h +2 -1
@@ -126,10 +126,11 @@ struct cmos_rtc_board_info {#endif /* ARCH_RTC_LOCATION */ bool mc146818_does_rtc_work (void ); -int mc146818_get_time (struct rtc_time *time); +int mc146818_get_time (struct rtc_time *time,int timeout);int mc146818_set_time (struct rtc_time *time);bool mc146818_avoid_UIP (void (*callback)(unsigned char seconds,void *param), +int timeout,void *param);#endif /* _MC146818RTC_H */
Greg Kroah-Hartman
maintainerPATCH 119/331 rtc: Extend timeout for waiting for UIP to clear to 1s
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Mario Limonciello <[email protected]>
commit cef9ecc8e938dd48a560f7dd9be1246359248d20 upstream.
Specs don't say anything about UIP being cleared within 10ms. They only say that UIP won't occur for another 244uS. If a long NMI occurs while UIP is still updating it might not be possible to get valid data in 10ms.
This has been observed in the wild that around s2idle some calls can take up to 480ms before UIP is clear.
Adjust callers from outside an interrupt context to wait for up to a 1s instead of 10ms.
--- arch/x86/kernel/rtc.c | 2 +- drivers/base/power/trace.c | 2 +- drivers/rtc/rtc-cmos.c | 2 +- drivers/rtc/rtc-mc146818-lib.c | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/kernel/rtc.c +1 -1
@@ -67,7 +67,7 @@ void mach_get_cmos_time(struct timespec6return ; } -if (mc146818_get_time (&tm,10 )) { +if (mc146818_get_time (&tm,1000 )) {pr_err ("Unable to read current time from RTC\n" ); now->tv_sec = now->tv_nsec =0 ;return ;
drivers/base/power/trace.c +1 -1
@@ -120,7 +120,7 @@ static unsigned int read_magic_time(voidstruct rtc_time time;unsigned int val; -if (mc146818_get_time (&time,10 ) <0 ) { +if (mc146818_get_time (&time,1000 ) <0 ) {pr_err ("Unable to read current time from RTC\n" );return 0 ; }
drivers/rtc/rtc-cmos.c +1 -1
@@ -231,7 +231,7 @@ static int cmos_read_time(struct deviceif (!pm_trace_rtc_valid ())return -EIO ; - ret =mc146818_get_time (t,10 ); + ret =mc146818_get_time (t,1000 );if (ret <0 ) {dev_err_ratelimited (dev,"unable to read current time\n" );return ret;
drivers/rtc/rtc-mc146818-lib.c +1 -1
@@ -93,7 +93,7 @@ EXPORT_SYMBOL_GPL(mc146818_avoid_UIP);*/ bool mc146818_does_rtc_work (void ) { -return mc146818_avoid_UIP (NULL ,10 ,NULL ); +return mc146818_avoid_UIP (NULL ,1000 ,NULL ); }EXPORT_SYMBOL_GPL (mc146818_does_rtc_work);
Greg Kroah-Hartman
maintainerPATCH 120/331 nouveau/vmm: dont set addr on the fail path to avoid warning
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Dave Airlie <[email protected]>
commit cacea81390fd8c8c85404e5eb2adeb83d87a912e upstream.
nvif_vmm_put gets called if addr is set, but if the allocation fails we don't need to call put, otherwise we get a warning like
[523232.435671] ------------[ cut here ]------------ [523232.435674] WARNING: CPU: 8 PID: 1505697 at drivers/gpu/drm/nouveau/nvif/vmm.c:68 nvif_vmm_put+0x72/0x80 [nouveau] [523232.435795] Modules linked in: uinput rfcomm snd_seq_dummy snd_hrtimer nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink qrtr bnep sunrpc binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common isst_if_common iwlmvm nfit libnvdimm vfat fat x86_pkg_temp_thermal intel_powerclamp mac80211 snd_soc_avs snd_soc_hda_codec coretemp snd_hda_ext_core snd_soc_core snd_hda_codec_realtek kvm_intel snd_hda_codec_hdmi snd_compress snd_hda_codec_generic ac97_bus snd_pcm_dmaengine snd_hda_intel libarc4 snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec kvm iwlwifi snd_hda_core btusb snd_hwdep btrtl snd_seq btintel irqbypass btbcm rapl snd_seq_device eeepc_wmi btmtk intel_cstate iTCO_wdt cfg80211 snd_pcm asus_wmi bluetooth intel_pmc_bxt iTCO_vendor_support snd_timer ledtrig_audio pktcdvd snd mei_me [523232.435828] sparse_keymap intel_uncore i2c_i801 platform_profile wmi_bmof mei pcspkr ioatdma soundcore i2c_smbus rfkill idma64 dca joydev acpi_tad loop zram nouveau drm_ttm_helper ttm video drm_exec drm_gpuvm gpu_sched crct10dif_pclmul i2c_algo_bit nvme crc32_pclmul crc32c_intel drm_display_helper polyval_clmulni nvme_core polyval_generic e1000e mxm_wmi cec ghash_clmulni_intel r8169 sha512_ssse3 nvme_common wmi pinctrl_sunrisepoint uas usb_storage ip6_tables ip_tables fuse [523232.435849] CPU: 8 PID: 1505697 Comm: gnome-shell Tainted: G W 6.6.0-rc7-nvk-uapi+ #12 [523232.435851] Hardware name: System manufacturer System Product Name/ROG STRIX X299-E GAMING II, BIOS 1301 09/24/2021 [523232.435852] RIP: 0010:nvif_vmm_put+0x72/0x80 [nouveau] [523232.435934] Code: 00 00 48 89 e2 be 02 00 00 00 48 c7 04 24 00 00 00 00 48 89 44 24 08 e8 fc bf ff ff 85 c0 75 0a 48 c7 43 08 00 00 00 00 eb b3 <0f> 0b eb f2 e8 f5 c9 b2 e6 0f 1f 44 00 00 90 90 90 90 90 90 90 90 [523232.435936] RSP: 0018:ffffc900077ffbd8 EFLAGS: 00010282 [523232.435937] RAX: 00000000fffffffe RBX: ffffc900077ffc00 RCX: 0000000000000010 [523232.435938] RDX: 0000000000000010 RSI: ffffc900077ffb38 RDI: ffffc900077ffbd8 [523232.435940] RBP: ffff888e1c4f2140 R08: 0000000000000000 R09: 0000000000000000 [523232.435940] R10: 0000000000000000 R11: 0000000000000000 R12: ffff888503811800 [523232.435941] R13: ffffc900077ffca0 R14: ffff888e1c4f2140 R15: ffff88810317e1e0 [523232.435942] FS: 00007f933a769640(0000) GS:ffff88905fa00000(0000) knlGS:0000000000000000 [523232.435943] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [523232.435944] CR2: 00007f930bef7000 CR3: 00000005d0322001 CR4: 00000000003706e0 [523232.435945] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [523232.435946] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [523232.435964] Call Trace: [523232.435965] <TASK> [523232.435966] ? nvif_vmm_put+0x72/0x80 [nouveau] [523232.436051] ? __warn+0x81/0x130 [523232.436055] ? nvif_vmm_put+0x72/0x80 [nouveau] [523232.436138] ? report_bug+0x171/0x1a0 [523232.436142] ? handle_bug+0x3c/0x80 [523232.436144] ? exc_invalid_op+0x17/0x70 [523232.436145] ? asm_exc_invalid_op+0x1a/0x20 [523232.436149] ? nvif_vmm_put+0x72/0x80 [nouveau] [523232.436230] ? nvif_vmm_put+0x64/0x80 [nouveau] [523232.436342] nouveau_vma_del+0x80/0xd0 [nouveau] [523232.436506] nouveau_vma_new+0x1a0/0x210 [nouveau] [523232.436671] nouveau_gem_object_open+0x1d0/0x1f0 [nouveau] [523232.436835] drm_gem_handle_create_tail+0xd1/0x180 [523232.436840] drm_prime_fd_to_handle_ioctl+0x12e/0x200 [523232.436844] ? __pfx_drm_prime_fd_to_handle_ioctl+0x10/0x10 [523232.436847] drm_ioctl_kernel+0xd3/0x180 [523232.436849] drm_ioctl+0x26d/0x4b0 [523232.436851] ? __pfx_drm_prime_fd_to_handle_ioctl+0x10/0x10 [523232.436855] nouveau_drm_ioctl+0x5a/0xb0 [nouveau] [523232.437032] __x64_sys_ioctl+0x94/0xd0 [523232.437036] do_syscall_64+0x5d/0x90 [523232.437040] ? syscall_exit_to_user_mode+0x2b/0x40 [523232.437044] ? do_syscall_64+0x6c/0x90 [523232.437046] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
--- drivers/gpu/drm/nouveau/nouveau_vmm.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/gpu/drm/nouveau/nouveau_vmm.c +3 -0
@@ -108,6 +108,9 @@ nouveau_vma_new(struct nouveau_bo *nvbo, }else { ret =nvif_vmm_get (&vmm->vmm,PTES , false, mem->mem.page,0 , mem->mem.size, &tmp); +if (ret) + goto done; + vma->addr = tmp.addr; }
Greg Kroah-Hartman
maintainerPATCH 121/331 efi: disable mirror feature during crashkernel
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Ma Wupeng <[email protected]>
commit 7ea6ec4c25294e8bc8788148ef854df92ee8dc5e upstream.
If the system has no mirrored memory or uses crashkernel.high while kernelcore=mirror is enabled on the command line then during crashkernel, there will be limited mirrored memory and this usually leads to OOM.
To solve this problem, disable the mirror feature during crashkernel.
--- mm/mm_init.c | 6 ++++++ 1 file changed, 6 insertions(+)
mm/mm_init.c +6 -0
@@ -26,6 +26,7 @@#include <linux/pgtable.h> #include <linux/swap.h> #include <linux/cma.h> +#include <linux/crash_dump.h> #include "internal.h" #include "slab.h" #include "shuffle.h"
@@ -381,6 +382,11 @@ static void __init find_zone_movable_pfn goto out; } +if (is_kdump_kernel ()) { +pr_warn ("The system is under kdump, ignore kernelcore=mirror.\n" ); + goto out; + } +for_each_mem_region (r) {if (memblock_is_mirror (r))continue ;
Greg Kroah-Hartman
maintainerPATCH 122/331 ubifs: ubifs_symlink: Fix memleak of inode->i_link in error path
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Zhihao Cheng <[email protected]>
commit 1e022216dcd248326a5bb95609d12a6815bca4e2 upstream.
For error handling path in ubifs_symlink(), inode will be marked as bad first, then iput() is invoked. If inode->i_link is initialized by fscrypt_encrypt_symlink() in encryption scenario, inode->i_link won't be freed by callchain ubifs_free_inode -> fscrypt_free_inode in error handling path, because make_bad_inode() has changed 'inode->i_mode' as 'S_IFREG'.
Following kmemleak is easy to be reproduced by injecting error in ubifs_jnl_update() when doing symlink in encryption scenario:
unreferenced object 0xffff888103da3d98 (size 8):
comm "ln", pid 1692, jiffies 4294914701 (age 12.045s) backtrace:
kmemdup+0x32/0x70 __fscrypt_encrypt_symlink+0xed/0x1c0 ubifs_symlink+0x210/0x300 [ubifs] vfs_symlink+0x216/0x360 do_symlinkat+0x11a/0x190 do_syscall_64+0x3b/0xe0
There are two ways fixing it:
1. Remove make_bad_inode() in error handling path. We can do that because ubifs_evict_inode() will do same processes for good symlink inode and bad symlink inode, for inode->i_nlink checking is before is_bad_inode().
2. Free inode->i_link before marking inode bad.
Method 2 is picked, it has less influence, personally, I think.
--- fs/ubifs/dir.c | 2 ++ 1 file changed, 2 insertions(+)
fs/ubifs/dir.c +2 -0
@@ -1225,6 +1225,8 @@ out_cancel: dir_ui->ui_size = dir->i_size;mutex_unlock (&dir_ui->ui_mutex); out_inode: +/* Free inode->i_link before inode is marked as bad. */ +fscrypt_free_inode (inode);make_bad_inode (inode);iput (inode); out_fname:
Greg Kroah-Hartman
maintainerPATCH 123/331 kexec: do syscore_shutdown() in kernel_kexec
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: James Gowans <[email protected]>
commit 7bb943806ff61e83ae4cceef8906b7fe52453e8a upstream.
syscore_shutdown() runs driver and module callbacks to get the system into a state where it can be correctly shut down. In commit 6f389a8f1dd2 ("PM / reboot: call syscore_shutdown() after disable_nonboot_cpus()") syscore_shutdown() was removed from kernel_restart_prepare() and hence got (incorrectly?) removed from the kexec flow. This was innocuous until commit 6735150b6997 ("KVM: Use syscore_ops instead of reboot_notifier to hook restart/shutdown") changed the way that KVM registered its shutdown callbacks, switching from reboot notifiers to syscore_ops.shutdown. As syscore_shutdown() is missing from kexec, KVM's shutdown hook is not run and virtualisation is left enabled on the boot CPU which results in triple faults when switching to the new kernel on Intel x86 VT-x with VMXE enabled.
Fix this by adding syscore_shutdown() to the kexec sequence. In terms of where to add it, it is being added after migrating the kexec task to the boot CPU, but before APs are shut down. It is not totally clear if this is the best place: in commit 6f389a8f1dd2 ("PM / reboot: call syscore_shutdown() after disable_nonboot_cpus()") it is stated that "syscore_ops operations should be carried with one CPU on-line and interrupts disabled." APs are only offlined later in machine_shutdown(), so this syscore_shutdown() is being run while APs are still online. This seems to be the correct place as it matches where syscore_shutdown() is run in the reboot and halt flows - they also run it before APs are shut down. The assumption is that the commit message in commit 6f389a8f1dd2 ("PM / reboot: call syscore_shutdown() after disable_nonboot_cpus()") is no longer valid.
KVM has been discussed here as it is what broke loudly by not having syscore_shutdown() in kexec, but this change impacts more than just KVM; all drivers/modules which register a syscore_ops.shutdown callback will now be invoked in the kexec flow. Looking at some of them like x86 MCE it is probably more correct to also shut these down during kexec. Maintainers of all drivers which use syscore_ops.shutdown are added on CC for visibility. They are:
arch/powerpc/platforms/cell/spu_base.c .shutdown = spu_shutdown,arch/x86/kernel/cpu/mce/core.c .shutdown = mce_syscore_shutdown,arch/x86/kernel/i8259.c .shutdown = i8259A_shutdown,drivers/irqchip/irq-i8259.c .shutdown = i8259A_shutdown,drivers/irqchip/irq-sun6i-r.c .shutdown = sun6i_r_intc_shutdown,drivers/leds/trigger/ledtrig-cpu.c .shutdown = ledtrig_cpu_syscore_shutdown,drivers/power/reset/sc27xx-poweroff.c .shutdown = sc27xx_poweroff_shutdown,kernel/irq/generic-chip.c .shutdown = irq_gc_shutdown,virt/kvm/kvm_main.c .shutdown = kvm_shutdown,
This has been tested by doing a kexec on x86_64 and aarch64.
--- kernel/kexec_core.c | 1 + 1 file changed, 1 insertion(+)
kernel/kexec_core.c +1 -0
@@ -1271,6 +1271,7 @@ int kernel_kexec(void) kexec_in_progress = true;kernel_restart_prepare ("kexec reboot" );migrate_to_reboot_cpu (); +syscore_shutdown ();/* * migrate_to_reboot_cpu() disables CPU hotplug assuming that
Greg Kroah-Hartman
maintainerPATCH 124/331 selftests: mm: hugepage-vmemmap fails on 64K page size systems
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Donet Tom <[email protected]>
commit 00bcfcd47a52f50f07a2e88d730d7931384cb073 upstream.
The kernel sefltest mm/hugepage-vmemmap fails on architectures which has different page size other than 4K. In hugepage-vmemmap page size used is 4k so the pfn calculation will go wrong on systems which has different page size .The length of MAP_HUGETLB memory must be hugepage aligned but in hugepage-vmemmap map length is 2M so this will not get aligned if the system has differnet hugepage size.
Added psize() to get the page size and default_huge_page_size() to get the default hugepage size at run time, hugepage-vmemmap test pass on powerpc with 64K page size and x86 with 4K page size.
Result on powerpc without patch (page size 64K)
*# ./hugepage-vmemmap
Returned address is 0x7effff000000 whose pfn is 0
Head page flags (100000000) is invalidcheck_page_flags: Invalid argument
*#
Result on powerpc with patch (page size 64K) *# ./hugepage-vmemmap
Returned address is 0x7effff000000 whose pfn is 600 *#
Result on x86 with patch (page size 4K)
*# ./hugepage-vmemmap
Returned address is 0x7fc7c2c00000 whose pfn is 1dac00 *#
--- tools/testing/selftests/mm/hugepage-vmemmap.c | 29 ++++++++++++++++---------- 1 file changed, 18 insertions(+), 11 deletions(-)
tools/testing/selftests/mm/hugepage-vmemmap.c +18 -11
@@ -10,10 +10,7 @@#include <unistd.h> #include <sys/mman.h> #include <fcntl.h> - -#define MAP_LENGTH (2UL * 1024 * 1024) - -#define PAGE_SIZE 4096 +#include "vm_util.h" #define PAGE_COMPOUND_HEAD (1UL << 15)#define PAGE_COMPOUND_TAIL (1UL << 16)
@@ -39,6 +36,9 @@#define MAP_FLAGS (MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB)#endif +static size_t pagesize; +static size_t maplength; +static void write_bytes (char *addr,size_t length) {unsigned long i;
@@ -56,7 +56,7 @@ static unsigned long virt_to_pfn(void *aif (fd <0 )return -1UL ; -lseek (fd, (unsigned long )addr /PAGE_SIZE *sizeof (pagemap),SEEK_SET ); +lseek (fd, (unsigned long )addr / pagesize *sizeof (pagemap),SEEK_SET );read (fd, &pagemap,sizeof (pagemap));close (fd);
@@ -86,7 +86,7 @@ static int check_page_flags(unsigned lon* this also verifies kernel has correctly set the fake page_head to tail * while hugetlb_free_vmemmap is enabled. */ -for (i =1 ; i <MAP_LENGTH /PAGE_SIZE ; i++) { +for (i =1 ; i < maplength / pagesize; i++) {read (fd, &pageflags,sizeof (pageflags));if ((pageflags &TAIL_PAGE_FLAGS ) !=TAIL_PAGE_FLAGS || (pageflags &HEAD_PAGE_FLAGS ) ==HEAD_PAGE_FLAGS ) {
@@ -106,18 +106,25 @@ int main(int argc, char **argv)void *addr;unsigned long pfn; - addr =mmap (MAP_ADDR ,MAP_LENGTH ,PROT_READ |PROT_WRITE ,MAP_FLAGS ,-1 ,0 ); + pagesize =psize (); + maplength =default_huge_page_size (); +if (!maplength) { +printf ("Unable to determine huge page size\n" ); +exit (1 ); + } + + addr =mmap (MAP_ADDR , maplength,PROT_READ |PROT_WRITE ,MAP_FLAGS ,-1 ,0 );if (addr ==MAP_FAILED ) {perror ("mmap" );exit (1 ); }/* Trigger allocation of HugeTLB page. */ -write_bytes (addr,MAP_LENGTH ); +write_bytes (addr, maplength); pfn =virt_to_pfn (addr);if (pfn ==-1UL ) { -munmap (addr,MAP_LENGTH ); +munmap (addr, maplength);perror ("virt_to_pfn" );exit (1 ); }
@@ -125,13 +132,13 @@ int main(int argc, char **argv)printf ("Returned address is %p whose pfn is %lx\n" , addr, pfn);if (check_page_flags (pfn) <0 ) { -munmap (addr,MAP_LENGTH ); +munmap (addr, maplength);perror ("check_page_flags" );exit (1 ); }/* munmap() length of MAP_HUGETLB memory must be hugepage aligned */ -if (munmap (addr,MAP_LENGTH )) { +if (munmap (addr, maplength)) {perror ("munmap" );exit (1 ); }
Greg Kroah-Hartman
maintainerPATCH 125/331 mm/rmap: fix misplaced parenthesis of a likely()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Steven Rostedt (Google) <[email protected]>
commit f67f8d4a8c1e1ebc85a6cbdb9a7266f14863461c upstream.
Running my yearly branch profiler to see where likely/unlikely annotation may be added or removed, I discovered this:
correct incorrect % Function File Line
------- --------- - -------- ---- ----
0 457918 100 page_try_dup_anon_rmap rmap.h 264
[..]
458021 0 0 page_try_dup_anon_rmap rmap.h 265
I thought it was interesting that line 264 of rmap.h had a 100% incorrect annotation, but the line directly below it was 100% correct. Looking at the code:
if (likely(!is_device_private_page(page) && unlikely(page_needs_cow_for_dma(vma, page))))
It didn't make sense. The "likely()" was around the entire if statement (not just the "!is_device_private_page(page)"), which also included the "unlikely()" portion of that if condition.
If the unlikely portion is unlikely to be true, that would make the entire if condition unlikely to be true, so it made no sense at all to say the entire if condition is true.
What is more likely to be likely is just the first part of the if statement before the && operation. It's likely to be a misplaced parenthesis. And after making the if condition broken into a likely() && unlikely(), both now appear to be correct!
--- include/linux/rmap.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/rmap.h +2 -2
@@ -261,8 +261,8 @@ static inline int page_try_dup_anon_rmap* guarantee the pinned page won't be randomly replaced in the * future on write faults. */ -if (likely (!is_device_private_page (page) && -unlikely (page_needs_cow_for_dma (vma, page)))) +if (likely (!is_device_private_page (page)) && +unlikely (page_needs_cow_for_dma (vma, page)))return -EBUSY ;ClearPageAnonExclusive (page);
Greg Kroah-Hartman
maintainerPATCH 126/331 mm/sparsemem: fix race in accessing memory_section->usage
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Charan Teja Kalla <[email protected]>
commit 5ec8e8ea8b7783fab150cf86404fc38cb4db8800 upstream.
The below race is observed on a PFN which falls into the device memory region with the system memory configuration where PFN's are such that [ZONE_NORMAL ZONE_DEVICE ZONE_NORMAL]. Since normal zone start and end pfn contains the device memory PFN's as well, the compaction triggered will try on the device memory PFN's too though they end up in NOP(because pfn_to_online_page() returns NULL for ZONE_DEVICE memory sections). When from other core, the section mappings are being removed for the ZONE_DEVICE region, that the PFN in question belongs to, on which compaction is currently being operated is resulting into the kernel crash with CONFIG_SPASEMEM_VMEMAP enabled. The crash logs can be seen at [1].
compact_zone() memunmap_pages
------------- ---------------__pageblock_pfn_to_page
......
(a)pfn_valid():
valid_section()//return true
(b)__remove_pages()->
sparse_remove_section()->
section_deactivate():
[Free the array ms->usage and set
ms->usage = NULL]
pfn_section_valid()
[Access ms->usage which
is NULL]NOTE: From the above it can be said that the race is reduced to between the pfn_valid()/pfn_section_valid() and the section deactivate with SPASEMEM_VMEMAP enabled.
The commit b943f045a9af("mm/sparse: fix kernel crash with pfn_section_valid check") tried to address the same problem by clearing the SECTION_HAS_MEM_MAP with the expectation of valid_section() returns false thus ms->usage is not accessed.
Fix this issue by the below steps:
a) Clear SECTION_HAS_MEM_MAP before freeing the ->usage.
b) RCU protected read side critical section will either return NULL when SECTION_HAS_MEM_MAP is cleared or can successfully access ->usage.
c) Free the ->usage with kfree_rcu() and set ms->usage = NULL. No attempt will be made to access ->usage after this as the SECTION_HAS_MEM_MAP is cleared thus valid_section() return false.
Thanks to David/Pavan for their inputs on this patch.
[1] https://lore.kernel.org/linux-mm/[email protected]/
On Snapdragon SoC, with the mentioned memory configuration of PFN's as [ZONE_NORMAL ZONE_DEVICE ZONE_NORMAL], we are able to see bunch of issues daily while testing on a device farm.
For this particular issue below is the log. Though the below log is not directly pointing to the pfn_section_valid(){ ms->usage;}, when we loaded this dump on T32 lauterbach tool, it is pointing.
[ 540.578056] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 [ 540.578068] Mem abort info: [ 540.578070] ESR = 0x0000000096000005 [ 540.578073] EC = 0x25: DABT (current EL), IL = 32 bits [ 540.578077] SET = 0, FnV = 0 [ 540.578080] EA = 0, S1PTW = 0 [ 540.578082] FSC = 0x05: level 1 translation fault [ 540.578085] Data abort info: [ 540.578086] ISV = 0, ISS = 0x00000005 [ 540.578088] CM = 0, WnR = 0 [ 540.579431] pstate: 82400005 (Nzcv daif +PAN -UAO +TCO -DIT -SSBSBTYPE=--) [ 540.579436] pc : __pageblock_pfn_to_page+0x6c/0x14c [ 540.579454] lr : compact_zone+0x994/0x1058 [ 540.579460] sp : ffffffc03579b510 [ 540.579463] x29: ffffffc03579b510 x28: 0000000000235800 x27:000000000000000c [ 540.579470] x26: 0000000000235c00 x25: 0000000000000068 x24:ffffffc03579b640 [ 540.579477] x23: 0000000000000001 x22: ffffffc03579b660 x21:0000000000000000 [ 540.579483] x20: 0000000000235bff x19: ffffffdebf7e3940 x18:ffffffdebf66d140 [ 540.579489] x17: 00000000739ba063 x16: 00000000739ba063 x15:00000000009f4bff [ 540.579495] x14: 0000008000000000 x13: 0000000000000000 x12:0000000000000001 [ 540.579501] x11: 0000000000000000 x10: 0000000000000000 x9 :ffffff897d2cd440 [ 540.579507] x8 : 0000000000000000 x7 : 0000000000000000 x6 :ffffffc03579b5b4 [ 540.579512] x5 : 0000000000027f25 x4 : ffffffc03579b5b8 x3 :0000000000000001 [ 540.579518] x2 : ffffffdebf7e3940 x1 : 0000000000235c00 x0 :0000000000235800 [ 540.579524] Call trace: [ 540.579527] __pageblock_pfn_to_page+0x6c/0x14c [ 540.579533] compact_zone+0x994/0x1058 [ 540.579536] try_to_compact_pages+0x128/0x378 [ 540.579540] __alloc_pages_direct_compact+0x80/0x2b0 [ 540.579544] __alloc_pages_slowpath+0x5c0/0xe10 [ 540.579547] __alloc_pages+0x250/0x2d0 [ 540.579550] __iommu_dma_alloc_noncontiguous+0x13c/0x3fc [ 540.579561] iommu_dma_alloc+0xa0/0x320 [ 540.579565] dma_alloc_attrs+0xd4/0x108
--- include/linux/mmzone.h | 14 +++++++++++--- mm/sparse.c | 17 +++++++++-------- 2 files changed, 20 insertions(+), 11 deletions(-)
include/linux/mmzone.h +11 -3
@@ -1774,6 +1774,7 @@ static inline unsigned long section_nr_t#define SUBSECTION_ALIGN_DOWN (pfn) ((pfn) & PAGE_SUBSECTION_MASK)struct mem_section_usage { +struct rcu_head rcu;#ifdef CONFIG_SPARSEMEM_VMEMMAP DECLARE_BITMAP(subsection_map, SUBSECTIONS_PER_SECTION);#endif
@@ -1967,7 +1968,7 @@ static inline int pfn_section_valid(stru {int idx =subsection_map_index (pfn); -return test_bit (idx, ms->usage->subsection_map); +return test_bit (idx,READ_ONCE (ms->usage)->subsection_map); }#else static inline int pfn_section_valid (struct mem_section *ms,unsigned long pfn)
@@ -1991,6 +1992,7 @@ static inline int pfn_section_valid(strustatic inline int pfn_valid (unsigned long pfn) {struct mem_section *ms; +int ret;/* * Ensure the upper PAGE_SHIFT bits are clear in the
@@ -2004,13 +2006,19 @@ static inline int pfn_valid(unsigned lonif (pfn_to_section_nr (pfn) >=NR_MEM_SECTIONS )return 0 ; ms =__pfn_to_section (pfn); -if (!valid_section (ms)) +rcu_read_lock (); +if (!valid_section (ms)) { +rcu_read_unlock ();return 0 ; + }/* * Traditionally early sections always returned pfn_valid() for * the entire section-sized span. */ - return early_section(ms) || pfn_section_valid(ms, pfn); + ret =early_section (ms) ||pfn_section_valid (ms, pfn); +rcu_read_unlock (); + + return ret; }#endif
mm/sparse.c +9 -8
@@ -792,6 +792,13 @@ static void section_deactivate(unsignedunsigned long section_nr =pfn_to_section_nr (pfn);/* +* Mark the section invalid so that valid_section() +* return false. This prevents code from dereferencing +* ms->usage array. +*/ + ms->section_mem_map &= ~SECTION_HAS_MEM_MAP ; + +/* * When removing an early section, the usage map is kept (as the * usage maps of other sections fall into the same page). It * will be re-used when re-adding the section - which is then no
@@ -799,16 +806,10 @@ static void section_deactivate(unsigned* was allocated during boot. */ if (!PageReserved (virt_to_page (ms->usage))) { -kfree (ms->usage); - ms->usage =NULL ; +kfree_rcu (ms->usage, rcu); +WRITE_ONCE (ms->usage,NULL ); } memmap =sparse_decode_mem_map (ms->section_mem_map, section_nr); -/* -* Mark the section invalid so that valid_section() -* return false. This prevents code from dereferencing -* ms->usage array. -*/ - ms->section_mem_map &= ~SECTION_HAS_MEM_MAP ; }/*
Greg Kroah-Hartman
maintainerPATCH 127/331 rename(): fix the locking of subdirectories
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Al Viro <[email protected]>
commit 22e111ed6c83dcde3037fc81176012721bc34c0b upstream.
We should never lock two subdirectories without having taken
->s_vfs_rename_mutex; inode pointer order or not, the "order" proposed in 28eceeda130f "fs: Lock moved directories" is not transitive, with the usual consequences.
The rationale for locking renamed subdirectory in all cases was
the possibility of race between rename modifying .. in a subdirectory to reflect the new parent and another thread modifying the same subdirectory. For a lot of filesystems that's not a problem, but for some it can lead to trouble (e.g. the case when short directory contents is kept in the inode, but creating a file in it might push it across the size limit and copy its contents into separate data block(s)).
However, we need that only in case when the parent does change -
otherwise ->rename() doesn't need to do anything with .. entry in the first place. Some instances are lazy and do a tautological update anyway, but it's really not hard to avoid.
Amended locking rules for rename():
find the parent(s) of source and target
if source and target have the same parent
lock the common parent
else
lock ->s_vfs_rename_mutex
lock both parents, in ancestor-first order; if neither is an ancestor of another, lock the parent of source first.
find the source and target.
if source and target have the same parent
if operation is an overwriting rename of a subdirectory
lock the target subdirectory
else
if source is a subdirectory
lock the source
if target is a subdirectory
lock the target
lock non-directories involved, in inode pointer order if both source and target are such.
That way we are guaranteed that parents are locked (for obvious reasons), that any renamed non-directory is locked (nfsd relies upon that), that any victim is locked (emptiness check needs that, among other things) and subdirectory that changes parent is locked (needed to protect the update of .. entries). We are also guaranteed that any operation locking more than one directory either takes ->s_vfs_rename_mutex or locks a parent followed by its child.
--- Documentation/filesystems/directory-locking.rst | 29 ++++++----- Documentation/filesystems/locking.rst | 5 +- Documentation/filesystems/porting.rst | 18 +++++++ fs/namei.c | 60 ++++++++++++++---------- 4 files changed, 74 insertions(+), 38 deletions(-)
Documentation/filesystems/directory-locking.rst +15 -14
@@ -22,13 +22,16 @@ exclusive. 3) object removal. Locking rules: caller locks parent, finds victim, locks victim and calls the method. Locks are exclusive. -4) rename() that is _not_ cross-directory. Locking rules: caller locks the -parent and finds source and target. We lock both (provided they exist). If we -need to lock two inodes of different type (dir vs non-dir), we lock directory -first. If we need to lock two inodes of the same type, lock them in inode -pointer order. Then call the method. All locks are exclusive. -NB: we might get away with locking the source (and target in exchange -case) shared. +4) rename() that is _not_ cross-directory. Locking rules: caller locks +the parent and finds source and target. Then we decide which of the +source and target need to be locked. Source needs to be locked if it's a +non-directory; target - if it's a non-directory or about to be removed. +Take the locks that need to be taken, in inode pointer order if need +to take both (that can happen only when both source and target are +non-directories - the source because it wouldn't be locked otherwise +and the target because mixing directory and non-directory is allowed +only with RENAME_EXCHANGE, and that won't be removing the target). +After the locks had been taken, call the method. All locks are exclusive. 5) link creation. Locking rules:
@@ -44,20 +47,17 @@ rules: * lock the filesystem * lock parents in "ancestors first" order. If one is not ancestor of - the other, lock them in inode pointer order. + the other, lock the parent of source first. * find source and target. * if old parent is equal to or is a descendent of target fail with -ENOTEMPTY * if new parent is equal to or is a descendent of source fail with -ELOOP - * Lock both the source and the target provided they exist. If we - need to lock two inodes of different type (dir vs non-dir), we lock - the directory first. If we need to lock two inodes of the same type, - lock them in inode pointer order. + * Lock subdirectories involved (source before target). + * Lock non-directories involved, in inode pointer order. * call the method. -All ->i_rwsem are taken exclusive. Again, we might get away with locking -the source (and target in exchange case) shared. +All ->i_rwsem are taken exclusive. The rules above obviously guarantee that all directories that are going to be read, modified or removed by method will be locked by caller.
@@ -67,6 +67,7 @@ If no directory is its own ancestor, the Proof: +[XXX: will be updated once we are done massaging the lock_rename()] First of all, at any moment we have a linear ordering of the objects - A < B iff (A is an ancestor of B) or (B is not an ancestor of A and ptr(A) < ptr(B)).
Documentation/filesystems/locking.rst +4 -1
@@ -101,7 +101,7 @@ symlink: exclusive mkdir: exclusive unlink: exclusive (both) rmdir: exclusive (both)(see below) -rename: exclusive (all) (see below) +rename: exclusive (both parents, some children) (see below) readlink: no get_link: no setattr: exclusive
@@ -123,6 +123,9 @@ get_offset_ctx no Additionally, ->rmdir(), ->unlink() and ->rename() have ->i_rwsem exclusive on victim. cross-directory ->rename() has (per-superblock) ->s_vfs_rename_sem. + ->unlink() and ->rename() have ->i_rwsem exclusive on all non-directories + involved. + ->rename() has ->i_rwsem exclusive on any subdirectory that changes parent. See Documentation/filesystems/directory-locking.rst for more detailed discussion of the locking scheme for directory operations.
Documentation/filesystems/porting.rst +18 -0
@@ -1045,3 +1045,21 @@ filesystem type is now moved to a later As this is a VFS level change it has no practical consequences for filesystems other than that all of them must use one of the provided kill_litter_super(), kill_anon_super(), or kill_block_super() helpers. + +--- + +**mandatory** + +If ->rename() update of .. on cross-directory move needs an exclusion with +directory modifications, do *not* lock the subdirectory in question in your +->rename() - it's done by the caller now [that item should've been added in +28eceeda130f "fs: Lock moved directories"]. + +--- + +**mandatory** + +On same-directory ->rename() the (tautological) update of .. is not protected +by any locks; just don't do it if the old parent is the same as the new one. +We really can't lock two subdirectories in same-directory rename - not without +deadlocks.
fs/namei.c +37 -23
97 lines not on this page · raw message
Greg Kroah-Hartman
maintainerPATCH 128/331 serial: sc16is7xx: improve regmap debugfs by using one regmap per port
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Hugo Villeneuve <[email protected]>
commit 3837a0379533aabb9e4483677077479f7c6aa910 upstream.
With this current driver regmap implementation, it is hard to make sense of the register addresses displayed using the regmap debugfs interface, because they do not correspond to the actual register addresses documented in the datasheet. For example, register 1 is displayed as registers 04 thru 07:
$ cat /sys/kernel/debug/regmap/spi0.0/registers 04: 10 -> Port 0, register offset 1 05: 10 -> Port 1, register offset 1 06: 00 -> Port 2, register offset 1 -> invalid 07: 00 -> port 3, register offset 1 -> invalid ...
The reason is that bits 0 and 1 of the register address correspond to the channel (port) bits, so the register address itself starts at bit 2, and we must 'mentally' shift each register address by 2 bits to get its real address/offset.
Also, only channels 0 and 1 are supported by the chip, so channel mask combinations of 10b and 11b are invalid, and the display of these registers is useless.
This patch adds a separate regmap configuration for each port, similar to what is done in the max310x driver, so that register addresses displayed match the register addresses in the chip datasheet. Also, each port now has its own debugfs entry.
Example with new regmap implementation:
$ cat /sys/kernel/debug/regmap/spi0.0-port0/registers 1: 10 2: 01 3: 00 ...
$ cat /sys/kernel/debug/regmap/spi0.0-port1/registers 1: 10 2: 01 3: 00
As an added bonus, this also simplifies some operations (read/write/modify) because it is no longer necessary to manually shift register addresses.
--- drivers/tty/serial/sc16is7xx.c | 143 +++++++++++++++++++++++------------------ 1 file changed, 81 insertions(+), 62 deletions(-)
drivers/tty/serial/sc16is7xx.c +81 -62
@@ -301,8 +301,8 @@/* Misc definitions */ +#define SC16IS7XX_SPI_READ_BIT BIT(7)#define SC16IS7XX_FIFO_SIZE (64) -#define SC16IS7XX_REG_SHIFT 2#define SC16IS7XX_GPIOS_PER_BANK 4struct sc16is7xx_devtype {
@@ -324,6 +324,7 @@ struct sc16is7xx_one_config {struct sc16is7xx_one {struct uart_port port;u8 line; +struct regmap *regmap;struct kthread_work tx_work;struct kthread_work reg_work;struct kthread_delayed_work ms_work;
@@ -362,48 +363,37 @@ static void sc16is7xx_stop_tx(struct uar#define to_sc16is7xx_port (p,e) ((container_of((p), struct sc16is7xx_port, e)))#define to_sc16is7xx_one (p,e) ((container_of((p), struct sc16is7xx_one, e))) -static int sc16is7xx_line (struct uart_port *port) -{ -struct sc16is7xx_one *one =to_sc16is7xx_one (port, port); - -return one->line; -} -static u8 sc16is7xx_port_read (struct uart_port *port,u8 reg) { -struct sc16is7xx_port *s =dev_get_drvdata (port->dev); +struct sc16is7xx_one *one =to_sc16is7xx_one (port, port);unsigned int val =0 ; -const u8 line =sc16is7xx_line (port); -regmap_read (s->regmap, (reg <<SC16IS7XX_REG_SHIFT ) | line, &val); +regmap_read (one->regmap, reg, &val);return val; }static void sc16is7xx_port_write (struct uart_port *port,u8 reg,u8 val) { -struct sc16is7xx_port *s =dev_get_drvdata (port->dev); -const u8 line =sc16is7xx_line (port); +struct sc16is7xx_one *one =to_sc16is7xx_one (port, port); -regmap_write (s->regmap, (reg <<SC16IS7XX_REG_SHIFT ) | line, val); +regmap_write (one->regmap, reg, val); }static void sc16is7xx_fifo_read (struct uart_port *port,unsigned int rxlen) {struct sc16is7xx_port *s =dev_get_drvdata (port->dev); -const u8 line =sc16is7xx_line (port); -u8 addr = (SC16IS7XX_RHR_REG <<SC16IS7XX_REG_SHIFT ) | line; +struct sc16is7xx_one *one =to_sc16is7xx_one (port, port); -regcache_cache_bypass (s->regmap, true); -regmap_raw_read (s->regmap, addr, s->buf, rxlen); -regcache_cache_bypass (s->regmap, false); +regcache_cache_bypass (one->regmap, true); +regmap_raw_read (one->regmap,SC16IS7XX_RHR_REG , s->buf, rxlen); +regcache_cache_bypass (one->regmap, false); }static void sc16is7xx_fifo_write (struct uart_port *port,u8 to_send) {struct sc16is7xx_port *s =dev_get_drvdata (port->dev); -const u8 line =sc16is7xx_line (port); -u8 addr = (SC16IS7XX_THR_REG <<SC16IS7XX_REG_SHIFT ) | line; +struct sc16is7xx_one *one =to_sc16is7xx_one (port, port);/* * Don't send zero-length data, at least on SPI it confuses the chip
@@ -412,19 +402,17 @@ static void sc16is7xx_fifo_write(structif (unlikely (!to_send))return ; -regcache_cache_bypass (s->regmap, true); -regmap_raw_write (s->regmap, addr, s->buf, to_send); -regcache_cache_bypass (s->regmap, false); +regcache_cache_bypass (one->regmap, true); +regmap_raw_write (one->regmap,SC16IS7XX_THR_REG , s->buf, to_send); +regcache_cache_bypass (one->regmap, false); }static void sc16is7xx_port_update (struct uart_port *port,u8 reg,u8 mask,u8 val) { -struct sc16is7xx_port *s =dev_get_drvdata (port->dev); -const u8 line =sc16is7xx_line (port); +struct sc16is7xx_one *one =to_sc16is7xx_one (port, port); -regmap_update_bits (s->regmap, (reg <<SC16IS7XX_REG_SHIFT ) | line, - mask, val); +regmap_update_bits (one->regmap, reg, mask, val); }static int sc16is7xx_alloc_line (void )
@@ -479,7 +467,7 @@ static const struct sc16is7xx_devtype scstatic bool sc16is7xx_regmap_volatile (struct device *dev,unsigned int reg) { -switch (reg >>SC16IS7XX_REG_SHIFT ) { +switch (reg) {case SC16IS7XX_RHR_REG :case SC16IS7XX_IIR_REG :case SC16IS7XX_LSR_REG :
@@ -498,7 +486,7 @@ static bool sc16is7xx_regmap_volatile(ststatic bool sc16is7xx_regmap_precious (struct device *dev,unsigned int reg) { -switch (reg >>SC16IS7XX_REG_SHIFT ) { +switch (reg) {case SC16IS7XX_RHR_REG :return true;default :
@@ -511,6 +499,7 @@ static bool sc16is7xx_regmap_precious(ststatic int sc16is7xx_set_baud (struct uart_port *port,int baud) {struct sc16is7xx_port *s =dev_get_drvdata (port->dev); +struct sc16is7xx_one *one =to_sc16is7xx_one (port, port);u8 lcr;u8 prescaler =0 ;unsigned long clk = port->uartclk, div = clk /16 / baud;
@@ -542,12 +531,12 @@ static int sc16is7xx_set_baud(struct uar SC16IS7XX_LCR_CONF_MODE_B);/* Enable enhanced features */ -regcache_cache_bypass (s->regmap, true); +regcache_cache_bypass (one->regmap, true);sc16is7xx_port_update (port,SC16IS7XX_EFR_REG ,SC16IS7XX_EFR_ENABLE_BIT ,SC16IS7XX_EFR_ENABLE_BIT ); -regcache_cache_bypass (s->regmap, false); +regcache_cache_bypass (one->regmap, false);/* Put LCR back to the normal mode */ sc16is7xx_port_write (port,SC16IS7XX_LCR_REG , lcr);
@@ -563,10 +552,10 @@ static int sc16is7xx_set_baud(struct uar SC16IS7XX_LCR_CONF_MODE_A);/* Write the new divisor */ -regcache_cache_bypass (s->regmap, true); +regcache_cache_bypass (one->regmap, true);sc16is7xx_port_write (port,SC16IS7XX_DLH_REG , div /256 );sc16is7xx_port_write (port,SC16IS7XX_DLL_REG , div %256 ); -regcache_cache_bypass (s->regmap, false); +regcache_cache_bypass (one->regmap, false);/* Put LCR back to the normal mode */ sc16is7xx_port_write (port,SC16IS7XX_LCR_REG , lcr);
@@ -1092,7 +1081,7 @@ static void sc16is7xx_set_termios(struct SC16IS7XX_LCR_CONF_MODE_B);/* Configure flow control */ -regcache_cache_bypass (s->regmap, true); +regcache_cache_bypass (one->regmap, true);sc16is7xx_port_write (port,SC16IS7XX_XON1_REG , termios->c_cc[VSTART ]);sc16is7xx_port_write (port,SC16IS7XX_XOFF1_REG , termios->c_cc[VSTOP ]);
@@ -1111,7 +1100,7 @@ static void sc16is7xx_set_termios(structSC16IS7XX_EFR_REG ,SC16IS7XX_EFR_FLOWCTRL_BITS , flow); -regcache_cache_bypass (s->regmap, false); +regcache_cache_bypass (one->regmap, false);/* Update LCR register */ sc16is7xx_port_write (port,SC16IS7XX_LCR_REG , lcr);
@@ -1162,7 +1151,6 @@ static int sc16is7xx_config_rs485(structstatic int sc16is7xx_startup (struct uart_port *port) {struct sc16is7xx_one *one =to_sc16is7xx_one (port, port); -struct sc16is7xx_port *s =dev_get_drvdata (port->dev);unsigned int val;unsigned long flags;
@@ -1179,7 +1167,7 @@ static int sc16is7xx_startup(struct uartsc16is7xx_port_write (port,SC16IS7XX_LCR_REG ,SC16IS7XX_LCR_CONF_MODE_B ); -regcache_cache_bypass (s->regmap, true); +regcache_cache_bypass (one->regmap, true);/* Enable write access to enhanced features and internal clock div */ sc16is7xx_port_update(port, SC16IS7XX_EFR_REG,
@@ -1197,7 +1185,7 @@ static int sc16is7xx_startup(struct uartSC16IS7XX_TCR_RX_RESUME (24 ) | SC16IS7XX_TCR_RX_HALT(48)); -regcache_cache_bypass (s->regmap, false); +regcache_cache_bypass (one->regmap, false);/* Now, initialize the UART */ sc16is7xx_port_write (port,SC16IS7XX_LCR_REG ,SC16IS7XX_LCR_WORD_LEN_8 );
@@ -1455,7 +1443,7 @@ static int sc16is7xx_setup_mctrl_ports(sif (s->mctrl_mask)regmap_update_bits ( s->regmap, -SC16IS7XX_IOCONTROL_REG <<SC16IS7XX_REG_SHIFT , +SC16IS7XX_IOCONTROL_REG ,SC16IS7XX_IOCONTROL_MODEM_A_BIT |SC16IS7XX_IOCONTROL_MODEM_B_BIT , s->mctrl_mask);
@@ -1470,7 +1458,7 @@ static const struct serial_rs485 sc16is7static int sc16is7xx_probe (struct device *dev,const struct sc16is7xx_devtype *devtype, -struct regmap *regmap,int irq) +struct regmap *regmaps[],int irq) {unsigned long freq =0 , *pfreq =dev_get_platdata (dev);unsigned int val;
@@ -1478,16 +1466,16 @@ static int sc16is7xx_probe(struct deviceint i, ret;struct sc16is7xx_port *s; -if (IS_ERR (regmap)) -return PTR_ERR (regmap); +for (i =0 ; i < devtype->nr_uart; i++) +if (IS_ERR (regmaps[i])) +return PTR_ERR (regmaps[i]);/* * This device does not have an identification register that would * tell us if we are really connected to the correct device. * The best we can do is to check if communication is at all possible. */ - ret =regmap_read (regmap, -SC16IS7XX_LSR_REG <<SC16IS7XX_REG_SHIFT , &val); + ret =regmap_read (regmaps[0 ],SC16IS7XX_LSR_REG , &val);if (ret <0 )return -EPROBE_DEFER ;
@@ -1521,7 +1509,7 @@ static int sc16is7xx_probe(struct devicereturn -EINVAL ; } - s->regmap = regmap; + s->regmap = regmaps[0 ]; s->devtype = devtype;dev_set_drvdata (dev, s);mutex_init (&s->efr_lock);
@@ -1536,8 +1524,8 @@ static int sc16is7xx_probe(struct devicesched_set_fifo (s->kworker_task);/* reset device, purging any pending irq / data */ -regmap_write (s->regmap,SC16IS7XX_IOCONTROL_REG <<SC16IS7XX_REG_SHIFT , -SC16IS7XX_IOCONTROL_SRESET_BIT ); +regmap_write (s->regmap,SC16IS7XX_IOCONTROL_REG , +SC16IS7XX_IOCONTROL_SRESET_BIT );for (i =0 ; i < devtype->nr_uart; ++i) { s->p[i].line = i;
@@ -1561,6 +1549,7 @@ static int sc16is7xx_probe(struct device s->p[i].port.ops = &sc16is7xx_ops; s->p[i].old_mctrl =0 ; s->p[i].port.line =sc16is7xx_alloc_line (); + s->p[i].regmap = regmaps[i];if (s->p[i].port.line >=SC16IS7XX_MAX_DEVS ) { ret = -ENOMEM ;
@@ -1589,13 +1578,13 @@ static int sc16is7xx_probe(struct devicesc16is7xx_port_write (&s->p[i].port,SC16IS7XX_LCR_REG ,SC16IS7XX_LCR_CONF_MODE_B ); -regcache_cache_bypass (s->regmap, true); +regcache_cache_bypass (regmaps[i], true);/* Enable write access to enhanced features */ sc16is7xx_port_write (&s->p[i].port,SC16IS7XX_EFR_REG ,SC16IS7XX_EFR_ENABLE_BIT ); -regcache_cache_bypass (s->regmap, false); +regcache_cache_bypass (regmaps[i], false);/* Restore access to general registers */ sc16is7xx_port_write (&s->p[i].port,SC16IS7XX_LCR_REG ,0x00 );
@@ -1698,19 +1687,36 @@ static const struct of_device_id __maybeMODULE_DEVICE_TABLE (of, sc16is7xx_dt_ids);static struct regmap_config regcfg = { - .reg_bits =7 , - .pad_bits =1 , + .reg_bits =5 , + .pad_bits =3 , .val_bits =8 , .cache_type =REGCACHE_RBTREE , .volatile_reg = sc16is7xx_regmap_volatile, .precious_reg = sc16is7xx_regmap_precious, + .max_register =SC16IS7XX_EFCR_REG , }; +static const char *sc16is7xx_regmap_name (unsigned int port_id) +{ +static char buf[6 ]; + +snprintf (buf,sizeof (buf),"port%d" , port_id); + +return buf; +} + +static unsigned int sc16is7xx_regmap_port_mask (unsigned int port_id) +{ +/* CH1,CH0 are at bits 2:1. */ +return port_id <<1 ; +} +#ifdef CONFIG_SERIAL_SC16IS7XX_SPI static int sc16is7xx_spi_probe (struct spi_device *spi) {const struct sc16is7xx_devtype *devtype; -struct regmap *regmap; +struct regmap *regmaps[2 ]; +unsigned int i;int ret;/* Setup SPI bus */
@@ -1735,11 +1741,20 @@ static int sc16is7xx_spi_probe(struct sp devtype = (struct sc16is7xx_devtype *)id_entry->driver_data; } - regcfg.max_register = (0xf <<SC16IS7XX_REG_SHIFT ) | - (devtype->nr_uart -1 ); - regmap =devm_regmap_init_spi (spi, ®cfg); +for (i =0 ; i < devtype->nr_uart; i++) { + regcfg.name =sc16is7xx_regmap_name (i); +/* +* If read_flag_mask is 0, the regmap code sets it to a default +* of 0x80. Since we specify our own mask, we must add the READ +* bit ourselves: +*/ + regcfg.read_flag_mask =sc16is7xx_regmap_port_mask (i) | +SC16IS7XX_SPI_READ_BIT ; + regcfg.write_flag_mask =sc16is7xx_regmap_port_mask (i); + regmaps[i] =devm_regmap_init_spi (spi, ®cfg); + } -return sc16is7xx_probe (&spi->dev, devtype, regmap, spi->irq); +return sc16is7xx_probe (&spi->dev, devtype, regmaps, spi->irq); }static void sc16is7xx_spi_remove (struct spi_device *spi)
@@ -1778,7 +1793,8 @@ static int sc16is7xx_i2c_probe(struct i2 {const struct i2c_device_id *id =i2c_client_get_device_id (i2c);const struct sc16is7xx_devtype *devtype; -struct regmap *regmap; +struct regmap *regmaps[2 ]; +unsigned int i;if (i2c->dev.of_node) { devtype =device_get_match_data (&i2c->dev);
@@ -1788,11 +1804,14 @@ static int sc16is7xx_i2c_probe(struct i2 devtype = (struct sc16is7xx_devtype *)id->driver_data; } - regcfg.max_register = (0xf <<SC16IS7XX_REG_SHIFT ) | - (devtype->nr_uart -1 ); - regmap =devm_regmap_init_i2c (i2c, ®cfg); +for (i =0 ; i < devtype->nr_uart; i++) { + regcfg.name =sc16is7xx_regmap_name (i); + regcfg.read_flag_mask =sc16is7xx_regmap_port_mask (i); + regcfg.write_flag_mask =sc16is7xx_regmap_port_mask (i); + regmaps[i] =devm_regmap_init_i2c (i2c, ®cfg); + } -return sc16is7xx_probe (&i2c->dev, devtype, regmap, i2c->irq); +return sc16is7xx_probe (&i2c->dev, devtype, regmaps, i2c->irq); }static void sc16is7xx_i2c_remove (struct i2c_client *client)
Greg Kroah-Hartman
maintainerPATCH 129/331 serial: sc16is7xx: remove wasteful static buffer in sc16is7xx_regmap_name()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Hugo Villeneuve <[email protected]>
commit 6bcab3c8acc88e265c570dea969fd04f137c8a4c upstream.
Using a static buffer inside sc16is7xx_regmap_name() was a convenient and simple way to set the regmap name without having to allocate and free a buffer each time it is called. The drawback is that the static buffer wastes memory for nothing once regmap is fully initialized.
Remove static buffer and use constant strings instead.
This also avoids a truncation warning when using "%d" or "%u" in snprintf which was flagged by kernel test robot.
--- drivers/tty/serial/sc16is7xx.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
drivers/tty/serial/sc16is7xx.c +8 -6
@@ -1696,13 +1696,15 @@ static struct regmap_config regcfg = { .max_register = SC16IS7XX_EFCR_REG, }; -static const char *sc16is7xx_regmap_name (unsigned int port_id) +static const char *sc16is7xx_regmap_name (u8 port_id) { -static char buf[6 ]; - -snprintf (buf,sizeof (buf),"port%d" , port_id); - -return buf; +switch (port_id) { +case 0 :return "port0" ; +case 1 :return "port1" ; +default : +WARN_ON (true); +return NULL ; + } }static unsigned int sc16is7xx_regmap_port_mask (unsigned int port_id)
Greg Kroah-Hartman
maintainerPATCH 130/331 serial: sc16is7xx: remove global regmap from struct sc16is7xx_port
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Hugo Villeneuve <[email protected]>
commit f6959c5217bd799bcb770b95d3c09b3244e175c6 upstream.
Remove global struct regmap so that it is more obvious that this regmap is to be used only in the probe function.
Also add a comment to that effect in probe function.
--- drivers/tty/serial/sc16is7xx.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
drivers/tty/serial/sc16is7xx.c +9 -6
@@ -335,7 +335,6 @@ struct sc16is7xx_one {struct sc16is7xx_port {const struct sc16is7xx_devtype *devtype; -struct regmap *regmap;struct clk *clk;#ifdef CONFIG_GPIOLIB struct gpio_chip gpio;
@@ -1413,7 +1412,8 @@ static int sc16is7xx_setup_gpio_chip(str/* * Configure ports designated to operate as modem control lines. */ -static int sc16is7xx_setup_mctrl_ports (struct sc16is7xx_port *s) +static int sc16is7xx_setup_mctrl_ports (struct sc16is7xx_port *s, +struct regmap *regmap) {int i;int ret;
@@ -1442,7 +1442,7 @@ static int sc16is7xx_setup_mctrl_ports(sif (s->mctrl_mask)regmap_update_bits ( - s->regmap, + regmap,SC16IS7XX_IOCONTROL_REG ,SC16IS7XX_IOCONTROL_MODEM_A_BIT |SC16IS7XX_IOCONTROL_MODEM_B_BIT , s->mctrl_mask);
@@ -1474,6 +1474,10 @@ static int sc16is7xx_probe(struct device* This device does not have an identification register that would * tell us if we are really connected to the correct device. * The best we can do is to check if communication is at all possible. +* +* Note: regmap[0] is used in the probe function to access registers +* common to all channels/ports, as it is guaranteed to be present on +* all variants. */ ret =regmap_read (regmaps[0 ],SC16IS7XX_LSR_REG , &val);if (ret <0 )
@@ -1509,7 +1513,6 @@ static int sc16is7xx_probe(struct devicereturn -EINVAL ; } - s->regmap = regmaps[0 ]; s->devtype = devtype;dev_set_drvdata (dev, s);mutex_init (&s->efr_lock);
@@ -1524,7 +1527,7 @@ static int sc16is7xx_probe(struct devicesched_set_fifo (s->kworker_task);/* reset device, purging any pending irq / data */ -regmap_write (s->regmap,SC16IS7XX_IOCONTROL_REG , +regmap_write (regmaps[0 ],SC16IS7XX_IOCONTROL_REG ,SC16IS7XX_IOCONTROL_SRESET_BIT );for (i =0 ; i < devtype->nr_uart; ++i) {
@@ -1604,7 +1607,7 @@ static int sc16is7xx_probe(struct device s->p[u].irda_mode = true; } - ret =sc16is7xx_setup_mctrl_ports (s); + ret =sc16is7xx_setup_mctrl_ports (s, regmaps[0 ]);if (ret) goto out_ports;
Greg Kroah-Hartman
maintainerPATCH 131/331 serial: sc16is7xx: remove unused line structure member
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Hugo Villeneuve <[email protected]>
commit 41a308cbedb2a68a6831f0f2e992e296c4b8aff0 upstream.
Now that the driver has been converted to use one regmap per port, the line structure member is no longer used, so remove it.
--- drivers/tty/serial/sc16is7xx.c | 2 -- 1 file changed, 2 deletions(-)
drivers/tty/serial/sc16is7xx.c +0 -2
@@ -323,7 +323,6 @@ struct sc16is7xx_one_config {struct sc16is7xx_one {struct uart_port port; -u8 line;struct regmap *regmap;struct kthread_work tx_work;struct kthread_work reg_work;
@@ -1531,7 +1530,6 @@ static int sc16is7xx_probe(struct device SC16IS7XX_IOCONTROL_SRESET_BIT);for (i =0 ; i < devtype->nr_uart; ++i) { - s->p[i].line = i;/* Initialize port data */ s->p[i].port.dev = dev; s->p[i].port.irq = irq;
Greg Kroah-Hartman
maintainerPATCH 132/331 serial: sc16is7xx: change EFR lock to operate on each channels
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Hugo Villeneuve <[email protected]>
commit 4409df5866b7ff7686ba27e449ca97a92ee063c9 upstream.
Now that the driver has been converted to use one regmap per port, change efr locking to operate on a channel basis instead of on the whole IC.
--- drivers/tty/serial/sc16is7xx.c | 49 +++++++++++++++++++++-------------------- 1 file changed, 26 insertions(+), 23 deletions(-)
drivers/tty/serial/sc16is7xx.c +26 -23
@@ -324,6 +324,7 @@ struct sc16is7xx_one_config {struct sc16is7xx_one {struct uart_port port;struct regmap *regmap; +struct mutex efr_lock;/* EFR registers access */ struct kthread_work tx_work;struct kthread_work reg_work;struct kthread_delayed_work ms_work;
@@ -343,7 +344,6 @@ struct sc16is7xx_port {unsigned char buf[SC16IS7XX_FIFO_SIZE ];struct kthread_worker kworker;struct task_struct *kworker_task; -struct mutex efr_lock;struct sc16is7xx_one p[]; };
@@ -496,7 +496,6 @@ static bool sc16is7xx_regmap_precious(ststatic int sc16is7xx_set_baud (struct uart_port *port,int baud) { -struct sc16is7xx_port *s =dev_get_drvdata (port->dev);struct sc16is7xx_one *one =to_sc16is7xx_one (port, port);u8 lcr;u8 prescaler =0 ;
@@ -520,7 +519,7 @@ static int sc16is7xx_set_baud(struct uar* because the bulk of the interrupt processing is run as a workqueue * job in thread context. */ -mutex_lock (&s->efr_lock); +mutex_lock (&one->efr_lock); lcr =sc16is7xx_port_read (port,SC16IS7XX_LCR_REG );
@@ -539,7 +538,7 @@ static int sc16is7xx_set_baud(struct uar/* Put LCR back to the normal mode */ sc16is7xx_port_write (port,SC16IS7XX_LCR_REG , lcr); -mutex_unlock (&s->efr_lock); +mutex_unlock (&one->efr_lock); sc16is7xx_port_update(port, SC16IS7XX_MCR_REG, SC16IS7XX_MCR_CLKSEL_BIT,
@@ -707,11 +706,10 @@ static unsigned int sc16is7xx_get_hwmctrstatic void sc16is7xx_update_mlines (struct sc16is7xx_one *one) {struct uart_port *port = &one->port; -struct sc16is7xx_port *s =dev_get_drvdata (port->dev);unsigned long flags;unsigned int status, changed; -lockdep_assert_held_once (&s->efr_lock); +lockdep_assert_held_once (&one->efr_lock); status =sc16is7xx_get_hwmctrl (port); changed = status ^ one->old_mctrl;
@@ -737,15 +735,20 @@ static void sc16is7xx_update_mlines(strustatic bool sc16is7xx_port_irq (struct sc16is7xx_port *s,int portno) { +bool rc = true;struct uart_port *port = &s->p[portno].port; +struct sc16is7xx_one *one =to_sc16is7xx_one (port, port); + +mutex_lock (&one->efr_lock); do { unsigned int iir, rxlen; - struct sc16is7xx_one *one = to_sc16is7xx_one(port, port); iir = sc16is7xx_port_read(port, SC16IS7XX_IIR_REG); - if (iir & SC16IS7XX_IIR_NO_INT_BIT) - return false; + if (iir & SC16IS7XX_IIR_NO_INT_BIT) { + rc = false; + goto out_port_irq; + } iir &= SC16IS7XX_IIR_ID_MASK;
@@ -785,15 +788,17 @@ static bool sc16is7xx_port_irq(struct scbreak ; } }while (0 ); -return true; + +out_port_irq: +mutex_unlock (&one->efr_lock); + +return rc; }static irqreturn_t sc16is7xx_irq (int irq,void *dev_id) {struct sc16is7xx_port *s = (struct sc16is7xx_port *)dev_id; -mutex_lock (&s->efr_lock); -while (1 ) {bool keep_polling = false;int i;
@@ -804,24 +809,22 @@ static irqreturn_t sc16is7xx_irq(int irqbreak ; } -mutex_unlock (&s->efr_lock); -return IRQ_HANDLED ; }static void sc16is7xx_tx_proc (struct kthread_work *ws) {struct uart_port *port = &(to_sc16is7xx_one (ws, tx_work)->port); -struct sc16is7xx_port *s =dev_get_drvdata (port->dev); +struct sc16is7xx_one *one =to_sc16is7xx_one (port, port);unsigned long flags;if ((port->rs485.flags &SER_RS485_ENABLED ) && (port->rs485.delay_rts_before_send >0 ))msleep (port->rs485.delay_rts_before_send); -mutex_lock (&s->efr_lock); +mutex_lock (&one->efr_lock);sc16is7xx_handle_tx (port); -mutex_unlock (&s->efr_lock); +mutex_unlock (&one->efr_lock);spin_lock_irqsave (&port->lock, flags);sc16is7xx_ier_set (port,SC16IS7XX_IER_THRI_BIT );
@@ -928,9 +931,9 @@ static void sc16is7xx_ms_proc(struct kthstruct sc16is7xx_port *s =dev_get_drvdata (one->port.dev);if (one->port.state) { -mutex_lock (&s->efr_lock); +mutex_lock (&one->efr_lock);sc16is7xx_update_mlines (one); -mutex_unlock (&s->efr_lock); +mutex_unlock (&one->efr_lock);kthread_queue_delayed_work (&s->kworker, &one->ms_work,HZ ); }
@@ -1014,7 +1017,6 @@ static void sc16is7xx_set_termios(structstruct ktermios *termios, const struct ktermios *old) { - struct sc16is7xx_port *s = dev_get_drvdata(port->dev); struct sc16is7xx_one *one = to_sc16is7xx_one(port, port);unsigned int lcr, flow =0 ;int baud;
@@ -1073,7 +1075,7 @@ static void sc16is7xx_set_termios(struct port->ignore_status_mask |=SC16IS7XX_LSR_BRK_ERROR_MASK ;/* As above, claim the mutex while accessing the EFR. */ -mutex_lock (&s->efr_lock); +mutex_lock (&one->efr_lock);sc16is7xx_port_write (port,SC16IS7XX_LCR_REG ,SC16IS7XX_LCR_CONF_MODE_B );
@@ -1103,7 +1105,7 @@ static void sc16is7xx_set_termios(struct/* Update LCR register */ sc16is7xx_port_write (port,SC16IS7XX_LCR_REG , lcr); -mutex_unlock (&s->efr_lock); +mutex_unlock (&one->efr_lock);/* Get baud rate generator configuration */ baud = uart_get_baud_rate(port, termios, old,
@@ -1514,7 +1516,6 @@ static int sc16is7xx_probe(struct device s->devtype = devtype;dev_set_drvdata (dev, s); -mutex_init (&s->efr_lock);kthread_init_worker (&s->kworker); s->kworker_task = kthread_run(kthread_worker_fn, &s->kworker,
@@ -1557,6 +1558,8 @@ static int sc16is7xx_probe(struct device goto out_ports; } +mutex_init (&s->p[i].efr_lock); + ret =uart_get_rs485_mode (&s->p[i].port);if (ret) goto out_ports;
Greg Kroah-Hartman
maintainerPATCH 133/331 serial: sc16is7xx: convert from _raw_ to _noinc_ regmap functions for FIFO
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Hugo Villeneuve <[email protected]>
commit dbf4ab821804df071c8b566d9813083125e6d97b upstream.
The SC16IS7XX IC supports a burst mode to access the FIFOs where the initial register address is sent ($00), followed by all the FIFO data without having to resend the register address each time. In this mode, the IC doesn't increment the register address for each R/W byte.
The regmap_raw_read() and regmap_raw_write() are functions which can perform IO over multiple registers. They are currently used to read/write from/to the FIFO, and although they operate correctly in this burst mode on the SPI bus, they would corrupt the regmap cache if it was not disabled manually. The reason is that when the R/W size is more than 1 byte, these functions assume that the register address is incremented and handle the cache accordingly.
Convert FIFO R/W functions to use the regmap _noinc_ versions in order to remove the manual cache control which was a workaround when using the _raw_ versions. FIFO registers are properly declared as volatile so cache will not be used/updated for FIFO accesses.
--- drivers/tty/serial/sc16is7xx.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-)
drivers/tty/serial/sc16is7xx.c +11 -6
@@ -383,9 +383,7 @@ static void sc16is7xx_fifo_read(struct ustruct sc16is7xx_port *s =dev_get_drvdata (port->dev);struct sc16is7xx_one *one =to_sc16is7xx_one (port, port); -regcache_cache_bypass (one->regmap, true); -regmap_raw_read (one->regmap,SC16IS7XX_RHR_REG , s->buf, rxlen); -regcache_cache_bypass (one->regmap, false); +regmap_noinc_read (one->regmap,SC16IS7XX_RHR_REG , s->buf, rxlen); }static void sc16is7xx_fifo_write (struct uart_port *port,u8 to_send)
@@ -400,9 +398,7 @@ static void sc16is7xx_fifo_write(structif (unlikely (!to_send))return ; -regcache_cache_bypass (one->regmap, true); -regmap_raw_write (one->regmap,SC16IS7XX_THR_REG , s->buf, to_send); -regcache_cache_bypass (one->regmap, false); +regmap_noinc_write (one->regmap,SC16IS7XX_THR_REG , s->buf, to_send); } static void sc16is7xx_port_update(struct uart_port *port, u8 reg,
@@ -494,6 +490,11 @@ static bool sc16is7xx_regmap_precious(streturn false; } +static bool sc16is7xx_regmap_noinc (struct device *dev,unsigned int reg) +{ +return reg ==SC16IS7XX_RHR_REG ; +} +static int sc16is7xx_set_baud (struct uart_port *port,int baud) {struct sc16is7xx_one *one =to_sc16is7xx_one (port, port);
@@ -1697,6 +1698,10 @@ static struct regmap_config regcfg = { .cache_type = REGCACHE_RBTREE, .volatile_reg = sc16is7xx_regmap_volatile, .precious_reg = sc16is7xx_regmap_precious, + .writeable_noinc_reg = sc16is7xx_regmap_noinc, + .readable_noinc_reg = sc16is7xx_regmap_noinc, + .max_raw_read = SC16IS7XX_FIFO_SIZE, + .max_raw_write = SC16IS7XX_FIFO_SIZE, .max_register = SC16IS7XX_EFCR_REG, };
Greg Kroah-Hartman
maintainerPATCH 134/331 serial: sc16is7xx: fix invalid sc16is7xx_lines bitfield in case of probe error
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Hugo Villeneuve <[email protected]>
commit 8a1060ce974919f2a79807527ad82ac39336eda2 upstream.
If an error occurs during probing, the sc16is7xx_lines bitfield may be left in a state that doesn't represent the correct state of lines allocation.
For example, in a system with two SC16 devices, if an error occurs only during probing of channel (port) B of the second device, sc16is7xx_lines final state will be 00001011b instead of the expected 00000011b.
This is caused in part because of the "i--" in the for/loop located in the out_ports: error path.
Fix this by checking the return value of uart_add_one_port() and set line allocation bit only if this was successful. This allows the refactor of the obfuscated for(i--...) loop in the error path, and properly call uart_remove_one_port() only when needed, and properly unset line allocation bits.
Also use same mechanism in remove() when calling uart_remove_one_port().
--- drivers/tty/serial/sc16is7xx.c | 44 ++++++++++++++++------------------------- 1 file changed, 18 insertions(+), 26 deletions(-)
drivers/tty/serial/sc16is7xx.c +18 -26
@@ -409,19 +409,6 @@ static void sc16is7xx_port_update(structregmap_update_bits (one->regmap, reg, mask, val); } -static int sc16is7xx_alloc_line (void ) -{ -int i; - -BUILD_BUG_ON (SC16IS7XX_MAX_DEVS >BITS_PER_LONG ); - -for (i =0 ; i <SC16IS7XX_MAX_DEVS ; i++) -if (!test_and_set_bit (i, &sc16is7xx_lines)) -break ; - -return i; -} -static void sc16is7xx_power (struct uart_port *port,int on) { sc16is7xx_port_update(port, SC16IS7XX_IER_REG,
@@ -1532,6 +1519,13 @@ static int sc16is7xx_probe(struct device SC16IS7XX_IOCONTROL_SRESET_BIT);for (i =0 ; i < devtype->nr_uart; ++i) { + s->p[i].port.line =find_first_zero_bit (&sc16is7xx_lines, +SC16IS7XX_MAX_DEVS ); +if (s->p[i].port.line >=SC16IS7XX_MAX_DEVS ) { + ret = -ERANGE ; + goto out_ports; + } +/* Initialize port data */ s->p[i].port.dev = dev; s->p[i].port.irq = irq;
@@ -1551,14 +1545,8 @@ static int sc16is7xx_probe(struct device s->p[i].port.rs485_supported = sc16is7xx_rs485_supported; s->p[i].port.ops = &sc16is7xx_ops; s->p[i].old_mctrl =0 ; - s->p[i].port.line =sc16is7xx_alloc_line (); s->p[i].regmap = regmaps[i]; -if (s->p[i].port.line >=SC16IS7XX_MAX_DEVS ) { - ret = -ENOMEM ; - goto out_ports; - } -mutex_init (&s->p[i].efr_lock); ret =uart_get_rs485_mode (&s->p[i].port);
@@ -1576,8 +1564,13 @@ static int sc16is7xx_probe(struct devicekthread_init_work (&s->p[i].tx_work, sc16is7xx_tx_proc);kthread_init_work (&s->p[i].reg_work, sc16is7xx_reg_proc);kthread_init_delayed_work (&s->p[i].ms_work, sc16is7xx_ms_proc); +/* Register port */ -uart_add_one_port (&sc16is7xx_uart, &s->p[i].port); + ret =uart_add_one_port (&sc16is7xx_uart, &s->p[i].port); +if (ret) + goto out_ports; + +set_bit (s->p[i].port.line, &sc16is7xx_lines);/* Enable EFR */ sc16is7xx_port_write(&s->p[i].port, SC16IS7XX_LCR_REG,
@@ -1644,10 +1637,9 @@ static int sc16is7xx_probe(struct device#endif out_ports: -for (i--; i >=0 ; i--) { -uart_remove_one_port (&sc16is7xx_uart, &s->p[i].port); -clear_bit (s->p[i].port.line, &sc16is7xx_lines); - } +for (i =0 ; i < devtype->nr_uart; i++) +if (test_and_clear_bit (s->p[i].port.line, &sc16is7xx_lines)) +uart_remove_one_port (&sc16is7xx_uart, &s->p[i].port);kthread_stop (s->kworker_task);
@@ -1669,8 +1661,8 @@ static void sc16is7xx_remove(struct devifor (i =0 ; i < s->devtype->nr_uart; i++) {kthread_cancel_delayed_work_sync (&s->p[i].ms_work); -uart_remove_one_port (&sc16is7xx_uart, &s->p[i].port); -clear_bit (s->p[i].port.line, &sc16is7xx_lines); +if (test_and_clear_bit (s->p[i].port.line, &sc16is7xx_lines)) +uart_remove_one_port (&sc16is7xx_uart, &s->p[i].port);sc16is7xx_power (&s->p[i].port,0 ); }
Greg Kroah-Hartman
maintainerPATCH 135/331 serial: sc16is7xx: remove obsolete loop in sc16is7xx_port_irq()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Hugo Villeneuve <[email protected]>
commit ed647256e8f226241ecff7baaecdb8632ffc7ec1 upstream.
Commit 834449872105 ("sc16is7xx: Fix for multi-channel stall") changed sc16is7xx_port_irq() from looping multiple times when there was still interrupts to serve. It simply changed the do {} while(1) loop to a do {} while(0) loop, which makes the loop itself now obsolete.
Clean the code by removing this obsolete do {} while(0) loop.
--- drivers/tty/serial/sc16is7xx.c | 89 +++++++++++++++++++---------------------- 1 file changed, 43 insertions(+), 46 deletions(-)
drivers/tty/serial/sc16is7xx.c +43 -46
@@ -724,58 +724,55 @@ static void sc16is7xx_update_mlines(strustatic bool sc16is7xx_port_irq (struct sc16is7xx_port *s,int portno) {bool rc = true; +unsigned int iir, rxlen;struct uart_port *port = &s->p[portno].port;struct sc16is7xx_one *one =to_sc16is7xx_one (port, port);mutex_lock (&one->efr_lock); -do { -unsigned int iir, rxlen; - - iir =sc16is7xx_port_read (port,SC16IS7XX_IIR_REG ); -if (iir &SC16IS7XX_IIR_NO_INT_BIT ) { - rc = false; - goto out_port_irq; - } - - iir &=SC16IS7XX_IIR_ID_MASK ; - -switch (iir) { -case SC16IS7XX_IIR_RDI_SRC : -case SC16IS7XX_IIR_RLSE_SRC : -case SC16IS7XX_IIR_RTOI_SRC : -case SC16IS7XX_IIR_XOFFI_SRC : - rxlen =sc16is7xx_port_read (port,SC16IS7XX_RXLVL_REG ); - -/* -* There is a silicon bug that makes the chip report a -* time-out interrupt but no data in the FIFO. This is -* described in errata section 18.1.4. -* -* When this happens, read one byte from the FIFO to -* clear the interrupt. -*/ -if (iir ==SC16IS7XX_IIR_RTOI_SRC && !rxlen) - rxlen =1 ; - -if (rxlen) -sc16is7xx_handle_rx (port, rxlen, iir); -break ; + iir =sc16is7xx_port_read (port,SC16IS7XX_IIR_REG ); +if (iir &SC16IS7XX_IIR_NO_INT_BIT ) { + rc = false; + goto out_port_irq; + } + + iir &=SC16IS7XX_IIR_ID_MASK ; + +switch (iir) { +case SC16IS7XX_IIR_RDI_SRC : +case SC16IS7XX_IIR_RLSE_SRC : +case SC16IS7XX_IIR_RTOI_SRC : +case SC16IS7XX_IIR_XOFFI_SRC : + rxlen =sc16is7xx_port_read (port,SC16IS7XX_RXLVL_REG ); + +/* +* There is a silicon bug that makes the chip report a +* time-out interrupt but no data in the FIFO. This is +* described in errata section 18.1.4. +* +* When this happens, read one byte from the FIFO to +* clear the interrupt. +*/ +if (iir ==SC16IS7XX_IIR_RTOI_SRC && !rxlen) + rxlen =1 ; + +if (rxlen) +sc16is7xx_handle_rx (port, rxlen, iir); +break ;/* CTSRTS interrupt comes only when CTS goes inactive */ -case SC16IS7XX_IIR_CTSRTS_SRC : -case SC16IS7XX_IIR_MSI_SRC : -sc16is7xx_update_mlines (one); -break ; -case SC16IS7XX_IIR_THRI_SRC : -sc16is7xx_handle_tx (port); -break ; -default : -dev_err_ratelimited (port->dev, -"ttySC%i: Unexpected interrupt: %x" , - port->line, iir); -break ; - } - }while (0 ); +case SC16IS7XX_IIR_CTSRTS_SRC : +case SC16IS7XX_IIR_MSI_SRC : +sc16is7xx_update_mlines (one); +break ; +case SC16IS7XX_IIR_THRI_SRC : +sc16is7xx_handle_tx (port); +break ; +default : +dev_err_ratelimited (port->dev, +"ttySC%i: Unexpected interrupt: %x" , + port->line, iir); +break ; + } out_port_irq:mutex_unlock (&one->efr_lock);
Greg Kroah-Hartman
maintainerPATCH 136/331 serial: sc16is7xx: improve do/while loop in sc16is7xx_irq()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Hugo Villeneuve <[email protected]>
commit d5078509c8b06c5c472a60232815e41af81c6446 upstream.
Simplify and improve readability by replacing while(1) loop with do {} while, and by using the keep_polling variable as the exit condition, making it more explicit.
--- drivers/tty/serial/sc16is7xx.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
drivers/tty/serial/sc16is7xx.c +6 -5
@@ -782,17 +782,18 @@ out_port_irq:static irqreturn_t sc16is7xx_irq (int irq,void *dev_id) { +bool keep_polling; +struct sc16is7xx_port *s = (struct sc16is7xx_port *)dev_id; -while (1 ) { -bool keep_polling = false; +do {int i; + keep_polling = false; +for (i =0 ; i < s->devtype->nr_uart; ++i) keep_polling |=sc16is7xx_port_irq (s, i); -if (!keep_polling) -break ; - } + }while (keep_polling);return IRQ_HANDLED ; }
Greg Kroah-Hartman
maintainerPATCH 137/331 LoongArch/smp: Call rcutree_report_cpu_starting() earlier
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Huacai Chen <[email protected]>
commit a2ccf46333d7b2cf9658f0d82ac74097c1542fae upstream.
rcutree_report_cpu_starting() must be called before cpu_probe() to avoid the following lockdep splat that triggered by calling __alloc_pages() when CONFIG_PROVE_RCU_LIST=y:
=============================
WARNING: suspicious RCU usage
6.6.0+ #980 Not tainted
-----------------------------
kernel/locking/lockdep.c:3761 RCU-list traversed in non-reader section!! other info that might help us debug this:
RCU used illegally from offline CPU!
rcu_scheduler_active = 1, debug_locks = 1
1 lock held by swapper/1/0:
#0: 900000000c82ef98 (&pcp->lock){+.+.}-{2:2}, at: get_page_from_freelist+0x894/0x1790
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.6.0+ #980
Stack : 0000000000000001 9000000004f79508 9000000004893670 9000000100310000
90000001003137d0 0000000000000000 90000001003137d8 9000000004f79508
0000000000000000 0000000000000001 0000000000000000 90000000048a3384
203a656d616e2065 ca43677b3687e616 90000001002c3480 0000000000000008
000000000000009d 0000000000000000 0000000000000001 80000000ffffe0b8
000000000000000d 0000000000000033 0000000007ec0000 13bbf50562dad831
9000000005140748 0000000000000000 9000000004f79508 0000000000000004
0000000000000000 9000000005140748 90000001002bad40 0000000000000000
90000001002ba400 0000000000000000 9000000003573ec8 0000000000000000
00000000000000b0 0000000000000004 0000000000000000 0000000000070000
...
Call Trace:
[<9000000003573ec8>] show_stack+0x38/0x150
[<9000000004893670>] dump_stack_lvl+0x74/0xa8
[<900000000360d2bc>] lockdep_rcu_suspicious+0x14c/0x190
[<900000000361235c>] __lock_acquire+0xd0c/0x2740
[<90000000036146f4>] lock_acquire+0x104/0x2c0
[<90000000048a955c>] _raw_spin_lock_irqsave+0x5c/0x90
[<900000000381cd5c>] rmqueue_bulk+0x6c/0x950
[<900000000381fc0c>] get_page_from_freelist+0xd4c/0x1790
[<9000000003821c6c>] __alloc_pages+0x1bc/0x3e0
[<9000000003583b40>] tlb_init+0x150/0x2a0
[<90000000035742a0>] per_cpu_trap_init+0xf0/0x110
[<90000000035712fc>] cpu_probe+0x3dc/0x7a0
[<900000000357ed20>] start_secondary+0x40/0xb0
[<9000000004897138>] smpboot_entry+0x54/0x58raw_smp_processor_id() is required in order to avoid calling into lockdep before RCU has declared the CPU to be watched for readers.
See also commit 29368e093921 ("x86/smpboot: Move rcu_cpu_starting() earlier"), commit de5d9dae150c ("s390/smp: move rcu_cpu_starting() earlier") and commit 99f070b62322 ("powerpc/smp: Call rcu_cpu_starting() earlier").
--- arch/loongarch/kernel/smp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/loongarch/kernel/smp.c +2 -1
@@ -504,8 +504,9 @@ asmlinkage void start_secondary(void)unsigned int cpu;sync_counter (); - cpu =smp_processor_id (); + cpu =raw_smp_processor_id ();set_my_cpu_offset (per_cpu_offset (cpu)); +rcu_cpu_starting (cpu);cpu_probe ();constant_clockevent_init ();
Greg Kroah-Hartman
maintainerPATCH 138/331 mm: page_alloc: unreserve highatomic page blocks before oom
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Charan Teja Kalla <[email protected]>
commit ac3f3b0a55518056bc80ed32a41931c99e1f7d81 upstream.
__alloc_pages_direct_reclaim() is called from slowpath allocation where high atomic reserves can be unreserved after there is a progress in reclaim and yet no suitable page is found. Later should_reclaim_retry() gets called from slow path allocation to decide if the reclaim needs to be retried before OOM kill path is taken.
should_reclaim_retry() checks the available(reclaimable + free pages) memory against the min wmark levels of a zone and returns:
a) true, if it is above the min wmark so that slow path allocation will do the reclaim retries.
b) false, thus slowpath allocation takes oom kill path.
should_reclaim_retry() can also unreserves the high atomic reserves **but only after all the reclaim retries are exhausted.**
In a case where there are almost none reclaimable memory and free pages contains mostly the high atomic reserves but allocation context can't use these high atomic reserves, makes the available memory below min wmark levels hence false is returned from should_reclaim_retry() leading the allocation request to take OOM kill path. This can turn into a early oom kill if high atomic reserves are holding lot of free memory and unreserving of them is not attempted.
(early)OOM is encountered on a VM with the below state:
[ 295.998653] Normal free:7728kB boost:0kB min:804kB low:1004kB
high:1204kB reserved_highatomic:8192KB active_anon:4kB inactive_anon:0kB active_file:24kB inactive_file:24kB unevictable:1220kB writepending:0kB present:70732kB managed:49224kB mlocked:0kB bounce:0kB free_pcp:688kB local_pcp:492kB free_cma:0kB
[ 295.998656] lowmem_reserve[]: 0 32 [ 295.998659] Normal: 508*4kB (UMEH) 241*8kB (UMEH) 143*16kB (UMEH)
33*32kB (UH) 7*64kB (UH) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 7752kB
Per above log, the free memory of ~7MB exist in the high atomic reserves is not freed up before falling back to oom kill path.
Fix it by trying to unreserve the high atomic reserves in should_reclaim_retry() before __alloc_pages_direct_reclaim() can fallback to oom kill path.
--- mm/page_alloc.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
mm/page_alloc.c +8 -8
@@ -3809,14 +3809,9 @@ should_reclaim_retry(gfp_t gfp_mask, unselse (*no_progress_loops)++; -/* -* Make sure we converge to OOM if we cannot make any progress -* several times in the row. -*/ -if (*no_progress_loops >MAX_RECLAIM_RETRIES ) { -/* Before OOM, exhaust highatomic_reserve */ -return unreserve_highatomic_pageblock (ac, true); - } +if (*no_progress_loops >MAX_RECLAIM_RETRIES ) + goto out; +/* * Keep reclaiming pages while there is a chance this will lead
@@ -3859,6 +3854,11 @@ should_reclaim_retry(gfp_t gfp_mask, unsschedule_timeout_uninterruptible (1 ); elsecond_resched (); +out: +/* Before OOM, exhaust highatomic_reserve */ +if (!ret) +return unreserve_highatomic_pageblock (ac, true); +return ret; }
Greg Kroah-Hartman
maintainerPATCH 139/331 serial: Do not hold the port lock when setting rx-during-tx GPIO
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Lino Sanfilippo <[email protected]>
commit 07c30ea5861fb26a77dade8cdc787252f6122fb1 upstream.
Both the imx and stm32 driver set the rx-during-tx GPIO in rs485_config(). Since this function is called with the port lock held, this can be a problem in case that setting the GPIO line can sleep (e.g. if a GPIO expander is used which is connected via SPI or I2C).
Avoid this issue by moving the GPIO setting outside of the port lock into the serial core and thus making it a generic feature.
Also with commit c54d48543689 ("serial: stm32: Add support for rs485 RX_DURING_TX output GPIO") the SER_RS485_RX_DURING_TX flag is only set if a rx-during-tx GPIO is _not_ available, which is wrong. Fix this, too.
Furthermore reset old GPIO settings in case that changing the RS485 configuration failed.
--- drivers/tty/serial/imx.c | 4 ---- drivers/tty/serial/serial_core.c | 26 ++++++++++++++++++++++++-- drivers/tty/serial/stm32-usart.c | 8 ++------ 3 files changed, 26 insertions(+), 12 deletions(-)
drivers/tty/serial/imx.c +0 -4
@@ -1947,10 +1947,6 @@ static int imx_uart_rs485_config(struct rs485conf->flags & SER_RS485_RX_DURING_TX)imx_uart_start_rx (port); -if (port->rs485_rx_during_tx_gpio) -gpiod_set_value_cansleep (port->rs485_rx_during_tx_gpio, - !!(rs485conf->flags &SER_RS485_RX_DURING_TX )); -return 0 ; }
drivers/tty/serial/serial_core.c +24 -2
@@ -1409,6 +1409,16 @@ static void uart_set_rs485_termination(s !!(rs485->flags & SER_RS485_TERMINATE_BUS)); } +static void uart_set_rs485_rx_during_tx (struct uart_port *port, +const struct serial_rs485 *rs485) +{ +if (!(rs485->flags &SER_RS485_ENABLED )) +return ; + +gpiod_set_value_cansleep (port->rs485_rx_during_tx_gpio, + !!(rs485->flags &SER_RS485_RX_DURING_TX )); +} +static int uart_rs485_config (struct uart_port *port) {struct serial_rs485 *rs485 = &port->rs485;
@@ -1420,12 +1430,17 @@ static int uart_rs485_config(struct uartuart_sanitize_serial_rs485 (port, rs485);uart_set_rs485_termination (port, rs485); +uart_set_rs485_rx_during_tx (port, rs485);spin_lock_irqsave (&port->lock, flags); ret = port->rs485_config (port,NULL , rs485);spin_unlock_irqrestore (&port->lock, flags); -if (ret) +if (ret) {memset (rs485,0 ,sizeof (*rs485)); +/* unset GPIOs */ +gpiod_set_value_cansleep (port->rs485_term_gpio,0 ); +gpiod_set_value_cansleep (port->rs485_rx_during_tx_gpio,0 ); + }return ret; }
@@ -1464,6 +1479,7 @@ static int uart_set_rs485_config(structreturn ret;uart_sanitize_serial_rs485 (port, &rs485);uart_set_rs485_termination (port, &rs485); +uart_set_rs485_rx_during_tx (port, &rs485);spin_lock_irqsave (&port->lock, flags); ret = port->rs485_config (port, &tty->termios, &rs485);
@@ -1475,8 +1491,14 @@ static int uart_set_rs485_config(struct port->ops->set_mctrl (port, port->mctrl); }spin_unlock_irqrestore (&port->lock, flags); -if (ret) +if (ret) { +/* restore old GPIO settings */ +gpiod_set_value_cansleep (port->rs485_term_gpio, + !!(port->rs485.flags &SER_RS485_TERMINATE_BUS )); +gpiod_set_value_cansleep (port->rs485_rx_during_tx_gpio, + !!(port->rs485.flags &SER_RS485_RX_DURING_TX ));return ret; + }if (copy_to_user (rs485_user, &port->rs485,sizeof (port->rs485)))return -EFAULT ;
drivers/tty/serial/stm32-usart.c +2 -6
@@ -226,12 +226,6 @@ static int stm32_usart_config_rs485(strustm32_usart_clr_bits (port, ofs->cr1,BIT (cfg->uart_enable_bit)); -if (port->rs485_rx_during_tx_gpio) -gpiod_set_value_cansleep (port->rs485_rx_during_tx_gpio, - !!(rs485conf->flags &SER_RS485_RX_DURING_TX )); -else - rs485conf->flags |=SER_RS485_RX_DURING_TX ; -if (rs485conf->flags &SER_RS485_ENABLED ) { cr1 =readl_relaxed (port->membase + ofs->cr1); cr3 =readl_relaxed (port->membase + ofs->cr3);
@@ -256,6 +250,8 @@ static int stm32_usart_config_rs485(struwritel_relaxed (cr3, port->membase + ofs->cr3);writel_relaxed (cr1, port->membase + ofs->cr1); + + rs485conf->flags |=SER_RS485_RX_DURING_TX ; }else {stm32_usart_clr_bits (port, ofs->cr3,USART_CR3_DEM |USART_CR3_DEP );
Greg Kroah-Hartman
maintainerPATCH 140/331 ksmbd: set v2 lease version on lease upgrade
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Namjae Jeon <[email protected]>
[ Upstream commit bb05367a66a9990d2c561282f5620bb1dbe40c28 ]
If file opened with v2 lease is upgraded with v1 lease, smb server should response v2 lease create context to client.
This patch fix smb2.lease.v2_epoch2 test failure.
This test case assumes the following scenario:
1. smb2 create with v2 lease(R, LEASE1 key)
2. smb server return smb2 create response with v2 lease context(R,
LEASE1 key, epoch + 1)
3. smb2 create with v1 lease(RH, LEASE1 key)
4. smb server return smb2 create response with v2 lease context(RH,
LEASE1 key, epoch + 2)
i.e. If same client(same lease key) try to open a file that is being opened with v2 lease with v1 lease, smb server should return v2 lease.
--- fs/smb/server/oplock.c | 1 + 1 file changed, 1 insertion(+)
fs/smb/server/oplock.c +1 -0
@@ -1036,6 +1036,7 @@ static void copy_lease(struct oplock_inf lease2->duration = lease1->duration; lease2->flags = lease1->flags; lease2->epoch = lease1->epoch++; + lease2->version = lease1->version; }static int add_lease_global_list (struct oplock_info *opinfo)
Greg Kroah-Hartman
maintainerPATCH 141/331 ksmbd: fix potential circular locking issue in smb2_set_ea()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Namjae Jeon <[email protected]>
[ Upstream commit 6fc0a265e1b932e5e97a038f99e29400a93baad0 ]
smb2_set_ea() can be called in parent inode lock range. So add get_write argument to smb2_set_ea() not to call nested mnt_want_write().
--- fs/smb/server/smb2pdu.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
fs/smb/server/smb2pdu.c +4 -3
@@ -2323,11 +2323,12 @@ out:* @eabuf: set info command buffer * @buf_len: set info command buffer length * @path: dentry path for get ea +* @get_write: get write access to a mount * * Return: 0 on success, otherwise error */ static int smb2_set_ea (struct smb2_ea_info *eabuf,unsigned int buf_len, -const struct path *path) +const struct path *path,bool get_write) {struct mnt_idmap *idmap =mnt_idmap (path->mnt);char *attr_name =NULL , *value;
@@ -3015,7 +3016,7 @@ int smb2_open(struct ksmbd_work *work) rc =smb2_set_ea (&ea_buf->ea,le32_to_cpu (ea_buf->ccontext.DataLength), - &path); + &path, false);if (rc == -EOPNOTSUPP ) rc =0 ;else if (rc)
@@ -5992,7 +5993,7 @@ static int smb2_set_info_file(struct ksmreturn -EINVAL ;return smb2_set_ea ((struct smb2_ea_info *)req->Buffer, - buf_len, &fp->filp->f_path); + buf_len, &fp->filp->f_path, true); }case FILE_POSITION_INFORMATION : {
Greg Kroah-Hartman
maintainerPATCH 142/331 ksmbd: dont increment epoch if current state and request state are same
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Namjae Jeon <[email protected]>
[ Upstream commit b6e9a44e99603fe10e1d78901fdd97681a539612 ]
If existing lease state and request state are same, don't increment epoch in create context.
--- fs/smb/server/oplock.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
fs/smb/server/oplock.c +6 -3
@@ -105,7 +105,7 @@ static int alloc_lease(struct oplock_inf lease->is_dir = lctx->is_dir;memcpy (lease->parent_lease_key, lctx->parent_lease_key,SMB2_LEASE_KEY_SIZE ); lease->version = lctx->version; - lease->epoch =le16_to_cpu (lctx->epoch); + lease->epoch =le16_to_cpu (lctx->epoch) +1 ;INIT_LIST_HEAD (&opinfo->lease_entry); opinfo->o_lease = lease;
@@ -541,6 +541,9 @@ static struct oplock_info *same_client_hcontinue ; } +if (lctx->req_state != lease->state) + lease->epoch++; +/* upgrading lease */ if ((atomic_read (&ci->op_count) +atomic_read (&ci->sop_count)) ==1 ) {
@@ -1035,7 +1038,7 @@ static void copy_lease(struct oplock_inf SMB2_LEASE_KEY_SIZE); lease2->duration = lease1->duration; lease2->flags = lease1->flags; - lease2->epoch = lease1->epoch++; + lease2->epoch = lease1->epoch; lease2->version = lease1->version; }
@@ -1454,7 +1457,7 @@ void create_lease_buf(u8 *rbuf, struct lmemcpy (buf->lcontext.LeaseKey, lease->lease_key,SMB2_LEASE_KEY_SIZE ); buf->lcontext.LeaseFlags = lease->flags; - buf->lcontext.Epoch =cpu_to_le16 (++lease->epoch); + buf->lcontext.Epoch =cpu_to_le16 (lease->epoch); buf->lcontext.LeaseState = lease->state;memcpy (buf->lcontext.ParentLeaseKey, lease->parent_lease_key,SMB2_LEASE_KEY_SIZE );
Greg Kroah-Hartman
maintainerPATCH 143/331 ksmbd: send lease break notification on FILE_RENAME_INFORMATION
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Namjae Jeon <[email protected]>
[ Upstream commit 3fc74c65b367476874da5fe6f633398674b78e5a ]
Send lease break notification on FILE_RENAME_INFORMATION request. This patch fix smb2.lease.v2_epoch2 test failure.
--- fs/smb/server/oplock.c | 12 +++++++----- fs/smb/server/smb2pdu.c | 1 + 2 files changed, 8 insertions(+), 5 deletions(-)
fs/smb/server/oplock.c +7 -5
@@ -541,14 +541,12 @@ static struct oplock_info *same_client_hcontinue ; } -if (lctx->req_state != lease->state) - lease->epoch++; -/* upgrading lease */ if ((atomic_read (&ci->op_count) +atomic_read (&ci->sop_count)) ==1 ) {if (lease->state !=SMB2_LEASE_NONE_LE && lease->state == (lctx->req_state & lease->state)) { + lease->epoch++; lease->state |= lctx->req_state;if (lctx->req_state &SMB2_LEASE_WRITE_CACHING_LE )
@@ -559,13 +557,17 @@ static struct oplock_info *same_client_h atomic_read(&ci->sop_count)) > 1) {if (lctx->req_state == (SMB2_LEASE_READ_CACHING_LE | -SMB2_LEASE_HANDLE_CACHING_LE )) +SMB2_LEASE_HANDLE_CACHING_LE )) { + lease->epoch++; lease->state = lctx->req_state; + } }if (lctx->req_state && lease->state == -SMB2_LEASE_NONE_LE ) +SMB2_LEASE_NONE_LE ) { + lease->epoch++;lease_none_upgrade (opinfo, lctx->req_state); + } }read_lock (&ci->m_lock); }
fs/smb/server/smb2pdu.c +1 -0
@@ -5581,6 +5581,7 @@ static int smb2_rename(struct ksmbd_workif (!file_info->ReplaceIfExists) flags =RENAME_NOREPLACE ; +smb_break_all_levII_oplock (work, fp,0 ); rc =ksmbd_vfs_rename (work, &fp->filp->f_path, new_name, flags); out:kfree (new_name);
Greg Kroah-Hartman
maintainerPATCH 144/331 ksmbd: Add missing set_freezable() for freezable kthread
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Namjae Jeon <[email protected]>
From: Kevin Hao <[email protected]>
[ Upstream commit 8fb7b723924cc9306bc161f45496497aec733904 ]
The kernel thread function ksmbd_conn_handler_loop() invokes the try_to_freeze() in its loop. But all the kernel threads are non-freezable by default. So if we want to make a kernel thread to be freezable, we have to invoke set_freezable() explicitly.
--- fs/smb/server/connection.c | 1 + 1 file changed, 1 insertion(+)
fs/smb/server/connection.c +1 -0
@@ -284,6 +284,7 @@ int ksmbd_conn_handler_loop(void *p) goto out; conn->last_active = jiffies; +set_freezable ();while (ksmbd_conn_alive (conn)) {if (try_to_freeze ())continue ;
Greg Kroah-Hartman
maintainerPATCH 145/331 dt-bindings: net: snps,dwmac: Tx coe unsupported
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Rohan G Thomas <[email protected]>
commit 6fb8c20a04be234cf1cfd4bdd8cfb8860c9d2d3b upstream.
Add dt-bindings for coe-unsupported property per tx queue. Some DWMAC IPs support tx checksum offloading(coe) only for a few tx queues.
DW xGMAC IP can be synthesized such that it can support tx coe only for a few initial tx queues. Also as Serge pointed out, for the DW QoS IP tx coe can be individually configured for each tx queue. This property is added to have sw fallback for checksum calculation if a tx queue doesn't support tx coe.
--- Documentation/devicetree/bindings/net/snps,dwmac.yaml | 5 +++++ 1 file changed, 5 insertions(+)
Documentation/devicetree/bindings/net/snps,dwmac.yaml +5 -0
@@ -394,6 +394,11 @@ properties: When a PFC frame is received with priorities matching the bitmask, the queue is blocked from transmitting for the pause time specified in the PFC frame. + + snps,coe-unsupported: + type: boolean + description: TX checksum offload is unsupported by the TX queue. + allOf: - if: required:
Greg Kroah-Hartman
maintainerPATCH 146/331 bpf: move explored_state() closer to the beginning of verifier.c
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 3c4e420cb6536026ddd50eaaff5f30e4f144200d upstream.
Subsequent patches would make use of explored_state() function. Move it up to avoid adding unnecessary prototype.
--- kernel/bpf/verifier.c | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-)
kernel/bpf/verifier.c +13 -15
@@ -1786,6 +1786,19 @@ static int copy_verifier_state(struct bpreturn 0 ; } +static u32 state_htab_size (struct bpf_verifier_env *env) +{ +return env->prog->len; +} + +static struct bpf_verifier_state_list **explored_state (struct bpf_verifier_env *env,int idx) +{ +struct bpf_verifier_state *cur = env->cur_state; +struct bpf_func_state *state = cur->frame[cur->curframe]; + +return &env->explored_states[(idx ^ state->callsite) %state_htab_size (env)]; +} +static void update_branch_counts (struct bpf_verifier_env *env,struct bpf_verifier_state *st) {while (st) {
@@ -14702,21 +14715,6 @@ enum { BRANCH = 2, }; -static u32 state_htab_size (struct bpf_verifier_env *env) -{ -return env->prog->len; -} - -static struct bpf_verifier_state_list **explored_state ( -struct bpf_verifier_env *env, -int idx) -{ -struct bpf_verifier_state *cur = env->cur_state; -struct bpf_func_state *state = cur->frame[cur->curframe]; - -return &env->explored_states[(idx ^ state->callsite) %state_htab_size (env)]; -} -static void mark_prune_point (struct bpf_verifier_env *env,int idx) { env->insn_aux_data[idx].prune_point = true;
Greg Kroah-Hartman
maintainerPATCH 147/331 bpf: extract same_callsites() as utility function
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 4c97259abc9bc8df7712f76f58ce385581876857 upstream.
Extract same_callsites() from clean_live_states() as a utility function. This function would be used by the next patch in the set.
--- kernel/bpf/verifier.c | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-)
kernel/bpf/verifier.c +15 -5
@@ -1799,6 +1799,20 @@ static struct bpf_verifier_state_list **return &env->explored_states[(idx ^ state->callsite) %state_htab_size (env)]; } +static bool same_callsites (struct bpf_verifier_state *a,struct bpf_verifier_state *b) +{ +int fr; + +if (a->curframe != b->curframe) +return false; + +for (fr = a->curframe; fr >=0 ; fr--) +if (a->frame[fr]->callsite != b->frame[fr]->callsite) +return false; + +return true; +} +static void update_branch_counts (struct bpf_verifier_env *env,struct bpf_verifier_state *st) {while (st) {
@@ -15521,18 +15535,14 @@ static void clean_live_states(struct bpf struct bpf_verifier_state *cur) {struct bpf_verifier_state_list *sl; -int i; sl = *explored_state (env, insn);while (sl) {if (sl->state.branches) goto next;if (sl->state.insn_idx != insn || - sl->state.curframe != cur->curframe) + !same_callsites (&sl->state, cur)) goto next; -for (i =0 ; i <= cur->curframe; i++) -if (sl->state.frame[i]->callsite != cur->frame[i]->callsite) - goto next;clean_verifier_state (env, &sl->state); next: sl = sl->next;
Greg Kroah-Hartman
maintainerPATCH 148/331 bpf: exact states comparison for iterator convergence checks
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 2793a8b015f7f1caadb9bce9c63dc659f7522676 upstream.
Convergence for open coded iterators is computed in is_state_visited() by examining states with branches count > 1 and using states_equal(). states_equal() computes sub-state relation using read and precision marks. Read and precision marks are propagated from children states, thus are not guaranteed to be complete inside a loop when branches count > 1. This could be demonstrated using the following unsafe program:
1. r7 = -16
2. r6 = bpf_get_prandom_u32()
3. while (bpf_iter_num_next(&fp[-8])) {
4. if (r6 != 42) {
5. r7 = -32
6. r6 = bpf_get_prandom_u32()
7. continue
8. }
9. r0 = r10
10. r0 += r7
11. r8 = *(u64 *)(r0 + 0)
12. r6 = bpf_get_prandom_u32()
13. }
Here verifier would first visit path 1-3, create a checkpoint at 3 with r7=-16, continue to 4-7,3 with r7=-32.
Because instructions at 9-12 had not been visitied yet existing checkpoint at 3 does not have read or precision mark for r7. Thus states_equal() would return true and verifier would discard current state, thus unsafe memory access at 11 would not be caught.
This commit fixes this loophole by introducing exact state comparisons for iterator convergence logic:
- registers are compared using regs_exact() regardless of read or precision marks;
- stack slots have to have identical type.
Unfortunately, this is too strict even for simple programs like below:
i = 0;
while(iter_next(&it))
i++;At each iteration step i++ would produce a new distinct state and eventually instruction processing limit would be reached.
To avoid such behavior speculatively forget (widen) range for imprecise scalar registers, if those registers were not precise at the end of the previous iteration and do not match exactly.
This a conservative heuristic that allows to verify wide range of programs, however it precludes verification of programs that conjure an imprecise value on the first loop iteration and use it as precise on the second.
Test case iter_task_vma_for_each() presents one of such cases:
unsigned int seen = 0;
...
bpf_for_each(task_vma, vma, task, 0) {
if (seen >= 1000)
break;
...
seen++;
}Here clang generates the following code:
<LBB0_4>:
24: r8 = r6 ; stash current value of
... body ... 'seen'
29: r1 = r10
30: r1 += -0x8
31: call bpf_iter_task_vma_next
32: r6 += 0x1 ; seen++;
33: if r0 == 0x0 goto +0x2 <LBB0_6> ; exit on next() == NULL
34: r7 += 0x10
35: if r8 < 0x3e7 goto -0xc <LBB0_4> ; loop on seen < 1000<LBB0_6>:
... exit ...
Note that counter in r6 is copied to r8 and then incremented, conditional jump is done using r8. Because of this precision mark for r6 lags one state behind of precision mark on r8 and widening logic kicks in.
Adding barrier_var(seen) after conditional is sufficient to force clang use the same register for both counting and conditional jump.
This issue was discussed in the thread [1] which was started by Andrew Werner <[email protected]> demonstrating a similar bug in callback functions handling. The callbacks would be addressed in a followup patch.
[1] https://lore.kernel.org/bpf/[email protected]/
--- include/linux/bpf_verifier.h | 1 kernel/bpf/verifier.c | 218 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 188 insertions(+), 31 deletions(-)
include/linux/bpf_verifier.h +1 -0
@@ -383,6 +383,7 @@ struct bpf_verifier_state {*/ struct bpf_idx_pair *jmp_history;u32 jmp_history_cnt; +u32 dfs_depth; };#define bpf_get_spilled_reg (slot, frame) \
kernel/bpf/verifier.c +187 -31
349 lines not on this page · raw message
Greg Kroah-Hartman
maintainerPATCH 149/331 selftests/bpf: tests with delayed read/precision makrs in loop body
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 389ede06c2974b2f878a7ebff6b0f4f707f9db74 upstream.
These test cases try to hide read and precision marks from loop convergence logic: marks would only be assigned on subsequent loop iterations or after exploring states pushed to env->head stack first. Without verifier fix to use exact states comparison logic for iterators convergence these tests (except 'triple_continue') would be errorneously marked as safe.
--- tools/testing/selftests/bpf/progs/iters.c | 518 ++++++++++++++++++++++++++++++ 1 file changed, 518 insertions(+)
tools/testing/selftests/bpf/progs/iters.c +518 -0
@@ -14,6 +14,13 @@ int my_pid;int arr[256 ]; int small_arr[16] SEC(".data.small_arr"); +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(max_entries, 10); + __type(key, int); + __type(value, int); +} amap SEC(".maps"); +#ifdef REAL_TEST #define MY_PID_GUARD () if (my_pid != (bpf_get_current_pid_tgid() >> 32)) return 0#else
@@ -716,4 +723,515 @@ int iter_pass_iter_ptr_to_subprog(constreturn 0 ; } +SEC("?raw_tp") +__failure +__msg("R1 type=scalar expected=fp") +__naked int delayed_read_mark(void) +{ +/* This is equivalent to C program below. +* The call to bpf_iter_num_next() is reachable with r7 values &fp[-16] and 0xdead. +* State with r7=&fp[-16] is visited first and follows r6 != 42 ... continue branch. +* At this point iterator next() call is reached with r7 that has no read mark. +* Loop body with r7=0xdead would only be visited if verifier would decide to continue +* with second loop iteration. Absence of read mark on r7 might affect state +* equivalent logic used for iterator convergence tracking. +* +* r7 = &fp[-16] +* fp[-16] = 0 +* r6 = bpf_get_prandom_u32() +* bpf_iter_num_new(&fp[-8], 0, 10) +* while (bpf_iter_num_next(&fp[-8])) { +* r6++ +* if (r6 != 42) { +* r7 = 0xdead +* continue; +* } +* bpf_probe_read_user(r7, 8, 0xdeadbeef); // this is not safe +* } +* bpf_iter_num_destroy(&fp[-8]) +* return 0 +*/ + asmvolatile ( +"r7 = r10;" +"r7 += -16;" +"r0 = 0;" +"*(u64 *)(r7 + 0) = r0;" +"call %[bpf_get_prandom_u32];" +"r6 = r0;" +"r1 = r10;" +"r1 += -8;" +"r2 = 0;" +"r3 = 10;" +"call %[bpf_iter_num_new];" +"1:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_next];" +"if r0 == 0 goto 2f;" +"r6 += 1;" +"if r6 != 42 goto 3f;" +"r7 = 0xdead;" +"goto 1b;" +"3:" +"r1 = r7;" +"r2 = 8;" +"r3 = 0xdeadbeef;" +"call %[bpf_probe_read_user];" +"goto 1b;" +"2:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_destroy];" +"r0 = 0;" +"exit;" + : + : __imm(bpf_get_prandom_u32), +__imm (bpf_iter_num_new), +__imm (bpf_iter_num_next), +__imm (bpf_iter_num_destroy), +__imm (bpf_probe_read_user) + : __clobber_all + ); +} + +SEC("?raw_tp") +__failure +__msg("math between fp pointer and register with unbounded") +__naked int delayed_precision_mark(void) +{ +/* This is equivalent to C program below. +* The test is similar to delayed_iter_mark but verifies that incomplete +* precision don't fool verifier. +* The call to bpf_iter_num_next() is reachable with r7 values -16 and -32. +* State with r7=-16 is visited first and follows r6 != 42 ... continue branch. +* At this point iterator next() call is reached with r7 that has no read +* and precision marks. +* Loop body with r7=-32 would only be visited if verifier would decide to continue +* with second loop iteration. Absence of precision mark on r7 might affect state +* equivalent logic used for iterator convergence tracking. +* +* r8 = 0 +* fp[-16] = 0 +* r7 = -16 +* r6 = bpf_get_prandom_u32() +* bpf_iter_num_new(&fp[-8], 0, 10) +* while (bpf_iter_num_next(&fp[-8])) { +* if (r6 != 42) { +* r7 = -32 +* r6 = bpf_get_prandom_u32() +* continue; +* } +* r0 = r10 +* r0 += r7 +* r8 = *(u64 *)(r0 + 0) // this is not safe +* r6 = bpf_get_prandom_u32() +* } +* bpf_iter_num_destroy(&fp[-8]) +* return r8 +*/ + asmvolatile ( +"r8 = 0;" +"*(u64 *)(r10 - 16) = r8;" +"r7 = -16;" +"call %[bpf_get_prandom_u32];" +"r6 = r0;" +"r1 = r10;" +"r1 += -8;" +"r2 = 0;" +"r3 = 10;" +"call %[bpf_iter_num_new];" +"1:" +"r1 = r10;" +"r1 += -8;\n" +"call %[bpf_iter_num_next];" +"if r0 == 0 goto 2f;" +"if r6 != 42 goto 3f;" +"r7 = -32;" +"call %[bpf_get_prandom_u32];" +"r6 = r0;" +"goto 1b;\n" +"3:" +"r0 = r10;" +"r0 += r7;" +"r8 = *(u64 *)(r0 + 0);" +"call %[bpf_get_prandom_u32];" +"r6 = r0;" +"goto 1b;\n" +"2:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_destroy];" +"r0 = r8;" +"exit;" + : + : __imm(bpf_get_prandom_u32), +__imm (bpf_iter_num_new), +__imm (bpf_iter_num_next), +__imm (bpf_iter_num_destroy), +__imm (bpf_probe_read_user) + : __clobber_all + ); +} + +SEC("?raw_tp") +__failure +__msg("math between fp pointer and register with unbounded") +__flag (BPF_F_TEST_STATE_FREQ) +__naked int loop_state_deps1(void) +{ +/* This is equivalent to C program below. +* +* The case turns out to be tricky in a sense that: +* - states with c=-25 are explored only on a second iteration +* of the outer loop; +* - states with read+precise mark on c are explored only on +* second iteration of the inner loop and in a state which +* is pushed to states stack first. +* +* Depending on the details of iterator convergence logic +* verifier might stop states traversal too early and miss +* unsafe c=-25 memory access. +* +* j = iter_new(); // fp[-16] +* a = 0; // r6 +* b = 0; // r7 +* c = -24; // r8 +* while (iter_next(j)) { +* i = iter_new(); // fp[-8] +* a = 0; // r6 +* b = 0; // r7 +* while (iter_next(i)) { +* if (a == 1) { +* a = 0; +* b = 1; +* } else if (a == 0) { +* a = 1; +* if (random() == 42) +* continue; +* if (b == 1) { +* *(r10 + c) = 7; // this is not safe +* iter_destroy(i); +* iter_destroy(j); +* return; +* } +* } +* } +* iter_destroy(i); +* a = 0; +* b = 0; +* c = -25; +* } +* iter_destroy(j); +* return; +*/ + asmvolatile ( +"r1 = r10;" +"r1 += -16;" +"r2 = 0;" +"r3 = 10;" +"call %[bpf_iter_num_new];" +"r6 = 0;" +"r7 = 0;" +"r8 = -24;" +"j_loop_%=:" +"r1 = r10;" +"r1 += -16;" +"call %[bpf_iter_num_next];" +"if r0 == 0 goto j_loop_end_%=;" +"r1 = r10;" +"r1 += -8;" +"r2 = 0;" +"r3 = 10;" +"call %[bpf_iter_num_new];" +"r6 = 0;" +"r7 = 0;" +"i_loop_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_next];" +"if r0 == 0 goto i_loop_end_%=;" +"check_one_r6_%=:" +"if r6 != 1 goto check_zero_r6_%=;" +"r6 = 0;" +"r7 = 1;" +"goto i_loop_%=;" +"check_zero_r6_%=:" +"if r6 != 0 goto i_loop_%=;" +"r6 = 1;" +"call %[bpf_get_prandom_u32];" +"if r0 != 42 goto check_one_r7_%=;" +"goto i_loop_%=;" +"check_one_r7_%=:" +"if r7 != 1 goto i_loop_%=;" +"r0 = r10;" +"r0 += r8;" +"r1 = 7;" +"*(u64 *)(r0 + 0) = r1;" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_destroy];" +"r1 = r10;" +"r1 += -16;" +"call %[bpf_iter_num_destroy];" +"r0 = 0;" +"exit;" +"i_loop_end_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_destroy];" +"r6 = 0;" +"r7 = 0;" +"r8 = -25;" +"goto j_loop_%=;" +"j_loop_end_%=:" +"r1 = r10;" +"r1 += -16;" +"call %[bpf_iter_num_destroy];" +"r0 = 0;" +"exit;" + : + : __imm(bpf_get_prandom_u32), +__imm (bpf_iter_num_new), +__imm (bpf_iter_num_next), +__imm (bpf_iter_num_destroy) + : __clobber_all + ); +} + +SEC("?raw_tp") +__success +__naked int triple_continue (void ) +{ +/* This is equivalent to C program below. +* High branching factor of the loop body turned out to be +* problematic for one of the iterator convergence tracking +* algorithms explored. +* +* r6 = bpf_get_prandom_u32() +* bpf_iter_num_new(&fp[-8], 0, 10) +* while (bpf_iter_num_next(&fp[-8])) { +* if (bpf_get_prandom_u32() != 42) +* continue; +* if (bpf_get_prandom_u32() != 42) +* continue; +* if (bpf_get_prandom_u32() != 42) +* continue; +* r0 += 0; +* } +* bpf_iter_num_destroy(&fp[-8]) +* return 0 +*/ + asmvolatile ( +"r1 = r10;" +"r1 += -8;" +"r2 = 0;" +"r3 = 10;" +"call %[bpf_iter_num_new];" +"loop_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_next];" +"if r0 == 0 goto loop_end_%=;" +"call %[bpf_get_prandom_u32];" +"if r0 != 42 goto loop_%=;" +"call %[bpf_get_prandom_u32];" +"if r0 != 42 goto loop_%=;" +"call %[bpf_get_prandom_u32];" +"if r0 != 42 goto loop_%=;" +"r0 += 0;" +"goto loop_%=;" +"loop_end_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_destroy];" +"r0 = 0;" +"exit;" + : + : __imm(bpf_get_prandom_u32), +__imm (bpf_iter_num_new), +__imm (bpf_iter_num_next), +__imm (bpf_iter_num_destroy) + : __clobber_all + ); +} + +SEC("?raw_tp") +__success +__naked int widen_spill (void ) +{ +/* This is equivalent to C program below. +* The counter is stored in fp[-16], if this counter is not widened +* verifier states representing loop iterations would never converge. +* +* fp[-16] = 0 +* bpf_iter_num_new(&fp[-8], 0, 10) +* while (bpf_iter_num_next(&fp[-8])) { +* r0 = fp[-16]; +* r0 += 1; +* fp[-16] = r0; +* } +* bpf_iter_num_destroy(&fp[-8]) +* return 0 +*/ + asmvolatile ( +"r0 = 0;" +"*(u64 *)(r10 - 16) = r0;" +"r1 = r10;" +"r1 += -8;" +"r2 = 0;" +"r3 = 10;" +"call %[bpf_iter_num_new];" +"loop_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_next];" +"if r0 == 0 goto loop_end_%=;" +"r0 = *(u64 *)(r10 - 16);" +"r0 += 1;" +"*(u64 *)(r10 - 16) = r0;" +"goto loop_%=;" +"loop_end_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_destroy];" +"r0 = 0;" +"exit;" + : + : __imm(bpf_iter_num_new), +__imm (bpf_iter_num_next), +__imm (bpf_iter_num_destroy) + : __clobber_all + ); +} + +SEC("raw_tp") +__success +__naked int checkpoint_states_deletion (void ) +{ +/* This is equivalent to C program below. +* +* int *a, *b, *c, *d, *e, *f; +* int i, sum = 0; +* bpf_for(i, 0, 10) { +* a = bpf_map_lookup_elem(&amap, &i); +* b = bpf_map_lookup_elem(&amap, &i); +* c = bpf_map_lookup_elem(&amap, &i); +* d = bpf_map_lookup_elem(&amap, &i); +* e = bpf_map_lookup_elem(&amap, &i); +* f = bpf_map_lookup_elem(&amap, &i); +* if (a) sum += 1; +* if (b) sum += 1; +* if (c) sum += 1; +* if (d) sum += 1; +* if (e) sum += 1; +* if (f) sum += 1; +* } +* return 0; +* +* The body of the loop spawns multiple simulation paths +* with different combination of NULL/non-NULL information for a/b/c/d/e/f. +* Each combination is unique from states_equal() point of view. +* Explored states checkpoint is created after each iterator next call. +* Iterator convergence logic expects that eventually current state +* would get equal to one of the explored states and thus loop +* exploration would be finished (at-least for a specific path). +* Verifier evicts explored states with high miss to hit ratio +* to to avoid comparing current state with too many explored +* states per instruction. +* This test is designed to "stress test" eviction policy defined using formula: +* +* sl->miss_cnt > sl->hit_cnt * N + N // if true sl->state is evicted +* +* Currently N is set to 64, which allows for 6 variables in this test. +*/ + asmvolatile ( +"r6 = 0;" /* a */ +"r7 = 0;" /* b */ +"r8 = 0;" /* c */ +"*(u64 *)(r10 - 24) = r6;" /* d */ +"*(u64 *)(r10 - 32) = r6;" /* e */ +"*(u64 *)(r10 - 40) = r6;" /* f */ +"r9 = 0;" /* sum */ +"r1 = r10;" +"r1 += -8;" +"r2 = 0;" +"r3 = 10;" +"call %[bpf_iter_num_new];" +"loop_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_next];" +"if r0 == 0 goto loop_end_%=;" + +"*(u64 *)(r10 - 16) = r0;" + +"r1 = %[amap] ll;" +"r2 = r10;" +"r2 += -16;" +"call %[bpf_map_lookup_elem];" +"r6 = r0;" + +"r1 = %[amap] ll;" +"r2 = r10;" +"r2 += -16;" +"call %[bpf_map_lookup_elem];" +"r7 = r0;" + +"r1 = %[amap] ll;" +"r2 = r10;" +"r2 += -16;" +"call %[bpf_map_lookup_elem];" +"r8 = r0;" + +"r1 = %[amap] ll;" +"r2 = r10;" +"r2 += -16;" +"call %[bpf_map_lookup_elem];" +"*(u64 *)(r10 - 24) = r0;" + +"r1 = %[amap] ll;" +"r2 = r10;" +"r2 += -16;" +"call %[bpf_map_lookup_elem];" +"*(u64 *)(r10 - 32) = r0;" + +"r1 = %[amap] ll;" +"r2 = r10;" +"r2 += -16;" +"call %[bpf_map_lookup_elem];" +"*(u64 *)(r10 - 40) = r0;" + +"if r6 == 0 goto +1;" +"r9 += 1;" +"if r7 == 0 goto +1;" +"r9 += 1;" +"if r8 == 0 goto +1;" +"r9 += 1;" +"r0 = *(u64 *)(r10 - 24);" +"if r0 == 0 goto +1;" +"r9 += 1;" +"r0 = *(u64 *)(r10 - 32);" +"if r0 == 0 goto +1;" +"r9 += 1;" +"r0 = *(u64 *)(r10 - 40);" +"if r0 == 0 goto +1;" +"r9 += 1;" + +"goto loop_%=;" +"loop_end_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_destroy];" +"r0 = 0;" +"exit;" + : + : __imm(bpf_map_lookup_elem), +__imm (bpf_iter_num_new), +__imm (bpf_iter_num_next), +__imm (bpf_iter_num_destroy), +__imm_addr (amap) + : __clobber_all + ); +} + char _license[] SEC("license") = "GPL";
Greg Kroah-Hartman
maintainerPATCH 150/331 bpf: correct loop detection for iterators convergence
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 2a0992829ea3864939d917a5c7b48be6629c6217 upstream.
It turns out that .branches > 0 in is_state_visited() is not a sufficient condition to identify if two verifier states form a loop when iterators convergence is computed. This commit adds logic to distinguish situations like below:
(I) initial (II) initial
| |
V V
.---------> hdr ..
| | |
| V V
| .------... .------..
| | | | |
| V V V V
| ... ... .-> hdr ..
| | | | | |
| V V | V V
| succ <- cur | succ <- cur
| | | |
| V | V
| ... | ...
| | | |
'----' '----'For both (I) and (II) successor 'succ' of the current state 'cur' was previously explored and has branches count at 0. However, loop entry 'hdr' corresponding to 'succ' might be a part of current DFS path. If that is the case 'succ' and 'cur' are members of the same loop and have to be compared exactly.
--- include/linux/bpf_verifier.h | 15 +++ kernel/bpf/verifier.c | 207 ++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 218 insertions(+), 4 deletions(-)
include/linux/bpf_verifier.h +15 -0
@@ -372,10 +372,25 @@ struct bpf_verifier_state {struct bpf_active_lock active_lock;bool speculative;bool active_rcu_lock; +/* If this state was ever pointed-to by other state's loop_entry field +* this flag would be set to true. Used to avoid freeing such states +* while they are still in use. +*/ +bool used_as_loop_entry;/* first and last insn idx of this verifier state */ u32 first_insn_idx;u32 last_insn_idx; +/* If this state is a part of states loop this field points to some +* parent of this state such that: +* - it is also a member of the same states loop; +* - DFS states traversal starting from initial state visits loop_entry +* state before this state. +* Used to compute topmost loop entry for state loops. +* State loops might appear because of open coded iterators logic. +* See get_loop_entry() for more information. +*/ +struct bpf_verifier_state *loop_entry;/* jmp history recorded from first to last. * backtracking is using it to go from last to first. * For most states jmp_history_cnt is [0-3].
kernel/bpf/verifier.c +203 -4
252 lines not on this page · raw message
Greg Kroah-Hartman
maintainerPATCH 151/331 selftests/bpf: test if state loops are detected in a tricky case
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 64870feebecb7130291a55caf0ce839a87405a70 upstream.
A convoluted test case for iterators convergence logic that demonstrates that states with branch count equal to 0 might still be a part of not completely explored loop.
E.g. consider the following state diagram:
initial Here state 'succ' was processed first,
| it was eventually tracked to produce a
V state identical to 'hdr'.
.---------> hdr All branches from 'succ' had been explored
| | and thus 'succ' has its .branches == 0.
| V
| .------... Suppose states 'cur' and 'succ' correspond
| | | to the same instruction + callsites.
| V V In such case it is necessary to check
| ... ... whether 'succ' and 'cur' are identical.
| | | If 'succ' and 'cur' are a part of the same loop
| V V they have to be compared exactly.
| succ <- cur
| |
| V
| ...
| |
'----'--- tools/testing/selftests/bpf/progs/iters.c | 177 ++++++++++++++++++++++++++++++ 1 file changed, 177 insertions(+)
tools/testing/selftests/bpf/progs/iters.c +177 -0
@@ -999,6 +999,183 @@ __naked int loop_state_deps1(void) } SEC("?raw_tp") +__failure +__msg("math between fp pointer and register with unbounded") +__flag (BPF_F_TEST_STATE_FREQ) +__naked int loop_state_deps2(void) +{ +/* This is equivalent to C program below. +* +* The case turns out to be tricky in a sense that: +* - states with read+precise mark on c are explored only on a second +* iteration of the first inner loop and in a state which is pushed to +* states stack first. +* - states with c=-25 are explored only on a second iteration of the +* second inner loop and in a state which is pushed to states stack +* first. +* +* Depending on the details of iterator convergence logic +* verifier might stop states traversal too early and miss +* unsafe c=-25 memory access. +* +* j = iter_new(); // fp[-16] +* a = 0; // r6 +* b = 0; // r7 +* c = -24; // r8 +* while (iter_next(j)) { +* i = iter_new(); // fp[-8] +* a = 0; // r6 +* b = 0; // r7 +* while (iter_next(i)) { +* if (a == 1) { +* a = 0; +* b = 1; +* } else if (a == 0) { +* a = 1; +* if (random() == 42) +* continue; +* if (b == 1) { +* *(r10 + c) = 7; // this is not safe +* iter_destroy(i); +* iter_destroy(j); +* return; +* } +* } +* } +* iter_destroy(i); +* i = iter_new(); // fp[-8] +* a = 0; // r6 +* b = 0; // r7 +* while (iter_next(i)) { +* if (a == 1) { +* a = 0; +* b = 1; +* } else if (a == 0) { +* a = 1; +* if (random() == 42) +* continue; +* if (b == 1) { +* a = 0; +* c = -25; +* } +* } +* } +* iter_destroy(i); +* } +* iter_destroy(j); +* return; +*/ + asmvolatile ( +"r1 = r10;" +"r1 += -16;" +"r2 = 0;" +"r3 = 10;" +"call %[bpf_iter_num_new];" +"r6 = 0;" +"r7 = 0;" +"r8 = -24;" +"j_loop_%=:" +"r1 = r10;" +"r1 += -16;" +"call %[bpf_iter_num_next];" +"if r0 == 0 goto j_loop_end_%=;" + +/* first inner loop */ +"r1 = r10;" +"r1 += -8;" +"r2 = 0;" +"r3 = 10;" +"call %[bpf_iter_num_new];" +"r6 = 0;" +"r7 = 0;" +"i_loop_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_next];" +"if r0 == 0 goto i_loop_end_%=;" +"check_one_r6_%=:" +"if r6 != 1 goto check_zero_r6_%=;" +"r6 = 0;" +"r7 = 1;" +"goto i_loop_%=;" +"check_zero_r6_%=:" +"if r6 != 0 goto i_loop_%=;" +"r6 = 1;" +"call %[bpf_get_prandom_u32];" +"if r0 != 42 goto check_one_r7_%=;" +"goto i_loop_%=;" +"check_one_r7_%=:" +"if r7 != 1 goto i_loop_%=;" +"r0 = r10;" +"r0 += r8;" +"r1 = 7;" +"*(u64 *)(r0 + 0) = r1;" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_destroy];" +"r1 = r10;" +"r1 += -16;" +"call %[bpf_iter_num_destroy];" +"r0 = 0;" +"exit;" +"i_loop_end_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_destroy];" + +/* second inner loop */ +"r1 = r10;" +"r1 += -8;" +"r2 = 0;" +"r3 = 10;" +"call %[bpf_iter_num_new];" +"r6 = 0;" +"r7 = 0;" +"i2_loop_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_next];" +"if r0 == 0 goto i2_loop_end_%=;" +"check2_one_r6_%=:" +"if r6 != 1 goto check2_zero_r6_%=;" +"r6 = 0;" +"r7 = 1;" +"goto i2_loop_%=;" +"check2_zero_r6_%=:" +"if r6 != 0 goto i2_loop_%=;" +"r6 = 1;" +"call %[bpf_get_prandom_u32];" +"if r0 != 42 goto check2_one_r7_%=;" +"goto i2_loop_%=;" +"check2_one_r7_%=:" +"if r7 != 1 goto i2_loop_%=;" +"r6 = 0;" +"r8 = -25;" +"goto i2_loop_%=;" +"i2_loop_end_%=:" +"r1 = r10;" +"r1 += -8;" +"call %[bpf_iter_num_destroy];" + +"r6 = 0;" +"r7 = 0;" +"goto j_loop_%=;" +"j_loop_end_%=:" +"r1 = r10;" +"r1 += -16;" +"call %[bpf_iter_num_destroy];" +"r0 = 0;" +"exit;" + : + : __imm(bpf_get_prandom_u32), +__imm (bpf_iter_num_new), +__imm (bpf_iter_num_next), +__imm (bpf_iter_num_destroy) + : __clobber_all + ); +} + +SEC("?raw_tp") __success__naked int triple_continue (void ) {
Greg Kroah-Hartman
maintainerPATCH 152/331 bpf: print full verifier states on infinite loop detection
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit b4d8239534fddc036abe4a0fdbf474d9894d4641 upstream.
Additional logging in is_state_visited(): if infinite loop is detected print full verifier state for both current and equivalent states.
--- kernel/bpf/verifier.c | 4 ++++ 1 file changed, 4 insertions(+)
kernel/bpf/verifier.c +4 -0
@@ -16540,6 +16540,10 @@ static int is_state_visited(struct bpf_v !iter_active_depths_differ(&sl->state, cur)) {verbose_linfo (env, insn_idx,"; " );verbose (env,"infinite loop detected at insn %d\n" , insn_idx); +verbose (env,"cur state:" ); +print_verifier_state (env, cur->frame[cur->curframe], true); +verbose (env,"old state:" ); +print_verifier_state (env, sl->state.frame[cur->curframe], true);return -EINVAL ; }/* if the verifier is processing a loop, avoid adding new state
Greg Kroah-Hartman
maintainerPATCH 153/331 selftests/bpf: track tcp payload offset as scalar in xdp_synproxy
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 977bc146d4eb7070118d8a974919b33bb52732b4 upstream.
This change prepares syncookie_{tc,xdp} for update in callbakcs verification logic. To allow bpf_loop() verification converge when multiple callback itreations are considered:
- track offset inside TCP payload explicitly, not as a part of the pointer;
- make sure that offset does not exceed MAX_PACKET_OFF enforced by verifier;
- make sure that offset is tracked as unbound scalar between iterations, otherwise verifier won't be able infer that bpf_loop callback reaches identical states.
--- tools/testing/selftests/bpf/progs/xdp_synproxy_kern.c | 84 +++++++++++------- 1 file changed, 52 insertions(+), 32 deletions(-)
tools/testing/selftests/bpf/progs/xdp_synproxy_kern.c +52 -32
@@ -53,6 +53,8 @@#define DEFAULT_TTL 64#define MAX_ALLOWED_PORTS 8 +#define MAX_PACKET_OFF 0xffff +#define swap (a, b) \ do { typeof(a) __tmp = (a); (a) = (b); (b) = __tmp; } while (0)
@@ -183,63 +185,76 @@ static __always_inline __u32 tcp_time_st }struct tcpopt_context { -__u8 *ptr; -__u8 *end; +void *data;void *data_end;__be32 *tsecr;__u8 wscale;bool option_timestamp;bool option_sack; +__u32 off; }; -static int tscookie_tcpopt_parse (struct tcpopt_context *ctx) +static __always_inline u8 *next(struct tcpopt_context *ctx, __u32 sz) { -__u8 opcode, opsize; +__u64 off = ctx->off; +__u8 *data; -if (ctx->ptr >= ctx->end) -return 1 ; -if (ctx->ptr >= ctx->data_end) -return 1 ; +/* Verifier forbids access to packet when offset exceeds MAX_PACKET_OFF */ +if (off >MAX_PACKET_OFF - sz) +return NULL ; + + data = ctx->data + off; +barrier_var (data); +if (data + sz >= ctx->data_end) +return NULL ; - opcode = ctx->ptr[0 ]; + ctx->off += sz; +return data; +} -if (opcode ==TCPOPT_EOL ) -return 1 ; -if (opcode ==TCPOPT_NOP ) { - ++ctx->ptr; -return 0 ; - } +static int tscookie_tcpopt_parse (struct tcpopt_context *ctx) +{ +__u8 *opcode, *opsize, *wscale, *tsecr; +__u32 off = ctx->off; -if (ctx->ptr +1 >= ctx->end) + opcode =next (ctx,1 ); +if (!opcode)return 1 ; -if (ctx->ptr +1 >= ctx->data_end) -return 1 ; - opsize = ctx->ptr[1 ]; -if (opsize <2 ) + +if (*opcode ==TCPOPT_EOL )return 1 ; +if (*opcode ==TCPOPT_NOP ) +return 0 ; -if (ctx->ptr + opsize > ctx->end) + opsize =next (ctx,1 ); +if (!opsize || *opsize <2 )return 1 ; -switch (opcode) { +switch (*opcode) {case TCPOPT_WINDOW : -if (opsize ==TCPOLEN_WINDOW && ctx->ptr +TCPOLEN_WINDOW <= ctx->data_end) - ctx->wscale = ctx->ptr[2 ] <TCP_MAX_WSCALE ? ctx->ptr[2 ] :TCP_MAX_WSCALE ; + wscale =next (ctx,1 ); +if (!wscale) +return 1 ; +if (*opsize ==TCPOLEN_WINDOW ) + ctx->wscale = *wscale <TCP_MAX_WSCALE ? *wscale :TCP_MAX_WSCALE ;break ;case TCPOPT_TIMESTAMP : -if (opsize ==TCPOLEN_TIMESTAMP && ctx->ptr +TCPOLEN_TIMESTAMP <= ctx->data_end) { + tsecr =next (ctx,4 ); +if (!tsecr) +return 1 ; +if (*opsize ==TCPOLEN_TIMESTAMP ) { ctx->option_timestamp = true;/* Client's tsval becomes our tsecr. */ - *ctx->tsecr =get_unaligned ((__be32 *)(ctx->ptr +2 )); + *ctx->tsecr =get_unaligned ((__be32 *)tsecr); }break ;case TCPOPT_SACK_PERM : -if (opsize ==TCPOLEN_SACK_PERM ) +if (*opsize ==TCPOLEN_SACK_PERM ) ctx->option_sack = true;break ; } - ctx->ptr += opsize; + ctx->off = off + *opsize;return 0 ; }
@@ -256,16 +271,21 @@ static int tscookie_tcpopt_parse_batch(_ static __always_inline bool tscookie_init(struct tcphdr *tcp_header,__u16 tcp_len,__be32 *tsval, -__be32 *tsecr,void *data_end) +__be32 *tsecr,void *data,void *data_end) {struct tcpopt_context loop_ctx = { - .ptr = (__u8 *)(tcp_header +1 ), - .end = (__u8 *)tcp_header + tcp_len, + .data = data, .data_end = data_end, .tsecr = tsecr, .wscale =TS_OPT_WSCALE_MASK , .option_timestamp = false, .option_sack = false, +/* Note: currently verifier would track .off as unbound scalar. +* In case if verifier would at some point get smarter and +* compute bounded value for this var, beware that it might +* hinder bpf_loop() convergence validation. +*/ + .off = (__u8 *)(tcp_header +1 ) - (__u8 *)data, };u32 cookie;
@@ -635,7 +655,7 @@ static __always_inline int syncookie_han cookie = (__u32 )value;if (tscookie_init ((void *)hdr->tcp, hdr->tcp_len, - &tsopt_buf[0 ], &tsopt_buf[1 ], data_end)) + &tsopt_buf[0 ], &tsopt_buf[1 ], data, data_end)) tsopt = tsopt_buf;/* Check that there is enough space for a SYNACK. It also covers
Greg Kroah-Hartman
maintainerPATCH 154/331 selftests/bpf: track string payload offset as scalar in strobemeta
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 87eb0152bcc102ecbda866978f4e54db5a3be1ef upstream.
This change prepares strobemeta for update in callbacks verification logic. To allow bpf_loop() verification converge when multiple callback iterations are considered:
- track offset inside strobemeta_payload->payload directly as scalar value;
- at each iteration make sure that remaining strobemeta_payload->payload capacity is sufficient for execution of read_{map,str}_var functions;
- make sure that offset is tracked as unbound scalar between iterations, otherwise verifier won't be able infer that bpf_loop callback reaches identical states.
--- tools/testing/selftests/bpf/progs/strobemeta.h | 78 +++++++++++++++---------- 1 file changed, 48 insertions(+), 30 deletions(-)
tools/testing/selftests/bpf/progs/strobemeta.h +48 -30
@@ -24,9 +24,11 @@ struct task_struct {};#define STACK_TABLE_EPOCH_SHIFT 20#define STROBE_MAX_STR_LEN 1#define STROBE_MAX_CFGS 32 +#define READ_MAP_VAR_PAYLOAD_CAP \ + ((1 + STROBE_MAX_MAP_ENTRIES * 2) * STROBE_MAX_STR_LEN)#define STROBE_MAX_PAYLOAD \ (STROBE_MAX_STRS * STROBE_MAX_STR_LEN + \ - STROBE_MAX_MAPS * (1 + STROBE_MAX_MAP_ENTRIES * 2) * STROBE_MAX_STR_LEN) + STROBE_MAX_MAPS * READ_MAP_VAR_PAYLOAD_CAP)struct strobe_value_header {/*
@@ -355,7 +357,7 @@ static __always_inline uint64_t read_str size_t idx, void *tls_base, struct strobe_value_generic *value, struct strobemeta_payload *data, - void *payload) + size_t off) { void *location;uint64_t len;
@@ -366,7 +368,7 @@ static __always_inline uint64_t read_strreturn 0 ;bpf_probe_read_user (value,sizeof (struct strobe_value_generic ), location); - len =bpf_probe_read_user_str (payload,STROBE_MAX_STR_LEN , value->ptr); + len =bpf_probe_read_user_str (&data->payload[off],STROBE_MAX_STR_LEN , value->ptr);/* * if bpf_probe_read_user_str returns error (<0), due to casting to * unsinged int, it will become big number, so next check is
@@ -378,14 +380,14 @@ static __always_inline uint64_t read_strreturn 0 ; data->str_lens[idx] = len; -return len; +return off + len; } -static __always_inline void *read_map_var(struct strobemeta_cfg *cfg, -size_t idx,void *tls_base, -struct strobe_value_generic *value, -struct strobemeta_payload *data, -void *payload) +static __always_inline uint64_t read_map_var(struct strobemeta_cfg *cfg, +size_t idx,void *tls_base, +struct strobe_value_generic *value, +struct strobemeta_payload *data, +size_t off) {struct strobe_map_descr * descr = &data->map_descrs[idx];struct strobe_map_raw map;
@@ -397,11 +399,11 @@ static __always_inline void *read_map_va location =calc_location (&cfg->map_locs[idx], tls_base);if (!location) -return payload; +return off;bpf_probe_read_user (value,sizeof (struct strobe_value_generic ), location);if (bpf_probe_read_user (&map,sizeof (struct strobe_map_raw ), value->ptr)) -return payload; +return off; descr->id = map.id; descr->cnt = map.cnt;
@@ -410,10 +412,10 @@ static __always_inline void *read_map_va data->req_meta_valid =1 ; } - len =bpf_probe_read_user_str (payload,STROBE_MAX_STR_LEN , map.tag); + len =bpf_probe_read_user_str (&data->payload[off],STROBE_MAX_STR_LEN , map.tag);if (len <=STROBE_MAX_STR_LEN ) { descr->tag_len = len; - payload += len; + off += len; }#ifdef NO_UNROLL
@@ -426,22 +428,22 @@ static __always_inline void *read_map_vabreak ; descr->key_lens[i] =0 ; - len =bpf_probe_read_user_str (payload,STROBE_MAX_STR_LEN , + len =bpf_probe_read_user_str (&data->payload[off],STROBE_MAX_STR_LEN , map.entries[i].key);if (len <=STROBE_MAX_STR_LEN ) { descr->key_lens[i] = len; - payload += len; + off += len; } descr->val_lens[i] =0 ; - len =bpf_probe_read_user_str (payload,STROBE_MAX_STR_LEN , + len =bpf_probe_read_user_str (&data->payload[off],STROBE_MAX_STR_LEN , map.entries[i].val);if (len <=STROBE_MAX_STR_LEN ) { descr->val_lens[i] = len; - payload += len; + off += len; } } -return payload; +return off; }#ifdef USE_BPF_LOOP
@@ -455,14 +457,20 @@ struct read_var_ctx {struct strobemeta_payload *data;void *tls_base;struct strobemeta_cfg *cfg; -void *payload; +size_t payload_off;/* value gets mutated */ struct strobe_value_generic *value;enum read_type type; }; -static int read_var_callback (__u32 index,struct read_var_ctx *ctx) +static int read_var_callback (__u64 index,struct read_var_ctx *ctx) { +/* lose precision info for ctx->payload_off, verifier won't track +* double xor, barrier_var() is needed to force clang keep both xors. +*/ + ctx->payload_off ^= index; +barrier_var (ctx->payload_off); + ctx->payload_off ^= index;switch (ctx->type) {case READ_INT_VAR :if (index >=STROBE_MAX_INTS )
@@ -472,14 +480,18 @@ static int read_var_callback(__u32 indexcase READ_MAP_VAR :if (index >=STROBE_MAX_MAPS )return 1 ; - ctx->payload =read_map_var (ctx->cfg, index, ctx->tls_base, - ctx->value, ctx->data, ctx->payload); +if (ctx->payload_off >sizeof (ctx->data->payload) -READ_MAP_VAR_PAYLOAD_CAP ) +return 1 ; + ctx->payload_off =read_map_var (ctx->cfg, index, ctx->tls_base, + ctx->value, ctx->data, ctx->payload_off);break ;case READ_STR_VAR :if (index >=STROBE_MAX_STRS )return 1 ; - ctx->payload +=read_str_var (ctx->cfg, index, ctx->tls_base, - ctx->value, ctx->data, ctx->payload); +if (ctx->payload_off >sizeof (ctx->data->payload) -STROBE_MAX_STR_LEN ) +return 1 ; + ctx->payload_off =read_str_var (ctx->cfg, index, ctx->tls_base, + ctx->value, ctx->data, ctx->payload_off);break ; }return 0 ;
@@ -501,7 +513,8 @@ static void *read_strobe_meta(struct taspid_t pid =bpf_get_current_pid_tgid () >>32 ;struct strobe_value_generic value = {0 };struct strobemeta_cfg *cfg; -void *tls_base, *payload; +size_t payload_off; +void *tls_base; cfg =bpf_map_lookup_elem (&strobemeta_cfgs, &pid);if (!cfg)
@@ -509,7 +522,7 @@ static void *read_strobe_meta(struct tas data->int_vals_set_mask =0 ; data->req_meta_valid =0 ; - payload = data->payload; + payload_off =0 ;/* * we don't have struct task_struct definition, it should be: * tls_base = (void *)task->thread.fsbase;
@@ -522,7 +535,7 @@ static void *read_strobe_meta(struct tas .tls_base = tls_base, .value = &value, .data = data, - .payload = payload, + .payload_off = 0, };int err;
@@ -540,6 +553,11 @@ static void *read_strobe_meta(struct tas err =bpf_loop (STROBE_MAX_MAPS , read_var_callback, &ctx,0 );if (err !=STROBE_MAX_MAPS )return NULL ; + + payload_off = ctx.payload_off; +/* this should not really happen, here only to satisfy verifer */ +if (payload_off >sizeof (data->payload)) + payload_off =sizeof (data->payload);#else #ifdef NO_UNROLL #pragma clang loop unroll(disable)
@@ -555,7 +573,7 @@ static void *read_strobe_meta(struct tas#pragma unroll#endif /* NO_UNROLL */ for (int i =0 ; i <STROBE_MAX_STRS ; ++i) { - payload +=read_str_var (cfg, i, tls_base, &value, data, payload); + payload_off =read_str_var (cfg, i, tls_base, &value, data, payload_off); }#ifdef NO_UNROLL #pragma clang loop unroll(disable)
@@ -563,7 +581,7 @@ static void *read_strobe_meta(struct tas#pragma unroll#endif /* NO_UNROLL */ for (int i =0 ; i <STROBE_MAX_MAPS ; ++i) { - payload =read_map_var (cfg, i, tls_base, &value, data, payload); + payload_off =read_map_var (cfg, i, tls_base, &value, data, payload_off); }#endif /* USE_BPF_LOOP */
@@ -571,7 +589,7 @@ static void *read_strobe_meta(struct tas* return pointer right after end of payload, so it's possible to * calculate exact amount of useful data that needs to be sent */ -return payload; +return &data->payload[payload_off]; }SEC ("raw_tracepoint/kfree_skb" )
Greg Kroah-Hartman
maintainerPATCH 155/331 bpf: extract __check_reg_arg() utility function
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 683b96f9606ab7308ffb23c46ab43cecdef8a241 upstream.
Split check_reg_arg() into two utility functions:
- check_reg_arg() operating on registers from current verifier state;
- __check_reg_arg() operating on a specific set of registers passed as a parameter;
The __check_reg_arg() function would be used by a follow-up change for callbacks handling.
--- kernel/bpf/verifier.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-)
kernel/bpf/verifier.c +13 -6
@@ -3321,13 +3321,11 @@ static void mark_insn_zext(struct bpf_ve reg->subreg_def =DEF_NOT_SUBREG ; } -static int check_reg_arg (struct bpf_verifier_env *env,u32 regno, -enum reg_arg_type t) +static int __check_reg_arg (struct bpf_verifier_env *env,struct bpf_reg_state *regs,u32 regno, +enum reg_arg_type t) { -struct bpf_verifier_state *vstate = env->cur_state; -struct bpf_func_state *state = vstate->frame[vstate->curframe];struct bpf_insn *insn = env->prog->insnsi + env->insn_idx; -struct bpf_reg_state *reg, *regs = state->regs; +struct bpf_reg_state *reg;bool rw64;if (regno >=MAX_BPF_REG ) {
@@ -3368,6 +3366,15 @@ static int check_reg_arg(struct bpf_verireturn 0 ; } +static int check_reg_arg (struct bpf_verifier_env *env,u32 regno, +enum reg_arg_type t) +{ +struct bpf_verifier_state *vstate = env->cur_state; +struct bpf_func_state *state = vstate->frame[vstate->curframe]; + +return __check_reg_arg (env, state->regs, regno, t); +} +static void mark_jmp_point (struct bpf_verifier_env *env,int idx) { env->insn_aux_data[idx].jmp_point = true;
@@ -9147,7 +9154,7 @@ static void clear_caller_saved_regs(stru/* after the call registers r0 - r5 were scratched */ for (i =0 ; i <CALLER_SAVED_REGS ; i++) {mark_reg_not_init (env, regs, caller_saved[i]); -check_reg_arg (env, caller_saved[i],DST_OP_NO_MARK ); +__check_reg_arg (env, regs, caller_saved[i],DST_OP_NO_MARK ); } }
Greg Kroah-Hartman
maintainerPATCH 156/331 bpf: extract setup_func_entry() utility function
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 58124a98cb8eda69d248d7f1de954c8b2767c945 upstream.
Move code for simulated stack frame creation to a separate utility function. This function would be used in the follow-up change for callbacks handling.
--- kernel/bpf/verifier.c | 84 ++++++++++++++++++++++++++++---------------------- 1 file changed, 48 insertions(+), 36 deletions(-)
kernel/bpf/verifier.c +48 -36
@@ -9167,11 +9167,10 @@ static int set_callee_state(struct bpf_vstruct bpf_func_state *caller, struct bpf_func_state *callee, int insn_idx); -static int __check_func_call (struct bpf_verifier_env *env,struct bpf_insn *insn, -int *insn_idx,int subprog, -set_callee_state_fn set_callee_state_cb) +static int setup_func_entry (struct bpf_verifier_env *env,int subprog,int callsite, +set_callee_state_fn set_callee_state_cb, +struct bpf_verifier_state *state) { -struct bpf_verifier_state *state = env->cur_state;struct bpf_func_state *caller, *callee;int err;
@@ -9181,13 +9180,53 @@ static int __check_func_call(struct bpf_return -E2BIG ; } - caller = state->frame[state->curframe];if (state->frame[state->curframe +1 ]) {verbose (env,"verifier bug. Frame %d already allocated\n" , state->curframe +1 );return -EFAULT ; } + caller = state->frame[state->curframe]; + callee =kzalloc (sizeof (*callee),GFP_KERNEL ); +if (!callee) +return -ENOMEM ; + state->frame[state->curframe +1 ] = callee; + +/* callee cannot access r0, r6 - r9 for reading and has to write +* into its own stack before reading from it. +* callee can read/write into caller's stack +*/ +init_func_state (env, callee, +/* remember the callsite, it will be used by bpf_exit */ + callsite, + state->curframe +1 /* frameno within this callchain */ , + subprog/* subprog number within this prog */ ); +/* Transfer references to the callee */ + err =copy_reference_state (callee, caller); + err = err ?:set_callee_state_cb (env, caller, callee, callsite); +if (err) + goto err_out; + +/* only increment it after check_reg_arg() finished */ + state->curframe++; + +return 0 ; + +err_out: +free_func_state (callee); + state->frame[state->curframe +1 ] =NULL ; +return err; +} + +static int __check_func_call (struct bpf_verifier_env *env,struct bpf_insn *insn, +int *insn_idx,int subprog, +set_callee_state_fn set_callee_state_cb) +{ +struct bpf_verifier_state *state = env->cur_state; +struct bpf_func_state *caller, *callee; +int err; + + caller = state->frame[state->curframe]; err =btf_check_subprog_call (env, subprog, caller->regs);if (err == -EFAULT )return err;
@@ -9256,35 +9295,12 @@ static int __check_func_call(struct bpf_return 0 ; } - callee =kzalloc (sizeof (*callee),GFP_KERNEL ); -if (!callee) -return -ENOMEM ; - state->frame[state->curframe +1 ] = callee; - -/* callee cannot access r0, r6 - r9 for reading and has to write -* into its own stack before reading from it. -* callee can read/write into caller's stack -*/ -init_func_state (env, callee, -/* remember the callsite, it will be used by bpf_exit */ - *insn_idx/* callsite */ , - state->curframe +1 /* frameno within this callchain */ , - subprog/* subprog number within this prog */ ); - -/* Transfer references to the callee */ - err =copy_reference_state (callee, caller); + err =setup_func_entry (env, subprog, *insn_idx, set_callee_state_cb, state);if (err) - goto err_out; - - err =set_callee_state_cb (env, caller, callee, *insn_idx); -if (err) - goto err_out; +return err;clear_caller_saved_regs (env, caller->regs); -/* only increment it after check_reg_arg() finished */ - state->curframe++; -/* and go analyze first insn of the callee */ *insn_idx = env->subprog_info[subprog].start -1 ;
@@ -9292,14 +9308,10 @@ static int __check_func_call(struct bpf_verbose (env,"caller:\n" );print_verifier_state (env, caller, true);verbose (env,"callee:\n" ); -print_verifier_state (env, callee, true); +print_verifier_state (env, state->frame[state->curframe], true); } -return 0 ; -err_out: -free_func_state (callee); - state->frame[state->curframe +1 ] =NULL ; -return err; +return 0 ; } int map_set_for_each_callback_args(struct bpf_verifier_env *env,
Greg Kroah-Hartman
maintainerPATCH 157/331 bpf: verify callbacks as if they are called unknown number of times
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit ab5cfac139ab8576fb54630d4cca23c3e690ee90 upstream.
Prior to this patch callbacks were handled as regular function calls, execution of callback body was modeled exactly once.
This patch updates callbacks handling logic as follows:
- introduces a function push_callback_call() that schedules callback body verification in env->head stack;
- updates prepare_func_exit() to reschedule callback body verification upon BPF_EXIT;
- as calls to bpf_*_iter_next(), calls to callback invoking functions are marked as checkpoints;
- is_state_visited() is updated to stop callback based iteration when some identical parent state is found.
Paths with callback function invoked zero times are now verified first, which leads to necessity to modify some selftests:
- the following negative tests required adding release/unlock/drop calls to avoid previously masked unrelated error reports:
- cb_refs.c:underflow_prog
- exceptions_fail.c:reject_rbtree_add_throw
- exceptions_fail.c:reject_with_cp_reference
- the following precision tracking selftests needed change in expected log trace:
- verifier_subprog_precision.c:callback_result_precise
(note: r0 precision is no longer propagated inside callback and
I think this is a correct behavior)
- verifier_subprog_precision.c:parent_callee_saved_reg_precise_with_callback
- verifier_subprog_precision.c:parent_stack_slot_precise_with_callback
--- include/linux/bpf_verifier.h | 5 kernel/bpf/verifier.c | 272 ++++++---- tools/testing/selftests/bpf/progs/cb_refs.c | 1 tools/testing/selftests/bpf/progs/verifier_subprog_precision.c | 71 ++ 4 files changed, 237 insertions(+), 112 deletions(-)
include/linux/bpf_verifier.h +5 -0
@@ -399,6 +399,7 @@ struct bpf_verifier_state {struct bpf_idx_pair *jmp_history;u32 jmp_history_cnt;u32 dfs_depth; +u32 callback_unroll_depth; };#define bpf_get_spilled_reg (slot, frame) \
@@ -506,6 +507,10 @@ struct bpf_insn_aux_data {* this instruction, regardless of any heuristics */ bool force_checkpoint; +/* true if instruction is a call to a helper function that +* accepts callback function as a parameter. +*/ +bool calls_callback; };#define MAX_USED_MAPS 64/* max number of maps accessed by one eBPF program */
kernel/bpf/verifier.c +174 -98
437 lines not on this page · raw message
tools/testing/selftests/bpf/progs/cb_refs.c +1 -0
@@ -33,6 +33,7 @@ int underflow_prog(void *ctx)if (!p)return 0 ;bpf_for_each_map_elem (&array_map, cb1, &p,0 ); +bpf_kfunc_call_test_release (p);return 0 ; }
tools/testing/selftests/bpf/progs/verifier_subprog_precision.c +57 -14
114 lines not on this page · raw message
Greg Kroah-Hartman
maintainerPATCH 158/331 selftests/bpf: tests for iterating callbacks
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 958465e217dbf5fc6677d42d8827fb3073d86afd upstream.
A set of test cases to check behavior of callback handling logic, check if verifier catches the following situations:
- program not safe on second callback iteration;
- program not safe on zero callback iterations;
- infinite loop inside a callback.
Verify that callback logic works for bpf_loop, bpf_for_each_map_elem, bpf_user_ringbuf_drain, bpf_find_vma.
--- tools/testing/selftests/bpf/prog_tests/verifier.c | 2 tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c | 147 ++++++++++ 2 files changed, 149 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
@@ -31,6 +31,7 @@#include "verifier_helper_restricted.skel.h" #include "verifier_helper_value_access.skel.h" #include "verifier_int_ptr.skel.h" +#include "verifier_iterating_callbacks.skel.h" #include "verifier_jeq_infer_not_null.skel.h" #include "verifier_ld_ind.skel.h" #include "verifier_ldsx.skel.h"
@@ -138,6 +139,7 @@ void test_verifier_helper_packet_access(void test_verifier_helper_restricted (void ) {RUN (verifier_helper_restricted); }void test_verifier_helper_value_access (void ) {RUN (verifier_helper_value_access); }void test_verifier_int_ptr (void ) {RUN (verifier_int_ptr); } +void test_verifier_iterating_callbacks (void ) {RUN (verifier_iterating_callbacks); }void test_verifier_jeq_infer_not_null (void ) {RUN (verifier_jeq_infer_not_null); }void test_verifier_ld_ind (void ) {RUN (verifier_ld_ind); }void test_verifier_ldsx (void ) {RUN (verifier_ldsx); }
tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c new file +147 -0
147 lines not on this page · raw message
Greg Kroah-Hartman
maintainerPATCH 159/331 bpf: widening for callback iterators
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit cafe2c21508a38cdb3ed22708842e957b2572c3e upstream.
Callbacks are similar to open coded iterators, so add imprecise widening logic for callback body processing. This makes callback based loops behave identically to open coded iterators, e.g. allowing to verify programs like below:
struct ctx { u32 i; };
int cb(u32 idx, struct ctx* ctx)
{
++ctx->i;
return 0;
}
...
struct ctx ctx = { .i = 0 };
bpf_loop(100, cb, &ctx, 0);
...--- kernel/bpf/verifier.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-)
kernel/bpf/verifier.c +22 -2
@@ -9595,9 +9595,10 @@ static bool in_rbtree_lock_required_cb(sstatic int prepare_func_exit (struct bpf_verifier_env *env,int *insn_idx) { -struct bpf_verifier_state *state = env->cur_state; +struct bpf_verifier_state *state = env->cur_state, *prev_st;struct bpf_func_state *caller, *callee;struct bpf_reg_state *r0; +bool in_callback_fn;int err; callee = state->frame[state->curframe];
@@ -9659,7 +9660,8 @@ static int prepare_func_exit(struct bpf_* there function call logic would reschedule callback visit. If iteration * converges is_state_visited() would prune that visit eventually. */ -if (callee->in_callback_fn) + in_callback_fn = callee->in_callback_fn; +if (in_callback_fn) *insn_idx = callee->callsite;else *insn_idx = callee->callsite +1 ;
@@ -9673,6 +9675,24 @@ static int prepare_func_exit(struct bpf_/* clear everything in the callee */ free_func_state (callee); state->frame[state->curframe--] =NULL ; + +/* for callbacks widen imprecise scalars to make programs like below verify: +* +* struct ctx { int i; } +* void cb(int idx, struct ctx *ctx) { ctx->i++; ... } +* ... +* struct ctx = { .i = 0; } +* bpf_loop(100, cb, &ctx, 0); +* +* This is similar to what is done in process_iter_next_call() for open +* coded iterators. +*/ + prev_st = in_callback_fn ?find_prev_entry (env, state, *insn_idx) :NULL ; +if (prev_st) { + err =widen_imprecise_scalars (env, prev_st, state); +if (err) +return err; + }return 0 ; }
Greg Kroah-Hartman
maintainerPATCH 160/331 selftests/bpf: test widening for iterating callbacks
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 9f3330aa644d6d979eb064c46e85c62d4b4eac75 upstream.
A test case to verify that imprecise scalars widening is applied to callback entering state, when callback call is simulated repeatedly.
--- tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c | 20 ++++++++++ 1 file changed, 20 insertions(+)
tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c +20 -0
@@ -25,6 +25,7 @@ struct buf_context {struct num_context {__u64 i; +__u64 j; };__u8 choice_arr[2 ] = {0 ,1 };
@@ -69,6 +70,25 @@ int unsafe_on_zero_iter(void *unused)return choice_arr[loop_ctx.i]; } +static int widening_cb (__u32 idx,struct num_context *ctx) +{ + ++ctx->i; +return 0 ; +} + +SEC("?raw_tp") +__success +int widening (void *unused) +{ +struct num_context loop_ctx = { .i =0 , .j =1 }; + +bpf_loop (100 , widening_cb, &loop_ctx,0 ); +/* loop_ctx.j is not changed during callback iteration, +* verifier should not apply widening to it. +*/ +return choice_arr[loop_ctx.j]; +} +static int loop_detection_cb (__u32 idx,struct num_context *ctx) {for (;;) {}
Greg Kroah-Hartman
maintainerPATCH 161/331 bpf: keep track of max number of bpf_loop callback iterations
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit bb124da69c47dd98d69361ec13244ece50bec63e upstream.
In some cases verifier can't infer convergence of the bpf_loop() iteration. E.g. for the following program:
static int cb(__u32 idx, struct num_context* ctx)
{
ctx->i++;
return 0;
}
SEC("?raw_tp")
int prog(void *_)
{
struct num_context ctx = { .i = 0 };
__u8 choice_arr[2] = { 0, 1 };
bpf_loop(2, cb, &ctx, 0);
return choice_arr[ctx.i];
}Each 'cb' simulation would eventually return to 'prog' and reach 'return choice_arr[ctx.i]' statement. At which point ctx.i would be marked precise, thus forcing verifier to track multitude of separate states with {.i=0}, {.i=1}, ... at bpf_loop() callback entry.
This commit allows "brute force" handling for such cases by limiting number of callback body simulations using 'umax' value of the first bpf_loop() parameter.
For this, extend bpf_func_state with 'callback_depth' field. Increment this field when callback visiting state is pushed to states traversal stack. For frame #N it's 'callback_depth' field counts how many times callback with frame depth N+1 had been executed. Use bpf_func_state specifically to allow independent tracking of callback depths when multiple nested bpf_loop() calls are present.
--- include/linux/bpf_verifier.h | 11 +++ kernel/bpf/verifier.c | 19 ++++- tools/testing/selftests/bpf/progs/verifier_subprog_precision.c | 35 +++++++--- 3 files changed, 53 insertions(+), 12 deletions(-)
include/linux/bpf_verifier.h +11 -0
@@ -300,6 +300,17 @@ struct bpf_func_state {bool in_callback_fn;struct tnum callback_ret_range;bool in_async_callback_fn; +/* For callback calling functions that limit number of possible +* callback executions (e.g. bpf_loop) keeps track of current +* simulated iteration number. +* Value in frame N refers to number of times callback with frame +* N+1 was simulated, e.g. for the following call: +* +* bpf_loop(..., fn, ...); | suppose current frame is N +* | fn would be simulated in frame N+1 +* | number of simulations is tracked in frame N +*/ +u32 callback_depth;/* The following fields should be last. See copy_func_state() */ int acquired_refs;
kernel/bpf/verifier.c +17 -2
@@ -9301,6 +9301,8 @@ static int push_callback_call(struct bpfreturn err; callback_state->callback_unroll_depth++; + callback_state->frame[callback_state->curframe -1 ]->callback_depth++; + caller->callback_depth =0 ;return 0 ; }
@@ -10090,8 +10092,21 @@ static int check_helper_call(struct bpf_break ;case BPF_FUNC_loop:update_loop_inline_state (env, meta.subprogno); - err =push_callback_call (env, insn, insn_idx, meta.subprogno, - set_loop_callback_state); +/* Verifier relies on R1 value to determine if bpf_loop() iteration +* is finished, thus mark it precise. +*/ + err =mark_chain_precision (env,BPF_REG_1 ); +if (err) +return err; +if (cur_func (env)->callback_depth < regs[BPF_REG_1 ].umax_value) { + err =push_callback_call (env, insn, insn_idx, meta.subprogno, + set_loop_callback_state); + }else { +cur_func (env)->callback_depth =0 ; +if (env->log.level &BPF_LOG_LEVEL2 ) +verbose (env,"frame%d bpf_loop iteration limit reached\n" , + env->cur_state->curframe); + }break ;case BPF_FUNC_dynptr_from_mem:if (regs[BPF_REG_1 ].type !=PTR_TO_MAP_VALUE ) {
tools/testing/selftests/bpf/progs/verifier_subprog_precision.c +25 -10
@@ -119,7 +119,23 @@ __naked int global_subprog_result_precis SEC("?raw_tp") __success __log_level(2) -/* First simulated path does not include callback body */ +/* First simulated path does not include callback body, +* r1 and r4 are always precise for bpf_loop() calls. +*/ +__msg("9: (85) call bpf_loop#181") +__msg("mark_precise: frame0: last_idx 9 first_idx 9 subseq_idx -1") +__msg("mark_precise: frame0: parent state regs=r4 stack=:") +__msg("mark_precise: frame0: last_idx 8 first_idx 0 subseq_idx 9") +__msg("mark_precise: frame0: regs=r4 stack= before 8: (b7) r4 = 0") +__msg("mark_precise: frame0: last_idx 9 first_idx 9 subseq_idx -1") +__msg("mark_precise: frame0: parent state regs=r1 stack=:") +__msg("mark_precise: frame0: last_idx 8 first_idx 0 subseq_idx 9") +__msg("mark_precise: frame0: regs=r1 stack= before 8: (b7) r4 = 0") +__msg("mark_precise: frame0: regs=r1 stack= before 7: (b7) r3 = 0") +__msg("mark_precise: frame0: regs=r1 stack= before 6: (bf) r2 = r8") +__msg("mark_precise: frame0: regs=r1 stack= before 5: (bf) r1 = r6") +__msg ("mark_precise: frame0: regs=r6 stack= before 4: (b7) r6 = 3" ) +/* r6 precision propagation */ __msg("14: (0f) r1 += r6") __msg("mark_precise: frame0: last_idx 14 first_idx 9")__msg ("mark_precise: frame0: regs=r6 stack= before 13: (bf) r1 = r7" )
@@ -134,10 +150,9 @@ __msg("17: (b7) r0 = 0") __msg("18: (95) exit") __msg("returning from callee:") __msg("to caller at 9:") -/* r4 (flags) is always precise for bpf_loop() */ -__msg("frame 0: propagating r4") +__msg("frame 0: propagating r1,r4") __msg("mark_precise: frame0: last_idx 9 first_idx 9 subseq_idx -1") -__msg("mark_precise: frame0: regs=r4 stack= before 18: (95) exit") +__msg("mark_precise: frame0: regs=r1,r4 stack= before 18: (95) exit") __msg("from 18 to 9: safe")__naked int callback_result_precise (void ) {
@@ -264,12 +279,12 @@ __msg("15: (b7) r0 = 0") __msg("16: (95) exit") __msg("returning from callee:")__msg ("to caller at 9:" ) -/* r4 (flags) is always precise for bpf_loop(), +/* r1, r4 are always precise for bpf_loop(), * r6 was marked before backtracking to callback body. */ -__msg("frame 0: propagating r4,r6") +__msg("frame 0: propagating r1,r4,r6") __msg("mark_precise: frame0: last_idx 9 first_idx 9 subseq_idx -1") -__msg("mark_precise: frame0: regs=r4,r6 stack= before 16: (95) exit") +__msg("mark_precise: frame0: regs=r1,r4,r6 stack= before 16: (95) exit") __msg("mark_precise: frame1: regs= stack= before 15: (b7) r0 = 0") __msg("mark_precise: frame1: regs= stack= before 9: (85) call bpf_loop")__msg ("mark_precise: frame0: parent state regs= stack=:" )
@@ -422,12 +437,12 @@ __msg("17: (b7) r0 = 0") __msg("18: (95) exit") __msg("returning from callee:")__msg ("to caller at 10:" ) -/* r4 (flags) is always precise for bpf_loop(), +/* r1, r4 are always precise for bpf_loop(), * fp-8 was marked before backtracking to callback body. */ -__msg("frame 0: propagating r4,fp-8") +__msg("frame 0: propagating r1,r4,fp-8") __msg("mark_precise: frame0: last_idx 10 first_idx 10 subseq_idx -1") -__msg("mark_precise: frame0: regs=r4 stack=-8 before 18: (95) exit") +__msg("mark_precise: frame0: regs=r1,r4 stack=-8 before 18: (95) exit") __msg("mark_precise: frame1: regs= stack= before 17: (b7) r0 = 0") __msg("mark_precise: frame1: regs= stack= before 10: (85) call bpf_loop#181")__msg ("mark_precise: frame0: parent state regs= stack=:" )
Greg Kroah-Hartman
maintainerPATCH 162/331 selftests/bpf: check if max number of bpf_loop iterations is tracked
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eduard Zingerman <[email protected]>
commit 57e2a52deeb12ab84c15c6d0fb93638b5b94001b upstream.
Check that even if bpf_loop() callback simulation does not converge to a specific state, verification could proceed via "brute force" simulation of maximal number of callback calls.
--- tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c | 75 ++++++++++ 1 file changed, 75 insertions(+)
tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c +75 -0
@@ -164,4 +164,79 @@ int unsafe_find_vma(void *unused)return choice_arr[loop_ctx.i]; } +static int iter_limit_cb (__u32 idx,struct num_context *ctx) +{ + ctx->i++; +return 0 ; +} + +SEC("?raw_tp") +__success +int bpf_loop_iter_limit_ok (void *unused) +{ +struct num_context ctx = { .i =0 }; + +bpf_loop (1 , iter_limit_cb, &ctx,0 ); +return choice_arr[ctx.i]; +} + +SEC("?raw_tp") +__failure __msg("invalid access to map value, value_size=2 off=2 size=1") +int bpf_loop_iter_limit_overflow(void *unused) +{ +struct num_context ctx = { .i =0 }; + +bpf_loop (2 , iter_limit_cb, &ctx,0 ); +return choice_arr[ctx.i]; +} + +static int iter_limit_level2a_cb (__u32 idx,struct num_context *ctx) +{ + ctx->i +=100 ; +return 0 ; +} + +static int iter_limit_level2b_cb (__u32 idx,struct num_context *ctx) +{ + ctx->i +=10 ; +return 0 ; +} + +static int iter_limit_level1_cb (__u32 idx,struct num_context *ctx) +{ + ctx->i +=1 ; +bpf_loop (1 , iter_limit_level2a_cb, ctx,0 ); +bpf_loop (1 , iter_limit_level2b_cb, ctx,0 ); +return 0 ; +} + +/* Check that path visiting every callback function once had been +* reached by verifier. Variables 'ctx{1,2}i' below serve as flags, +* with each decimal digit corresponding to a callback visit marker. +*/ +SEC("socket") +__success __retval(111111) +int bpf_loop_iter_limit_nested(void *unused) +{ +struct num_context ctx1 = { .i =0 }; +struct num_context ctx2 = { .i =0 }; +__u64 a, b, c; + +bpf_loop (1 , iter_limit_level1_cb, &ctx1,0 ); +bpf_loop (1 , iter_limit_level1_cb, &ctx2,0 ); + a = ctx1.i; + b = ctx2.i; +/* Force 'ctx1.i' and 'ctx2.i' precise. */ + c = choice_arr[(a + b) %2 ]; +/* This makes 'c' zero, but neither clang nor verifier know it. */ + c /=10 ; +/* Make sure that verifier does not visit 'impossible' states: +* enumerate all possible callback visit masks. +*/ +if (a !=0 && a !=1 && a !=11 && a !=101 && a !=111 && + b !=0 && b !=1 && b !=11 && b !=101 && b !=111 ) + asmvolatile ("r0 /= 0;" :::"r0" ); +return 1000 * a + b + c; +} + char _license[] SEC("license") = "GPL";
Greg Kroah-Hartman
maintainerPATCH 163/331 Revert "drm/amd: Enable PCIe PME from D3"
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Jonathan Gray <[email protected]>
This reverts commit 847e6947afd3c46623172d2eabcfc2481ee8668e.
duplicated a change made in 6.6.549227bea27ebcd260f0c94a3055b14bbd8605c5e
--- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 2 -- 1 file changed, 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +0 -2
@@ -2197,8 +2197,6 @@ retry_init:pci_wake_from_d3 (pdev, TRUE); -pci_wake_from_d3 (pdev, TRUE); -/* * For runpm implemented via BACO, PMFW will handle the * timing for BACO in and out:
Greg Kroah-Hartman
maintainerPATCH 164/331 cifs: fix lock ordering while disabling multichannel
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Shyam Prasad N <[email protected]>
commit 5eef12c4e3230f2025dc46ad8c4a3bc19978e5d7 upstream.
The code to handle the case of server disabling multichannel was picking iface_lock with chan_lock held. This goes against the lock ordering rules, as iface_lock is a higher order lock (even if it isn't so obvious).
This change fixes the lock ordering by doing the following in that order for each secondary channel:
1. store iface and server pointers in local variable
2. remove references to iface and server in channels
3. unlock chan_lock
4. lock iface_lock
5. dec ref count for iface
6. unlock iface_lock
7. dec ref count for server
8. lock chan_lock again
Since this function can only be called in smb2_reconnect, and that cannot be called by two parallel processes, we should not have races due to dropping chan_lock between steps 3 and 8.
--- fs/smb/client/sess.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)
fs/smb/client/sess.c +13 -9
@@ -316,28 +316,32 @@ cifs_disable_secondary_channels(struct c iface = ses->chans[i].iface; server = ses->chans[i].server; +/* +* remove these references first, since we need to unlock +* the chan_lock here, since iface_lock is a higher lock +*/ + ses->chans[i].iface =NULL ; + ses->chans[i].server =NULL ; +spin_unlock (&ses->chan_lock); +if (iface) {spin_lock (&ses->iface_lock);kref_put (&iface->refcount, release_iface); - ses->chans[i].iface =NULL ; iface->num_channels--;if (iface->weight_fulfilled) iface->weight_fulfilled--;spin_unlock (&ses->iface_lock); } -spin_unlock (&ses->chan_lock); -if (server && !server->terminate) { - server->terminate = true; -cifs_signal_cifsd_for_reconnect (server, false); - } -spin_lock (&ses->chan_lock); -if (server) { - ses->chans[i].server =NULL ; +if (!server->terminate) { + server->terminate = true; +cifs_signal_cifsd_for_reconnect (server, false); + }cifs_put_tcp_session (server, false); } +spin_lock (&ses->chan_lock); } done:
Greg Kroah-Hartman
maintainerPATCH 165/331 cifs: fix a pending undercount of srv_count
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Shyam Prasad N <[email protected]>
commit f30bbc38704e279c06d073ecb18fea376791ecab upstream.
The following commit reverted the changes to ref count
the server struct while scheduling a reconnect work:
823342524868 Revert "cifs: reconnect work should have reference on server struct"
However, a following change also introduced scheduling of reconnect work, and assumed ref counting. This change fixes that as well.
Fixes umount problems like:
[73496.157838] CPU: 5 PID: 1321389 Comm: umount Tainted: G W OE 6.7.0-060700rc6-generic #202312172332 [73496.157841] Hardware name: LENOVO 20MAS08500/20MAS08500, BIOS N2CET67W (1.50 ) 12/15/2022 [73496.157843] RIP: 0010:cifs_put_tcp_session+0x17d/0x190 [cifs] [73496.157906] Code: 5d 31 c0 31 d2 31 f6 31 ff c3 cc cc cc cc e8 4a 6e 14 e6 e9 f6 fe ff ff be 03 00 00 00 48 89 d7 e8 78 26 b3 e5 e9 e4 fe ff ff <0f> 0b e9 b1 fe ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 [73496.157908] RSP: 0018:ffffc90003bcbcb8 EFLAGS: 00010286 [73496.157911] RAX: 00000000ffffffff RBX: ffff8885830fa800 RCX: 0000000000000000 [73496.157913] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 [73496.157915] RBP: ffffc90003bcbcc8 R08: 0000000000000000 R09: 0000000000000000 [73496.157917] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 [73496.157918] R13: ffff8887d56ba800 R14: 00000000ffffffff R15: ffff8885830fa800 [73496.157920] FS: 00007f1ff0e33800(0000) GS:ffff88887ba80000(0000) knlGS:0000000000000000 [73496.157922] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [73496.157924] CR2: 0000115f002e2010 CR3: 00000003d1e24005 CR4: 00000000003706f0 [73496.157926] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [73496.157928] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [73496.157929] Call Trace: [73496.157931] <TASK> [73496.157933] ? show_regs+0x6d/0x80 [73496.157936] ? __warn+0x89/0x160 [73496.157939] ? cifs_put_tcp_session+0x17d/0x190 [cifs] [73496.157976] ? report_bug+0x17e/0x1b0 [73496.157980] ? handle_bug+0x51/0xa0 [73496.157983] ? exc_invalid_op+0x18/0x80 [73496.157985] ? asm_exc_invalid_op+0x1b/0x20 [73496.157989] ? cifs_put_tcp_session+0x17d/0x190 [cifs] [73496.158023] ? cifs_put_tcp_session+0x1e/0x190 [cifs] [73496.158057] __cifs_put_smb_ses+0x2b5/0x540 [cifs] [73496.158090] ? tconInfoFree+0xc2/0x120 [cifs] [73496.158130] cifs_put_tcon.part.0+0x108/0x2b0 [cifs] [73496.158173] cifs_put_tlink+0x49/0x90 [cifs] [73496.158220] cifs_umount+0x56/0xb0 [cifs] [73496.158258] cifs_kill_sb+0x52/0x60 [cifs] [73496.158306] deactivate_locked_super+0x32/0xc0 [73496.158309] deactivate_super+0x46/0x60 [73496.158311] cleanup_mnt+0xc3/0x170 [73496.158314] __cleanup_mnt+0x12/0x20 [73496.158330] task_work_run+0x5e/0xa0 [73496.158333] exit_to_user_mode_loop+0x105/0x130 [73496.158336] exit_to_user_mode_prepare+0xa5/0xb0 [73496.158338] syscall_exit_to_user_mode+0x29/0x60 [73496.158341] do_syscall_64+0x6c/0xf0 [73496.158344] ? syscall_exit_to_user_mode+0x37/0x60 [73496.158346] ? do_syscall_64+0x6c/0xf0 [73496.158349] ? exit_to_user_mode_prepare+0x30/0xb0 [73496.158353] ? syscall_exit_to_user_mode+0x37/0x60 [73496.158355] ? do_syscall_64+0x6c/0xf0
--- fs/smb/client/smb2pdu.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
fs/smb/client/smb2pdu.c +1 -2
@@ -440,8 +440,7 @@ skip_sess_setup: skip_add_channels:if (smb2_command !=SMB2_INTERNAL_CMD ) -if (mod_delayed_work (cifsiod_wq, &server->reconnect,0 )) -cifs_put_tcp_session (server, false); +mod_delayed_work (cifsiod_wq, &server->reconnect,0 );atomic_inc (&tconInfoReconnectCount); out:
Greg Kroah-Hartman
maintainerPATCH 166/331 cifs: after disabling multichannel, mark tcon for reconnect
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Shyam Prasad N <[email protected]>
commit 27e1fd343f80168ff456785c2443136b6b7ca3cc upstream.
Once the server disables multichannel for an active multichannel session, on the following reconnect, the client would reduce the number of channels to 1. However, it could be the case that the tree connect was active on one of these disabled channels. This results in an unrecoverable state.
This change fixes that by making sure that whenever a channel is being terminated, the session and tcon are marked for reconnect too. This could mean a few redundant tree connect calls to the server, but considering that this is not a frequent event, we should be okay.
--- fs/smb/client/connect.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
fs/smb/client/connect.c +12 -8
@@ -212,17 +212,21 @@ cifs_mark_tcp_ses_conns_for_reconnect(st/* If server is a channel, select the primary channel */ pserver =SERVER_IS_CHAN (server) ? server->primary_server : server; +/* +* if the server has been marked for termination, there is a +* chance that the remaining channels all need reconnect. To be +* on the safer side, mark the session and trees for reconnect +* for this scenario. This might cause a few redundant session +* setup and tree connect requests, but it is better than not doing +* a tree connect when needed, and all following requests failing +*/ +if (server->terminate) { + mark_smb_session = true; + server = pserver; + }spin_lock (&cifs_tcp_ses_lock);list_for_each_entry_safe (ses, nses, &pserver->smb_ses_list, smb_ses_list) { -/* -* if channel has been marked for termination, nothing to do -* for the channel. in fact, we cannot find the channel for the -* server. So safe to exit here -*/ -if (server->terminate) -break ; -/* check if iface is still active */ spin_lock (&ses->chan_lock);if (!cifs_chan_is_iface_active (ses, server)) {
Greg Kroah-Hartman
maintainerPATCH 167/331 SUNRPC: use request size to initialize bio_vec in svc_udp_sendto()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Lucas Stach <[email protected]>
[ Upstream commit 1d9cabe2817edd215779dc9c2fe5e7ab9aac0704 ]
Use the proper size when setting up the bio_vec, as otherwise only zero-length UDP packets will be sent.
--- net/sunrpc/svcsock.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
net/sunrpc/svcsock.c +2 -2
@@ -717,12 +717,12 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) ARRAY_SIZE(rqstp->rq_bvec), xdr);iov_iter_bvec (&msg.msg_iter,ITER_SOURCE , rqstp->rq_bvec, - count,0 ); + count, rqstp->rq_res.len); err =sock_sendmsg (svsk->sk_sock, &msg);if (err == -ECONNREFUSED ) {/* ICMP error on earlier request. */ iov_iter_bvec (&msg.msg_iter,ITER_SOURCE , rqstp->rq_bvec, - count,0 ); + count, rqstp->rq_res.len); err =sock_sendmsg (svsk->sk_sock, &msg); }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 168/331 wifi: mac80211: fix potential sta-link leak
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Johannes Berg <[email protected]>
[ Upstream commit b01a74b3ca6fd51b62c67733ba7c3280fa6c5d26 ]
When a station is allocated, links are added but not set to valid yet (e.g. during connection to an AP MLD), we might remove the station without ever marking links valid, and leak them. Fix that.
--- net/mac80211/sta_info.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
net/mac80211/sta_info.c +4 -1
@@ -398,7 +398,10 @@ void sta_info_free(struct ieee80211_local *local, struct sta_info *sta)int i;for (i =0 ; i <ARRAY_SIZE (sta->link); i++) { -if (!(sta->sta.valid_links &BIT (i))) +struct link_sta_info *link_sta; + + link_sta =rcu_access_pointer (sta->link[i]); +if (!link_sta)continue ;sta_remove_link (sta, i, false);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 169/331 net/smc: fix illegal rmb_desc access in SMC-D connection dump
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Wen Gu <[email protected]>
[ Upstream commit dbc153fd3c142909e564bb256da087e13fbf239c ]
A crash was found when dumping SMC-D connections. It can be reproduced by following steps:
- run nginx/wrk test:
smc_run nginx
smc_run wrk -t 16 -c 1000 -d <duration> -H 'Connection: Close' <URL>
- continuously dump SMC-D connections in parallel:
watch -n 1 'smcss -D'
BUG: kernel NULL pointer dereference, address: 0000000000000030 CPU: 2 PID: 7204 Comm: smcss Kdump: loaded Tainted: G E 6.7.0+ #55 RIP: 0010:__smc_diag_dump.constprop.0+0x5e5/0x620 [smc_diag] Call Trace: <TASK> ? __die+0x24/0x70 ? page_fault_oops+0x66/0x150 ? exc_page_fault+0x69/0x140 ? asm_exc_page_fault+0x26/0x30 ? __smc_diag_dump.constprop.0+0x5e5/0x620 [smc_diag] ? __kmalloc_node_track_caller+0x35d/0x430 ? __alloc_skb+0x77/0x170 smc_diag_dump_proto+0xd0/0xf0 [smc_diag] smc_diag_dump+0x26/0x60 [smc_diag] netlink_dump+0x19f/0x320 __netlink_dump_start+0x1dc/0x300 smc_diag_handler_dump+0x6a/0x80 [smc_diag] ? __pfx_smc_diag_dump+0x10/0x10 [smc_diag] sock_diag_rcv_msg+0x121/0x140 ? __pfx_sock_diag_rcv_msg+0x10/0x10 netlink_rcv_skb+0x5a/0x110 sock_diag_rcv+0x28/0x40 netlink_unicast+0x22a/0x330 netlink_sendmsg+0x1f8/0x420 __sock_sendmsg+0xb0/0xc0 ____sys_sendmsg+0x24e/0x300 ? copy_msghdr_from_user+0x62/0x80 ___sys_sendmsg+0x7c/0xd0 ? __do_fault+0x34/0x160 ? do_read_fault+0x5f/0x100 ? do_fault+0xb0/0x110 ? __handle_mm_fault+0x2b0/0x6c0 __sys_sendmsg+0x4d/0x80 do_syscall_64+0x69/0x180 entry_SYSCALL_64_after_hwframe+0x6e/0x76
It is possible that the connection is in process of being established when we dump it. Assumed that the connection has been registered in a link group by smc_conn_create() but the rmb_desc has not yet been initialized by smc_buf_create(), thus causing the illegal access to conn->rmb_desc. So fix it by checking before dump.
--- net/smc/smc_diag.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/smc/smc_diag.c +1 -1
@@ -163,7 +163,7 @@ static int __smc_diag_dump(struct sock *sk, struct sk_buff *skb, }if (smc_conn_lgr_valid (&smc->conn) && smc->conn.lgr->is_smcd && (req->diag_ext & (1 << (SMC_DIAG_DMBINFO -1 ))) && - !list_empty (&smc->conn.lgr->list)) { + !list_empty (&smc->conn.lgr->list) && smc->conn.rmb_desc) {struct smc_connection *conn = &smc->conn;struct smcd_diag_dmbinfo dinfo;struct smcd_dev *smcd = conn->lgr->smcd;
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 170/331 selftests: bonding: Increase timeout to 1200s
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Benjamin Poirier <[email protected]>
[ Upstream commit b01f15a7571b7aa222458bc9bf26ab59bd84e384 ]
When tests are run by runner.sh, bond_options.sh gets killed before it can complete:
make -C tools/testing/selftests run_tests TARGETS="drivers/net/bonding"
[...]
# timeout set to 120
# selftests: drivers/net/bonding: bond_options.sh
# TEST: prio (active-backup miimon primary_reselect 0) [ OK ]
# TEST: prio (active-backup miimon primary_reselect 1) [ OK ]
# TEST: prio (active-backup miimon primary_reselect 2) [ OK ]
# TEST: prio (active-backup arp_ip_target primary_reselect 0) [ OK ]
# TEST: prio (active-backup arp_ip_target primary_reselect 1) [ OK ]
# TEST: prio (active-backup arp_ip_target primary_reselect 2) [ OK ]
#
not ok 7 selftests: drivers/net/bonding: bond_options.sh # TIMEOUT 120 seconds
This test includes many sleep statements, at least some of which are related to timers in the operation of the bonding driver itself. Increase the test timeout to allow the test to complete.
I ran the test in slightly different VMs (including one without HW virtualization support) and got runtimes of 13m39.760s, 13m31.238s, and 13m2.956s. Use a ~1.5x "safety factor" and set the timeout to 1200s.
--- tools/testing/selftests/drivers/net/bonding/settings | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/drivers/net/bonding/settings +1 -1
@@ -1 +1 @@ -timeout=120 +timeout=1200
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 171/331 tcp: make sure init the accept_queues spinlocks once
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Zhengchao Shao <[email protected]>
[ Upstream commit 198bc90e0e734e5f98c3d2833e8390cac3df61b2 ]
When I run syz's reproduction C program locally, it causes the following issue:
pvqspinlock: lock 0xffff9d181cd5c660 has corrupted value 0x0!
WARNING: CPU: 19 PID: 21160 at __pv_queued_spin_unlock_slowpath (kernel/locking/qspinlock_paravirt.h:508) Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 RIP: 0010:__pv_queued_spin_unlock_slowpath (kernel/locking/qspinlock_paravirt.h:508) Code: 73 56 3a ff 90 c3 cc cc cc cc 8b 05 bb 1f 48 01 85 c0 74 05 c3 cc cc cc cc 8b 17 48 89 fe 48 c7 c7 30 20 ce 8f e8 ad 56 42 ff <0f> 0b c3 cc cc cc cc 0f 0b 0f 1f 40 00 90 90 90 90 90 90 90 90 90 RSP: 0018:ffffa8d200604cb8 EFLAGS: 00010282 RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff9d1ef60e0908 RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff9d1ef60e0900 RBP: ffff9d181cd5c280 R08: 0000000000000000 R09: 00000000ffff7fff R10: ffffa8d200604b68 R11: ffffffff907dcdc8 R12: 0000000000000000 R13: ffff9d181cd5c660 R14: ffff9d1813a3f330 R15: 0000000000001000 FS: 00007fa110184640(0000) GS:ffff9d1ef60c0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000020000000 CR3: 000000011f65e000 CR4: 00000000000006f0 Call Trace: <IRQ> _raw_spin_unlock (kernel/locking/spinlock.c:186) inet_csk_reqsk_queue_add (net/ipv4/inet_connection_sock.c:1321) inet_csk_complete_hashdance (net/ipv4/inet_connection_sock.c:1358) tcp_check_req (net/ipv4/tcp_minisocks.c:868) tcp_v4_rcv (net/ipv4/tcp_ipv4.c:2260) ip_protocol_deliver_rcu (net/ipv4/ip_input.c:205) ip_local_deliver_finish (net/ipv4/ip_input.c:234) __netif_receive_skb_one_core (net/core/dev.c:5529) process_backlog (./include/linux/rcupdate.h:779) __napi_poll (net/core/dev.c:6533) net_rx_action (net/core/dev.c:6604) __do_softirq (./arch/x86/include/asm/jump_label.h:27) do_softirq (kernel/softirq.c:454 kernel/softirq.c:441) </IRQ> <TASK> __local_bh_enable_ip (kernel/softirq.c:381) __dev_queue_xmit (net/core/dev.c:4374) ip_finish_output2 (./include/net/neighbour.h:540 net/ipv4/ip_output.c:235) __ip_queue_xmit (net/ipv4/ip_output.c:535) __tcp_transmit_skb (net/ipv4/tcp_output.c:1462) tcp_rcv_synsent_state_process (net/ipv4/tcp_input.c:6469) tcp_rcv_state_process (net/ipv4/tcp_input.c:6657) tcp_v4_do_rcv (net/ipv4/tcp_ipv4.c:1929) __release_sock (./include/net/sock.h:1121 net/core/sock.c:2968) release_sock (net/core/sock.c:3536) inet_wait_for_connect (net/ipv4/af_inet.c:609) __inet_stream_connect (net/ipv4/af_inet.c:702) inet_stream_connect (net/ipv4/af_inet.c:748) __sys_connect (./include/linux/file.h:45 net/socket.c:2064) __x64_sys_connect (net/socket.c:2073 net/socket.c:2070 net/socket.c:2070) do_syscall_64 (arch/x86/entry/common.c:51 arch/x86/entry/common.c:82) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:129) RIP: 0033:0x7fa10ff05a3d Code: 5b 41 5c c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ab a3 0e 00 f7 d8 64 89 01 48 RSP: 002b:00007fa110183de8 EFLAGS: 00000202 ORIG_RAX: 000000000000002a RAX: ffffffffffffffda RBX: 0000000020000054 RCX: 00007fa10ff05a3d RDX: 000000000000001c RSI: 0000000020000040 RDI: 0000000000000003 RBP: 00007fa110183e20 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000202 R12: 00007fa110184640 R13: 0000000000000000 R14: 00007fa10fe8b060 R15: 00007fff73e23b20 </TASK>
The issue triggering process is analyzed as follows:
Thread A Thread Btcp_v4_rcv //receive ack TCP packet inet_shutdown
tcp_check_req tcp_disconnect //disconnect sock
... tcp_set_state(sk, TCP_CLOSE)
inet_csk_complete_hashdance ...
inet_csk_reqsk_queue_add inet_listen //start listen
spin_lock(&queue->rskq_lock) inet_csk_listen_start
... reqsk_queue_alloc
... spin_lock_init
spin_unlock(&queue->rskq_lock) //warningWhen the socket receives the ACK packet during the three-way handshake, it will hold spinlock. And then the user actively shutdowns the socket and listens to the socket immediately, the spinlock will be initialized. When the socket is going to release the spinlock, a warning is generated. Also the same issue to fastopenq.lock.
Move init spinlock to inet_create and inet_accept to make sure init the accept_queue's spinlocks once.
--- include/net/inet_connection_sock.h | 8 ++++++++ net/core/request_sock.c | 3 --- net/ipv4/af_inet.c | 3 +++ net/ipv4/inet_connection_sock.c | 4 ++++ 4 files changed, 15 insertions(+), 3 deletions(-)
include/net/inet_connection_sock.h +8 -0
@@ -347,4 +347,12 @@ static inline bool inet_csk_has_ulp(const struct sock *sk) return inet_test_bit(IS_ICSK, sk) && !!inet_csk(sk)->icsk_ulp_ops; } +static inline void inet_init_csk_locks (struct sock *sk) +{ +struct inet_connection_sock *icsk =inet_csk (sk); + +spin_lock_init (&icsk->icsk_accept_queue.rskq_lock); +spin_lock_init (&icsk->icsk_accept_queue.fastopenq.lock); +} +#endif /* _INET_CONNECTION_SOCK_H */
net/core/request_sock.c +0 -3
@@ -33,9 +33,6 @@void reqsk_queue_alloc (struct request_sock_queue *queue) { -spin_lock_init (&queue->rskq_lock); - -spin_lock_init (&queue->fastopenq.lock); queue->fastopenq.rskq_rst_head =NULL ; queue->fastopenq.rskq_rst_tail =NULL ; queue->fastopenq.qlen =0 ;
net/ipv4/af_inet.c +3 -0
@@ -330,6 +330,9 @@ static int inet_create(struct net *net, struct socket *sock, int protocol,if (INET_PROTOSW_REUSE & answer_flags) sk->sk_reuse =SK_CAN_REUSE ; +if (INET_PROTOSW_ICSK & answer_flags) +inet_init_csk_locks (sk); + inet =inet_sk (sk);inet_assign_bit (IS_ICSK , sk,INET_PROTOSW_ICSK & answer_flags);
net/ipv4/inet_connection_sock.c +4 -0
@@ -730,6 +730,10 @@ struct sock *inet_csk_accept(struct sock *sk, int flags, int *err, bool kern) }if (req)reqsk_put (req); + +if (newsk) +inet_init_csk_locks (newsk); +return newsk; out_err: newsk =NULL ;
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 172/331 bnxt_en: Wait for FLR to complete during probe
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Michael Chan <[email protected]>
[ Upstream commit 3c1069fa42872f95cf3c6fedf80723d391e12d57 ]
The first message to firmware may fail if the device is undergoing FLR. The driver has some recovery logic for this failure scenario but we must wait 100 msec for FLR to complete before proceeding. Otherwise the recovery will always fail.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +5 -0
@@ -12261,6 +12261,11 @@ static int bnxt_fw_init_one_p1(struct bnxt *bp) bp->fw_cap =0 ; rc =bnxt_hwrm_ver_get (bp); +/* FW may be unresponsive after FLR. FLR must complete within 100 msec +* so wait before continuing with recovery. +*/ +if (rc) +msleep (100 );bnxt_try_map_fw_health_reg (bp);if (rc) { rc =bnxt_try_recover_fw (bp);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 173/331 bnxt_en: Prevent kernel warning when running offline self test
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Michael Chan <[email protected]>
[ Upstream commit c20f482129a582455f02eb9a6dcb2a4215274599 ]
We call bnxt_half_open_nic() to setup the chip partially to run loopback tests. The rings and buffers are initialized normally so that we can transmit and receive packets in loopback mode. That means page pool buffers are allocated for the aggregation ring just like the normal case. NAPI is not needed because we are just polling for the loopback packets.
When we're done with the loopback tests, we call bnxt_half_close_nic() to clean up. When freeing the page pools, we hit a WARN_ON() in page_pool_unlink_napi() because the NAPI state linked to the page pool is uninitialized.
The simplest way to avoid this warning is just to initialize the NAPIs during half open and delete the NAPIs during half close. Trying to skip the page pool initialization or skip linking of NAPI during half open will be more complicated.
This fix avoids this warning:
WARNING: CPU: 4 PID: 46967 at net/core/page_pool.c:946 page_pool_unlink_napi+0x1f/0x30 CPU: 4 PID: 46967 Comm: ethtool Tainted: G S W 6.7.0-rc5+ #22 Hardware name: Dell Inc. PowerEdge R750/06V45N, BIOS 1.3.8 08/31/2021 RIP: 0010:page_pool_unlink_napi+0x1f/0x30 Code: 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 48 8b 47 18 48 85 c0 74 1b 48 8b 50 10 83 e2 01 74 08 8b 40 34 83 f8 ff 74 02 <0f> 0b 48 c7 47 18 00 00 00 00 c3 cc cc cc cc 66 90 90 90 90 90 90 RSP: 0018:ffa000003d0dfbe8 EFLAGS: 00010246 RAX: ff110003607ce640 RBX: ff110010baf5d000 RCX: 0000000000000008 RDX: 0000000000000000 RSI: ff110001e5e522c0 RDI: ff110010baf5d000 RBP: ff11000145539b40 R08: 0000000000000001 R09: ffffffffc063f641 R10: ff110001361eddb8 R11: 000000000040000f R12: 0000000000000001 R13: 000000000000001c R14: ff1100014553a080 R15: 0000000000003fc0 FS: 00007f9301c4f740(0000) GS:ff1100103fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f91344fa8f0 CR3: 00000003527cc005 CR4: 0000000000771ef0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 PKRU: 55555554 Call Trace: <TASK> ? __warn+0x81/0x140 ? page_pool_unlink_napi+0x1f/0x30 ? report_bug+0x102/0x200 ? handle_bug+0x44/0x70 ? exc_invalid_op+0x13/0x60 ? asm_exc_invalid_op+0x16/0x20 ? bnxt_free_ring.isra.123+0xb1/0xd0 [bnxt_en] ? page_pool_unlink_napi+0x1f/0x30 page_pool_destroy+0x3e/0x150 bnxt_free_mem+0x441/0x5e0 [bnxt_en] bnxt_half_close_nic+0x2a/0x40 [bnxt_en] bnxt_self_test+0x21d/0x450 [bnxt_en] __dev_ethtool+0xeda/0x2e30 ? native_queued_spin_lock_slowpath+0x17f/0x2b0 ? __link_object+0xa1/0x160 ? _raw_spin_unlock_irqrestore+0x23/0x40 ? __create_object+0x5f/0x90 ? __kmem_cache_alloc_node+0x317/0x3c0 ? dev_ethtool+0x59/0x170 dev_ethtool+0xa7/0x170 dev_ioctl+0xc3/0x530 sock_do_ioctl+0xa8/0xf0 sock_ioctl+0x270/0x310 __x64_sys_ioctl+0x8c/0xc0 do_syscall_64+0x3e/0xf0 entry_SYSCALL_64_after_hwframe+0x6e/0x76
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +3 -0
@@ -10598,10 +10598,12 @@ int bnxt_half_open_nic(struct bnxt *bp)netdev_err (bp->dev,"bnxt_alloc_mem err: %x\n" , rc); goto half_open_err; } +bnxt_init_napi (bp);set_bit (BNXT_STATE_HALF_OPEN , &bp->state); rc =bnxt_init_nic (bp, true);if (rc) {clear_bit (BNXT_STATE_HALF_OPEN , &bp->state); +bnxt_del_napi (bp);netdev_err (bp->dev,"bnxt_init_nic err: %x\n" , rc); goto half_open_err; }
@@ -10620,6 +10622,7 @@ int bnxt_half_open_nic(struct bnxt *bp)void bnxt_half_close_nic (struct bnxt *bp) {bnxt_hwrm_resource_free (bp, false, true); +bnxt_del_napi (bp);bnxt_free_skbs (bp);bnxt_free_mem (bp, true);clear_bit (BNXT_STATE_HALF_OPEN , &bp->state);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 174/331 vlan: skip nested type that is not IFLA_VLAN_QOS_MAPPING
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Lin Ma <[email protected]>
[ Upstream commit 6c21660fe221a15c789dee2bc2fd95516bc5aeaf ]
In the vlan_changelink function, a loop is used to parse the nested attributes IFLA_VLAN_EGRESS_QOS and IFLA_VLAN_INGRESS_QOS in order to obtain the struct ifla_vlan_qos_mapping. These two nested attributes are checked in the vlan_validate_qos_map function, which calls nla_validate_nested_deprecated with the vlan_map_policy.
However, this deprecated validator applies a LIBERAL strictness, allowing the presence of an attribute with the type IFLA_VLAN_QOS_UNSPEC. Consequently, the loop in vlan_changelink may parse an attribute of type IFLA_VLAN_QOS_UNSPEC and believe it carries a payload of struct ifla_vlan_qos_mapping, which is not necessarily true.
To address this issue and ensure compatibility, this patch introduces two type checks that skip attributes whose type is not IFLA_VLAN_QOS_MAPPING.
--- net/8021q/vlan_netlink.c | 4 ++++ 1 file changed, 4 insertions(+)
net/8021q/vlan_netlink.c +4 -0
@@ -118,12 +118,16 @@ static int vlan_changelink(struct net_device *dev, struct nlattr *tb[], }if (data[IFLA_VLAN_INGRESS_QOS ]) {nla_for_each_nested (attr, data[IFLA_VLAN_INGRESS_QOS ], rem) { +if (nla_type (attr) !=IFLA_VLAN_QOS_MAPPING ) +continue ; m =nla_data (attr);vlan_dev_set_ingress_priority (dev, m->to, m->from); } }if (data[IFLA_VLAN_EGRESS_QOS ]) {nla_for_each_nested (attr, data[IFLA_VLAN_EGRESS_QOS ], rem) { +if (nla_type (attr) !=IFLA_VLAN_QOS_MAPPING ) +continue ; m =nla_data (attr); err =vlan_dev_set_egress_priority (dev, m->from, m->to);if (err)
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 175/331 llc: make llc_ui_sendmsg() more robust against bonding changes
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eric Dumazet <[email protected]>
[ Upstream commit dad555c816a50c6a6a8a86be1f9177673918c647 ]
syzbot was able to trick llc_ui_sendmsg(), allocating an skb with no headroom, but subsequently trying to push 14 bytes of Ethernet header [1]
Like some others, llc_ui_sendmsg() releases the socket lock before calling sock_alloc_send_skb().
Then it acquires it again, but does not redo all the sanity checks that were performed.
This fix:
- Uses LL_RESERVED_SPACE() to reserve space.
- Check all conditions again after socket lock is held again.
- Do not account Ethernet header for mtu limitation.
[1]
skbuff: skb_under_panic: text:ffff800088baa334 len:1514 put:14 head:ffff0000c9c37000 data:ffff0000c9c36ff2 tail:0x5dc end:0x6c0 dev:bond0
kernel BUG at net/core/skbuff.c:193 !
Internal error: Oops - BUG: 00000000f2000800 [#1] PREEMPT SMP Modules linked in: CPU: 0 PID: 6875 Comm: syz-executor.0 Not tainted 6.7.0-rc8-syzkaller-00101-g0802e17d9aca-dirty #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023 pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : skb_panic net/core/skbuff.c:189 [inline] pc : skb_under_panic+0x13c/0x140 net/core/skbuff.c:203 lr : skb_panic net/core/skbuff.c:189 [inline] lr : skb_under_panic+0x13c/0x140 net/core/skbuff.c:203
sp : ffff800096f97000
x29: ffff800096f97010 x28: ffff80008cc8d668 x27: dfff800000000000
x26: ffff0000cb970c90 x25: 00000000000005dc x24: ffff0000c9c36ff2
x23: ffff0000c9c37000 x22: 00000000000005ea x21: 00000000000006c0
x20: 000000000000000e x19: ffff800088baa334 x18: 1fffe000368261ce
x17: ffff80008e4ed000 x16: ffff80008a8310f8 x15: 0000000000000001
x14: 1ffff00012df2d58 x13: 0000000000000000 x12: 0000000000000000
x11: 0000000000000001 x10: 0000000000ff0100 x9 : e28a51f1087e8400 x8 : e28a51f1087e8400 x7 : ffff80008028f8d0 x6 : 0000000000000000 x5 : 0000000000000001 x4 : 0000000000000001 x3 : ffff800082b78714 x2 : 0000000000000001 x1 : 0000000100000000 x0 : 0000000000000089
Call trace: skb_panic net/core/skbuff.c:189 [inline] skb_under_panic+0x13c/0x140 net/core/skbuff.c:203 skb_push+0xf0/0x108 net/core/skbuff.c:2451 eth_header+0x44/0x1f8 net/ethernet/eth.c:83 dev_hard_header include/linux/netdevice.h:3188 [inline] llc_mac_hdr_init+0x110/0x17c net/llc/llc_output.c:33 llc_sap_action_send_xid_c+0x170/0x344 net/llc/llc_s_ac.c:85 llc_exec_sap_trans_actions net/llc/llc_sap.c:153 [inline] llc_sap_next_state net/llc/llc_sap.c:182 [inline] llc_sap_state_process+0x1ec/0x774 net/llc/llc_sap.c:209 llc_build_and_send_xid_pkt+0x12c/0x1c0 net/llc/llc_sap.c:270 llc_ui_sendmsg+0x7bc/0xb1c net/llc/af_llc.c:997 sock_sendmsg_nosec net/socket.c:730 [inline] __sock_sendmsg net/socket.c:745 [inline] sock_sendmsg+0x194/0x274 net/socket.c:767 splice_to_socket+0x7cc/0xd58 fs/splice.c:881 do_splice_from fs/splice.c:933 [inline] direct_splice_actor+0xe4/0x1c0 fs/splice.c:1142 splice_direct_to_actor+0x2a0/0x7e4 fs/splice.c:1088 do_splice_direct+0x20c/0x348 fs/splice.c:1194 do_sendfile+0x4bc/0xc70 fs/read_write.c:1254 __do_sys_sendfile64 fs/read_write.c:1322 [inline] __se_sys_sendfile64 fs/read_write.c:1308 [inline] __arm64_sys_sendfile64+0x160/0x3b4 fs/read_write.c:1308 __invoke_syscall arch/arm64/kernel/syscall.c:37 [inline] invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:51 el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:136 do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:155 el0_svc+0x54/0x158 arch/arm64/kernel/entry-common.c:678 el0t_64_sync_handler+0x84/0xfc arch/arm64/kernel/entry-common.c:696 el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:595 Code: aa1803e6 aa1903e7 a90023f5 94792f6a (d4210000)
--- net/llc/af_llc.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
net/llc/af_llc.c +16 -8
@@ -928,14 +928,15 @@ static int llc_ui_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,*/ static int llc_ui_sendmsg (struct socket *sock,struct msghdr *msg,size_t len) { + DECLARE_SOCKADDR(struct sockaddr_llc *, addr, msg->msg_name);struct sock *sk = sock->sk;struct llc_sock *llc =llc_sk (sk); - DECLARE_SOCKADDR(struct sockaddr_llc *, addr, msg->msg_name);int flags = msg->msg_flags;int noblock = flags &MSG_DONTWAIT ; +int rc = -EINVAL , copied =0 , hdrlen, hh_len;struct sk_buff *skb =NULL ; +struct net_device *dev;size_t size =0 ; -int rc = -EINVAL , copied =0 , hdrlen;dprintk ("%s: sending from %02X to %02X\n" , __func__, llc->laddr.lsap, llc->daddr.lsap);
@@ -955,22 +956,29 @@ static int llc_ui_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)if (rc) goto out; } - hdrlen = llc->dev->hard_header_len +llc_ui_header_len (sk, addr); + dev = llc->dev; + hh_len =LL_RESERVED_SPACE (dev); + hdrlen =llc_ui_header_len (sk, addr); size = hdrlen + len; -if (size > llc->dev->mtu) - size = llc->dev->mtu; + size =min_t (size_t, size,READ_ONCE (dev->mtu)); copied = size - hdrlen; rc = -EINVAL ;if (copied <0 ) goto out;release_sock (sk); - skb =sock_alloc_send_skb (sk, size, noblock, &rc); + skb =sock_alloc_send_skb (sk, hh_len + size, noblock, &rc);lock_sock (sk);if (!skb) goto out; - skb->dev = llc->dev; +if (sock_flag (sk,SOCK_ZAPPED ) || + llc->dev != dev || + hdrlen !=llc_ui_header_len (sk, addr) || + hh_len !=LL_RESERVED_SPACE (dev) || + size >READ_ONCE (dev->mtu)) + goto out; + skb->dev = dev; skb->protocol =llc_proto_type (addr->sllc_arphrd); -skb_reserve (skb, hdrlen); +skb_reserve (skb, hh_len + hdrlen); rc =memcpy_from_msg (skb_put (skb, copied), msg, copied);if (rc) goto out;
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 176/331 llc: Drop support for ETH_P_TR_802_2.
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Kuniyuki Iwashima <[email protected]>
[ Upstream commit e3f9bed9bee261e3347131764e42aeedf1ffea61 ]
syzbot reported an uninit-value bug below. [0]
llc supports ETH_P_802_2 (0x0004) and used to support ETH_P_TR_802_2 (0x0011), and syzbot abused the latter to trigger the bug.
write$tun(r0, &(0x7f0000000040)={@val={0x0, 0x11}, @val, @mpls={[], @llc={@snap={0xaa, 0x1, ')', "90e5dd"}}}}, 0x16)
llc_conn_handler() initialises local variables {saddr,daddr}.mac based on skb in llc_pdu_decode_sa()/llc_pdu_decode_da() and passes them to __llc_lookup().
However, the initialisation is done only when skb->protocol is htons(ETH_P_802_2), otherwise, __llc_lookup_established() and __llc_lookup_listener() will read garbage.
The missing initialisation existed prior to commit 211ed865108e ("net: delete all instances of special processing for token ring").
It removed the part to kick out the token ring stuff but forgot to close the door allowing ETH_P_TR_802_2 packets to sneak into llc_rcv().
Let's remove llc_tr_packet_type and complete the deprecation.
[0]:
BUG: KMSAN: uninit-value in __llc_lookup_established+0xe9d/0xf90 __llc_lookup_established+0xe9d/0xf90 __llc_lookup net/llc/llc_conn.c:611 [inline] llc_conn_handler+0x4bd/0x1360 net/llc/llc_conn.c:791 llc_rcv+0xfbb/0x14a0 net/llc/llc_input.c:206 __netif_receive_skb_one_core net/core/dev.c:5527 [inline] __netif_receive_skb+0x1a6/0x5a0 net/core/dev.c:5641 netif_receive_skb_internal net/core/dev.c:5727 [inline] netif_receive_skb+0x58/0x660 net/core/dev.c:5786 tun_rx_batched+0x3ee/0x980 drivers/net/tun.c:1555 tun_get_user+0x53af/0x66d0 drivers/net/tun.c:2002 tun_chr_write_iter+0x3af/0x5d0 drivers/net/tun.c:2048 call_write_iter include/linux/fs.h:2020 [inline] new_sync_write fs/read_write.c:491 [inline] vfs_write+0x8ef/0x1490 fs/read_write.c:584 ksys_write+0x20f/0x4c0 fs/read_write.c:637 __do_sys_write fs/read_write.c:649 [inline] __se_sys_write fs/read_write.c:646 [inline] __x64_sys_write+0x93/0xd0 fs/read_write.c:646 do_syscall_x64 arch/x86/entry/common.c:51 [inline] do_syscall_64+0x44/0x110 arch/x86/entry/common.c:82 entry_SYSCALL_64_after_hwframe+0x63/0x6b
Local variable daddr created at:
llc_conn_handler+0x53/0x1360 net/llc/llc_conn.c:783 llc_rcv+0xfbb/0x14a0 net/llc/llc_input.c:206 CPU: 1 PID: 5004 Comm: syz-executor994 Not tainted 6.6.0-syzkaller-14500-g1c41041124bd #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/09/2023
--- include/net/llc_pdu.h | 6 ++---- net/llc/llc_core.c | 7 ------- 2 files changed, 2 insertions(+), 11 deletions(-)
include/net/llc_pdu.h +2 -4
@@ -262,8 +262,7 @@ static inline void llc_pdu_header_init(struct sk_buff *skb, u8 type,*/ static inline void llc_pdu_decode_sa (struct sk_buff *skb,u8 *sa) { -if (skb->protocol ==htons (ETH_P_802_2 )) -memcpy (sa,eth_hdr (skb)->h_source,ETH_ALEN ); +memcpy (sa,eth_hdr (skb)->h_source,ETH_ALEN ); }/**
@@ -275,8 +274,7 @@ static inline void llc_pdu_decode_sa(struct sk_buff *skb, u8 *sa)*/ static inline void llc_pdu_decode_da (struct sk_buff *skb,u8 *da) { -if (skb->protocol ==htons (ETH_P_802_2 )) -memcpy (da,eth_hdr (skb)->h_dest,ETH_ALEN ); +memcpy (da,eth_hdr (skb)->h_dest,ETH_ALEN ); }/**
net/llc/llc_core.c +0 -7
@@ -135,22 +135,15 @@ static struct packet_type llc_packet_type __read_mostly = { .func = llc_rcv, }; -static struct packet_type llc_tr_packet_type__read_mostly = { - .type =cpu_to_be16 (ETH_P_TR_802_2 ), - .func = llc_rcv, -}; -static int __init llc_init (void ) {dev_add_pack (&llc_packet_type); -dev_add_pack (&llc_tr_packet_type);return 0 ; } static void __exit llc_exit(void) {dev_remove_pack (&llc_packet_type); -dev_remove_pack (&llc_tr_packet_type); }module_init (llc_init);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 177/331 udp: fix busy polling
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Eric Dumazet <[email protected]>
[ Upstream commit a54d51fb2dfb846aedf3751af501e9688db447f5 ]
Generic sk_busy_loop_end() only looks at sk->sk_receive_queue for presence of packets.
Problem is that for UDP sockets after blamed commit, some packets could be present in another queue: udp_sk(sk)->reader_queue
In some cases, a busy poller could spin until timeout expiration, even if some packets are available in udp_sk(sk)->reader_queue.
v3: - make sk_busy_loop_end() nicer (Willem)
v2: - add a READ_ONCE(sk->sk_family) in sk_is_inet() to avoid KCSAN splats.
- add a sk_is_inet() check in sk_is_udp() (Willem feedback)
- add a sk_is_inet() check in sk_is_tcp().
--- include/linux/skmsg.h | 6 ------ include/net/inet_sock.h | 5 ----- include/net/sock.h | 18 +++++++++++++++++- net/core/sock.c | 11 +++++++++-- 4 files changed, 26 insertions(+), 14 deletions(-)
include/linux/skmsg.h +0 -6
@@ -500,12 +500,6 @@ static inline bool sk_psock_strp_enabled(struct sk_psock *psock)return !!psock->saved_data_ready; } -static inline bool sk_is_udp (const struct sock *sk) -{ -return sk->sk_type ==SOCK_DGRAM && - sk->sk_protocol ==IPPROTO_UDP ; -} -#if IS_ENABLED (CONFIG_NET_SOCK_MSG )#define BPF_F_STRPARSER (1UL << 1)
include/net/inet_sock.h +0 -5
@@ -301,11 +301,6 @@ static inline unsigned long inet_cmsg_flags(const struct inet_sock *inet)#define inet_assign_bit (nr, sk, val) \ assign_bit(INET_FLAGS_##nr, &inet_sk(sk)->inet_flags, val) -static inline bool sk_is_inet (struct sock *sk) -{ -return sk->sk_family ==AF_INET || sk->sk_family ==AF_INET6 ; -} -/** * sk_to_full_sk - Access to a full socket * @sk: pointer to a socket
include/net/sock.h +17 -1
@@ -2793,9 +2793,25 @@ static inline void skb_setup_tx_timestamp(struct sk_buff *skb, __u16 tsflags) &skb_shinfo(skb)->tskey); } +static inline bool sk_is_inet (const struct sock *sk) +{ +int family =READ_ONCE (sk->sk_family); + +return family ==AF_INET || family ==AF_INET6 ; +} +static inline bool sk_is_tcp (const struct sock *sk) { -return sk->sk_type ==SOCK_STREAM && sk->sk_protocol ==IPPROTO_TCP ; +return sk_is_inet (sk) && + sk->sk_type ==SOCK_STREAM && + sk->sk_protocol ==IPPROTO_TCP ; +} + +static inline bool sk_is_udp (const struct sock *sk) +{ +return sk_is_inet (sk) && + sk->sk_type ==SOCK_DGRAM && + sk->sk_protocol ==IPPROTO_UDP ; }static inline bool sk_is_stream_unix (const struct sock *sk)
net/core/sock.c +9 -2
@@ -107,6 +107,7 @@#include <linux/interrupt.h> #include <linux/poll.h> #include <linux/tcp.h> +#include <linux/udp.h> #include <linux/init.h> #include <linux/highmem.h> #include <linux/user_namespace.h>
@@ -4136,8 +4137,14 @@ bool sk_busy_loop_end(void *p, unsigned long start_time) {struct sock *sk = p; -return !skb_queue_empty_lockless (&sk->sk_receive_queue) || -sk_busy_loop_timeout (sk, start_time); +if (!skb_queue_empty_lockless (&sk->sk_receive_queue)) +return true; + +if (sk_is_udp (sk) && + !skb_queue_empty_lockless (&udp_sk (sk)->reader_queue)) +return true; + +return sk_busy_loop_timeout (sk, start_time); }EXPORT_SYMBOL (sk_busy_loop_end);#endif /* CONFIG_NET_RX_BUSY_POLL */
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 178/331 net: fix removing a namespace with conflicting altnames
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Jakub Kicinski <[email protected]>
[ Upstream commit d09486a04f5da0a812c26217213b89a3b1acf836 ]
Mark reports a BUG() when a net namespace is removed.
kernel BUG at net/core/dev.c:11520!
Physical interfaces moved outside of init_net get "refunded" to init_net when that namespace disappears. The main interface name may get overwritten in the process if it would have conflicted. We need to also discard all conflicting altnames. Recent fixes addressed ensuring that altnames get moved with the main interface, which surfaced this problem.
--- net/core/dev.c | 9 +++++++++ net/core/dev.h | 3 +++ 2 files changed, 12 insertions(+)
net/core/dev.c +9 -0
@@ -11491,6 +11491,7 @@ static struct pernet_operations __net_initdata netdev_net_ops = { static void __net_exit default_device_exit_net(struct net *net) { +struct netdev_name_node *name_node, *tmp;struct net_device *dev, *aux;/* * Push all migratable network devices back to the
@@ -11513,6 +11514,14 @@ static void __net_exit default_device_exit_net(struct net *net)snprintf (fb_name,IFNAMSIZ ,"dev%d" , dev->ifindex);if (netdev_name_in_use (&init_net, fb_name))snprintf (fb_name,IFNAMSIZ ,"dev%%d" ); + +netdev_for_each_altname_safe (dev, name_node, tmp) +if (netdev_name_in_use (&init_net, name_node->name)) { +netdev_name_node_del (name_node); +synchronize_rcu (); +__netdev_name_node_alt_destroy (name_node); + } + err =dev_change_net_namespace (dev, &init_net, fb_name);if (err) { pr_emerg("%s: failed to move %s to init_net: %d\n",
net/core/dev.h +3 -0
@@ -64,6 +64,9 @@ int dev_change_name(struct net_device *dev, const char *newname);#define netdev_for_each_altname (dev, namenode) \ list_for_each_entry((namenode), &(dev)->name_node->list, list) +#define netdev_for_each_altname_safe (dev, namenode, next) \ + list_for_each_entry_safe((namenode), (next), &(dev)->name_node->list, \ + list)int netdev_name_node_alt_create (struct net_device *dev,const char *name);int netdev_name_node_alt_destroy (struct net_device *dev,const char *name);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 179/331 tun: fix missing dropped counter in tun_xdp_act
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Yunjian Wang <[email protected]>
[ Upstream commit 5744ba05e7c4bff8fec133dd0f9e51ddffba92f5 ]
The commit 8ae1aff0b331 ("tuntap: split out XDP logic") includes dropped counter for XDP_DROP, XDP_ABORTED, and invalid XDP actions. Unfortunately, that commit missed the dropped counter when error occurs during XDP_TX and XDP_REDIRECT actions. This patch fixes this issue.
--- drivers/net/tun.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/net/tun.c +6 -2
@@ -1630,13 +1630,17 @@ static int tun_xdp_act(struct tun_struct *tun, struct bpf_prog *xdp_prog,switch (act) {case XDP_REDIRECT : err =xdp_do_redirect (tun->dev, xdp, xdp_prog); -if (err) +if (err) { +dev_core_stats_rx_dropped_inc (tun->dev);return err; + }break ;case XDP_TX : err =tun_xdp_tx (tun->dev, xdp); -if (err <0 ) +if (err <0 ) { +dev_core_stats_rx_dropped_inc (tun->dev);return err; + }break ;case XDP_PASS :break ;
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 180/331 tun: add missing rx stats accounting in tun_xdp_act
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Yunjian Wang <[email protected]>
[ Upstream commit f1084c427f55d573fcd5688d9ba7b31b78019716 ]
The TUN can be used as vhost-net backend, and it is necessary to count the packets transmitted from TUN to vhost-net/virtio-net. However, there are some places in the receive path that were not taken into account when using XDP. It would be beneficial to also include new accounting for successfully received bytes using dev_sw_netstats_rx_add.
--- drivers/net/tun.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/tun.c +2 -0
@@ -1634,6 +1634,7 @@ static int tun_xdp_act(struct tun_struct *tun, struct bpf_prog *xdp_prog,dev_core_stats_rx_dropped_inc (tun->dev);return err; } +dev_sw_netstats_rx_add (tun->dev, xdp->data_end - xdp->data);break ;case XDP_TX : err =tun_xdp_tx (tun->dev, xdp);
@@ -1641,6 +1642,7 @@ static int tun_xdp_act(struct tun_struct *tun, struct bpf_prog *xdp_prog,dev_core_stats_rx_dropped_inc (tun->dev);return err; } +dev_sw_netstats_rx_add (tun->dev, xdp->data_end - xdp->data);break ;case XDP_PASS :break ;
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 181/331 net: micrel: Fix PTP frame parsing for lan8814
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Horatiu Vultur <[email protected]>
[ Upstream commit aaf632f7ab6dec57bc9329a438f94504fe8034b9 ]
The HW has the capability to check each frame if it is a PTP frame, which domain it is, which ptp frame type it is, different ip address in the frame. And if one of these checks fail then the frame is not timestamp. Most of these checks were disabled except checking the field minorVersionPTP inside the PTP header. Meaning that once a partner sends a frame compliant to 8021AS which has minorVersionPTP set to 1, then the frame was not timestamp because the HW expected by default a value of 0 in minorVersionPTP. This is exactly the same issue as on lan8841. Fix this issue by removing this check so the userspace can decide on this.
--- drivers/net/phy/micrel.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
drivers/net/phy/micrel.c +11 -0
@@ -120,6 +120,11 @@*/ #define LAN8814_1PPM_FORMAT 17179 +#define PTP_RX_VERSION 0x0248 +#define PTP_TX_VERSION 0x0288 +#define PTP_MAX_VERSION (x) (((x) & GENMASK(7, 0)) << 8) +#define PTP_MIN_VERSION (x) ((x) & GENMASK(7, 0)) +#define PTP_RX_MOD 0x024F#define PTP_RX_MOD_BAD_UDPV4_CHKSUM_FORCE_FCS_DIS_ BIT(3)#define PTP_RX_TIMESTAMP_EN 0x024D
@@ -3125,6 +3130,12 @@ static void lan8814_ptp_init(struct phy_device *phydev)lanphy_write_page_reg (phydev,5 ,PTP_TX_PARSE_IP_ADDR_EN ,0 );lanphy_write_page_reg (phydev,5 ,PTP_RX_PARSE_IP_ADDR_EN ,0 ); +/* Disable checking for minorVersionPTP field */ +lanphy_write_page_reg (phydev,5 ,PTP_RX_VERSION , +PTP_MAX_VERSION (0xff ) |PTP_MIN_VERSION (0x0 )); +lanphy_write_page_reg (phydev,5 ,PTP_TX_VERSION , +PTP_MAX_VERSION (0xff ) |PTP_MIN_VERSION (0x0 )); +skb_queue_head_init (&ptp_priv->tx_queue);skb_queue_head_init (&ptp_priv->rx_queue);INIT_LIST_HEAD (&ptp_priv->rx_ts_list);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 182/331 net/rds: Fix UBSAN: array-index-out-of-bounds in rds_cmsg_recv
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Sharath Srinivasan <[email protected]>
[ Upstream commit 13e788deb7348cc88df34bed736c3b3b9927ea52 ]
Syzcaller UBSAN crash occurs in rds_cmsg_recv(),
which reads inc->i_rx_lat_trace[j + 1] with index 4 (3 + 1), but with array size of 4 (RDS_RX_MAX_TRACES).
Here 'j' is assigned from rs->rs_rx_trace[i] and in-turn from trace.rx_trace_pos[i] in rds_recv_track_latency(),
with both arrays sized 3 (RDS_MSG_RX_DGRAM_TRACE_MAX). So fix the off-by-one bounds check in rds_recv_track_latency() to prevent a potential crash in rds_cmsg_recv().
Found by syzcaller:
=================================================================
UBSAN: array-index-out-of-bounds in net/rds/recv.c:585:39 index 4 is out of range for type 'u64 [4]' CPU: 1 PID: 8058 Comm: syz-executor228 Not tainted 6.6.0-gd2f51b3516da #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014 Call Trace: <TASK> __dump_stack lib/dump_stack.c:88 [inline] dump_stack_lvl+0x136/0x150 lib/dump_stack.c:106 ubsan_epilogue lib/ubsan.c:217 [inline] __ubsan_handle_out_of_bounds+0xd5/0x130 lib/ubsan.c:348 rds_cmsg_recv+0x60d/0x700 net/rds/recv.c:585 rds_recvmsg+0x3fb/0x1610 net/rds/recv.c:716 sock_recvmsg_nosec net/socket.c:1044 [inline] sock_recvmsg+0xe2/0x160 net/socket.c:1066 __sys_recvfrom+0x1b6/0x2f0 net/socket.c:2246 __do_sys_recvfrom net/socket.c:2264 [inline] __se_sys_recvfrom net/socket.c:2260 [inline] __x64_sys_recvfrom+0xe0/0x1b0 net/socket.c:2260 do_syscall_x64 arch/x86/entry/common.c:51 [inline] do_syscall_64+0x40/0x110 arch/x86/entry/common.c:82 entry_SYSCALL_64_after_hwframe+0x63/0x6b ==================================================================
--- net/rds/af_rds.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/rds/af_rds.c +1 -1
@@ -419,7 +419,7 @@ static int rds_recv_track_latency(struct rds_sock *rs, sockptr_t optval, rs->rs_rx_traces = trace.rx_traces;for (i =0 ; i < rs->rs_rx_traces; i++) { -if (trace.rx_trace_pos[i] >RDS_MSG_RX_DGRAM_TRACE_MAX ) { +if (trace.rx_trace_pos[i] >=RDS_MSG_RX_DGRAM_TRACE_MAX ) { rs->rs_rx_traces =0 ;return -EFAULT ; }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 183/331 netfs, fscache: Prevent Oops in fscache_put_cache()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Dan Carpenter <[email protected]>
[ Upstream commit 3be0b3ed1d76c6703b9ee482b55f7e01c369cc68 ]
This function dereferences "cache" and then checks if it's IS_ERR_OR_NULL(). Check first, then dereference.
--- fs/fscache/cache.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/fscache/cache.c +2 -1
@@ -179,13 +179,14 @@ EXPORT_SYMBOL(fscache_acquire_cache);void fscache_put_cache (struct fscache_cache *cache,enum fscache_cache_trace where) { -unsigned int debug_id = cache->debug_id; +unsigned int debug_id;bool zero;int ref;if (IS_ERR_OR_NULL (cache))return ; + debug_id = cache->debug_id; zero =__refcount_dec_and_test (&cache->ref, &ref);trace_fscache_cache (debug_id, ref -1 , where);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 184/331 tracing: Ensure visibility when inserting an element into tracing_map
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Petr Pavlu <[email protected]>
[ Upstream commit 2b44760609e9eaafc9d234a6883d042fc21132a7 ]
Running the following two commands in parallel on a multi-processor AArch64 machine can sporadically produce an unexpected warning about duplicate histogram entries:
$ while true; do
echo hist:key=id.syscall:val=hitcount > \
/sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/trigger
cat /sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/hist
sleep 0.001
done
$ stress-ng --sysbadaddr $(nproc)The warning looks as follows:
[ 2911.172474] ------------[ cut here ]------------ [ 2911.173111] Duplicates detected: 1 [ 2911.173574] WARNING: CPU: 2 PID: 12247 at kernel/trace/tracing_map.c:983 tracing_map_sort_entries+0x3e0/0x408 [ 2911.174702] Modules linked in: iscsi_ibft(E) iscsi_boot_sysfs(E) rfkill(E) af_packet(E) nls_iso8859_1(E) nls_cp437(E) vfat(E) fat(E) ena(E) tiny_power_button(E) qemu_fw_cfg(E) button(E) fuse(E) efi_pstore(E) ip_tables(E) x_tables(E) xfs(E) libcrc32c(E) aes_ce_blk(E) aes_ce_cipher(E) crct10dif_ce(E) polyval_ce(E) polyval_generic(E) ghash_ce(E) gf128mul(E) sm4_ce_gcm(E) sm4_ce_ccm(E) sm4_ce(E) sm4_ce_cipher(E) sm4(E) sm3_ce(E) sm3(E) sha3_ce(E) sha512_ce(E) sha512_arm64(E) sha2_ce(E) sha256_arm64(E) nvme(E) sha1_ce(E) nvme_core(E) nvme_auth(E) t10_pi(E) sg(E) scsi_mod(E) scsi_common(E) efivarfs(E) [ 2911.174738] Unloaded tainted modules: cppc_cpufreq(E):1 [ 2911.180985] CPU: 2 PID: 12247 Comm: cat Kdump: loaded Tainted: G E 6.7.0-default #2 1b58bbb22c97e4399dc09f92d309344f69c44a01 [ 2911.182398] Hardware name: Amazon EC2 c7g.8xlarge/, BIOS 1.0 11/1/2018 [ 2911.183208] pstate: 61400005 (nZCv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--) [ 2911.184038] pc : tracing_map_sort_entries+0x3e0/0x408 [ 2911.184667] lr : tracing_map_sort_entries+0x3e0/0x408 [ 2911.185310] sp : ffff8000a1513900 [ 2911.185750] x29: ffff8000a1513900 x28: ffff0003f272fe80 x27: 0000000000000001 [ 2911.186600] x26: ffff0003f272fe80 x25: 0000000000000030 x24: 0000000000000008 [ 2911.187458] x23: ffff0003c5788000 x22: ffff0003c16710c8 x21: ffff80008017f180 [ 2911.188310] x20: ffff80008017f000 x19: ffff80008017f180 x18: ffffffffffffffff [ 2911.189160] x17: 0000000000000000 x16: 0000000000000000 x15: ffff8000a15134b8 [ 2911.190015] x14: 0000000000000000 x13: 205d373432323154 x12: 5b5d313131333731 [ 2911.190844] x11: 00000000fffeffff x10: 00000000fffeffff x9 : ffffd1b78274a13c [ 2911.191716] x8 : 000000000017ffe8 x7 : c0000000fffeffff x6 : 000000000057ffa8 [ 2911.192554] x5 : ffff0012f6c24ec0 x4 : 0000000000000000 x3 : ffff2e5b72b5d000 [ 2911.193404] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff0003ff254480 [ 2911.194259] Call trace: [ 2911.194626] tracing_map_sort_entries+0x3e0/0x408 [ 2911.195220] hist_show+0x124/0x800 [ 2911.195692] seq_read_iter+0x1d4/0x4e8 [ 2911.196193] seq_read+0xe8/0x138 [ 2911.196638] vfs_read+0xc8/0x300 [ 2911.197078] ksys_read+0x70/0x108 [ 2911.197534] __arm64_sys_read+0x24/0x38 [ 2911.198046] invoke_syscall+0x78/0x108 [ 2911.198553] el0_svc_common.constprop.0+0xd0/0xf8 [ 2911.199157] do_el0_svc+0x28/0x40 [ 2911.199613] el0_svc+0x40/0x178 [ 2911.200048] el0t_64_sync_handler+0x13c/0x158 [ 2911.200621] el0t_64_sync+0x1a8/0x1b0 [ 2911.201115] ---[ end trace 0000000000000000 ]---
The problem appears to be caused by CPU reordering of writes issued from __tracing_map_insert().
The check for the presence of an element with a given key in this function is:
val = READ_ONCE(entry->val);
if (val && keys_match(key, val->key, map->key_size)) ...
The write of a new entry is:
elt = get_free_elt(map);
memcpy(elt->key, key, map->key_size);
entry->val = elt;
The "memcpy(elt->key, key, map->key_size);" and "entry->val = elt;" stores may become visible in the reversed order on another CPU. This second CPU might then incorrectly determine that a new key doesn't match an already present val->key and subsequently insert a new element, resulting in a duplicate.
Fix the problem by adding a write barrier between "memcpy(elt->key, key, map->key_size);" and "entry->val = elt;", and for good measure, also use WRITE_ONCE(entry->val, elt) for publishing the element. The sequence pairs with the mentioned "READ_ONCE(entry->val);" and the "val->key" check which has an address dependency.
The barrier is placed on a path executed when adding an element for a new key. Subsequent updates targeting the same key remain unaffected.
Quoted 1 line
From the user's perspective, the issue was introduced by commit
c193707dde77 ("tracing: Remove code which merges duplicates"), which followed commit cbf4100efb8f ("tracing: Add support to detect and avoid duplicates"). The previous code operated differently; it inherently expected potential races which result in duplicates but merged them later when they occurred.
--- kernel/trace/tracing_map.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
kernel/trace/tracing_map.c +6 -1
@@ -574,7 +574,12 @@ __tracing_map_insert(struct tracing_map *map, void *key, bool lookup_only) }memcpy (elt->key, key, map->key_size); - entry->val = elt; +/* +* Ensure the initialization is visible and +* publish the elt. +*/ +smp_wmb (); +WRITE_ONCE (entry->val, elt);atomic64_inc (&map->hits);return entry->val;
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 185/331 afs: Hide silly-rename files from userspace
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: David Howells <[email protected]>
[ Upstream commit 57e9d49c54528c49b8bffe6d99d782ea051ea534 ]
There appears to be a race between silly-rename files being created/removed and various userspace tools iterating over the contents of a directory, leading to such errors as:
find: './kernel/.tmp_cpio_dir/include/dt-bindings/reset/.__afs2080': No such file or directory
tar: ./include/linux/greybus/.__afs3C95: File removed before we read it
when building a kernel.
Fix afs_readdir() so that it doesn't return .__afsXXXX silly-rename files to userspace. This doesn't stop them being looked up directly by name as we need to be able to look them up from within the kernel as part of the silly-rename algorithm.
--- fs/afs/dir.c | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/afs/dir.c +8 -0
@@ -474,6 +474,14 @@ static int afs_dir_iterate_block(struct afs_vnode *dvnode,continue ; } +/* Don't expose silly rename entries to userspace. */ +if (nlen >6 && + dire->u.name[0 ] =='.' && + ctx->actor != afs_lookup_filldir && + ctx->actor != afs_lookup_one_filldir && +memcmp (dire->u.name,".__afs" ,6 ) ==0 ) +continue ; +/* found the next entry */ if (!dir_emit(ctx, dire->u.name, nlen, ntohl(dire->u.vnode),
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 186/331 tcp: Add memory barrier to tcp_push()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Salvatore Dipietro <[email protected]>
[ Upstream commit 7267e8dcad6b2f9fce05a6a06335d7040acbc2b6 ]
On CPUs with weak memory models, reads and updates performed by tcp_push to the sk variables can get reordered leaving the socket throttled when it should not. The tasklet running tcp_wfree() may also not observe the memory updates in time and will skip flushing any packets throttled by tcp_push(), delaying the sending. This can pathologically cause 40ms extra latency due to bad interactions with delayed acks.
Adding a memory barrier in tcp_push removes the bug, similarly to the previous commit bf06200e732d ("tcp: tsq: fix nonagle handling"). smp_mb__after_atomic() is used to not incur in unnecessary overhead on x86 since not affected.
Patch has been tested using an AWS c7g.2xlarge instance with Ubuntu 22.04 and Apache Tomcat 9.0.83 running the basic servlet below:
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
public class HelloWorldServlet extends HttpServlet {
@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
response.setContentType("text/html;charset=utf-8");
OutputStreamWriter osw = new OutputStreamWriter(response.getOutputStream(),"UTF-8");
String s = "a".repeat(3096);
osw.write(s,0,s.length());
osw.flush();
}
}Load was applied using wrk2 (https://github.com/kinvolk/wrk2) from an AWS c6i.8xlarge instance. Before the patch an additional 40ms latency from P99.99+ values is observed while, with the patch, the extra latency disappears.
No patch and tcp_autocorking=1
./wrk -t32 -c128 -d40s --latency -R10000 http://172.31.60.173:8080/hello/hello
...
50.000% 0.91ms 75.000% 1.13ms 90.000% 1.46ms 99.000% 1.74ms 99.900% 1.89ms 99.990% 41.95ms <<< 40+ ms extra latency 99.999% 48.32ms 100.000% 48.96ms
With patch and tcp_autocorking=1
./wrk -t32 -c128 -d40s --latency -R10000 http://172.31.60.173:8080/hello/hello
...
50.000% 0.90ms 75.000% 1.13ms 90.000% 1.45ms 99.000% 1.72ms 99.900% 1.83ms 99.990% 2.11ms <<< no 40+ ms extra latency 99.999% 2.53ms 100.000% 2.62ms
Patch has been also tested on x86 (m7i.2xlarge instance) which it is not affected by this issue and the patch doesn't introduce any additional delay.
--- net/ipv4/tcp.c | 1 + 1 file changed, 1 insertion(+)
net/ipv4/tcp.c +1 -0
@@ -722,6 +722,7 @@ void tcp_push(struct sock *sk, int flags, int mss_now,if (!test_bit (TSQ_THROTTLED , &sk->sk_tsq_flags)) {NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPAUTOCORKING );set_bit (TSQ_THROTTLED , &sk->sk_tsq_flags); +smp_mb__after_atomic (); }/* It is possible TX completion already happened * before we set TSQ_THROTTLED.
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 187/331 selftest: Dont reuse port for SO_INCOMING_CPU test.
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Kuniyuki Iwashima <[email protected]>
[ Upstream commit 97de5a15edf2d22184f5ff588656030bbb7fa358 ]
Jakub reported that ASSERT_EQ(cpu, i) in so_incoming_cpu.c seems to fire somewhat randomly.
# # RUN so_incoming_cpu.before_reuseport.test3 ...
# # so_incoming_cpu.c:191:test3:Expected cpu (32) == i (0) # # test3: Test terminated by assertion
# # FAIL so_incoming_cpu.before_reuseport.test3
# not ok 3 so_incoming_cpu.before_reuseport.test3
When the test failed, not-yet-accepted CLOSE_WAIT sockets received SYN with a "challenging" SEQ number, which was sent from an unexpected CPU that did not create the receiver.
The test basically does:
1. for each cpu:
1-1. create a server
1-2. set SO_INCOMING_CPU
2. for each cpu:
2-1. set cpu affinity
2-2. create some clients
2-3. let clients connect() to the server on the same cpu 2-4. close() clients
3. for each server:
3-1. accept() all child sockets
3-2. check if all children have the same SO_INCOMING_CPU with the server
The root cause was the close() in 2-4. and net.ipv4.tcp_tw_reuse.
In a loop of 2., close() changed the client state to FIN_WAIT_2, and the peer transitioned to CLOSE_WAIT.
In another loop of 2., connect() happened to select the same port of the FIN_WAIT_2 socket, and it was reused as the default value of net.ipv4.tcp_tw_reuse is 2.
As a result, the new client sent SYN to the CLOSE_WAIT socket from a different CPU, and the receiver's sk_incoming_cpu was overwritten with unexpected CPU ID.
Also, the SYN had a different SEQ number, so the CLOSE_WAIT socket responded with Challenge ACK. The new client properly returned RST and effectively killed the CLOSE_WAIT socket.
This way, all clients were created successfully, but the error was detected later by 3-2., ASSERT_EQ(cpu, i).
To avoid the failure, let's make sure that (i) the number of clients is less than the number of available ports and (ii) such reuse never happens.
--- tools/testing/selftests/net/so_incoming_cpu.c | 68 ++++++++++++++----- 1 file changed, 50 insertions(+), 18 deletions(-)
tools/testing/selftests/net/so_incoming_cpu.c +50 -18
@@ -3,19 +3,16 @@#define _GNU_SOURCE#include <sched.h> +#include <fcntl.h> +#include <netinet/in.h> #include <sys/socket.h> #include <sys/sysinfo.h> #include "../kselftest_harness.h" -#define CLIENT_PER_SERVER 32/* More sockets, more reliable */ -#define NR_SERVER self->nproc -#define NR_CLIENT (CLIENT_PER_SERVER * NR_SERVER) -FIXTURE (so_incoming_cpu) { -int nproc;int *servers;union {struct sockaddr addr;
@@ -56,12 +53,47 @@ FIXTURE_VARIANT_ADD(so_incoming_cpu, after_all_listen) .when_to_set = AFTER_ALL_LISTEN, }; +static void write_sysctl (struct __test_metadata *_metadata, +char *filename,char *string) +{ +int fd, len, ret; + + fd =open (filename,O_WRONLY ); +ASSERT_NE (fd,-1 ); + + len =strlen (string); + ret =write (fd, string, len); +ASSERT_EQ (ret, len); +} + +static void setup_netns (struct __test_metadata *_metadata) +{ +ASSERT_EQ (unshare (CLONE_NEWNET ),0 ); +ASSERT_EQ (system ("ip link set lo up" ),0 ); + +write_sysctl (_metadata,"/proc/sys/net/ipv4/ip_local_port_range" ,"10000 60001" ); +write_sysctl (_metadata,"/proc/sys/net/ipv4/tcp_tw_reuse" ,"0" ); +} + +#define NR_PORT (60001 - 10000 - 1) +#define NR_CLIENT_PER_SERVER_DEFAULT 32 +static int nr_client_per_server, nr_server, nr_client; +FIXTURE_SETUP (so_incoming_cpu) { - self->nproc =get_nprocs (); -ASSERT_LE (2 , self->nproc); +setup_netns (_metadata); + + nr_server =get_nprocs (); +ASSERT_LE (2 , nr_server); + +if (NR_CLIENT_PER_SERVER_DEFAULT * nr_server <NR_PORT ) + nr_client_per_server =NR_CLIENT_PER_SERVER_DEFAULT ; +else + nr_client_per_server =NR_PORT / nr_server; + + nr_client = nr_client_per_server * nr_server; - self->servers =malloc (sizeof (int ) *NR_SERVER ); + self->servers =malloc (sizeof (int ) * nr_server);ASSERT_NE (self->servers,NULL ); self->in_addr.sin_family =AF_INET ;
@@ -74,7 +106,7 @@ FIXTURE_TEARDOWN(so_incoming_cpu) {int i; -for (i =0 ; i <NR_SERVER ; i++) +for (i =0 ; i < nr_server; i++)close (self->servers[i]);free (self->servers);
@@ -110,10 +142,10 @@ int create_server(struct __test_metadata *_metadata,if (variant->when_to_set ==BEFORE_LISTEN )set_so_incoming_cpu (_metadata, fd, cpu); -/* We don't use CLIENT_PER_SERVER here not to block +/* We don't use nr_client_per_server here not to block * this test at connect() if SO_INCOMING_CPU is broken. */ - ret =listen (fd,NR_CLIENT ); + ret =listen (fd, nr_client);ASSERT_EQ (ret,0 );if (variant->when_to_set ==AFTER_LISTEN )
@@ -128,7 +160,7 @@ void create_servers(struct __test_metadata *_metadata, {int i, ret; -for (i =0 ; i <NR_SERVER ; i++) { +for (i =0 ; i < nr_server; i++) { self->servers[i] =create_server (_metadata, self, variant, i);if (i ==0 ) {
@@ -138,7 +170,7 @@ void create_servers(struct __test_metadata *_metadata, }if (variant->when_to_set ==AFTER_ALL_LISTEN ) { -for (i =0 ; i <NR_SERVER ; i++) +for (i =0 ; i < nr_server; i++)set_so_incoming_cpu (_metadata, self->servers[i], i); } }
@@ -149,7 +181,7 @@ void create_clients(struct __test_metadata *_metadata,cpu_set_t cpu_set;int i, j, fd, ret; -for (i =0 ; i <NR_SERVER ; i++) { +for (i =0 ; i < nr_server; i++) {CPU_ZERO (&cpu_set);CPU_SET (i, &cpu_set);
@@ -162,7 +194,7 @@ void create_clients(struct __test_metadata *_metadata, ret =sched_setaffinity (0 ,sizeof (cpu_set), &cpu_set);ASSERT_EQ (ret,0 ); -for (j =0 ; j <CLIENT_PER_SERVER ; j++) { +for (j =0 ; j < nr_client_per_server; j++) { fd =socket (AF_INET ,SOCK_STREAM ,0 );ASSERT_NE (fd,-1 );
@@ -180,8 +212,8 @@ void verify_incoming_cpu(struct __test_metadata *_metadata,int i, j, fd, cpu, ret, total =0 ;socklen_t len =sizeof (int ); -for (i =0 ; i <NR_SERVER ; i++) { -for (j =0 ; j <CLIENT_PER_SERVER ; j++) { +for (i =0 ; i < nr_server; i++) { +for (j =0 ; j < nr_client_per_server; j++) {/* If we see -EAGAIN here, SO_INCOMING_CPU is broken */ fd =accept (self->servers[i], &self->addr, &self->addrlen);ASSERT_NE (fd,-1 );
@@ -195,7 +227,7 @@ void verify_incoming_cpu(struct __test_metadata *_metadata, } } -ASSERT_EQ (total,NR_CLIENT ); +ASSERT_EQ (total, nr_client);TH_LOG ("SO_INCOMING_CPU is very likely to be " "working correctly with %d sockets." , total); }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 188/331 netlink: fix potential sleeping issue in mqueue_flush_file
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Zhengchao Shao <[email protected]>
[ Upstream commit 234ec0b6034b16869d45128b8cd2dc6ffe596f04 ]
I analyze the potential sleeping issue of the following processes:
Thread A Thread B ... netlink_create //ref = 1 do_mq_notify ... sock = netlink_getsockbyfilp ... //ref = 2 info->notify_sock = sock; ... ... netlink_sendmsg
... skb = netlink_alloc_large_skb //skb->head is vmalloced
... netlink_unicast
... sk = netlink_getsockbyportid //ref = 3
... netlink_sendskb
... __netlink_sendskb
... skb_queue_tail //put skb to sk_receive_queue
... sock_put //ref = 2
... ...
... netlink_release
... deferred_put_nlk_sk //ref = 1mqueue_flush_file
spin_lock
remove_notification
netlink_sendskb
sock_put //ref = 0
sk_free
...
__sk_destruct
netlink_sock_destruct
skb_queue_purge //get skb from sk_receive_queue
...
__skb_queue_purge_reason
kfree_skb_reason
__kfree_skb
...
skb_release_all
skb_release_head_state
netlink_skb_destructor
vfree(skb->head) //sleeping while holding spinlockIn netlink_sendmsg, if the memory pointed to by skb->head is allocated by vmalloc, and is put to sk_receive_queue queue, also the skb is not freed. When the mqueue executes flush, the sleeping bug will occur. Use vfree_atomic instead of vfree in netlink_skb_destructor to solve the issue.
--- net/netlink/af_netlink.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/netlink/af_netlink.c +1 -1
@@ -374,7 +374,7 @@ static void netlink_skb_destructor(struct sk_buff *skb)if (is_vmalloc_addr (skb->head)) {if (!skb->cloned || !atomic_dec_return (&(skb_shinfo (skb)->dataref))) -vfree (skb->head); +vfree_atomic (skb->head); skb->head =NULL ; }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 189/331 ipv6: init the accept_queues spinlocks in inet6_create
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Zhengchao Shao <[email protected]>
[ Upstream commit 435e202d645c197dcfd39d7372eb2a56529b6640 ]
In commit 198bc90e0e73("tcp: make sure init the accept_queue's spinlocks once"), the spinlocks of accept_queue are initialized only when socket is created in the inet4 scenario. The locks are not initialized when socket is created in the inet6 scenario. The kernel reports the following error:
INFO: trying to register non-static key.
The code is fine but needs lockdep annotation, or maybe
you didn't initialize this object before use?
turning off the locking correctness validator.
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 Call Trace: <TASK> dump_stack_lvl (lib/dump_stack.c:107) register_lock_class (kernel/locking/lockdep.c:1289) __lock_acquire (kernel/locking/lockdep.c:5015) lock_acquire.part.0 (kernel/locking/lockdep.c:5756) _raw_spin_lock_bh (kernel/locking/spinlock.c:178) inet_csk_listen_stop (net/ipv4/inet_connection_sock.c:1386) tcp_disconnect (net/ipv4/tcp.c:2981) inet_shutdown (net/ipv4/af_inet.c:935) __sys_shutdown (./include/linux/file.h:32 net/socket.c:2438) __x64_sys_shutdown (net/socket.c:2445) do_syscall_64 (arch/x86/entry/common.c:52) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:129) RIP: 0033:0x7f52ecd05a3d Code: 5b 41 5c c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ab a3 0e 00 f7 d8 64 89 01 48
RSP: 002b:00007f52ecf5dde8 EFLAGS: 00000293 ORIG_RAX: 0000000000000030
RAX: ffffffffffffffda RBX: 00007f52ecf5e640 RCX: 00007f52ecd05a3d
RDX: 00007f52ecc8b188 RSI: 0000000000000000 RDI: 0000000000000004
RBP: 00007f52ecf5de20 R08: 00007ffdae45c69f R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000293 R12: 00007f52ecf5e640
R13: 0000000000000000 R14: 00007f52ecc8b060 R15: 00007ffdae45c6e0
--- net/ipv6/af_inet6.c | 3 +++ 1 file changed, 3 insertions(+)
net/ipv6/af_inet6.c +3 -0
@@ -199,6 +199,9 @@ static int inet6_create(struct net *net, struct socket *sock, int protocol,if (INET_PROTOSW_REUSE & answer_flags) sk->sk_reuse =SK_CAN_REUSE ; +if (INET_PROTOSW_ICSK & answer_flags) +inet_init_csk_locks (sk); + inet =inet_sk (sk);inet_assign_bit (IS_ICSK , sk,INET_PROTOSW_ICSK & answer_flags);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 190/331 selftests: fill in some missing configs for net
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Jakub Kicinski <[email protected]>
[ Upstream commit 04fe7c5029cbdbcdb28917f09a958d939a8f19f7 ]
We are missing a lot of config options from net selftests, it seems:
tun/tap: CONFIG_TUN, CONFIG_MACVLAN, CONFIG_MACVTAP
fib_tests: CONFIG_NET_SCH_FQ_CODEL
l2tp: CONFIG_L2TP, CONFIG_L2TP_V3, CONFIG_L2TP_IP, CONFIG_L2TP_ETH
sctp-vrf: CONFIG_INET_DIAG
txtimestamp: CONFIG_NET_CLS_U32
vxlan_mdb: CONFIG_BRIDGE_VLAN_FILTERING
gre_gso: CONFIG_NET_IPGRE_DEMUX, CONFIG_IP_GRE, CONFIG_IPV6_GRE
srv6_end_dt*_l3vpn: CONFIG_IPV6_SEG6_LWTUNNEL
ip_local_port_range: CONFIG_MPTCP
fib_test: CONFIG_NET_CLS_BASIC
rtnetlink: CONFIG_MACSEC, CONFIG_NET_SCH_HTB, CONFIG_XFRM_INTERFACE
CONFIG_NET_IPGRE, CONFIG_BONDING
fib_nexthops: CONFIG_MPLS, CONFIG_MPLS_ROUTING
vxlan_mdb: CONFIG_NET_ACT_GACT
tls: CONFIG_TLS, CONFIG_CRYPTO_CHACHA20POLY1305
psample: CONFIG_PSAMPLE
fcnal: CONFIG_TCP_MD5SIGTry to add them in a semi-alphabetical order.
--- tools/testing/selftests/net/config | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
tools/testing/selftests/net/config +28 -0
@@ -1,5 +1,6 @@ CONFIG_USER_NS=y CONFIG_NET_NS=y +CONFIG_BONDING=m CONFIG_BPF_SYSCALL=y CONFIG_TEST_BPF=m CONFIG_NUMA=y
@@ -14,9 +15,13 @@ CONFIG_VETH=y CONFIG_NET_IPVTI=y CONFIG_IPV6_VTI=y CONFIG_DUMMY=y +CONFIG_BRIDGE_VLAN_FILTERING=y CONFIG_BRIDGE=y +CONFIG_CRYPTO_CHACHA20POLY1305=m CONFIG_VLAN_8021Q=y CONFIG_IFB=y +CONFIG_INET_DIAG=y +CONFIG_IP_GRE=m CONFIG_NETFILTER=y CONFIG_NETFILTER_ADVANCED=y CONFIG_NF_CONNTRACK=m
@@ -25,15 +30,36 @@ CONFIG_IP6_NF_IPTABLES=m CONFIG_IP_NF_IPTABLES=m CONFIG_IP6_NF_NAT=m CONFIG_IP_NF_NAT=m +CONFIG_IPV6_GRE=m +CONFIG_IPV6_SEG6_LWTUNNEL=y +CONFIG_L2TP_ETH=m +CONFIG_L2TP_IP=m +CONFIG_L2TP=m +CONFIG_L2TP_V3=y +CONFIG_MACSEC=m +CONFIG_MACVLAN=y +CONFIG_MACVTAP=y +CONFIG_MPLS=y +CONFIG_MPTCP=y CONFIG_NF_TABLES=m CONFIG_NF_TABLES_IPV6=y CONFIG_NF_TABLES_IPV4=y CONFIG_NFT_NAT=m +CONFIG_NET_ACT_GACT=m +CONFIG_NET_CLS_BASIC=m +CONFIG_NET_CLS_U32=m +CONFIG_NET_IPGRE_DEMUX=m +CONFIG_NET_IPGRE=m +CONFIG_NET_SCH_FQ_CODEL=m +CONFIG_NET_SCH_HTB=m CONFIG_NET_SCH_FQ=m CONFIG_NET_SCH_ETF=m CONFIG_NET_SCH_NETEM=y +CONFIG_PSAMPLE=m +CONFIG_TCP_MD5SIG=y CONFIG_TEST_BLACKHOLE_DEV=m CONFIG_KALLSYMS=y +CONFIG_TLS=m CONFIG_TRACEPOINTS=y CONFIG_NET_DROP_MONITOR=m CONFIG_NETDEVSIM=m
@@ -48,7 +74,9 @@ CONFIG_BAREUDP=m CONFIG_IPV6_IOAM6_LWTUNNEL=y CONFIG_CRYPTO_SM4_GENERIC=y CONFIG_AMT=m +CONFIG_TUN=y CONFIG_VXLAN=m CONFIG_IP_SCTP=m CONFIG_NETFILTER_XT_MATCH_POLICY=m CONFIG_CRYPTO_ARIA=y +CONFIG_XFRM_INTERFACE=m
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 191/331 net/sched: flower: Fix chain template offload
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Ido Schimmel <[email protected]>
[ Upstream commit 32f2a0afa95fae0d1ceec2ff06e0e816939964b8 ]
When a qdisc is deleted from a net device the stack instructs the underlying driver to remove its flow offload callback from the associated filter block using the 'FLOW_BLOCK_UNBIND' command. The stack then continues to replay the removal of the filters in the block for this driver by iterating over the chains in the block and invoking the 'reoffload' operation of the classifier being used. In turn, the classifier in its 'reoffload' operation prepares and emits a 'FLOW_CLS_DESTROY' command for each filter.
However, the stack does not do the same for chain templates and the underlying driver never receives a 'FLOW_CLS_TMPLT_DESTROY' command when a qdisc is deleted. This results in a memory leak [1] which can be reproduced using [2].
Fix by introducing a 'tmplt_reoffload' operation and have the stack invoke it with the appropriate arguments as part of the replay. Implement the operation in the sole classifier that supports chain templates (flower) by emitting the 'FLOW_CLS_TMPLT_{CREATE,DESTROY}' command based on whether a flow offload callback is being bound to a filter block or being unbound from one.
As far as I can tell, the issue happens since cited commit which reordered tcf_block_offload_unbind() before tcf_block_flush_all_chains() in __tcf_block_put(). The order cannot be reversed as the filter block is expected to be freed after flushing all the chains.
[1]
unreferenced object 0xffff888107e28800 (size 2048):
comm "tc", pid 1079, jiffies 4294958525 (age 3074.287s) hex dump (first 32 bytes):
b1 a6 7c 11 81 88 ff ff e0 5b b3 10 81 88 ff ff ..|......[......
01 00 00 00 00 00 00 00 e0 aa b0 84 ff ff ff ff ................
backtrace:
[<ffffffff81c06a68>] __kmem_cache_alloc_node+0x1e8/0x320
[<ffffffff81ab374e>] __kmalloc+0x4e/0x90
[<ffffffff832aec6d>] mlxsw_sp_acl_ruleset_get+0x34d/0x7a0
[<ffffffff832bc195>] mlxsw_sp_flower_tmplt_create+0x145/0x180
[<ffffffff832b2e1a>] mlxsw_sp_flow_block_cb+0x1ea/0x280
[<ffffffff83a10613>] tc_setup_cb_call+0x183/0x340
[<ffffffff83a9f85a>] fl_tmplt_create+0x3da/0x4c0
[<ffffffff83a22435>] tc_ctl_chain+0xa15/0x1170
[<ffffffff838a863c>] rtnetlink_rcv_msg+0x3cc/0xed0
[<ffffffff83ac87f0>] netlink_rcv_skb+0x170/0x440
[<ffffffff83ac6270>] netlink_unicast+0x540/0x820
[<ffffffff83ac6e28>] netlink_sendmsg+0x8d8/0xda0
[<ffffffff83793def>] ____sys_sendmsg+0x30f/0xa80
[<ffffffff8379d29a>] ___sys_sendmsg+0x13a/0x1e0
[<ffffffff8379d50c>] __sys_sendmsg+0x11c/0x1f0
[<ffffffff843b9ce0>] do_syscall_64+0x40/0xe0
unreferenced object 0xffff88816d2c0400 (size 1024):
comm "tc", pid 1079, jiffies 4294958525 (age 3074.287s)
hex dump (first 32 bytes):
40 00 00 00 00 00 00 00 57 f6 38 be 00 00 00 00 @.......W.8.....
10 04 2c 6d 81 88 ff ff 10 04 2c 6d 81 88 ff ff ..,m......,m....
backtrace:
[<ffffffff81c06a68>] __kmem_cache_alloc_node+0x1e8/0x320
[<ffffffff81ab36c1>] __kmalloc_node+0x51/0x90
[<ffffffff81a8ed96>] kvmalloc_node+0xa6/0x1f0
[<ffffffff82827d03>] bucket_table_alloc.isra.0+0x83/0x460
[<ffffffff82828d2b>] rhashtable_init+0x43b/0x7c0
[<ffffffff832aed48>] mlxsw_sp_acl_ruleset_get+0x428/0x7a0
[<ffffffff832bc195>] mlxsw_sp_flower_tmplt_create+0x145/0x180
[<ffffffff832b2e1a>] mlxsw_sp_flow_block_cb+0x1ea/0x280
[<ffffffff83a10613>] tc_setup_cb_call+0x183/0x340
[<ffffffff83a9f85a>] fl_tmplt_create+0x3da/0x4c0
[<ffffffff83a22435>] tc_ctl_chain+0xa15/0x1170
[<ffffffff838a863c>] rtnetlink_rcv_msg+0x3cc/0xed0
[<ffffffff83ac87f0>] netlink_rcv_skb+0x170/0x440
[<ffffffff83ac6270>] netlink_unicast+0x540/0x820
[<ffffffff83ac6e28>] netlink_sendmsg+0x8d8/0xda0
[<ffffffff83793def>] ____sys_sendmsg+0x30f/0xa80[2]
# tc qdisc add dev swp1 clsact # tc chain add dev swp1 ingress proto ip chain 1 flower dst_ip 0.0.0.0/32 # tc qdisc del dev swp1 clsact # devlink dev reload pci/0000:06:00.0
--- include/net/sch_generic.h | 4 ++++ net/sched/cls_api.c | 9 ++++++++- net/sched/cls_flower.c | 23 +++++++++++++++++++++++ 3 files changed, 35 insertions(+), 1 deletion(-)
include/net/sch_generic.h +4 -0
@@ -376,6 +376,10 @@ struct tcf_proto_ops {struct nlattr **tca, struct netlink_ext_ack *extack);void (*tmplt_destroy)(void *tmplt_priv); +void (*tmplt_reoffload)(struct tcf_chain *chain, +bool add, +flow_setup_cb_t *cb, +void *cb_priv);struct tcf_exts * (*get_exts)(const struct tcf_proto *tp,u32 handle);
net/sched/cls_api.c +8 -1
@@ -1536,6 +1536,9 @@ tcf_block_playback_offloads(struct tcf_block *block, flow_setup_cb_t *cb, chain_prev = chain, chain = __tcf_get_next_chain(block, chain), tcf_chain_put(chain_prev)) { +if (chain->tmplt_ops && add) + chain->tmplt_ops->tmplt_reoffload (chain, true, cb, + cb_priv); for (tp = __tcf_get_next_proto(chain, NULL); tp; tp_prev = tp, tp = __tcf_get_next_proto(chain, tp),
@@ -1551,6 +1554,9 @@ tcf_block_playback_offloads(struct tcf_block *block, flow_setup_cb_t *cb, goto err_playback_remove; } } +if (chain->tmplt_ops && !add) + chain->tmplt_ops->tmplt_reoffload (chain, false, cb, + cb_priv); }return 0 ;
@@ -2950,7 +2956,8 @@ static int tc_chain_tmplt_add(struct tcf_chain *chain, struct net *net, ops =tcf_proto_lookup_ops (name, true, extack);if (IS_ERR (ops))return PTR_ERR (ops); -if (!ops->tmplt_create || !ops->tmplt_destroy || !ops->tmplt_dump) { +if (!ops->tmplt_create || !ops->tmplt_destroy || !ops->tmplt_dump || + !ops->tmplt_reoffload) {NL_SET_ERR_MSG (extack,"Chain templates are not supported with specified classifier" );module_put (ops->owner);return -EOPNOTSUPP ;
net/sched/cls_flower.c +23 -0
@@ -2721,6 +2721,28 @@ static void fl_tmplt_destroy(void *tmplt_priv)kfree (tmplt); } +static void fl_tmplt_reoffload (struct tcf_chain *chain,bool add, +flow_setup_cb_t *cb,void *cb_priv) +{ +struct fl_flow_tmplt *tmplt = chain->tmplt_priv; +struct flow_cls_offload cls_flower = {}; + + cls_flower.rule =flow_rule_alloc (0 ); +if (!cls_flower.rule) +return ; + + cls_flower.common.chain_index = chain->index; + cls_flower.command = add ?FLOW_CLS_TMPLT_CREATE : +FLOW_CLS_TMPLT_DESTROY ; + cls_flower.cookie = (unsigned long ) tmplt; + cls_flower.rule->match.dissector = &tmplt->dissector; + cls_flower.rule->match.mask = &tmplt->mask; + cls_flower.rule->match.key = &tmplt->dummy_key; + +cb (TC_SETUP_CLSFLOWER , &cls_flower, cb_priv); +kfree (cls_flower.rule); +} +static int fl_dump_key_val (struct sk_buff *skb,void *val,int val_type,void *mask,int mask_type,int len)
@@ -3628,6 +3650,7 @@ static struct tcf_proto_ops cls_fl_ops __read_mostly = { .bind_class = fl_bind_class, .tmplt_create = fl_tmplt_create, .tmplt_destroy = fl_tmplt_destroy, + .tmplt_reoffload = fl_tmplt_reoffload, .tmplt_dump = fl_tmplt_dump, .get_exts = fl_get_exts, .owner = THIS_MODULE,
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 192/331 net/mlx5e: Fix operation precedence bug in port timestamping napi_poll context
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Rahul Rameshbabu <[email protected]>
[ Upstream commit 3876638b2c7ebb2c9d181de1191db0de8cac143a ]
Indirection (*) is of lower precedence than postfix increment (++). Logic in napi_poll context would cause an out-of-bound read by first increment the pointer address by byte address space and then dereference the value. Rather, the intended logic was to dereference first and then increment the underlying value.
--- drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c +1 -1
@@ -213,7 +213,7 @@ static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq,mlx5e_ptpsq_mark_ts_cqes_undelivered (ptpsq, hwtstamp); out:napi_consume_skb (skb, budget); - md_buff[*md_buff_sz++] = metadata_id; + md_buff[(*md_buff_sz)++] = metadata_id;if (unlikely (mlx5e_ptp_metadata_map_unhealthy (&ptpsq->metadata_map)) && !test_and_set_bit (MLX5E_SQ_STATE_RECOVERING , &sq->state))queue_work (ptpsq->txqsq.priv->wq, &ptpsq->report_unhealthy_work);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 193/331 net/mlx5e: Fix peer flow lists handling
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Vlad Buslov <[email protected]>
[ Upstream commit d76fdd31f953ac5046555171620f2562715e9b71 ]
The cited change refactored mlx5e_tc_del_fdb_peer_flow() to only clear DUP flag when list of peer flows has become empty. However, if any concurrent user holds a reference to a peer flow (for example, the neighbor update workqueue task is updating peer flow's parent encap entry concurrently), then the flow will not be removed from the peer list and, consecutively, DUP flag will remain set. Since mlx5e_tc_del_fdb_peers_flow() calls mlx5e_tc_del_fdb_peer_flow() for every possible peer index the algorithm will try to remove the flow from eswitch instances that it has never peered with causing either NULL pointer dereference when trying to remove the flow peer list head of peer_index that was never initialized or a warning if the list debug config is enabled[0].
Fix the issue by always removing the peer flow from the list even when not releasing the last reference to it.
[0]:
[ 3102.985806] ------------[ cut here ]------------ [ 3102.986223] list_del corruption, ffff888139110698->next is NULL [ 3102.986757] WARNING: CPU: 2 PID: 22109 at lib/list_debug.c:53 __list_del_entry_valid_or_report+0x4f/0xc0 [ 3102.987561] Modules linked in: act_ct nf_flow_table bonding act_tunnel_key act_mirred act_skbedit vxlan cls_matchall nfnetlink_cttimeout act_gact cls_flower sch_ingress mlx5_vdpa vringh vhost_iotlb vdpa openvswitch nsh xt_MASQUERADE nf_conntrack_netlink nfnetlink iptable_nat xt_addrtype xt_conntrack nf_nat br_netfilter rpcsec_gss_krb5 auth_rpcg ss oid_registry overlay rpcrdma rdma_ucm ib_iser libiscsi scsi_transport_iscsi ib_umad rdma_cm ib_ipoib iw_cm ib_cm mlx5_ib ib_uverbs ib_core mlx5_core [last unloaded: bonding] [ 3102.991113] CPU: 2 PID: 22109 Comm: revalidator28 Not tainted 6.6.0-rc6+ #3 [ 3102.991695] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 [ 3102.992605] RIP: 0010:__list_del_entry_valid_or_report+0x4f/0xc0 [ 3102.993122] Code: 39 c2 74 56 48 8b 32 48 39 fe 75 62 48 8b 51 08 48 39 f2 75 73 b8 01 00 00 00 c3 48 89 fe 48 c7 c7 48 fd 0a 82 e8 41 0b ad ff <0f> 0b 31 c0 c3 48 89 fe 48 c7 c7 70 fd 0a 82 e8 2d 0b ad ff 0f 0b [ 3102.994615] RSP: 0018:ffff8881383e7710 EFLAGS: 00010286 [ 3102.995078] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 0000000000000000 [ 3102.995670] RDX: 0000000000000001 RSI: ffff88885f89b640 RDI: ffff88885f89b640 [ 3102.997188] DEL flow 00000000be367878 on port 0 [ 3102.998594] RBP: dead000000000122 R08: 0000000000000000 R09: c0000000ffffdfff [ 3102.999604] R10: 0000000000000008 R11: ffff8881383e7598 R12: dead000000000100 [ 3103.000198] R13: 0000000000000002 R14: ffff888139110000 R15: ffff888101901240 [ 3103.000790] FS: 00007f424cde4700(0000) GS:ffff88885f880000(0000) knlGS:0000000000000000 [ 3103.001486] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3103.001986] CR2: 00007fd42e8dcb70 CR3: 000000011e68a003 CR4: 0000000000370ea0 [ 3103.002596] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 3103.003190] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 3103.003787] Call Trace: [ 3103.004055] <TASK> [ 3103.004297] ? __warn+0x7d/0x130 [ 3103.004623] ? __list_del_entry_valid_or_report+0x4f/0xc0 [ 3103.005094] ? report_bug+0xf1/0x1c0 [ 3103.005439] ? console_unlock+0x4a/0xd0 [ 3103.005806] ? handle_bug+0x3f/0x70 [ 3103.006149] ? exc_invalid_op+0x13/0x60 [ 3103.006531] ? asm_exc_invalid_op+0x16/0x20 [ 3103.007430] ? __list_del_entry_valid_or_report+0x4f/0xc0 [ 3103.007910] mlx5e_tc_del_fdb_peers_flow+0xcf/0x240 [mlx5_core] [ 3103.008463] mlx5e_tc_del_flow+0x46/0x270 [mlx5_core] [ 3103.008944] mlx5e_flow_put+0x26/0x50 [mlx5_core] [ 3103.009401] mlx5e_delete_flower+0x25f/0x380 [mlx5_core] [ 3103.009901] tc_setup_cb_destroy+0xab/0x180 [ 3103.010292] fl_hw_destroy_filter+0x99/0xc0 [cls_flower] [ 3103.010779] __fl_delete+0x2d4/0x2f0 [cls_flower] [ 3103.011207] fl_delete+0x36/0x80 [cls_flower] [ 3103.011614] tc_del_tfilter+0x56f/0x750 [ 3103.011982] rtnetlink_rcv_msg+0xff/0x3a0 [ 3103.012362] ? netlink_ack+0x1c7/0x4e0 [ 3103.012719] ? rtnl_calcit.isra.44+0x130/0x130 [ 3103.013134] netlink_rcv_skb+0x54/0x100 [ 3103.013533] netlink_unicast+0x1ca/0x2b0 [ 3103.013902] netlink_sendmsg+0x361/0x4d0 [ 3103.014269] __sock_sendmsg+0x38/0x60 [ 3103.014643] ____sys_sendmsg+0x1f2/0x200 [ 3103.015018] ? copy_msghdr_from_user+0x72/0xa0 [ 3103.015265] ___sys_sendmsg+0x87/0xd0 [ 3103.016608] ? copy_msghdr_from_user+0x72/0xa0 [ 3103.017014] ? ___sys_recvmsg+0x9b/0xd0 [ 3103.017381] ? ttwu_do_activate.isra.137+0x58/0x180 [ 3103.017821] ? wake_up_q+0x49/0x90 [ 3103.018157] ? futex_wake+0x137/0x160 [ 3103.018521] ? __sys_sendmsg+0x51/0x90 [ 3103.018882] __sys_sendmsg+0x51/0x90 [ 3103.019230] ? exit_to_user_mode_prepare+0x56/0x130 [ 3103.019670] do_syscall_64+0x3c/0x80 [ 3103.020017] entry_SYSCALL_64_after_hwframe+0x46/0xb0 [ 3103.020469] RIP: 0033:0x7f4254811ef4 [ 3103.020816] Code: 89 f3 48 83 ec 10 48 89 7c 24 08 48 89 14 24 e8 42 eb ff ff 48 8b 14 24 41 89 c0 48 89 de 48 8b 7c 24 08 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 30 44 89 c7 48 89 04 24 e8 78 eb ff ff 48 8b [ 3103.022290] RSP: 002b:00007f424cdd9480 EFLAGS: 00000293 ORIG_RAX: 000000000000002e [ 3103.022970] RAX: ffffffffffffffda RBX: 00007f424cdd9510 RCX: 00007f4254811ef4 [ 3103.023564] RDX: 0000000000000000 RSI: 00007f424cdd9510 RDI: 0000000000000012 [ 3103.024158] RBP: 00007f424cdda238 R08: 0000000000000000 R09: 00007f41d801a4b0 [ 3103.024748] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000001 [ 3103.025341] R13: 00007f424cdd9510 R14: 00007f424cdda240 R15: 00007f424cdd99a0 [ 3103.025931] </TASK> [ 3103.026182] ---[ end trace 0000000000000000 ]--- [ 3103.027033] ------------[ cut here ]------------
--- drivers/net/ethernet/mellanox/mlx5/core/en_tc.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +2 -1
@@ -2012,9 +2012,10 @@ static void mlx5e_tc_del_fdb_peer_flow(struct mlx5e_tc_flow *flow,list_for_each_entry_safe (peer_flow, tmp, &flow->peer_flows, peer_flows) {if (peer_index !=mlx5_get_dev_index (peer_flow->priv->mdev))continue ; + +list_del (&peer_flow->peer_flows);if (refcount_dec_and_test (&peer_flow->refcnt)) {mlx5e_tc_del_fdb_flow (peer_flow->priv, peer_flow); -list_del (&peer_flow->peer_flows);kfree (peer_flow); } }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 194/331 net/mlx5: Fix a WARN upon a callback command failure
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Yishai Hadas <[email protected]>
[ Upstream commit cc8091587779cfaddb6b29c9e9edb9079a282cad ]
The below WARN [1] is reported once a callback command failed.
As a callback runs under an interrupt context, needs to use the IRQ save/restore variant.
[1]
DEBUG_LOCKS_WARN_ON(lockdep_hardirq_context())
WARNING: CPU: 15 PID: 0 at kernel/locking/lockdep.c:4353
lockdep_hardirqs_on_prepare+0x11b/0x180
Modules linked in: vhost_net vhost tap mlx5_vfio_pcivfio_pci vfio_pci_core vfio_iommu_type1 vfio mlx5_vdpa vringh vhost_iotlb vdpa nfnetlink_cttimeout openvswitch nsh ip6table_mangle ip6table_nat ip6table_filter ip6_tables iptable_mangle xt_conntrackxt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat br_netfilter rpcsec_gss_krb5 auth_rpcgss oid_registry overlay rpcrdma rdma_ucm ib_iser libiscsi scsi_transport_iscsi rdma_cm iw_cm ib_umad ib_ipoib ib_cm mlx5_ib ib_uverbs ib_core fuse mlx5_core
CPU: 15 PID: 0 Comm: swapper/15 Tainted: G W 6.7.0-rc4+ #1587
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
RIP: 0010:lockdep_hardirqs_on_prepare+0x11b/0x180
Code: 00 5b c3 c3 e8 e6 0d 58 00 85 c0 74 d6 8b 15 f0 c3
76 01 85 d2 75 cc 48 c7 c6 04 a5 3b 82 48 c7 c7 f1
e9 39 82 e8 95 12 f9 ff <0f> 0b 5b c3 e8 bc 0d 58 00
85 c0 74 ac 8b 3d c6 c3 76 01 85 ff 75
RSP: 0018:ffffc900003ecd18 EFLAGS: 00010086
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000027
RDX: 0000000000000000 RSI: ffff88885fbdb880 RDI: ffff88885fbdb888
RBP: 00000000ffffff87 R08: 0000000000000000 R09: 0000000000000001
R10: 0000000000000000 R11: 284e4f5f4e524157 R12: 00000000002c9aa1
R13: ffff88810aace980 R14: ffff88810aace9b8 R15: 0000000000000003
FS: 0000000000000000(0000) GS:ffff88885fbc0000(0000)
knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f731436f4c8 CR3: 000000010aae6001 CR4: 0000000000372eb0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
? __warn+0x81/0x170
? lockdep_hardirqs_on_prepare+0x11b/0x180
? report_bug+0xf8/0x1c0
? handle_bug+0x3f/0x70
? exc_invalid_op+0x13/0x60
? asm_exc_invalid_op+0x16/0x20
? lockdep_hardirqs_on_prepare+0x11b/0x180
? lockdep_hardirqs_on_prepare+0x11b/0x180
trace_hardirqs_on+0x4a/0xa0
raw_spin_unlock_irq+0x24/0x30
cmd_status_err+0xc0/0x1a0 [mlx5_core]
cmd_status_err+0x1a0/0x1a0 [mlx5_core]
mlx5_cmd_exec_cb_handler+0x24/0x40 [mlx5_core]
mlx5_cmd_comp_handler+0x129/0x4b0 [mlx5_core]
cmd_comp_notifier+0x1a/0x20 [mlx5_core]
notifier_call_chain+0x3e/0xe0
atomic_notifier_call_chain+0x5f/0x130
mlx5_eq_async_int+0xe7/0x200 [mlx5_core]
notifier_call_chain+0x3e/0xe0
atomic_notifier_call_chain+0x5f/0x130
irq_int_handler+0x11/0x20 [mlx5_core]
__handle_irq_event_percpu+0x99/0x220
? tick_irq_enter+0x5d/0x80
handle_irq_event_percpu+0xf/0x40
handle_irq_event+0x3a/0x60
handle_edge_irq+0xa2/0x1c0
__common_interrupt+0x55/0x140
common_interrupt+0x7d/0xa0
</IRQ>
<TASK>
asm_common_interrupt+0x22/0x40
RIP: 0010:default_idle+0x13/0x20
Code: c0 08 00 00 00 4d 29 c8 4c 01 c7 4c 29 c2 e9 72 ffff ff cc cc cc cc 8b 05 ea 08 25 01 85 c0 7e 07 0f 00 2d 7f b0 26 00 fb f4 <fa> c3 90 66 2e 0f 1f 84 00 00 00 00 00 65 48 8b 04 25 80 d0 02 00
RSP: 0018:ffffc9000010fec8 EFLAGS: 00000242
RAX: 0000000000000001 RBX: 000000000000000f RCX: 4000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff811c410c
RBP: ffffffff829478c0 R08: 0000000000000001 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
? do_idle+0x1ec/0x210 default_idle_call+0x6c/0x90 do_idle+0x1ec/0x210 cpu_startup_entry+0x26/0x30 start_secondary+0x11b/0x150 secondary_startup_64_no_verify+0x165/0x16b </TASK> irq event stamp: 833284 hardirqs last enabled at (833283): [<ffffffff811c410c>] do_idle+0x1ec/0x210 hardirqs last disabled at (833284): [<ffffffff81daf9ef>] common_interrupt+0xf/0xa0 softirqs last enabled at (833224): [<ffffffff81dc199f>] __do_softirq+0x2bf/0x40e softirqs last disabled at (833177): [<ffffffff81178ddf>] irq_exit_rcu+0x7f/0xa0
--- drivers/net/ethernet/mellanox/mlx5/core/cmd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/cmd.c +3 -2
@@ -1921,6 +1921,7 @@ static void cmd_status_log(struct mlx5_core_dev *dev, u16 opcode, u8 status, {const char *namep =mlx5_command_str (opcode);struct mlx5_cmd_stats *stats; +unsigned long flags;if (!err || !(strcmp (namep,"unknown command opcode" )))return ;
@@ -1928,7 +1929,7 @@ static void cmd_status_log(struct mlx5_core_dev *dev, u16 opcode, u8 status, stats =xa_load (&dev->cmd.stats, opcode);if (!stats)return ; -spin_lock_irq (&stats->lock); +spin_lock_irqsave (&stats->lock, flags); stats->failed++;if (err <0 ) stats->last_failed_errno = -err;
@@ -1937,7 +1938,7 @@ static void cmd_status_log(struct mlx5_core_dev *dev, u16 opcode, u8 status, stats->last_failed_mbox_status = status; stats->last_failed_syndrome = syndrome; } -spin_unlock_irq (&stats->lock); +spin_unlock_irqrestore (&stats->lock, flags); }/* preserve -EREMOTEIO for outbox.status != OK, otherwise return err as is */
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 195/331 net/mlx5: Bridge, Enable mcast in smfs steering mode
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Erez Shitrit <[email protected]>
[ Upstream commit 653b7eb9d74426397c95061fd57da3063625af65 ]
In order to have mcast offloads the driver needs the following:
It should know if that mcast comes from wire port, in addition the flow should not be marked as any specific source, that way it will give the flexibility for the driver not to be depended on the way iterator implemented in the FW.
Stable-dep-of: ec7cc38ef9f8 ("net/mlx5: Bridge, fix multicast packets sent to uplink")
--- .../ethernet/mellanox/mlx5/core/esw/bridge_mcast.c | 11 ++--------- include/linux/mlx5/fs.h | 1 + 2 files changed, 3 insertions(+), 9 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_mcast.c +2 -9
@@ -78,6 +78,8 @@ mlx5_esw_bridge_mdb_flow_create(u16 esw_owner_vhca_id, struct mlx5_esw_bridge_mdxa_for_each (&entry->ports, idx, port) { dests[i].type =MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE ; dests[i].ft = port->mcast.ft; +if (port->vport_num ==MLX5_VPORT_UPLINK ) + dests[i].ft->flags |=MLX5_FLOW_TABLE_UPLINK_VPORT ; i++; }
@@ -585,10 +587,6 @@ mlx5_esw_bridge_mcast_vlan_flow_create(u16 vlan_proto, struct mlx5_esw_bridge_poif (!rule_spec)return ERR_PTR (-ENOMEM ); -if (MLX5_CAP_ESW_FLOWTABLE (bridge->br_offloads->esw->dev, flow_source) && - port->vport_num ==MLX5_VPORT_UPLINK ) - rule_spec->flow_context.flow_source = -MLX5_FLOW_CONTEXT_FLOW_SOURCE_LOCAL_VPORT ; rule_spec->match_criteria_enable =MLX5_MATCH_OUTER_HEADERS ; flow_act.action |=MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT ;
@@ -660,11 +658,6 @@ mlx5_esw_bridge_mcast_fwd_flow_create(struct mlx5_esw_bridge_port *port)if (!rule_spec)return ERR_PTR (-ENOMEM ); -if (MLX5_CAP_ESW_FLOWTABLE (bridge->br_offloads->esw->dev, flow_source) && - port->vport_num ==MLX5_VPORT_UPLINK ) - rule_spec->flow_context.flow_source = -MLX5_FLOW_CONTEXT_FLOW_SOURCE_LOCAL_VPORT ; -if (MLX5_CAP_ESW (bridge->br_offloads->esw->dev, merged_eswitch)) { dest.vport.flags =MLX5_FLOW_DEST_VPORT_VHCA_ID ; dest.vport.vhca_id = port->esw_owner_vhca_id;
include/linux/mlx5/fs.h +1 -0
@@ -67,6 +67,7 @@ enum {MLX5_FLOW_TABLE_TERMINATION =BIT (2 ),MLX5_FLOW_TABLE_UNMANAGED =BIT (3 ),MLX5_FLOW_TABLE_OTHER_VPORT =BIT (4 ), + MLX5_FLOW_TABLE_UPLINK_VPORT = BIT(5), };#define LEFTOVERS_RULE_NUM 2
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 196/331 net/mlx5: Bridge, fix multicast packets sent to uplink
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Moshe Shemesh <[email protected]>
[ Upstream commit ec7cc38ef9f83553102e84c82536971a81630739 ]
To enable multicast packets which are offloaded in bridge multicast offload mode to be sent also to uplink, FTE bit uplink_hairpin_en should be set. Add this bit to FTE for the bridge multicast offload rules.
--- drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_mcast.c | 3 +++ drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c | 2 ++ include/linux/mlx5/fs.h | 1 + include/linux/mlx5/mlx5_ifc.h | 2 +- 4 files changed, 7 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_mcast.c +3 -0
@@ -83,6 +83,7 @@ mlx5_esw_bridge_mdb_flow_create(u16 esw_owner_vhca_id, struct mlx5_esw_bridge_md i++; } + rule_spec->flow_context.flags |=FLOW_CONTEXT_UPLINK_HAIRPIN_EN ; rule_spec->match_criteria_enable =MLX5_MATCH_OUTER_HEADERS ; dmac_v =MLX5_ADDR_OF (fte_match_param, rule_spec->match_value, outer_headers.dmac_47_16);ether_addr_copy (dmac_v, entry->key.addr);
@@ -587,6 +588,7 @@ mlx5_esw_bridge_mcast_vlan_flow_create(u16 vlan_proto, struct mlx5_esw_bridge_poif (!rule_spec)return ERR_PTR (-ENOMEM ); + rule_spec->flow_context.flags |=FLOW_CONTEXT_UPLINK_HAIRPIN_EN ; rule_spec->match_criteria_enable =MLX5_MATCH_OUTER_HEADERS ; flow_act.action |=MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT ;
@@ -662,6 +664,7 @@ mlx5_esw_bridge_mcast_fwd_flow_create(struct mlx5_esw_bridge_port *port) dest.vport.flags =MLX5_FLOW_DEST_VPORT_VHCA_ID ; dest.vport.vhca_id = port->esw_owner_vhca_id; } + rule_spec->flow_context.flags |=FLOW_CONTEXT_UPLINK_HAIRPIN_EN ; handle =mlx5_add_flow_rules (port->mcast.ft, rule_spec, &flow_act, &dest,1 );kvfree (rule_spec);
drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c +2 -0
@@ -566,6 +566,8 @@ static int mlx5_cmd_set_fte(struct mlx5_core_dev *dev, fte->flow_context.flow_tag);MLX5_SET (flow_context, in_flow_context, flow_source, fte->flow_context.flow_source); +MLX5_SET (flow_context, in_flow_context, uplink_hairpin_en, + !!(fte->flow_context.flags &FLOW_CONTEXT_UPLINK_HAIRPIN_EN ));MLX5_SET (flow_context, in_flow_context, extended_destination, extended_dest);
include/linux/mlx5/fs.h +1 -0
@@ -132,6 +132,7 @@ struct mlx5_flow_handle;enum {FLOW_CONTEXT_HAS_TAG =BIT (0 ), +FLOW_CONTEXT_UPLINK_HAIRPIN_EN =BIT (1 ), };struct mlx5_flow_context {
include/linux/mlx5/mlx5_ifc.h +1 -1
@@ -3536,7 +3536,7 @@ struct mlx5_ifc_flow_context_bits {u8 action[0x10 ];u8 extended_destination[0x1 ]; -u8 reserved_at_81[0x1 ]; +u8 uplink_hairpin_en[0x1 ];u8 flow_source[0x2 ];u8 encrypt_decrypt_type[0x4 ];u8 destination_list_size[0x18 ];
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 197/331 net/mlx5: DR, Use the right GVMI number for drop action
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Yevgeny Kliteynik <[email protected]>
[ Upstream commit 5665954293f13642f9c052ead83c1e9d8cff186f ]
When FW provides ICM addresses for drop RX/TX, the provided capability is 64 bits that contain its GVMI as well as the ICM address itself. In case of TX DROP this GVMI is different from the GVMI that the domain is operating on.
This patch fixes the action to use these GVMI IDs, as provided by FW.
--- drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c +1 -0
@@ -781,6 +781,7 @@ int mlx5dr_actions_build_ste_arr(struct mlx5dr_matcher *matcher,switch (action_type) {case DR_ACTION_TYP_DROP : attr.final_icm_addr = nic_dmn->drop_icm_addr; + attr.hit_gvmi = nic_dmn->drop_icm_addr >>48 ;break ;case DR_ACTION_TYP_FT : dest_action = action;
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 198/331 net/mlx5: DR, Cant go to uplink vport on RX rule
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Yevgeny Kliteynik <[email protected]>
[ Upstream commit 5b2a2523eeea5f03d39a9d1ff1bad2e9f8eb98d2 ]
Go-To-Vport action on RX is not allowed when the vport is uplink. In such case, the packet should be dropped.
--- .../mellanox/mlx5/core/steering/dr_action.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c +11 -5
@@ -867,11 +867,17 @@ int mlx5dr_actions_build_ste_arr(struct mlx5dr_matcher *matcher, action->sampler->tx_icm_addr;break ;case DR_ACTION_TYP_VPORT : - attr.hit_gvmi = action->vport->caps->vhca_gvmi; - dest_action = action; - attr.final_icm_addr = rx_rule ? - action->vport->caps->icm_address_rx : - action->vport->caps->icm_address_tx; +if (unlikely (rx_rule && action->vport->caps->num ==MLX5_VPORT_UPLINK )) { +/* can't go to uplink on RX rule - dropping instead */ + attr.final_icm_addr = nic_dmn->drop_icm_addr; + attr.hit_gvmi = nic_dmn->drop_icm_addr >>48 ; + }else { + attr.hit_gvmi = action->vport->caps->vhca_gvmi; + dest_action = action; + attr.final_icm_addr = rx_rule ? + action->vport->caps->icm_address_rx : + action->vport->caps->icm_address_tx; + }break ;case DR_ACTION_TYP_POP_VLAN : if (!rx_rule && !(dmn->ste_ctx->actions_caps &
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 199/331 net/mlx5: Use mlx5 device constant for selecting CQ period mode for ASO
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Rahul Rameshbabu <[email protected]>
[ Upstream commit 20cbf8cbb827094197f3b17db60d71449415db1e ]
mlx5 devices have specific constants for choosing the CQ period mode. These constants do not have to match the constants used by the kernel software API for DIM period mode selection.
--- drivers/net/ethernet/mellanox/mlx5/core/lib/aso.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/lib/aso.c +1 -1
@@ -98,7 +98,7 @@ static int create_aso_cq(struct mlx5_aso_cq *cq, void *cqc_data)mlx5_fill_page_frag_array (&cq->wq_ctrl.buf, (__be64 *)MLX5_ADDR_OF (create_cq_in, in, pas)); -MLX5_SET (cqc, cqc, cq_period_mode,DIM_CQ_PERIOD_MODE_START_FROM_EQE ); +MLX5_SET (cqc, cqc, cq_period_mode,MLX5_CQ_PERIOD_MODE_START_FROM_EQE );MLX5_SET (cqc, cqc, c_eqn_or_apu_element, eqn);MLX5_SET (cqc, cqc, uar_page, mdev->priv.uar->index); MLX5_SET(cqc, cqc, log_page_size, cq->wq_ctrl.buf.page_shift -
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 200/331 net/mlx5e: Allow software parsing when IPsec crypto is enabled
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Leon Romanovsky <[email protected]>
[ Upstream commit 20f5468a7988dedd94a57ba8acd65ebda6a59723 ]
All ConnectX devices have software parsing capability enabled, but it is more correct to set allow_swp only if capability exists, which for IPsec means that crypto offload is supported.
--- drivers/net/ethernet/mellanox/mlx5/core/en/params.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en/params.c +2 -2
@@ -1062,8 +1062,8 @@ void mlx5e_build_sq_param(struct mlx5_core_dev *mdev,void *wq =MLX5_ADDR_OF (sqc, sqc, wq);bool allow_swp; - allow_swp = -mlx5_geneve_tx_allowed (mdev) || !!mlx5_ipsec_device_caps (mdev); + allow_swp =mlx5_geneve_tx_allowed (mdev) || + (mlx5_ipsec_device_caps (mdev) &MLX5_IPSEC_CAP_CRYPTO );mlx5e_build_sq_param_common (mdev, param);MLX5_SET (wq, wq, log_wq_sz, params->log_sq_size);MLX5_SET (sqc, sqc, allow_swp, allow_swp);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 201/331 net/mlx5e: Ignore IPsec replay window values on sender side
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Leon Romanovsky <[email protected]>
[ Upstream commit 315a597f9bcfe7fe9980985031413457bee95510 ]
XFRM stack doesn't prevent from users to configure replay window in TX side and strongswan sets replay_window to be 1. It causes to failures in validation logic when trying to offload the SA.
Replay window is not relevant in TX side and should be ignored.
--- .../net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c +8 -2
@@ -336,12 +336,17 @@ void mlx5e_ipsec_build_accel_xfrm_attrs(struct mlx5e_ipsec_sa_entry *sa_entry,/* iv len */ aes_gcm->icv_len = x->aead->alg_icv_len; + attrs->dir = x->xso.dir; +/* esn */ if (x->props.flags &XFRM_STATE_ESN ) { attrs->replay_esn.trigger = true; attrs->replay_esn.esn = sa_entry->esn_state.esn; attrs->replay_esn.esn_msb = sa_entry->esn_state.esn_msb; attrs->replay_esn.overlap = sa_entry->esn_state.overlap; +if (attrs->dir ==XFRM_DEV_OFFLOAD_OUT ) + goto skip_replay_window; +switch (x->replay_esn->replay_window) {case 32 : attrs->replay_esn.replay_window =
@@ -365,7 +370,7 @@ void mlx5e_ipsec_build_accel_xfrm_attrs(struct mlx5e_ipsec_sa_entry *sa_entry, } } - attrs->dir = x->xso.dir; +skip_replay_window:/* spi */ attrs->spi =be32_to_cpu (x->id.spi);
@@ -501,7 +506,8 @@ static int mlx5e_xfrm_validate_state(struct mlx5_core_dev *mdev,return -EINVAL ; } -if (x->replay_esn && x->replay_esn->replay_window !=32 && +if (x->replay_esn && x->xso.dir ==XFRM_DEV_OFFLOAD_IN && + x->replay_esn->replay_window !=32 && x->replay_esn->replay_window !=64 && x->replay_esn->replay_window !=128 && x->replay_esn->replay_window !=256 ) {
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 202/331 net/mlx5e: fix a double-free in arfs_create_groups
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Zhipeng Lu <[email protected]>
[ Upstream commit 3c6d5189246f590e4e1f167991558bdb72a4738b ]
When in allocated by kvzalloc fails, arfs_create_groups will free ft->g and return an error. However, arfs_create_table, the only caller of arfs_create_groups, will hold this error and call to mlx5e_destroy_flow_table, in which the ft->g will be freed again.
--- .../net/ethernet/mellanox/mlx5/core/en_arfs.c | 26 +++++++++++-------- 1 file changed, 15 insertions(+), 11 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c +15 -11
@@ -254,11 +254,13 @@ static int arfs_create_groups(struct mlx5e_flow_table *ft, ft->g =kcalloc (MLX5E_ARFS_NUM_GROUPS ,sizeof (*ft->g),GFP_KERNEL ); - in =kvzalloc (inlen,GFP_KERNEL ); -if (!in || !ft->g) { -kfree (ft->g); -kvfree (in); +if (!ft->g)return -ENOMEM ; + + in =kvzalloc (inlen,GFP_KERNEL ); +if (!in) { + err = -ENOMEM ; + goto err_free_g; } mc =MLX5_ADDR_OF (create_flow_group_in, in, match_criteria);
@@ -278,7 +280,7 @@ static int arfs_create_groups(struct mlx5e_flow_table *ft,break ;default : err = -EINVAL ; - goto out; + goto err_free_in; }switch (type) {
@@ -300,7 +302,7 @@ static int arfs_create_groups(struct mlx5e_flow_table *ft,break ;default : err = -EINVAL ; - goto out; + goto err_free_in; }MLX5_SET_CFG (in, match_criteria_enable,MLX5_MATCH_OUTER_HEADERS );
@@ -309,7 +311,7 @@ static int arfs_create_groups(struct mlx5e_flow_table *ft,MLX5_SET_CFG (in, end_flow_index, ix -1 ); ft->g[ft->num_groups] =mlx5_create_flow_group (ft->t, in);if (IS_ERR (ft->g[ft->num_groups])) - goto err; + goto err_clean_group; ft->num_groups++;memset (in,0 , inlen);
@@ -318,18 +320,20 @@ static int arfs_create_groups(struct mlx5e_flow_table *ft,MLX5_SET_CFG (in, end_flow_index, ix -1 ); ft->g[ft->num_groups] =mlx5_create_flow_group (ft->t, in);if (IS_ERR (ft->g[ft->num_groups])) - goto err; + goto err_clean_group; ft->num_groups++;kvfree (in);return 0 ; -err: +err_clean_group: err =PTR_ERR (ft->g[ft->num_groups]); ft->g[ft->num_groups] =NULL ; -out: +err_free_in:kvfree (in); - +err_free_g: +kfree (ft->g); + ft->g =NULL ;return err; }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 203/331 net/mlx5e: fix a potential double-free in fs_any_create_groups
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Dinghao Liu <[email protected]>
[ Upstream commit aef855df7e1bbd5aa4484851561211500b22707e ]
When kcalloc() for ft->g succeeds but kvzalloc() for in fails, fs_any_create_groups() will free ft->g. However, its caller fs_any_create_table() will free ft->g again through calling mlx5e_destroy_flow_table(), which will lead to a double-free. Fix this by setting ft->g to NULL in fs_any_create_groups().
--- drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.c +1 -0
@@ -436,6 +436,7 @@ static int fs_any_create_groups(struct mlx5e_flow_table *ft) in =kvzalloc (inlen,GFP_KERNEL );if (!in || !ft->g) {kfree (ft->g); + ft->g =NULL ;kvfree (in);return -ENOMEM ; }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 204/331 rcu: Defer RCU kthreads wakeup when CPU is dying
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Frederic Weisbecker <[email protected]>
[ Upstream commit e787644caf7628ad3269c1fbd321c3255cf51710 ]
When the CPU goes idle for the last time during the CPU down hotplug process, RCU reports a final quiescent state for the current CPU. If this quiescent state propagates up to the top, some tasks may then be woken up to complete the grace period: the main grace period kthread and/or the expedited main workqueue (or kworker).
If those kthreads have a SCHED_FIFO policy, the wake up can indirectly arm the RT bandwith timer to the local offline CPU. Since this happens after hrtimers have been migrated at CPUHP_AP_HRTIMERS_DYING stage, the timer gets ignored. Therefore if the RCU kthreads are waiting for RT bandwidth to be available, they may never be actually scheduled.
This triggers TREE03 rcutorture hangs:
rcu: INFO: rcu_preempt self-detected stall on CPU rcu: 4-...!: (1 GPs behind) idle=9874/1/0x4000000000000000 softirq=0/0 fqs=20 rcuc=21071 jiffies(starved) rcu: (t=21035 jiffies g=938281 q=40787 ncpus=6) rcu: rcu_preempt kthread starved for 20964 jiffies! g938281 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. rcu: RCU grace-period kthread stack dump: task:rcu_preempt state:R running task stack:14896 pid:14 tgid:14 ppid:2 flags:0x00004000 Call Trace: <TASK> __schedule+0x2eb/0xa80 schedule+0x1f/0x90 schedule_timeout+0x163/0x270 ? __pfx_process_timeout+0x10/0x10 rcu_gp_fqs_loop+0x37c/0x5b0 ? __pfx_rcu_gp_kthread+0x10/0x10 rcu_gp_kthread+0x17c/0x200 kthread+0xde/0x110 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x40 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1b/0x30 </TASK>
The situation can't be solved with just unpinning the timer. The hrtimer infrastructure and the nohz heuristics involved in finding the best remote target for an unpinned timer would then also need to handle enqueues from an offline CPU in the most horrendous way.
So fix this on the RCU side instead and defer the wake up to an online CPU if it's too late for the local one.
--- kernel/rcu/tree.c | 34 +++++++++++++++++++++++++++++++++- kernel/rcu/tree_exp.h | 3 +-- 2 files changed, 34 insertions(+), 3 deletions(-)
kernel/rcu/tree.c +33 -1
@@ -1013,6 +1013,38 @@ static bool rcu_future_gp_cleanup(struct rcu_node *rnp)return needmore; } +static void swake_up_one_online_ipi (void *arg) +{ +struct swait_queue_head *wqh = arg; + +swake_up_one (wqh); +} + +static void swake_up_one_online (struct swait_queue_head *wqh) +{ +int cpu =get_cpu (); + +/* +* If called from rcutree_report_cpu_starting(), wake up +* is dangerous that late in the CPU-down hotplug process. The +* scheduler might queue an ignored hrtimer. Defer the wake up +* to an online CPU instead. +*/ +if (unlikely (cpu_is_offline (cpu))) { +int target; + + target =cpumask_any_and (housekeeping_cpumask (HK_TYPE_RCU ), + cpu_online_mask); + +smp_call_function_single (target, swake_up_one_online_ipi, + wqh,0 ); +put_cpu (); + }else { +put_cpu (); +swake_up_one (wqh); + } +} +/* * Awaken the grace-period kthread. Don't do a self-awaken (unless in an * interrupt or softirq handler, in which case we just might immediately
@@ -1037,7 +1069,7 @@ static void rcu_gp_kthread_wake(void)return ;WRITE_ONCE (rcu_state.gp_wake_time, jiffies);WRITE_ONCE (rcu_state.gp_wake_seq,READ_ONCE (rcu_state.gp_seq)); -swake_up_one (&rcu_state.gp_wq); +swake_up_one_online (&rcu_state.gp_wq); }/*
kernel/rcu/tree_exp.h +1 -2
@@ -173,7 +173,6 @@ static bool sync_rcu_exp_done_unlocked(struct rcu_node *rnp)return ret; } -/* * Report the exit from RCU read-side critical section for the last task * that queued itself during or before the current expedited preemptible-RCU
@@ -201,7 +200,7 @@ static void __rcu_report_exp_rnp(struct rcu_node *rnp,raw_spin_unlock_irqrestore_rcu_node (rnp, flags);if (wake) {smp_mb ();/* EGP done before wake_up(). */ -swake_up_one (&rcu_state.expedited_wq); +swake_up_one_online (&rcu_state.expedited_wq); }break ; }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 205/331 netfilter: nft_limit: reject configurations that cause integer overflow
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Florian Westphal <[email protected]>
[ Upstream commit c9d9eb9c53d37cdebbad56b91e40baf42d5a97aa ]
Reject bogus configs where internal token counter wraps around. This only occurs with very very large requests, such as 17gbyte/s.
Its better to reject this rather than having incorrect ratelimit.
--- net/netfilter/nft_limit.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)
net/netfilter/nft_limit.c +16 -7
@@ -58,17 +58,19 @@ static inline bool nft_limit_eval(struct nft_limit_priv *priv, u64 cost)static int nft_limit_init (struct nft_limit_priv *priv,const struct nlattr *const tb[],bool pkts) { +u64 unit, tokens, rate_with_burst;bool invert = false; -u64 unit, tokens;if (tb[NFTA_LIMIT_RATE ] ==NULL || tb[NFTA_LIMIT_UNIT ] ==NULL )return -EINVAL ; priv->rate =be64_to_cpu (nla_get_be64 (tb[NFTA_LIMIT_RATE ])); +if (priv->rate ==0 ) +return -EINVAL ; + unit =be64_to_cpu (nla_get_be64 (tb[NFTA_LIMIT_UNIT ])); - priv->nsecs = unit *NSEC_PER_SEC ; -if (priv->rate ==0 || priv->nsecs < unit) +if (check_mul_overflow (unit,NSEC_PER_SEC , &priv->nsecs))return -EOVERFLOW ;if (tb[NFTA_LIMIT_BURST ])
@@ -77,18 +79,25 @@ static int nft_limit_init(struct nft_limit_priv *priv,if (pkts && priv->burst ==0 ) priv->burst =NFT_LIMIT_PKT_BURST_DEFAULT ; -if (priv->rate + priv->burst < priv->rate) +if (check_add_overflow (priv->rate, priv->burst, &rate_with_burst))return -EOVERFLOW ;if (pkts) { - tokens =div64_u64 (priv->nsecs, priv->rate) * priv->burst; +u64 tmp =div64_u64 (priv->nsecs, priv->rate); + +if (check_mul_overflow (tmp, priv->burst, &tokens)) +return -EOVERFLOW ; }else { +u64 tmp; +/* The token bucket size limits the number of tokens can be * accumulated. tokens_max specifies the bucket size. * tokens_max = unit * (rate + burst) / rate. */ - tokens =div64_u64 (priv->nsecs * (priv->rate + priv->burst), - priv->rate); +if (check_mul_overflow (priv->nsecs, rate_with_burst, &tmp)) +return -EOVERFLOW ; + + tokens =div64_u64 (tmp, priv->rate); }if (tb[NFTA_LIMIT_FLAGS ]) {
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 206/331 netfilter: nf_tables: restrict anonymous set and map names to 16 bytes
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Florian Westphal <[email protected]>
[ Upstream commit b462579b2b86a8f5230543cadd3a4836be27baf7 ]
nftables has two types of sets/maps, one where userspace defines the name, and anonymous sets/maps, where userspace defines a template name.
For the latter, kernel requires presence of exactly one "%d". nftables uses "__set%d" and "__map%d" for this. The kernel will expand the format specifier and replaces it with the smallest unused number.
As-is, userspace could define a template name that allows to move the set name past the 256 bytes upperlimit (post-expansion).
I don't see how this could be a problem, but I would prefer if userspace cannot do this, so add a limit of 16 bytes for the '%d' template name.
16 bytes is the old total upper limit for set names that existed when nf_tables was merged initially.
--- net/netfilter/nf_tables_api.c | 4 ++++ 1 file changed, 4 insertions(+)
net/netfilter/nf_tables_api.c +4 -0
@@ -24,6 +24,7 @@#include <net/sock.h> #define NFT_MODULE_AUTOLOAD_LIMIT (MODULE_NAME_LEN - sizeof("nft-expr-255-")) +#define NFT_SET_MAX_ANONLEN 16unsigned int nf_tables_net_id__read_mostly ;
@@ -4351,6 +4352,9 @@ static int nf_tables_set_alloc_name(struct nft_ctx *ctx, struct nft_set *set,if (p[1 ] !='d' ||strchr (p +2 ,'%' ))return -EINVAL ; +if (strnlen (name,NFT_SET_MAX_ANONLEN ) >=NFT_SET_MAX_ANONLEN ) +return -EINVAL ; + inuse = (unsigned long *)get_zeroed_page (GFP_KERNEL );if (inuse ==NULL )return -ENOMEM ;
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 207/331 netfilter: nf_tables: validate NFPROTO_* family
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Pablo Neira Ayuso <[email protected]>
[ Upstream commit d0009effa8862c20a13af4cb7475d9771b905693 ]
Several expressions explicitly refer to NF_INET_* hook definitions from expr->ops->validate, however, family is not validated.
Bail out with EOPNOTSUPP in case they are used from unsupported families.
--- net/netfilter/nft_compat.c | 12 ++++++++++++ net/netfilter/nft_flow_offload.c | 5 +++++ net/netfilter/nft_nat.c | 5 +++++ net/netfilter/nft_rt.c | 5 +++++ net/netfilter/nft_socket.c | 5 +++++ net/netfilter/nft_synproxy.c | 7 +++++-- net/netfilter/nft_tproxy.c | 5 +++++ net/netfilter/nft_xfrm.c | 5 +++++ 8 files changed, 47 insertions(+), 2 deletions(-)
net/netfilter/nft_compat.c +12 -0
@@ -350,6 +350,12 @@ static int nft_target_validate(const struct nft_ctx *ctx,unsigned int hook_mask =0 ;int ret; +if (ctx->family !=NFPROTO_IPV4 && + ctx->family !=NFPROTO_IPV6 && + ctx->family !=NFPROTO_BRIDGE && + ctx->family !=NFPROTO_ARP ) +return -EOPNOTSUPP ; +if (nft_is_base_chain (ctx->chain)) {const struct nft_base_chain *basechain =nft_base_chain (ctx->chain);
@@ -595,6 +601,12 @@ static int nft_match_validate(const struct nft_ctx *ctx,unsigned int hook_mask =0 ;int ret; +if (ctx->family !=NFPROTO_IPV4 && + ctx->family !=NFPROTO_IPV6 && + ctx->family !=NFPROTO_BRIDGE && + ctx->family !=NFPROTO_ARP ) +return -EOPNOTSUPP ; +if (nft_is_base_chain (ctx->chain)) {const struct nft_base_chain *basechain =nft_base_chain (ctx->chain);
net/netfilter/nft_flow_offload.c +5 -0
@@ -384,6 +384,11 @@ static int nft_flow_offload_validate(const struct nft_ctx *ctx, {unsigned int hook_mask = (1 <<NF_INET_FORWARD ); +if (ctx->family !=NFPROTO_IPV4 && + ctx->family !=NFPROTO_IPV6 && + ctx->family !=NFPROTO_INET ) +return -EOPNOTSUPP ; +return nft_chain_validate_hooks (ctx->chain, hook_mask); }
net/netfilter/nft_nat.c +5 -0
@@ -143,6 +143,11 @@ static int nft_nat_validate(const struct nft_ctx *ctx,struct nft_nat *priv =nft_expr_priv (expr);int err; +if (ctx->family !=NFPROTO_IPV4 && + ctx->family !=NFPROTO_IPV6 && + ctx->family !=NFPROTO_INET ) +return -EOPNOTSUPP ; + err =nft_chain_validate_dependency (ctx->chain,NFT_CHAIN_T_NAT );if (err <0 )return err;
net/netfilter/nft_rt.c +5 -0
@@ -166,6 +166,11 @@ static int nft_rt_validate(const struct nft_ctx *ctx, const struct nft_expr *expconst struct nft_rt *priv =nft_expr_priv (expr);unsigned int hooks; +if (ctx->family !=NFPROTO_IPV4 && + ctx->family !=NFPROTO_IPV6 && + ctx->family !=NFPROTO_INET ) +return -EOPNOTSUPP ; +switch (priv->key) {case NFT_RT_NEXTHOP4 :case NFT_RT_NEXTHOP6 :
net/netfilter/nft_socket.c +5 -0
@@ -242,6 +242,11 @@ static int nft_socket_validate(const struct nft_ctx *ctx,const struct nft_expr *expr, const struct nft_data **data) { + if (ctx->family != NFPROTO_IPV4 && + ctx->family != NFPROTO_IPV6 && + ctx->family != NFPROTO_INET) + return -EOPNOTSUPP; + return nft_chain_validate_hooks(ctx->chain, (1 << NF_INET_PRE_ROUTING) | (1 << NF_INET_LOCAL_IN) |
net/netfilter/nft_synproxy.c +5 -2
@@ -186,7 +186,6 @@ static int nft_synproxy_do_init(const struct nft_ctx *ctx,break ;#endif case NFPROTO_INET : -case NFPROTO_BRIDGE : err =nf_synproxy_ipv4_init (snet, ctx->net);if (err) goto nf_ct_failure;
@@ -219,7 +218,6 @@ static void nft_synproxy_do_destroy(const struct nft_ctx *ctx)break ;#endif case NFPROTO_INET : -case NFPROTO_BRIDGE :nf_synproxy_ipv4_fini (snet, ctx->net);nf_synproxy_ipv6_fini (snet, ctx->net);break ;
@@ -253,6 +251,11 @@ static int nft_synproxy_validate(const struct nft_ctx *ctx,const struct nft_expr *expr, const struct nft_data **data) { + if (ctx->family != NFPROTO_IPV4 && + ctx->family != NFPROTO_IPV6 && + ctx->family != NFPROTO_INET) + return -EOPNOTSUPP; + return nft_chain_validate_hooks(ctx->chain, (1 << NF_INET_LOCAL_IN) | (1 << NF_INET_FORWARD)); }
net/netfilter/nft_tproxy.c +5 -0
@@ -316,6 +316,11 @@ static int nft_tproxy_validate(const struct nft_ctx *ctx,const struct nft_expr *expr, const struct nft_data **data) { + if (ctx->family != NFPROTO_IPV4 && + ctx->family != NFPROTO_IPV6 && + ctx->family != NFPROTO_INET) + return -EOPNOTSUPP; + return nft_chain_validate_hooks(ctx->chain, 1 << NF_INET_PRE_ROUTING); }
net/netfilter/nft_xfrm.c +5 -0
@@ -235,6 +235,11 @@ static int nft_xfrm_validate(const struct nft_ctx *ctx, const struct nft_expr *econst struct nft_xfrm *priv =nft_expr_priv (expr);unsigned int hooks; +if (ctx->family !=NFPROTO_IPV4 && + ctx->family !=NFPROTO_IPV6 && + ctx->family !=NFPROTO_INET ) +return -EOPNOTSUPP ; +switch (priv->dir) {case XFRM_POLICY_IN : hooks = (1 << NF_INET_FORWARD) |
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 208/331 net: stmmac: Wait a bit for the reset to take effect
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Bernd Edlinger <[email protected]>
[ Upstream commit a5f5eee282a0aae80227697e1d9c811b1726d31d ]
otherwise the synopsys_id value may be read out wrong, because the GMAC_VERSION register might still be in reset state, for at least 1 us after the reset is de-asserted.
Add a wait for 10 us before continuing to be on the safe side.
Quoted 1 line
From what have you got that delay value?
Just try and error, with very old linux versions and old gcc versions the synopsys_id was read out correctly most of the time (but not always), with recent linux versions and recnet gcc versions it was read out wrongly most of the time, but again not always.
I don't have access to the VHDL code in question, so I cannot tell why it takes so long to get the correct values, I also do not have more than a few hardware samples, so I cannot tell how long this timeout must be in worst case.
Experimentally I can tell that the register is read several times as zero immediately after the reset is de-asserted, also adding several no-ops is not enough, adding a printk is enough, also udelay(1) seems to be enough but I tried that not very often, and I have not access to many hardware samples to be 100% sure about the necessary delay. And since the udelay here is only executed once per device instance, it seems acceptable to delay the boot for 10 us.
BTW: my hardware's synopsys id is 0x37.
--- drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +3 -0
@@ -7441,6 +7441,9 @@ int stmmac_dvr_probe(struct device *device,dev_err (priv->device,"unable to bring out of ahb reset: %pe\n" ,ERR_PTR (ret)); +/* Wait a bit for the reset to take effect */ +udelay (10 ); +/* Init MAC and get the capabilities */ ret =stmmac_hw_init (priv);if (ret)
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 209/331 net: mvpp2: clear BM pool before initialization
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Jenishkumar Maheshbhai Patel <[email protected]>
[ Upstream commit 9f538b415db862e74b8c5d3abbccfc1b2b6caa38 ]
Register value persist after booting the kernel using kexec which results in kernel panic. Thus clear the BM pool registers before initialisation to fix the issue.
--- .../net/ethernet/marvell/mvpp2/mvpp2_main.c | 27 ++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +26 -1
@@ -614,12 +614,38 @@ static void mvpp23_bm_set_8pool_mode(struct mvpp2 *priv)mvpp2_write (priv,MVPP22_BM_POOL_BASE_ADDR_HIGH_REG , val); } +/* Cleanup pool before actual initialization in the OS */ +static void mvpp2_bm_pool_cleanup (struct mvpp2 *priv,int pool_id) +{ +unsigned int thread =mvpp2_cpu_to_thread (priv,get_cpu ()); +u32 val; +int i; + +/* Drain the BM from all possible residues left by firmware */ +for (i =0 ; i <MVPP2_BM_POOL_SIZE_MAX ; i++) +mvpp2_thread_read (priv, thread,MVPP2_BM_PHY_ALLOC_REG (pool_id)); + +put_cpu (); + +/* Stop the BM pool */ + val =mvpp2_read (priv,MVPP2_BM_POOL_CTRL_REG (pool_id)); + val |=MVPP2_BM_STOP_MASK ; +mvpp2_write (priv,MVPP2_BM_POOL_CTRL_REG (pool_id), val); +} +static int mvpp2_bm_init (struct device *dev,struct mvpp2 *priv) {enum dma_data_direction dma_dir =DMA_FROM_DEVICE ;int i, err, poolnum =MVPP2_BM_POOLS_NUM ;struct mvpp2_port *port; +if (priv->percpu_pools) + poolnum =mvpp2_get_nrxqs (priv) *2 ; + +/* Clean up the pool state in case it contains stale state */ +for (i =0 ; i < poolnum; i++) +mvpp2_bm_pool_cleanup (priv, i); +if (priv->percpu_pools) {for (i =0 ; i < priv->port_count; i++) { port = priv->port_list[i];
@@ -629,7 +655,6 @@ static int mvpp2_bm_init(struct device *dev, struct mvpp2 *priv) } } - poolnum =mvpp2_get_nrxqs (priv) *2 ;for (i =0 ; i < poolnum; i++) {/* the pool in use */ int pn = i / (poolnum /2 );
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 210/331 selftests: net: fix rps_default_mask with >32 CPUs
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Jakub Kicinski <[email protected]>
[ Upstream commit 0719b5338a0cbe80d1637a5fb03d8141b5bfc7a1 ]
If there is more than 32 cpus the bitmask will start to contain commas, leading to:
./rps_default_mask.sh: line 36: [: 00000000,00000000: integer expression expected
Remove the commas, bash doesn't interpret leading zeroes as oct so that should be good enough. Switch to bash, Simon reports that not all shells support this type of substitution.
--- tools/testing/selftests/net/rps_default_mask.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
tools/testing/selftests/net/rps_default_mask.sh +5 -1
@@ -1,4 +1,4 @@ -#!/bin/sh +#!/bin/bash # SPDX-License-Identifier: GPL-2.0 readonly ksft_skip=4
@@ -33,6 +33,10 @@ chk_rps() { rps_mask=$($cmd /sys/class/net/$dev_name/queues/rx-0/rps_cpus) printf "%-60s" "$msg" + + # In case there is more than 32 CPUs we need to remove commas from masks + rps_mask=${rps_mask//,} + expected_rps_mask=${expected_rps_mask//,} if [ $rps_mask -eq $expected_rps_mask ]; then echo "[ ok ]" else
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 211/331 selftests: netdevsim: fix the udp_tunnel_nic test
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Jakub Kicinski <[email protected]>
[ Upstream commit 0879020a7817e7ce636372c016b4528f541c9f4d ]
This test is missing a whole bunch of checks for interface renaming and one ifup. Presumably it was only used on a system with renaming disabled and NetworkManager running.
--- .../selftests/drivers/net/netdevsim/udp_tunnel_nic.sh | 9 +++++++++ 1 file changed, 9 insertions(+)
tools/testing/selftests/drivers/net/netdevsim/udp_tunnel_nic.sh +9 -0
@@ -269,6 +269,7 @@ for port in 0 1; do echo 1 > $NSIM_DEV_SYS/new_port fi NSIM_NETDEV=`get_netdev_name old_netdevs` + ifconfig $NSIM_NETDEV up msg="new NIC device created" exp0=( 0 0 0 0 )
@@ -430,6 +431,7 @@ for port in 0 1; do fi echo $port > $NSIM_DEV_SYS/new_port + NSIM_NETDEV=`get_netdev_name old_netdevs` ifconfig $NSIM_NETDEV up overflow_table0 "overflow NIC table"
@@ -487,6 +489,7 @@ for port in 0 1; do fi echo $port > $NSIM_DEV_SYS/new_port + NSIM_NETDEV=`get_netdev_name old_netdevs` ifconfig $NSIM_NETDEV up overflow_table0 "overflow NIC table"
@@ -543,6 +546,7 @@ for port in 0 1; do fi echo $port > $NSIM_DEV_SYS/new_port + NSIM_NETDEV=`get_netdev_name old_netdevs` ifconfig $NSIM_NETDEV up overflow_table0 "destroy NIC"
@@ -572,6 +576,7 @@ for port in 0 1; do fi echo $port > $NSIM_DEV_SYS/new_port + NSIM_NETDEV=`get_netdev_name old_netdevs` ifconfig $NSIM_NETDEV up msg="create VxLANs v6"
@@ -632,6 +637,7 @@ for port in 0 1; do fi echo $port > $NSIM_DEV_SYS/new_port + NSIM_NETDEV=`get_netdev_name old_netdevs` ifconfig $NSIM_NETDEV up echo 110 > $NSIM_DEV_DFS/ports/$port/udp_ports_inject_error
@@ -687,6 +693,7 @@ for port in 0 1; do fi echo $port > $NSIM_DEV_SYS/new_port + NSIM_NETDEV=`get_netdev_name old_netdevs` ifconfig $NSIM_NETDEV up msg="create VxLANs v6"
@@ -746,6 +753,7 @@ for port in 0 1; do fi echo $port > $NSIM_DEV_SYS/new_port + NSIM_NETDEV=`get_netdev_name old_netdevs` ifconfig $NSIM_NETDEV up msg="create VxLANs v6"
@@ -876,6 +884,7 @@ msg="re-add a port" echo 2 > $NSIM_DEV_SYS/del_port echo 2 > $NSIM_DEV_SYS/new_port +NSIM_NETDEV=`get_netdev_name old_netdevs` check_tables msg="replace VxLAN in overflow table"
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 212/331 xsk: recycle buffer in case Rx queue was full
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Maciej Fijalkowski <[email protected]>
[ Upstream commit 269009893146c495f41e9572dd9319e787c2eba9 ]
Add missing xsk_buff_free() call when __xsk_rcv_zc() failed to produce descriptor to XSK Rx queue.
--- net/xdp/xsk.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
net/xdp/xsk.c +8 -4
@@ -166,8 +166,10 @@ static int xsk_rcv_zc(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len) contd =XDP_PKT_CONTD ; err =__xsk_rcv_zc (xs, xskb, len, contd); -if (err ||likely (!frags)) - goto out; +if (err) + goto err; +if (likely (!frags)) +return 0 ; xskb_list = &xskb->pool->xskb_list;list_for_each_entry_safe (pos, tmp, xskb_list, xskb_list_node) {
@@ -176,11 +178,13 @@ static int xsk_rcv_zc(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len) len = pos->xdp.data_end - pos->xdp.data; err =__xsk_rcv_zc (xs, pos, len, contd);if (err) -return err; + goto err;list_del (&pos->xskb_list_node); } -out: +return 0 ; +err: +xsk_buff_free (xdp);return err; }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 213/331 xsk: make xsk_buff_pool responsible for clearing xdp_buff::flags
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Maciej Fijalkowski <[email protected]>
[ Upstream commit f7f6aa8e24383fbb11ac55942e66da9660110f80 ]
XDP multi-buffer support introduced XDP_FLAGS_HAS_FRAGS flag that is used by drivers to notify data path whether xdp_buff contains fragments or not. Data path looks up mentioned flag on first buffer that occupies the linear part of xdp_buff, so drivers only modify it there. This is sufficient for SKB and XDP_DRV modes as usually xdp_buff is allocated on stack or it resides within struct representing driver's queue and fragments are carried via skb_frag_t structs. IOW, we are dealing with only one xdp_buff.
ZC mode though relies on list of xdp_buff structs that is carried via xsk_buff_pool::xskb_list, so ZC data path has to make sure that fragments do *not* have XDP_FLAGS_HAS_FRAGS set. Otherwise, xsk_buff_free() could misbehave if it would be executed against xdp_buff that carries a frag with XDP_FLAGS_HAS_FRAGS flag set. Such scenario can take place when within supplied XDP program bpf_xdp_adjust_tail() is used with negative offset that would in turn release the tail fragment from multi-buffer frame.
Calling xsk_buff_free() on tail fragment with XDP_FLAGS_HAS_FRAGS would result in releasing all the nodes from xskb_list that were produced by driver before XDP program execution, which is not what is intended - only tail fragment should be deleted from xskb_list and then it should be put onto xsk_buff_pool::free_list. Such multi-buffer frame will never make it up to user space, so from AF_XDP application POV there would be no traffic running, however due to free_list getting constantly new nodes, driver will be able to feed HW Rx queue with recycled buffers. Bottom line is that instead of traffic being redirected to user space, it would be continuously dropped.
To fix this, let us clear the mentioned flag on xsk_buff_pool side during xdp_buff initialization, which is what should have been done right from the start of XSK multi-buffer support.
--- drivers/net/ethernet/intel/i40e/i40e_xsk.c | 1 - drivers/net/ethernet/intel/ice/ice_xsk.c | 1 - include/net/xdp_sock_drv.h | 1 + net/xdp/xsk_buff_pool.c | 1 + 4 files changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/intel/i40e/i40e_xsk.c +0 -1
@@ -503,7 +503,6 @@ int i40e_clean_rx_irq_zc(struct i40e_ring *rx_ring, int budget) xdp_res =i40e_run_xdp_zc (rx_ring, first, xdp_prog);i40e_handle_xdp_result_zc (rx_ring, first, rx_desc, &rx_packets, &rx_bytes, xdp_res, &failure); - first->flags =0 ; next_to_clean = next_to_process;if (failure)break ;
drivers/net/ethernet/intel/ice/ice_xsk.c +0 -1
@@ -897,7 +897,6 @@ int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, int budget)if (!first) { first = xdp; -xdp_buff_clear_frags_flag (first); }else if (ice_add_xsk_frag (rx_ring, first, xdp, size)) {break ; }
include/net/xdp_sock_drv.h +1 -0
@@ -152,6 +152,7 @@ static inline void xsk_buff_set_size(struct xdp_buff *xdp, u32 size) xdp->data = xdp->data_hard_start +XDP_PACKET_HEADROOM ; xdp->data_meta = xdp->data; xdp->data_end = xdp->data + size; + xdp->flags =0 ; } static inline dma_addr_t xsk_buff_raw_get_dma(struct xsk_buff_pool *pool,
net/xdp/xsk_buff_pool.c +1 -0
@@ -541,6 +541,7 @@ struct xdp_buff *xp_alloc(struct xsk_buff_pool *pool) xskb->xdp.data = xskb->xdp.data_hard_start +XDP_PACKET_HEADROOM ; xskb->xdp.data_meta = xskb->xdp.data; + xskb->xdp.flags =0 ;if (pool->dma_need_sync) { dma_sync_single_range_for_device(pool->dev, xskb->dma, 0,
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 214/331 bpf: Propagate modified uaddrlen from cgroup sockaddr programs
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Daan De Meyer <[email protected]>
[ Upstream commit fefba7d1ae198dcbf8b3b432de46a4e29f8dbd8c ]
As prep for adding unix socket support to the cgroup sockaddr hooks, let's propagate the sockaddr length back to the caller after running a bpf cgroup sockaddr hook program. While not important for AF_INET or AF_INET6, the sockaddr length is important when working with AF_UNIX sockaddrs as the size of the sockaddr cannot be determined just from the address family or the sockaddr's contents.
__cgroup_bpf_run_filter_sock_addr() is modified to take the uaddrlen as an input/output argument. After running the program, the modified sockaddr length is stored in the uaddrlen pointer.
Stable-dep-of: c5114710c8ce ("xsk: fix usage of multi-buffer BPF helpers for ZC XDP")
--- include/linux/bpf-cgroup.h | 73 +++++++++++++++++++------------------- include/linux/filter.h | 1 + kernel/bpf/cgroup.c | 17 +++++++-- net/ipv4/af_inet.c | 7 ++-- net/ipv4/ping.c | 2 +- net/ipv4/tcp_ipv4.c | 2 +- net/ipv4/udp.c | 9 +++-- net/ipv6/af_inet6.c | 9 ++--- net/ipv6/ping.c | 2 +- net/ipv6/tcp_ipv6.c | 2 +- net/ipv6/udp.c | 6 ++-- 11 files changed, 76 insertions(+), 54 deletions(-)
include/linux/bpf-cgroup.h +37 -36
@@ -120,6 +120,7 @@ int __cgroup_bpf_run_filter_sk(struct sock *sk,int __cgroup_bpf_run_filter_sock_addr (struct sock *sk,struct sockaddr *uaddr, +int *uaddrlen,enum cgroup_bpf_attach_type atype,void *t_ctx,u32 *flags);
@@ -230,22 +231,22 @@ static inline bool cgroup_bpf_sock_enabled(struct sock *sk,#define BPF_CGROUP_RUN_PROG_INET6_POST_BIND (sk) \ BPF_CGROUP_RUN_SK_PROG(sk, CGROUP_INET6_POST_BIND) -#define BPF_CGROUP_RUN_SA_PROG (sk, uaddr, atype) \ +#define BPF_CGROUP_RUN_SA_PROG (sk, uaddr, uaddrlen, atype) \ ({ \ int __ret = 0; \ if (cgroup_bpf_enabled(atype)) \ - __ret = __cgroup_bpf_run_filter_sock_addr(sk, uaddr, atype, \ - NULL, NULL); \ + __ret = __cgroup_bpf_run_filter_sock_addr(sk, uaddr, uaddrlen, \ + atype, NULL, NULL); \ __ret; \ }) -#define BPF_CGROUP_RUN_SA_PROG_LOCK (sk, uaddr, atype, t_ctx) \ +#define BPF_CGROUP_RUN_SA_PROG_LOCK (sk, uaddr, uaddrlen, atype, t_ctx) \ ({ \ int __ret = 0; \ if (cgroup_bpf_enabled(atype)) { \ lock_sock(sk); \ - __ret = __cgroup_bpf_run_filter_sock_addr(sk, uaddr, atype, \ - t_ctx, NULL); \ + __ret = __cgroup_bpf_run_filter_sock_addr(sk, uaddr, uaddrlen, \ + atype, t_ctx, NULL); \ release_sock(sk); \ } \ __ret; \
@@ -256,14 +257,14 @@ static inline bool cgroup_bpf_sock_enabled(struct sock *sk,* (at bit position 0) is to indicate CAP_NET_BIND_SERVICE capability check * should be bypassed (BPF_RET_BIND_NO_CAP_NET_BIND_SERVICE). */ -#define BPF_CGROUP_RUN_PROG_INET_BIND_LOCK (sk, uaddr, atype, bind_flags) \ +#define BPF_CGROUP_RUN_PROG_INET_BIND_LOCK (sk, uaddr, uaddrlen, atype, bind_flags) \ ({ \ u32 __flags = 0; \ int __ret = 0; \ if (cgroup_bpf_enabled(atype)) { \ lock_sock(sk); \ - __ret = __cgroup_bpf_run_filter_sock_addr(sk, uaddr, atype, \ - NULL, &__flags); \ + __ret = __cgroup_bpf_run_filter_sock_addr(sk, uaddr, uaddrlen, \ + atype, NULL, &__flags); \ release_sock(sk); \ if (__flags & BPF_RET_BIND_NO_CAP_NET_BIND_SERVICE) \ *bind_flags |= BIND_NO_CAP_NET_BIND_SERVICE; \
@@ -276,29 +277,29 @@ static inline bool cgroup_bpf_sock_enabled(struct sock *sk, cgroup_bpf_enabled(CGROUP_INET6_CONNECT)) && \ (sk)->sk_prot->pre_connect) -#define BPF_CGROUP_RUN_PROG_INET4_CONNECT (sk, uaddr) \ - BPF_CGROUP_RUN_SA_PROG(sk, uaddr, CGROUP_INET4_CONNECT) +#define BPF_CGROUP_RUN_PROG_INET4_CONNECT (sk, uaddr, uaddrlen) \ + BPF_CGROUP_RUN_SA_PROG(sk, uaddr, uaddrlen, CGROUP_INET4_CONNECT) -#define BPF_CGROUP_RUN_PROG_INET6_CONNECT (sk, uaddr) \ - BPF_CGROUP_RUN_SA_PROG(sk, uaddr, CGROUP_INET6_CONNECT) +#define BPF_CGROUP_RUN_PROG_INET6_CONNECT (sk, uaddr, uaddrlen) \ + BPF_CGROUP_RUN_SA_PROG(sk, uaddr, uaddrlen, CGROUP_INET6_CONNECT) -#define BPF_CGROUP_RUN_PROG_INET4_CONNECT_LOCK (sk, uaddr) \ - BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, CGROUP_INET4_CONNECT, NULL) +#define BPF_CGROUP_RUN_PROG_INET4_CONNECT_LOCK (sk, uaddr, uaddrlen) \ + BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, uaddrlen, CGROUP_INET4_CONNECT, NULL) -#define BPF_CGROUP_RUN_PROG_INET6_CONNECT_LOCK (sk, uaddr) \ - BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, CGROUP_INET6_CONNECT, NULL) +#define BPF_CGROUP_RUN_PROG_INET6_CONNECT_LOCK (sk, uaddr, uaddrlen) \ + BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, uaddrlen, CGROUP_INET6_CONNECT, NULL) -#define BPF_CGROUP_RUN_PROG_UDP4_SENDMSG_LOCK (sk, uaddr, t_ctx) \ - BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, CGROUP_UDP4_SENDMSG, t_ctx) +#define BPF_CGROUP_RUN_PROG_UDP4_SENDMSG_LOCK (sk, uaddr, uaddrlen, t_ctx) \ + BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, uaddrlen, CGROUP_UDP4_SENDMSG, t_ctx) -#define BPF_CGROUP_RUN_PROG_UDP6_SENDMSG_LOCK (sk, uaddr, t_ctx) \ - BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, CGROUP_UDP6_SENDMSG, t_ctx) +#define BPF_CGROUP_RUN_PROG_UDP6_SENDMSG_LOCK (sk, uaddr, uaddrlen, t_ctx) \ + BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, uaddrlen, CGROUP_UDP6_SENDMSG, t_ctx) -#define BPF_CGROUP_RUN_PROG_UDP4_RECVMSG_LOCK (sk, uaddr) \ - BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, CGROUP_UDP4_RECVMSG, NULL) +#define BPF_CGROUP_RUN_PROG_UDP4_RECVMSG_LOCK (sk, uaddr, uaddrlen) \ + BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, uaddrlen, CGROUP_UDP4_RECVMSG, NULL) -#define BPF_CGROUP_RUN_PROG_UDP6_RECVMSG_LOCK (sk, uaddr) \ - BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, CGROUP_UDP6_RECVMSG, NULL) +#define BPF_CGROUP_RUN_PROG_UDP6_RECVMSG_LOCK (sk, uaddr, uaddrlen) \ + BPF_CGROUP_RUN_SA_PROG_LOCK(sk, uaddr, uaddrlen, CGROUP_UDP6_RECVMSG, NULL)/* The SOCK_OPS"_SK" macro should be used when sock_ops->sk is not a * fullsock and its parent fullsock cannot be traced by
@@ -477,24 +478,24 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map, }#define cgroup_bpf_enabled (atype) (0) -#define BPF_CGROUP_RUN_SA_PROG_LOCK (sk, uaddr, atype, t_ctx) ({ 0; }) -#define BPF_CGROUP_RUN_SA_PROG (sk, uaddr, atype) ({ 0; }) +#define BPF_CGROUP_RUN_SA_PROG_LOCK (sk, uaddr, uaddrlen, atype, t_ctx) ({ 0; }) +#define BPF_CGROUP_RUN_SA_PROG (sk, uaddr, uaddrlen, atype) ({ 0; })#define BPF_CGROUP_PRE_CONNECT_ENABLED (sk) (0)#define BPF_CGROUP_RUN_PROG_INET_INGRESS (sk,skb) ({ 0; })#define BPF_CGROUP_RUN_PROG_INET_EGRESS (sk,skb) ({ 0; })#define BPF_CGROUP_RUN_PROG_INET_SOCK (sk) ({ 0; })#define BPF_CGROUP_RUN_PROG_INET_SOCK_RELEASE (sk) ({ 0; }) -#define BPF_CGROUP_RUN_PROG_INET_BIND_LOCK (sk, uaddr, atype, flags) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_INET_BIND_LOCK (sk, uaddr, uaddrlen, atype, flags) ({ 0; })#define BPF_CGROUP_RUN_PROG_INET4_POST_BIND (sk) ({ 0; })#define BPF_CGROUP_RUN_PROG_INET6_POST_BIND (sk) ({ 0; }) -#define BPF_CGROUP_RUN_PROG_INET4_CONNECT (sk, uaddr) ({ 0; }) -#define BPF_CGROUP_RUN_PROG_INET4_CONNECT_LOCK (sk, uaddr) ({ 0; }) -#define BPF_CGROUP_RUN_PROG_INET6_CONNECT (sk, uaddr) ({ 0; }) -#define BPF_CGROUP_RUN_PROG_INET6_CONNECT_LOCK (sk, uaddr) ({ 0; }) -#define BPF_CGROUP_RUN_PROG_UDP4_SENDMSG_LOCK (sk, uaddr, t_ctx) ({ 0; }) -#define BPF_CGROUP_RUN_PROG_UDP6_SENDMSG_LOCK (sk, uaddr, t_ctx) ({ 0; }) -#define BPF_CGROUP_RUN_PROG_UDP4_RECVMSG_LOCK (sk, uaddr) ({ 0; }) -#define BPF_CGROUP_RUN_PROG_UDP6_RECVMSG_LOCK (sk, uaddr) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_INET4_CONNECT (sk, uaddr, uaddrlen) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_INET4_CONNECT_LOCK (sk, uaddr, uaddrlen) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_INET6_CONNECT (sk, uaddr, uaddrlen) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_INET6_CONNECT_LOCK (sk, uaddr, uaddrlen) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_UDP4_SENDMSG_LOCK (sk, uaddr, uaddrlen, t_ctx) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_UDP6_SENDMSG_LOCK (sk, uaddr, uaddrlen, t_ctx) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_UDP4_RECVMSG_LOCK (sk, uaddr, uaddrlen) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_UDP6_RECVMSG_LOCK (sk, uaddr, uaddrlen) ({ 0; })#define BPF_CGROUP_RUN_PROG_SOCK_OPS (sock_ops) ({ 0; })#define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP (atype, major, minor, access) ({ 0; })#define BPF_CGROUP_RUN_PROG_SYSCTL (head,table,write,buf,count,pos) ({ 0; })
include/linux/filter.h +1 -0
@@ -1285,6 +1285,7 @@ struct bpf_sock_addr_kern {*/ u64 tmp_reg;void *t_ctx;/* Attach type specific context. */ +u32 uaddrlen; };struct bpf_sock_ops_kern {
kernel/bpf/cgroup.c +15 -2
@@ -1450,6 +1450,9 @@ EXPORT_SYMBOL(__cgroup_bpf_run_filter_sk);* provided by user sockaddr * @sk: sock struct that will use sockaddr * @uaddr: sockaddr struct provided by user +* @uaddrlen: Pointer to the size of the sockaddr struct provided by user. It is +* read-only for AF_INET[6] uaddr but can be modified for AF_UNIX +* uaddr. * @atype: The type of program to be executed * @t_ctx: Pointer to attach type specific context * @flags: Pointer to u32 which contains higher bits of BPF program
@@ -1462,6 +1465,7 @@ EXPORT_SYMBOL(__cgroup_bpf_run_filter_sk);*/ int __cgroup_bpf_run_filter_sock_addr (struct sock *sk,struct sockaddr *uaddr, +int *uaddrlen,enum cgroup_bpf_attach_type atype,void *t_ctx,u32 *flags)
@@ -1473,6 +1477,7 @@ int __cgroup_bpf_run_filter_sock_addr(struct sock *sk, };struct sockaddr_storage unspec;struct cgroup *cgrp; +int ret;/* Check socket family since not all sockets represent network * endpoint (e.g. AF_UNIX).
@@ -1483,11 +1488,19 @@ int __cgroup_bpf_run_filter_sock_addr(struct sock *sk,if (!ctx.uaddr) {memset (&unspec,0 ,sizeof (unspec)); ctx.uaddr = (struct sockaddr *)&unspec; + ctx.uaddrlen =0 ; + }else { + ctx.uaddrlen = *uaddrlen; } cgrp =sock_cgroup_ptr (&sk->sk_cgrp_data); -return bpf_prog_run_array_cg (&cgrp->bpf, atype, &ctx, bpf_prog_run, -0 , flags); + ret =bpf_prog_run_array_cg (&cgrp->bpf, atype, &ctx, bpf_prog_run, +0 , flags); + +if (!ret && uaddr) + *uaddrlen = ctx.uaddrlen; + +return ret; }EXPORT_SYMBOL (__cgroup_bpf_run_filter_sock_addr);
net/ipv4/af_inet.c +4 -3
@@ -455,7 +455,7 @@ int inet_bind_sk(struct sock *sk, struct sockaddr *uaddr, int addr_len)/* BPF prog is run before any checks are done so that if the prog * changes context in a wrong way it will be caught. */ - err =BPF_CGROUP_RUN_PROG_INET_BIND_LOCK (sk, uaddr, + err =BPF_CGROUP_RUN_PROG_INET_BIND_LOCK (sk, uaddr, &addr_len,CGROUP_INET4_BIND , &flags);if (err)return err;
@@ -797,6 +797,7 @@ int inet_getname(struct socket *sock, struct sockaddr *uaddr,struct sock *sk = sock->sk;struct inet_sock *inet =inet_sk (sk); DECLARE_SOCKADDR(struct sockaddr_in *, sin, uaddr); +int sin_addr_len =sizeof (*sin); sin->sin_family =AF_INET ;lock_sock (sk);
@@ -809,7 +810,7 @@ int inet_getname(struct socket *sock, struct sockaddr *uaddr, } sin->sin_port = inet->inet_dport; sin->sin_addr.s_addr = inet->inet_daddr; -BPF_CGROUP_RUN_SA_PROG (sk, (struct sockaddr *)sin, +BPF_CGROUP_RUN_SA_PROG (sk, (struct sockaddr *)sin, &sin_addr_len,CGROUP_INET4_GETPEERNAME ); }else {__be32 addr = inet->inet_rcv_saddr;
@@ -817,7 +818,7 @@ int inet_getname(struct socket *sock, struct sockaddr *uaddr, addr = inet->inet_saddr; sin->sin_port = inet->inet_sport; sin->sin_addr.s_addr = addr; -BPF_CGROUP_RUN_SA_PROG (sk, (struct sockaddr *)sin, +BPF_CGROUP_RUN_SA_PROG (sk, (struct sockaddr *)sin, &sin_addr_len,CGROUP_INET4_GETSOCKNAME ); }release_sock (sk);
net/ipv4/ping.c +1 -1
@@ -301,7 +301,7 @@ static int ping_pre_connect(struct sock *sk, struct sockaddr *uaddr,if (addr_len <sizeof (struct sockaddr_in ))return -EINVAL ; -return BPF_CGROUP_RUN_PROG_INET4_CONNECT_LOCK (sk, uaddr); +return BPF_CGROUP_RUN_PROG_INET4_CONNECT_LOCK (sk, uaddr, &addr_len); }/* Checks the bind address and possibly modifies sk->sk_bound_dev_if. */
net/ipv4/tcp_ipv4.c +1 -1
@@ -194,7 +194,7 @@ static int tcp_v4_pre_connect(struct sock *sk, struct sockaddr *uaddr,sock_owned_by_me (sk); -return BPF_CGROUP_RUN_PROG_INET4_CONNECT (sk, uaddr); +return BPF_CGROUP_RUN_PROG_INET4_CONNECT (sk, uaddr, &addr_len); }/* This will initiate an outgoing connection. */
net/ipv4/udp.c +6 -3
@@ -1143,7 +1143,9 @@ int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)if (cgroup_bpf_enabled (CGROUP_UDP4_SENDMSG ) && !connected) { err =BPF_CGROUP_RUN_PROG_UDP4_SENDMSG_LOCK (sk, - (struct sockaddr *)usin, &ipc.addr); + (struct sockaddr *)usin, + &msg->msg_namelen, + &ipc.addr);if (err) goto out_free;if (usin) {
@@ -1865,7 +1867,8 @@ int udp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, int flags, *addr_len =sizeof (*sin);BPF_CGROUP_RUN_PROG_UDP4_RECVMSG_LOCK (sk, - (struct sockaddr *)sin); + (struct sockaddr *)sin, + addr_len); }if (udp_test_bit (GRO_ENABLED , sk))
@@ -1904,7 +1907,7 @@ int udp_pre_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len)if (addr_len <sizeof (struct sockaddr_in ))return -EINVAL ; -return BPF_CGROUP_RUN_PROG_INET4_CONNECT_LOCK (sk, uaddr); +return BPF_CGROUP_RUN_PROG_INET4_CONNECT_LOCK (sk, uaddr, &addr_len); }EXPORT_SYMBOL (udp_pre_connect);
net/ipv6/af_inet6.c +5 -4
@@ -456,7 +456,7 @@ int inet6_bind_sk(struct sock *sk, struct sockaddr *uaddr, int addr_len)/* BPF prog is run before any checks are done so that if the prog * changes context in a wrong way it will be caught. */ - err =BPF_CGROUP_RUN_PROG_INET_BIND_LOCK (sk, uaddr, + err =BPF_CGROUP_RUN_PROG_INET_BIND_LOCK (sk, uaddr, &addr_len,CGROUP_INET6_BIND , &flags);if (err)return err;
@@ -522,6 +522,7 @@ int inet6_getname(struct socket *sock, struct sockaddr *uaddr, int peer) {struct sockaddr_in6 *sin = (struct sockaddr_in6 *)uaddr; +int sin_addr_len =sizeof (*sin);struct sock *sk = sock->sk;struct inet_sock *inet =inet_sk (sk);struct ipv6_pinfo *np =inet6_sk (sk);
@@ -541,7 +542,7 @@ int inet6_getname(struct socket *sock, struct sockaddr *uaddr, sin->sin6_addr = sk->sk_v6_daddr;if (np->sndflow) sin->sin6_flowinfo = np->flow_label; -BPF_CGROUP_RUN_SA_PROG (sk, (struct sockaddr *)sin, +BPF_CGROUP_RUN_SA_PROG (sk, (struct sockaddr *)sin, &sin_addr_len,CGROUP_INET6_GETPEERNAME ); }else {if (ipv6_addr_any (&sk->sk_v6_rcv_saddr))
@@ -549,13 +550,13 @@ int inet6_getname(struct socket *sock, struct sockaddr *uaddr,else sin->sin6_addr = sk->sk_v6_rcv_saddr; sin->sin6_port = inet->inet_sport; -BPF_CGROUP_RUN_SA_PROG (sk, (struct sockaddr *)sin, +BPF_CGROUP_RUN_SA_PROG (sk, (struct sockaddr *)sin, &sin_addr_len,CGROUP_INET6_GETSOCKNAME ); } sin->sin6_scope_id =ipv6_iface_scope_id (&sin->sin6_addr, sk->sk_bound_dev_if);release_sock (sk); -return sizeof (*sin); +return sin_addr_len; }EXPORT_SYMBOL (inet6_getname);
net/ipv6/ping.c +1 -1
@@ -56,7 +56,7 @@ static int ping_v6_pre_connect(struct sock *sk, struct sockaddr *uaddr,if (addr_len <SIN6_LEN_RFC2133 )return -EINVAL ; -return BPF_CGROUP_RUN_PROG_INET6_CONNECT_LOCK (sk, uaddr); +return BPF_CGROUP_RUN_PROG_INET6_CONNECT_LOCK (sk, uaddr, &addr_len); }static int ping_v6_sendmsg (struct sock *sk,struct msghdr *msg,size_t len)
net/ipv6/tcp_ipv6.c +1 -1
@@ -135,7 +135,7 @@ static int tcp_v6_pre_connect(struct sock *sk, struct sockaddr *uaddr,sock_owned_by_me (sk); -return BPF_CGROUP_RUN_PROG_INET6_CONNECT (sk, uaddr); +return BPF_CGROUP_RUN_PROG_INET6_CONNECT (sk, uaddr, &addr_len); } static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr,
net/ipv6/udp.c +4 -2
@@ -410,7 +410,8 @@ int udpv6_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, *addr_len =sizeof (*sin6);BPF_CGROUP_RUN_PROG_UDP6_RECVMSG_LOCK (sk, - (struct sockaddr *)sin6); + (struct sockaddr *)sin6, + addr_len); }if (udp_test_bit (GRO_ENABLED , sk))
@@ -1157,7 +1158,7 @@ static int udpv6_pre_connect(struct sock *sk, struct sockaddr *uaddr,if (addr_len <SIN6_LEN_RFC2133 )return -EINVAL ; -return BPF_CGROUP_RUN_PROG_INET6_CONNECT_LOCK (sk, uaddr); +return BPF_CGROUP_RUN_PROG_INET6_CONNECT_LOCK (sk, uaddr, &addr_len); }/**
@@ -1510,6 +1511,7 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)if (cgroup_bpf_enabled (CGROUP_UDP6_SENDMSG ) && !connected) { err =BPF_CGROUP_RUN_PROG_UDP6_SENDMSG_LOCK (sk, (struct sockaddr *)sin6, + &addr_len, &fl6->saddr);if (err) goto out_no_dst;
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 215/331 bpf: Add bpf_sock_addr_set_sun_path() to allow writing unix sockaddr from bpf
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Daan De Meyer <[email protected]>
[ Upstream commit 53e380d21441909b12b6e0782b77187ae4b971c4 ]
As prep for adding unix socket support to the cgroup sockaddr hooks, let's add a kfunc bpf_sock_addr_set_sun_path() that allows modifying a unix sockaddr from bpf. While this is already possible for AF_INET and AF_INET6, we'll need this kfunc when we add unix socket support since modifying the address for those requires modifying both the address and the sockaddr length.
Stable-dep-of: c5114710c8ce ("xsk: fix usage of multi-buffer BPF helpers for ZC XDP")
--- kernel/bpf/btf.c | 1 + net/core/filter.c | 35 ++++++++++++++++++++++++++++++++++- 2 files changed, 35 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +1 -0
@@ -7832,6 +7832,7 @@ static int bpf_prog_type_to_kfunc_hook(enum bpf_prog_type prog_type)case BPF_PROG_TYPE_SYSCALL :return BTF_KFUNC_HOOK_SYSCALL ;case BPF_PROG_TYPE_CGROUP_SKB : +case BPF_PROG_TYPE_CGROUP_SOCK_ADDR :return BTF_KFUNC_HOOK_CGROUP_SKB ;case BPF_PROG_TYPE_SCHED_ACT :return BTF_KFUNC_HOOK_SCHED_ACT ;
net/core/filter.c +34 -1
@@ -81,6 +81,7 @@#include <net/xdp.h> #include <net/mptcp.h> #include <net/netfilter/nf_conntrack_bpf.h> +#include <linux/un.h> static const struct bpf_func_proto *bpf_sk_base_func_proto (enum bpf_func_id func_id);
@@ -11772,6 +11773,27 @@ __bpf_kfunc int bpf_dynptr_from_xdp(struct xdp_buff *xdp, u64 flags,return 0 ; } + +__bpf_kfunc int bpf_sock_addr_set_sun_path(struct bpf_sock_addr_kern *sa_kern, +const u8 *sun_path,u32 sun_path__sz) +{ +struct sockaddr_un *un; + +if (sa_kern->sk->sk_family !=AF_UNIX ) +return -EINVAL ; + +/* We do not allow changing the address to unnamed or larger than the +* maximum allowed address size for a unix sockaddr. +*/ +if (sun_path__sz ==0 || sun_path__sz >UNIX_PATH_MAX ) +return -EINVAL ; + + un = (struct sockaddr_un *)sa_kern->uaddr; +memcpy (un->sun_path, sun_path, sun_path__sz); + sa_kern->uaddrlen = offsetof(struct sockaddr_un , sun_path) + sun_path__sz; + +return 0 ; +}__diag_pop (); int bpf_dynptr_from_skb_rdonly(struct sk_buff *skb, u64 flags,
@@ -11796,6 +11818,10 @@ BTF_SET8_START(bpf_kfunc_check_set_xdp) BTF_ID_FLAGS(func, bpf_dynptr_from_xdp)BTF_SET8_END (bpf_kfunc_check_set_xdp) +BTF_SET8_START (bpf_kfunc_check_set_sock_addr ) +BTF_ID_FLAGS (func, bpf_sock_addr_set_sun_path) +BTF_SET8_END (bpf_kfunc_check_set_sock_addr) + static const struct btf_kfunc_id_set bpf_kfunc_set_skb = { .owner =THIS_MODULE , .set = &bpf_kfunc_check_set_skb,
@@ -11806,6 +11832,11 @@ static const struct btf_kfunc_id_set bpf_kfunc_set_xdp = { .set = &bpf_kfunc_check_set_xdp, }; +static const struct btf_kfunc_id_set bpf_kfunc_set_sock_addr = { + .owner =THIS_MODULE , + .set = &bpf_kfunc_check_set_sock_addr, +}; +static int __init bpf_kfunc_init (void ) {int ret;
@@ -11820,7 +11851,9 @@ static int __init bpf_kfunc_init(void) ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_LWT_XMIT , &bpf_kfunc_set_skb); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_LWT_SEG6LOCAL , &bpf_kfunc_set_skb); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_NETFILTER , &bpf_kfunc_set_skb); -return ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_XDP , &bpf_kfunc_set_xdp); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_XDP , &bpf_kfunc_set_xdp); +return ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_CGROUP_SOCK_ADDR , + &bpf_kfunc_set_sock_addr); }late_initcall (bpf_kfunc_init);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 216/331 xsk: fix usage of multi-buffer BPF helpers for ZC XDP
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Maciej Fijalkowski <[email protected]>
[ Upstream commit c5114710c8ce86b8317e9b448f4fd15c711c2a82 ]
Currently when packet is shrunk via bpf_xdp_adjust_tail() and memory type is set to MEM_TYPE_XSK_BUFF_POOL, null ptr dereference happens:
[1136314.192256] BUG: kernel NULL pointer dereference, address: 0000000000000034 [1136314.203943] #PF: supervisor read access in kernel mode [1136314.213768] #PF: error_code(0x0000) - not-present page [1136314.223550] PGD 0 P4D 0 [1136314.230684] Oops: 0000 [#1] PREEMPT SMP NOPTI [1136314.239621] CPU: 8 PID: 54203 Comm: xdpsock Not tainted 6.6.0+ #257 [1136314.250469] Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.02.01.0008.031920191559 03/19/2019 [1136314.265615] RIP: 0010:__xdp_return+0x6c/0x210 [1136314.274653] Code: ad 00 48 8b 47 08 49 89 f8 a8 01 0f 85 9b 01 00 00 0f 1f 44 00 00 f0 41 ff 48 34 75 32 4c 89 c7 e9 79 cd 80 ff 83 fe 03 75 17 <f6> 41 34 01 0f 85 02 01 00 00 48 89 cf e9 22 cc 1e 00 e9 3d d2 86 [1136314.302907] RSP: 0018:ffffc900089f8db0 EFLAGS: 00010246 [1136314.312967] RAX: ffffc9003168aed0 RBX: ffff8881c3300000 RCX: 0000000000000000 [1136314.324953] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffffc9003168c000 [1136314.336929] RBP: 0000000000000ae0 R08: 0000000000000002 R09: 0000000000010000 [1136314.348844] R10: ffffc9000e495000 R11: 0000000000000040 R12: 0000000000000001 [1136314.360706] R13: 0000000000000524 R14: ffffc9003168aec0 R15: 0000000000000001 [1136314.373298] FS: 00007f8df8bbcb80(0000) GS:ffff8897e0e00000(0000) knlGS:0000000000000000 [1136314.386105] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [1136314.396532] CR2: 0000000000000034 CR3: 00000001aa912002 CR4: 00000000007706f0 [1136314.408377] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [1136314.420173] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [1136314.431890] PKRU: 55555554 [1136314.439143] Call Trace: [1136314.446058] <IRQ> [1136314.452465] ? __die+0x20/0x70 [1136314.459881] ? page_fault_oops+0x15b/0x440 [1136314.468305] ? exc_page_fault+0x6a/0x150 [1136314.476491] ? asm_exc_page_fault+0x22/0x30 [1136314.484927] ? __xdp_return+0x6c/0x210 [1136314.492863] bpf_xdp_adjust_tail+0x155/0x1d0 [1136314.501269] bpf_prog_ccc47ae29d3b6570_xdp_sock_prog+0x15/0x60 [1136314.511263] ice_clean_rx_irq_zc+0x206/0xc60 [ice] [1136314.520222] ? ice_xmit_zc+0x6e/0x150 [ice] [1136314.528506] ice_napi_poll+0x467/0x670 [ice] [1136314.536858] ? ttwu_do_activate.constprop.0+0x8f/0x1a0 [1136314.546010] __napi_poll+0x29/0x1b0 [1136314.553462] net_rx_action+0x133/0x270 [1136314.561619] __do_softirq+0xbe/0x28e [1136314.569303] do_softirq+0x3f/0x60
This comes from __xdp_return() call with xdp_buff argument passed as NULL which is supposed to be consumed by xsk_buff_free() call.
To address this properly, in ZC case, a node that represents the frag being removed has to be pulled out of xskb_list. Introduce appropriate xsk helpers to do such node operation and use them accordingly within bpf_xdp_adjust_tail().
--- include/net/xdp_sock_drv.h | 26 +++++++++++++++++++++++ net/core/filter.c | 42 ++++++++++++++++++++++++++++++++------ 2 files changed, 62 insertions(+), 6 deletions(-)
include/net/xdp_sock_drv.h +26 -0
@@ -147,6 +147,23 @@ static inline struct xdp_buff *xsk_buff_get_frag(struct xdp_buff *first)return ret; } +static inline void xsk_buff_del_tail (struct xdp_buff *tail) +{ + struct xdp_buff_xsk *xskb = container_of(tail, struct xdp_buff_xsk, xdp); + +list_del (&xskb->xskb_list_node); +} + +static inline struct xdp_buff *xsk_buff_get_tail (struct xdp_buff *first) +{ + struct xdp_buff_xsk *xskb = container_of(first, struct xdp_buff_xsk, xdp); +struct xdp_buff_xsk *frag; + + frag = list_last_entry(&xskb->pool->xskb_list, struct xdp_buff_xsk, + xskb_list_node); +return &frag->xdp; +} +static inline void xsk_buff_set_size (struct xdp_buff *xdp,u32 size) { xdp->data = xdp->data_hard_start +XDP_PACKET_HEADROOM ;
@@ -310,6 +327,15 @@ static inline struct xdp_buff *xsk_buff_get_frag(struct xdp_buff *first)return NULL ; } +static inline void xsk_buff_del_tail (struct xdp_buff *tail) +{ +} + +static inline struct xdp_buff *xsk_buff_get_tail (struct xdp_buff *first) +{ +return NULL ; +} +static inline void xsk_buff_set_size (struct xdp_buff *xdp,u32 size) { }
net/core/filter.c +36 -6
60 lines not on this page · raw message
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 217/331 ice: work on pre-XDP prog frag count
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Maciej Fijalkowski <[email protected]>
[ Upstream commit ad2047cf5d9313200e308612aed516548873d124 ]
Fix an OOM panic in XDP_DRV mode when a XDP program shrinks a multi-buffer packet by 4k bytes and then redirects it to an AF_XDP socket.
Since support for handling multi-buffer frames was added to XDP, usage of bpf_xdp_adjust_tail() helper within XDP program can free the page that given fragment occupies and in turn decrease the fragment count within skb_shared_info that is embedded in xdp_buff struct. In current ice driver codebase, it can become problematic when page recycling logic decides not to reuse the page. In such case, __page_frag_cache_drain() is used with ice_rx_buf::pagecnt_bias that was not adjusted after refcount of page was changed by XDP prog which in turn does not drain the refcount to 0 and page is never freed.
To address this, let us store the count of frags before the XDP program was executed on Rx ring struct. This will be used to compare with current frag count from skb_shared_info embedded in xdp_buff. A smaller value in the latter indicates that XDP prog freed frag(s). Then, for given delta decrement pagecnt_bias for XDP_DROP verdict.
While at it, let us also handle the EOP frag within ice_set_rx_bufs_act() to make our life easier, so all of the adjustments needed to be applied against freed frags are performed in the single place.
--- drivers/net/ethernet/intel/ice/ice_txrx.c | 14 ++++++--- drivers/net/ethernet/intel/ice/ice_txrx.h | 1 + drivers/net/ethernet/intel/ice/ice_txrx_lib.h | 31 +++++++++++++------ 3 files changed, 32 insertions(+), 14 deletions(-)
drivers/net/ethernet/intel/ice/ice_txrx.c +9 -5
@@ -600,9 +600,7 @@ ice_run_xdp(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp, ret =ICE_XDP_CONSUMED ; } exit: - rx_buf->act = ret; -if (unlikely (xdp_buff_has_frags (xdp))) -ice_set_rx_bufs_act (xdp, rx_ring, ret); +ice_set_rx_bufs_act (xdp, rx_ring, ret); }/**
@@ -890,14 +888,17 @@ ice_add_xdp_frag(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp, }if (unlikely (sinfo->nr_frags ==MAX_SKB_FRAGS )) { -if (unlikely (xdp_buff_has_frags (xdp))) -ice_set_rx_bufs_act (xdp, rx_ring,ICE_XDP_CONSUMED ); +ice_set_rx_bufs_act (xdp, rx_ring,ICE_XDP_CONSUMED );return -ENOMEM ; }__skb_fill_page_desc_noacc (sinfo, sinfo->nr_frags++, rx_buf->page, rx_buf->page_offset, size); sinfo->xdp_frags_size += size; +/* remember frag count before XDP prog execution; bpf_xdp_adjust_tail() +* can pop off frags but driver has to handle it on its own +*/ + rx_ring->nr_frags = sinfo->nr_frags;if (page_is_pfmemalloc (rx_buf->page))xdp_buff_set_frag_pfmemalloc (xdp);
@@ -1249,6 +1250,7 @@ int ice_clean_rx_irq(struct ice_rx_ring *rx_ring, int budget) xdp->data =NULL ; rx_ring->first_desc = ntc; + rx_ring->nr_frags =0 ;continue ; construct_skb:if (likely (ice_ring_uses_build_skb (rx_ring)))
@@ -1264,10 +1266,12 @@ int ice_clean_rx_irq(struct ice_rx_ring *rx_ring, int budget) ICE_XDP_CONSUMED); xdp->data =NULL ; rx_ring->first_desc = ntc; + rx_ring->nr_frags =0 ;break ; } xdp->data =NULL ; rx_ring->first_desc = ntc; + rx_ring->nr_frags =0 ; stat_err_bits =BIT (ICE_RX_FLEX_DESC_STATUS0_RXE_S ); if (unlikely(ice_test_staterr(rx_desc->wb.status_error0,
drivers/net/ethernet/intel/ice/ice_txrx.h +1 -0
@@ -333,6 +333,7 @@ struct ice_rx_ring {struct ice_channel *ch;struct ice_tx_ring *xdp_ring;struct xsk_buff_pool *xsk_pool; +u32 nr_frags;dma_addr_t dma;/* physical address of ring */ u64 cached_phctime;u16 rx_buf_len;
drivers/net/ethernet/intel/ice/ice_txrx_lib.h +22 -9
@@ -12,26 +12,39 @@* act: action to store onto Rx buffers related to XDP buffer parts * * Set action that should be taken before putting Rx buffer from first frag -* to one before last. Last one is handled by caller of this function as it -* is the EOP frag that is currently being processed. This function is -* supposed to be called only when XDP buffer contains frags. +* to the last. */ static inline void ice_set_rx_bufs_act (struct xdp_buff *xdp,const struct ice_rx_ring *rx_ring,const unsigned int act) { -const struct skb_shared_info *sinfo =xdp_get_shared_info_from_buff (xdp); -u32 first = rx_ring->first_desc; -u32 nr_frags = sinfo->nr_frags; +u32 sinfo_frags =xdp_get_shared_info_from_buff (xdp)->nr_frags; +u32 nr_frags = rx_ring->nr_frags +1 ; +u32 idx = rx_ring->first_desc;u32 cnt = rx_ring->count;struct ice_rx_buf *buf;for (int i =0 ; i < nr_frags; i++) { - buf = &rx_ring->rx_buf[first]; + buf = &rx_ring->rx_buf[idx]; buf->act = act; -if (++first == cnt) - first =0 ; +if (++idx == cnt) + idx =0 ; + } + +/* adjust pagecnt_bias on frags freed by XDP prog */ +if (sinfo_frags < rx_ring->nr_frags && act ==ICE_XDP_CONSUMED ) { +u32 delta = rx_ring->nr_frags - sinfo_frags; + +while (delta) { +if (idx ==0 ) + idx = cnt -1 ; +else + idx--; + buf = &rx_ring->rx_buf[idx]; + buf->pagecnt_bias--; + delta--; + } } }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 218/331 i40e: handle multi-buffer packets that are shrunk by xdp prog
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Tirthendu Sarkar <[email protected]>
[ Upstream commit 83014323c642b8faa2d64a5f303b41c019322478 ]
XDP programs can shrink packets by calling the bpf_xdp_adjust_tail() helper function. For multi-buffer packets this may lead to reduction of frag count stored in skb_shared_info area of the xdp_buff struct. This results in issues with the current handling of XDP_PASS and XDP_DROP cases.
For XDP_PASS, currently skb is being built using frag count of xdp_buffer before it was processed by XDP prog and thus will result in an inconsistent skb when frag count gets reduced by XDP prog. To fix this, get correct frag count while building the skb instead of using pre-obtained frag count.
For XDP_DROP, current page recycling logic will not reuse the page but instead will adjust the pagecnt_bias so that the page can be freed. This again results in inconsistent behavior as the page refcnt has already been changed by the helper while freeing the frag(s) as part of shrinking the packet. To fix this, only adjust pagecnt_bias for buffers that are stillpart of the packet post-xdp prog run.
--- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 40 ++++++++++++--------- 1 file changed, 23 insertions(+), 17 deletions(-)
drivers/net/ethernet/intel/i40e/i40e_txrx.c +23 -17
@@ -2100,7 +2100,8 @@ static void i40e_put_rx_buffer(struct i40e_ring *rx_ring,static void i40e_process_rx_buffs (struct i40e_ring *rx_ring,int xdp_res,struct xdp_buff *xdp) { -u32 next = rx_ring->next_to_clean; +u32 nr_frags =xdp_get_shared_info_from_buff (xdp)->nr_frags; +u32 next = rx_ring->next_to_clean, i =0 ;struct i40e_rx_buffer *rx_buffer; xdp->flags =0 ;
@@ -2113,10 +2114,10 @@ static void i40e_process_rx_buffs(struct i40e_ring *rx_ring, int xdp_res,if (!rx_buffer->page)continue ; -if (xdp_res ==I40E_XDP_CONSUMED ) - rx_buffer->pagecnt_bias++; -else +if (xdp_res !=I40E_XDP_CONSUMED )i40e_rx_buffer_flip (rx_buffer, xdp->frame_sz); +else if (i++ <= nr_frags) + rx_buffer->pagecnt_bias++;/* EOP buffer will be put in i40e_clean_rx_irq() */ if (next == rx_ring->next_to_process)
@@ -2130,20 +2131,20 @@ static void i40e_process_rx_buffs(struct i40e_ring *rx_ring, int xdp_res,* i40e_construct_skb - Allocate skb and populate it * @rx_ring: rx descriptor ring to transact packets on * @xdp: xdp_buff pointing to the data -* @nr_frags: number of buffers for the packet * * This function allocates an skb. It then populates it with the page * data from the current receive descriptor, taking care to set up the * skb correctly. */ static struct sk_buff *i40e_construct_skb (struct i40e_ring *rx_ring, -struct xdp_buff *xdp, -u32 nr_frags) +struct xdp_buff *xdp) {unsigned int size = xdp->data_end - xdp->data;struct i40e_rx_buffer *rx_buffer; +struct skb_shared_info *sinfo;unsigned int headlen;struct sk_buff *skb; +u32 nr_frags =0 ;/* prefetch first cache line of first page */ net_prefetch (xdp->data);
@@ -2181,6 +2182,10 @@ static struct sk_buff *i40e_construct_skb(struct i40e_ring *rx_ring,memcpy (__skb_put (skb, headlen), xdp->data,ALIGN (headlen,sizeof (long ))); +if (unlikely (xdp_buff_has_frags (xdp))) { + sinfo =xdp_get_shared_info_from_buff (xdp); + nr_frags = sinfo->nr_frags; + } rx_buffer =i40e_rx_bi (rx_ring, rx_ring->next_to_clean);/* update all of the pointers */ size -= headlen;
@@ -2200,9 +2205,8 @@ static struct sk_buff *i40e_construct_skb(struct i40e_ring *rx_ring, }if (unlikely (xdp_buff_has_frags (xdp))) { -struct skb_shared_info *sinfo, *skinfo =skb_shinfo (skb); +struct skb_shared_info *skinfo =skb_shinfo (skb); - sinfo =xdp_get_shared_info_from_buff (xdp);memcpy (&skinfo->frags[skinfo->nr_frags], &sinfo->frags[0 ],sizeof (skb_frag_t) * nr_frags);
@@ -2225,17 +2229,17 @@ static struct sk_buff *i40e_construct_skb(struct i40e_ring *rx_ring,* i40e_build_skb - Build skb around an existing buffer * @rx_ring: Rx descriptor ring to transact packets on * @xdp: xdp_buff pointing to the data -* @nr_frags: number of buffers for the packet * * This function builds an skb around an existing Rx buffer, taking care * to set up the skb correctly and avoid any memcpy overhead. */ static struct sk_buff *i40e_build_skb (struct i40e_ring *rx_ring, -struct xdp_buff *xdp, -u32 nr_frags) +struct xdp_buff *xdp) {unsigned int metasize = xdp->data - xdp->data_meta; +struct skb_shared_info *sinfo;struct sk_buff *skb; +u32 nr_frags;/* Prefetch first cache line of first page. If xdp->data_meta * is unused, this points exactly as xdp->data, otherwise we
@@ -2244,6 +2248,11 @@ static struct sk_buff *i40e_build_skb(struct i40e_ring *rx_ring,*/ net_prefetch (xdp->data_meta); +if (unlikely (xdp_buff_has_frags (xdp))) { + sinfo =xdp_get_shared_info_from_buff (xdp); + nr_frags = sinfo->nr_frags; + } +/* build an skb around the page buffer */ skb =napi_build_skb (xdp->data_hard_start, xdp->frame_sz);if (unlikely (!skb))
@@ -2256,9 +2265,6 @@ static struct sk_buff *i40e_build_skb(struct i40e_ring *rx_ring,skb_metadata_set (skb, metasize);if (unlikely (xdp_buff_has_frags (xdp))) { -struct skb_shared_info *sinfo; - - sinfo =xdp_get_shared_info_from_buff (xdp); xdp_update_skb_shared_info(skb, nr_frags, sinfo->xdp_frags_size, nr_frags * xdp->frame_sz,
@@ -2603,9 +2609,9 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget, total_rx_bytes += size; }else {if (ring_uses_build_skb (rx_ring)) - skb =i40e_build_skb (rx_ring, xdp, nfrags); + skb =i40e_build_skb (rx_ring, xdp);else - skb =i40e_construct_skb (rx_ring, xdp, nfrags); + skb =i40e_construct_skb (rx_ring, xdp);/* drop if we failed to retrieve a buffer */ if (!skb) {
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 219/331 ice: remove redundant xdp_rxq_info registration
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Maciej Fijalkowski <[email protected]>
[ Upstream commit 2ee788c06493d02ee85855414cca39825e768aaf ]
xdp_rxq_info struct can be registered by drivers via two functions - xdp_rxq_info_reg() and __xdp_rxq_info_reg(). The latter one allows drivers that support XDP multi-buffer to set up xdp_rxq_info::frag_size which in turn will make it possible to grow the packet via bpf_xdp_adjust_tail() BPF helper.
Currently, ice registers xdp_rxq_info in two spots:
1) ice_setup_rx_ring() // via xdp_rxq_info_reg(), BUG
2) ice_vsi_cfg_rxq() // via __xdp_rxq_info_reg(), OK
Cited commit under fixes tag took care of setting up frag_size and updated registration scheme in 2) but it did not help as
1) is called before 2) and as shown above it uses old registration function. This means that 2) sees that xdp_rxq_info is already registered and never calls __xdp_rxq_info_reg() which leaves us with xdp_rxq_info::frag_size being set to 0.
To fix this misbehavior, simply remove xdp_rxq_info_reg() call from ice_setup_rx_ring().
--- drivers/net/ethernet/intel/ice/ice_txrx.c | 5 ----- 1 file changed, 5 deletions(-)
drivers/net/ethernet/intel/ice/ice_txrx.c +0 -5
@@ -513,11 +513,6 @@ int ice_setup_rx_ring(struct ice_rx_ring *rx_ring)if (ice_is_xdp_ena_vsi (rx_ring->vsi))WRITE_ONCE (rx_ring->xdp_prog, rx_ring->vsi->xdp_prog); -if (rx_ring->vsi->type ==ICE_VSI_PF && - !xdp_rxq_info_is_reg (&rx_ring->xdp_rxq)) -if (xdp_rxq_info_reg (&rx_ring->xdp_rxq, rx_ring->netdev, - rx_ring->q_index, rx_ring->q_vector->napi.napi_id)) - goto err;return 0 ; err:
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 220/331 intel: xsk: initialize skb_frag_t::bv_offset in ZC drivers
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Maciej Fijalkowski <[email protected]>
[ Upstream commit 290779905d09d5fdf6caa4f58ddefc3f4db0c0a9 ]
Ice and i40e ZC drivers currently set offset of a frag within skb_shared_info to 0, which is incorrect. xdp_buffs that come from xsk_buff_pool always have 256 bytes of a headroom, so they need to be taken into account to retrieve xdp_buff::data via skb_frag_address(). Otherwise, bpf_xdp_frags_increase_tail() would be starting its job from xdp_buff::data_hard_start which would result in overwriting existing payload.
--- drivers/net/ethernet/intel/i40e/i40e_xsk.c | 3 ++- drivers/net/ethernet/intel/ice/ice_xsk.c | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-)
drivers/net/ethernet/intel/i40e/i40e_xsk.c +2 -1
@@ -418,7 +418,8 @@ i40e_add_xsk_frag(struct i40e_ring *rx_ring, struct xdp_buff *first, }__skb_fill_page_desc_noacc (sinfo, sinfo->nr_frags++, -virt_to_page (xdp->data_hard_start),0 , size); +virt_to_page (xdp->data_hard_start), +XDP_PACKET_HEADROOM , size); sinfo->xdp_frags_size += size;xsk_buff_add_frag (xdp);
drivers/net/ethernet/intel/ice/ice_xsk.c +2 -1
@@ -826,7 +826,8 @@ ice_add_xsk_frag(struct ice_rx_ring *rx_ring, struct xdp_buff *first, }__skb_fill_page_desc_noacc (sinfo, sinfo->nr_frags++, -virt_to_page (xdp->data_hard_start),0 , size); +virt_to_page (xdp->data_hard_start), +XDP_PACKET_HEADROOM , size); sinfo->xdp_frags_size += size;xsk_buff_add_frag (xdp);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 221/331 ice: update xdp_rxq_info::frag_size for ZC enabled Rx queue
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Maciej Fijalkowski <[email protected]>
[ Upstream commit 3de38c87174225487fc93befeea7d380db80aef6 ]
Now that ice driver correctly sets up frag_size in xdp_rxq_info, let us make it work for ZC multi-buffer as well. ice_rx_ring::rx_buf_len for ZC is being set via xsk_pool_get_rx_frame_size() and this needs to be propagated up to xdp_rxq_info.
Use a bigger hammer and instead of unregistering only xdp_rxq_info's memory model, unregister it altogether and register it again and have xdp_rxq_info with correct frag_size value.
--- drivers/net/ethernet/intel/ice/ice_base.c | 37 ++++++++++++++--------- 1 file changed, 23 insertions(+), 14 deletions(-)
drivers/net/ethernet/intel/ice/ice_base.c +23 -14
@@ -534,19 +534,27 @@ int ice_vsi_cfg_rxq(struct ice_rx_ring *ring) ring->rx_buf_len = ring->vsi->rx_buf_len;if (ring->vsi->type ==ICE_VSI_PF ) { -if (!xdp_rxq_info_is_reg (&ring->xdp_rxq)) -/* coverity[check_return] */ -__xdp_rxq_info_reg (&ring->xdp_rxq, ring->netdev, - ring->q_index, - ring->q_vector->napi.napi_id, - ring->vsi->rx_buf_len); +if (!xdp_rxq_info_is_reg (&ring->xdp_rxq)) { + err =__xdp_rxq_info_reg (&ring->xdp_rxq, ring->netdev, + ring->q_index, + ring->q_vector->napi.napi_id, + ring->rx_buf_len); +if (err) +return err; + } ring->xsk_pool =ice_xsk_pool (ring);if (ring->xsk_pool) { -xdp_rxq_info_unreg_mem_model (&ring->xdp_rxq); +xdp_rxq_info_unreg (&ring->xdp_rxq); ring->rx_buf_len =xsk_pool_get_rx_frame_size (ring->xsk_pool); + err =__xdp_rxq_info_reg (&ring->xdp_rxq, ring->netdev, + ring->q_index, + ring->q_vector->napi.napi_id, + ring->rx_buf_len); +if (err) +return err; err =xdp_rxq_info_reg_mem_model (&ring->xdp_rxq,MEM_TYPE_XSK_BUFF_POOL ,NULL );
@@ -557,13 +565,14 @@ int ice_vsi_cfg_rxq(struct ice_rx_ring *ring)dev_info (dev,"Registered XDP mem model MEM_TYPE_XSK_BUFF_POOL on Rx ring %d\n" , ring->q_index); }else { -if (!xdp_rxq_info_is_reg (&ring->xdp_rxq)) -/* coverity[check_return] */ -__xdp_rxq_info_reg (&ring->xdp_rxq, - ring->netdev, - ring->q_index, - ring->q_vector->napi.napi_id, - ring->vsi->rx_buf_len); +if (!xdp_rxq_info_is_reg (&ring->xdp_rxq)) { + err =__xdp_rxq_info_reg (&ring->xdp_rxq, ring->netdev, + ring->q_index, + ring->q_vector->napi.napi_id, + ring->rx_buf_len); +if (err) +return err; + } err = xdp_rxq_info_reg_mem_model(&ring->xdp_rxq, MEM_TYPE_PAGE_SHARED,
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 222/331 xdp: reflect tail increase for MEM_TYPE_XSK_BUFF_POOL
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Maciej Fijalkowski <[email protected]>
[ Upstream commit fbadd83a612c3b7aad2987893faca6bd24aaebb3 ]
XSK ZC Rx path calculates the size of data that will be posted to XSK Rx queue via subtracting xdp_buff::data_end from xdp_buff::data.
In bpf_xdp_frags_increase_tail(), when underlying memory type of xdp_rxq_info is MEM_TYPE_XSK_BUFF_POOL, add offset to data_end in tail fragment, so that later on user space will be able to take into account the amount of bytes added by XDP program.
--- net/core/filter.c | 2 ++ 1 file changed, 2 insertions(+)
net/core/filter.c +2 -0
@@ -4081,6 +4081,8 @@ static int bpf_xdp_frags_increase_tail(struct xdp_buff *xdp, int offset)memset (skb_frag_address (frag) +skb_frag_size (frag),0 , offset);skb_frag_size_add (frag, offset); sinfo->xdp_frags_size += offset; +if (rxq->mem.type ==MEM_TYPE_XSK_BUFF_POOL ) +xsk_buff_get_tail (xdp)->data_end += offset;return 0 ; }
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 223/331 i40e: set xdp_rxq_info::frag_size
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Maciej Fijalkowski <[email protected]>
[ Upstream commit a045d2f2d03d23e7db6772dd83e0ba2705dfad93 ]
i40e support XDP multi-buffer so it is supposed to use __xdp_rxq_info_reg() instead of xdp_rxq_info_reg() and set the frag_size. It can not be simply converted at existing callsite because rx_buf_len could be un-initialized, so let us register xdp_rxq_info within i40e_configure_rx_ring(), which happen to be called with already initialized rx_buf_len value.
Commit 5180ff1364bc ("i40e: use int for i40e_status") converted 'err' to int, so two variables to deal with return codes are not needed within i40e_configure_rx_ring(). Remove 'ret' and use 'err' to handle status from xdp_rxq_info registration.
--- drivers/net/ethernet/intel/i40e/i40e_main.c | 40 ++++++++++++--------- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 9 ----- 2 files changed, 24 insertions(+), 25 deletions(-)
drivers/net/ethernet/intel/i40e/i40e_main.c +24 -16
@@ -3578,40 +3578,48 @@ static int i40e_configure_rx_ring(struct i40e_ring *ring)struct i40e_hmc_obj_rxq rx_ctx;int err =0 ;bool ok; -int ret;bitmap_zero (ring->state, __I40E_RING_STATE_NBITS);/* clear the context structure first */ memset (&rx_ctx,0 ,sizeof (rx_ctx)); -if (ring->vsi->type ==I40E_VSI_MAIN ) -xdp_rxq_info_unreg_mem_model (&ring->xdp_rxq); + ring->rx_buf_len = vsi->rx_buf_len; + +/* XDP RX-queue info only needed for RX rings exposed to XDP */ +if (ring->vsi->type !=I40E_VSI_MAIN ) + goto skip; + +if (!xdp_rxq_info_is_reg (&ring->xdp_rxq)) { + err =__xdp_rxq_info_reg (&ring->xdp_rxq, ring->netdev, + ring->queue_index, + ring->q_vector->napi.napi_id, + ring->rx_buf_len); +if (err) +return err; + } ring->xsk_pool =i40e_xsk_pool (ring);if (ring->xsk_pool) { - ring->rx_buf_len = -xsk_pool_get_rx_frame_size (ring->xsk_pool); - ret =xdp_rxq_info_reg_mem_model (&ring->xdp_rxq, + ring->rx_buf_len =xsk_pool_get_rx_frame_size (ring->xsk_pool); + err =xdp_rxq_info_reg_mem_model (&ring->xdp_rxq,MEM_TYPE_XSK_BUFF_POOL ,NULL ); -if (ret) -return ret; +if (err) +return err;dev_info (&vsi->back->pdev->dev,"Registered XDP mem model MEM_TYPE_XSK_BUFF_POOL on Rx ring %d\n" , ring->queue_index); }else { - ring->rx_buf_len = vsi->rx_buf_len; -if (ring->vsi->type ==I40E_VSI_MAIN ) { - ret =xdp_rxq_info_reg_mem_model (&ring->xdp_rxq, -MEM_TYPE_PAGE_SHARED , -NULL ); -if (ret) -return ret; - } + err =xdp_rxq_info_reg_mem_model (&ring->xdp_rxq, +MEM_TYPE_PAGE_SHARED , +NULL ); +if (err) +return err; } +skip:xdp_init_buff (&ring->xdp,i40e_rx_pg_size (ring) /2 , &ring->xdp_rxq); rx_ctx.dbuff = DIV_ROUND_UP(ring->rx_buf_len,
drivers/net/ethernet/intel/i40e/i40e_txrx.c +0 -9
@@ -1556,7 +1556,6 @@ void i40e_free_rx_resources(struct i40e_ring *rx_ring)int i40e_setup_rx_descriptors (struct i40e_ring *rx_ring) {struct device *dev = rx_ring->dev; -int err;u64_stats_init (&rx_ring->syncp);
@@ -1577,14 +1576,6 @@ int i40e_setup_rx_descriptors(struct i40e_ring *rx_ring) rx_ring->next_to_process =0 ; rx_ring->next_to_use =0 ; -/* XDP RX-queue info only needed for RX rings exposed to XDP */ -if (rx_ring->vsi->type ==I40E_VSI_MAIN ) { - err =xdp_rxq_info_reg (&rx_ring->xdp_rxq, rx_ring->netdev, - rx_ring->queue_index, rx_ring->q_vector->napi.napi_id); -if (err <0 ) -return err; - } - rx_ring->xdp_prog = rx_ring->vsi->xdp_prog; rx_ring->rx_bi =
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 224/331 i40e: update xdp_rxq_info::frag_size for ZC enabled Rx queue
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Maciej Fijalkowski <[email protected]>
[ Upstream commit 0cbb08707c932b3f004bc1a8ec6200ef572c1f5f ]
Now that i40e driver correctly sets up frag_size in xdp_rxq_info, let us make it work for ZC multi-buffer as well. i40e_ring::rx_buf_len for ZC is being set via xsk_pool_get_rx_frame_size() and this needs to be propagated up to xdp_rxq_info.
--- drivers/net/ethernet/intel/i40e/i40e_main.c | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/net/ethernet/intel/i40e/i40e_main.c +7 -0
@@ -3601,7 +3601,14 @@ static int i40e_configure_rx_ring(struct i40e_ring *ring) ring->xsk_pool =i40e_xsk_pool (ring);if (ring->xsk_pool) { +xdp_rxq_info_unreg (&ring->xdp_rxq); ring->rx_buf_len =xsk_pool_get_rx_frame_size (ring->xsk_pool); + err =__xdp_rxq_info_reg (&ring->xdp_rxq, ring->netdev, + ring->queue_index, + ring->q_vector->napi.napi_id, + ring->rx_buf_len); +if (err) +return err; err =xdp_rxq_info_reg_mem_model (&ring->xdp_rxq,MEM_TYPE_XSK_BUFF_POOL ,NULL );
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 225/331 fjes: fix memleaks in fjes_hw_setup
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Zhipeng Lu <[email protected]>
[ Upstream commit f6cc4b6a3ae53df425771000e9c9540cce9b7bb1 ]
In fjes_hw_setup, it allocates several memory and delay the deallocation to the fjes_hw_exit in fjes_probe through the following call chain:
fjes_probe
|-> fjes_hw_init
|-> fjes_hw_setup
|-> fjes_hw_exit
However, when fjes_hw_setup fails, fjes_hw_exit won't be called and thus all the resources allocated in fjes_hw_setup will be leaked. In this patch, we free those resources in fjes_hw_setup and prevents such leaks.
--- drivers/net/fjes/fjes_hw.c | 37 ++++++++++++++++++++++++++++++------- 1 file changed, 30 insertions(+), 7 deletions(-)
drivers/net/fjes/fjes_hw.c +30 -7
@@ -221,21 +221,25 @@ static int fjes_hw_setup(struct fjes_hw *hw) mem_size =FJES_DEV_REQ_BUF_SIZE (hw->max_epid); hw->hw_info.req_buf =kzalloc (mem_size,GFP_KERNEL ); -if (!(hw->hw_info.req_buf)) -return -ENOMEM ; +if (!(hw->hw_info.req_buf)) { + result = -ENOMEM ; + goto free_ep_info; + } hw->hw_info.req_buf_size = mem_size; mem_size =FJES_DEV_RES_BUF_SIZE (hw->max_epid); hw->hw_info.res_buf =kzalloc (mem_size,GFP_KERNEL ); -if (!(hw->hw_info.res_buf)) -return -ENOMEM ; +if (!(hw->hw_info.res_buf)) { + result = -ENOMEM ; + goto free_req_buf; + } hw->hw_info.res_buf_size = mem_size; result =fjes_hw_alloc_shared_status_region (hw);if (result) -return result; + goto free_res_buf; hw->hw_info.buffer_share_bit =0 ; hw->hw_info.buffer_unshare_reserve_bit =0 ;
@@ -246,11 +250,11 @@ static int fjes_hw_setup(struct fjes_hw *hw) result =fjes_hw_alloc_epbuf (&buf_pair->tx);if (result) -return result; + goto free_epbuf; result =fjes_hw_alloc_epbuf (&buf_pair->rx);if (result) -return result; + goto free_epbuf;spin_lock_irqsave (&hw->rx_status_lock, flags); fjes_hw_setup_epbuf(&buf_pair->tx, mac,
@@ -273,6 +277,25 @@ static int fjes_hw_setup(struct fjes_hw *hw)fjes_hw_init_command_registers (hw, ¶m);return 0 ; + +free_epbuf: +for (epidx =0 ; epidx < hw->max_epid ; epidx++) { +if (epidx == hw->my_epid) +continue ; +fjes_hw_free_epbuf (&hw->ep_shm_info[epidx].tx); +fjes_hw_free_epbuf (&hw->ep_shm_info[epidx].rx); + } +fjes_hw_free_shared_status_region (hw); +free_res_buf: +kfree (hw->hw_info.res_buf); + hw->hw_info.res_buf =NULL ; +free_req_buf: +kfree (hw->hw_info.req_buf); + hw->hw_info.req_buf =NULL ; +free_ep_info: +kfree (hw->ep_shm_info); + hw->ep_shm_info =NULL ; +return result; }static void fjes_hw_cleanup (struct fjes_hw *hw)
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 226/331 selftests: bonding: do not test arp/ns target with mode balance-alb/tlb
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Hangbin Liu <[email protected]>
[ Upstream commit a2933a8759a62269754e54733d993b19de870e84 ]
The prio_arp/ns tests hard code the mode to active-backup. At the same time, The balance-alb/tlb modes do not support arp/ns target. So remove the prio_arp/ns tests from the loop and only test active-backup mode.
--- .../testing/selftests/drivers/net/bonding/bond_options.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tools/testing/selftests/drivers/net/bonding/bond_options.sh +4 -4
@@ -162,7 +162,7 @@ prio_arp() local mode=$1 for primary_reselect in 0 1 2; do - prio_test "mode active-backup arp_interval 100 arp_ip_target ${g_ip4} primary eth1 primary_reselect $primary_reselect" + prio_test "mode $mode arp_interval 100 arp_ip_target ${g_ip4} primary eth1 primary_reselect $primary_reselect" log_test "prio" "$mode arp_ip_target primary_reselect $primary_reselect" done }
@@ -178,7 +178,7 @@ prio_ns() fi for primary_reselect in 0 1 2; do - prio_test "mode active-backup arp_interval 100 ns_ip6_target ${g_ip6} primary eth1 primary_reselect $primary_reselect" + prio_test "mode $mode arp_interval 100 ns_ip6_target ${g_ip6} primary eth1 primary_reselect $primary_reselect" log_test "prio" "$mode ns_ip6_target primary_reselect $primary_reselect" done }
@@ -194,9 +194,9 @@ prio() for mode in $modes; do prio_miimon $mode - prio_arp $mode - prio_ns $mode done + prio_arp "active-backup" + prio_ns "active-backup" } arp_validate_test()
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 227/331 net: fec: fix the unhandled context fault from smmu
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Shenwei Wang <[email protected]>
[ Upstream commit 5e344807735023cd3a67c37a1852b849caa42620 ]
When repeatedly changing the interface link speed using the command below:
ethtool -s eth0 speed 100 duplex full
ethtool -s eth0 speed 1000 duplex full
The following errors may sometimes be reported by the ARM SMMU driver:
[ 5395.035364] fec 5b040000.ethernet eth0: Link is Down [ 5395.039255] arm-smmu 51400000.iommu: Unhandled context fault: fsr=0x402, iova=0x00000000, fsynr=0x100001, cbfrsynra=0x852, cb=2 [ 5398.108460] fec 5b040000.ethernet eth0: Link is Up - 100Mbps/Full -
flow control off
It is identified that the FEC driver does not properly stop the TX queue during the link speed transitions, and this results in the invalid virtual I/O address translations from the SMMU and causes the context faults.
--- drivers/net/ethernet/freescale/fec_main.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/ethernet/freescale/fec_main.c +2 -0
@@ -2011,6 +2011,7 @@ static void fec_enet_adjust_link(struct net_device *ndev)/* if any of the above changed restart the FEC */ if (status_change) { +netif_stop_queue (ndev);napi_disable (&fep->napi);netif_tx_lock_bh (ndev);fec_restart (ndev);
@@ -2020,6 +2021,7 @@ static void fec_enet_adjust_link(struct net_device *ndev) } }else {if (fep->link) { +netif_stop_queue (ndev);napi_disable (&fep->napi);netif_tx_lock_bh (ndev);fec_stop (ndev);
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 228/331 tsnep: Remove FCS for XDP data path
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Gerhard Engleder <[email protected]>
[ Upstream commit 50bad6f797d4d501c5ef416a6f92e1912ab5aa8b ]
The RX data buffer includes the FCS. The FCS is already stripped for the normal data path. But for the XDP data path the FCS is included and acts like additional/useless data.
Remove the FCS from the RX data buffer also for XDP.
--- drivers/net/ethernet/engleder/tsnep_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/engleder/tsnep_main.c +2 -2
@@ -1434,7 +1434,7 @@ static int tsnep_rx_poll(struct tsnep_rx *rx, struct napi_struct *napi,xdp_prepare_buff (&xdp,page_address (entry->page),XDP_PACKET_HEADROOM +TSNEP_RX_INLINE_METADATA_SIZE , - length, false); + length -ETH_FCS_LEN , false); consume =tsnep_xdp_run_prog (rx, prog, &xdp, &xdp_status, tx_nq, tx);
@@ -1517,7 +1517,7 @@ static int tsnep_rx_poll_zc(struct tsnep_rx *rx, struct napi_struct *napi,prefetch (entry->xdp->data); length =__le32_to_cpu (entry->desc_wb->properties) &TSNEP_DESC_LENGTH_MASK ; -xsk_buff_set_size (entry->xdp, length); +xsk_buff_set_size (entry->xdp, length -ETH_FCS_LEN );xsk_buff_dma_sync_for_cpu (entry->xdp, rx->xsk_pool);/* RX metadata with timestamps is in front of actual data,
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 229/331 tsnep: Fix XDP_RING_NEED_WAKEUP for empty fill ring
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Gerhard Engleder <[email protected]>
[ Upstream commit 9a91c05f4bd6f6bdd6b8f90445e0da92e3ac956c ]
The fill ring of the XDP socket may contain not enough buffers to completey fill the RX queue during socket creation. In this case the flag XDP_RING_NEED_WAKEUP is not set as this flag is only set if the RX queue is not completely filled during polling.
Set XDP_RING_NEED_WAKEUP flag also if RX queue is not completely filled during XDP socket creation.
--- drivers/net/ethernet/engleder/tsnep_main.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
drivers/net/ethernet/engleder/tsnep_main.c +13 -0
@@ -1711,6 +1711,19 @@ static void tsnep_rx_reopen_xsk(struct tsnep_rx *rx) allocated--; } } + +/* set need wakeup flag immediately if ring is not filled completely, +* first polling would be too late as need wakeup signalisation would +* be delayed for an indefinite time +*/ +if (xsk_uses_need_wakeup (rx->xsk_pool)) { +int desc_available =tsnep_rx_desc_available (rx); + +if (desc_available) +xsk_set_rx_need_wakeup (rx->xsk_pool); +else +xsk_clear_rx_need_wakeup (rx->xsk_pool); + } }static bool tsnep_pending (struct tsnep_queue *queue)
Signature · 1 line
2.43.0
Greg Kroah-Hartman
maintainerPATCH 230/331 btrfs: scrub: avoid use-after-free when chunk length is not 64K aligned
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Qu Wenruo <[email protected]>
commit f546c4282673497a06ecb6190b50ae7f6c85b02f upstream.
[BUG]
There is a bug report that, on a ext4-converted btrfs, scrub leads to various problems, including:
- "unable to find chunk map" errors
BTRFS info (device vdb): scrub: started on devid 1
BTRFS critical (device vdb): unable to find chunk map for logical 2214744064 length 4096 BTRFS critical (device vdb): unable to find chunk map for logical 2214744064 length 45056
This would lead to unrepariable errors.
- Use-after-free KASAN reports:
==================================================================
BUG: KASAN: slab-use-after-free in __blk_rq_map_sg+0x18f/0x7c0 Read of size 8 at addr ffff8881013c9040 by task btrfs/909 CPU: 0 PID: 909 Comm: btrfs Not tainted 6.7.0-x64v3-dbg #11 c50636e9419a8354555555245df535e380563b2b Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 2023.11-2 12/24/2023 Call Trace: <TASK> dump_stack_lvl+0x43/0x60 print_report+0xcf/0x640 kasan_report+0xa6/0xd0 __blk_rq_map_sg+0x18f/0x7c0 virtblk_prep_rq.isra.0+0x215/0x6a0 [virtio_blk 19a65eeee9ae6fcf02edfad39bb9ddee07dcdaff] virtio_queue_rqs+0xc4/0x310 [virtio_blk 19a65eeee9ae6fcf02edfad39bb9ddee07dcdaff] blk_mq_flush_plug_list.part.0+0x780/0x860 __blk_flush_plug+0x1ba/0x220 blk_finish_plug+0x3b/0x60 submit_initial_group_read+0x10a/0x290 [btrfs e57987a360bed82fe8756dcd3e0de5406ccfe965] flush_scrub_stripes+0x38e/0x430 [btrfs e57987a360bed82fe8756dcd3e0de5406ccfe965] scrub_stripe+0x82a/0xae0 [btrfs e57987a360bed82fe8756dcd3e0de5406ccfe965] scrub_chunk+0x178/0x200 [btrfs e57987a360bed82fe8756dcd3e0de5406ccfe965] scrub_enumerate_chunks+0x4bc/0xa30 [btrfs e57987a360bed82fe8756dcd3e0de5406ccfe965] btrfs_scrub_dev+0x398/0x810 [btrfs e57987a360bed82fe8756dcd3e0de5406ccfe965] btrfs_ioctl+0x4b9/0x3020 [btrfs e57987a360bed82fe8756dcd3e0de5406ccfe965] __x64_sys_ioctl+0xbd/0x100 do_syscall_64+0x5d/0xe0 entry_SYSCALL_64_after_hwframe+0x63/0x6b RIP: 0033:0x7f47e5e0952b
- Crash, mostly due to above use-after-free
[CAUSE]
The converted fs has the following data chunk layout:
item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 2214658048) itemoff 16025 itemsize 80
length 86016 owner 2 stripe_len 65536 type DATA|single
For above logical bytenr 2214744064, it's at the chunk end (2214658048 + 86016 = 2214744064).
This means btrfs_submit_bio() would split the bio, and trigger endio function for both of the two halves.
However scrub_submit_initial_read() would only expect the endio function to be called once, not any more.
This means the first endio function would already free the bbio::bio, leaving the bvec freed, thus the 2nd endio call would lead to use-after-free.
[FIX]
- Make sure scrub_read_endio() only updates bits in its range Since we may read less than 64K at the end of the chunk, we should not touch the bits beyond chunk boundary.
- Make sure scrub_submit_initial_read() only to read the chunk range This is done by calculating the real number of sectors we need to read, and add sector-by-sector to the bio.
Thankfully the scrub read repair path won't need extra fixes:
- scrub_stripe_submit_repair_read()
With above fixes, we won't update error bit for range beyond chunk, thus scrub_stripe_submit_repair_read() should never submit any read beyond the chunk.
--- fs/btrfs/scrub.c | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)
fs/btrfs/scrub.c +22 -7
@@ -1099,12 +1099,22 @@ out:static void scrub_read_endio (struct btrfs_bio *bbio) {struct scrub_stripe *stripe = bbio->private; +struct bio_vec *bvec; +int sector_nr =calc_sector_number (stripe,bio_first_bvec_all (&bbio->bio)); +int num_sectors; +u32 bio_size =0 ; +int i; + +ASSERT (sector_nr < stripe->nr_sectors); +bio_for_each_bvec_all (bvec, &bbio->bio, i) + bio_size += bvec->bv_len; + num_sectors = bio_size >> stripe->bg->fs_info->sectorsize_bits;if (bbio->bio.bi_status) { -bitmap_set (&stripe->io_error_bitmap,0 , stripe->nr_sectors); -bitmap_set (&stripe->error_bitmap,0 , stripe->nr_sectors); +bitmap_set (&stripe->io_error_bitmap, sector_nr, num_sectors); +bitmap_set (&stripe->error_bitmap, sector_nr, num_sectors); }else { -bitmap_clear (&stripe->io_error_bitmap,0 , stripe->nr_sectors); +bitmap_clear (&stripe->io_error_bitmap, sector_nr, num_sectors); }bio_put (&bbio->bio);if (atomic_dec_and_test (&stripe->pending_io)) {
@@ -1640,6 +1650,9 @@ static void scrub_submit_initial_read(st {struct btrfs_fs_info *fs_info = sctx->fs_info;struct btrfs_bio *bbio; +unsigned int nr_sectors =min_t (u64,BTRFS_STRIPE_LEN , stripe->bg->start + + stripe->bg->length - stripe->logical) >> + fs_info->sectorsize_bits;int mirror = stripe->mirror_num;ASSERT (stripe->bg);
@@ -1649,14 +1662,16 @@ static void scrub_submit_initial_read(st bbio =btrfs_bio_alloc (SCRUB_STRIPE_PAGES ,REQ_OP_READ , fs_info, scrub_read_endio, stripe); -/* Read the whole stripe. */ bbio->bio.bi_iter.bi_sector = stripe->logical >>SECTOR_SHIFT ; -for (int i =0 ; i <BTRFS_STRIPE_LEN >>PAGE_SHIFT ; i++) { +/* Read the whole range inside the chunk boundary. */ +for (unsigned int cur =0 ; cur < nr_sectors; cur++) { +struct page *page =scrub_stripe_get_page (stripe, cur); +unsigned int pgoff =scrub_stripe_get_page_offset (stripe, cur);int ret; - ret =bio_add_page (&bbio->bio, stripe->pages[i],PAGE_SIZE ,0 ); + ret =bio_add_page (&bbio->bio, page, fs_info->sectorsize, pgoff);/* We should have allocated enough bio vectors. */ -ASSERT (ret ==PAGE_SIZE ); +ASSERT (ret == fs_info->sectorsize); }atomic_inc (&stripe->pending_io);
Greg Kroah-Hartman
maintainerPATCH 231/331 btrfs: zoned: fix lock ordering in btrfs_zone_activate()
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Naohiro Aota <[email protected]>
commit b18f3b60b35a8c01c9a2a0f0d6424c6d73971dc3 upstream.
The btrfs CI reported a lockdep warning as follows by running generic generic/129.
WARNING: possible circular locking dependency detected
6.7.0-rc5+ #1 Not tainted
------------------------------------------------------
kworker/u5:5/793427 is trying to acquire lock:
ffff88813256d028 (&cache->lock){+.+.}-{2:2}, at: btrfs_zone_finish_one_bg+0x5e/0x130 but task is already holding lock:
ffff88810a23a318 (&fs_info->zone_active_bgs_lock){+.+.}-{2:2}, at: btrfs_zone_finish_one_bg+0x34/0x130 which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #1 (&fs_info->zone_active_bgs_lock){+.+.}-{2:2}:
...
-> #0 (&cache->lock){+.+.}-{2:2}:
...
This is because we take fs_info->zone_active_bgs_lock after a block_group's lock in btrfs_zone_activate() while doing the opposite in other places.
Fix the issue by expanding the fs_info->zone_active_bgs_lock's critical section and taking it before a block_group's lock.
--- fs/btrfs/zoned.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
fs/btrfs/zoned.c +2 -6
@@ -1975,6 +1975,7 @@ bool btrfs_zone_activate(struct btrfs_bl map = block_group->physical_map; +spin_lock (&fs_info->zone_active_bgs_lock);spin_lock (&block_group->lock);if (test_bit (BLOCK_GROUP_FLAG_ZONE_IS_ACTIVE , &block_group->runtime_flags)) { ret = true;
@@ -1987,7 +1988,6 @@ bool btrfs_zone_activate(struct btrfs_bl goto out_unlock; } -spin_lock (&fs_info->zone_active_bgs_lock);for (i =0 ; i < map->num_stripes; i++) {struct btrfs_zoned_device_info *zinfo;int reserved =0 ;
@@ -2007,20 +2007,17 @@ bool btrfs_zone_activate(struct btrfs_bl*/ if (atomic_read (&zinfo->active_zones_left) <= reserved) { ret = false; -spin_unlock (&fs_info->zone_active_bgs_lock); goto out_unlock; }if (!btrfs_dev_set_active_zone (device, physical)) {/* Cannot activate the zone */ ret = false; -spin_unlock (&fs_info->zone_active_bgs_lock); goto out_unlock; }if (!is_data) zinfo->reserved_active_zones--; } -spin_unlock (&fs_info->zone_active_bgs_lock);/* Successfully activated all the zones */ set_bit (BLOCK_GROUP_FLAG_ZONE_IS_ACTIVE , &block_group->runtime_flags);
@@ -2028,8 +2025,6 @@ bool btrfs_zone_activate(struct btrfs_bl/* For the active block group list */ btrfs_get_block_group (block_group); - -spin_lock (&fs_info->zone_active_bgs_lock);list_add_tail (&block_group->active_bg_list, &fs_info->zone_active_bgs);spin_unlock (&fs_info->zone_active_bgs_lock);
@@ -2037,6 +2032,7 @@ bool btrfs_zone_activate(struct btrfs_bl out_unlock:spin_unlock (&block_group->lock); +spin_unlock (&fs_info->zone_active_bgs_lock);return ret; }
Greg Kroah-Hartman
maintainerPATCH 232/331 btrfs: avoid copying BTRFS_ROOT_SUBVOL_DEAD flag to snapshot of subvolume being deleted
In reply to Greg Kroah-Hartman6.6-stable review patch. If anyone has any objections, please let me know.
------------------
From: Omar Sandoval <[email protected]>
commit 3324d0547861b16cf436d54abba7052e0c8aa9de upstream.
Sweet Tea spotted a race between subvolume deletion and snapshotting that can result in the root item for the snapshot having the BTRFS_ROOT_SUBVOL_DEAD flag set. The race is:
Thread 1 | Thread 2
----------------------------------------------|----------
btrfs_delete_subvolume |
btrfs_set_root_flags(BTRFS_ROOT_SUBVOL_DEAD)|
|btrfs_mksubvol
| down_read(subvol_sem)
| create_snapshot
| ...
| create_pending_snapshot
| copy root item from source
down_write(subvol_sem) |This flag is only checked in send and swap activate, which this would cause to fail mysteriously.
create_snapshot() now checks the root refs to reject a deleted subvolume, so we can fix this by locking subvol_sem earlier so that the BTRFS_ROOT_SUBVOL_DEAD flag and the root refs are updated atomically.
--- fs/btrfs/inode.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)
fs/btrfs/inode.c +13 -9
@@ -4447,6 +4447,8 @@ int btrfs_delete_subvolume(struct btrfs_u64 root_flags;int ret; +down_write (&fs_info->subvol_sem); +/* * Don't allow to delete a subvolume with send in progress. This is * inside the inode lock so the error handling that has to drop the bit
@@ -4458,25 +4460,25 @@ int btrfs_delete_subvolume(struct btrfs_btrfs_warn (fs_info,"attempt to delete subvolume %llu during send" , dest->root_key.objectid); -return -EPERM ; + ret = -EPERM ; + goto out_up_write; }if (atomic_read (&dest->nr_swapfiles)) {spin_unlock (&dest->root_item_lock);btrfs_warn (fs_info,"attempt to delete subvolume %llu with active swapfile" , root->root_key.objectid); -return -EPERM ; + ret = -EPERM ; + goto out_up_write; } root_flags =btrfs_root_flags (&dest->root_item);btrfs_set_root_flags (&dest->root_item, root_flags |BTRFS_ROOT_SUBVOL_DEAD );spin_unlock (&dest->root_item_lock); -down_write (&fs_info->subvol_sem); - ret =may_destroy_subvol (dest);if (ret) - goto out_up_write; + goto out_undead;btrfs_init_block_rsv (&block_rsv,BTRFS_BLOCK_RSV_TEMP );/*
@@ -4486,7 +4488,7 @@ int btrfs_delete_subvolume(struct btrfs_*/ ret =btrfs_subvolume_reserve_metadata (root, &block_rsv,5 , true);if (ret) - goto out_up_write; + goto out_undead; trans =btrfs_start_transaction (root,0 );if (IS_ERR (trans)) {
@@ -4552,15 +4554,17 @@ out_end_trans: inode->i_flags |=S_DEAD ; out_release:btrfs_subvolume_release_metadata (root, &block_rsv); -out_up_write: -up_write (&fs_info->subvol_sem); +out_undead:if (ret) {spin_lock (&dest->root_item_lock); root_flags =btrfs_root_flags (&dest->root_item);btrfs_set_root_flags (&dest->root_item, root_flags & ~BTRFS_ROOT_SUBVOL_DEAD );spin_unlock (&dest->root_item_lock); - }else { + } +out_up_write: +up_write (&fs_info->subvol_sem); +if (!ret) {d_invalidate (dentry);btrfs_prune_dentries (dest);ASSERT (dest->send_in_progress ==0 );