x86/math64: handle #DE in mul_u64_u64_div_u64()
Subsystems (3)
v3 cover x86/math64: handle #DE in mul_u64_u64_div_u64()
Hello,
Changes since V2:
1/2 - added the acks from David and Li (thank you) 2/2 - new and trivial
Oleg.
---
arch/x86/include/asm/div64.h | 13 ++++++++----- block/blk-throttle.c | 6 ------ 2 files changed, 8 insertions(+), 11 deletions(-)
PATCH v3 1/2 x86/math64: handle #DE in mul_u64_u64_div_u64()
Change mul_u64_u64_div_u64() to return ULONG_MAX if the result doesn't fit into u64 or div == 0. The former matches the generic implementation in lib/math/div64.c, the latter doesn't. Perhaps we will add a WARN() into the fixup_exception() paths later.
No need to use _ASM_EXTABLE_TYPE_REG(), we know that the target register is pt_regs->ax with offset == 0, so a simple EX_DATA_REG(0) should work just fine.
--- arch/x86/include/asm/div64.h | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
arch/x86/include/asm/div64.h +8 -5
@@ -79,18 +79,21 @@ static inline u64 mul_u32_u32(u32 a, u32 b)#else # include <asm-generic/div64.h> +# include <asm/asm.h> /* -* Will generate an #DE when the result doesn't fit u64, could fix with an -* __ex_table[] entry when it becomes an issue. +* Returns U64_MAX if the result doesn't fit u64 or div == 0. */ static inline u64 mul_u64_u64_div_u64 (u64 a,u64 mul,u64 div) {u64 q; - asm ("mulq %2; divq %3" :"=a" (q) - :"a" (a),"rm" (mul),"rm" (div) - :"rdx" ); + asm ("mulq %2; 1: divq %3; 2:\n" + _ASM_EXTABLE_TYPE(1b, 2b, + EX_TYPE_IMM_REG | EX_DATA_REG(0) | EX_DATA_IMM(-1)) + : "=a" (q) + : "a" (a), "rm" (mul), "rm" (div) + : "rdx");return q; }
Signature · 1 line
2.25.1.362.g51ebf55
PATCH v3 2/2 blk-throttle: kill the no longer needed overflow check in calculate_bytes_allowed()
In reply to Oleg NesterovNow that mul_u64_u64_div_u64() can't crash there is no need to check for possible overflow in calculate_bytes_allowed().
--- block/blk-throttle.c | 6 ------ 1 file changed, 6 deletions(-)
block/blk-throttle.c +0 -6
@@ -601,12 +601,6 @@ static unsigned int calculate_io_allowed(u32 iops_limit,static u64 calculate_bytes_allowed (u64 bps_limit,unsigned long jiffy_elapsed) { -/* -* Can result be wider than 64 bits? -* We check against 62, not 64, due to ilog2 truncation. -*/ -if (ilog2 (bps_limit) +ilog2 (jiffy_elapsed) -ilog2 (HZ ) >62 ) -return U64_MAX ;return mul_u64_u64_div_u64 (bps_limit, (u64 )jiffy_elapsed, (u64 )HZ ); }
Signature · 1 line
2.25.1.362.g51ebf55
Re: [PATCH v3 2/2] blk-throttle: kill the no longer needed overflow check in calculate_bytes_allowed()
Quoted 13 lines · On Fri, 15 Aug 2025 18:41:02 +0200 Oleg Nesterov <[email protected]> wrote:
Now that
mul_u64_u64_div_u64()can't crash there is no need to check for possible overflow incalculate_bytes_allowed().Signed-off-by Oleg Nesterov <[email protected]>--- block/blk-throttle.c | 6 ------ 1 file changed, 6 deletions(-)block/blk-throttle.c +0
-0
block/blk-throttle.c static unsigned int calculate_io_allowed(u32 iops_limit, · line 604
Quoted 10 lines
block/blk-throttle.c +0
-6@@ -601,12 +601,6 @@ static unsigned int calculate_io_allowed(u32 iops_limit,static u64 calculate_bytes_allowed (u64 bps_limit,unsigned long jiffy_elapsed) { -/* -* Can result be wider than 64 bits? -* We check against 62, not 64, due to ilog2 truncation. -*/ -if (ilog2 (bps_limit) +ilog2 (jiffy_elapsed) -ilog2 (HZ ) >62 ) -return U64_MAX ;return mul_u64_u64_div_u64 (bps_limit, (u64 )jiffy_elapsed, (u64 )HZ );
Not directly related, but the two (u64) casts are pointless and can be removed.
David
Quoted 2 lines
block/blk-throttle.c +0
-0}
On Fri, 15 Aug 2025 18:40:09 +0200
Oleg Nesterov <[email protected]> wrote:
One of my 'idea patches' is to make mul_u64_u64_div_u64() a wrapper for another function that takes in extra 'int *overflowed' parameter that is set zero/non-zero for success/overflow.
The 'overflowed' parameter can either be a compile-time NULL or a valid pointer.
So the x86-x64 asm implementation would use different code - you need the 'jump around fail label' to write the ~0 return value to *overflowed. The extra pointer check in the C version normal path may not be worth worrying about (but the '*overflow = 0' could easily be inlined).
The typical use would be:
quotient = mul_u64_u64_div_u64_overflow(..., &overflowed); if (quotient == ~0ull && overflowed) ...
That will generate better code than returning 'overflowed' and the quotient by reference.
Although I wonder how often ~0ull is a valid result?
David
H. Peter Anvin
reviewerRe: [PATCH v3 2/2] blk-throttle: kill the no longer needed overflow check in calculate_bytes_allowed()
In reply to David LaightQuoted 29 lines · David Laight
Quoted 13 lines · On Fri, 15 Aug 2025 18:41:02 +0200 Oleg Nesterov <[email protected]> wrote:
Now that
mul_u64_u64_div_u64()can't crash there is no need to check for possible overflow incalculate_bytes_allowed().Signed-off-by Oleg Nesterov <[email protected]>--- block/blk-throttle.c | 6 ------ 1 file changed, 6 deletions(-)block/blk-throttle.c +0
-0block/blk-throttle.c static unsigned int calculate_io_allowed(u32 iops_limit, · line 604
Quoted 10 lines
block/blk-throttle.c +0
-6@@ -601,12 +601,6 @@ static unsigned int calculate_io_allowed(u32 iops_limit,static u64 calculate_bytes_allowed (u64 bps_limit,unsigned long jiffy_elapsed) { -/* -* Can result be wider than 64 bits? -* We check against 62, not 64, due to ilog2 truncation. -*/ -if (ilog2 (bps_limit) +ilog2 (jiffy_elapsed) -ilog2 (HZ ) >62 ) -return U64_MAX ;return mul_u64_u64_div_u64 (bps_limit, (u64 )jiffy_elapsed, (u64 )HZ );Not directly related, but the two (u64) casts are pointless and can be removed.
Quoted 5 lines
David
Quoted 2 lines
block/blk-throttle.c +0
-0}
It's also rather broken, because a division with a constant can be implemented as a multiply, and both gcc and clang knows how to do that.
David,
We had a lengthy discussion and you have already acked this fix.
I thought that we agreed on that a) we need to fix the problem first and b) x86 version should be consistent with the generic implementation regarding ~0ull on overflow.
Can we finally merge this fix, then discuss the possible improvements and possibly change both implementation?
Oleg.
Quoted 26 lines · David Laight
On Fri, 15 Aug 2025 18:40:09 +0200
Oleg Nesterov <[email protected]> wrote:One of my 'idea patches' is to make
mul_u64_u64_div_u64()a wrapper for another function that takes in extra 'int *overflowed' parameter that is set zero/non-zero for success/overflow.
The 'overflowed' parameter can either be a compile-time NULL or a valid pointer.So the x86-x64 asm implementation would use different code - you need the 'jump around fail label' to write the ~0 return value to *overflowed. The extra pointer check in the C version normal path may not be worth worrying about (but the '*overflow = 0' could easily be inlined).
The typical use would be:
quotient = mul_u64_u64_div_u64_overflow(..., &overflowed); if (quotient == ~0ull && overflowed) ...That will generate better code than returning 'overflowed' and the quotient by reference.
Although I wonder how often ~0ull is a valid result?
David
Re: [PATCH v3 2/2] blk-throttle: kill the no longer needed overflow check in calculate_bytes_allowed()
In reply to David LaightQuoted 18 lines · David Laight
Quoted 14 lines · On Fri, 15 Aug 2025 18:41:02 +0200 Oleg Nesterov <[email protected]> wrote:
Now that
mul_u64_u64_div_u64()can't crash there is no need to check for possible overflow incalculate_bytes_allowed().Signed-off-by Oleg Nesterov <[email protected]>--- block/blk-throttle.c | 6 ------ 1 file changed, 6 deletions(-)block/blk-throttle.c +0
-0@@ -601,12 +601,6 @@ static unsigned int calculate_io_allowed(u32 iops_limit,
Quoted 11 lines
block/blk-throttle.c static unsigned int calculate_io_allowed(u32 iops_limit, · line 604
Quoted 9 lines
block/blk-throttle.c +0
-6static u64 calculate_bytes_allowed (u64 bps_limit,unsigned long jiffy_elapsed) { -/* -* Can result be wider than 64 bits? -* We check against 62, not 64, due to ilog2 truncation. -*/ -if (ilog2 (bps_limit) +ilog2 (jiffy_elapsed) -ilog2 (HZ ) >62 ) -return U64_MAX ;return mul_u64_u64_div_u64 (bps_limit, (u64 )jiffy_elapsed, (u64 )HZ );Not directly related, but the two (u64) casts are pointless and can be removed.
Yeah... I only reverted 2dd710d476f2 ("blk-throttle: check for overflow in calculate_bytes_allowed")
Oleg.
Quoted 10 lines · On Mon, 18 Aug 2025 14:39:01 +0200 Oleg Nesterov <[email protected]> wrote:
David,
We had a lengthy discussion and you have already acked this fix.
I thought that we agreed on that a) we need to fix the problem first and b) x86 version should be consistent with the generic implementation regarding ~0ull on overflow.
Can we finally merge this fix, then discuss the possible improvements and possibly change both implementation?
I deliberately put this comment on 0/2 because it is 'future thought'. I didn't want to delay the patch going in.
David
Quoted 31 lines
Oleg.
Quoted 26 lines · David Laight
On Fri, 15 Aug 2025 18:40:09 +0200
Oleg Nesterov <[email protected]> wrote:One of my 'idea patches' is to make
mul_u64_u64_div_u64()a wrapper for another function that takes in extra 'int *overflowed' parameter that is set zero/non-zero for success/overflow.
The 'overflowed' parameter can either be a compile-time NULL or a valid pointer.So the x86-x64 asm implementation would use different code - you need the 'jump around fail label' to write the ~0 return value to *overflowed. The extra pointer check in the C version normal path may not be worth worrying about (but the '*overflow = 0' could easily be inlined).
The typical use would be:
quotient = mul_u64_u64_div_u64_overflow(..., &overflowed); if (quotient == ~0ull && overflowed) ...That will generate better code than returning 'overflowed' and the quotient by reference.
Although I wonder how often ~0ull is a valid result?
David