Skip to contentSkip to conversation

x86/math64: handle #DE in mul_u64_u64_div_u64()

PATCH In review Oleg Nesterov 3 participants 9 messages Aug 15, 2025 – Aug 18, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v3 cover x86/math64: handle #DE in mul_u64_u64_div_u64()

Hello,

Changes since V2:

1/2 - added the acks from David and Li (thank you) 2/2 - new and trivial

Oleg.
---

 arch/x86/include/asm/div64.h | 13 ++++++++-----
 block/blk-throttle.c         |  6 ------
 2 files changed, 8 insertions(+), 11 deletions(-)
#1 lore raw

PATCH v3 1/2 x86/math64: handle #DE in mul_u64_u64_div_u64()

Change mul_u64_u64_div_u64() to return ULONG_MAX if the result doesn't fit into u64 or div == 0. The former matches the generic implementation in lib/math/div64.c, the latter doesn't. Perhaps we will add a WARN() into the fixup_exception() paths later.

No need to use _ASM_EXTABLE_TYPE_REG(), we know that the target register is pt_regs->ax with offset == 0, so a simple EX_DATA_REG(0) should work just fine.

---
 arch/x86/include/asm/div64.h | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
arch/x86/include/asm/div64.h +8 -5
index 9931e4c7d73f..c2532d38ed0a 100644
@@ -79,18 +79,21 @@ static inline u64 mul_u32_u32(u32 a, u32 b)
 
 #else
 # include <asm-generic/div64.h>
+# include <asm/asm.h>
 
 /*
- * Will generate an #DE when the result doesn't fit u64, could fix with an
- * __ex_table[] entry when it becomes an issue.
+ * Returns U64_MAX if the result doesn't fit u64 or div == 0.
  */
 static inline u64 mul_u64_u64_div_u64(u64 a, u64 mul, u64 div)
 {
 	u64 q;
 
-	asm ("mulq %2; divq %3" : "=a" (q)
-				: "a" (a), "rm" (mul), "rm" (div)
-				: "rdx");
+	asm ("mulq %2; 1: divq %3; 2:\n"
+		_ASM_EXTABLE_TYPE(1b, 2b,
+			EX_TYPE_IMM_REG | EX_DATA_REG(0) | EX_DATA_IMM(-1))
+		: "=a" (q)
+		: "a" (a), "rm" (mul), "rm" (div)
+		: "rdx");
 
 	return q;
 }
Signature · 1 line
2.25.1.362.g51ebf55
#2 lore raw

PATCH v3 2/2 blk-throttle: kill the no longer needed overflow check in calculate_bytes_allowed()

In reply to Oleg Nesterov

Now that mul_u64_u64_div_u64() can't crash there is no need to check for possible overflow in calculate_bytes_allowed().

---
 block/blk-throttle.c | 6 ------
 1 file changed, 6 deletions(-)
block/blk-throttle.c +0 -6
index 397b6a410f9e..66339e22cc85 100644
@@ -601,12 +601,6 @@ static unsigned int calculate_io_allowed(u32 iops_limit,
 
 static u64 calculate_bytes_allowed(u64 bps_limit, unsigned long jiffy_elapsed)
 {
-	/*
-	 * Can result be wider than 64 bits?
-	 * We check against 62, not 64, due to ilog2 truncation.
-	 */
-	if (ilog2(bps_limit) + ilog2(jiffy_elapsed) - ilog2(HZ) > 62)
-		return U64_MAX;
 	return mul_u64_u64_div_u64(bps_limit, (u64)jiffy_elapsed, (u64)HZ);
 }
 
Signature · 1 line
2.25.1.362.g51ebf55
#3 lore raw

Re: [PATCH v3 2/2] blk-throttle: kill the no longer needed overflow check in calculate_bytes_allowed()

Quoted 13 lines · On Fri, 15 Aug 2025 18:41:02 +0200 Oleg Nesterov <[email protected]> wrote:

Now that mul_u64_u64_div_u64() can't crash there is no need to check for possible overflow in calculate_bytes_allowed().

---
 block/blk-throttle.c | 6 ------
 1 file changed, 6 deletions(-)
block/blk-throttle.c +0 -0
index 397b6a410f9e..66339e22cc85 100644

block/blk-throttle.c static unsigned int calculate_io_allowed(u32 iops_limit, · line 604

Quoted 10 lines
block/blk-throttle.c +0 -6
@@ -601,12 +601,6 @@ static unsigned int calculate_io_allowed(u32 iops_limit,
 
 static u64 calculate_bytes_allowed(u64 bps_limit, unsigned long jiffy_elapsed)
 {
-	/*
-	 * Can result be wider than 64 bits?
-	 * We check against 62, not 64, due to ilog2 truncation.
-	 */
-	if (ilog2(bps_limit) + ilog2(jiffy_elapsed) - ilog2(HZ) > 62)
-		return U64_MAX;
 	return mul_u64_u64_div_u64(bps_limit, (u64)jiffy_elapsed, (u64)HZ);

Not directly related, but the two (u64) casts are pointless and can be removed.

David

Quoted 2 lines
block/blk-throttle.c +0 -0
 }
 
#4 lore raw

Re: [PATCH v3 2/2] blk-throttle: kill the no longer needed overflow check in calculate_bytes_allowed()

Quoted 29 lines · David Laight
Quoted 13 lines · On Fri, 15 Aug 2025 18:41:02 +0200 Oleg Nesterov <[email protected]> wrote:

Now that mul_u64_u64_div_u64() can't crash there is no need to check for possible overflow in calculate_bytes_allowed().

---
 block/blk-throttle.c | 6 ------
 1 file changed, 6 deletions(-)
block/blk-throttle.c +0 -0
index 397b6a410f9e..66339e22cc85 100644

block/blk-throttle.c static unsigned int calculate_io_allowed(u32 iops_limit, · line 604

Quoted 10 lines
block/blk-throttle.c +0 -6
@@ -601,12 +601,6 @@ static unsigned int calculate_io_allowed(u32 iops_limit,
 
 static u64 calculate_bytes_allowed(u64 bps_limit, unsigned long jiffy_elapsed)
 {
-	/*
-	 * Can result be wider than 64 bits?
-	 * We check against 62, not 64, due to ilog2 truncation.
-	 */
-	if (ilog2(bps_limit) + ilog2(jiffy_elapsed) - ilog2(HZ) > 62)
-		return U64_MAX;
 	return mul_u64_u64_div_u64(bps_limit, (u64)jiffy_elapsed, (u64)HZ);

Not directly related, but the two (u64) casts are pointless and can be removed.

Quoted 5 lines

David

Quoted 2 lines
block/blk-throttle.c +0 -0
 }
 

It's also rather broken, because a division with a constant can be implemented as a multiply, and both gcc and clang knows how to do that.

#5 lore raw

Re: [PATCH v3 2/2] blk-throttle: kill the no longer needed overflow check in calculate_bytes_allowed()

In reply to David Laight
Quoted 18 lines · David Laight
Quoted 14 lines · On Fri, 15 Aug 2025 18:41:02 +0200 Oleg Nesterov <[email protected]> wrote:

Now that mul_u64_u64_div_u64() can't crash there is no need to check for possible overflow in calculate_bytes_allowed().

---
 block/blk-throttle.c | 6 ------
 1 file changed, 6 deletions(-)
block/blk-throttle.c +0 -0
index 397b6a410f9e..66339e22cc85 100644
@@ -601,12 +601,6 @@ static unsigned int calculate_io_allowed(u32 iops_limit,
 
Quoted 11 lines

block/blk-throttle.c static unsigned int calculate_io_allowed(u32 iops_limit, · line 604

Quoted 9 lines
block/blk-throttle.c +0 -6
 static u64 calculate_bytes_allowed(u64 bps_limit, unsigned long jiffy_elapsed)
 {
-	/*
-	 * Can result be wider than 64 bits?
-	 * We check against 62, not 64, due to ilog2 truncation.
-	 */
-	if (ilog2(bps_limit) + ilog2(jiffy_elapsed) - ilog2(HZ) > 62)
-		return U64_MAX;
 	return mul_u64_u64_div_u64(bps_limit, (u64)jiffy_elapsed, (u64)HZ);

Not directly related, but the two (u64) casts are pointless and can be removed.

Yeah... I only reverted 2dd710d476f2 ("blk-throttle: check for overflow in calculate_bytes_allowed")

Oleg.

#6 lore raw
In reply to Oleg Nesterov

On Fri, 15 Aug 2025 18:40:09 +0200
Oleg Nesterov <[email protected]> wrote:

One of my 'idea patches' is to make mul_u64_u64_div_u64() a wrapper for another function that takes in extra 'int *overflowed' parameter that is set zero/non-zero for success/overflow.
The 'overflowed' parameter can either be a compile-time NULL or a valid pointer.

So the x86-x64 asm implementation would use different code - you need the 'jump around fail label' to write the ~0 return value to *overflowed. The extra pointer check in the C version normal path may not be worth worrying about (but the '*overflow = 0' could easily be inlined).

The typical use would be:

	quotient = mul_u64_u64_div_u64_overflow(..., &overflowed);
	if (quotient == ~0ull && overflowed)
		...

That will generate better code than returning 'overflowed' and the quotient by reference.

Although I wonder how often ~0ull is a valid result?

David

#7 lore raw

David,

We had a lengthy discussion and you have already acked this fix.

I thought that we agreed on that a) we need to fix the problem first and b) x86 version should be consistent with the generic implementation regarding ~0ull on overflow.

Can we finally merge this fix, then discuss the possible improvements and possibly change both implementation?

Oleg.

Quoted 26 lines · David Laight

On Fri, 15 Aug 2025 18:40:09 +0200
Oleg Nesterov <[email protected]> wrote:

One of my 'idea patches' is to make mul_u64_u64_div_u64() a wrapper for another function that takes in extra 'int *overflowed' parameter that is set zero/non-zero for success/overflow.
The 'overflowed' parameter can either be a compile-time NULL or a valid pointer.

So the x86-x64 asm implementation would use different code - you need the 'jump around fail label' to write the ~0 return value to *overflowed. The extra pointer check in the C version normal path may not be worth worrying about (but the '*overflow = 0' could easily be inlined).

The typical use would be:

	quotient = mul_u64_u64_div_u64_overflow(..., &overflowed);
	if (quotient == ~0ull && overflowed)
		...

That will generate better code than returning 'overflowed' and the quotient by reference.

Although I wonder how often ~0ull is a valid result?

David

#8 lore raw
Quoted 10 lines · On Mon, 18 Aug 2025 14:39:01 +0200 Oleg Nesterov <[email protected]> wrote:

David,

We had a lengthy discussion and you have already acked this fix.

I thought that we agreed on that a) we need to fix the problem first and b) x86 version should be consistent with the generic implementation regarding ~0ull on overflow.

Can we finally merge this fix, then discuss the possible improvements and possibly change both implementation?

I deliberately put this comment on 0/2 because it is 'future thought'. I didn't want to delay the patch going in.

David

Quoted 31 lines

Oleg.

Quoted 26 lines · David Laight

On Fri, 15 Aug 2025 18:40:09 +0200
Oleg Nesterov <[email protected]> wrote:

One of my 'idea patches' is to make mul_u64_u64_div_u64() a wrapper for another function that takes in extra 'int *overflowed' parameter that is set zero/non-zero for success/overflow.
The 'overflowed' parameter can either be a compile-time NULL or a valid pointer.

So the x86-x64 asm implementation would use different code - you need the 'jump around fail label' to write the ~0 return value to *overflowed. The extra pointer check in the C version normal path may not be worth worrying about (but the '*overflow = 0' could easily be inlined).

The typical use would be:

	quotient = mul_u64_u64_div_u64_overflow(..., &overflowed);
	if (quotient == ~0ull && overflowed)
		...

That will generate better code than returning 'overflowed' and the quotient by reference.

Although I wonder how often ~0ull is a valid result?

David