Skip to contentSkip to conversation

drm/amdkfd: Fix the shift-out-of-bounds warning

PATCH Superseded Ma Jun 5 participants 6 messages Jan 10, 2024 – Jan 11, 2024 AMD KFD

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Ma Jun

author
#0 lore raw

PATCH 1/1 drm/amdkfd: Fix the shift-out-of-bounds warning

There is following shift-out-of-bounds warning if ecode=0.
"shift exponent 4294967295 is too large for 64-bit type 'long long unsigned int'"

---
 include/uapi/linux/kfd_ioctl.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/uapi/linux/kfd_ioctl.h +1 -1
index 2aa88afe305b..129325b02a91 100644
@@ -1004,7 +1004,7 @@ enum kfd_dbg_trap_exception_code {
 };
 
 /* Mask generated by ecode in kfd_dbg_trap_exception_code */
-#define KFD_EC_MASK(ecode)	(1ULL << (ecode - 1))
+#define KFD_EC_MASK(ecode)	(BIT(ecode) - 1)
 
 /* Masks for exception code type checks below */
 #define KFD_EC_MASK_QUEUE	(KFD_EC_MASK(EC_QUEUE_WAVE_ABORT) |	\
Signature · 1 line
2.34.1
maintainer
#1 lore raw
Quoted 13 lines · Ma Jun

There is following shift-out-of-bounds warning if ecode=0.
"shift exponent 4294967295 is too large for 64-bit type 'long long unsigned int'"

---
  include/uapi/linux/kfd_ioctl.h | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
include/uapi/linux/kfd_ioctl.h +0 -0
index 2aa88afe305b..129325b02a91 100644

include/uapi/linux/kfd_ioctl.h enum kfd_dbg_trap_exception_code { · line 1007

Quoted 5 lines
include/uapi/linux/kfd_ioctl.h +1 -1
@@ -1004,7 +1004,7 @@ enum kfd_dbg_trap_exception_code {
  };
 
  /* Mask generated by ecode in kfd_dbg_trap_exception_code */
-#define KFD_EC_MASK(ecode)	(1ULL << (ecode - 1))
+#define KFD_EC_MASK(ecode)	(BIT(ecode) - 1)

This is not the same thing. We want a bit mask with one bit set. And ecode=1 should set bit 0. ecode=0 is not a valid code and doesn't have a valid mask. You could use BIT((ecode) - 1), but I think that would give you the same warning for ecode=0. I also don't see BIT defined anywhere under include/uapi, so I think using this in the API header would break the build in user mode.

Where are you seeing the warning about the bad shift exponent? Looks like someone is using the KFD_EC_MASK macro incorrectly. Or if there is a legitimate use of it with ecode=0, then the correct fix would be

#define KFD_EC_MASK(ecode) ((ecode) ? 1ULL << (ecode - 1) : 0ULL)

Regards,
  Felix

Quoted 3 lines
include/uapi/linux/kfd_ioctl.h +0 -0
  /* Masks for exception code type checks below */
  #define KFD_EC_MASK_QUEUE	(KFD_EC_MASK(EC_QUEUE_WAVE_ABORT) |	\
#2 lore raw

Hi Felix,

Quoted 27 lines · Felix Kuehling
Quoted 13 lines · Ma Jun

There is following shift-out-of-bounds warning if ecode=0.
"shift exponent 4294967295 is too large for 64-bit type 'long long unsigned int'"

---
  include/uapi/linux/kfd_ioctl.h | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
include/uapi/linux/kfd_ioctl.h +0 -0
index 2aa88afe305b..129325b02a91 100644

include/uapi/linux/kfd_ioctl.h enum kfd_dbg_trap_exception_code { · line 1007

Quoted 5 lines
include/uapi/linux/kfd_ioctl.h +1 -1
@@ -1004,7 +1004,7 @@ enum kfd_dbg_trap_exception_code {
  };
 
  /* Mask generated by ecode in kfd_dbg_trap_exception_code */
-#define KFD_EC_MASK(ecode)	(1ULL << (ecode - 1))
+#define KFD_EC_MASK(ecode)	(BIT(ecode) - 1)

This is not the same thing. We want a bit mask with one bit set. And ecode=1 should set bit 0. ecode=0 is not a valid code and doesn't have a valid mask. You could use BIT((ecode) - 1), but I think that would give you the same warning for ecode=0. I also don't see BIT defined anywhere under include/uapi, so I think using this in the API header would break the build in user mode.

Quoted 4 lines

Where are you seeing the warning about the bad shift exponent? Looks like someone is using the KFD_EC_MASK macro incorrectly. Or if there is a legitimate use of it with ecode=0, then the correct fix would be

This warning is caused by following code in function event_interrupt_wq_v10()

else if (source_id == SOC15_INTSRC_CP_BAD_OPCODE) {
	kfd_set_dbg_ev_from_interrupt(dev, pasid,
	KFD_DEBUG_DOORBELL_ID(context_id0),
	KFD_EC_MASK(KFD_DEBUG_CP_BAD_OP_ECODE(context_id0)),
	NULL,
	0);
}
Quoted 1 line

#define KFD_EC_MASK(ecode) ((ecode) ? 1ULL << (ecode - 1) : 0ULL)

This can fix the warning.

Regards
Ma Jun

Quoted 8 lines

Regards,
  Felix

Quoted 3 lines
include/uapi/linux/kfd_ioctl.h +0 -0
  /* Masks for exception code type checks below */
  #define KFD_EC_MASK_QUEUE	(KFD_EC_MASK(EC_QUEUE_WAVE_ABORT) |	\
maintainer
#3 lore raw

[+Jon]

Quoted 35 lines · Jun

Hi Felix,

Quoted 26 lines · Felix Kuehling
Quoted 13 lines · Ma Jun

There is following shift-out-of-bounds warning if ecode=0.
"shift exponent 4294967295 is too large for 64-bit type 'long long unsigned int'"

---
   include/uapi/linux/kfd_ioctl.h | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)
include/uapi/linux/kfd_ioctl.h +0 -0
index 2aa88afe305b..129325b02a91 100644

include/uapi/linux/kfd_ioctl.h enum kfd_dbg_trap_exception_code { · line 1007

Quoted 5 lines
include/uapi/linux/kfd_ioctl.h +1 -1
@@ -1004,7 +1004,7 @@ enum kfd_dbg_trap_exception_code {
   };
 
   /* Mask generated by ecode in kfd_dbg_trap_exception_code */
-#define KFD_EC_MASK(ecode)	(1ULL << (ecode - 1))
+#define KFD_EC_MASK(ecode)	(BIT(ecode) - 1)

This is not the same thing. We want a bit mask with one bit set. And ecode=1 should set bit 0. ecode=0 is not a valid code and doesn't have a valid mask. You could use BIT((ecode) - 1), but I think that would give you the same warning for ecode=0. I also don't see BIT defined anywhere under include/uapi, so I think using this in the API header would break the build in user mode.

Quoted 4 lines

Where are you seeing the warning about the bad shift exponent? Looks like someone is using the KFD_EC_MASK macro incorrectly. Or if there is a legitimate use of it with ecode=0, then the correct fix would be

This warning is caused by following code in function event_interrupt_wq_v10()

Quoted 7 lines
else if (source_id == SOC15_INTSRC_CP_BAD_OPCODE) {
	kfd_set_dbg_ev_from_interrupt(dev, pasid,
	KFD_DEBUG_DOORBELL_ID(context_id0),
	KFD_EC_MASK(KFD_DEBUG_CP_BAD_OP_ECODE(context_id0)),
	NULL,
	0);
}

This looks OK. The compiler must be warning about a potential problem here, not a definite one.

Question for Jon, how does the firmware encode the error code in the context ID? I see these macros:

#define KFD_DEBUG_CP_BAD_OP_ECODE_MASK 0x3fffc00
#define KFD_DEBUG_CP_BAD_OP_ECODE_SHIFT 10

#define KFD_DEBUG_CP_BAD_OP_ECODE(ctxid0) (((ctxid0) &                  \
                                KFD_DEBUG_CP_BAD_OP_ECODE_MASK)         \
                                >> KFD_DEBUG_CP_BAD_OP_ECODE_SHIFT)

It looks like we have 16 bits for the ECODE. That's enough to have a bit mask. Do we really need KFD_EC_MASK to convert an error number into a bitmask here?

Quoted 4 lines
Quoted 3 lines

#define KFD_EC_MASK(ecode) ((ecode) ? 1ULL << (ecode - 1) : 0ULL)

This can fix the warning.

In the code above, if ecode is 0, that would lead to calling kfd_set_dbg_ev_from_interrupt with a event mask of 0. Not sure if that even makes sense. Jon, so we need special handling of cases where the error code is 0 or out of range, so we can warn about buggy firmware rather than creating nonsensical events for the debugger?

Regards,
  Felix

Quoted 10 lines

Regards
Ma Jun

Quoted 7 lines

Regards,
  Felix

Quoted 3 lines
include/uapi/linux/kfd_ioctl.h +0 -0
   /* Masks for exception code type checks below */
   #define KFD_EC_MASK_QUEUE	(KFD_EC_MASK(EC_QUEUE_WAVE_ABORT) |	\
#4 lore raw

[Public]

Quoted 64 lines

-----Original Message-----
From: Kuehling, Felix <[email protected]>
Sent: Thursday, January 11, 2024 11:03 AM
To: Ma, Jun <[email protected]>; Ma, Jun <[email protected]>; amd- [email protected]; Kim, Jonathan <[email protected]>
Subject: Re: [PATCH] drm/amdkfd: Fix the shift-out-of-bounds warning

[+Jon]

Quoted 6 lines · Jun

Hi Felix,

Quoted 3 lines · Felix Kuehling
Quoted 2 lines · Ma Jun

There is following shift-out-of-bounds warning if ecode=0. "shift exponent 4294967295 is too large for 64-bit type 'long long

unsigned int'"

Quoted 23 lines
Quoted 23 lines
Quoted 11 lines
---
   include/uapi/linux/kfd_ioctl.h | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)
include/uapi/linux/kfd_ioctl.h +0 -0
index 2aa88afe305b..129325b02a91 100644

include/uapi/linux/kfd_ioctl.h enum kfd_dbg_trap_exception_code { · line 1007

Quoted 5 lines
include/uapi/linux/kfd_ioctl.h +1 -1
@@ -1004,7 +1004,7 @@ enum kfd_dbg_trap_exception_code {
   };
 
   /* Mask generated by ecode in kfd_dbg_trap_exception_code */
-#define KFD_EC_MASK(ecode)       (1ULL << (ecode - 1))
+#define KFD_EC_MASK(ecode)       (BIT(ecode) - 1)

This is not the same thing. We want a bit mask with one bit set. And ecode=1 should set bit 0. ecode=0 is not a valid code and doesn't have a valid mask. You could use BIT((ecode) - 1), but I think that would give you the same warning for ecode=0. I also don't see BIT defined anywhere under include/uapi, so I think using this in the API header would break the build in user mode.

Quoted 5 lines
Quoted 4 lines

Where are you seeing the warning about the bad shift exponent? Looks like someone is using the KFD_EC_MASK macro incorrectly. Or if there is a legitimate use of it with ecode=0, then the correct fix would be

This warning is caused by following code in function

event_interrupt_wq_v10()

Quoted 8 lines
else if (source_id == SOC15_INTSRC_CP_BAD_OPCODE) {
    kfd_set_dbg_ev_from_interrupt(dev, pasid,
    KFD_DEBUG_DOORBELL_ID(context_id0),
    KFD_EC_MASK(KFD_DEBUG_CP_BAD_OP_ECODE(context_id0)),
    NULL,
    0);
}

This looks OK. The compiler must be warning about a potential problem here, not a definite one.

Question for Jon, how does the firmware encode the error code in the context ID? I see these macros:

#define KFD_DEBUG_CP_BAD_OP_ECODE_MASK 0x3fffc00
#define KFD_DEBUG_CP_BAD_OP_ECODE_SHIFT 10

Quoted 7 lines
#define KFD_DEBUG_CP_BAD_OP_ECODE(ctxid0) (((ctxid0) &                  \
                                 KFD_DEBUG_CP_BAD_OP_ECODE_MASK)         \
                                 >> KFD_DEBUG_CP_BAD_OP_ECODE_SHIFT)

It looks like we have 16 bits for the ECODE. That's enough to have a bit mask. Do we really need KFD_EC_MASK to convert an error number into a bitmask here?

Added Jay for confirmation.
I could be wrong but IIRC (and I'm quite fuzzy on this ... probably should document this), unlike the wave trap code interrupt mask (bit mask) the CP bad op code is a single error code that directly points to one of the exception code enums that we defined in the user API header. If that's the case, the KFD_EC_MASK is convenient for the kfd debugger code to mask the payload to send to the debugger or runtime.
If that's been wrong this whole time (i.e. the bad ops code is actually a bitwise mask of ecodes), then I'm not sure how we were able to get away with running the runtime negative tests for as long as we have and we'd need to recheck those tests.

Quoted 4 lines
Quoted 4 lines
Quoted 8 lines
Quoted 2 lines
Quoted 1 line

#define KFD_EC_MASK(ecode) ((ecode) ? 1ULL << (ecode - 1) : 0ULL)

This can fix the warning.

In the code above, if ecode is 0, that would lead to calling kfd_set_dbg_ev_from_interrupt with a event mask of 0. Not sure if that even makes sense. Jon, so we need special handling of cases where the error code is 0 or out of range, so we can warn about buggy firmware rather than creating nonsensical events for the debugger?

That makes sense. Again, deferring to Jay if a NULL cp bad op code is expected under any circumstances.
Either way, raising undefined events to the debugger or runtime isn't useful so range checking to filter out non-encoded cp bad op interrupts would be needed.

Thanks,

Jon

Quoted 16 lines

Regards,
Felix

Quoted 10 lines

Regards
Ma Jun

Quoted 7 lines

Regards,
Felix

Quoted 3 lines
include/uapi/linux/kfd_ioctl.h +0 -0
   /* Masks for exception code type checks below */
   #define KFD_EC_MASK_QUEUE
      (KFD_EC_MASK(EC_QUEUE_WAVE_ABORT) |     \
#5 lore raw
Quoted 12 lines · Jonathan
Quoted 12 lines

This looks OK. The compiler must be warning about a potential problem here, not a definite one.

Question for Jon, how does the firmware encode the error code in the context ID? I see these macros:

#define KFD_DEBUG_CP_BAD_OP_ECODE_MASK 0x3fffc00
#define KFD_DEBUG_CP_BAD_OP_ECODE_SHIFT 10

#define KFD_DEBUG_CP_BAD_OP_ECODE(ctxid0) (((ctxid0) &                  \
                                 KFD_DEBUG_CP_BAD_OP_ECODE_MASK)         \
                                 >> KFD_DEBUG_CP_BAD_OP_ECODE_SHIFT)
Quoted 8 lines
Quoted 3 lines

It looks like we have 16 bits for the ECODE. That's enough to have a bit mask. Do we really need KFD_EC_MASK to convert an error number into a bitmask here?

Added Jay for confirmation.
I could be wrong but IIRC (and I'm quite fuzzy on this ... probably should document this), unlike the wave trap code interrupt mask (bit mask) the CP bad op code is a single error code that directly points to one of the exception code enums that we defined in the user API header. If that's the case, the KFD_EC_MASK is convenient for the kfd debugger code to mask the payload to send to the debugger or runtime.
If that's been wrong this whole time (i.e. the bad ops code is actually a bitwise mask of ecodes), then I'm not sure how we were able to get away with running the runtime negative tests for as long as we have and we'd need to recheck those tests.

That's right. Queue errors are serialized. The error code is recorded directly.

Wavefront errors may occur concurrently within a wavefront. Those are recorded as a bitmask.

Quoted 8 lines
Quoted 5 lines

In the code above, if ecode is 0, that would lead to calling kfd_set_dbg_ev_from_interrupt with a event mask of 0. Not sure if that even makes sense. Jon, so we need special handling of cases where the error code is 0 or out of range, so we can warn about buggy firmware rather than creating nonsensical events for the debugger?

That makes sense. Again, deferring to Jay if a NULL cp bad op code is expected under any circumstances.
Either way, raising undefined events to the debugger or runtime isn't useful so range checking to filter out non-encoded cp bad op interrupts would be needed.

On AQL queues this interrupt carries an error code beginning from 16.