Skip to contentSkip to conversation

skbuff: align sk_buff::cb to 64 bit

PATCH Superseded Felix Fietkau 4 participants 6 messages Jan 29, 2010 – Jan 30, 2010

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 skbuff: align sk_buff::cb to 64 bit

The alignment requirement for 64-bit load/store instructions on ARM is implementation defined. Some CPUs (such as Marvell Feroceon) do not generate an exception, if such an instruction is executed with an address that is not 64 bit aligned.
In such a case, the Feroceon corrupts adjacent memory, which showed up in my tests as a crash in the rx path of ath9k that only occured with CONFIG_XFRM set. This crash happened, because the first field of the mac80211 rx status info in the cb is an u64, and changing it corrupted the skb->sp field.

---

include/linux/skbuff.h +1 -1
@@ -329,7 +329,7 @@ struct sk_buff {
 	 * want to keep them across layers you have to do a skb_clone()
 	 * first. This is owned by whoever has the skb queued ATM.
 	 */
-	char			cb[48];
+	char			cb[48] __attribute__((aligned(8)));
 
 	unsigned int		len,
 				data_len;
#1 lore raw
Quoted 15 lines · Felix Fietkau

The alignment requirement for 64-bit load/store instructions on ARM is
implementation defined. Some CPUs (such as Marvell Feroceon) do not
generate an exception, if such an instruction is executed with an
address that is not 64 bit aligned. In such a case, the Feroceon corrupts adjacent memory, which showed up in my tests as a crash in the rx path of ath9k that only occured with
CONFIG_XFRM set. This crash happened, because the first field of the
mac80211 rx status info in the cb is an u64, and changing it corrupted
the skb->sp field.

---

include/linux/skbuff.h struct sk_buff { · line 333

Quoted 6 lines
include/linux/skbuff.h +1 -1
@@ -329,7 +329,7 @@ struct sk_buff {
 	 * want to keep them across layers you have to do a skb_clone()
 	 * first. This is owned by whoever has the skb queued ATM.
 	 */
-	char			cb[48];
+	char			cb[48] __attribute__((aligned(8)));
 

s/__attribute__((aligned(8)))/__aligned(8)/

Could the same thing be achieved by swapping the order of the dev and tstamp fields instead of adding the alignment attribute?

What is the sizeof(void *) on this thing?

David Daney

#2 lore raw
Quoted 27 lines · David Daney
Quoted 16 lines · Felix Fietkau

The alignment requirement for 64-bit load/store instructions on ARM is implementation defined. Some CPUs (such as Marvell Feroceon) do not generate an exception, if such an instruction is executed with an address that is not 64 bit aligned.
In such a case, the Feroceon corrupts adjacent memory, which showed up in my tests as a crash in the rx path of ath9k that only occured with CONFIG_XFRM set. This crash happened, because the first field of the mac80211 rx status info in the cb is an u64, and changing it corrupted the skb->sp field.

---

include/linux/skbuff.h struct sk_buff { · line 333

Quoted 6 lines
include/linux/skbuff.h +1 -1
@@ -329,7 +329,7 @@ struct sk_buff {
 	 * want to keep them across layers you have to do a skb_clone()
 	 * first. This is owned by whoever has the skb queued ATM.
 	 */
-	char			cb[48];
+	char			cb[48] __attribute__((aligned(8)));
 

s/__attribute__((aligned(8)))/__aligned(8)/

Quoted 4 lines

Could the same thing be achieved by swapping the order of the dev and tstamp fields instead of adding the alignment attribute?

What is the sizeof(void *) on this thing?

I'd guess the problem is with the accessor to the thing. something in the opaque cb ptr its beeing accessed wroingly causing overwrites.

Cheers,
ed

#4 lore raw
In reply to David Daney
Quoted 11 lines · David Daney
Quoted 10 lines · Felix Fietkau

The alignment requirement for 64-bit load/store instructions on ARM is implementation defined. Some CPUs (such as Marvell Feroceon) do not generate an exception, if such an instruction is executed with an address that is not 64 bit aligned.
In such a case, the Feroceon corrupts adjacent memory, which showed up in my tests as a crash in the rx path of ath9k that only occured with CONFIG_XFRM set. This crash happened, because the first field of the mac80211 rx status info in the cb is an u64, and changing it corrupted the skb->sp field.

Quoted 15 lines

include/linux/skbuff.h struct sk_buff { · line 333

Quoted 12 lines

---

include/linux/skbuff.h +1 -1
@@ -329,7 +329,7 @@ struct sk_buff {
 	 * want to keep them across layers you have to do a skb_clone()
 	 * first. This is owned by whoever has the skb queued ATM.
 	 */
-	char			cb[48];
+	char			cb[48] __attribute__((aligned(8)));
 

s/__attribute__((aligned(8)))/__aligned(8)/

OK. Will send a v2.

Quoted 4 lines

Could the same thing be achieved by swapping the order of the dev and tstamp fields instead of adding the alignment attribute?

What is the sizeof(void *) on this thing?

sizeof(void *) == 4
This is a 32 bit arch, which happens to have 64-bit load/store ops, which the compiler emits where appropriate.
Reordering the dev and tstamp does not help, as the cb gets moved by 4 bytes, depending on whether CONFIG_XFRM is set or unset.
As far as I know, ARM EABI requires an 8 byte alignment anyway, so I think it's better to make this explicit, since cb[] is meant to be used as opaque storage for other data structures.

- Felix

#3 lore raw
In reply to Edgar E. Iglesias
Quoted 12 lines · Edgar E. Iglesias
Quoted 11 lines · David Daney
Quoted 10 lines · Felix Fietkau

The alignment requirement for 64-bit load/store instructions on ARM is implementation defined. Some CPUs (such as Marvell Feroceon) do not generate an exception, if such an instruction is executed with an address that is not 64 bit aligned.
In such a case, the Feroceon corrupts adjacent memory, which showed up in my tests as a crash in the rx path of ath9k that only occured with CONFIG_XFRM set. This crash happened, because the first field of the mac80211 rx status info in the cb is an u64, and changing it corrupted the skb->sp field.

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines

---
--- a/include/linux/skbuff.h

Quoted 8 lines

include/linux/skbuff.h struct sk_buff { · line 333

Quoted 8 lines

+++ b/include/linux/skbuff.h

include/linux/skbuff.h +1 -1
@@ -329,7 +329,7 @@ struct sk_buff {
 	 * want to keep them across layers you have to do a skb_clone()
 	 * first. This is owned by whoever has the skb queued ATM.
 	 */
-	char			cb[48];
+	char			cb[48] __attribute__((aligned(8)));
 

I'd guess the problem is with the accessor to the thing. something in the opaque cb ptr its beeing accessed wroingly causing overwrites.

Nope, the problem is not with the accessor, I verified that. The start address of the datastructure that gets put into skb->cb correctly points at the start of the cb, and it is only aligned to 4 byte, not 8. Also, this issue only happens on CPUs that do not throw an unaligned exception for this instruction.

- Felix

#5 lore raw
In reply to Felix Fietkau
Quoted 9 lines · Felix Fietkau

The alignment requirement for 64-bit load/store instructions on ARM is implementation defined. Some CPUs (such as Marvell Feroceon) do not generate an exception, if such an instruction is executed with an address that is not 64 bit aligned.
In such a case, the Feroceon corrupts adjacent memory, which showed up in my tests as a crash in the rx path of ath9k that only occured with CONFIG_XFRM set. This crash happened, because the first field of the mac80211 rx status info in the cb is an u64, and changing it corrupted the skb->sp field.

Actually, only older Marvell CPU cores have this behavior.

The ARMv5 Architecture Reference Manual permits the behavior of not throwing alignment exceptions for doubleword (64bit) load/stores to 4mod8 aligned addresses, and early Feroceons indeed don't, but anything recent will throw an alignment exception.