Skip to contentSkip to conversation

crypto: sha256_ssse3 - fix stack corruption with SSSE3 and AVX implementations

PATCH In review Jussi Kivilinna 4 participants 4 messages May 21, 2013 – May 28, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 crypto: sha256_ssse3 - fix stack corruption with SSSE3 and AVX implementations

The _XFER stack element size was set too small, 8 bytes, when it needs to be 16 bytes. As _XFER is the last stack element used by these implementations, the 16 byte stores with 'movdqa' corrupt the stack where the value of register %r12 is temporarily stored. As these implementations align the stack pointer to 16 bytes, this corruption did not happen every time.

Patch corrects this issue.

---
 arch/x86/crypto/sha256-avx-asm.S   |    2 +-
 arch/x86/crypto/sha256-ssse3-asm.S |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/crypto/sha256-avx-asm.S +1 -1
index 56610c4..642f156 100644
@@ -118,7 +118,7 @@ y2 = %r15d
 
 _INP_END_SIZE = 8
 _INP_SIZE = 8
-_XFER_SIZE = 8
+_XFER_SIZE = 16
 _XMM_SAVE_SIZE = 0
 
 _INP_END = 0
arch/x86/crypto/sha256-ssse3-asm.S +1 -1
index 98d3c39..f833b74 100644
@@ -111,7 +111,7 @@ y2 = %r15d
 
 _INP_END_SIZE = 8
 _INP_SIZE = 8
-_XFER_SIZE = 8
+_XFER_SIZE = 16
 _XMM_SAVE_SIZE = 0
 
 _INP_END = 0
Tested-by on v1 1/1
#1 lore raw

Hi,

Quoted 7 lines

The _XFER stack element size was set too small, 8 bytes, when it needs to be 16 bytes. As _XFER is the last stack element used by these implementations, the 16 byte stores with 'movdqa' corrupt the stack where the value of register %r12 is temporarily stored. As these implementations align the stack pointer to 16 bytes, this corruption did not happen every time.

Quoted 5 lines

Patch corrects this issue.

Thank you for fixing this issue.

Best regards,
Julian Wollrath

Quoted 32 lines
---
 arch/x86/crypto/sha256-avx-asm.S   |    2 +-
 arch/x86/crypto/sha256-ssse3-asm.S |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
(quoted diff) +0 -0

b/arch/x86/crypto/sha256-avx-asm.S index 56610c4..642f156 100644

arch/x86/crypto/sha256-avx-asm.S +1 -1
@@ -118,7 +118,7 @@ y2 = %r15d
 
 _INP_END_SIZE = 8
 _INP_SIZE = 8
-_XFER_SIZE = 8
+_XFER_SIZE = 16
 _XMM_SAVE_SIZE = 0
 
 _INP_END = 0
(quoted diff) +0 -0

b/arch/x86/crypto/sha256-ssse3-asm.S index 98d3c39..f833b74 100644

arch/x86/crypto/sha256-ssse3-asm.S +1 -1
@@ -111,7 +111,7 @@ y2 = %r15d
 
 _INP_END_SIZE = 8
 _INP_SIZE = 8
-_XFER_SIZE = 8
+_XFER_SIZE = 16
 _XMM_SAVE_SIZE = 0
 
 _INP_END = 0
Acked-by on v1 1/1
#2 lore raw
In reply to Jussi Kivilinna
Quoted 7 lines · Jussi Kivilinna

The _XFER stack element size was set too small, 8 bytes, when it needs to be 16 bytes. As _XFER is the last stack element used by these implementations, the 16 byte stores with 'movdqa' corrupt the stack where the value of register %r12 is temporarily stored. As these implementations align the stack pointer to 16 bytes, this corruption did not happen every time.

Patch corrects this issue.

Thanks for catching and fixing the issue.

Tim

maintainer
#3 lore raw
Quoted 7 lines · Tim Chen
Quoted 6 lines · Jussi Kivilinna

The _XFER stack element size was set too small, 8 bytes, when it needs to be 16 bytes. As _XFER is the last stack element used by these implementations, the 16 byte stores with 'movdqa' corrupt the stack where the value of register %r12 is temporarily stored. As these implementations align the stack pointer to 16 bytes, this corruption did not happen every time.

Quoted 5 lines
Quoted 1 line

Patch corrects this issue.

Thanks for catching and fixing the issue.

Patch applied. Thanks!

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt