lib: zstd: Fix -Wstringop-overflow warning
PATCH v2 1/1 lib: zstd: Fix -Wstringop-overflow warning
Fix the following -Wstringop-overflow warning when building with GCC 11+:
lib/zstd/decompress/huf_decompress.c: In function ‘HUF_readDTableX2_wksp’:lib/zstd/decompress/huf_decompress.c:700:5: warning: ‘HUF_fillDTableX2.constprop’ accessing 624 bytes in a region of size 52 [-Wstringop-overflow=]
700 | HUF_fillDTableX2(dt, maxTableLog,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
701 | wksp->sortedSymbol, sizeOfSort,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
702 | wksp->rankStart0, wksp->rankVal, maxW,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
703 | tableLog+1,
| ~~~~~~~~~~~
704 | wksp->calleeWksp, sizeof(wksp->calleeWksp) / sizeof(U32));
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~lib/zstd/decompress/huf_decompress.c:700:5: note: referencing argument 6 of type ‘U32 (*)[13]’ {aka ‘unsigned int (*)[13]’} lib/zstd/decompress/huf_decompress.c:571:13: note: in a call to function ‘HUF_fillDTableX2.constprop’
571 | static void HUF_fillDTableX2(HUF_DEltX2* DTable, const U32 targetLog,
| ^~~~~~~~~~~~~~~~
by using pointer notation instead of array notation.
This is one of the last remaining warnings to be fixed before globally enabling -Wstringop-overflow.
---
v2: use "rankValCol_t *" instead of U32
v1: https://lore.kernel.org/lkml/20220330193352.GA119296@embeddedor/
--- lib/zstd/decompress/huf_decompress.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/zstd/decompress/huf_decompress.c +1 -1
@@ -985,7 +985,7 @@ static void HUF_fillDTableX2Level2(HUF_DEltX2* DTable, U32 targetLog, const U32static void HUF_fillDTableX2 (HUF_DEltX2 * DTable,const U32 targetLog,const sortedSymbol_t * sortedList, -const U32 * rankStart,rankVal_t rankValOrigin,const U32 maxWeight, +const U32 * rankStart,rankValCol_t *rankValOrigin,const U32 maxWeight,const U32 nbBitsBaseline) {U32 *const rankVal = rankValOrigin[0 ];
Signature · 1 line
2.34.1
Quoted 27 lines · Kees Cook
!-------------------------------------------------------------------|
This Message Is From an External Sender|-------------------------------------------------------------------!
Fix the following -Wstringop-overflow warning when building with GCC 11+:
lib/zstd/decompress/huf_decompress.c: In function ‘HUF_readDTableX2_wksp’:lib/zstd/decompress/huf_decompress.c:700:5: warning: ‘HUF_fillDTableX2.constprop’ accessing 624 bytes in a region of size 52 [-Wstringop-overflow=]700 | HUF_fillDTableX2(dt, maxTableLog, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 701 | wksp->sortedSymbol, sizeOfSort, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 702 | wksp->rankStart0, wksp->rankVal, maxW, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 703 | tableLog+1, | ~~~~~~~~~~~ 704 | wksp->calleeWksp, sizeof(wksp->calleeWksp) / sizeof(U32)); |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/zstd/decompress/huf_decompress.c:700:5: note: referencing argument 6 of type ‘U32 (*)[13]’ {aka ‘unsigned int (*)[13]’}lib/zstd/decompress/huf_decompress.c:571:13: note: in a call to function ‘HUF_fillDTableX2.constprop’
571 | static void HUF_fillDTableX2(HUF_DEltX2* DTable, const U32 targetLog,
Quoted 6 lines
| ^~~~~~~~~~~~~~~~by using pointer notation instead of array notation.
This is one of the last remaining warnings to be fixed before globally enabling -Wstringop-overflow.
The patch looks correct to me, thanks for reviving it. But, I was attempting to reproduce the issue, so I could better understand what's going on, and I wasn't able to reproduce it myself.
To attempt to reproduce, I applied this patch
---
lib/zstd/Makefile +2 -0
@@ -12,6 +12,8 @@ obj-$(CONFIG_ZSTD_COMPRESS) += zstd_compress.o obj-$(CONFIG_ZSTD_DECOMPRESS) += zstd_decompress.o obj-$(CONFIG_ZSTD_COMMON) += zstd_common.o +ccflags-y := -Wstringop-overflow=4 -Werror + zstd_compress-y := \ zstd_compress_module.o \ compress/fse_compress.o \
---
Then compiled on x86-64 with gcc 12.2.0 on tag v6.2-rc3. I saw no errors. I also tried with just -Wstringop-overflow, and on upstream zstd. I tried to make a minimal reproducer on godbolt, so I could see if it was the gcc version, but wasn't able to make it fail with any of them https://gcc.godbolt.org/z/Exzq9arMr.
Could you please tell me how to reproduce this warning?
Best,
Nick Terrell
Quoted 27 lines
Co-developed-by Gustavo A. R. Silva <[email protected]>Signed-off-by Gustavo A. R. Silva <[email protected]>Cc Nick Terrell <[email protected]>Signed-off-by Kees Cook <[email protected]>---
v2: use "rankValCol_t*" instead of U32
v1: https://lore.kernel.org/lkml/20220330193352.GA119296@embeddedor/
---lib/zstd/decompress/huf_decompress.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)lib/zstd/decompress/huf_decompress.c +1
-1@@ -985,7 +985,7 @@ static void HUF_fillDTableX2Level2(HUF_DEltX2* DTable, U32 targetLog, const U32static void HUF_fillDTableX2 (HUF_DEltX2 * DTable,const U32 targetLog,const sortedSymbol_t * sortedList, -const U32 * rankStart,rankVal_t rankValOrigin,const U32 maxWeight, +const U32 * rankStart,rankValCol_t *rankValOrigin,const U32 maxWeight,const U32 nbBitsBaseline) {U32 *const rankVal = rankValOrigin[0 ];Signature · 1 line
2.34.1
Quoted 55 lines · Nick Terrell
Quoted 27 lines · Kees Cook
!-------------------------------------------------------------------|
This Message Is From an External Sender|-------------------------------------------------------------------!
Fix the following -Wstringop-overflow warning when building with GCC 11+:
lib/zstd/decompress/huf_decompress.c: In function ‘HUF_readDTableX2_wksp’:lib/zstd/decompress/huf_decompress.c:700:5: warning: ‘HUF_fillDTableX2.constprop’ accessing 624 bytes in a region of size 52 [-Wstringop-overflow=]700 | HUF_fillDTableX2(dt, maxTableLog, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 701 | wksp->sortedSymbol, sizeOfSort, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 702 | wksp->rankStart0, wksp->rankVal, maxW, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 703 | tableLog+1, | ~~~~~~~~~~~ 704 | wksp->calleeWksp, sizeof(wksp->calleeWksp) / sizeof(U32)); |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/zstd/decompress/huf_decompress.c:700:5: note: referencing argument 6 of type ‘U32 (*)[13]’ {aka ‘unsigned int (*)[13]’}lib/zstd/decompress/huf_decompress.c:571:13: note: in a call to function ‘HUF_fillDTableX2.constprop’
571 | static void HUF_fillDTableX2(HUF_DEltX2* DTable, const U32 targetLog,Quoted 6 lines
| ^~~~~~~~~~~~~~~~by using pointer notation instead of array notation.
This is one of the last remaining warnings to be fixed before globally enabling -Wstringop-overflow.
The patch looks correct to me, thanks for reviving it. But, I was attempting to reproduce the issue, so I could better understand what's going on, and I wasn't able to reproduce it myself.
To attempt to reproduce, I applied this patch
---
lib/zstd/Makefile +2
-1@@ -12,6 +12,8 @@ obj-$(CONFIG_ZSTD_COMPRESS) += zstd_compress.o obj-$(CONFIG_ZSTD_DECOMPRESS) += zstd_decompress.o obj-$(CONFIG_ZSTD_COMMON) += zstd_common.o +ccflags-y := -Wstringop-overflow=4 -Werror + zstd_compress-y := \ zstd_compress_module.o \ compress/fse_compress.o \ ---
Quoted 6 lines
Then compiled on x86-64 with gcc 12.2.0 on tag v6.2-rc3. I saw no errors. I also tried with just
-Wstringop-overflow, and on upstream zstd. I tried to make a minimal reproducer on godbolt, so I could see if it was the gcc version, but wasn't able to make it fail with any of them https://gcc.godbolt.org/z/Exzq9arMr.Could you please tell me how to reproduce this warning?
I saw it like so with next-20230113 on x86_64:
$ gcc --version gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0 ... $ make KCFLAGS=-Wstringop-overflow allmodconfig lib/zstd/decompress/huf_decompress.o
Signature · 1 line
Kees Cook
Nick Terrell
Reviewed-byQuoted 70 lines · Kees Cook
!-------------------------------------------------------------------|
This Message Is From an External Sender|-------------------------------------------------------------------!
Quoted 60 lines · Nick Terrell
Quoted 27 lines · Kees Cook
!-------------------------------------------------------------------| This Message Is From an External Sender
|-------------------------------------------------------------------!
Fix the following -Wstringop-overflow warning when building with GCC 11+:
lib/zstd/decompress/huf_decompress.c: In function ‘HUF_readDTableX2_wksp’:lib/zstd/decompress/huf_decompress.c:700:5: warning: ‘HUF_fillDTableX2.constprop’ accessing 624 bytes in a region of size 52 [-Wstringop-overflow=]700 | HUF_fillDTableX2(dt, maxTableLog, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 701 | wksp->sortedSymbol, sizeOfSort, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 702 | wksp->rankStart0, wksp->rankVal, maxW, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 703 | tableLog+1, | ~~~~~~~~~~~ 704 | wksp->calleeWksp, sizeof(wksp->calleeWksp) / sizeof(U32)); |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/zstd/decompress/huf_decompress.c:700:5: note: referencing argument 6 of type ‘U32 (*)[13]’ {aka ‘unsigned int (*)[13]’}lib/zstd/decompress/huf_decompress.c:571:13: note: in a call to function ‘HUF_fillDTableX2.constprop’ 571 | static void HUF_fillDTableX2(HUF_DEltX2* DTable, const U32 targetLog,Quoted 6 lines
| ^~~~~~~~~~~~~~~~by using pointer notation instead of array notation.
This is one of the last remaining warnings to be fixed before globally enabling -Wstringop-overflow.
The patch looks correct to me, thanks for reviving it. But, I was attempting to reproduce the issue, so I could better understand what's going on, and I wasn't able to reproduce it myself.
To attempt to reproduce, I applied this patch
---
lib/zstd/Makefile +0
-0@@ -12,6 +12,8 @@ obj-$(CONFIG_ZSTD_COMPRESS) += zstd_compress.oobj-$(CONFIG_ZSTD_DECOMPRESS) += zstd_decompress.o obj-$(CONFIG_ZSTD_COMMON) += zstd_common.o
lib/zstd/Makefile +2
-0+ccflags-y := -Wstringop-overflow=4 -Werror +zstd_compress-y := \ zstd_compress_module.o \ compress/fse_compress.o \---
Then compiled on x86-64 with gcc 12.2.0 on tag v6.2-rc3. I saw no errors. I also tried with just
-Wstringop-overflow, and on upstream zstd. I tried to make a minimal reproducer on godbolt, so I could see if it was the gcc version, but wasn't able to make it fail with any of them https://gcc.godbolt.org/z/Exzq9arMr .
Quoted 8 lines
Quoted 1 line
Could you please tell me how to reproduce this warning?
I saw it like so with next-20230113 on
x86_64:$ gcc --version gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0 ... $ make KCFLAGS=-Wstringop-overflow allmodconfig lib/zstd/decompress/huf_decompress.o
Thanks, I was able to repro it! I will merge this patch into my tree.
If you would like to submit the same patch upstream yourself, I will accept the PR, otherwise I can submit an upstream PR.
Just to be certain, this patch is to work around a shortcoming in -Wstringop-overflow, but the code was otherwise correct?
Best,
Nick Terrell
Quoted 2 lines
Signature · 1 line
Kees Cook
Quoted 74 lines · Nick Terrell
Quoted 74 lines · Kees Cook
!-------------------------------------------------------------------|
This Message Is From an External Sender|-------------------------------------------------------------------!
Quoted 53 lines · Nick Terrell
Quoted 27 lines · Kees Cook
!-------------------------------------------------------------------| This Message Is From an External Sender
|-------------------------------------------------------------------!
Fix the following -Wstringop-overflow warning when building with GCC 11+:
lib/zstd/decompress/huf_decompress.c: In function ‘HUF_readDTableX2_wksp’:lib/zstd/decompress/huf_decompress.c:700:5: warning: ‘HUF_fillDTableX2.constprop’ accessing 624 bytes in a region of size 52 [-Wstringop-overflow=]700 | HUF_fillDTableX2(dt, maxTableLog, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 701 | wksp->sortedSymbol, sizeOfSort, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 702 | wksp->rankStart0, wksp->rankVal, maxW, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 703 | tableLog+1, | ~~~~~~~~~~~ 704 | wksp->calleeWksp, sizeof(wksp->calleeWksp) / sizeof(U32)); |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/zstd/decompress/huf_decompress.c:700:5: note: referencing argument 6 of type ‘U32 (*)[13]’ {aka ‘unsigned int (*)[13]’}lib/zstd/decompress/huf_decompress.c:571:13: note: in a call to function ‘HUF_fillDTableX2.constprop’ 571 | static void HUF_fillDTableX2(HUF_DEltX2* DTable, const U32 targetLog,Quoted 6 lines
| ^~~~~~~~~~~~~~~~by using pointer notation instead of array notation.
This is one of the last remaining warnings to be fixed before globally enabling -Wstringop-overflow.
The patch looks correct to me, thanks for reviving it. But, I was attempting to reproduce the issue, so I could better understand what's going on, and I wasn't able to reproduce it myself.
To attempt to reproduce, I applied this patch
---
lib/zstd/Makefile +0
-0@@ -12,6 +12,8 @@ obj-$(CONFIG_ZSTD_COMPRESS) += zstd_compress.oobj-$(CONFIG_ZSTD_DECOMPRESS) += zstd_decompress.o obj-$(CONFIG_ZSTD_COMMON) += zstd_common.o
lib/zstd/Makefile +2
-0+ccflags-y := -Wstringop-overflow=4 -Werror +zstd_compress-y := \ zstd_compress_module.o \ compress/fse_compress.o \Quoted 8 lines
---
Then compiled on x86-64 with gcc 12.2.0 on tag v6.2-rc3. I saw no errors. I also tried with just
-Wstringop-overflow, and on upstream zstd. I tried to make a minimal reproducer on godbolt, so I could see if it was the gcc version, but wasn't able to make it fail with any of them https://gcc.godbolt.org/z/Exzq9arMr .Could you please tell me how to reproduce this warning?
I saw it like so with next-20230113 on
x86_64:
Quoted 6 lines
Quoted 4 lines
$ gcc --version gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0 ... $ make KCFLAGS=-Wstringop-overflow allmodconfig lib/zstd/decompress/huf_decompress.oThanks, I was able to repro it! I will merge this patch into my tree.
Thanks!
Quoted 1 line
If you would like to submit the same patch upstream yourself, I will accept the PR, otherwise I can submit an upstream PR.
I don't know the process there, so if you could do it, I'd appreciate it.
Quoted 2 lines
Just to be certain, this patch is to work around a shortcoming in -Wstringop-overflow, but the code was otherwise correct?
It's the same result for the binary output. The types indicated in the function prototype meant there was cross-struct-member overflow (i.e. writing to the second array dimension when only 1 was specified), but the way to clear up intention isn't as clean here, so I'd kind of say half a code correctness issue, and half a work-around. *makes wavey hands gesture*
-Kees
Signature · 1 line
Kees Cook