riscv: optimized mem* functions

Hi Matteo,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linux/master]
[also build test ERROR on linus/master v5.15-rc1 next-20210917] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matteo-Croce/riscv-optimized-mem-functions/20210920-032303
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git bdb575f872175ed0ecf2638369da1cb7a6e86a14
config: riscv-randconfig-r004-20210919 (attached as .config)
compiler: riscv64-linux-gcc (GCC) 11.2.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/9a948fd7d78a58890608e9dd0f77e5ff84f36e3e
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matteo-Croce/riscv-optimized-mem-functions/20210920-032303 git checkout 9a948fd7d78a58890608e9dd0f77e5ff84f36e3e
# save the attached .config to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=riscv SHELL=/bin/bash

If you fix the issue, kindly add following tag as appropriate

Reported-by kernel test robot <[email protected]>

All errors (new ones prefixed by >>):

arch/riscv/lib/string.c: In function '__memmove':

Quoted 1 line

Quoted 1 line

arch/riscv/lib/string.c:89:7: error: inlining failed in call to 'always_inline' 'memcpy': function body can be overwritten at link time

      89 | void *memcpy(void *dest, const void *src, size_t count) __weak __alias(__memcpy);
         |       ^~~~~~
   arch/riscv/lib/string.c:99:24: note: called from here
      99 |                 return memcpy(dest, src, count);
         |                        ^~~~~~~~~~~~~~~~~~~~~~~~

vim +89 arch/riscv/lib/string.c

86c5866e9b7fdd Matteo Croce 2021-09-19 88
86c5866e9b7fdd Matteo Croce 2021-09-19 @89 void *memcpy(void *dest, const void *src, size_t count) __weak __alias(__memcpy); 86c5866e9b7fdd Matteo Croce 2021-09-19 90 EXPORT_SYMBOL(memcpy);
9a948fd7d78a58 Matteo Croce 2021-09-19 91

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

Quoted 47 lines · kernel test robot

Hi Matteo,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linux/master]
[also build test ERROR on linus/master v5.15-rc1 next-20210917] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matteo-Croce/riscv-optimized-mem-functions/20210920-032303
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git bdb575f872175ed0ecf2638369da1cb7a6e86a14
config: riscv-randconfig-r004-20210919 (attached as .config)
compiler: riscv64-linux-gcc (GCC) 11.2.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/9a948fd7d78a58890608e9dd0f77e5ff84f36e3e
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matteo-Croce/riscv-optimized-mem-functions/20210920-032303 git checkout 9a948fd7d78a58890608e9dd0f77e5ff84f36e3e
# save the attached .config to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=riscv SHELL=/bin/bash

If you fix the issue, kindly add following tag as appropriate

Reported-by kernel test robot <[email protected]>

All errors (new ones prefixed by >>):

arch/riscv/lib/string.c: In function '__memmove':

Quoted 1 line

Quoted 1 line

arch/riscv/lib/string.c:89:7: error: inlining failed in call to 'always_inline' 'memcpy': function body can be overwritten at link time

      89 | void *memcpy(void *dest, const void *src, size_t count) __weak __alias(__memcpy);
         |       ^~~~~~
   arch/riscv/lib/string.c:99:24: note: called from here
      99 |                 return memcpy(dest, src, count);
         |                        ^~~~~~~~~~~~~~~~~~~~~~~~

vim +89 arch/riscv/lib/string.c

86c5866e9b7fdd Matteo Croce 2021-09-19 88
86c5866e9b7fdd Matteo Croce 2021-09-19 @89 void *memcpy(void *dest, const void *src, size_t count) __weak __alias(__memcpy); 86c5866e9b7fdd Matteo Croce 2021-09-19 90 EXPORT_SYMBOL(memcpy);
9a948fd7d78a58 Matteo Croce 2021-09-19 91

Quoted 3 lines

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

How can we fix this? Maybe calling __memcpy() instead?

From: Matteo Croce <[email protected]>

Replace the assembly mem{cpy,move,set} with C equivalent.

Try to access RAM with the largest bit width possible, but without doing unaligned accesses.

A further improvement could be to use multiple read and writes as the assembly version was trying to do.

Tested on a BeagleV Starlight with a SiFive U74 core, where the improvement is noticeable.

v3 -> v4:
- incorporate changes from proposed generic version:
https://lore.kernel.org/lkml/[email protected]/

From: Matteo Croce <[email protected]>

Replace the assembly mem{cpy,move,set} with C equivalent.

Try to access RAM with the largest bit width possible, but without doing unaligned accesses.

A further improvement could be to use multiple read and writes as the assembly version was trying to do.

Tested on a BeagleV Starlight with a SiFive U74 core, where the improvement is noticeable.

v3 -> v4:
- incorporate changes from proposed generic version:
https://lore.kernel.org/lkml/[email protected]/

v2 -> v3:
- alias mem* to __mem* and not viceversa
- use __alias instead of a tail call

v1 -> v2:
- reduce the threshold from 64 to 16 bytes
- fix KASAN build
- optimize memset

Matteo Croce (3):
riscv: optimized memcpy
riscv: optimized memmove
riscv: optimized memset

 arch/riscv/include/asm/string.h |  18 ++--
 arch/riscv/kernel/Makefile      |   1 -
 arch/riscv/kernel/riscv_ksyms.c |  17 ----
 arch/riscv/lib/Makefile         |   4 +-
 arch/riscv/lib/memcpy.S         | 108 ----------------------
 arch/riscv/lib/memmove.S        |  64 -------------
 arch/riscv/lib/memset.S         | 113 -----------------------
 arch/riscv/lib/string.c         | 154 ++++++++++++++++++++++++++++++++
 8 files changed, 164 insertions(+), 315 deletions(-)
 delete mode 100644 arch/riscv/kernel/riscv_ksyms.c
 delete mode 100644 arch/riscv/lib/memcpy.S
 delete mode 100644 arch/riscv/lib/memmove.S
 delete mode 100644 arch/riscv/lib/memset.S
 create mode 100644 arch/riscv/lib/string.c

Quoted 31 lines · On Sun, 19 Sep 2021 12:21:01 PDT (-0700), [email protected] wrote:

From: Matteo Croce <[email protected]>

Replace the assembly mem{cpy,move,set} with C equivalent.

Try to access RAM with the largest bit width possible, but without doing unaligned accesses.

A further improvement could be to use multiple read and writes as the assembly version was trying to do.

Tested on a BeagleV Starlight with a SiFive U74 core, where the improvement is noticeable.

v3 -> v4:
- incorporate changes from proposed generic version:
https://lore.kernel.org/lkml/[email protected]/

v2 -> v3:
- alias mem* to __mem* and not viceversa
- use __alias instead of a tail call

v1 -> v2:
- reduce the threshold from 64 to 16 bytes
- fix KASAN build
- optimize memset

Matteo Croce (3):
riscv: optimized memcpy
riscv: optimized memmove
riscv: optimized memset

Quoted 14 lines

 arch/riscv/include/asm/string.h |  18 ++--
 arch/riscv/kernel/Makefile      |   1 -
 arch/riscv/kernel/riscv_ksyms.c |  17 ----
 arch/riscv/lib/Makefile         |   4 +-
 arch/riscv/lib/memcpy.S         | 108 ----------------------
 arch/riscv/lib/memmove.S        |  64 -------------
 arch/riscv/lib/memset.S         | 113 -----------------------
 arch/riscv/lib/string.c         | 154 ++++++++++++++++++++++++++++++++
 8 files changed, 164 insertions(+), 315 deletions(-)
 delete mode 100644 arch/riscv/kernel/riscv_ksyms.c
 delete mode 100644 arch/riscv/lib/memcpy.S
 delete mode 100644 arch/riscv/lib/memmove.S
 delete mode 100644 arch/riscv/lib/memset.S
 create mode 100644 arch/riscv/lib/string.c

Thanks. These generally look good, but they're failing to build for me. I'm getting errors along the lines of

    arch/riscv/lib/string.c:89:7: error: inlining failed in call to ‘always_inline’ ‘memcpy’: function body can be overwritten at link time
       89 | void *memcpy(void *dest, const void *src, size_t count) __weak __alias(__memcpy);      |       ^~~~~~
    arch/riscv/lib/string.c:99:10: note: called from here
       99 |   return memcpy(dest, src, count);
          |          ^~~~~~~~~~~~~~~~~~~~~~~~

I'm still a bit behind on email so I'm going to keep going through patches, but if there's no v5 by the time I get back here then I'll take a look.