undefined reference to `__multi3' when building with gcc 7.x

PATCH In review Thomas Petazzoni 8 participants 21 messages Aug 3, 2017 – Dec 27, 2017 MIPS

Share & tools

Thomas Petazzoni

author

2017-08-03 20:56 UTC #0 lore raw

Hello,

When trying to build the current Linux master with a gcc 7.x toolchain for mips64r6-n32, I'm getting the following build failure:

crypto/scompress.o: In function `.L31':
scompress.c:(.text+0x2a0): undefined reference to __multi3' drivers/base/component.o: In function .L97':
component.c:(.text+0x4a4): undefined reference to __multi3' drivers/base/component.o: In function component_master_add_with_match':
component.c:(.text+0x8c4): undefined reference to __multi3' net/core/ethtool.o: In function ethtool_set_per_queue_coalesce':
ethtool.c:(.text+0x1ab0): undefined reference to `__multi3' Makefile:1000: recipe for target 'vmlinux' failed
make[2]: *** [vmlinux] Error 1

Taking the example from net/core/ethtool.o, objdump says:

    1aac:       00408025        move    s0,v0
    1ab0:       e8000000        balc    1ab4 <ethtool_set_per_queue_coalesce+0x7c>
    1ab4:       14600000        bnez    v1,1ab8 <ethtool_set_per_queue_coalesce+0x80>

And readelf tells us:

Relocation section '.rela.text' at offset 0xaa00 contains 1189 entries:
Offset Info Type Sym. Value Sym. Name + Addend
[...]
000000001ab0 023a0000003d R_MIPS_PC26_S2 0000000000000000 __multi3 - 4
Type2: R_MIPS_NONE
Type3: R_MIPS_NONE
[...]
Symbol table '.symtab' contains 586 entries:
[...]
570: 0000000000000000 0 OBJECT GLOBAL DEFAULT UND __multi3

__multi3() is normally provided by libgcc, but of course the kernel doesn't link with libgcc.

The bug can be reproduced by building with the toolchain available at
http://toolchains.free-electrons.com/downloads/2017.08-rc1-fix-binutils/toolchains/mips64r6el-n32/tarballs/mips64r6el-n32--glibc--bleeding-edge-2017.05-1453-ga703fdd-1.tar.bz2
and building with the attached kernel configuration file.

It is not clear to me if this is a kernel issue (lack of __multi3 in arch/mips/lib/), or a gcc bug in that it shouldn't emit a call to this function.

FWIW, sparc64 had a similar issue, and they added __multi3 in their libgcc replacement, see commit
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/sparc/lib?id=1b4af13ff2cc6897557bb0b8d9e2fad4fa4d67aa.

Best regards,

Thomas Petazzoni

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Ralf Baechle

2017-08-04 00:06 UTC #1 lore raw

Quoted 42 lines · Thomas Petazzoni

Hello,
When trying to build the current Linux master with a gcc 7.x toolchain for mips64r6-n32, I'm getting the following build failure:
crypto/scompress.o: In function `.L31':
scompress.c:(.text+0x2a0): undefined reference to __multi3' drivers/base/component.o: In function .L97':
component.c:(.text+0x4a4): undefined reference to __multi3' drivers/base/component.o: In function component_master_add_with_match':
component.c:(.text+0x8c4): undefined reference to __multi3' net/core/ethtool.o: In function ethtool_set_per_queue_coalesce':
ethtool.c:(.text+0x1ab0): undefined reference to `__multi3' Makefile:1000: recipe for target 'vmlinux' failed
make[2]: *** [vmlinux] Error 1
Taking the example from net/core/ethtool.o, objdump says:
 1aac: 00408025 move s0,v0
 1ab0: e8000000 balc 1ab4 <ethtool_set_per_queue_coalesce+0x7c>
 1ab4: 14600000 bnez v1,1ab8 <ethtool_set_per_queue_coalesce+0x80>
And readelf tells us:
Relocation section '.rela.text' at offset 0xaa00 contains 1189 entries:
Offset Info Type Sym. Value Sym. Name + Addend
[...]
000000001ab0 023a0000003d R_MIPS_PC26_S2 0000000000000000 __multi3 - 4
Type2: R_MIPS_NONE
Type3: R_MIPS_NONE
[...]
Symbol table '.symtab' contains 586 entries:
[...]
570: 0000000000000000 0 OBJECT GLOBAL DEFAULT UND __multi3
__multi3() is normally provided by libgcc, but of course the kernel doesn't link with libgcc.
The bug can be reproduced by building with the toolchain available at
http://toolchains.free-electrons.com/downloads/2017.08-rc1-fix-binutils/toolchains/mips64r6el-n32/tarballs/mips64r6el-n32--glibc--bleeding-edge-2017.05-1453-ga703fdd-1.tar.bz2
and building with the attached kernel configuration file.

Quoted 7 lines

It is not clear to me if this is a kernel issue (lack of __multi3 in arch/mips/lib/), or a gcc bug in that it shouldn't emit a call to this function.
FWIW, sparc64 had a similar issue, and they added __multi3 in their libgcc replacement, see commit
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/sparc/lib?id=1b4af13ff2cc6897557bb0b8d9e2fad4fa4d67aa.

I think these days we've given up the stuborn resistance of the old days against adding new libgcc1 functions to the kernel, so we should probably just add it.

I'm looking into this but a small wrench into the gear is that I'm still on GCC 6 so I'm off to building myself a cross-gcc first ...

Ralf

Ralf Baechle

2017-08-04 15:19 UTC #2 lore raw

Quoted 49 lines · Ralf Baechle

Quoted 41 lines
When trying to build the current Linux master with a gcc 7.x toolchain for mips64r6-n32, I'm getting the following build failure:
crypto/scompress.o: In function `.L31':
scompress.c:(.text+0x2a0): undefined reference to __multi3' drivers/base/component.o: In function .L97':
component.c:(.text+0x4a4): undefined reference to __multi3' drivers/base/component.o: In function component_master_add_with_match':
component.c:(.text+0x8c4): undefined reference to __multi3' net/core/ethtool.o: In function ethtool_set_per_queue_coalesce':
ethtool.c:(.text+0x1ab0): undefined reference to `__multi3' Makefile:1000: recipe for target 'vmlinux' failed
make[2]: *** [vmlinux] Error 1
Taking the example from net/core/ethtool.o, objdump says:
 1aac: 00408025 move s0,v0
 1ab0: e8000000 balc 1ab4 <ethtool_set_per_queue_coalesce+0x7c>
 1ab4: 14600000 bnez v1,1ab8 <ethtool_set_per_queue_coalesce+0x80>
And readelf tells us:
Relocation section '.rela.text' at offset 0xaa00 contains 1189 entries:
Offset Info Type Sym. Value Sym. Name + Addend
[...]
000000001ab0 023a0000003d R_MIPS_PC26_S2 0000000000000000 __multi3 - 4
Type2: R_MIPS_NONE
Type3: R_MIPS_NONE
[...]
Symbol table '.symtab' contains 586 entries:
[...]
570: 0000000000000000 0 OBJECT GLOBAL DEFAULT UND __multi3
__multi3() is normally provided by libgcc, but of course the kernel doesn't link with libgcc.
The bug can be reproduced by building with the toolchain available at
http://toolchains.free-electrons.com/downloads/2017.08-rc1-fix-binutils/toolchains/mips64r6el-n32/tarballs/mips64r6el-n32--glibc--bleeding-edge-2017.05-1453-ga703fdd-1.tar.bz2
and building with the attached kernel configuration file.
Quoted 7 lines
It is not clear to me if this is a kernel issue (lack of __multi3 in arch/mips/lib/), or a gcc bug in that it shouldn't emit a call to this function.
FWIW, sparc64 had a similar issue, and they added __multi3 in their libgcc replacement, see commit
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/sparc/lib?id=1b4af13ff2cc6897557bb0b8d9e2fad4fa4d67aa.

Quoted 6 lines

I think these days we've given up the stuborn resistance of the old days against adding new libgcc1 functions to the kernel, so we should probably just add it.
I'm looking into this but a small wrench into the gear is that I'm still on GCC 6 so I'm off to building myself a cross-gcc first ...

I now can reproduce the issue with vanilla FSF binutils 2.28 and GCC 7.1.0.

Ralf

Thomas Petazzoni

author

2017-08-04 15:42 UTC #3 lore raw

Hello,

Quoted 12 lines · Ralf Baechle

Quoted 12 lines
Quoted 11 lines
The bug can be reproduced by building with the toolchain available at
http://toolchains.free-electrons.com/downloads/2017.08-rc1-fix-binutils/toolchains/mips64r6el-n32/tarballs/mips64r6el-n32--glibc--bleeding-edge-2017.05-1453-ga703fdd-1.tar.bz2
and building with the attached kernel configuration file.
It is not clear to me if this is a kernel issue (lack of __multi3 in arch/mips/lib/), or a gcc bug in that it shouldn't emit a call to this function.
FWIW, sparc64 had a similar issue, and they added __multi3 in their libgcc replacement, see commit
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/sparc/lib?id=1b4af13ff2cc6897557bb0b8d9e2fad4fa4d67aa.

Quoted 8 lines

Quoted 6 lines
I think these days we've given up the stuborn resistance of the old days against adding new libgcc1 functions to the kernel, so we should probably just add it.
I'm looking into this but a small wrench into the gear is that I'm still on GCC 6 so I'm off to building myself a cross-gcc first ...
I now can reproduce the issue with vanilla FSF binutils 2.28 and GCC 7.1.0.

Great! However, looking at the functions that end up calling __multi3, I'm wondering why suddenly gcc 7.x needs to call such a function, while the same code was compiling without __multi3 in libgcc with gcc 6.x.

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Ralf Baechle

2017-08-04 22:25 UTC #4 lore raw

Quoted 16 lines · Thomas Petazzoni

Quoted 16 lines
Quoted 16 lines
Quoted 11 lines
The bug can be reproduced by building with the toolchain available at
http://toolchains.free-electrons.com/downloads/2017.08-rc1-fix-binutils/toolchains/mips64r6el-n32/tarballs/mips64r6el-n32--glibc--bleeding-edge-2017.05-1453-ga703fdd-1.tar.bz2
and building with the attached kernel configuration file.
It is not clear to me if this is a kernel issue (lack of __multi3 in arch/mips/lib/), or a gcc bug in that it shouldn't emit a call to this function.
FWIW, sparc64 had a similar issue, and they added __multi3 in their libgcc replacement, see commit
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/sparc/lib?id=1b4af13ff2cc6897557bb0b8d9e2fad4fa4d67aa.
I think these days we've given up the stuborn resistance of the old days against adding new libgcc1 functions to the kernel, so we should probably just add it.

Quoted 8 lines

Quoted 4 lines
Quoted 2 lines
I'm looking into this but a small wrench into the gear is that I'm still on GCC 6 so I'm off to building myself a cross-gcc first ...
I now can reproduce the issue with vanilla FSF binutils 2.28 and GCC 7.1.0.
Great! However, looking at the functions that end up calling __multi3, I'm wondering why suddenly gcc 7.x needs to call such a function, while the same code was compiling without __multi3 in libgcc with gcc 6.x.

Chances are it's something specific to MIPS64 R6. Before trying your config file I also tried a number of other defconfigs and all built well.

Here's a test case which generates a reference to __multi3:

unsigned long func(unsigned long a, unsigned long b)

{
        return a > (~0UL) / b;
}

GCC rearanges above statement to:

	return (unsigned __int128)a * (unsigned __int128) b > 0xffffffff;

computing which requires 128 bit intermediate results. This is the code generated for MIPS64 R2:

	dmultu	$4,$5
	mfhi	$2
	sltu	$2,$0,$2

And this is for R6:

	move	$2,$4
	move	$7,$5			# $6/$7 contain the second op
	move	$6,$0
	move	$4,$0
	move	$5,$2			# $4/$5 contain the first op
	sd	$31,8($sp)
	balc	__multi3
	sltu	$2,$0,$2

Ralf

Thomas Petazzoni

author

2017-08-05 11:57 UTC #5 lore raw

Hello,

Quoted 10 lines · Ralf Baechle

Quoted 3 lines
Great! However, looking at the functions that end up calling __multi3, I'm wondering why suddenly gcc 7.x needs to call such a function, while the same code was compiling without __multi3 in libgcc with gcc 6.x.
Chances are it's something specific to MIPS64 R6. Before trying your config file I also tried a number of other defconfigs and all built well.
Here's a test case which generates a reference to __multi3:

Quoted 8 lines

unsigned long func(unsigned long a, unsigned long b)
{
        return a > (~0UL) / b;
}
GCC rearanges above statement to:
	return (unsigned __int128)a * (unsigned __int128) b > 0xffffffff;

And this is normal/expected ?

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Ralf Baechle

2017-08-07 08:35 UTC #6 lore raw

Quoted 17 lines · Thomas Petazzoni

Quoted 15 lines · Ralf Baechle
Quoted 3 lines
Great! However, looking at the functions that end up calling __multi3, I'm wondering why suddenly gcc 7.x needs to call such a function, while the same code was compiling without __multi3 in libgcc with gcc 6.x.
Chances are it's something specific to MIPS64 R6. Before trying your config file I also tried a number of other defconfigs and all built well.
Here's a test case which generates a reference to __multi3:
unsigned long func(unsigned long a, unsigned long b)
{
        return a > (~0UL) / b;
}

Quoted 5 lines

Quoted 3 lines
GCC rearanges above statement to:
	return (unsigned __int128)a * (unsigned __int128) b > 0xffffffff;
And this is normal/expected ?

Without consideration of performance, It's certainly is valid code. And with that I can't drop the issue as a GCC code generation bug.

However it seems GCC itself doesn't seem to have a __multi3 in its libgcc2 - which indeed would be a GCC issue - at least none I was easily able to find with grep so I'm adding Matthew Fortune to cc in the hope he can shed some light on this.

Ralf

Thomas Petazzoni

author

2017-08-13 20:46 UTC #7 lore raw

Hello,

Quoted 17 lines · Ralf Baechle

Quoted 11 lines
Quoted 11 lines
Chances are it's something specific to MIPS64 R6. Before trying your config file I also tried a number of other defconfigs and all built well.
Here's a test case which generates a reference to __multi3:
unsigned long func(unsigned long a, unsigned long b)
{
        return a > (~0UL) / b;
}
Quoted 5 lines
Quoted 3 lines
GCC rearanges above statement to:
	return (unsigned __int128)a * (unsigned __int128) b > 0xffffffff;  
And this is normal/expected ?

Quoted 7 lines

Without consideration of performance, It's certainly is valid code. And with that I can't drop the issue as a GCC code generation bug.
However it seems GCC itself doesn't seem to have a __multi3 in its libgcc2 - which indeed would be a GCC issue - at least none I was easily able to find with grep so I'm adding Matthew Fortune to cc in the hope he can shed some light on this.

Indeed, I don't see __multi3 implemented in libgcc in the source code, but it's probably because it's tricky to see its implementation, as it really is there:

$ ./bin/mips64el-linux-readelf -a ./mips64el-buildroot-linux-uclibc/sysroot/lib/libgcc_s.so.1 | grep multi3
  1747: 00011700   100 FUNC    GLOBAL DEFAULT   11 __multi3@@GCC_3.0
  5511: 00011700   100 FUNC    GLOBAL DEFAULT   11 __multi3
  000435e4 -32236(gp) 00011700 00011700 FUNC     11 __multi3

Objdump says:

00011700 <__multi3>:

   11700:       0006103e        dsrl32  v0,a2,0x0
   11704:       7c89f803        dext    a5,a0,0x0,0x20
   11708:       0004403e        dsrl32  a4,a0,0x0
   1170c:       7ccaf803        dext    a6,a2,0x0,0x20
   11710:       012a589c        dmul    a7,a5,a6
   11714:       010a509c        dmul    a6,a4,a6
   11718:       0122489c        dmul    a5,a5,v0
   1171c:       0102409c        dmul    a4,a4,v0
   11720:       012a482d        daddu   a5,a5,a6
   11724:       000b103e        dsrl32  v0,a7,0x0
   11728:       0049102d        daddu   v0,v0,a5
   1172c:       184a0003        bgeuc   v0,a6,1173c <.L2>
   11730:       24090001        li      a5,1
   11734:       0009483c        dsll32  a5,a5,0x0
   11738:       0109402d        daddu   a4,a4,a5

The same __multi3 problem also occurred on SPARC64, and they fixed it by adding a __multi3 implementation in the kernel. See:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/sparc?id=1b4af13ff2cc6897557bb0b8d9e2fad4fa4d67aa.

Best regards,

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Ralf Baechle

2017-08-17 07:15 UTC #8 lore raw

Quoted 49 lines · Thomas Petazzoni

Date: Sun, 13 Aug 2017 22:46:02 +0200
From: Thomas Petazzoni <[email protected]>
To: Ralf Baechle <[email protected]>
Cc: Matthew Fortune <[email protected]>,
[email protected], Waldemar Brodkorb <[email protected]>
Subject: Re: undefined reference to `__multi3' when building with gcc 7.x
Content-Type: text/plain; charset=US-ASCII
Hello,
Quoted 17 lines · Ralf Baechle
Quoted 11 lines
Quoted 11 lines
Chances are it's something specific to MIPS64 R6. Before trying your config file I also tried a number of other defconfigs and all built well.
Here's a test case which generates a reference to __multi3:
unsigned long func(unsigned long a, unsigned long b)
{
 return a > (~0UL) / b;
}
Quoted 5 lines
Quoted 3 lines
GCC rearanges above statement to:
	return (unsigned __int128)a * (unsigned __int128) b > 0xffffffff; 
And this is normal/expected ?
Quoted 7 lines
Without consideration of performance, It's certainly is valid code. And with that I can't drop the issue as a GCC code generation bug.
However it seems GCC itself doesn't seem to have a __multi3 in its libgcc2 - which indeed would be a GCC issue - at least none I was easily able to find with grep so I'm adding Matthew Fortune to cc in the hope he can shed some light on this.
Indeed, I don't see __multi3 implemented in libgcc in the source code, but it's probably because it's tricky to see its implementation, as it really is there:
$ ./bin/mips64el-linux-readelf -a ./mips64el-buildroot-linux-uclibc/sysroot/lib/libgcc_s.so.1 | grep multi3
 1747: 00011700 100 FUNC GLOBAL DEFAULT 11 __multi3@@GCC_3.0
 5511: 00011700 100 FUNC GLOBAL DEFAULT 11 __multi3
 000435e4 -32236(gp) 00011700 00011700 FUNC 11 __multi3
Objdump says:
00011700 <__multi3>:

Quoted 15 lines

   11700:       0006103e        dsrl32  v0,a2,0x0
   11704:       7c89f803        dext    a5,a0,0x0,0x20
   11708:       0004403e        dsrl32  a4,a0,0x0
   1170c:       7ccaf803        dext    a6,a2,0x0,0x20
   11710:       012a589c        dmul    a7,a5,a6
   11714:       010a509c        dmul    a6,a4,a6
   11718:       0122489c        dmul    a5,a5,v0
   1171c:       0102409c        dmul    a4,a4,v0
   11720:       012a482d        daddu   a5,a5,a6
   11724:       000b103e        dsrl32  v0,a7,0x0
   11728:       0049102d        daddu   v0,v0,a5
   1172c:       184a0003        bgeuc   v0,a6,1173c <.L2>
   11730:       24090001        li      a5,1
   11734:       0009483c        dsll32  a5,a5,0x0
   11738:       0109402d        daddu   a4,a4,a5

I happened to have a GCC build dir around so I greped for __multi3 and found it hiding in muldi3.o.

Maybe that'obvious for those in the know, not me :) Also that .o file contained MIPS III code and I was able to get GCC to emit a reference to __multi3 for MIPS III or MIPS64R1 targets, so version of __multi3 even seems unused.

Ralf

Matthew Fortune

2017-08-17 08:49 UTC #9 lore raw

Quoted 67 lines · Ralf Baechle

Quoted 37 lines · Thomas Petazzoni
Date: Sun, 13 Aug 2017 22:46:02 +0200
From: Thomas Petazzoni <[email protected]>
To: Ralf Baechle <[email protected]>
Cc: Matthew Fortune <[email protected]>,
[email protected], Waldemar Brodkorb <[email protected]>
Subject: Re: undefined reference to `__multi3' when building with gcc 7.x
Content-Type: text/plain; charset=US-ASCII
Hello,
Quoted 17 lines · Ralf Baechle
Quoted 11 lines
Quoted 11 lines
Chances are it's something specific to MIPS64 R6. Before trying your config file I also tried a number of other defconfigs and all built well.
Here's a test case which generates a reference to __multi3:
unsigned long func(unsigned long a, unsigned long b)
{
 return a > (~0UL) / b;
}
Quoted 5 lines
Quoted 3 lines
GCC rearanges above statement to:
	return (unsigned __int128)a * (unsigned __int128) b > 0xffffffff;
And this is normal/expected ?
Quoted 7 lines
Without consideration of performance, It's certainly is valid code. And with that I can't drop the issue as a GCC code generation bug.
However it seems GCC itself doesn't seem to have a __multi3 in its libgcc2 - which indeed would be a GCC issue - at least none I was easily able to find with grep so I'm adding Matthew Fortune to cc in the hope he can shed some light on this.
Quoted 5 lines
Indeed, I don't see __multi3 implemented in libgcc in the source code, but it's probably because it's tricky to see its implementation, as it really is there:
$ ./bin/mips64el-linux-readelf -a ./mips64el-buildroot-linux-
uclibc/sysroot/lib/libgcc_s.so.1 | grep multi3
> 1747: 00011700 100 FUNC GLOBAL DEFAULT 11 __multi3@@GCC_3.0
> 5511: 00011700 100 FUNC GLOBAL DEFAULT 11 __multi3
Quoted 5 lines
000435e4 -32236(gp) 00011700 00011700 FUNC 11 __multi3
Objdump says:
00011700 <__multi3>:
Quoted 15 lines
 11700: 0006103e dsrl32 v0,a2,0x0
 11704: 7c89f803 dext a5,a0,0x0,0x20
 11708: 0004403e dsrl32 a4,a0,0x0
 1170c: 7ccaf803 dext a6,a2,0x0,0x20
 11710: 012a589c dmul a7,a5,a6
 11714: 010a509c dmul a6,a4,a6
 11718: 0122489c dmul a5,a5,v0
 1171c: 0102409c dmul a4,a4,v0
 11720: 012a482d daddu a5,a5,a6
 11724: 000b103e dsrl32 v0,a7,0x0
 11728: 0049102d daddu v0,v0,a5
 1172c: 184a0003 bgeuc v0,a6,1173c <.L2>
 11730: 24090001 li a5,1
 11734: 0009483c dsll32 a5,a5,0x0
 11738: 0109402d daddu a4,a4,a5

Quoted 7 lines

I happened to have a GCC build dir around so I greped for __multi3 and found it hiding in muldi3.o.
Maybe that'obvious for those in the know, not me :) Also that .o file contained MIPS III code and I was able to get GCC to emit a reference to __multi3 for MIPS III or MIPS64R1 targets, so version of __multi3 even seems unused.

Sorry for the long delay in replying.

I think this could be considered a GCC bug. The multiply pattern support in MIPS GCC is incredibly complex but the net effect is supposed to be a guarantee that certain multiply operations will never need to use helpers and as such we don't provide the helpers in those cases. There are however special cases of course. The widening multiply from DImode to TImode can't be generated by the compiler when working around R4000 errata and the extension is unsigned from DImode to TImode. I don't believe TImode multiplies (i.e. full width 128-bit multiplies) are generated from MIPS GCC but I don't know whether we have sleep-walked into having __int128 support which would muddy the water somewhat.

So... In an R6 build I don't think the __multi3 helper should be generated and I think the offending pattern needs extending for R6 support:

(define_expand "mulditi3"
[(set (match_operand:TI 0 "register_operand")
(mult:TI (any_extend:TI (match_operand:DI 1 "register_operand"))
(any_extend:TI (match_operand:DI 2 "register_operand"))))]
"ISA_HAS_DMULT && !(<CODE> == ZERO_EXTEND && TARGET_FIX_VR4120)"
{
rtx hilo;

if (TARGET_MIPS16)

    {
      hilo = gen_rtx_REG (TImode, MD_REG_FIRST);
      emit_insn (gen_<u>mulditi3_internal (hilo, operands[1], operands[2]));
      emit_move_insn (operands[0], hilo);
    }
  else if (TARGET_FIX_R4000)
    emit_insn (gen_<u>mulditi3_r4000 (operands[0], operands[1], operands[2]));
  else
    emit_insn (gen_<u>mulditi3_internal (operands[0], operands[1],
                                         operands[2]));

DONE;
})

For SI->DI mode multiplies on 32bit R6 we have the following which naturally ports to DI->TI:

(define_expand "mulsidi3_32bit_r6"
[(set (match_operand:DI 0 "register_operand")
(mult:DI (any_extend:DI (match_operand:SI 1 "register_operand"))
(any_extend:DI (match_operand:SI 2 "register_operand"))))]
"!TARGET_64BIT && ISA_HAS_R6MUL"
{
rtx dest = gen_reg_rtx (DImode);
rtx low = mips_subword (dest, 0);
rtx high = mips_subword (dest, 1);

  emit_insn (gen_mulsi3_mul3_nohilo (low, operands[1], operands[2]));
  emit_insn (gen_<su>mulsi3_highpart_r6 (high, operands[1], operands[2]));

emit_move_insn (mips_subword (operands[0], 0), low); emit_move_insn (mips_subword (operands[0], 1), high); DONE;
})

Despite the theory being simple, wiring this up will take time as it also involves getting the costing calculations updated.

Please can you submit it as a GCC bug?

As a workaround you may want to include a version of __multi3 in the kernel until it is resolved.

Thanks,
Matthew

Ralf Baechle

2017-08-17 22:19 UTC #10 lore raw

Quoted 144 lines · Matthew Fortune

Date: Thu, 17 Aug 2017 08:49:13 +0000
From: Matthew Fortune <[email protected]>
To: Ralf Baechle <[email protected]>, Thomas Petazzoni
<[email protected]>
CC: "[email protected]" <[email protected]>, Waldemar
Brodkorb <[email protected]>
Subject: RE: undefined reference to `__multi3' when building with gcc 7.x
Content-Type: text/plain; charset="us-ascii"
Quoted 67 lines · Ralf Baechle
Quoted 37 lines · Thomas Petazzoni
Date: Sun, 13 Aug 2017 22:46:02 +0200
From: Thomas Petazzoni <[email protected]>
To: Ralf Baechle <[email protected]>
Cc: Matthew Fortune <[email protected]>,
[email protected], Waldemar Brodkorb <[email protected]>
Subject: Re: undefined reference to `__multi3' when building with gcc 7.x
Content-Type: text/plain; charset=US-ASCII
Hello,
Quoted 17 lines · Ralf Baechle
Quoted 11 lines
Quoted 11 lines
Chances are it's something specific to MIPS64 R6. Before trying your config file I also tried a number of other defconfigs and all built well.
Here's a test case which generates a reference to __multi3:
unsigned long func(unsigned long a, unsigned long b)
{
 return a > (~0UL) / b;
}
Quoted 5 lines
Quoted 3 lines
GCC rearanges above statement to:
	return (unsigned __int128)a * (unsigned __int128) b > 0xffffffff;
And this is normal/expected ?
Quoted 7 lines
Without consideration of performance, It's certainly is valid code. And with that I can't drop the issue as a GCC code generation bug.
However it seems GCC itself doesn't seem to have a __multi3 in its libgcc2 - which indeed would be a GCC issue - at least none I was easily able to find with grep so I'm adding Matthew Fortune to cc in the hope he can shed some light on this.
Quoted 5 lines
Indeed, I don't see __multi3 implemented in libgcc in the source code, but it's probably because it's tricky to see its implementation, as it really is there:
$ ./bin/mips64el-linux-readelf -a ./mips64el-buildroot-linux-
uclibc/sysroot/lib/libgcc_s.so.1 | grep multi3
> 1747: 00011700 100 FUNC GLOBAL DEFAULT 11 __multi3@@GCC_3.0
> 5511: 00011700 100 FUNC GLOBAL DEFAULT 11 __multi3
Quoted 5 lines
000435e4 -32236(gp) 00011700 00011700 FUNC 11 __multi3
Objdump says:
00011700 <__multi3>:
Quoted 15 lines
 11700: 0006103e dsrl32 v0,a2,0x0
 11704: 7c89f803 dext a5,a0,0x0,0x20
 11708: 0004403e dsrl32 a4,a0,0x0
 1170c: 7ccaf803 dext a6,a2,0x0,0x20
 11710: 012a589c dmul a7,a5,a6
 11714: 010a509c dmul a6,a4,a6
 11718: 0122489c dmul a5,a5,v0
 1171c: 0102409c dmul a4,a4,v0
 11720: 012a482d daddu a5,a5,a6
 11724: 000b103e dsrl32 v0,a7,0x0
 11728: 0049102d daddu v0,v0,a5
 1172c: 184a0003 bgeuc v0,a6,1173c <.L2>
 11730: 24090001 li a5,1
 11734: 0009483c dsll32 a5,a5,0x0
 11738: 0109402d daddu a4,a4,a5
Quoted 7 lines
I happened to have a GCC build dir around so I greped for __multi3 and found it hiding in muldi3.o.
Maybe that'obvious for those in the know, not me :) Also that .o file contained MIPS III code and I was able to get GCC to emit a reference to __multi3 for MIPS III or MIPS64R1 targets, so version of __multi3 even seems unused.
Sorry for the long delay in replying.
I think this could be considered a GCC bug. The multiply pattern support in MIPS GCC is incredibly complex but the net effect is supposed to be a guarantee that certain multiply operations will never need to use helpers and as such we don't provide the helpers in those cases. There are however special cases of course. The widening multiply from DImode to TImode can't be generated by the compiler when working around R4000 errata and the extension is unsigned from DImode to TImode. I don't believe TImode multiplies (i.e. full width 128-bit multiplies) are generated from MIPS GCC but I don't know whether we have sleep-walked into having __int128 support which would muddy the water somewhat.
So... In an R6 build I don't think the __multi3 helper should be generated and I think the offending pattern needs extending for R6 support:
(define_expand "mulditi3"
[(set (match_operand:TI 0 "register_operand")
(mult:TI (any_extend:TI (match_operand:DI 1 "register_operand"))
(any_extend:TI (match_operand:DI 2 "register_operand"))))]
"ISA_HAS_DMULT && !(<CODE> == ZERO_EXTEND && TARGET_FIX_VR4120)"
{
rtx hilo;
if (TARGET_MIPS16)
 {
 hilo = gen_rtx_REG (TImode, MD_REG_FIRST);
 emit_insn (gen_mulditi3_internal (hilo, operands[1], operands[2]));
 emit_move_insn (operands[0], hilo);
 }
 else if (TARGET_FIX_R4000)
 emit_insn (gen_mulditi3_r4000 (operands[0], operands[1], operands[2]));
 else
 emit_insn (gen_mulditi3_internal (operands[0], operands[1],
 operands[2]));
DONE;
})
For SI->DI mode multiplies on 32bit R6 we have the following which naturally ports to DI->TI:
(define_expand "mulsidi3_32bit_r6"
[(set (match_operand:DI 0 "register_operand")
(mult:DI (any_extend:DI (match_operand:SI 1 "register_operand"))
(any_extend:DI (match_operand:SI 2 "register_operand"))))]
"!TARGET_64BIT && ISA_HAS_R6MUL"
{
rtx dest = gen_reg_rtx (DImode);
rtx low = mips_subword (dest, 0);
rtx high = mips_subword (dest, 1);
 emit_insn (gen_mulsi3_mul3_nohilo (low, operands[1], operands[2]));
 emit_insn (gen_<su>mulsi3_highpart_r6 (high, operands[1], operands[2]));
emit_move_insn (mips_subword (operands[0], 0), low); emit_move_insn (mips_subword (operands[0], 1), high); DONE;
})

Quoted 4 lines

Despite the theory being simple, wiring this up will take time as it also involves getting the costing calculations updated.
Please can you submit it as a GCC bug?

Will do.

Quoted 2 lines

As a workaround you may want to include a version of __multi3 in the kernel until it is resolved.

Yes, working on that. This has been made harder by the fact that the implementation of __umulti3 is well hidden in the source :) I now have functioning implementation of __multi3 but it's still too ugly to be committed to the kernel.

And while I agree it should be fixed in GCC at the same time the generated code while convoluted and unnecessarily slow appears to be correct so I think we should support this by adding a suitable __umulti3 to the kernel code as you suggest.

Ralf

Thomas Petazzoni

author

2017-10-07 19:23 UTC #11 lore raw

Hello,

Quoted 15 lines · Ralf Baechle

Quoted 4 lines
Despite the theory being simple, wiring this up will take time as it also involves getting the costing calculations updated.
Please can you submit it as a GCC bug?
Will do.
Quoted 2 lines
As a workaround you may want to include a version of __multi3 in the kernel until it is resolved.
Yes, working on that. This has been made harder by the fact that the implementation of __umulti3 is well hidden in the source :) I now have functioning implementation of __multi3 but it's still too ugly to be committed to the kernel.

Quoted 4 lines

And while I agree it should be fixed in GCC at the same time the generated code while convoluted and unnecessarily slow appears to be correct so I think we should support this by adding a suitable __umulti3 to the kernel code as you suggest.

Has there been any progress on solving this issue ? Either on the GCC side or the kernel side ?

Thanks a lot!

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Thomas Petazzoni

author

2017-11-07 17:39 UTC #12 lore raw

In reply to Ralf Baechle

Hello,

Quoted 8 lines · Ralf Baechle

Quoted 2 lines
As a workaround you may want to include a version of __multi3 in the kernel until it is resolved.
Yes, working on that. This has been made harder by the fact that the implementation of __umulti3 is well hidden in the source :) I now have functioning implementation of __multi3 but it's still too ugly to be committed to the kernel.

Quoted 4 lines

And while I agree it should be fixed in GCC at the same time the generated code while convoluted and unnecessarily slow appears to be correct so I think we should support this by adding a suitable __umulti3 to the kernel code as you suggest.

Has there been some progress on this front? I'm willing to test patches if needed.

Thanks!

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Maciej W. Rozycki

2017-08-17 01:01 UTC #13 lore raw

In reply to Ralf Baechle

Quoted 24 lines · Ralf Baechle

Chances are it's something specific to MIPS64 R6. Before trying your config file I also tried a number of other defconfigs and all built well.
Here's a test case which generates a reference to __multi3:
unsigned long func(unsigned long a, unsigned long b)
{
        return a > (~0UL) / b;
}
GCC rearanges above statement to:
	return (unsigned __int128)a * (unsigned __int128) b > 0xffffffff;
computing which requires 128 bit intermediate results. This is the code generated for MIPS64 R2:
	dmultu	$4,$5
	mfhi	$2
	sltu	$2,$0,$2
And this is for R6:

Quoted 8 lines

	move	$2,$4
	move	$7,$5			# $6/$7 contain the second op
	move	$6,$0
	move	$4,$0
	move	$5,$2			# $4/$5 contain the first op
	sd	$31,8($sp)
	balc	__multi3
	sltu	$2,$0,$2

This looks silly. Why doesn't GCC simply emit:

	dmuhu	$2,$4,$5
	sltu	$2,$0,$2

instead? It's even shorter than the R2 equivalent.

Maciej

Thomas Petazzoni

author

2017-12-03 09:56 UTC #14 lore raw

In reply to Thomas Petazzoni

Hello,

+James Hogan in Cc.

Quoted 13 lines · Thomas Petazzoni

When trying to build the current Linux master with a gcc 7.x toolchain for mips64r6-n32, I'm getting the following build failure:
crypto/scompress.o: In function `.L31':
scompress.c:(.text+0x2a0): undefined reference to __multi3' drivers/base/component.o: In function .L97':
component.c:(.text+0x4a4): undefined reference to __multi3' drivers/base/component.o: In function component_master_add_with_match':
component.c:(.text+0x8c4): undefined reference to __multi3' net/core/ethtool.o: In function ethtool_set_per_queue_coalesce':
ethtool.c:(.text+0x1ab0): undefined reference to `__multi3' Makefile:1000: recipe for target 'vmlinux' failed
make[2]: *** [vmlinux] Error 1

I'm still facing this problem. There was a lengthy thread about it back in August when I reported the problem, but then it calmed down, with no real solution proposed.

Are there plans to fix this at some point?

Thanks a lot,

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

James Hogan

2017-12-05 23:51 UTC #15 lore raw

Quoted 20 lines · Thomas Petazzoni

Hello,
+James Hogan in Cc.
Quoted 13 lines · Thomas Petazzoni
When trying to build the current Linux master with a gcc 7.x toolchain for mips64r6-n32, I'm getting the following build failure:
crypto/scompress.o: In function `.L31':
scompress.c:(.text+0x2a0): undefined reference to __multi3' drivers/base/component.o: In function .L97':
component.c:(.text+0x4a4): undefined reference to __multi3' drivers/base/component.o: In function component_master_add_with_match':
component.c:(.text+0x8c4): undefined reference to __multi3' net/core/ethtool.o: In function ethtool_set_per_queue_coalesce':
ethtool.c:(.text+0x1ab0): undefined reference to `__multi3' Makefile:1000: recipe for target 'vmlinux' failed
make[2]: *** [vmlinux] Error 1

Quoted 5 lines

I'm still facing this problem. There was a lengthy thread about it back in August when I reported the problem, but then it calmed down, with no real solution proposed.
Are there plans to fix this at some point?

I recently fixed a similar issue in 64r6[el]_defconfig, but its not the same as it applies to all gcc versions on mips64r6. Given Ralf appears to be busy I'll take a look.

Cheers
James

Thomas Petazzoni

author

2017-12-06 07:50 UTC #16 lore raw

Hello,

Quoted 9 lines · James Hogan

Quoted 5 lines
I'm still facing this problem. There was a lengthy thread about it back in August when I reported the problem, but then it calmed down, with no real solution proposed.
Are there plans to fix this at some point?
I recently fixed a similar issue in 64r6[el]_defconfig, but its not the same as it applies to all gcc versions on mips64r6. Given Ralf appears to be busy I'll take a look.

Thanks a lot. If you need some help to reproduce the problem, let me know, I can provide simple instructions to produce it.

Best regards,

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

James Hogan

2017-12-07 07:21 UTC #17 lore raw

PATCH 1/1 MIPS: Implement __multi3 for GCC7 MIPS64r6 builds

GCC7 is a bit too eager to generate suboptimal __multi3 calls (128bit multiply with 128bit result) for MIPS64r6 builds, even in code which doesn't explicitly use 128bit types, such as the following:

unsigned long func(unsigned long a, unsigned long b)

{
	return a > (~0UL) / b;
}

Which GCC rearanges to:

return (unsigned __int128)a * (unsigned __int128)b > 0xffffffff;

Therefore implement __multi3, but only for MIPS64r6 with GCC7 as under normal circumstances we wouldn't expect any calls to __multi3 to be generated from kernel code.

Reported-by Thomas Petazzoni <[email protected]>

Signed-off-by James Hogan <[email protected]>

Cc Ralf Baechle <[email protected]>

Cc Maciej W. Rozycki <[email protected]>

Cc Matthew Fortune <[email protected]>

Cc Florian Fainelli <[email protected]>

Cc Waldemar Brodkorb <[email protected]>

Cc [email protected]

---
This should fix the issue Thomas. Thanks for reporting.

---
 arch/mips/lib/Makefile |  3 ++-
 arch/mips/lib/libgcc.h | 17 +++++++++++++++++
 arch/mips/lib/multi3.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 71 insertions(+), 1 deletion(-)
 create mode 100644 arch/mips/lib/multi3.c

arch/mips/lib/Makefile +2 -1

index 78c2affeabf8..e84e12655fa8 100644
@@ -16,4 +16,5 @@ obj-$(CONFIG_CPU_R3000)		+= r3k_dump_tlb.o
 obj-$(CONFIG_CPU_TX39XX)	+= r3k_dump_tlb.o
 
 # libgcc-style stuff needed in the kernel
-obj-y += ashldi3.o ashrdi3.o bswapsi.o bswapdi.o cmpdi2.o lshrdi3.o ucmpdi2.o
+obj-y += ashldi3.o ashrdi3.o bswapsi.o bswapdi.o cmpdi2.o lshrdi3.o multi3.o \
+	 ucmpdi2.o

arch/mips/lib/libgcc.h +17 -0

index 28002ed90c2c..199a7f96282f 100644
@@ -10,10 +10,18 @@ typedef int word_type __attribute__ ((mode (__word__)));
 struct DWstruct {
 	int high, low;
 };
+
+struct TWstruct {
+	long long high, low;
+};
 #elif defined(__LITTLE_ENDIAN)
 struct DWstruct {
 	int low, high;
 };
+
+struct TWstruct {
+	long long low, high;
+};
 #else
 #error I feel sick.
 #endif

@@ -23,4 +31,13 @@ typedef union {
 	long long ll;
 } DWunion;
 
+#if defined(CONFIG_64BIT) && defined(CONFIG_CPU_MIPSR6)
+typedef int ti_type __attribute__((mode(TI)));
+
+typedef union {
+	struct TWstruct s;
+	ti_type ti;
+} TWunion;
+#endif
+
 #endif /* __ASM_LIBGCC_H */

arch/mips/lib/multi3.c new file +52 -0

52 lines not on this page · raw message

Signature · 1 line

2.13.6

Maciej W. Rozycki

2017-12-08 23:56 UTC #18 lore raw

Re: [PATCH] MIPS: Implement __multi3 for GCC7 MIPS64r6 builds

Quoted 4 lines · James Hogan

GCC7 is a bit too eager to generate suboptimal __multi3 calls (128bit multiply with 128bit result) for MIPS64r6 builds, even in code which doesn't explicitly use 128bit types, such as the following:

Quoted 8 lines

unsigned long func(unsigned long a, unsigned long b)
{
	return a > (~0UL) / b;
}
Which GCC rearanges to:
return (unsigned __int128)a * (unsigned __int128)b > 0xffffffff;

You mean:

return (unsigned __int128)a * (unsigned __int128)b > 0xffffffffffffffff;

presumably, or is there another bug here?

Quoted 3 lines

Therefore implement __multi3, but only for MIPS64r6 with GCC7 as under normal circumstances we wouldn't expect any calls to __multi3 to be generated from kernel code.

That does look bad; I'd expect a `umulditi3' (widening 64-bit by 64-bit
unsigned multiplication) kind of operation instead, which should expand internally. And we only really need to execute DMUHU and then check the result for non-zero here, because the value of the low 64 bits of the product does not matter for the evaluation of the expression.

I don't know offhand if such a transformation can be handled by GCC as it
stands by tweaking the MIPS backend without a corresponding update to the middle end though.

Maciej

James Hogan

2017-12-09 07:21 UTC #19 lore raw

Re: [PATCH] MIPS: Implement __multi3 for GCC7 MIPS64r6 builds

Quoted 13 lines · Maciej W. Rozycki

Quoted 11 lines · James Hogan
GCC7 is a bit too eager to generate suboptimal __multi3 calls (128bit multiply with 128bit result) for MIPS64r6 builds, even in code which doesn't explicitly use 128bit types, such as the following:
unsigned long func(unsigned long a, unsigned long b)
{
	return a > (~0UL) / b;
}
Which GCC rearanges to:

Quoted 7 lines

Quoted 1 line
return (unsigned __int128)a * (unsigned __int128)b > 0xffffffff;
You mean:
return (unsigned __int128)a * (unsigned __int128)b > 0xffffffffffffffff;
presumably, or is there another bug here?

Yes, thats what was meant. It was copy + pasted from Ralf's analysis.

Thanks
James

Quoted 17 lines

Quoted 4 lines
Therefore implement __multi3, but only for MIPS64r6 with GCC7 as under normal circumstances we wouldn't expect any calls to __multi3 to be generated from kernel code.
That does look bad; I'd expect a `umulditi3' (widening 64-bit by 64-bit
unsigned multiplication) kind of operation instead, which should expand internally. And we only really need to execute DMUHU and then check the result for non-zero here, because the value of the low 64 bits of the product does not matter for the evaluation of the expression.
I don't know offhand if such a transformation can be handled by GCC as it
stands by tweaking the MIPS backend without a corresponding update to the middle end though.
Maciej

Waldemar Brodkorb

2017-12-27 08:31 UTC #20 lore raw

Re: [PATCH] MIPS: Implement __multi3 for GCC7 MIPS64r6 builds

In reply to James Hogan

Hi James,
James Hogan wrote,

Quoted 9 lines

GCC7 is a bit too eager to generate suboptimal __multi3 calls (128bit multiply with 128bit result) for MIPS64r6 builds, even in code which doesn't explicitly use 128bit types, such as the following:
unsigned long func(unsigned long a, unsigned long b)
{
	return a > (~0UL) / b;
}

Quoted 7 lines

Which GCC rearanges to:
return (unsigned __int128)a * (unsigned __int128)b > 0xffffffff;
Therefore implement __multi3, but only for MIPS64r6 with GCC7 as under normal circumstances we wouldn't expect any calls to __multi3 to be generated from kernel code.

I tested the patch and it works fine for me. I can build a mips64r6 kernel and run the uClibc-ng testsuite inside qemu system emulation.

It works for me with 4.9.x and 4.14.x kernels.

Thanks
Waldemar