SIGILL errors in strncpu (NAT consumption)

Has anyone seen random SIGILL failures in the strncpy function in glibc-2.2.4-19.3?

The failure is caused by a NAT consumption fault in the code sequence shown below.

I'm still analyzing the failure, but it _appears_ that the failure occurs:
- if a VHPT fault occurs at <strncpy+450>
- then a NAT consumption occurs at <strncpy+560>

(preliminary analysis - it may be more complicated than this)

In the failing case, neither source or destination crosses or is near to a page boundary. Source address is reg 1, dest is the stack in reg 4. Length 25 bytes.

If no one else has seen this failure, I'll gather more information about it & try to create a simple failing test case.

We are running 2.4.17 with B0 stepping Itanium.

Note:

	rotating registers/predicates
	speculative loads


	....
	<strncpy+416>:       [MIB] (p16) ld8.s r32=[r20],8
	<strncpy+417>:             (p18) chk.s.i r34,0x20000000001f8c90 <strncpy+944>
	<strncpy+418>:                   nop.b 0x0
	<strncpy+432>:       [MII] (p18) mov r31=r34
	<strncpy+433>:             (p18) czx1.r r24=r34;;
	<strncpy+434>:             (p18) cmp.eq p0,p7=8,r24
	<strncpy+448>:       [MFB] (p18) adds r21=-8,r21
	<strncpy+449>:                   nop.f 0x0
	<strncpy+450>:             (p07) br.cond.dpnt.few 0x20000000001f8b40 <strncpy+608>

	<strncpy+464>:       [MBB] (p18) st8 [r18]=r34,8		<<<--------- if VHPT occurs here

	<strncpy+465>:                   nop.b 0x0
	<strncpy+466>:                   br.ctop.dptk.few 0x20000000001f8a80 <strncpy+416>;;

	<strncpy+480>:       [MFB]       chk.s.m r33,0x20000000001f8cb0 <strncpy+976>
	<strncpy+481>:                   nop.f 0x0
	<strncpy+482>:                   nop.b 0x0
	<strncpy+496>:       [MFB]       mov r31=r33
	<strncpy+497>:                   nop.f 0x0
	<strncpy+498>:                   nop.b 0x0
	`
	<strncpy+512>:       [MIB]       cmp.eq p5,p6=r21,r0
	<strncpy+513>:                   adds r21=-1,r21
	<strncpy+514>:             (p05) br.cond.dptk.few 0x20000000001f8bf0 <strncpy+784>;;

	<strncpy+528>:       [MFI]       nop.m 0x0
	<strncpy+529>:                   nop.f 0x0
	<strncpy+530>:                   mov.i ar.lc=r21

	<strncpy+544>:       [MII]       nop.m 0x0
	<strncpy+545>:             (p06) extr.u r27=r31,0,8
	<strncpy+546>:             (p06) shr.u r31=r31,8;;

	<strncpy+560>:       [MIB]       st1 [r18]=r27,1     <<<<<<<<<<<<<<<<<<<< fails here

Thanks

Jack Steiner ([email protected])          651-683-5302
Principal Engineer                      Core OS/Strategic Software Org
SGI - Silicon Graphics, Inc.            Eagan, MN

Whoops. Wrong version. I dont have the steppings chart at home but:

        family     : Itanium
        model      : 0
        revision   : 6
        cpu MHz    : 799.942992

Thanks

Jack Steiner    (651-683-5302)   (vnet 233-5302)      [email protected]

I've seen NaT consumption faults with an ISV application in a loop involving speculative loads, too. It was very hard to trace back (occured at different simulation times in a huge case) and disappeared after we rewrote the loop and used the latest Intel Fortran compiler. It occured with both B3 and C0 CPUs under 2.4.7 and 2.4.17. I don't have a testcase for this, sorry.

Regards,
Erich

I isolated the strncpy problem to a simple test program. It fails with the new glibc-2.2.4-19.3 within a few seconds.

Works fine with older versions of glibc.

David Mosberger took a look at the strncpy code & spotted the error:

From David:

----
Test case - run ~12 copies of this in parallel.

#include <stdio.h>
#include <signal.h>
#include <string.h>
#include <time.h>

char *dest, *src;

void
sigill_handler(int sig)

{
        fprintf(stderr,"SIGILL: pid %d, dest 0x%lx, src 0x%lx\n",
                getpid(), (long)dest, (long)src);
        exit(1);
}

int
main() {
time_t temp1;
char *p, buffer[1024];

  signal(SIGILL, sigill_handler);
  
  time(&temp1);
  src = ctime(&temp1);

  dest = buffer;

  printf("%d\n", strlen(src));

while(1)

      strncpy(buffer,src,strlen(src));
}

Thanks

Jack Steiner    (651-683-5302)   (vnet 233-5302)      [email protected]

Erich> I've seen NaT consumption faults with an ISV application in a Erich> loop involving speculative loads, too. It was very hard to Erich> trace back (occured at different simulation times in a huge Erich> case) and disappeared after we rewrote the loop and used the Erich> latest Intel Fortran compiler. It occured with both B3 and C0 Erich> CPUs under 2.4.7 and 2.4.17. I don't have a testcase for Erich> this, sorry.

It's due to a glibc bug that was introduced last August when strncpy() was rewritten. I sent a bug report (and preliminary patch) to the author and am waiting to hear back.

	--david

Do you have the bug-report ID on GNATS? I am not able to find it in the database to known if it is being worked-out by the maintainer.

FiX

Francois-Xavier "FiX" KOWALSKI

Hi there,

I have a favor to ask David.
If possible, plase give me the patch you had resolved this problem. I have been looking for the patch in all email(linux-ia64) but I could not find this patch.

Thanks.

Hideki> I have a favor to ask David. If possible, plase give me Hideki> the patch you had resolved this problem. I have been looking Hideki> for the patch in all email(linux-ia64) but I could not find Hideki> this patch.

Jack just sent the patch. Caveat: it has not been tested much. You may be better off reverting to the pre-August 2001 version of strncpy() if stability is what you want.

	--david

Francois-Xavier> Do you have the bug-report ID on GNATS? I am not Francois-Xavier> able to find it in the database to known if it is Francois-Xavier> being worked-out by the maintainer.

No, I don't. I just reported it to Jakub Jelenik.

	--david

The error I've seen didn't have anything to do with strncpy(), the loop where the strange SIGILL and "NaT consumption" came from was:

      DO 502 IB=1,NBCUT0
      if(IB.GT.NBNCST.AND.IB.LE.NBNCEN) GO TO 502
      IP1=LCU(1,IB)
      IP2=LCU(2,IB)
      IF(LQ(1,IP1).LT.-NBC.OR.LQ(1,IP2).LT.-NBC) GO TO 502
      IDP1=NDIR(ICU(1,IB))
      LQ(IDP1,IP1)=LCB(1,IB)
      IDP2=NDIR(ICU(2,IB))
      LQ(IDP2,IP2)=LCB(2,IB)

502 CONTINUE

You shouldn't blame me for the first IF condition, it's a third party (ISV) code. The assembler code produced by the Fortran compiler looked correct.

What change did you make for strncpy()? Did it somehow produce a NaT somewhere where it could influence a Fortran program? I'd like to understand whether the problem comes from a strange combination of instructions or somehow propagates from glibc. Splitting the loop and eliminating the first IF helped in this case, so it's improbable that strncpy() is related to this.

Thanks,
best regards,
Erich

Erich> What change did you make for strncpy()? Did it somehow Erich> produce a NaT somewhere where it could influence a Fortran Erich> program? I'd like to understand whether the problem comes Erich> from a strange combination of instructions or somehow Erich> propagates from glibc.

The glibc routine simply was buggy. Garbage in, garbage out, not surprise here. I suspect your Fortran problem is something entirely different.

	--david

Thanks, I should have read the previous messages to this thread, the question was answered before... The disadvantage of reading the digest form of the mailing list traffic.

Regards,
Erich

Hi Jack-san and David-san.

Thank you for sending the patch.
Yes, I will take care to do it.

At Thu, 4 Apr 2002 07:54:57 -0800,

Just to avoid confusing others, I'd like to clarify this point:

This was a brain fart. It is of course legal to do this as there is no dependency violation (both instructions only read r[MEMLAT]) and hence the effect of the instructions is as if they had been executed sequentially. Thanks to Jim Hull for setting me straight.

The rest of the patch should still apply.

	--david