tar hangs on 715/75

Hi,
tar (and nscd) hang on my 715/75. Same binaries/libraries work on
the B180. The hang is in __pthread_acquire() called from pthread_initialize(). On the B180 the code sequence goes like this (tracing branches):

IAOQ: 40161eff 4015df47 <pthread_initialize+348>
IAOQ: 4015df53 4006b63f <_init+1136>
IAOQ: 4006b65f 00000103 <__sigprocmask+32>

IAOQ: 4006b66f 4006b687 <__sigprocmask+48>
IAOQ: 4006b693 40161f03 <__sigprocmask+84>
IAOQ: 40161f37 4015df37 <pthread_initialize+404>
IAOQ: 4015df43 4006e983 <_init+1120>
IAOQ: 4006e9af 4006e82f <__cxa_atexit+44>
IAOQ: 4006e867 400860cf <__new_exitfn+56>
IAOQ: 400860db 40160ed7 <____strtod_l_internal+15484>
IAOQ: 40160f07 40160f0b <__pthread_mutex_lock+48>
IAOQ: 40160f0f 4016107f <__pthread_mutex_lock+56>
IAOQ: 40161083 40164c63 <__pthread_mutex_lock+428>
IAOQ: 40164c93 4016508f <__pthread_alt_lock+48>
IAOQ: 401650cf 40165107 <__pthread_acquire+64>
IAOQ: 4016511f 40164c97 <__pthread_acquire+144>
IAOQ: 40164cab 40164d53 <__pthread_alt_lock+72>
IAOQ: 40164d57 40164d63 <__pthread_alt_lock+244>
IAOQ: 40164d7b 40161087 <__pthread_alt_lock+280>
IAOQ: 4016109b 4006e86b <__pthread_mutex_lock+452>
IAOQ: 4006e893 4006e8c7 <__new_exitfn+100>
etc

while on the 715/75 it goes like this:

IAOQ: 40161eff 4015df47 <pthread_initialize+348>
IAOQ: 4015df53 4006b63f <_init+1136>
IAOQ: 4006b65f 00000103 <__sigprocmask+32>

IAOQ: 4006b66f 4006b687
IAOQ: 4006b693 40161f03
IAOQ: 40161f37 4015df37
IAOQ: 4015df43 4006e983
IAOQ: 4006e9af 4006e82f
IAOQ: 4006e867 400860cf
IAOQ: 400860db 40160ed7
IAOQ: 40160f07 40160f0b
IAOQ: 40160f0f 4016107f
IAOQ: 40161083 40164c63
IAOQ: 40164c93 4016508f <__pthread_alt_lock+48>
IAOQ: 401650df 4015dba7 <__pthread_acquire+80>
IAOQ: 4015dbb3 400f546f <_init+208>
IAOQ: 400f5473 00000103 <sched_yield+4>

from which I conclude that __pthread_acquire(int *spinlock) found the spinlock held.

Relevant bit of pthread_initialize() looks like this:

sigprocmask(SIG_BLOCK, &mask, NULL);
/* Register an exit function to kill all other threads. */ /* Do it early so that user-registered atexit functions are called
before pthread_*exit_process. */
#ifndef HAVE_Z_NODELETE
if (__builtin_expect (&__dso_handle != NULL, 1))

    __cxa_atexit ((void (*) (void *)) pthread_atexit_process, NULL,
                  __dso_handle);
  else
#endif
    __on_exit (pthread_onexit_process, NULL);

Do other people see this problem on 715/old machines? Just type 'tar'; if you have the problem you'll never see the usage message.

Even better, does anyone have any suggestions as to what might be causing it? I am using taggarts new tarball nfsroot.

Richard

On Mon, 22 Jan 2001, Richard Hirst wrote:

[ snip! ]

My C100 barfs (Bus error) when I run tar or make (and probably other programs I haven't discovered). dmesg shows:

!!die_if_kernel: tar(6969): Unaligned data reference 28

YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
PSW: 00000000000001001111111100001011

r0-3     00000000 40149938 4015dc7b 401470d4
r4-7     40168148 00000000 00000031 001e8481
r8-11    bb0001a8 bb0001a0 00000001 4001ada0
r12-15   4001ad7c 00000000 0009c5b0 00000000
r16-19   00000000 00000001 0000b71b 40168148
r20-23   001e8000 40159f14 bb000454 00000008
r24-27   40161460 00000000 401470d4 00076808
r28-31   00000016 00000000 bb000700 4006e987
sr0-3    00005187 00005187 00000000 00005187
sr4-7    00005187 00005187 00005187 00005187

IASQ: 00005187 00005187 IAOQ: 4015e0ab 4015e0af
IIR: 0c6011d4 ISR: 00005187 IOR: 401470d4
ORIG_R28: 00000000

I don't know if this may in any way be related.

- Greg

This is because ldcw behaves differently on the 715/75 and the B180.

Take this code (which is basically a bit of libpthread):

========================= ldcw.c ============================= #include <stdio.h>
#include <time.h>

#define MAX_SPIN_COUNT 32
#define SPIN_SLEEP_DURATION 2000000

extern inline long int
testandset (int *spinlock)
{
int ret;

  __asm__ __volatile__(
       "ldcw 0(%2),%0"
       : "=r"(ret), "=m"(*spinlock)
       : "r"(spinlock));

return ret == 0;
}

static void __pthread_acquire(int * spinlock)
{
int cnt = 0;
struct timespec tm;

  while (testandset(spinlock)) {
    if (cnt < MAX_SPIN_COUNT) {
      sched_yield();
      cnt++;
    } else {
      tm.tv_sec = 0;
      tm.tv_nsec = SPIN_SLEEP_DURATION;
      nanosleep(&tm, NULL);
      cnt = 0;
    }

}
}

int s = 1;

int main(int argc, char **argv)
{

//      printf("&s = %p\n", &s);
        __pthread_acquire(&s);

        return 0;
}

================================================================

and compile with "gcc -O -Wall -o ldcw ldcw.c"

Run it on a B180 and it completes; run it on a 715/75 and it loops in __pthread_acquire().

If you uncomment the printf at the beginning of main() it completes on the 715/75 also.

Is there some cacheline requirements on spinlocks that libpthread needs to take in to account?

Richard

Grant tells me spinlock words have to be the first word of a cacheline, so that would be why my example code broke. However, libpthreads uses spinlocks and doesn't appear to force any alignment.

I think the libpthreads spinlock definitions come from glibc/linuxthreads/sysdeps/pthread/bits/pthreadtypes.h, struct pthread_mutex_t, spinlock is inside __m_lock. No alignment is specified. When debugging tar, I found the spinlock word at 0x4014df4c.

Richard

16-byte aligned.

Yeah - this is definitely broken. taggart and jsm are going to play with the alignments. jsm couldn't say for sure off-hand which processors don't really care about the alignment. But we agreed it doesn't matter since the architecture requires it and some of the processor models do care.

I spoke with Scott Norton (http://www.hp.com/hpbooks/authors/norton.html)
and he made it clear the POSIX spec defines pthread_mutex_t to be an "opaque" type - ie it's arch specific. _pthread_fastlock and pthread_spinlock_t types must be aligned to correctly work under parisc and Scott made it clear these are not POSIX data types. Perhaps UNIX98 or some other standard.

What's baffled me is did this ever work under HPUX?

HP published a CD with a bunch of GNU/open source tools for HPUX 11. The glibc version shipped was 1.2.6. Only has references to pthread_mutex_t and none for "spinlock".

So HPUX only supports the pthread_mutex_t and not pthread_spinlock_t. The HPUX kernel support for pthread_mutex_t seems to be __pthread_spinlock_t. __pthread_spinlock_t contains an array where the init sequence figures out where the 16byte alignment falls in the array and uses that offset. I don't want to endorse that as *the* strategy to use since the linux kernel obviously uses compiler built-ins to get's it's alignments right.

thanks,
grant

Grant Grundler
Unix Systems Enablement Lab
+1.408.447.7253

Then add appropriate alignment. What's the problem? We just don't have to do it for any architecture.

The leading underscore should tell you that this is an internal, private type.

Glibc has no support for HPUX. I know some people inside HP worked on it but none of this code is available. Besides, I doubt that the LinuxThreads add-on is used on HPUX.

---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------