Skip to contentSkip to conversation

cyclictest hangs with posix interval timers !?

DISCUSSION Steven Scholz 3 participants 24 messages Jan 4, 2007 – Jan 8, 2007

lore
Share & tools
lore
#0 lore raw

Hi,

These are my very first steps into CONFIG_PREEMPT_RT.

I built a kernel 2.6.20-rc3-rt0, booted it on a x86 based single board computer and tried to run cyclictest.

"cyclictest -t 1 -p 80 -i 10000 -l 10000 -n" runs fine.

But "cyclictest -t 1 -p 80 -i 10000 -l 10000" hangs.

Putting in some printfs I got the impression that the timerthread() blocks at sigwait()

		case MODE_SYS_ITIMER:
			if (sigwait(&sigset, &sigs) < 0) {
				goto out;
			}
			break;

So I I booted the OSADL Live CD provided at

http://www.osadl.org/projects-live-cd.0.html

on the same board and cyclictest runs fine in both modes.

I build a kernel 2.6.17 + 2.6.17-rt3 patch with the same kernel config used on this CD. But cyclictest hangs when using interval timer.

So I recon it has something to do with the rootfs I am using (created with PTXdist).

Will I need a special version of glibc or libpthread? NTPL? I am using a cross tool chain gcc-3.4.4-glibc-2.3.5 for i686 built with "crosstool".

Any ideas what I could be missing? Or how I could find out.

Thanks a million!

--
Steven

#1 lore raw
Quoted 3 lines · Steven Scholz

Will I need a special version of glibc or libpthread? NTPL? I am using a cross tool chain gcc-3.4.4-glibc-2.3.5 for i686 built with "crosstool".

You usually need NPTL, and if I remember correctly you also needed the hrtimer-support lib for older glibcs.

Robert

Signature · 5 lines
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9
#2 lore raw

Robert,

Thanks for your fast reply.

Quoted 6 lines
Quoted 3 lines

Will I need a special version of glibc or libpthread? NTPL? I am using a cross tool chain gcc-3.4.4-glibc-2.3.5 for i686 built with "crosstool".

You usually need NPTL, and if I remember correctly you also needed the hrtimer-support lib for older glibcs.

How could I find out if the glibc on the OASDL CD uses a patch glibc? I only see libc-2.3.5.so ...

Steven

#3 lore raw
In reply to Robert Schwebel
Quoted 7 lines · Robert Schwebel
Quoted 3 lines · Steven Scholz

Will I need a special version of glibc or libpthread? NTPL? I am using a cross tool chain gcc-3.4.4-glibc-2.3.5 for i686 built with "crosstool".

You usually need NPTL, and if I remember correctly you also needed the hrtimer-support lib for older glibcs.

Any pointers where I could find this hrtimer-support?

Steven

#4 lore raw
Quoted 9 lines · Steven Scholz
Quoted 7 lines · Robert Schwebel
Quoted 3 lines · Steven Scholz

Will I need a special version of glibc or libpthread? NTPL? I am using a cross tool chain gcc-3.4.4-glibc-2.3.5 for i686 built with "crosstool".

You usually need NPTL, and if I remember correctly you also needed the hrtimer-support lib for older glibcs.

Any pointers where I could find this hrtimer-support?

cyclictest does not need any extra functionality. You only need a new glibc if you want to use priority inheritance mutexes.

tglx

#5 lore raw

Hi Thomas,

Quoted 9 lines
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines

Will I need a special version of glibc or libpthread? NTPL? I am using a cross tool chain gcc-3.4.4-glibc-2.3.5 for i686 built with "crosstool".

You usually need NPTL, and if I remember correctly you also needed the hrtimer-support lib for older glibcs.

Any pointers where I could find this hrtimer-support?

cyclictest does not need any extra functionality. You only need a new glibc if you want to use priority inheritance mutexes.

Ok. Then any idea how I could find out why sigwait() is blocking?

Thanks!

Steven

#6 lore raw
Quoted 4 lines · Steven Scholz
Quoted 2 lines

cyclictest does not need any extra functionality. You only need a new glibc if you want to use priority inheritance mutexes.

Ok. Then any idea how I could find out why sigwait() is blocking?

Can you strace cyclictest and send me the output ?

# strace -f cyclictest -t 1 -p 80 -i 10000 -l 10 -q

tglx

#7 lore raw
In reply to Thomas Gleixner
Quoted 6 lines · Thomas Gleixner
Quoted 5 lines · Steven Scholz
Quoted 4 lines · Robert Schwebel
Quoted 3 lines · Steven Scholz

Will I need a special version of glibc or libpthread? NTPL? I am using a cross tool chain gcc-3.4.4-glibc-2.3.5 for i686 built with "crosstool".

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

You usually need NPTL, and if I remember correctly you also needed the hrtimer-support lib for older glibcs.

Any pointers where I could find this hrtimer-support?

cyclictest does not need any extra functionality. You only need a new glibc if you want to use priority inheritance mutexes.

So IIUC the cyclictest should run and give results even on kernels with no real time, preempt, lowlatency support at all?

Just tried it on a plain 2.6.19. posix timer version still hangs ... --
Steven

#8 lore raw
Quoted 5 lines · Steven Scholz
Quoted 2 lines

cyclictest does not need any extra functionality. You only need a new glibc if you want to use priority inheritance mutexes.

So IIUC the cyclictest should run and give results even on kernels with no real time, preempt, lowlatency support at all?

Right.

Quoted 1 line

Just tried it on a plain 2.6.19. posix timer version still hangs ...

That's caused by user space breakage. Looking at the strace you sent:

[pid   932] getpid()                    = 932
[pid   932] rt_sigprocmask(SIG_SETMASK, [ALRM RTMIN], NULL, 8) = 0
[pid   932] gettid()                    = 932
[pid   932] rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
[pid   932] sched_setscheduler(0, SCHED_FIFO, { 80 }) = 0
[pid   932] clock_gettime(CLOCK_MONOTONIC, {325, 223507890}) = 0
[pid   932] rt_sigaction(SIGALRM, {0xb7ed72c0, ~[], 0}, NULL, 8) = 0
[pid   932] rt_sigprocmask(SIG_BLOCK, NULL, [ALRM RTMIN], 8) = 0
[pid   932] rt_sigsuspend(~[ALRM RT_1] <unfinished ...>
[pid   930] <... nanosleep resumed> NULL) = 0

There is neither a call of timer_create() nor of timer_settime(). So the posix timer is not setup, which results in the endless wait for the signal.

This looks like a serious problem with your cross compiler / glibc setup. Can you compile cyclictest on one of your desktop machines and link it statically ?

tglx

#9 lore raw
Quoted 3 lines · Thomas Gleixner

This looks like a serious problem with your cross compiler / glibc setup. Can you compile cyclictest on one of your desktop machines and link it statically?

Steven, if this is a standard ptxdist toolchain it is built with stock crosstool, so if there is really a problem with the toolchain you might want to check if crosstool does everything right. However, we havn't seen such effects, although all our realtime testing is currently done with the soon-to-be-released OSELAS.Toolchain() toolchains, which are built with native PTXdist instead of crosstool.

Robert

Signature · 5 lines
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9
#10 lore raw
In reply to Thomas Gleixner

Hi Thomas,

Quoted 4 lines
Quoted 1 line

Just tried it on a plain 2.6.19. posix timer version still hangs ...

That's caused by user space breakage. Looking at the strace you sent:

Quoted 10 lines
[pid   932] getpid()                    = 932
[pid   932] rt_sigprocmask(SIG_SETMASK, [ALRM RTMIN], NULL, 8) = 0
[pid   932] gettid()                    = 932
[pid   932] rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
[pid   932] sched_setscheduler(0, SCHED_FIFO, { 80 }) = 0
[pid   932] clock_gettime(CLOCK_MONOTONIC, {325, 223507890}) = 0
[pid   932] rt_sigaction(SIGALRM, {0xb7ed72c0, ~[], 0}, NULL, 8) = 0
[pid   932] rt_sigprocmask(SIG_BLOCK, NULL, [ALRM RTMIN], 8) = 0
[pid   932] rt_sigsuspend(~[ALRM RT_1] <unfinished ...>
[pid   930] <... nanosleep resumed> NULL) = 0

Ah. Thanks for that hint!

Quoted 3 lines

This looks like a serious problem with your cross compiler / glibc setup. Can you compile cyclictest on one of your desktop machines and link it statically ?

gcc -Wall -O2 -lpthread -lrt cyclictest.c -o cyclictest -static

results in

/tmp/ccIsApGI.o: In function `main':
cyclictest.c:(.text+0x4a2): undefined reference to pthread_create' cyclictest.c:(.text+0x6d9): undefined reference to pthread_kill' cyclictest.c:(.text+0x6f0): undefined reference to pthread_join' /tmp/ccIsApGI.o: In function timerthread':
cyclictest.c:(.text+0x839): undefined reference to clock_gettime' cyclictest.c:(.text+0x881): undefined reference to timer_settime' cyclictest.c:(.text+0x90c): undefined reference to clock_gettime' cyclictest.c:(.text+0xa7b): undefined reference to clock_gettime' cyclictest.c:(.text+0xa8c): undefined reference to clock_nanosleep' cyclictest.c:(.text+0xb08): undefined reference to timer_delete' cyclictest.c:(.text+0xb79): undefined reference to clock_gettime' cyclictest.c:(.text+0xbd8): undefined reference to clock_nanosleep' cyclictest.c:(.text+0xc37): undefined reference to `timer_create'
collect2: ld returned 1 exit status

???

Shouldn't pthread_* be in libpthread?

Steven

#11 lore raw
In reply to Thomas Gleixner

Thomas,

Quoted 13 lines

That's caused by user space breakage. Looking at the strace you sent:

[pid   932] getpid()                    = 932
[pid   932] rt_sigprocmask(SIG_SETMASK, [ALRM RTMIN], NULL, 8) = 0
[pid   932] gettid()                    = 932
[pid   932] rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
[pid   932] sched_setscheduler(0, SCHED_FIFO, { 80 }) = 0
[pid   932] clock_gettime(CLOCK_MONOTONIC, {325, 223507890}) = 0
[pid   932] rt_sigaction(SIGALRM, {0xb7ed72c0, ~[], 0}, NULL, 8) = 0
[pid   932] rt_sigprocmask(SIG_BLOCK, NULL, [ALRM RTMIN], 8) = 0
[pid   932] rt_sigsuspend(~[ALRM RT_1] <unfinished ...>
[pid   930] <... nanosleep resumed> NULL) = 0
Quoted 3 lines

There is neither a call of timer_create() nor of timer_settime(). So the posix timer is not setup, which results in the endless wait for the signal.

cyclictest runs fine on my SuSE 10.0 host system. But: there are *no* calls to any timer_* function!!!
Has it to with using TLS?

:~/Projekte/Latency/cyclictest> ldd ./cyclictest

        linux-gate.so.1 =>  (0xffffe000)
        libpthread.so.0 => /lib/tls/libpthread.so.0 (0x4003a000)
        librt.so.1 => /lib/tls/librt.so.1 (0x4004c000)
        libc.so.6 => /lib/tls/libc.so.6 (0x40055000)
        /lib/ld-linux.so.2 (0x40000000)

Steven

#12 lore raw

Steven,

Quoted 6 lines · Steven Scholz
Quoted 3 lines

There is neither a call of timer_create() nor of timer_settime(). So the posix timer is not setup, which results in the endless wait for the signal.

cyclictest runs fine on my SuSE 10.0 host system. But: there are *no* calls to any timer_* function!!!

Did you check that with strace ?

Quoted 1 line

Has it to with using TLS?

No.

Quoted 6 lines

:~/Projekte/Latency/cyclictest> ldd ./cyclictest

        linux-gate.so.1 =>  (0xffffe000)
        libpthread.so.0 => /lib/tls/libpthread.so.0 (0x4003a000)
        librt.so.1 => /lib/tls/librt.so.1 (0x4004c000)
        libc.so.6 => /lib/tls/libc.so.6 (0x40055000)
        /lib/ld-linux.so.2 (0x40000000)

The timer_* functions are in librt.

tglx

#13 lore raw

Thomas,

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

There is neither a call of timer_create() nor of timer_settime(). So the posix timer is not setup, which results in the endless wait for the signal.

cyclictest runs fine on my SuSE 10.0 host system. But: there are *no* calls to any timer_* function!!!

Did you check that with strace ?

Yes. I attach that run.

gcc -Wall -O2 -lpthread -lrt cyclictest.c -o cyclictest

ldd ./cyclictest

        linux-gate.so.1 =>  (0xffffe000)
        libpthread.so.0 => /lib/tls/libpthread.so.0 (0x4003a000)
        librt.so.1 => /lib/tls/librt.so.1 (0x4004c000)
        libc.so.6 => /lib/tls/libc.so.6 (0x40055000)
        /lib/ld-linux.so.2 (0x40000000)

strace ./cyclictest -t 1 -p 80 -i 10000 -l 10 -v 2> cyclictest.txt

grep timer cyclictest.txt
<nothing>

Quoted 3 lines
Quoted 2 lines

Has it to with using TLS?

No.

IIUC TLS is only usefull for SMP systems?

Quoted 8 lines
Quoted 6 lines

:~/Projekte/Latency/cyclictest> ldd ./cyclictest

        linux-gate.so.1 =>  (0xffffe000)
        libpthread.so.0 => /lib/tls/libpthread.so.0 (0x4003a000)
        librt.so.1 => /lib/tls/librt.so.1 (0x4004c000)
        libc.so.6 => /lib/tls/libc.so.6 (0x40055000)
        /lib/ld-linux.so.2 (0x40000000)

The timer_* functions are in librt.

Then why does the static linking not work?

Steven

execve("./cyclictest", ["./cyclictest", "-t", "1", "-p", "80", "-i", "10000", "-l", "10", "-v"], [/* 69 vars */]) = 0 uname({sys="Linux", node="pinguin", ...}) = 0

brk(0)                                  = 0x804c000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3

fstat64(3, {st_mode=S_IFREG|0644, st_size=135508, ...}) = 0
old_mmap(NULL, 135508, PROT_READ, MAP_PRIVATE, 3, 0) = 0x40017000
close(3) = 0
open("/lib/tls/libpthread.so.0", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0pH\0\000"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=93266, ...}) = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40039000 old_mmap(NULL, 70104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x4003a000 madvise(0x4003a000, 70104, MADV_SEQUENTIAL|0x1) = 0

old_mmap(0x40048000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xd000) = 0x40048000
old_mmap(0x4004a000, 4568, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x4004a000

close(3) = 0
open("/lib/tls/librt.so.1", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0000\36\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=40833, ...}) = 0
old_mmap(NULL, 33384, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x4004c000
madvise(0x4004c000, 33384, MADV_SEQUENTIAL|0x1) = 0
old_mmap(0x40053000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x40053000
close(3) = 0
open("/lib/tls/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\240O\1"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1417095, ...}) = 0
old_mmap(NULL, 1174524, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x40055000
madvise(0x40055000, 1174524, MADV_SEQUENTIAL|0x1) = 0
old_mmap(0x4016e000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x119000) = 0x4016e000
old_mmap(0x40172000, 7164, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x40172000
close(3) = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40174000
mprotect(0x4016e000, 4096, PROT_READ) = 0
set_thread_area({entry_number:-1 -> 6, base_addr:0x401746c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
munmap(0x40017000, 135508) = 0
set_tid_address(0x40174708) = 5192
rt_sigaction(SIGRTMIN, {0x4003e3c0, [], SA_SIGINFO}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x4003e440, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
_sysctl({{CTL_KERN, KERN_VERSION, 0, 20529, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 2, 0xbf853c00, 34, (nil), 0}) = 0
geteuid32() = 0
open("/proc/version", O_RDONLY) = 3
read(3, "Linux version 2.6.13-15.8-smp (g"..., 255) = 138
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
rt_sigaction(SIGINT, {0x8048c00, [INT], SA_RESTART}, {SIG_DFL}, 8) = 0
rt_sigaction(SIGTERM, {0x8048c00, [TERM], SA_RESTART}, {SIG_DFL}, 8) = 0
brk(0) = 0x804c000
brk(0x806d000) = 0x806d000
mmap2(NULL, 2101248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40175000
mprotect(0x40175000, 4096, PROT_NONE) = 0
clone(child_stack=0x403754c4, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x40375bf8, {entry_number:6, base_addr:0x40375bb0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0x40375bf8) = 5193

nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 10000000}, NULL)          = 0

fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40017000
write(1, " 0: 0: 0\n", 27) = 27
nanosleep({0, 10000000}, NULL) = 0
write(1, " 0: 1: 3508\n", 27) = 27
nanosleep({0, 10000000}, NULL) = 0

write(1, "       0:       2:    5517\n", 27) = 27
write(1, "       0:       3:    7510\n", 27) = 27

nanosleep({0, 10000000}, NULL) = 0
write(1, " 0: 4: 9508\n", 27) = 27
nanosleep({0, 10000000}, NULL) = 0
write(1, " 0: 5: 11503\n", 27) = 27
nanosleep({0, 10000000}, NULL) = 0

write(1, "       0:       6:   13517\n", 27) = 27
write(1, "       0:       7:   15512\n", 27) = 27

nanosleep({0, 10000000}, NULL) = 0
write(1, " 0: 8: 17518\n", 27) = 27
nanosleep({0, 10000000}, NULL) = 0
write(1, " 0: 9: 19519\n", 27) = 27

nanosleep({0, 10000000}, NULL)          = 0
nanosleep({0, 50000000}, NULL)          = 0
munmap(0x40017000, 4096)                = 0
exit_group(0)                           = ?
#14 lore raw

Steven,

Quoted 1 line · Steven Scholz

strace ./cyclictest -t 1 -p 80 -i 10000 -l 10 -v 2> cyclictest.txt

You need to run "strace -f ...." otherwise you don't get the thread strace.

Quoted 3 lines
Quoted 2 lines
Quoted 1 line

Has it to with using TLS?

No.

IIUC TLS is only usefull for SMP systems?

Thread Local Storage is used to store per thread local information. This is needed as the threads of a process share the same address space. NPTL needs TLS support.

Quoted 2 lines
Quoted 1 line

The timer_* functions are in librt.

Then why does the static linking not work?

Are the static libs available ?

tglx

#15 lore raw
Quoted 7 lines · Thomas Gleixner

Steven,

Quoted 1 line · Steven Scholz

strace ./cyclictest -t 1 -p 80 -i 10000 -l 10 -v 2> cyclictest.txt

You need to run "strace -f ...." otherwise you don't get the thread strace.

Ok. My fault. Sorry. Of course now the trace contains

[pid 20910] timer_create(CLOCK_MONOTONIC, {(nil), 14, SIGEV_THREAD_ID, {20910}}, {(nil)}) = 0
[pid 20910] timer_settime(0, TIMER_ABSTIME, {it_interval={0, 10000000}, it_value={3908778, 132879754}}, NULL) = 0
[pid 20910] timer_delete(0) = 0

Quoted 4 lines
Quoted 2 lines
Quoted 1 line

The timer_* functions are in librt.

Then why does the static linking not work?

Are the static libs available ?

Ah. Maybe not. Which would that be? libpthread.a, librt.a, libc.a?

Steven

#16 lore raw
In reply to Thomas Gleixner

Thomas,

Quoted 4 lines
Quoted 2 lines
Quoted 1 line

The timer_* functions are in librt.

Then why does the static linking not work?

Are the static libs available ?

Well. On the host system with /lib/tls/librt.so.1 there is only a /usr/lib/librt.a. So maybe that does not work.

But the crosstool chain (built with crosstool) has all the static libs ...

Steven

#17 lore raw
In reply to Thomas Gleixner

Thomas,

Quoted 3 lines

This looks like a serious problem with your cross compiler / glibc setup. Can you compile cyclictest on one of your desktop machines and link it statically ?

As I wrote I have problems linking it statically!

gcc -Wall -O2 -lpthread -lrt cyclictest.c -o cyclictest -static

results in

/tmp/cc30ryGd.o: In function `main':
cyclictest.c:(.text+0x4a2): undefined reference to pthread_create' cyclictest.c:(.text+0x6d9): undefined reference to pthread_kill' cyclictest.c:(.text+0x6f0): undefined reference to pthread_join' /tmp/cc30ryGd.o: In function timerthread':
cyclictest.c:(.text+0x839): undefined reference to clock_gettime' cyclictest.c:(.text+0x881): undefined reference to timer_settime' cyclictest.c:(.text+0x90c): undefined reference to clock_gettime' cyclictest.c:(.text+0xa7b): undefined reference to clock_gettime' cyclictest.c:(.text+0xa8c): undefined reference to clock_nanosleep' cyclictest.c:(.text+0xb08): undefined reference to timer_delete' cyclictest.c:(.text+0xb79): undefined reference to clock_gettime' cyclictest.c:(.text+0xbd8): undefined reference to clock_nanosleep' cyclictest.c:(.text+0xc37): undefined reference to `timer_create'
collect2: ld returned 1 exit status

while

gcc -Wall -O2 cyclictest.c -o cyclictest -lrt -lpthread -static

compiles and links fine!

Any idea why the order matters?

--
Steven

#18 lore raw
Quoted 5 lines · Steven Scholz

As I wrote I have problems linking it statically!

gcc -Wall -O2 -lpthread -lrt cyclictest.c -o cyclictest -static

results in

...

Quoted 7 lines

while

gcc -Wall -O2 cyclictest.c -o cyclictest -lrt -lpthread -static

compiles and links fine!

Any idea why the order matters?

The order does always matter. From 'info gcc':

It makes a difference where in the command you write this option; the linker searches and processes libraries and object files in the order they are specified. Thus, foo.o -lz bar.o' searches library z' after file foo.o' but before bar.o'. If bar.o' refers to functions in z', those functions may not be loaded.

So in your first case, run with -v to see the linker options:

rsc@isonoe:~/tmp/cyclictest$ gcc -v -Wall -O2  -lpthread -lrt cyclictest.c -o cyclictest -static 2>&1 | grep collect2
 /usr/lib/gcc/i486-linux-gnu/4.1.2/collect2 -m elf_i386 -static -o
cyclictest /usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/crt1.o
/usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/crti.o
/usr/lib/gcc/i486-linux-gnu/4.1.2/crtbeginT.o
-L/usr/lib/gcc/i486-linux-gnu/4.1.2 -L/usr/lib/gcc/i486-linux-gnu/4.1.2
-L/usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib -L/lib/../lib
-L/usr/lib/../lib -lpthread -lrt /tmp/cc2ooNdo.o --start-group -lgcc
-lgcc_eh -lc --end-group /usr/lib/gcc/i486-linux-gnu/4.1.2/crtend.o
/usr/lib/gcc/i486-linux-gnu/4.1.2/../../../../lib/crtn.o
collect2: ld returned 1 exit status

... crt1.o crti.o crtbeginT.o -lpthread -lrt cc2ooNdo.o crtend.o

Robert

Signature · 5 lines
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9
#19 lore raw
In reply to Thomas Gleixner

Hi,

Quoted 7 lines

There is neither a call of timer_create() nor of timer_settime(). So the posix timer is not setup, which results in the endless wait for the signal.

This looks like a serious problem with your cross compiler / glibc setup. Can you compile cyclictest on one of your desktop machines and link it statically ?

Well the statically linked version on SuSE 10.0 hangs as well.

Using the crosstool chains

	gcc-4.1.1-glibc-2.3.6-linux-2.6.15.1 (2.6.12 headers)
	gcc-3.4.4-glibc-2.3.5-linux-2.6.12

the cyclictest segafaults on the SuSE host and target system. strace attached.

Any more ideas? Maybe? Please?

Steven

execve("./cyclictest", ["./cyclictest", "-t", "1", "-p", "80", "-i", "10000", "-l", "10", "-v"], [/* 69 vars */]) = 0
uname({sys="Linux", node="pinguin", ...}) = 0
set_thread_area({entry_number:-1 -> 6, base_addr:0x80b9580, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
getpid() = 17701
rt_sigaction(SIGRTMIN, {0x804db08, [], 0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x804dc31, [RTMIN], 0}, NULL, 8) = 0
rt_sigaction(SIGRT_2, {0x804dddc, [], 0}, NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [RTMIN], NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RT_1], NULL, 8) = 0
_sysctl({{CTL_KERN, KERN_VERSION, 0, 20431, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 2, 0xbfd40714, 34, (nil), 0}) = 0

brk(0)                                  = 0x8109000
brk(0x812a000)                          = 0x812a000
geteuid32()                             = 0
open("/proc/version", O_RDONLY)         = 3

read(3, "Linux version 2.6.13-15.8-smp (g"..., 255) = 138
close(3) = 0
rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
rt_sigaction(SIGINT, {0x804f724, [INT], SA_RESTART}, {SIG_DFL}, 8) = 0
rt_sigaction(SIGTERM, {0x804f724, [TERM], SA_RESTART}, {SIG_DFL}, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
pipe([3, 4]) = 0
clone(Process 17702 attached
child_stack=0x811b304, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND) = 17702
[pid 17701] write(4, "\0\0\0\0\5\0\0\0\0\0\0\0\0\0\0\20{J\5\10\0\0\0\0\17\0\0"..., 148 <unfinished ...>
[pid 17702] set_thread_area({entry_number: <unfinished ...>
[pid 17701] <... write resumed> ) = 148
[pid 17702] <... set_thread_area resumed> 6, base_addr:0x80b99a0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 [pid 17701] rt_sigprocmask(SIG_SETMASK, NULL, <unfinished ...>
[pid 17702] rt_sigprocmask(SIG_SETMASK, ~[TRAP RT_1], <unfinished ...>
[pid 17701] <... rt_sigprocmask resumed> [ALRM RTMIN], 8) = 0
[pid 17702] <... rt_sigprocmask resumed> NULL, 8) = 0
[pid 17701] write(4, "\200\225\v\10\0\0\0\0\0\0\0\0\0\202\4\10\350\222\20\10"..., 148 <unfinished ...>
[pid 17702] read(3, <unfinished ...>
[pid 17701] <... write resumed> ) = 148
[pid 17702] <... read resumed> "\0\0\0\0\5\0\0\0\0\0\0\0\0\0\0\20{J\5\10\0\0\0\0\17\0\0"..., 148) = 148
[pid 17701] rt_sigprocmask(SIG_SETMASK, NULL, <unfinished ...>
[pid 17702] poll( <unfinished ...>
[pid 17701] <... rt_sigprocmask resumed> [ALRM RTMIN], 8) = 0
[pid 17702] <... poll resumed> [{fd=3, events=POLLIN, revents=POLLIN}], 1, 2000) = 1
[pid 17701] rt_sigsuspend([ALRM] <unfinished ...>
[pid 17702] getppid() = 17701
[pid 17702] read(3, "\200\225\v\10\0\0\0\0\0\0\0\0\0\202\4\10\350\222\20\10"..., 148) = 148
[pid 17702] mmap2(NULL, 8388608, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40000000
[pid 17702] mprotect(0x40000000, 4096, PROT_NONE) = 0
[pid 17702] clone(Process 17703 attached
child_stack=0x407ffbc4, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|SIGRT_1) = 17703
[pid 17702] kill(17701, SIGRTMIN <unfinished ...>
[pid 17703] set_thread_area({entry_number: <unfinished ...>
[pid 17701] --- SIGRTMIN (Unknown signal 32) @ 0 (0) ---
[pid 17702] <... kill resumed> ) = 0
[pid 17703] <... set_thread_area resumed> 6, base_addr:0x407ffbe0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 [pid 17701] <... rt_sigsuspend resumed> ) = -1 EINTR (Interrupted system call)
[pid 17702] poll( <unfinished ...>
[pid 17703] getpid( <unfinished ...>
[pid 17701] sigreturn( <unfinished ...>
[pid 17703] <... getpid resumed> ) = 17703
[pid 17701] <... sigreturn resumed> ) = ? (mask now [ALRM RTMIN])
[pid 17703] rt_sigprocmask(SIG_SETMASK, [ALRM RTMIN], <unfinished ...>
[pid 17701] nanosleep({0, 10000000}, <unfinished ...>
[pid 17703] <... rt_sigprocmask resumed> NULL, 8) = 0
[pid 17703] gettid() = 17703
[pid 17703] rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
[pid 17703] sched_setscheduler(0, SCHED_FIFO, { 80 }) = 0
[pid 17703] clock_gettime(CLOCK_MONOTONIC, {3983601, 531609754}) = 0
[pid 17703] --- SIGSEGV (Segmentation fault) @ 0 (0) ---
Process 17703 detached
[pid 17701] <... nanosleep resumed> 0) = ? ERESTART_RESTARTBLOCK (To be restarted)
[pid 17702] <... poll resumed> [{fd=3, events=POLLIN}], 1, 2000) = -1 EINTR (Interrupted system call)
[pid 17701] +++ killed by SIGKILL +++
+++ killed by SIGKILL +++

#20 lore raw
Quoted 10 lines · Steven Scholz

Well the statically linked version on SuSE 10.0 hangs as well.

Using the crosstool chains

	gcc-4.1.1-glibc-2.3.6-linux-2.6.15.1 (2.6.12 headers)
	gcc-3.4.4-glibc-2.3.5-linux-2.6.12

the cyclictest segafaults on the SuSE host and target system. strace attached.

Any more ideas? Maybe? Please?

How much RAM does your box have?

Robert

Signature · 5 lines
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9
#21 lore raw
Quoted 5 lines · Robert Schwebel
Quoted 4 lines · Steven Scholz

Well the statically linked version on SuSE 10.0 hangs as well.

Using the crosstool chains

Quoted 8 lines
Quoted 6 lines
	gcc-4.1.1-glibc-2.3.6-linux-2.6.15.1 (2.6.12 headers)
	gcc-3.4.4-glibc-2.3.5-linux-2.6.12

the cyclictest segafaults on the SuSE host and target system. strace attached.

Any more ideas? Maybe? Please?

How much RAM does your box have?

The desktop has 1GB. The kontron cp306 target has 512MB.

--
Steven

#22 lore raw
In reply to Steven Scholz
Quoted 2 lines · Steven Scholz

So I recon it has something to do with the rootfs I am using (created with PTXdist).

Can you try to use a standard root filesystem (fedora, debian or whatever you have handy) ?

tglx

#23 lore raw
Quoted 6 lines · Thomas Gleixner
Quoted 2 lines · Steven Scholz

So I recon it has something to do with the rootfs I am using (created with PTXdist).

Can you try to use a standard root filesystem (fedora, debian or whatever you have handy) ?

Nothing handy. :-(

But of course I could download some and install it.

Which would you recommend? Shell I try some recent Fedora?

Steven