timer signal loss on RT

I'm trying to run one of our applications while using the RT kernel. I am having a problem receiving timer signals in a RT thread pinned to a processor. The thread is a spinning cpu hog. I am not able to provide source for this application but have written a simple test case that _seems_ to fail the same way. This works fine on a non-RT kernel. I have attached the test case. It basically sets up a 60Hz repeating timer, runs for 10 seconds, then displays how many signals the RT thread caught. Should be around 600. And it is on a vanilla kernel but not on an RT kernel. It does require an SMP box to run. It will NOT lockup your machine if you run it. Worse case would be for 10 seconds. The 2 RT priorities in use are less than 50 BTW.

I'm sure you can find many things wrong with this test case but I'm not sending it for discussion on the merits, or certainly the lack of, an application that contains a cpu hog, but only to show what I think is a problem with RT kernels.

compile with "cc rtc.c -o rtc -lrt -lpthread"

run it and in 10 seconds it should stop and indicate 600 timers signals were caught by the RT thread. On RT, I usually get 0 but often I get some, just not 600.

Thanks in advance
Mark

#include <signal.h>
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <unistd.h>
#include <time.h>
#include <errno.h>
#include <sys/time.h>
#include <sched.h>

#define __NR_sched_set_affinity 241
#define __NR_sched_get_affinity 242

int32_t sig_occured = 0;
int32_t sig_count = 0;
int32_t Sig;
int32_t exit_flg = 0;
int32_t thread_running = 0;

uint32_t pri_mask = 0x01;
uint32_t sec_mask = 0x02;

sigset_t pri_sset;
sigset_t sec_sset;
timer_t timerid;
pthread_t thread_id;

struct itimerspec RTC_value;
struct sigevent sig_event;
struct sigaction sigaction_block;
struct sched_param sparam;

void RTC_isr()

{
    sig_occured = 1;
}

void *thread_code(void *args)

{
    uint64_t nsecs;

    //
    // Pin self to a different processor 
    //
    syscall(__NR_sched_set_affinity, 0, sizeof(sec_mask), &sec_mask);

    //
    // Set self to SCHED_FIFO
    //
    sparam.sched_priority = 30;
    pthread_setschedparam(thread_id, SCHED_FIFO, &sparam);

    Sig = SIGRTMIN;
    sigaction_block.sa_handler = (__sighandler_t)RTC_isr;
    sigaction_block.sa_flags = SA_RESTART;

    if (sigaction(Sig, &sigaction_block, NULL) < 0) {
        perror("sigaction failed: ");
        goto out;
    }

    sig_event.sigev_signo  = Sig;
    sig_event.sigev_notify = SIGEV_SIGNAL;

    if (timer_create(CLOCK_MONOTONIC, &sig_event, &timerid) < 0) {
        perror("timer_create failed: ");
        goto out;
    }

    //
    // UNBlock signals
    //
    sigfillset(&sec_sset);
    pthread_sigmask(SIG_UNBLOCK, &sec_sset, NULL);

    //
    // start the 60 Hz repeating timer running
    //
    nsecs = (uint64_t) (1000000000LL / 60); 

    RTC_value.it_value.tv_sec     = nsecs / 1000000000LL;
    RTC_value.it_value.tv_nsec    = nsecs % 1000000000LL;
    RTC_value.it_interval.tv_sec  = RTC_value.it_value.tv_sec;
    RTC_value.it_interval.tv_nsec = RTC_value.it_value.tv_nsec;

    if (timer_settime(timerid, 0, &RTC_value, NULL) < 0) {
        perror("timer_settime failed: ");
        goto out;
    }

    thread_running = 1;

    while (!exit_flg) {
        if (sig_occured) {
            sig_occured = 0;
            sig_count++;
    }}

out:

    if (timerid)
        timer_delete(timerid);

    return (0);
}

int32_t main(int argc, char **argv)

{
    int32_t i = 0;
    int32_t sleep_cnt = 0;

    //
    // Pin self to a processor 
    //
    syscall(__NR_sched_set_affinity, 0, sizeof(pri_mask), &pri_mask);

    //
    // Set self to SCHED_FIFO
    //
    sparam.sched_priority = 35;
    sched_setscheduler(0, SCHED_FIFO, &sparam);

    //
    // Block signals to self
    //
    sigfillset(&pri_sset);
    sigprocmask(SIG_BLOCK, &pri_sset, NULL);

    pthread_create(&thread_id, NULL, thread_code, NULL);

    while(!thread_running)
        usleep(100);

    while (sleep_cnt < 10) {
        sleep(1);
        sleep_cnt++;
    }

    exit_flg = 1;
    pthread_cancel(thread_id);

    printf("Thread process received %d signals\n", sig_count);

    exit (0);
}

Increase the priority of ksoftirqd.

-Mike

Nope, PEBKAC.

Timer signals are delivered via ksoftirqd, so your cpu hog prevents ksoftirqd frum running.

Thanks,

tglx

Does ksoftirqd deliver timer signals in vanilla?

Thanks
Mark

Sometimes :-)

Softirqs in vanilla run in interrupt context. Mostly on return from an hardware interrupt handler, the softirqs are called and run anytime interrupts are enabled (and bottom halves enabled). But if there's too much work to do from the soft interrupt, then it pushes the rest of the work to ksoftirqd. Which on vanilla runs as a normal SCHED_OTHER task.

In -rt, the hardware handlers are run as thread, and all softirqs are handled by ksoftirqd. -rt does not process any softirqs in interrupt context.

The answer to your question is, the timerd softirq runs mostly from interrupt context, which will preempt your high priority CPU hog, but if too much work is being done, it may defer it to ksoftirqd, where your thread might block it. But chances are it wont.

-- Steve