Skip to contentSkip to conversation

anybody else OK with current CVS *and* SMP?

DISCUSSION Paul Davis 11 participants 27 messages Aug 5, 2003 – Aug 11, 2003

lore
Share & tools
lore

Paul Davis

author
#0 lore raw

i am having a repeated problem with system lockups using current CVS with JACK running SCHED_FIFO on an SMP system, with the Planet's 2.4.21 kernel.

can anyone report that a similar configuration is working for them? there is no way to diagnose the situation, because the kernel hangs totally and completely, and its not possible to get the output from either an oops or the magic-request key(s).

it seems to be less of a problem when JACK is running without SCHED_FIFO.

--p

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#1 lore raw

This seems like the problem I've always had with my single processor with JACK and SCHED_FIFO, with 2.4.20 kernel. I haven't tried the realtime setting for a while, though.

Maybe there's some other thing causing these? Should we gather a bit more in-depth information on our computers?

-a

Quoted 9 lines · Paul Davis

i am having a repeated problem with system lockups using current CVS with JACK running SCHED_FIFO on an SMP system, with the Planet's 2.4.21 kernel.

can anyone report that a similar configuration is working for them? there is no way to diagnose the situation, because the kernel hangs totally and completely, and its not possible to get the output from either an oops or the magic-request key(s).

Quoted 4 lines

it seems to be less of a problem when JACK is running without SCHED_FIFO.

--p

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#2 lore raw

At Tue, 05 Aug 2003 23:37:01 +0300,

Quoted 4 lines · Antti Boman

This seems like the problem I've always had with my single processor with JACK and SCHED_FIFO, with 2.4.20 kernel. I haven't tried the realtime setting for a while, though.

if the exactly same problem happens on UP, the cause would be different from what i thought originally. my guess was spin dead locks or other SMP-specific issues, possibly around the linked pcm streams.

Quoted 2 lines

Maybe there's some other thing causing these? Should we gather a bit more in-depth information on our computers?

yes, this will help.
at least, we need to investigate with the same driver/lib version (preferably with the same kernel and gcc).

btw, the dead-lock can be related to the lowlevel part, too. recently i found a bug in es1968 driver which can cause a dead-lock with linked streams and xrun (typically as jack does). but in this case, it can happen regardless of SCHED_FIFO...

Takashi

Quoted 20 lines

-a

Quoted 9 lines · Paul Davis

i am having a repeated problem with system lockups using current CVS with JACK running SCHED_FIFO on an SMP system, with the Planet's 2.4.21 kernel.

can anyone report that a similar configuration is working for them? there is no way to diagnose the situation, because the kernel hangs totally and completely, and its not possible to get the output from either an oops or the magic-request key(s).

Quoted 4 lines

it seems to be less of a problem when JACK is running without SCHED_FIFO.

--p

Quoted 10 lines

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
_______________________________________________
Alsa-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/alsa-devel

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#3 lore raw
Quoted 9 lines · Takashi Iwai

At Tue, 05 Aug 2003 23:37:01 +0300,

Quoted 1 line · Antti Boman

This seems like the problem I've always had with my single processor with JACK and SCHED_FIFO, with 2.4.20 kernel. I haven't tried the realtime setting for a while, though.

if the exactly same problem happens on UP, the cause would be different from what i thought originally. my guess was spin dead locks or other SMP-specific issues, possibly around the linked pcm streams.

It sounded as if it's the same problem, but naturally I don't know if it's caused by the same thing. I don't remember if it happened without X running, for example, so I have to test it without X, too. The frustrating part is that I really cannot debug anything, as the system completely locks up.

Quoted 5 lines
Quoted 1 line

Maybe there's some other thing causing these? Should we gather a bit more in-depth information on our computers?

yes, this will help.
at least, we need to investigate with the same driver/lib version (preferably with the same kernel and gcc).

Ok. I'll gather the information, do a few more tests, and post it on the list. I can also upgrade/downgrade the libs and programs if you think it's worth the while.

Quoted 4 lines

btw, the dead-lock can be related to the lowlevel part, too. recently i found a bug in es1968 driver which can cause a dead-lock with linked streams and xrun (typically as jack does). but in this case, it can happen regardless of SCHED_FIFO...

If you remember, I used to have huge problems with CMI8738 based integrated sound... "card", but they went away when installing newer ALSA drivers and libraries. Except from this SCHED_FIFO, of course.

Anyway, I'll first let you know about my current configuration, and you can tell me what to try.

Thanks,
-a

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#4 lore raw
In reply to Antti Boman
Quoted 6 lines · On 05-Aug-2003 Antti Boman wrote:

This seems like the problem I've always had with my single processor with JACK and SCHED_FIFO, with 2.4.20 kernel. I haven't tried the realtime setting for a while, though.

Maybe there's some other thing causing these? Should we gather a bit more in-depth information on our computers?

I'm also having random Oopses and crashes. I use alsa0.9.6-stable, linux 2.4.22p6-smp on a dual G4. I don't use Jack, it happens just playing music with xmms, but it's rare enough I'm not sure it's caused by ALSA or anything else. Yesterday I left my box in a endless kernel compile cycle for many hours without ALSA running and I had no problems. It also might be an hardware problem... I don't know. I never had problems with nearly the same config on a single processor G3.

Bye.

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#5 lore raw
In reply to Paul Davis
Quoted 5 lines · Paul Davis

i am having a repeated problem with system lockups using current CVS with JACK running SCHED_FIFO on an SMP system, with the Planet's 2.4.21 kernel.

can anyone report that a similar configuration is working for them?

I'm using alsa-0.9.6, slightly patched but otherwise vanilla linux-2.4.20 SMP, jack-0.74.1. Starting jack with

jackd -R -a --tmpdir /mnt/ramfs -d alsa -d terratec -n 3 -p 128

with an EWS88MT (ice1712) and it's fine. I'm fetching latest CVS now, will let you know.

HTH
John

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#6 lore raw
Quoted 7 lines · John Anderson
Quoted 5 lines · Paul Davis

i am having a repeated problem with system lockups using current CVS with JACK running SCHED_FIFO on an SMP system, with the Planet's 2.4.21 kernel.

can anyone report that a similar configuration is working for them?

Quoted 7 lines

I'm using alsa-0.9.6, slightly patched but otherwise vanilla linux-2.4.20 SMP, jack-0.74.1. Starting jack with

jackd -R -a --tmpdir /mnt/ramfs -d alsa -d terratec -n 3 -p 128

with an EWS88MT (ice1712) and it's fine. I'm fetching latest CVS now, will let you know.

Nope. Latest CVS (1/2 an hour ago) compile fails with

rawmidi_virt.c: In function `snd_rawmidi_virtual_open':
rawmidi_virt.c:342: `SND_RAWMIDI_TYPE_VIRTUAL' undeclared (first use in this function)

Maybe this is a SourceForge CVS problem?

bye
John

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#7 lore raw

At 06 Aug 2003 16:39:12 +0200,

Quoted 17 lines · John Anderson
Quoted 7 lines · John Anderson
Quoted 5 lines · Paul Davis

i am having a repeated problem with system lockups using current CVS with JACK running SCHED_FIFO on an SMP system, with the Planet's 2.4.21 kernel.

can anyone report that a similar configuration is working for them?

Quoted 7 lines

I'm using alsa-0.9.6, slightly patched but otherwise vanilla linux-2.4.20 SMP, jack-0.74.1. Starting jack with

jackd -R -a --tmpdir /mnt/ramfs -d alsa -d terratec -n 3 -p 128

with an EWS88MT (ice1712) and it's fine. I'm fetching latest CVS now, will let you know.

Quoted 6 lines

Nope. Latest CVS (1/2 an hour ago) compile fails with

rawmidi_virt.c: In function `snd_rawmidi_virtual_open':
rawmidi_virt.c:342: `SND_RAWMIDI_TYPE_VIRTUAL' undeclared (first use in this function)

Maybe this is a SourceForge CVS problem?

sorry, my fault. now it's committed to cvs tree. hope this is mirrored to anon cvs tree soon later...

the fix is just the change from SND_RAWMIDI_TYPE_VIRT to SND_RAWMIDI_TYPE_VIRTUAL in include/rawmidi.h, BTW.

Takashi

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#8 lore raw
Quoted 8 lines · Takashi Iwai
Quoted 6 lines
Quoted 6 lines · Paul Davis
Quoted 5 lines

i am having a repeated problem with system lockups using current CVS with JACK running SCHED_FIFO on an SMP system, with the Planet's 2.4.21 kernel.

can anyone report that a similar configuration is working for them?

the fix is just the change from SND_RAWMIDI_TYPE_VIRT to SND_RAWMIDI_TYPE_VIRTUAL in include/rawmidi.h, BTW.

OK, compiled, installed etc. But no lockups with a 2.4.20 kernel. Is there anything in particular that triggers it?

bye
John

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

author
#9 lore raw
Quoted 2 lines

OK, compiled, installed etc. But no lockups with a 2.4.20 kernel. Is there anything in particular that triggers it?

i don't know. i just know that both my system and that of my first paying customer lock up when JACK is run SCHED_FIFO, and it normally happens within 10 minutes.

i'll try to make sure to try it with my trident card as well.

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#10 lore raw

At Wed, 06 Aug 2003 11:13:48 -0400,

Quoted 7 lines · Paul Davis
Quoted 3 lines

OK, compiled, installed etc. But no lockups with a 2.4.20 kernel. Is there anything in particular that triggers it?

i don't know. i just know that both my system and that of my first paying customer lock up when JACK is run SCHED_FIFO, and it normally happens within 10 minutes.

with which sound card did it happen? rme9652?

Takashi

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

author
#11 lore raw
Quoted 1 line

with which sound card did it happen? rme9652?

yep, three different ones. and it didn't use to happen using an older version of ALSA CVS. i haven't narrowed the dates - the older version is the pre-linked-streams-spinlock stuff, so its not very helpful.

as i said, i need to try it on my trident card too.

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#12 lore raw
In reply to Paul Davis
Quoted 5 lines · Paul Davis
Quoted 2 lines

OK, compiled, installed etc. But no lockups with a 2.4.20 kernel. Is there anything in particular that triggers it?

i don't know. i just know that both my system and that of my first paying customer

ouch :-(

Quoted 4 lines

lock up when JACK is run SCHED_FIFO, and it
normally happens within 10 minutes.

i'll try to make sure to try it with my trident card as well.

Been listening to a CD here using alsaplayer and jack for about 30 minutes now. Alsaplayer crashed.

bye
John

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#13 lore raw
In reply to Paul Davis
Quoted 3 lines · Paul Davis

i don't know. i just know that both my system and that of my first paying customer lock up when JACK is run SCHED_FIFO, and it normally happens within 10 minutes.

Ok, there's a difference compared to my problem. My computer locks up within a couple of seconds. So, obviously we're not talking about the same problem. Also, I have to install newer ALSA and JACK to see if the lock-up is still there.

-a

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#14 lore raw
Quoted 7 lines · --- Antti Boman

Ok, there's a difference compared to my problem. My computer locks up
within a couple of seconds. So, obviously we're not talking about the
same problem. Also, I have to install newer ALSA and JACK to see if the
lock-up is still there.

That's interesting. I use CVS ALSA and SMP (although not JACK), and I have been having lock-ups too ever since last Sunday evening (3rd August). However, at least some of my lock-ups must be hardware related because I've had a horrible time fscking my root partition afterwards. (But THAT could be caused because I can't load my CPUs' microcode until the root partition is mounted OK...).

I shall try removing ALSA and seeing if that improves thing. It would be just my luck to have multiple lock-up problems all at once!

Chris

________________________________________________________________________
Want to chat instantly with your online friends? Get the FREE Yahoo! Messenger http://uk.messenger.yahoo.com/

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#15 lore raw
In reply to Paul Davis
Quoted 6 lines
Quoted 2 lines

OK, compiled, installed etc. But no lockups with a 2.4.20 kernel. Is there anything in particular that triggers it?

i don't know. i just know that both my system and that of my first paying customer lock up when JACK is run SCHED_FIFO, and it normally happens within 10 minutes.

There was a small program posted by Roger Larsson that monitored processes running with SCHED_FIFO and if one of them hogged cpu for more than a number of seconds it would get downgraded to SCHED_OTHER. If the bug is a lockup due to SCHED_FIFO this may revive the machine after it hangs (it did that for me when I was poking around trying to find the cause of lockups that latter were solved by Andrew Morton).

I'm including the source in attachments.
-- Fernando

CFLAGS=-g

all: rt rt_monitor

rt: rt.c

rt_monitor: rt_monitor.c

/* RT user.

Copyright (c) 2002 Roger Larsson <[email protected]>

This program is free software; you can redistribute it and/or modify it under the terms of version 2 of the GNU General Public License as published by the Free Software Foundation.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

Thanks to autor of KSysGuard Chris Schlaeger for borrowed code...
*/

#include <sys/types.h>
#include <sched.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <stdlib.h>

struct request

{
	pid_t pid;
	char _filler[32];
};

int isRT(pid_t pid)

 {
     int sched_class = sched_getscheduler( pid);
     if (sched_class == -1) {
	 fprintf(stderr, "Pid %d Exited?\n", pid);
	 return 0;
     }

     return sched_class != SCHED_OTHER;

}

int main(int argc, char *argv[])

{
	struct request request;
	FILE *reqf;
	int done=0, loops = 0;

	request.pid = getpid();
	while (!done) {
	    switch (getopt(argc, argv, "?c:p:q:" )) {
		case 'c':
		    loops = atoi(optarg);
		    break;
		case 'p':
		    request.pid = atoi(optarg);
		    printf("pid %d\n", request.pid);
		    break;
		case 'q':
		    request.pid = atoi(optarg);
		    printf("pid %d - %s\n", request.pid, isRT(request.pid) ? "is RT" : "is NOT RT");
		    return;
		case '?':
		    printf("%s: [-c|-p pid]\n", argv[0]);
		    printf("\t-c loops\tcheck monitor function by looping\n");
		    printf("\t   Note: I need 9000000 at 933MHz, start low and move up\n");
		    printf("\t-p pid\trequest on behalf of other process\n");
		    printf("\t-q pid\tquery if process is RT\n");
		    return 1;
		case -1:
		    // No more options
		    done = 1;
		    break;
	    }
	}

	printf("As long as no monitor runs, execution will sleep here...\n");
	reqf = fopen("/var/named/rt-request", "w");
	if (reqf == NULL) {
	    perror("fopen");
	    return errno;
	}

	printf("policy %d\n", sched_getscheduler(request.pid));
	printf("Wait...\n"); // Has to be here to get some chance to display...

	fwrite(&request, 32, 1, reqf);
	fclose(reqf); // important! (maybe flush?)

	// Wait until RT monitor raised prio
	while (sched_getscheduler(request.pid) == 0) {
	}
	printf("policy %d\n", sched_getscheduler(request.pid));

	// well behaved
	if (request.pid == getpid() && loops > 0) {

	    printf("\nsleep for 3 seconds then start with a\n");
	    printf("busy wait for %d loops (or until prio reduced)\n", loops);
	    printf(" move your mouse!\n");
	    sleep(3);

	    while (--loops > 0 && sched_getscheduler(request.pid) != 0) {
		// someone did listen to my request...
		// assume monitor is running
	    }

	    if (loops == 0)
		printf(" - normal loop finish, to short loop?\n");
	    else
		printf(" - monitor works! (priority got reduced)\n");
	}

	return 0;
}

/* RT monitor.

Copyright (c) 2002 Roger Larsson <[email protected]>

This program is free software; you can redistribute it and/or modify it under the terms of version 2 of the GNU General Public License as published by the Free Software Foundation.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

Thanks to autor of KSysGuard Chris Schlaeger for borrowed code...
*/

#include <sys/types.h>
#include <sched.h>
#include <stdio.h>
#include <dirent.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <ctype.h>
#include <sys/time.h>
#include <stdlib.h>
#include <signal.h>
#include <sys/resource.h>

int set_normal_priority(pid_t pid);

int isRT(pid_t pid)

 {
     int sched_class = sched_getscheduler( pid);
     if (sched_class == -1) {
	 fprintf(stderr, "Pid %d Exited?\n", pid);
	 return 0;
     }

     return sched_class != SCHED_OTHER;

}

struct rt_process_info

{
    /* This flag is set for all found processes at the beginning of the
     * process list update. Processes that do not have this flag set will
     * be assumed dead and removed from the list. The flag is cleared after
     * each list update. */
    int alive;
    int centStamp;


    pid_t pid;
    pid_t ppid;
    gid_t gid;

    unsigned int userTime;
    unsigned int sysTime;
    unsigned int vmSize; // enough?
    unsigned int vmRss; // enough?

    float sysLoad;
    float userLoad;
    float cpu_usage;
};

#define MAX_RT_PROCESSES 200
struct rt_process_info rt_process[MAX_RT_PROCESSES]; /* pid & alive == 0 */

struct rt_process_info *find_process(pid_t pid)

{
    unsigned ix;
    for (ix = 0; ix < MAX_RT_PROCESSES; ix++)
    {
	if (rt_process[ix].pid == pid) {
	    rt_process[ix].alive = 1;
	    return &rt_process[ix];
	}
    }

    return NULL;
}

struct rt_process_info *new_process(pid_t pid)

{
    unsigned ix;
    for (ix = 0; ix < MAX_RT_PROCESSES; ix++)
    {
	if (rt_process[ix].pid == 0) {
	    rt_process[ix].pid = pid;
	    rt_process[ix].alive = 2;
	    return &rt_process[ix];
	}
    }

    return NULL;
}

float cpu_usage(struct rt_process_info *ps)
{

#define BUFSIZE 1024
    char buf[BUFSIZE];
    FILE *fd;
    char status;
    unsigned int userTime, sysTime;

    snprintf(buf, BUFSIZE - 1, "/proc/%d/stat", ps->pid);
    buf[BUFSIZE - 1] = '\0';
    if ((fd = fopen(buf, "r")) == 0)
	return (-1);

    if (fscanf(fd, "%*d %*s %c %d %d %*d %*d %*d %*u %*u %*u %*u %*u %d %d"
	       "%*d %*d %*d %*d %*u %*u %*d %u %u",
	       &status, (int*) &ps->ppid, (int*) &ps->gid,
	       &userTime, &sysTime, &ps->vmSize,
	       &ps->vmRss) != 7) {
      perror("fscanf");
	fclose(fd);
	return (-1);
    }

    if (fclose(fd))
	return (-1);

    {
	unsigned int newCentStamp;
	int timeDiff, userDiff, sysDiff;
	struct timeval tv;

	gettimeofday(&tv, 0);
	newCentStamp = tv.tv_sec * 100 + tv.tv_usec / 10000;

	// calculate load
	if (ps->alive == 2)
	    ps->sysLoad = ps->userLoad = 0.0f; /* can't give relieable number at the moment... */
	else {
	    timeDiff = (int)(newCentStamp - ps->centStamp);
	    userDiff = userTime - ps->userTime;
	    sysDiff = sysTime - ps->sysTime;

			
	    if ((timeDiff > 0) && (userDiff >= 0) && (sysDiff >= 0)) /* protect from bad data */
	    {
		ps->userLoad = ((double) userDiff / timeDiff) * 100.0;
		ps->sysLoad = ((double) sysDiff / timeDiff) * 100.0;
	    }
	    else
		ps->sysLoad = ps->userLoad = 0.0;
	}

	// update fields
	ps->centStamp = newCentStamp;
	ps->userTime = userTime;
	ps->sysTime = sysTime;
    }
	
    ps->cpu_usage = ps->userLoad + ps->sysLoad;

    return ps->cpu_usage;
}

float process_cpu_usage(pid_t pid)

{
    struct rt_process_info *process = find_process(pid);
    float cpu_use;

    if (process == NULL)
    {
      // New process, but not RT - uninteresting!
        if (!isRT(pid))
	  return 0.0f;

	process = new_process(pid);
	if (process == NULL) {
	    // to many RT (or previous) processes!
	    //  this process is new - assume a DOS attack
	    printf("DOS attack? kill process %d\n", pid);
	    if (!kill(pid, SIGKILL))
	      perror("kill");
	    return 0.0f;
	}

						
	process->alive = 2; /* mark process new */
    }

    // Account current RT processes AND old ones!

    cpu_use = cpu_usage(process);

    process->alive = 1;

    return cpu_use;
}

/* process reading code from ksysguard */
float cpu_rt_usage(struct rt_process_info **rt_list_head)

{
    // Watch out for SMP effects...
	
    float result = 0.0f;
    pid_t myself = getpid();
    DIR* dir;
    struct dirent* entry;

    /* read in current process list via the /proc filesystem entry */
    if ((dir = opendir("/proc")) == NULL)
    {
	perror("Cannot open directory \'/proc\'!\n"
	       "The kernel needs to be compiled with support\n"
	       "for /proc filesystem enabled!\n");
	return 0;
    }
	
    // for all processes
    while ((entry = readdir(dir)))
    {
	if (isdigit(entry->d_name[0]))
	{
	    pid_t pid;

	    pid = atoi(entry->d_name);
			
	    if (pid != myself) {
		result += process_cpu_usage(pid);
	    }
	}
    }
    closedir(dir);
	
    return result;
}

void gc_rt_processes()

{
    unsigned ix;
    printf("C:");
    for (ix = 0; ix < MAX_RT_PROCESSES; ix++)
    {
	struct rt_process_info *rt_examine = &rt_process[ix];

	if (rt_examine->alive)
	{
	    rt_examine->alive = 0;
	}
	else if (rt_examine->pid)
	{
	  printf("%d ", rt_examine->pid);
	    rt_examine->pid = 0; /* delete it! */
	}
    }
}

int set_me_realtime(void)
{

struct sched_param schp;
	/*
	 * set the process to realtime privs
	 */
        memset(&schp, 0, sizeof(schp));
	schp.sched_priority = sched_get_priority_max(SCHED_FIFO);

	if (sched_setscheduler(0, SCHED_FIFO, &schp) != 0) {
		perror("sched_setscheduler");
		return -1;
	}

	if(mlockall(MCL_CURRENT|MCL_FUTURE))
	{
	    perror("mlockall() failed, exiting. mlock");
	    return -1;
	}

	return 0;

}

int set_realtime_priority(pid_t pid)

{
	struct sched_param schp;
	/*
	 * set the process to realtime privs
	 */

	printf("Attempt to set realtime for pid %d ", pid);

	if (pid == 0 || pid == getpid()) {
	    printf("- ignored! (that is me)\n");
	    return -1;
	}


        memset(&schp, 0, sizeof(schp));
	schp.sched_priority = sched_get_priority_min(SCHED_FIFO);

	if (sched_setscheduler(pid, SCHED_FIFO, &schp) != 0) {
		printf("- failed!\n");
		perror("sched_setscheduler");
		return -1;
	}
	printf("- done!\n");

	(void)process_cpu_usage(pid); // Add to RT process array ASAP

	return 0;

}

int set_normal_priority(pid_t pid)
{
struct sched_param schp;

	if (!isRT(pid))
	  return 0;

	/*
	 * set the process to normal privs, most nice
	 */
        memset(&schp, 0, sizeof(schp));
	schp.sched_priority = 0;

	printf("%d ", pid);

	if (pid == 0 || pid == getpid()) {
	    printf("- ignored! (that is me)\n");
	    return -1;
	}

	if (sched_setscheduler(pid, SCHED_OTHER, &schp) != 0) {
	  	printf("- failed!\n");
		perror("sched_setscheduler");
		return -1;
	}

	if (setpriority(PRIO_PROCESS, pid, 20) != 0) {
	  	printf("- failed!\n");
		perror("setpriority");
		return -1;
	}

	return 0;
}

void set_normal_priority_all()

{
    unsigned ix;
    printf("Removing RT scheduling from processes:\n");
    for (ix = 0; ix < MAX_RT_PROCESSES; ix++)
    {
	struct rt_process_info *process_info = &rt_process[ix];
	if (process_info->pid)
	    set_normal_priority(process_info->pid);
    }
    printf("\n");
}

struct request

{
	pid_t pid;
	char	_filler[32];
};

#define REQUEST_SIZE 32

int poll_request(int reqfd)

{
    // Be VERY careful not to
    // * block here...
    // * get buffer overruns...

	static int remaining = REQUEST_SIZE;
	static struct request request;
	char *next = ((char *)&request + REQUEST_SIZE - remaining);

	int ret = read(reqfd, (void *)next, remaining);
	if (ret == -1) {
		perror("read");
		return 0;
	}
	remaining -= ret;

	if (remaining == 0) {
		remaining = REQUEST_SIZE;

		if (request.pid == 0 || request.pid == getpid()) {
			fputs("attempt to forge the monitor\n", stderr);
			return 0;
		}

		set_realtime_priority(request.pid);

		return 1;
	}

	return 0;
}

#define RTREQUEST_FILE "/var/named/rt-request"
int main(int argc, char * argv[])

{
    struct rt_process_info *rt_list = NULL;
    int reqfd = open(RTREQUEST_FILE,  O_RDONLY | O_NONBLOCK | O_NDELAY);

    if (reqfd == -1) {
	perror("open " RTREQUEST_FILE);
	fputs("have you created it? use 'mkfifo -m 622 " RTREQUEST_FILE "'\n", stderr);
	exit(1);
    }
      
    // monitor process runs with realtime prio
    set_me_realtime();

#define MIN_IDLE 10
#define MAX_RT_USAGE 70

    while (1) {
        gc_rt_processes();
	poll_request(reqfd);

		
	if (cpu_rt_usage(&rt_list) > MAX_RT_USAGE) {
	    printf("Total CPU RT usage above MAX_RT_USAGE\n");

	    // build process trees from rt_list
	    // decide which tree to reduce to normal prio class
	    //   (assume only one for simplicitly...)
	    
	    // reduce all processes in that tree
	    set_normal_priority_all();

	    //   (may use nice to simulate prio levels)
	    // log a message

	}
	//	sleep(2);
	sleep(4);
    }
    
    // process exiting - free elements on rt_list...
    //free_rt_list(rt_list);
}
author
#16 lore raw
Quoted 6 lines

There was a small program posted by Roger Larsson that monitored processes running with SCHED_FIFO and if one of them hogged cpu for more than a number of seconds it would get downgraded to SCHED_OTHER. If the bug is a lockup due to SCHED_FIFO this may revive the machine after it hangs (it did that for me when I was poking around trying to find the cause of lockups that latter were solved by Andrew Morton).

true, but JACK almost does this by itself. when run with -R, a highpri SCHED_FIFO task runs every 5 (?) seconds, and requires that the engine has checked in since the last time. this prevents loops within JACK from stalling the entire system.

anyway, the fact that ctrl-alt-sysrq doesn't work either is a good indication of a deeper problem than a runaway SCHED_FIFO task. i think.

--p

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#17 lore raw
Quoted 11 lines
Quoted 6 lines

There was a small program posted by Roger Larsson that monitored processes running with SCHED_FIFO and if one of them hogged cpu for more than a number of seconds it would get downgraded to SCHED_OTHER. If the bug is a lockup due to SCHED_FIFO this may revive the machine after it hangs (it did that for me when I was poking around trying to find the cause of lockups that latter were solved by Andrew Morton).

true, but JACK almost does this by itself. when run with -R, a highpri SCHED_FIFO task runs every 5 (?) seconds, and requires that the engine has checked in since the last time. this prevents loops within JACK from stalling the entire system.

"should prevent" :-), in my case the machine would just lock. Whatever was making the timer happy was still happening. I think that when the extra program brought down the priority of jack then the watchdog timer would kick in, but a little too late.

Quoted 2 lines

anyway, the fact that ctrl-alt-sysrq doesn't work either is a good indication of a deeper problem than a runaway SCHED_FIFO task. i think.

Yup, that sounds much worse... althugh I seem to remember some similar issue with my lockups as well (those were caused by the ext3 bug), if you did not break out to a debugger or rebooted with sysrq soon, eventually the machine would not respond to anything and the only way out was to power off.

I'm building current alsa cvs and will try to find the time to test this on an smp machine.

-- Fernando

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#18 lore raw
In reply to Paul Davis
Quoted 7 lines · Paul Davis

true, but JACK almost does this by itself. when run with -R, a highpri SCHED_FIFO task runs every 5 (?) seconds, and requires that the engine has checked in since the last time. this prevents loops within JACK from stalling the entire system.

anyway, the fact that ctrl-alt-sysrq doesn't work either is a good indication of a deeper problem than a runaway SCHED_FIFO task. i think.

I've had little luck with ctrl-alt-sysrq and usb keyboards.

- Steve

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#19 lore raw
In reply to Paul Davis
Quoted 6 lines

i am having a repeated problem with system lockups using current CVS with JACK running SCHED_FIFO on an SMP system, with the Planet's 2.4.21 kernel.

Quoted 9 lines

can anyone report that a similar configuration is working for them? there is no way to diagnose the situation, because the kernel hangs totally and completely, and its not possible to get the output from either an oops or the magic-request key(s).

it seems to be less of a problem when JACK is running without SCHED_FIFO.

A couple of month agoo I've tried to upgrade the ALSA CVS, and I remember that I got almost exactly the same sympton that you are describing.

Sometimes when running JACK without X, the SysRQ key works, however
I did not get any useful information from it, I did not get any interactive response from the developer list, and I've been very lasy ....

Since that I'm using a very old ALSA CVS version from May 17, that seems to work fine with SMP.

/Benny

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#20 lore raw
In reply to Paul Davis
Quoted 3 lines · Paul Davis

i am having a repeated problem with system lockups using current CVS with JACK running SCHED_FIFO on an SMP system, with the Planet's 2.4.21 kernel.

No lockups so far with the following setup:

2.4.19-ll
ALSA CVS of today
JACK CVS of today
snd-ens1371/snd-ice1712/snd-dummy low-level drivers (tried all three) jackd run SCHED_FIFO with -n 3 -p 256
two or more JACK clients active

Do you get lockups if you use snd-dummy as the lowlevel driver? How about without ardour (just jackd and some example clients running)?

Signature · 10 lines
 http://www.eca.cx
 Audio software for Linux!



-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
#21 lore raw

Kai, did you make a multi out of your cards together?

_J

Quoted 15 lines · Kai Vehmanen
Quoted 3 lines · Paul Davis

i am having a repeated problem with system lockups using current CVS with JACK running SCHED_FIFO on an SMP system, with the Planet's 2.4.21 kernel.

No lockups so far with the following setup:

2.4.19-ll
ALSA CVS of today
JACK CVS of today
snd-ens1371/snd-ice1712/snd-dummy low-level drivers (tried all three) jackd run SCHED_FIFO with -n 3 -p 256
two or more JACK clients active

Quoted 19 lines

Do you get lockups if you use snd-dummy as the lowlevel driver? How about without ardour (just jackd and some example clients running)?

Signature · 14 lines
 http://www.eca.cx
 Audio software for Linux!



-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
_______________________________________________
Alsa-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/alsa-devel

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#22 lore raw
Quoted 1 line · Jeremy Hall

Kai, did you make a multi out of your cards together?

Nope, jackd accessed the individual soundcards directly (... "-d alsa -d hw:{0,1,2}").

Quoted 12 lines
Quoted 11 lines · Kai Vehmanen

No lockups so far with the following setup:

2.4.19-ll
ALSA CVS of today
JACK CVS of today
snd-ens1371/snd-ice1712/snd-dummy low-level drivers (tried all three) jackd run SCHED_FIFO with -n 3 -p 256
two or more JACK clients active

Do you get lockups if you use snd-dummy as the lowlevel driver? How about without ardour (just jackd and some example clients running)?

Signature · 10 lines
 http://www.eca.cx
 Audio software for Linux!



-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
#23 lore raw

WICKED!

How does THAT work? when was that feature added? how does it maintain sample-synch? Is it better to use a pcm_multi or this option?

I suspect people are finding locking issues in the multi code, but not sure. I'm doing ok with it in 2.5.75-mm1, although I'm having latency issues and clients get killed when you use jack_connect in rapid fire.

_J

Quoted 17 lines · Kai Vehmanen
Quoted 1 line · Jeremy Hall

Kai, did you make a multi out of your cards together?

Nope, jackd accessed the individual soundcards directly (... "-d alsa -d hw:{0,1,2}").

Quoted 10 lines
Quoted 9 lines · Kai Vehmanen

No lockups so far with the following setup:

2.4.19-ll
ALSA CVS of today
JACK CVS of today
snd-ens1371/snd-ice1712/snd-dummy low-level drivers (tried all three) jackd run SCHED_FIFO with -n 3 -p 256
two or more JACK clients active

Quoted 19 lines
Quoted 2 lines
Quoted 2 lines

Do you get lockups if you use snd-dummy as the lowlevel driver? How about without ardour (just jackd and some example clients running)?

Signature · 14 lines
 http://www.eca.cx
 Audio software for Linux!



-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
_______________________________________________
Alsa-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/alsa-devel

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

author
#24 lore raw
Quoted 4 lines

WICKED!

How does THAT work? when was that feature added? how does it maintain sample-synch? Is it better to use a pcm_multi or this option?

kai wasn't saying that he runs jack with multiple sound cards. he was just using shell syntax to indicate the possible commands.

--p

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

author
#25 lore raw
In reply to Kai Vehmanen
Quoted 2 lines

Do you get lockups if you use snd-dummy as the lowlevel driver? How about without ardour (just jackd and some example clients running)?

i'll have to try snd-dummy. and yes, it will happen with just jackd alone.

however, it will not happen unless jackd is run with SCHED_FIFO. my system has been running for several days now with jackd there the whole time. with SCHED_FIFO, i can't get more than about 10-15 minutes, or so it seemed.

the worrying thing here is that the problem may go back further in time, because until qjackctl came along, my default was to run without SCHED_FIFO, whereas with it the "default" is with SCHED_FIFO. this means that SCHED_FIFO operation may not have been tested well enough for quite a while (on my system, i mean). i do know that in further back (months), i've run jackd+ardour with SCHED_FIFO for hours without problems on this system.

--p

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

#26 lore raw

Hi Paul,

At Thu, 07 Aug 2003 09:22:57 -0400,

Quoted 6 lines · Paul Davis
Quoted 3 lines

Do you get lockups if you use snd-dummy as the lowlevel driver? How about without ardour (just jackd and some example clients running)?

i'll have to try snd-dummy. and yes, it will happen with just jackd alone.

did you check this already?

Quoted 5 lines

however, it will not happen unless jackd is run with SCHED_FIFO. my system has been running for several days now with jackd there the whole time. with SCHED_FIFO, i can't get more than about 10-15 minutes, or so it seemed.

Quoted 7 lines

the worrying thing here is that the problem may go back further in time, because until qjackctl came along, my default was to run without SCHED_FIFO, whereas with it the "default" is with SCHED_FIFO. this means that SCHED_FIFO operation may not have been tested well enough for quite a while (on my system, i mean). i do know that in further back (months), i've run jackd+ardour with SCHED_FIFO for hours without problems on this system.

i became now a bit skeptical whether this is really related with the new stream-linking codes, since this happens only with SCHED_FIFO. in theory, the dead stream-link locks can happen at the XRUN status, which may happen more often with SCHED_OTHERS.

anyway, the attached is a patch to clean up a bit the complicated spin-lock paths. this invokes always unlock -> lock instead of spin_trylock for simplicity. please give a try.

Takashi