Skip to contentSkip to conversation

Audio skips when RAM is ~full

DISCUSSION Anthony DiSante 5 participants 8 messages Nov 1, 2003 – Nov 11, 2003

lore
Share & tools
lore
#0 lore raw

Hello,

I have a memory-related question regarding a Slackware system I'm running (kernel 2.4.22, compiled for i386, on a VIA Eden 600MHz processor, with 256MB RAM).

First, the background info:

This system is primarily an mp3 player. It boots directly to /bin/bash bypassing everything under /etc/rc.d, and just runs a small script that does modprobing and a few other miscellaneous things, then starts the main control script (which is Perl).

I have about 250 CDs encoded as mp3s on the 40gig drive, laid out in directories as /music/band/album/songs.mp3. I also have a text file which contains one line for the full path to each album (so ~250 lines total). This file is used by my play-random-album function, which plays a random album each time I press the tick key. When I do that, the songs from the randomly-chosen album become the current playlist, and the first song starts playing.

Now, the problem/question:

Each time I press the tick key, a new directory is read from the disk (and a song is played, and the directory contents are enqueued). This causes the free memory (as reported by free) to drop by 1%, each time I press tick. I often press it 5 or 10 or 20 times in a row until I find something I feel like listening to, and the free memory will drop by almost exactly 5 or 10 or 20 percent by the time I settle on an album. If the free memory was already very low (for example, when I do this after having already listened to an album or two), then it will often get down to 2% or 3% at this point, and stay there until I reboot.

This system has one of those awful on-board "ac'97" sound chips, which (AIUI) uses the system CPU to do much of the audio work instead of using dedicated audio chips for it. When the free memory is less than about 10%, each song will "skip" when it first starts playing. (It sounds like 1/2 or 1 or 2 seconds of audio was deleted from the song; you don't hear silence, you hear the music at seconds 1, 2, and then 4, or just 2, 3, 4, ... skipping the very start, etc. But it's always some portion of the first ~5 seconds getting skipped.)

When the system is first booted, the free mem is around 90%, and this skipping doesn't happen. As time goes on, the free mem drops, and skipping starts happening somewhere along the line; by the time it's down to ~10% free, every song skips when it starts.

The disk is a modern one using DMA. And I even set up a ramdisk to try playing the songs from there (copying the "next" track into ramdisk while the current track is playing), but it still skips at the beginning of each track when the free mem is low enough.

My understanding/analysis/proposed solution:

The kernel is buffering the contents of each directory (album) that it reads (and also, mpg321 copies each mp3 file into RAM before playing it?). I understand that the idea is to stuff as much into RAM as possible to reduce pagefile usage, and that the kernel will reclaim memory utilized by buffers if/when it needs to. But apparently that isn't happening fast enough to allow a realtime process like music-playing to work skip-free on this system with this soundcard. I think that if I could regularly forcibly dump the buffered stuff out of the RAM (dropping the used-RAM percentage down to, say, 10%, like at boot time) then this would make the skipping stop.

So... do I have a correct understanding of the problem, and a correct analysis of the kernel/mem issues that are related to it? Is it possible to clear some of the RAM; if so, would that help?

Thanks,
Anthony

PS - you can see the system at:
http://nodivisions.com/tech/systems/musicbox/

#1 lore raw

Hello !

Quoted 10 lines · Anthony DiSante

The kernel is buffering the contents of each directory (album) that it reads (and also, mpg321 copies each mp3 file into RAM before playing it?). I understand that the idea is to stuff as much into RAM as possible to reduce pagefile usage, and that the kernel will reclaim memory utilized by buffers if/when it needs to. But apparently that isn't happening fast enough to allow a realtime process like music-playing to work skip-free on this system with this soundcard. I think that if I could regularly forcibly dump the buffered stuff out of the RAM (dropping the used-RAM percentage down to, say, 10%, like at boot time) then this would make the skipping stop.

1) are you certain that none of your programs (including mpg321) leaks memory ? As I understand it, it's really the cache which fills memory.
2) you can try to preload your file into the cache just before playing it :
cp $file /dev/null ; mpg321 $file

Quoted 3 lines

So... do I have a correct understanding of the problem, and a correct analysis of the kernel/mem issues that are related to it? Is it possible to clear some of the RAM; if so, would that help?

Unless a leak happens somewhere in the kernel, sound driver, etc..., memory consumed by programs is restored when your programs exit. The only part which is not restored immediately is the cache.

BTW, there may be one other reason for your problem. Considering that you scan your disk to find random albums, I think that the system updates all directories access time after 5s, thus preventing your player from reading an uncached file fast enough. You might be interested in mounting it with the 'noatime,nodiratime' options in /etc/fstab.

You might also want to try 2.4.23pre9 which includes VM changes compared to 2.4.22, and seems quite stable to me.

Regards,
Willy

#2 lore raw
Quoted 9 lines · Willy Tarreau

Hello !

Quoted 1 line · Anthony DiSante

The kernel is buffering the contents of each directory (album) that it reads (and also, mpg321 copies each mp3 file into RAM before playing it?). I understand that the idea is to stuff as much into RAM as possible to reduce pagefile usage, and that the kernel will reclaim memory utilized by buffers if/when it needs to. But apparently that isn't happening fast enough to allow a realtime process like music-playing to work skip-free on this system with this soundcard. I think that if I could regularly forcibly dump the buffered stuff out of the RAM (dropping the used-RAM percentage down to, say, 10%, like at boot time) then this would make the skipping stop.

1) are you certain that none of your programs (including mpg321) leaks memory ? As I understand it, it's really the cache which fills memory.

No, I'm not certain that mpg321 (or any other app, though there's next-to-nothing else running) is leak-free.

Quoted 2 lines

2) you can try to preload your file into the cache just before playing it :
cp $file /dev/null ; mpg321 $file

But wouldn't "cp $file /tmp/ramdisk0; mpg321 /tmp/ramdisk0/$file" accomplish the same thing? (I've done that.)

Quoted 7 lines
Quoted 1 line

So... do I have a correct understanding of the problem, and a correct analysis of the kernel/mem issues that are related to it? Is it possible to clear some of the RAM; if so, would that help?

Unless a leak happens somewhere in the kernel, sound driver, etc..., memory consumed by programs is restored when your programs exit. The only part which is not restored immediately is the cache.

Quoted 5 lines

BTW, there may be one other reason for your problem. Considering that you scan your disk to find random albums, I think that the system updates all directories access time after 5s, thus preventing your player from reading an uncached file fast enough. You might be interested in mounting it with the 'noatime,nodiratime' options in /etc/fstab.

Actually, I don't scan the disk to find random albums; I have a text file that contains a list of every album's full path, and I pick a random line from that file. So only the selected album's directory gets scanned. And the mp3 partition is mounted read-only (I should have mentioned that before), so the atimes shouldn't be getting written as it is.

Quoted 2 lines

You might also want to try 2.4.23pre9 which includes VM changes compared to 2.4.22, and seems quite stable to me.

Maybe I will give that a try.

So I'm guessing that there isn't actually a way to manually move buffer-data out of RAM?

-Anthony
http://nodivisions.com/

#3 lore raw
Quoted 5 lines · Anthony DiSante
Quoted 2 lines

2) you can try to preload your file into the cache just before playing it :
cp $file /dev/null ; mpg321 $file

But wouldn't "cp $file /tmp/ramdisk0; mpg321 /tmp/ramdisk0/$file" accomplish the same thing? (I've done that.)

It's different because you need to reserve a fixed amount of ram for the ramdisk while I was only using the caching features of the kernel as a fast read-ahead mechanism. But I agree with you that the result should be the same.

Quoted 5 lines

Actually, I don't scan the disk to find random albums; I have a text file that contains a list of every album's full path, and I pick a random line from that file. So only the selected album's directory gets scanned. And the mp3 partition is mounted read-only (I should have mentioned that before), so the atimes shouldn't be getting written as it is.

OK, so it's not a disk IO problem at all. It's really related to the sound driver it seems. Now that you say it, I remember having noticed skips on my laptop with a via 82cxxx chip after tens of minutes playing. At first I thought it was related to other activity on the system, but it did exactly what you describe, play seconds 1, 2, then 5 without a hole between them. There may be a problem with the way the audio buffer gets allocated or freed.

Quoted 2 lines

So I'm guessing that there isn't actually a way to manually move buffer-data out of RAM?

Yes, there is. I have a quick'n'dirty program which does exactly that. Basically, you tell it how many kB you want to free, then it allocates and uses that amount of memory, frees it and exits. Buffered data gets flushed very quickly. I sometimes give it a try before starting to work on large kernel trees, because it helps the entire directories to fit in cache.

Here it is if you're interested. Don't start it without an argument, it will try to allocate 4G !

Regards,
Willy

---
#include <stdio.h>

main(int argc, char **argv) {
unsigned long int i,k=0, max;
char *p;

  if (argc>1)
      max=atol(argv[1]);
  else
      max=0xffffffff;

printf("Freeing %lu kB ram\n",max);

  while (((p=(char *)malloc(1048576))!=NULL) && (k+1024<=max)) {
    for (i=0;i<256;p[4096*i++]=0);
    k+=1024;
    fprintf(stderr,"\r%d kB allocated",k);
  }
  while (((p=(char *)malloc(65536))!=NULL) && (k+64<=max)) {
    for (i=0;i<16;p[4096*i++]=0);
    k+=64;
    fprintf(stderr,"\r%d kB allocated",k);
  }
  while (((p=(char *)malloc(1024))!=NULL) && (k+1<=max)) {
    for (i=0;i<16;p[64*i++]=0);
    k+=1;
    fprintf(stderr,"\r%d kB allocated",k);

}
fprintf(stderr,"\nMemory freed.\n");
exit(0);
}

#6 lore raw
In reply to Anthony DiSante
Quoted 2 lines

So I'm guessing that there isn't actually a way to manually move buffer-data out of RAM?

Just the naive allocate X MB, set the first byte of every 4KB (PageSize) block to 1 and then free the memory - presto X MB free RAM.

Cheers,
MaZe.

#4 lore raw
In reply to Willy Tarreau
Quoted 9 lines · Willy Tarreau
Quoted 1 line

Actually, I don't scan the disk to find random albums; I have a text file that contains a list of every album's full path, and I pick a random line from that file. So only the selected album's directory gets scanned. And the mp3 partition is mounted read-only (I should have mentioned that before), so the atimes shouldn't be getting written as it is.

OK, so it's not a disk IO problem at all. It's really related to the sound driver it seems. Now that you say it, I remember having noticed skips on my laptop with a via 82cxxx chip after tens of minutes playing. At first I thought it was related to other activity on the system, but it did exactly what you describe, play seconds 1, 2, then 5 without a hole between them. There may be a problem with the way the audio buffer gets allocated or freed.

Yeah... in my experience, the whole ac'97 deal is pretty buggy (for one thing, with both ALSA and OSS drivers, output is distorted unless I keep the volume below ~70%...). But as I said, the skipping only happens when the memory is full, or getting close to full. If I select specific albums to play instead of using my random function, I can usually play a few albums and there aren't any skips, not until the memory gets pretty full.

Quoted 10 lines
Quoted 1 line

So I'm guessing that there isn't actually a way to manually move buffer-data out of RAM?

Yes, there is. I have a quick'n'dirty program which does exactly that. Basically, you tell it how many kB you want to free, then it allocates and uses that amount of memory, frees it and exits. Buffered data gets flushed very quickly. I sometimes give it a try before starting to work on large kernel trees, because it helps the entire directories to fit in cache.

Quoted 2 lines

Here it is if you're interested. Don't start it without an argument, it will try to allocate 4G !

4G!! (Note to self: s/ffffffff/0000ffff/...) Thanks a lot, I'll give this a try. It certainly works here on my home system.

-Anthony
http://nodivisions.com/

#5 lore raw
Quoted 3 lines · Anthony DiSante

Yeah... in my experience, the whole ac'97 deal is pretty buggy (for one thing, with both ALSA and OSS drivers, output is distorted unless I keep the volume below ~70%...).

It's ok. The AC97 codecs can apply gain (not only attenuation). Thus you can overdrive your analog output and hear distortion. It's analog behaviour not a bug.

Jaroslav

-----
Jaroslav Kysela <[email protected]>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs

#7 lore raw
In reply to Anthony DiSante

Hi!

Check your sound driver's sources . Perhaps it allocates GFP_ATOMIC memory nd drops data on the floor if it is not available.

Signature · 2 lines
				Pavel
Written on sharp zaurus, because my Velo1 broke. If you have Velo you don't need...