System lockup.
I have a terminal server that's supporting up to 240 lines. It's a 2.4.17 kernel, and is running squid, and using the reiser file system to store log files, squid cache and other data. About every day or so, the machine locks up. The screen is blank, keyboard doesn't respond, the serial console I set up shows no 'dying gasp' and there is nothing in any of the system logs.
This doesn't appear to be related to load as it has happened both during the busiest times and during the low times.
I'm still servicing interrupts from our serial devices (on IRQ 11), so it seems interrupts are still happening.
Beyond this, however, I have no idea where to go from here. If anyone has any hints on what the problem might be, or even a way to gather more information, I would be grateful.
Signature · 4 lines
Bill Leckey - Senior Software Design Engineer TPG Research and Development Ph: +61 2 62851711 Fax: +61 2 62853939
Quoted 10 lines · Bill Leckey
I have a terminal server that's supporting up to 240 lines. It's a 2.4.17 kernel, and is running squid, and using the reiser file system to store log files, squid cache and other data. About every day or so, the machine locks up. The screen is blank, keyboard doesn't respond, the serial console I set up shows no 'dying gasp' and there is nothing in any of the system logs.
This doesn't appear to be related to load as it has happened both during the busiest times and during the low times.
Quoted 6 lines
I'm still servicing interrupts from our serial devices (on IRQ 11), so it seems interrupts are still happening.
Beyond this, however, I have no idea where to go from here. If anyone has any hints on what the problem might be, or even a way to gather more information, I would be grateful.
Hardware details would be a useful starting point. Also if its uniprocessor or SMP. Finally have you considered 2.4.19 as 2.4.17 does have at least one known and fixed small PPP race. With 240 lines I guess you might actually hit that
Quoted 8 lines · Alan Cox
Quoted 6 lines · Bill Leckey
I have a terminal server that's supporting up to 240 lines. It's a 2.4.17 kernel, and is running squid, and using the reiser file system to store log files, squid cache and other data. About every day or so, the machine locks up. The screen is blank, keyboard doesn't respond, the serial console I set up shows no 'dying gasp' and there is nothing in any of the system logs.
This doesn't appear to be related to load as it has happened both during the busiest times and during the low times.
I'm still servicing interrupts from our serial devices (on IRQ 11), so it seems interrupts are still happening.
Quoted 7 lines
Quoted 1 line
Beyond this, however, I have no idea where to go from here. If anyone has any hints on what the problem might be, or even a way to gather more information, I would be grateful.
Hardware details would be a useful starting point. Also if its uniprocessor or SMP. Finally have you considered 2.4.19 as 2.4.17 does have at least one known and fixed small PPP race. With 240 lines I guess you might actually hit that
Thanks for the reply Alan, much appreciated.
This system is running on Uniprocessor Intel P III's or Celerons (a variety of clock speeds) with either 256 or 512Mb of memory and no swap (another experiment). The Serial Hardware is a proprietary card (with a driver to cope with that, mostly copied from the standard serial driver) giving us the 240 serial lines. I can give more detail on that and the driver if necessary.
Just offhand I have been through my driver (with others making comments/suggestions) a few times to see if it's all my fault (which it may well still be). The reason I know interrupts are still running is that on every interrupt I stick a different value onto the Parallel port and can see those changing as I would expect even when the system is locked up otherwise.
I am considering 2.4.19 but haven't had a chance to test with the driver/hardware yet. That will be soon as it seems a good path to go down.
One thing I forgot was that there was the same failure with a system running without the squid. It seems though, that a system running without squid will fail less often (two or more weeks between failures as opposed to a few days).
I hope I'm giving enough detail to be useful here.
Bill
Signature · 4 lines
Bill Leckey - Senior Software Design Engineer TPG Research and Development Ph: +61 2 62851711 Fax: +61 2 62853939
Just FYI, I've been running 2.4.19 on three systems for about 3 days now. Where before I would expect each system to hang at least once a day, I've had no hangs whatsoever.
I'm not sure if it was the PPP race or something else but 2.4.19 seems (at least so far) much more stable.
Thanks for the advice Alan.
Bill
Quoted 16 lines · Alan Cox
Quoted 7 lines · Bill Leckey
I have a terminal server that's supporting up to 240 lines. It's a 2.4.17 kernel, and is running squid, and using the reiser file system to store log files, squid cache and other data. About every day or so, the machine locks up. The screen is blank, keyboard doesn't respond, the serial console I set up shows no 'dying gasp' and there is nothing in any of the system logs.
This doesn't appear to be related to load as it has happened both during the busiest times and during the low times.
I'm still servicing interrupts from our serial devices (on IRQ 11), so it seems interrupts are still happening.
Beyond this, however, I have no idea where to go from here. If anyone has any hints on what the problem might be, or even a way to gather more information, I would be grateful.
Hardware details would be a useful starting point. Also if its uniprocessor or SMP. Finally have you considered 2.4.19 as 2.4.17 does have at least one known and fixed small PPP race. With 240 lines I guess you might actually hit that