uninterruptible sleep lockups
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 2 lines · linux-os
You don't seem to understand. A process that's stuck in 'D' state shows a SEVERE error, usually with a hardware driver.
Or a network filesystem mount to a no longer existing server or share.
Quoted 5 lines
For instance,
somebody may have coded something in a critical section that will wait forever for some bit to be set when, in fact, that bit may never be set because of a hardware glitch. Such problems must be found. One can't just suck some process out of the 'D' state.
But you can easily fall into one, e.g. by mounting a SMB share to ~/mnt, working until after the windows box breaks down and trying to save the work of the last hour (which involves enumerating and stat()ing all entries in ~).
Quoted 5 lines
The 'D' state usually stands for 'Down' where a task was '
down()' on a semaphore. To get out of that state, that task (and none other) needs to executeup(). This means that whatever that task was waiting for needs to happen or it won't call 'up()'.
Maybe the device/mountpoint causing the processes to hang can be declared dead (This is the more important part to me) and/or the syscall can be forced to fail. If it involves wasting some MB of RAM for copying all possibly affected memory in order to avoid corrupting used RAM, that will be the price to pay for not losing your data.
How to clean up the stuck processes: (This requires a MMU)
Add an error path to each syscall (or create some generic error paths) and keep the original stack frame. On errors, you can "longjump" (not exactly, but similar) to the error path after copying the memory. The semaphore will not be taken, and the code depending on the semaphore will not be executed.
BTW: Your Reply-To: should be omited if it's equal to the From:
Quoted 7 lines · Bodo Eggert
Quoted 2 lines · linux-os
You don't seem to understand. A process that's stuck in 'D' state shows a SEVERE error, usually with a hardware driver.
Or a network filesystem mount to a no longer existing server or share.
But that's a whole different problem. That's a systemic problem of "fail-over". Network file-systems really need to interface with an intermediate virtual device that can isolate failed systems and make them look "perfect" to individual machines.
If you don't do this, then as soon as somebody trips over a wire, your database is trashed. I'm surprised that NFS, PCNFS, SMB, etc., actually work as well as everybody seems to think they do. Until the architectural problem is resolved, there are still going to be hung processes, trashed databases, etc.
Quoted 11 lines
Quoted 5 lines
For instance,
somebody may have coded something in a critical section that will wait forever for some bit to be set when, in fact, that bit may never be set because of a hardware glitch. Such problems must be found. One can't just suck some process out of the 'D' state.But you can easily fall into one, e.g. by mounting a SMB share to ~/mnt, working until after the windows box breaks down and trying to save the work of the last hour (which involves enumerating and stat()ing all entries in ~).
Yes. See above.
Quoted 6 lines
Quoted 5 lines
The 'D' state usually stands for 'Down' where a task was '
down()' on a semaphore. To get out of that state, that task (and none other) needs to executeup(). This means that whatever that task was waiting for needs to happen or it won't call 'up()'.
Quoted 6 lines
Maybe the device/mountpoint causing the processes to hang can be declared dead (This is the more important part to me) and/or the syscall can be forced to fail. If it involves wasting some MB of RAM for copying all possibly affected memory in order to avoid corrupting used RAM, that will be the price to pay for not losing your data.
That's not how it's done.
Quoted 6 lines
How to clean up the stuck processes: (This requires a MMU)
Add an error path to each syscall (or create some generic error paths) and keep the original stack frame. On errors, you can "longjump" (not exactly, but similar) to the error path after copying the memory. The semaphore will not be taken, and the code depending on the semaphore will not be executed.
Again, you are attacking the symptom. The problem could be resolved by using a local disk (or a disk file) for the immediate I/O and the I/O to the file-servers could occur whenever they are available. It's just ordinary transaction processing. Nothing new. It's just that people continue to use primitive garbage (really, usually developed by amateur hackers with no formal education) that is then specified by the likes of Microsoft and then, to be compatible, other operating systems create clones with the same kinds of unfixable bugs.
Quoted 3 lines
BTW: Your Reply-To: should be omited if it's equal to the From:
The problem with From: is this machine is not "known" to the outside world, although somebody has entries in the auth02.ns.uu.net name-server that claims to be my machine, which gets cached and cloned everywhere. Mail to this system needs to go to the Reply-To: address.
Our network "experts" here have tried to track down the bad name-server entry and they say it's not here.
All of my machine names mysteriously appear in auth02.ns.uu.net with 204.178.40.nnn IP addresses. This really screws up email because email tries to verify the sender by contacting those bogus addresses.
Cheers,
Dick Johnson
Penguin : Linux version 2.6.10 on an i686 machine (5537.79 BogoMips).
Notice : All mail here is now cached for review by Dictator Bush.
98.36% of all statistics are fiction.
Quoted 13 lines · linux-os
Quoted 6 lines · Bodo Eggert
Quoted 2 lines · linux-os
You don't seem to understand. A process that's stuck in 'D' state shows a SEVERE error, usually with a hardware driver.
Or a network filesystem mount to a no longer existing server or share.
But that's a whole different problem. That's a systemic problem of "fail-over". Network file-systems really need to interface with an intermediate virtual device that can isolate failed systems and make them look "perfect" to individual machines.
Quoted 6 lines
If you don't do this, then as soon as somebody trips over a wire, your database is trashed. I'm surprised that NFS, PCNFS, SMB, etc., actually work as well as everybody seems to think they do. Until the architectural problem is resolved, there are still going to be hung processes, trashed databases, etc.
You don't run databases over a network filesystem unless you're begging for trouble. For the other common purposes you'll usurally get a more stable behaviour, since the failure on the client won't prevent the server from properly writing the metadata or flushing the cache.
Quoted 10 lines
Quoted 6 lines
How to clean up the stuck processes: (This requires a MMU)
Add an error path to each syscall (or create some generic error paths) and keep the original stack frame. On errors, you can "longjump" (not exactly, but similar) to the error path after copying the memory. The semaphore will not be taken, and the code depending on the semaphore will not be executed.Again, you are attacking the symptom. The problem could be resolved by using a local disk (or a disk file) for the immediate I/O and the I/O to the file-servers could occur whenever they are available.
a) There are systems without local storage.
b) It won't help while stat()ing a non-cached object.
c) This would involve race conditions for e.g. two disconnected nodes on reconnect. AFAI can see, this race can be solved by:
c1) The final transaction must be delayed until it's ACKed or
NACKed. This may delay the D-State for some seconds, but not enough.
c2) The server will have to keep track of the clients and need to be told
when a user left for a trip to the south pole without unmounting. Very undesirable.
c3) Ignoring. Very, very undesireable.
c4) Requiring explicit transaction handling by the applications.
Interesting, but not in the near future.
d) This won't allow synchronous updates without falling back to classic handling.
e) The users will update some files, get a positive reply and shut down their PCs before the changes can be commited to the server. If the server will not come back or the client is not rebooted within reasonable time, this will cause silent data loss.
f) This will require reliable identification of the network server.
g) I'm not only thinking of NFS/..., allthough I used it as _the_ example. E.g. if you see your IDE drive failing, you'll want to declare it dead instead of waiting $num_of_sectors times five minutes until the kernel decides to give up.
I agree that most D-states are problems that need to be fixed instead of being worked-around, but sometimes you can't fix the problem without access to the crystal-ball-device. Therefore all devices that can block will need a manual override (with different probability), and the processes that were stuck will need a way to recover or be stuck forever.
Obvoiusly the system is healthy enough to do some important and uninterruptible work after those errors occured, so having them stuck will be OK for now. Instead, the next task might be freeing the file descriptors preventing you from unmounting your removable media or network share or allowing really-forced umount.
Signature · 2 lines
Top 100 things you don't want the sysadmin to say: 54. Uh huh......"nu -k $USER".. no problem....sure thing...