df hangs on down nfs server mounted with hard,intr, can't kill

[I posted this to the Fedora list yesterday.]

I have a Fedora server with kernel 2.4.22-1-2163 SMP mounting a remote solaris server (hence choice of options):

rsize=32768,ro,hard,intr,tcp,nfsvers=3

When the remote is down or disconnected, a "df" hangs (as expected), but I can't kill it, even as root or with kill -9. The docs for mount indicate that the INTR option should allow for killing apps mounted with HARD. Is this a bug (glibc, 2.4 kernel, NFS, or Fedora's kernel)?

I also coded a test program that calls statvfs(2) and it hangs on the statvfs(2) call when run against a down NFS server. It too can't be interrupted or killed.

My questions are:

1) Is there a safe and reliable means to check for a down NFS server (e.g., is showmount -e <server> safe enough?)

2) Is the non-interruptable operation (even with INTR option) a bug or feature?

3) Is there a simple kernel call, /proc entry, or similar that can be used to reliably check for free/used disk space and for a down host, without hanging my application?

A showmount -e followed by a statvfs() might work, but there is the possibility of losing the host between the two calls, resulting in an application hang.

4) Is there a perl module to accomplish this?

This would be very useful for network monitoring, e.g., when the server goes down and stays down for >1 minute, generate an SNMP trap and write to a log file. It would be good if you can't put an SNMP agent on the server, but only on the client. It is also useful for writing a highly reliable client application.

As I have no control over the remote system, when it went down, I had to do a hard reboot of my Linux box to stop the hung apps. This is a Windows solution, not a Linux solution....

Note, I found this when writing some scripts for MRTG to check the disk utilization of partitions. My df's hung so I didn't even get the proper values for my local partitions. After a few days, I had LOTS of hung MRTG apps and had to reboot (this test server is down for a week or two).

Thanks

Wade Hampton


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

snip ...

Perhaps you could convert this bit of code to perl. There would surely be access to the RPC libraries from perl.

If the server is down the timeout is several seconds longer than the timeout first time the routine is called (anyone care to comment as to why?) but at least it returns.

#include <rpc/rpc.h>
#include <nfs/nfs.h>
#include <linux/nfs2.h>
#include <rpc/xdr.h>

int rpc_ping(const char *host, long seconds, long micros)

{
	CLIENT *client;
	struct timeval tout;
	enum clnt_stat stat;

	client = clnt_create(host, NFS_PROGRAM, NFS2_VERSION, "udp");
	if (client == NULL) {
		return 0;
	}

	tout.tv_sec = seconds;
	tout.tv_usec = micros;

	clnt_control(client, CLSET_TIMEOUT, (char *)&tout);
	clnt_control(client, CLSET_RETRY_TIMEOUT, (char *)&tout);

	stat = clnt_call(client, NFSPROC_NULL,
			 (xdrproc_t)xdr_void, 0, (xdrproc_t)xdr_void, 0, tout);

	clnt_destroy(client);

	if (stat != RPC_SUCCESS) {
		return 0;
	}

	return 1;
}

Ian

-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs