Skip to contentSkip to conversation

weird moutstats with pnfs

DISCUSSION Mkrtchyan, Tigran 2 participants 2 messages Aug 7, 2013 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Hi,

I am running some IO tests with pNFS and see some strange(at least to me) numbers:

WRITE:

	44316 ops (249%) 	0 retrans (0%) 	0 major timeouts
	avg bytes sent per op: 179112	avg bytes received per op: 111
	backlog wait: 2416.041204 	RTT: 11.033735 	total execute time: 2427.148953 (milliseconds)

249% What does it tell me?

This is fedora19 with 3.10.3-300.fc19.x86_64 kernel.

Tigran.

#2 lore raw

I see what's happening here. This is due to stat collection happening in the rpc layer, which works great until you have multiple RPC clients associated with one mountpoint (like pNFS with filelayout).

When we added pNFS READ/WRITE counting to /proc/self/mountstats, we incremented the read/write count on the rpc_client associated with the superblock of the mount being used (aka struct nfs_server), even though the write may have actually happened on a different rpc client (MDS != DS). The 'rpcsends' (and I'm sure recvs) counter is not being updated in this case, and that's what the WRITE count is being divided by (in the userland mountstats program) to come up with the percentage.

I'll look at fixing this - I think we just want to update send & recv counters on the superblock's rpc_client stats when it's a DS READ/WRITE.

-dros

Quoted 16 lines · Tigran

Hi,

I am running some IO tests with pNFS and see some strange(at least to me) numbers:

WRITE:

	44316 ops (249%) 	0 retrans (0%) 	0 major timeouts
	avg bytes sent per op: 179112	avg bytes received per op: 111
	backlog wait: 2416.041204 	RTT: 11.033735 	total execute time: 2427.148953 (milliseconds)

249% What does it tell me?

This is fedora19 with 3.10.3-300.fc19.x86_64 kernel.

Tigran.