Skip to contentSkip to conversation

NFS performance debuggins

NFS Adrian von Bidder 7 participants 18 messages Jun 23, 2008 – Jun 30, 2008 7 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

Environment:

several Debian based clients (Debian etch and etchnhalf kernels, this means 2.6.18 or 2.6.24); Debian etch (2.6.18 kernel) NFS (v3) server. Network seems basically ok ("ping -f -s 3000" works without losses, ifconfig and switch monitoring shows no errors) with no noticeable load. Disks seem to have very little load either, NFS server has no other tasks.

Performance is sluggish :-( Basically works, though -- no spurious errors.

tcpdump shows many "reply ERR 1448" etc. msgs whenever NFS activitiy is going on (both stat like with "find /home" or read/write with dd)

+++
16:49:24.778560 IP 10.0.1.2.2049 > 10.0.0.209.809066834: reply ERR 1448 16:49:24.790304 IP 10.0.1.2.2049 > 10.0.0.209.943279929: reply ERR 1448 16:49:24.801380 IP 10.0.1.2.2049 > 10.0.0.209.2001885801: reply ERR 1448 16:49:24.802173 IP 10.0.1.2.2049 > 10.0.0.209.860835666: reply ERR 1448 16:49:24.805286 IP 10.0.1.2.2049 > 10.0.0.209.1479697199: reply ERR 1332 16:49:24.807679 IP 10.0.1.2.2049 > 10.0.0.209.1096249460: reply ERR 1448 16:49:24.808358 IP 10.0.1.2.2049 > 10.0.0.209.2000902760: reply ERR 1332 16:49:24.809097 IP 10.0.1.2.2049 > 10.0.0.209.926298420: reply ERR 1448 16:49:24.809100 IP 10.0.1.2.2049 > 10.0.0.209.25105411: reply ERR 1332 16:49:24.817923 IP 10.0.1.2.2049 > 10.0.0.209.1366504235: reply ERR 1448 16:49:24.817927 IP 10.0.1.2.2049 > 10.0.0.209.352525071: reply ERR 1332 16:49:24.820397 IP 10.0.1.2.2049 > 10.0.0.209.269848846: reply ERR 1332 16:49:24.822097 IP 10.0.1.2.2049 > 10.0.0.209.1345540144: reply ERR 1448 16:49:24.822856 IP 10.0.1.2.2049 > 10.0.0.209.944780599: reply ERR 1448 16:49:24.825109 IP 10.0.1.2.2049 > 10.0.0.209.1395668559: reply ERR 1448 16:49:24.825112 IP 10.0.1.2.2049 > 10.0.0.209.1999335795: reply ERR 1332 16:49:24.827813 IP 10.0.1.2.2049 > 10.0.0.209.1685677906: reply ERR 1332 16:49:24.829439 IP 10.0.1.2.2049 > 10.0.0.209.1666084982: reply ERR 1448 16:49:24.829443 IP 10.0.1.2.2049 > 10.0.0.209.1415656037: reply ERR 1332 16:49:24.839013 IP 10.0.1.2.2049 > 10.0.0.209.911226680: reply ERR 1448 16:49:24.839017 IP 10.0.1.2.2049 > 10.0.0.209.1735414852: reply ERR 1332 16:49:24.841325 IP 10.0.1.2.2049 > 10.0.0.209.911358287: reply ERR 1332 16:49:24.842092 IP 10.0.1.2.2049 > 10.0.0.209.1364284211: reply ERR 1448 16:49:24.842800 IP 10.0.1.2.2049 > 10.0.0.209.258643250: reply ERR 1332 16:49:24.844256 IP 10.0.1.2.2049 > 10.0.0.209.1666017882: reply ERR 1448 16:49:24.844996 IP 10.0.1.2.2049 > 10.0.0.209.808595513: reply ERR 1448 16:49:24.845674 IP 10.0.1.2.2049 > 10.0.0.209.2000779112: reply ERR 1448 16:49:24.845677 IP 10.0.1.2.2049 > 10.0.0.209.1652175121: reply ERR 1332 16:49:24.847120 IP 10.0.1.2.2049 > 10.0.0.209.944722769: reply ERR 1448 16:49:24.847123 IP 10.0.1.2.2049 > 10.0.0.209.1682657874: reply ERR 1332 16:49:24.849334 IP 10.0.1.2.2049 > 10.0.0.209.944714835: reply ERR 1448 16:49:24.850873 IP 10.0.1.2.2049 > 10.0.0.209.1345861938: reply ERR 1448 16:49:24.918710 IP 10.0.1.2.2049 > 10.0.0.179.1936680564: reply ERR 1448 16:49:24.918719 IP 10.0.1.2.2049 > 10.0.0.179.1698508838: reply ERR 1448 16:49:24.921911 IP 10.0.1.2.2049 > 10.0.0.179.1633904741: reply ERR 1448
+++

Mount options: "rw,noatime,rsize=8192,wsize=8192,intr,hard,addr=10.0.1.2", it seems to pick tcp by default. I had problems with UDP from some of the clients due to a strangely buggy VDSL switch in the path, so I haven't tried that again (I want to keep the DSL clients and the non-DSL clients identical if this is at all possible, so I can switch equipment around without reconfiguration.)

That performance is not optimal whith todays desktop environments (tons of small configuration files in both oo.org and kde) at login/program start on cold caches is one thing, but performance

Now where do I start debugging this?

Signature · 5 lines
Development costs of average proprietary and free software don't differ
radically because the methods are pretty much the same.  The huge
difference lies in the way the developers try to recoup their costs, not
in the costs they have to compensate.
        -- Florian Weimer on debian-security

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 60 lines · Adrian von Bidder

Hi,

Environment:

several Debian based clients (Debian etch and etchnhalf kernels, this means 2.6.18 or 2.6.24); Debian etch (2.6.18 kernel) NFS (v3) server. Network seems basically ok ("ping -f -s 3000" works without losses, ifconfig and switch monitoring shows no errors) with no noticeable load. Disks seem to have very little load either, NFS server has no other tasks.

Performance is sluggish :-( Basically works, though -- no spurious errors.

tcpdump shows many "reply ERR 1448" etc. msgs whenever NFS activitiy is going on (both stat like with "find /home" or read/write with dd)

+++
16:49:24.778560 IP 10.0.1.2.2049 > 10.0.0.209.809066834: reply ERR 1448 16:49:24.790304 IP 10.0.1.2.2049 > 10.0.0.209.943279929: reply ERR 1448 16:49:24.801380 IP 10.0.1.2.2049 > 10.0.0.209.2001885801: reply ERR 1448 16:49:24.802173 IP 10.0.1.2.2049 > 10.0.0.209.860835666: reply ERR 1448 16:49:24.805286 IP 10.0.1.2.2049 > 10.0.0.209.1479697199: reply ERR 1332 16:49:24.807679 IP 10.0.1.2.2049 > 10.0.0.209.1096249460: reply ERR 1448 16:49:24.808358 IP 10.0.1.2.2049 > 10.0.0.209.2000902760: reply ERR 1332 16:49:24.809097 IP 10.0.1.2.2049 > 10.0.0.209.926298420: reply ERR 1448 16:49:24.809100 IP 10.0.1.2.2049 > 10.0.0.209.25105411: reply ERR 1332 16:49:24.817923 IP 10.0.1.2.2049 > 10.0.0.209.1366504235: reply ERR 1448 16:49:24.817927 IP 10.0.1.2.2049 > 10.0.0.209.352525071: reply ERR 1332 16:49:24.820397 IP 10.0.1.2.2049 > 10.0.0.209.269848846: reply ERR 1332 16:49:24.822097 IP 10.0.1.2.2049 > 10.0.0.209.1345540144: reply ERR 1448 16:49:24.822856 IP 10.0.1.2.2049 > 10.0.0.209.944780599: reply ERR 1448 16:49:24.825109 IP 10.0.1.2.2049 > 10.0.0.209.1395668559: reply ERR 1448 16:49:24.825112 IP 10.0.1.2.2049 > 10.0.0.209.1999335795: reply ERR 1332 16:49:24.827813 IP 10.0.1.2.2049 > 10.0.0.209.1685677906: reply ERR 1332 16:49:24.829439 IP 10.0.1.2.2049 > 10.0.0.209.1666084982: reply ERR 1448 16:49:24.829443 IP 10.0.1.2.2049 > 10.0.0.209.1415656037: reply ERR 1332 16:49:24.839013 IP 10.0.1.2.2049 > 10.0.0.209.911226680: reply ERR 1448 16:49:24.839017 IP 10.0.1.2.2049 > 10.0.0.209.1735414852: reply ERR 1332 16:49:24.841325 IP 10.0.1.2.2049 > 10.0.0.209.911358287: reply ERR 1332 16:49:24.842092 IP 10.0.1.2.2049 > 10.0.0.209.1364284211: reply ERR 1448 16:49:24.842800 IP 10.0.1.2.2049 > 10.0.0.209.258643250: reply ERR 1332 16:49:24.844256 IP 10.0.1.2.2049 > 10.0.0.209.1666017882: reply ERR 1448 16:49:24.844996 IP 10.0.1.2.2049 > 10.0.0.209.808595513: reply ERR 1448 16:49:24.845674 IP 10.0.1.2.2049 > 10.0.0.209.2000779112: reply ERR 1448 16:49:24.845677 IP 10.0.1.2.2049 > 10.0.0.209.1652175121: reply ERR 1332 16:49:24.847120 IP 10.0.1.2.2049 > 10.0.0.209.944722769: reply ERR 1448 16:49:24.847123 IP 10.0.1.2.2049 > 10.0.0.209.1682657874: reply ERR 1332 16:49:24.849334 IP 10.0.1.2.2049 > 10.0.0.209.944714835: reply ERR 1448 16:49:24.850873 IP 10.0.1.2.2049 > 10.0.0.209.1345861938: reply ERR 1448 16:49:24.918710 IP 10.0.1.2.2049 > 10.0.0.179.1936680564: reply ERR 1448 16:49:24.918719 IP 10.0.1.2.2049 > 10.0.0.179.1698508838: reply ERR 1448 16:49:24.921911 IP 10.0.1.2.2049 > 10.0.0.179.1633904741: reply ERR 1448
+++

Mount options: "rw,noatime,rsize=8192,wsize=8192,intr,hard,addr=10.0.1.2", it seems to pick tcp by default. I had problems with UDP from some of the clients due to a strangely buggy VDSL switch in the path, so I haven't tried that again (I want to keep the DSL clients and the non-DSL clients identical if this is at all possible, so I can switch equipment around without reconfiguration.)

Quoted 5 lines

That performance is not optimal whith todays desktop environments (tons of small configuration files in both oo.org and kde) at login/program start on cold caches is one thing, but performance

Now where do I start debugging this?

In the above dump 1448 is _not_ the error code, but rather the packet length. You might therefore try using the tcpdump option '-vvv' to see if you can obtain the actual error value (which should tell you why the NFS server is rejecting your packets).
Alternatively, you might consider using wireshark/tshark, which can display NFS packets in a much more readable fashion.

Cheers
Trond

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#3 lore raw
In reply to a message not in the archive
Quoted 11 lines · Adrian von Bidder

Hi,

Environment:

several Debian based clients (Debian etch and etchnhalf kernels, this means 2.6.18 or 2.6.24); Debian etch (2.6.18 kernel) NFS (v3) server. Network seems basically ok ("ping -f -s 3000" works without losses, ifconfig and switch monitoring shows no errors) with no noticeable load. Disks seem to have very little load either, NFS server has no other tasks.

Performance is sluggish :-( Basically works, though -- no spurious errors.

In what way exactly is it sluggish?

Quoted 2 lines

tcpdump shows many "reply ERR 1448" etc. msgs whenever NFS activitiy is going on (both stat like with "find /home" or read/write with dd)

I'm afraid I don't know how to read that tcpdump output.

--b.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#4 lore raw

Re: [NFS] NFS performance debugging

Hi again,

Thanks for your replies (You too, Trond)

On Monday 23 June 2008 21.28:36 you wrote:

[... NFS performance ...]

Quoted 1 line

In what way exactly is it sluggish?

Starting KDE, opening documents, sometimes also closing oo.org and saving documents takes several seconds longer than on local disk.

Certainly network latency (especially with these silly lots of small config files) takes some time, but I'm still surprised. At the same time, I don't have data to compare a "known good" NFS against ours, so perhaps NFS is indeed so slow?

Quoted 5 lines
Quoted 3 lines

tcpdump shows many "reply ERR 1448" etc. msgs whenever NFS activitiy is going on (both stat like with "find /home" or read/write with dd)

I'm afraid I don't know how to read that tcpdump output.

tcpdump "-vvv" doesn't give more information on these packets; at the same time wireshark doesn't show anything suspicious except tons of wrong TCP checksums caused (I hope...) by offloading. I'll have to look if I can get the raw traffic at the network switch to check this (but I think with 30% and more wrong tcp checksums, traffic would completely break down so I'm quite confident here.)

Slightly different topic: is there an NFS related mailing list I can subscribe to? This one is apparently closed for new subscribers, and the bounce instructs me to send mail to [email protected] which bounces :-( Reading others' NFS postings might just give me more ideas on where to look.

TODO today: play around with NFSv4 on the shaky assumption that nfsv3 is actually working but net latency is killing my performance.

cheers
-- vbi

Signature · 3 lines
The typewriting machine, when played with expression, is no more
annoying than the piano when played by a sister or near relation.
		-- Oscar Wilde

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw

Re: [NFS] NFS performance debugging

Quoted 3 lines · Adrian von Bidder

Hi again,

Thanks for your replies (You too, Trond)

The custom around here is to leave everyone on the cc: line.

Quoted 9 lines · On Monday 23 June 2008 21.28:36 you wrote:

[... NFS performance ...]

Quoted 1 line

In what way exactly is it sluggish?

Starting KDE, opening documents, sometimes also closing oo.org and saving documents takes several seconds longer than on local disk.

"close" on nfs is an operation that requires a round-trip to the server and waiting for the disk to commit any writes made before the close, so if you've got to do a lot of those it can take time. Fooling with the journaling on the exported filesystem may help.

Quoted 6 lines

Certainly network latency (especially with these silly lots of small config files) takes some time, but I'm still surprised. At the same time, I don't have data to compare a "known good" NFS against ours, so perhaps NFS is indeed so slow?

Quoted 1 line
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

tcpdump shows many "reply ERR 1448" etc. msgs whenever NFS activitiy is going on (both stat like with "find /home" or read/write with dd)

I'm afraid I don't know how to read that tcpdump output.

tcpdump "-vvv" doesn't give more information on these packets; at the same time wireshark doesn't show anything suspicious except tons of wrong TCP checksums caused (I hope...) by offloading.

Yes, that's normal.

Quoted 9 lines

I'll have to look if I can get
the raw traffic at the network switch to check this (but I think with 30% and more wrong tcp checksums, traffic would completely break down so I'm quite confident here.)

Slightly different topic: is there an NFS related mailing list I can subscribe to? This one is apparently closed for new subscribers, and the bounce instructs me to send mail to [email protected] which

						^^^^
						vger

Where'd the typo in that address get introduced?

Quoted 2 lines

bounces :-( Reading others' NFS postings might just give me more ideas on where to look.

Should be: http://vger.kernel.org/vger-lists.html#linux-nfs

Quoted 2 lines

TODO today: play around with NFSv4 on the shaky assumption that nfsv3 is actually working but net latency is killing my performance.

Delegations *might* help if the problem is really open latency.

--b.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#7 lore raw

Re: [NFS] NFS performance debugging

Quoted 9 lines · On Tuesday 24 June 2008 22.29:31 J. Bruce Fields wrote:
Quoted 2 lines · Adrian von Bidder

Starting KDE, opening documents, sometimes also closing oo.org and saving documents takes several seconds longer than on local disk.

"close" on nfs is an operation that requires a round-trip to the server and waiting for the disk to commit any writes made before the close, so if you've got to do a lot of those it can take time. Fooling with the journaling on the exported filesystem may help.

Are there tools to measure latencies on NFS? Given a network dump, desired output would be histograms of latencies by file operation? (Or maybe I can catch the information on the client, VFS side instead of NFS?

At this time, I really need to collect more data on where the problem is since all I'm doing right now is fooling around based on assumptions... :-(

OTOH I'd suspect KDE/oo.org startup to be mostly reads of those config files, so the problem shouldn't be close latencies. Assumptions again.

Quoted 4 lines
Quoted 2 lines

TODO today: play around with NFSv4 on the shaky assumption that nfsv3 is actually working but net latency is killing my performance.

Delegations *might* help if the problem is really open latency.

First tries showed
* There are no acl on my files now
* user id mapping seems funny: some users map to nobody, others map
correctly. Huh?
* Performance seems to be ok (timing desktop applications is always
difficult, and so far I'm working against on the production server with varying load anyway...)

Haven't investigated these yet...

cheers
-- vbi

Signature · 1 line
Today is Sweetmorn, the 30th day of Confusion in the YOLD 3174

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw

Re: [NFS] NFS performance debugging

Quoted 6 lines · Adrian von Bidder
Quoted 5 lines · On Tuesday 24 June 2008 22.29:31 J. Bruce Fields wrote:
Quoted 2 lines · Adrian von Bidder

Starting KDE, opening documents, sometimes also closing oo.org and saving documents takes several seconds longer than on local disk.

Quoted 8 lines
Quoted 4 lines

"close" on nfs is an operation that requires a round-trip to the server and waiting for the disk to commit any writes made before the close, so if you've got to do a lot of those it can take time. Fooling with the journaling on the exported filesystem may help.

Are there tools to measure latencies on NFS? Given a network dump, desired output would be histograms of latencies by file operation? (Or maybe I can catch the information on the client, VFS side instead of NFS?

You can catch the info on the client. See the 'nfs-iostat' tool in the latest nfs-utils git tree:

http://git.linux-nfs.org/?p=steved/nfs-utils.git;a=summary

Trond

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#10 lore raw

Re: [NFS] NFS performance debugging

Quoted 9 lines · Trond Myklebust
Quoted 8 lines · Adrian von Bidder
Quoted 7 lines · On Tuesday 24 June 2008 22.29:31 J. Bruce Fields wrote:
Quoted 2 lines · Adrian von Bidder

Starting KDE, opening documents, sometimes also closing oo.org and saving documents takes several seconds longer than on local disk.

"close" on nfs is an operation that requires a round-trip to the server and waiting for the disk to commit any writes made before the close, so if you've got to do a lot of those it can take time. Fooling with the journaling on the exported filesystem may help.

Quoted 6 lines
Quoted 3 lines

Are there tools to measure latencies on NFS? Given a network dump, desired output would be histograms of latencies by file operation? (Or maybe I can catch the information on the client, VFS side instead of NFS?

You can catch the info on the client. See the 'nfs-iostat' tool in the latest nfs-utils git tree:

One side note on the current state of the nfs-iostat tool... There is some talk about changing the name from 'nfs-iostat' to 'nfsiostat' so it will map better with the current command names that are in nfs-utils (i.e. no other commands use have a '-' in their names). So at this point, even though the script is in the tree, its not being installed (via a make install) which allows us to make these types of changes.

But please do not let this caveat stop you from trying this script and mountstats script. We definitely need feedback on how well they do or don't work (and as always... patches are welcomed! ;-) ) Once things harden up (via any feedback), I'll added the install code to make these scripts be installed (probably in /usr/sbin) which means they will be ready for prime time...

steved.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

A message that is not in the archive (sent off-list or not yet received): [email protected]

#12 lore raw

Re: [NFS] NFS performance debugging

[email protected] wrote on 06/25/2008 07:29:16 PM:

Quoted 3 lines

But please do not let this caveat stop you from trying this script and mountstats script. We definitely need feedback on how well they do or don't work (and as always... patches are welcomed! ;-) ) Once things

harden

Quoted 1 line

up (via any feedback), I'll added the install code to make these scripts

be

Quoted 2 lines

installed (probably in /usr/sbin) which means they will be ready for prime time...

Hi Steve,

When I ran this util, it worked the first time:

[root@localhost ~]# /tmp/nfs-iostat

localhost:/local mounted on /nfs:

op/s rpc bklog
1.27 0.01
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                  0.619          39.772          64.273        0 (0.0%)
16.889          16.927
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                  0.626          39.112          62.523        0 (0.0%)

26.357 4039.241

8.124.111.71:/mnt mounted on /mnt:

op/s rpc bklog
3.00 0.00
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                  0.000           0.000           0.000        0 (0.0%)
0.000           0.000
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                  0.000           0.000           0.000        0 (0.0%)

0.000 0.000

(KK: for some reason, the real NFS mount gives all zero. Also, for loopback, the read/write
BW is very poor: less than 40 Kb/s).

But from the next time I ran, and all subsequent runs (and even after a reboot),
I get:

[root@localhost init.d]# /tmp/nfs-iostat
localhost:/local mounted on /nfs:

op/s rpc bklog
0.04 0.00
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                  0.000           0.000           0.000        0 (0.0%)
0.000           0.000
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                  0.000           0.000           0.000        0 (0.0%)

0.000 0.000

8.124.111.71:/mnt mounted on /mnt:

op/s rpc bklog
6.00 0.00
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                  0.000           0.000           0.000        0 (0.0%)
0.000           0.000
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                  0.000           0.000           0.000        0 (0.0%)

0.000 0.000

(KK: every filesystem gives zero for all parameters).

What could be wrong?

Thanks,

- KK

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#13 lore raw

Re: [NFS] NFS performance debugging

Quoted 41 lines · On Fri, Jun 27, 2008 at 3:30 AM, Krishna Kumar2 <[email protected]> wrote:

[email protected] wrote on 06/25/2008 07:29:16 PM:

Quoted 3 lines

But please do not let this caveat stop you from trying this script and mountstats script. We definitely need feedback on how well they do or don't work (and as always... patches are welcomed! ;-) ) Once things

harden

Quoted 1 line

up (via any feedback), I'll added the install code to make these scripts

be

Quoted 2 lines

installed (probably in /usr/sbin) which means they will be ready for prime time...

Hi Steve,

When I ran this util, it worked the first time:

[root@localhost ~]# /tmp/nfs-iostat

localhost:/local mounted on /nfs:

op/s rpc bklog
1.27 0.01
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                 0.619          39.772          64.273        0 (0.0%)
16.889          16.927
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                 0.626          39.112          62.523        0 (0.0%)

26.357 4039.241

8.124.111.71:/mnt mounted on /mnt:

op/s rpc bklog
3.00 0.00
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                 0.000           0.000           0.000        0 (0.0%)
0.000           0.000
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                 0.000           0.000           0.000        0 (0.0%)
Quoted 5 lines

0.000 0.000

(KK: for some reason, the real NFS mount gives all zero. Also, for loopback, the read/write
BW is very poor: less than 40 Kb/s).

As an aside, Jim Rees pointed out to me that that should be KB/s, not Kb/s. I will post a patch to address that.

Quoted 28 lines

But from the next time I ran, and all subsequent runs (and even after a reboot), I get:

[root@localhost init.d]# /tmp/nfs-iostat
localhost:/local mounted on /nfs:

op/s rpc bklog
0.04 0.00
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                 0.000           0.000           0.000        0 (0.0%)
0.000           0.000
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                 0.000           0.000           0.000        0 (0.0%)

0.000 0.000

8.124.111.71:/mnt mounted on /mnt:

op/s rpc bklog
6.00 0.00
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                 0.000           0.000           0.000        0 (0.0%)
0.000           0.000
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                 0.000           0.000           0.000        0 (0.0%)
Quoted 5 lines

0.000 0.000

(KK: every filesystem gives zero for all parameters).

What could be wrong?

At a guess, it's because the client is now caching your file data?

Can you post a copy of your mount command line, and your /proc/self/mountstats file after a test run (but before you unmount)?

--
Chuck Lever

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

A message that is not in the archive (sent off-list or not yet received): 76bd70e30806270644j3e67c83and7b1f7fd6cc2f5f5-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#15 lore raw

Re: [NFS] NFS performance debugging

Quoted 7 lines · Chuck Lever
Quoted 4 lines

(KK: for some reason, the real NFS mount gives all zero. Also, for loopback, the read/write
BW is very poor: less than 40 Kb/s).

As an aside, Jim Rees pointed out to me that that should be KB/s, not Kb/s. I will post a patch to address that.

Strictly speaking, neither of the above are correct: the SI abbreviation for kilo is 'k'. IOW it should be 'kB/s'

Cheers
Trond

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#16 lore raw

Re: [NFS] NFS performance debugging

In reply to a message not in the archive

[email protected] wrote on 06/27/2008 07:14:38 PM:

Quoted 1 line

At a guess, it's because the client is now caching your file data?

Today it is working fine, so maybe something was wrong earlier. However, nfs-iostat shows very low BW:

op/s rpc bklog
42.60 0.00
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                 42.542         2734.324         64.273        0 (0.0%)
18.480   18.518
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)     avg exe (ms)
                  0.000           0.000           0.000        0 (0.0%)

0.000 0.000

A cat of the file (after fresh mount of /local and /nfs) on the NFS mountpoint to
/dev/null gives around 29.4 MB/s (vs 39.1 MB/s on /local).

Quoted 2 lines

Can you post a copy of your mount command line, and your /proc/self/mountstats file after a test run (but before you unmount)?

mount:
mount -o
rw,bg,hard,nointr,proto=tcp,vers=3,rsize=65536,wsize=65536,timeo=600,noatime
localhost:/local /nfs

mountstats file:

device rootfs mounted on / with fstype rootfs
device /dev/root mounted on / with fstype ext3
device /dev mounted on /dev with fstype tmpfs
device /proc mounted on /proc with fstype proc
device /sys mounted on /sys with fstype sysfs
device none mounted on /selinux with fstype selinuxfs
device /proc/bus/usb mounted on /proc/bus/usb with fstype usbfs
device devpts mounted on /dev/pts with fstype devpts
device tmpfs mounted on /dev/shm with fstype tmpfs
device none mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
device sunrpc mounted on /var/lib/nfs/rpc_pipefs with fstype rpc_pipefs
device /etc/auto.misc mounted on /misc with fstype autofs
device -hosts mounted on /net with fstype autofs
device nfsd mounted on /proc/fs/nfsd with fstype nfsd
device /dev/sda6 mounted on /local with fstype ext3
device localhost:/local mounted on /nfs with fstype nfs statvers=1.0
opts:
rw,vers=3,rsize=65536,wsize=65536,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys

        age:    79
        caps:   caps=0x9,wtmult=4096,dtsize=4096,bsize=0,namelen=255
        sec:    flavor=1,pseudoflavor=1
        events: 0 0 0 0 1 3 4 0 0 609 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
        bytes:  595722240 0 0 0 596639744 0 145664 0
        RPC iostats version: 1.0  p/v: 100003/3 (nfs)
        xprt:   tcp 855 0 2 0 60 9117 9117 0 72672 0
        per-op statistics
                NULL: 0 0 0 0 0 0 0 0
             GETATTR: 2 2 0 264 224 0 0 0
             SETATTR: 0 0 0 0 0 0 0 0
              LOOKUP: 3 3 0 448 684 0 1 1
              ACCESS: 4 4 0 568 480 0 0 0
            READLINK: 0 0 0 0 0 0 0 0
                READ: 9104 9104 0 1383808 597805056 15 168246 168587
               WRITE: 0 0 0 0 0 0 0 0
              CREATE: 0 0 0 0 0 0 0 0
               MKDIR: 0 0 0 0 0 0 0 0
             SYMLINK: 0 0 0 0 0 0 0 0
               MKNOD: 0 0 0 0 0 0 0 0
              REMOVE: 0 0 0 0 0 0 0 0
               RMDIR: 0 0 0 0 0 0 0 0
              RENAME: 0 0 0 0 0 0 0 0
                LINK: 0 0 0 0 0 0 0 0
             READDIR: 0 0 0 0 0 0 0 0
         READDIRPLUS: 0 0 0 0 0 0 0 0
              FSSTAT: 0 0 0 0 0 0 0 0
              FSINFO: 1 1 0 132 80 0 0 0
            PATHCONF: 0 0 0 0 0 0 0 0
              COMMIT: 0 0 0 0 0 0 0 0

And:

[root@localhost nfs]# nfsstat -rc
Client rpc stats:

calls      retrans    authrefrsh
713710     19         0

Thanks,

- KK

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#17 lore raw

Re: [NFS] NFS performance debugging

In reply to Krishna Kumar2
Quoted 82 lines · Krishna Kumar2 wrote:

[email protected] wrote on 06/25/2008 07:29:16 PM:

Quoted 4 lines

But please do not let this caveat stop you from trying this script and mountstats script. We definitely need feedback on how well they do or don't work (and as always... patches are welcomed! ;-) ) Once things

harden

Quoted 2 lines

up (via any feedback), I'll added the install code to make these scripts

be

Quoted 3 lines

installed (probably in /usr/sbin) which means they will be ready for prime time...

Hi Steve,

When I ran this util, it worked the first time:

[root@localhost ~]# /tmp/nfs-iostat

localhost:/local mounted on /nfs:

op/s rpc bklog
1.27 0.01
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                  0.619          39.772          64.273        0 (0.0%)
16.889          16.927
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                  0.626          39.112          62.523        0 (0.0%)

26.357 4039.241

8.124.111.71:/mnt mounted on /mnt:

op/s rpc bklog
3.00 0.00
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                  0.000           0.000           0.000        0 (0.0%)
0.000           0.000
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                  0.000           0.000           0.000        0 (0.0%)

0.000 0.000

(KK: for some reason, the real NFS mount gives all zero. Also, for loopback, the read/write
BW is very poor: less than 40 Kb/s).

But from the next time I ran, and all subsequent runs (and even after a reboot),
I get:

[root@localhost init.d]# /tmp/nfs-iostat
localhost:/local mounted on /nfs:

op/s rpc bklog
0.04 0.00
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                  0.000           0.000           0.000        0 (0.0%)
0.000           0.000
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                  0.000           0.000           0.000        0 (0.0%)

0.000 0.000

8.124.111.71:/mnt mounted on /mnt:

op/s rpc bklog
6.00 0.00
read: ops/s Kb/s Kb/op retrans
avg RTT (ms) avg exe (ms)

                  0.000           0.000           0.000        0 (0.0%)
0.000           0.000
write:            ops/s            Kb/s           Kb/op         retrans
avg RTT (ms)    avg exe (ms)
                  0.000           0.000           0.000        0 (0.0%)
Quoted 6 lines

0.000 0.000

(KK: every filesystem gives zero for all parameters).

What could be wrong?

Are you running nfs-iostat just once? While doing your testing, you should do "nfs-iostat 2" so that every 2secs nfs-iostat displays information about the on-going test (you can choose any other polling interval value).

Quoted 18 lines

Thanks,

- KK

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

Quoted 6 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#18 lore raw

Re: [NFS] NFS performance debugging

In reply to a message not in the archive
Quoted 21 lines · Adrian von Bidder
Quoted 9 lines · On Tuesday 24 June 2008 22.29:31 J. Bruce Fields wrote:
Quoted 2 lines · Adrian von Bidder

Starting KDE, opening documents, sometimes also closing oo.org and saving documents takes several seconds longer than on local disk.

"close" on nfs is an operation that requires a round-trip to the server and waiting for the disk to commit any writes made before the close, so if you've got to do a lot of those it can take time. Fooling with the journaling on the exported filesystem may help.

Are there tools to measure latencies on NFS? Given a network dump, desired output would be histograms of latencies by file operation? (Or maybe I can catch the information on the client, VFS side instead of NFS?

At this time, I really need to collect more data on where the problem is since all I'm doing right now is fooling around based on assumptions... :-(

OTOH I'd suspect KDE/oo.org startup to be mostly reads of those config files, so the problem shouldn't be close latencies. Assumptions again.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

TODO today: play around with NFSv4 on the shaky assumption that nfsv3 is actually working but net latency is killing my performance.

Delegations *might* help if the problem is really open latency.

First tries showed
* There are no acl on my files now

NFSv4 uses an entirely different type of ACL, for which you need different client-side tools; see

http://www.citi.umich.edu/projects/nfsv4/linux/nfs4-acl-tools/

Quoted 2 lines

* user id mapping seems funny: some users map to nobody, others map
correctly. Huh?

And whereas v2/v3 require only uid's and gid's to agree, v4 (if you're using auth_sys) requires uid's, gid's, *and* user and group names to agree.

--b.

Quoted 5 lines

* Performance seems to be ok (timing desktop applications is always
difficult, and so far I'm working against on the production server with varying load anyway...)

Haven't investigated these yet...

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#19 lore raw

Re: [NFS] NFS performance debugging

Quoted 27 lines · On Wednesday 25 June 2008 18.56:58 J. Bruce Fields wrote:
Quoted 26 lines · Adrian von Bidder
Quoted 8 lines · On Tuesday 24 June 2008 22.29:31 J. Bruce Fields wrote:
Quoted 2 lines · Adrian von Bidder

Starting KDE, opening documents, sometimes also closing oo.org and saving documents takes several seconds longer than on local disk.

"close" on nfs is an operation that requires a round-trip to the server and waiting for the disk to commit any writes made before the close, so if you've got to do a lot of those it can take time. Fooling with the journaling on the exported filesystem may help.

Are there tools to measure latencies on NFS? Given a network dump, desired output would be histograms of latencies by file operation? (Or maybe I can catch the information on the client, VFS side instead of NFS?

At this time, I really need to collect more data on where the problem is since all I'm doing right now is fooling around based on assumptions... :-(

OTOH I'd suspect KDE/oo.org startup to be mostly reads of those config files, so the problem shouldn't be close latencies. Assumptions again.

Quoted 4 lines
Quoted 3 lines

TODO today: play around with NFSv4 on the shaky assumption that nfsv3 is actually working but net latency is killing my performance.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Delegations *might* help if the problem is really open latency.

First tries showed
* There are no acl on my files now

NFSv4 uses an entirely different type of ACL, for which you need different client-side tools; see

I'm absolutely confused on the state of nfs4 acls. There are so many old mailing list posts around that it's difficult to see what information applies to the currentimplementation. So: are POSIX acls in the server filesystem somehow mapped to nfsv4 acls on the client side? I understand there understands an RFC proposal on how to do this, and
http://wiki.linux-nfs.org/wiki/index.php/ACLs is very interesting but doesn't really cover the current state of the implementation.

Quoted 1 line

http://www.citi.umich.edu/projects/nfsv4/linux/nfs4-acl-tools/

Hmm. Not packaged in Debian yet?????? ;-) So work to do for me if I end up using nfs4

Quoted 7 lines
Quoted 3 lines

* user id mapping seems funny: some users map to nobody, others map
correctly. Huh?

And whereas v2/v3 require only uid's and gid's to agree, v4 (if you're using auth_sys) requires uid's, gid's, *and* user and group names to agree.

I got that fixed. "localdomain" in idmapd.conf...

cheers
-- vbi

Quoted 2 lines

Maybe that question would be a good starting point: What's the use for a gender field there?

Stalking.
-- Miriam Ruiz, Marco d'Itri (im that order)

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

A message that is not in the archive (sent off-list or not yet received): [email protected]

#21 lore raw

Re: [NFS] NFS performance debugging

Quoted 8 lines · Adrian von Bidder
Quoted 6 lines · On Wednesday 25 June 2008 18.56:58 J. Bruce Fields wrote:
Quoted 2 lines · Adrian von Bidder

First tries showed
* There are no acl on my files now

NFSv4 uses an entirely different type of ACL, for which you need different client-side tools; see

Quoted 7 lines

I'm absolutely confused on the state of nfs4 acls. There are so many old mailing list posts around that it's difficult to see what information applies to the currentimplementation. So: are POSIX acls in the server filesystem somehow mapped to nfsv4 acls on the client side? I understand there understands an RFC proposal on how to do this, and
http://wiki.linux-nfs.org/wiki/index.php/ACLs is very interesting but doesn't really cover the current state of the implementation.

I've attempted to update that wiki page to fix that problem; could you take a look and tell me whether the changes help?

Quoted 4 lines
Quoted 1 line

http://www.citi.umich.edu/projects/nfsv4/linux/nfs4-acl-tools/

Hmm. Not packaged in Debian yet?????? ;-) So work to do for me if I end up using nfs4

Yes. Volunteers to package those things welcomed....

(Or maybe we should get them into nfs-utils at some point. I don't know.)

--b.

Quoted 18 lines
Quoted 6 lines
Quoted 2 lines

* user id mapping seems funny: some users map to nobody, others map
correctly. Huh?

And whereas v2/v3 require only uid's and gid's to agree, v4 (if you're using auth_sys) requires uid's, gid's, *and* user and group names to agree.

I got that fixed. "localdomain" in idmapd.conf...

cheers
-- vbi

Quoted 2 lines

Maybe that question would be a good starting point: What's the use for a gender field there?

Stalking.
-- Miriam Ruiz, Marco d'Itri (im that order)

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

#22 lore raw

Re: [NFS] NFS performance debugging

Quoted 18 lines · On Thursday 26 June 2008 19.03:42 you wrote:
Quoted 8 lines · Adrian von Bidder
Quoted 6 lines · On Wednesday 25 June 2008 18.56:58 J. Bruce Fields wrote:
Quoted 2 lines · Adrian von Bidder

First tries showed
* There are no acl on my files now

NFSv4 uses an entirely different type of ACL, for which you need different client-side tools; see

Quoted 8 lines

I'm absolutely confused on the state of nfs4 acls. There are so many old mailing list posts around that it's difficult to see what information applies to the currentimplementation. So: are POSIX acls in the server filesystem somehow mapped to nfsv4 acls on the client side? I understand there understands an RFC proposal on how to do this, and
http://wiki.linux-nfs.org/wiki/index.php/ACLs is very interesting but doesn't really cover the current state of the implementation.

Quoted 2 lines

I've attempted to update that wiki page to fix that problem; could you take a look and tell me whether the changes help?

The summary is: "POSIX ACLs on the server side turn (more or less) into NFSv4 ACL on the client.", correct? So our current ACL based system of permissions should work except that we can't modify permissions on the client side until I've deployed nfs4 acl tools ...

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

http://www.citi.umich.edu/projects/nfsv4/linux/nfs4-acl-tools/

Hmm. Not packaged in Debian yet?????? ;-) So work to do for me if I end up using nfs4

Yes. Volunteers to package those things welcomed....

... which means you've possibly got your Debian packager, and I might even be able to do that on company time.

Ok, I guess I know in which direction to put my work now, thanks a lot. (nfs-iostat will be the first thing I'll try, though, to get some more data.)

cheers
-- vbi

Signature · 4 lines
Why on earth should we teach children
that they are not allowed to share the toys.
        -- Patrick Harvie, Member of the Scottish Parliament
           Speaking at Debconf7

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs

A message that is not in the archive (sent off-list or not yet received): [email protected]

#24 lore raw

Re: [NFS] NFS performance debugging

Quoted 19 lines · Adrian von Bidder
Quoted 18 lines · On Thursday 26 June 2008 19.03:42 you wrote:
Quoted 16 lines · Adrian von Bidder
Quoted 6 lines · On Wednesday 25 June 2008 18.56:58 J. Bruce Fields wrote:
Quoted 2 lines · Adrian von Bidder

First tries showed
* There are no acl on my files now

NFSv4 uses an entirely different type of ACL, for which you need different client-side tools; see

I'm absolutely confused on the state of nfs4 acls. There are so many old mailing list posts around that it's difficult to see what information applies to the currentimplementation. So: are POSIX acls in the server filesystem somehow mapped to nfsv4 acls on the client side? I understand there understands an RFC proposal on how to do this, and
http://wiki.linux-nfs.org/wiki/index.php/ACLs is very interesting but doesn't really cover the current state of the implementation.

Quoted 5 lines
Quoted 2 lines

I've attempted to update that wiki page to fix that problem; could you take a look and tell me whether the changes help?

The summary is: "POSIX ACLs on the server side turn (more or less) into NFSv4 ACL on the client.", correct?

Yes (but note that that translation happens on the server).

Quoted 3 lines

So our current ACL based system of
permissions should work except that we can't modify permissions on the client side until I've deployed nfs4 acl tools ...

ACL's will definitely still be enforced correctly, yes--enforcement of permissions is entirely left up to the exported filesystem.

Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line

http://www.citi.umich.edu/projects/nfsv4/linux/nfs4-acl-tools/

Hmm. Not packaged in Debian yet?????? ;-) So work to do for me if I end up using nfs4

Yes. Volunteers to package those things welcomed....

... which means you've possibly got your Debian packager, and I might even be able to do that on company time.

If you get a chance to work on that, that would be great.

Quoted 3 lines

Ok, I guess I know in which direction to put my work now, thanks a lot. (nfs-iostat will be the first thing I'll try, though, to get some more data.)

Yes, that's probably best. (NFSv4 will be somewhat of a long shot, and in any case it'll help to first understand better where the time's going.)

--b.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
_______________________________________________
Please note that [email protected] is being discontinued. Please subscribe to [email protected] instead.
http://vger.kernel.org/vger-lists.html#linux-nfs