client: 2.6.7 client much slower than 2.4.26

DISCUSSION Bernd Schubert 2 participants 3 messages Jul 1, 2004

Subsystems (2)

NFS client NFS server

lore

Share & tools

lore

Bernd Schubert

author

2004-07-01 21:18 UTC #0 lore raw

Hi,

I'm currently testing 2.6.7 on one of our clients and the nfs transfer is much slower than when the same system is running 2.4.X

Well, the usual suggestion probably would be to run nfs over tcp now, but tcp is already enabled (from /proc/mounts):

taylor:/worka /mnt/test nfs rw,v3,rsize=32768,wsize=32768,hard,tcp,lock,addr=taylor 0 0

benchmarks:

copy a file to the server (2.6.7): ~ 3MB/s
copy a file to the server (2.4.26): ~ 9MB/s

copy a file from the server (2.6.7): ~ 3MB/s
copy a file from the server (2.4.26): ~ 11MB/s

The filesize was 300MB.

Here's the nfsstat output:

euklid:~# nfsstat
Client rpc stats:
calls      retrans    authrefrsh
385466     0          0
Client nfs v2:
null       getattr    setattr    root       lookup     readlink
0       0% 27475  66% 231     0% 0       0% 3963    9% 161     0%
read       wrcache    write      create     remove     rename
8303   20% 0       0% 560     1% 171     0% 198     0% 42      0%
link       symlink    mkdir      rmdir      readdir    fsstat
45      0% 0       0% 3       0% 3       0% 129     0% 2       0%

Client nfs v3:
null getattr setattr lookup access readlink
0 0% 56851 16% 1047 0% 17711 5% 36670 10% 936 0%
read write create mkdir symlink mknod
73640 21% 149802 43% 629 0% 86 0% 1 0% 0 0%
remove rmdir rename link readdir readdirplus
701 0% 105 0% 279 0% 36 0% 464 0% 1557 0%
fsstat fsinfo pathconf commit
260 0% 18 0% 0 0% 3387 0%

I already tried to reduce rsize and wsize up to 1024, but this didn't change anything.

This happens to two different servers, one server is running 2.6.7 and has Gigabit connection to the switch. The other server is running 2.4.25 and only has a 100MBit connection. All of our clients have 100MBit connections.

So I really thing this is a client problem. Well, the client is not the fastest of our systems (PII 450), but with 2.4.X it has the usual speed. Its also interesting top is showing that the rpciod is taking all cpu-time when copying a file to the server.

Any ideas whats going on?

Thanks,
Bernd

Trond Myklebust

2004-07-01 21:43 UTC #1 lore raw

P=E5 to , 01/07/2004 klokka 17:18, skreiv Bernd Schubert:

Quoted 7 lines

benchmarks:
=20
copy a file to the server (2.6.7): ~ 3MB/s
copy a file to the server (2.4.26): ~ 9MB/s
=20
copy a file from the server (2.6.7): ~ 3MB/s
copy a file from the server (2.4.26): ~ 11MB/s

Looks like the wire speeds on read/write are the same. How does the above compare to ttcp transfer speeds of a 300MB file?

Quoted 1 line

Here's the nfsstat output:

What's that for, 2.4.26 or 2.6.7? How does it compare?

Quoted 2 lines

So I really thing this is a client problem. Well, the client is not the=20 fastest of our systems (PII 450), but with 2.4.X it has the usual speed. =

Its=20

Quoted 1 line

also interesting top is showing that the rpciod is taking all cpu-time wh=

en=20

Quoted 1 line

copying a file to the server.

How much memory does this thing have? The main difference between 2.4.x and 2.6.x is the fact that the 2.6.x caches a lot more data and for longer (the VM is what decides when to flush the cache).
You might want to try twiddling around with the vm sysctl parameters to see if that makes a difference.

Cheers,
Trond

-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Bernd Schubert

author

2004-07-01 22:32 UTC #2 lore raw

Quoted 11 lines · you

P=E5 to , 01/07/2004 klokka 17:18, skreiv Bernd Schubert:
Quoted 7 lines
benchmarks:
copy a file to the server (2.6.7): ~ 3MB/s
copy a file to the server (2.4.26): ~ 9MB/s
copy a file from the server (2.6.7): ~ 3MB/s
copy a file from the server (2.4.26): ~ 11MB/s
Looks like the wire speeds on read/write are the same. How does the above compare to ttcp transfer speeds of a 300MB file?

Thanks, I didn't know about this tool and I think you are right, it seems t= hat=20
the general network speed is much slower with 2.6.7:

2.4.26:
bernd@euklid bernd>nttcp -T taylor
Bytes Real s CPU s Real-MBit/s CPU-MBit/s Calls Real-C/s CPU-=
C/s
l 8388608 0.70 0.06 95.2480 1118.4811 2048 2906.74 3413=
3.3
1 8388608 0.71 0.03 94.1283 2485.7897 5793 8125.38 21457=
9.4

2.6.7:
bernd@euklid bernd>nttcp -T taylor
Bytes Real s CPU s Real-MBit/s CPU-MBit/s Calls Real-C/s CPU-=
C/s
l 8388608 2.67 2.64 25.1472 25.4046 2048 767.43 77=
5.3
1 8388608 2.67 0.03 25.1442 2237.3350 6145 2302.40 20486=
7.5

Its already too late, so I will do the 300MB test tomorrow, but I think the= =20
numbers from the default 8MB test are already convincing.

Quoted 1 line

How much memory does this thing have? The main difference between 2.4.x

Somehow this system didn't like to have more than 2x128MB, when we added=20 another DIMM, the system crashed pretty often, though memtest86 showed no=20 problems.=20

Quoted 5 lines

and 2.6.x is the fact that the 2.6.x caches a lot more data and for longer (the VM is what decides when to flush the cache).
You might want to try twiddling around with the vm sysctl parameters to see if that makes a difference.

I will try to play with this over the weekend.

Thanks again a lot for your help!

Cheers,
Bernd