Skip to contentSkip to conversation

SMP, NFS, and 2.6

DISCUSSION Chris Penney 3 participants 5 messages Mar 29, 2005 – Apr 4, 2005

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I've heard from some (via google searchs) that there are issues with NFS + SMP when using the 2.6 kernel. Assuming I'm using 2.6.11.5 w/ the NFS-ALL patches applied should I be concerned about running SMP (dual cpu pentium 4 doing pretty much only NFSv3 TCP, but looking toward NFSv4)?

Assuming it's ok to use SMP on a heavy use NFS server, should I enable or disable hyperthreading? I'd assume that for NFS hyperthreading would be good, but that's only a guess.

Thanks,

Chris

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#1 lore raw
Quoted 6 lines · Chris Penney

I've heard from some (via google searchs) that there are issues with NFS + SMP when using the 2.6 kernel. Assuming I'm using 2.6.11.5 w/ the NFS-ALL patches applied should I be concerned about running SMP (dual cpu pentium 4 doing pretty much only NFSv3 TCP, but looking toward NFSv4)?

Quoted 8 lines

Assuming it's ok to use SMP on a heavy use NFS server, should I enable or disable hyperthreading? I'd assume that for NFS hyperthreading would be good, but that's only a guess.

Thanks,

Chris

Are you referring to the dcache problems with XFS? I think it was only a problem when the NFS server was exporting an XFS file system. I also haven't checked in some time, but I believe those problems have since been fixed.

Neil

Quoted 9 lines

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Signature · 18 lines
/***************************************************
 *Neil Horman
 *Software Engineer
 *Red Hat, Inc.
 *[email protected]
 *gpg keyid: 1024D / 0x92A74FA1
 *http://pgp.mit.edu
 ***************************************************/


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
#2 lore raw
Quoted 13 lines · Neil Horman
Quoted 12 lines · Chris Penney

I've heard from some (via google searchs) that there are issues with NFS + SMP when using the 2.6 kernel. Assuming I'm using 2.6.11.5 w/ the NFS-ALL patches applied should I be concerned about running SMP (dual cpu pentium 4 doing pretty much only NFSv3 TCP, but looking toward NFSv4)?

Assuming it's ok to use SMP on a heavy use NFS server, should I enable or disable hyperthreading? I'd assume that for NFS hyperthreading would be good, but that's only a guess.

Thanks,

Quoted 7 lines
Quoted 2 lines

Chris

Are you referring to the dcache problems with XFS? I think it was only a problem when the NFS server was exporting an XFS file system. I also haven't checked in some time, but I believe those problems have since been fixed.

Neil

Yes, most of what I read was regarding XFS. I was not sure if there were any issues outside of that (I'm using JFS right now).

As far as HyperThreading my testing so far has shown that under heavy loads enabling HyperThreading on a dual-cpu P4 box seems to be a good thing. I would grab 30s of packets with ethereal and look at the NFS stats and the responce time on writes/commits (which we see a fair bit of) were at least 33% faster if not more. I'm not sure if that's an accurate way to measure though -- I know enough to be dangerous, but I'm not an expert.

Chris

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#3 lore raw
Quoted 16 lines · Chris Penney
Quoted 15 lines · Neil Horman
Quoted 14 lines · Chris Penney

I've heard from some (via google searchs) that there are issues with NFS + SMP when using the 2.6 kernel. Assuming I'm using 2.6.11.5 w/ the NFS-ALL patches applied should I be concerned about running SMP (dual cpu pentium 4 doing pretty much only NFSv3 TCP, but looking toward NFSv4)?

Assuming it's ok to use SMP on a heavy use NFS server, should I enable or disable hyperthreading? I'd assume that for NFS hyperthreading would be good, but that's only a guess.

Thanks,

Chris

Quoted 9 lines
Quoted 5 lines

Are you referring to the dcache problems with XFS? I think it was only a problem when the NFS server was exporting an XFS file system. I also haven't checked in some time, but I believe those problems have since been fixed.

Neil

Yes, most of what I read was regarding XFS. I was not sure if there were any issues outside of that (I'm using JFS right now).

None that I'm aware of, but my knoweldge is far from exhaustive :)

Quoted 8 lines

As far as HyperThreading my testing so far has shown that under heavy loads enabling HyperThreading on a dual-cpu P4 box seems to be a good thing. I would grab 30s of packets with ethereal and look at the NFS stats and the responce time on writes/commits (which we see a fair bit of) were at least 33% faster if not more. I'm not sure if that's an accurate way to measure though -- I know enough to be dangerous, but I'm not an expert.

Neither am I :). You testing seems reasonable to me (at least for what you are trying to measure currently). IIRC the Linux scheduler is built to take advantage of the shared resources available on hyperthreaded processors and schedules accordingly. In general I think its a good idea to turn on. I believe there are some pessimal workloads that can cause worse performance with HT on, than with HT off, but I believe you really have to work hard to produce those.

Regards
Neil

Quoted 11 lines

Chris

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Signature · 18 lines
/***************************************************
 *Neil Horman
 *Software Engineer
 *Red Hat, Inc.
 *[email protected]
 *gpg keyid: 1024D / 0x92A74FA1
 *http://pgp.mit.edu
 ***************************************************/


-------------------------------------------------------
This SF.net email is sponsored by Demarc:
A global provider of Threat Management Solutions.
Download our HomeAdmin security software for free today!
http://www.demarc.com/Info/Sentarus/hamr30
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
#4 lore raw

Hi,

I don't know if you originally referred to some of my earlier posts here and on LKML regarding instability on 2.6 (2.6.8.1) with SMP...

There where, from my experience two separate problems:

The dcache problems which was fixed in 2.6.10.

One or more issues producing Oops's on SMP machines. I am not sure if they where NFS bound (actually I dont think they where, but I only saw them on one of our high-load NFS servers). I found out that running the system in UP mode made the server stable.

Back when I had the problems, I tested on 2.6.8.1 and haven't dared to try SMP since, so my knowledge is a bit ageing. But if you decide you are up for it, I would be very interested in the results! ....SMP would be nice on those XEON boxes.

//Saaby

Quoted 2 lines · Neil Horman
Quoted 1 line · Chris Penney

On Tue, 29 Mar 2005 16:13:24 -0500, Neil Horman <[email protected]>

Quoted 57 lines · wrote:
Quoted 20 lines
Quoted 20 lines · Chris Penney
Quoted 6 lines

I've heard from some (via google searchs) that there are issues with NFS + SMP when using the 2.6 kernel. Assuming I'm using 2.6.11.5 w/ the NFS-ALL patches applied should I be concerned about running SMP (dual cpu pentium 4 doing pretty much only NFSv3 TCP, but looking toward NFSv4)?

Quoted 7 lines

Assuming it's ok to use SMP on a heavy use NFS server, should I enable or disable hyperthreading? I'd assume that for NFS hyperthreading would be good, but that's only a guess.

Thanks,

Chris

Are you referring to the dcache problems with XFS? I think it was only a problem when the NFS server was exporting an XFS file system. I also haven't checked in some time, but I believe those problems have since been fixed.

Quoted 4 lines
Quoted 1 line

Neil

Yes, most of what I read was regarding XFS. I was not sure if there were any issues outside of that (I'm using JFS right now).

None that I'm aware of, but my knoweldge is far from exhaustive :)

Quoted 7 lines

As far as HyperThreading my testing so far has shown that under heavy loads enabling HyperThreading on a dual-cpu P4 box seems to be a good thing. I would grab 30s of packets with ethereal and look at the NFS stats and the responce time on writes/commits (which we see a fair bit of) were at least 33% faster if not more. I'm not sure if that's an accurate way to measure though -- I know enough to be dangerous, but I'm not an expert.

Neither am I :). You testing seems reasonable to me (at least for what you are trying to measure currently). IIRC the Linux scheduler is built to take advantage of the shared resources available on hyperthreaded processors and schedules accordingly. In general I think its a good idea to turn on. I believe there are some pessimal workloads that can cause worse performance with HT on, than with HT off, but I believe you really have to work hard to produce those.

Regards
Neil

Quoted 11 lines

Chris

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Signature · 19 lines
Med venlig hilsen - Best regards - Meilleures salutations

Anders Saaby
Systems Engineer
------------------------------------------------
Cohaesio A/S - Maglebjergvej 5D - DK-2800 Lyngby
Phone: +45 45 880 888 - Fax: +45 45 880 777
Mail: [email protected] - http://www.cohaesio.com
------------------------------------------------


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs