spnfs write performance issue

DISCUSSION Fu Liankun 3 participants 10 messages Feb 22, 2012 – Feb 28, 2012

Subsystems (2)

NFS client NFS server

lore

Share & tools

lore

Fu Liankun

author

2012-02-22 10:17 UTC #0 lore raw

When copy a big file(about 500MB) to nfs server using pnfs, it cost obvious longer time as compared with NFSv3 or NFSv4 protocol.

PNFS cost about 300-550s
NFSv3 cost about 49s
NFSv4 cost about 49s

My test environment as following:

MDS: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.100, ipaddr2:192.168.1.100
DS1: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.101, ipaddr2:192.168.1.101
DS2: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.102, ipaddr2:192.168.1.102
client: RHEL6.2GA(2.6.32-220.el6.i686) ipaddr1: 192.168.0.19,
ipaddr2:192.168.1.19

192.168.1.100 <-------- NFS4.1 connection --------->192.168.1.101 192.168.0.100 <-------- NFS4.1 connection --------->192.168.0.102 192.168.0.19 <-------- NFS4.1 connection --------->192.168.0.100

Dunring the write process, the client output the message:
FS-Cache: Loaded
FS-Cache: Netfs 'nfs' registered for caching
nfs4filelayout_init: NFSv4 File Layout Driver Registering...
nfs: server 192.168.1.101 not responding, timed out

The message "nfs: server 192.168.1.101 not responding, timed out" was output during the COMMIT request.

Who can tell me the reason why pnfs cost so long time.

J. Bruce Fields

2012-02-22 17:25 UTC #1 lore raw

Quoted 7 lines · Fu Liankun

When copy a big file(about 500MB) to nfs server using pnfs, it cost obvious longer time as compared with NFSv3 or NFSv4 protocol.
PNFS cost about 300-550s
NFSv3 cost about 49s
NFSv4 cost about 49s

Quoted 6 lines

My test environment as following:
MDS: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.100, ipaddr2:192.168.1.100
DS1: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.101, ipaddr2:192.168.1.101
DS2: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.102, ipaddr2:192.168.1.102
client: RHEL6.2GA(2.6.32-220.el6.i686) ipaddr1: 192.168.0.19,

What kind of server are you using?

--b.

Quoted 23 lines

ipaddr2:192.168.1.19
192.168.1.100 <-------- NFS4.1 connection --------->192.168.1.101 192.168.0.100 <-------- NFS4.1 connection --------->192.168.0.102 192.168.0.19 <-------- NFS4.1 connection --------->192.168.0.100
Dunring the write process, the client output the message:
FS-Cache: Loaded
FS-Cache: Netfs 'nfs' registered for caching
nfs4filelayout_init: NFSv4 File Layout Driver Registering...
nfs: server 192.168.1.101 not responding, timed out
The message "nfs: server 192.168.1.101 not responding, timed out" was output during the COMMIT request.
Who can tell me the reason why pnfs cost so long time.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Fu Liankun

author

2012-02-27 07:00 UTC #2 lore raw

Quoted 8 lines · J. Bruce Fields

Quoted 7 lines · Fu Liankun
When copy a big file(about 500MB) to nfs server using pnfs, it cost obvious longer time as compared with NFSv3 or NFSv4 protocol.
PNFS cost about 300-550s
NFSv3 cost about 49s
NFSv4 cost about 49s

Quoted 8 lines

Quoted 6 lines
My test environment as following:
MDS: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.100, ipaddr2:192.168.1.100
DS1: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.101, ipaddr2:192.168.1.101
DS2: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.102, ipaddr2:192.168.1.102
client: RHEL6.2GA(2.6.32-220.el6.i686) ipaddr1: 192.168.0.19,
What kind of server are you using?

Sorry for late response.

Fedora14 + 3.1.0-rc8-pnfs

--b

Quoted 31 lines

--b.
Quoted 15 lines
ipaddr2:192.168.1.19
192.168.1.100 <-------- NFS4.1 connection --------->192.168.1.101 192.168.0.100 <-------- NFS4.1 connection --------->192.168.0.102 192.168.0.19 <-------- NFS4.1 connection --------->192.168.0.100
Dunring the write process, the client output the message:
FS-Cache: Loaded
FS-Cache: Netfs 'nfs' registered for caching
nfs4filelayout_init: NFSv4 File Layout Driver Registering...
nfs: server 192.168.1.101 not responding, timed out
The message "nfs: server 192.168.1.101 not responding, timed out" was output during the COMMIT request.
Quoted 8 lines
Who can tell me the reason why pnfs cost so long time.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

J. Bruce Fields

2012-02-27 12:01 UTC #3 lore raw

Quoted 16 lines · Fu Liankun

Quoted 15 lines · J. Bruce Fields
Quoted 13 lines · Fu Liankun
When copy a big file(about 500MB) to nfs server using pnfs, it cost obvious longer time as compared with NFSv3 or NFSv4 protocol.
PNFS cost about 300-550s
NFSv3 cost about 49s
NFSv4 cost about 49s
My test environment as following:
MDS: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.100, ipaddr2:192.168.1.100
DS1: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.101, ipaddr2:192.168.1.101
DS2: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.102, ipaddr2:192.168.1.102
client: RHEL6.2GA(2.6.32-220.el6.i686) ipaddr1: 192.168.0.19,

Quoted 5 lines

Quoted 1 line
What kind of server are you using?
Sorry for late response.
Fedora14 + 3.1.0-rc8-pnfs

And you're exporting gfs2?

--b.

Quoted 35 lines

--b
Quoted 31 lines
--b.
Quoted 15 lines
ipaddr2:192.168.1.19
192.168.1.100 <-------- NFS4.1 connection --------->192.168.1.101 192.168.0.100 <-------- NFS4.1 connection --------->192.168.0.102 192.168.0.19 <-------- NFS4.1 connection --------->192.168.0.100
Dunring the write process, the client output the message:
FS-Cache: Loaded
FS-Cache: Netfs 'nfs' registered for caching
nfs4filelayout_init: NFSv4 File Layout Driver Registering...
nfs: server 192.168.1.101 not responding, timed out
The message "nfs: server 192.168.1.101 not responding, timed out" was output during the COMMIT request.
Quoted 8 lines
Who can tell me the reason why pnfs cost so long time.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Fu Liankun

author

2012-02-28 01:33 UTC #4 lore raw

Quoted 16 lines · J. Bruce Fields

Quoted 15 lines · Fu Liankun
Quoted 14 lines · J. Bruce Fields
Quoted 13 lines · Fu Liankun
When copy a big file(about 500MB) to nfs server using pnfs, it cost obvious longer time as compared with NFSv3 or NFSv4 protocol.
PNFS cost about 300-550s
NFSv3 cost about 49s
NFSv4 cost about 49s
My test environment as following:
MDS: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.100, ipaddr2:192.168.1.100
DS1: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.101, ipaddr2:192.168.1.101
DS2: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.102, ipaddr2:192.168.1.102
client: RHEL6.2GA(2.6.32-220.el6.i686) ipaddr1: 192.168.0.19,

Quoted 6 lines

Quoted 4 lines
Quoted 1 line
What kind of server are you using?
Sorry for late response.
Fedora14 + 3.1.0-rc8-pnfs
And you're exporting gfs2?

No. I'm exporting ext4.
Does the file system type would influence the result?

--b

Quoted 40 lines

--b.
Quoted 32 lines
--b
Quoted 30 lines
--b.
Quoted 15 lines
ipaddr2:192.168.1.19
192.168.1.100 <-------- NFS4.1 connection --------->192.168.1.101 192.168.0.100 <-------- NFS4.1 connection --------->192.168.0.102 192.168.0.19 <-------- NFS4.1 connection --------->192.168.0.100
Dunring the write process, the client output the message:
FS-Cache: Loaded
FS-Cache: Netfs 'nfs' registered for caching
nfs4filelayout_init: NFSv4 File Layout Driver Registering...
nfs: server 192.168.1.101 not responding, timed out
The message "nfs: server 192.168.1.101 not responding, timed out" was output during the COMMIT request.
Quoted 8 lines
Who can tell me the reason why pnfs cost so long time.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 3 lines

----
Best wishes
Fu Liankun

J. Bruce Fields

2012-02-28 01:46 UTC #5 lore raw

Quoted 18 lines · Fu Liankun

Quoted 17 lines · J. Bruce Fields
Quoted 16 lines · Fu Liankun
Quoted 15 lines · J. Bruce Fields
Quoted 7 lines · Fu Liankun
When copy a big file(about 500MB) to nfs server using pnfs, it cost obvious longer time as compared with NFSv3 or NFSv4 protocol.
PNFS cost about 300-550s
NFSv3 cost about 49s
NFSv4 cost about 49s
Quoted 6 lines
My test environment as following:
MDS: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.100, ipaddr2:192.168.1.100
DS1: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.101, ipaddr2:192.168.1.101
DS2: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.102, ipaddr2:192.168.1.102
client: RHEL6.2GA(2.6.32-220.el6.i686) ipaddr1: 192.168.0.19,
What kind of server are you using?

Quoted 8 lines

Quoted 5 lines
Quoted 3 lines
Sorry for late response.
Fedora14 + 3.1.0-rc8-pnfs
And you're exporting gfs2?
No. I'm exporting ext4.
Does the file system type would influence the result?

So you're using local-pnfs? Or spnfs? What setup instructions did you follow?

--b.

Fu Liankun

author

2012-02-28 01:49 UTC #6 lore raw

Quoted 21 lines · J. Bruce Fields

Quoted 20 lines · Fu Liankun
Quoted 19 lines · J. Bruce Fields
Quoted 18 lines · Fu Liankun
Quoted 8 lines · J. Bruce Fields
Quoted 7 lines · Fu Liankun
When copy a big file(about 500MB) to nfs server using pnfs, it cost obvious longer time as compared with NFSv3 or NFSv4 protocol.
PNFS cost about 300-550s
NFSv3 cost about 49s
NFSv4 cost about 49s
Quoted 7 lines
Quoted 6 lines
My test environment as following:
MDS: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.100, ipaddr2:192.168.1.100
DS1: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.101, ipaddr2:192.168.1.101
DS2: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.102, ipaddr2:192.168.1.102
client: RHEL6.2GA(2.6.32-220.el6.i686) ipaddr1: 192.168.0.19,
What kind of server are you using?
Sorry for late response.

Quoted 7 lines

Quoted 4 lines
Quoted 2 lines
Quoted 1 line
Fedora14 + 3.1.0-rc8-pnfs
And you're exporting gfs2?
No. I'm exporting ext4.
Does the file system type would influence the result?
So you're using local-pnfs? Or spnfs? What setup instructions did you follow?

I'm using spnfs.

--b

Quoted 3 lines

--b.

J. Bruce Fields

2012-02-28 01:53 UTC #7 lore raw

Quoted 23 lines · Fu Liankun

Quoted 22 lines · J. Bruce Fields
Quoted 21 lines · Fu Liankun
Quoted 20 lines · J. Bruce Fields
Quoted 19 lines · Fu Liankun
Quoted 8 lines · J. Bruce Fields
Quoted 7 lines · Fu Liankun
When copy a big file(about 500MB) to nfs server using pnfs, it cost obvious longer time as compared with NFSv3 or NFSv4 protocol.
PNFS cost about 300-550s
NFSv3 cost about 49s
NFSv4 cost about 49s
Quoted 7 lines
Quoted 6 lines
My test environment as following:
MDS: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.100, ipaddr2:192.168.1.100
DS1: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.101, ipaddr2:192.168.1.101
DS2: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.102, ipaddr2:192.168.1.102
client: RHEL6.2GA(2.6.32-220.el6.i686) ipaddr1: 192.168.0.19,
What kind of server are you using?
Sorry for late response.
Fedora14 + 3.1.0-rc8-pnfs

Quoted 8 lines

Quoted 6 lines
Quoted 3 lines
Quoted 1 line
And you're exporting gfs2?
No. I'm exporting ext4.
Does the file system type would influence the result?
So you're using local-pnfs? Or spnfs? What setup instructions did you follow?
I'm using spnfs.

OK, got it.

I'm not sure if anyone here can talk about the performance of spnfs. It's not really maintained as far as I know.

So, you're the expert.

--b.

Fu Liankun

author

2012-02-28 02:15 UTC #8 lore raw

Quoted 27 lines · J. Bruce Fields

Quoted 26 lines · Fu Liankun
Quoted 25 lines · J. Bruce Fields
Quoted 24 lines · Fu Liankun
Quoted 16 lines · J. Bruce Fields
Quoted 15 lines · Fu Liankun
Quoted 14 lines · J. Bruce Fields
Quoted 13 lines · Fu Liankun
When copy a big file(about 500MB) to nfs server using pnfs, it cost obvious longer time as compared with NFSv3 or NFSv4 protocol.
PNFS cost about 300-550s
NFSv3 cost about 49s
NFSv4 cost about 49s
My test environment as following:
MDS: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.100, ipaddr2:192.168.1.100
DS1: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.101, ipaddr2:192.168.1.101
DS2: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.102, ipaddr2:192.168.1.102
client: RHEL6.2GA(2.6.32-220.el6.i686) ipaddr1: 192.168.0.19,
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
What kind of server are you using?
Sorry for late response.
Fedora14 + 3.1.0-rc8-pnfs
And you're exporting gfs2?
No. I'm exporting ext4.
Does the file system type would influence the result?

Quoted 9 lines

Quoted 3 lines
Quoted 2 lines
So you're using local-pnfs? Or spnfs? What setup instructions did you follow?
I'm using spnfs.
OK, got it.
I'm not sure if anyone here can talk about the performance of spnfs. It's not really maintained as far as I know.

Got it.
I will keep on investigating the issue.

--b

Quoted 2 lines

So, you're the expert.

Signature · 3 lines

----
Best wishes
Fu Liankun

Benny Halevy

2012-02-28 09:27 UTC #9 lore raw

In reply to J. Bruce Fields

Quoted 31 lines · J. Bruce Fields

Quoted 30 lines · Fu Liankun
Quoted 28 lines · J. Bruce Fields
Quoted 17 lines · Fu Liankun
Quoted 16 lines · J. Bruce Fields
Quoted 15 lines · Fu Liankun
Quoted 14 lines · J. Bruce Fields
Quoted 13 lines · Fu Liankun
When copy a big file(about 500MB) to nfs server using pnfs, it cost obvious longer time as compared with NFSv3 or NFSv4 protocol.
PNFS cost about 300-550s
NFSv3 cost about 49s
NFSv4 cost about 49s
My test environment as following:
MDS: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.100, ipaddr2:192.168.1.100
DS1: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.101, ipaddr2:192.168.1.101
DS2: 3.1.0-rc8-pnfs ipaddr1: 192.168.0.102, ipaddr2:192.168.1.102
client: RHEL6.2GA(2.6.32-220.el6.i686) ipaddr1: 192.168.0.19,
Quoted 7 lines
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
What kind of server are you using?
Sorry for late response.
Fedora14 + 3.1.0-rc8-pnfs
And you're exporting gfs2?
No. I'm exporting ext4.
Does the file system type would influence the result?
So you're using local-pnfs? Or spnfs? What setup instructions did you follow?

Quoted 6 lines

Quoted 1 line
I'm using spnfs.
OK, got it.
I'm not sure if anyone here can talk about the performance of spnfs. It's not really maintained as far as I know.

True. spnfs is unsupported and not maintained by anybody. Sorry.

Benny

Quoted 8 lines

So, you're the expert.
--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html