Skip to contentSkip to conversation

CIFS is slooow, gvfs is fast

DISCUSSION J.A. Magallón 3 participants 8 messages Oct 21, 2010 – Oct 30, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi all...

I have a very strange behaviour with windows networking.
I have an HTPC, which I access from my Linux desktop. It runs Windows 7. Network is GigaBit, routed by a Linksys WRT320N running latest DD-WRT. I setup an automount path under /net/htpc with CIFS filesystem. I can also access the drives in HTPC via gvfs from Gnome DE.

Problem: speed is _VERY_ different.
R/W through the automount point (CIFS fs), is roughly 7-8 MB/s (ridiculous for gigabit speeds).
R/W through Gnome/gvfs (kind of fuse) goes around 35 - 40 MB/s (400 Mbits/s), that look much better.

I have tried with different rsize/wsize in mount, but speed is the same (perhaps I did not get the right values...).

BTW, I'm not alone:

http://ubuntuforums.org/showthread.php?t=1578068
http://ubuntuforums.org/showthread.php?t=1429532
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/471512

but I have not seen any diagnostic anywere.

Any ideas ?

Signature · 2 lines
J.A. Magallon <jamagallon()ono!com>     \               Software is like sex:
                                         \         It's better when it's free
#1 lore raw

(Cc [email protected])

Quoted 11 lines · J.A. Magallón

I have a very strange behaviour with windows networking.
I have an HTPC, which I access from my Linux desktop. It runs Windows 7. Network is GigaBit, routed by a Linksys WRT320N running latest DD-WRT. I setup an automount path under /net/htpc with CIFS filesystem. I can also access the drives in HTPC via gvfs from Gnome DE.

Problem: speed is _VERY_ different.
R/W through the automount point (CIFS fs), is roughly 7-8 MB/s (ridiculous for gigabit speeds).
R/W through Gnome/gvfs (kind of fuse) goes around 35 - 40 MB/s (400 Mbits/s), that look much better.

What is the kernel version In which you are seeing the problem? I'm not seeing this on 2.6.36-rc4. Do you have a wired interface as well? Are you sure that both are using the same network path?

Quoted 2 lines

I have tried with different rsize/wsize in mount, but speed is the same (perhaps I did not get the right values...).

Does the changes reflect in /proc/mounts?

Signature · 1 line
Suresh Jayaraman
#2 lore raw
Quoted 10 lines · Suresh Jayaraman

(Cc [email protected])

Quoted 5 lines · J.A. Magallón

Problem: speed is _VERY_ different.
R/W through the automount point (CIFS fs), is roughly 7-8 MB/s (ridiculous for gigabit speeds).
R/W through Gnome/gvfs (kind of fuse) goes around 35 - 40 MB/s (400 Mbits/s), that look much better.

Quoted 7 lines

What is the kernel version In which you are seeing the problem? I'm not seeing this on 2.6.36-rc4. Do you have a wired interface as well? Are you sure that both are using the same network path?

Quoted 2 lines

I have tried with different rsize/wsize in mount, but speed is the same (perhaps I did not get the right values...).

Read sizes for cifs are relatively small at 16K (unless you change the maximum buffer sizes at insmod time), and cifs write size is 56K. This relatively smaller read size and the serializing of relatively small reads in cifs_readpages can cause slower speeds than you might expect reading files over gigabit if only one process is using the network especially if long network latency (lots of dead time on the network leading to low network utilizations).   cifs will perform relatively well compared to other options as the number of processes accessing the network increases, because a lot of work has gone into locking.  dbench results will often show 50 simultaneous requests to each Samba server (50 is the traditional maximum number of simultaneous requests that smb/cifs servers support).   For workloads other than sequential read workloads - cifs does relatively better as well.

In the future with smb2 the maximum number of requests can vary, and the maximum read size is larger which helps, as does the parallelized smb2 write code which Pavel wrote over the summer for Google Summer of Code.

--
Thanks,

Steve

#3 lore raw
In reply to Suresh Jayaraman
Quoted 10 lines · Suresh Jayaraman

(Cc [email protected])

Quoted 6 lines · J.A. Magallón

I have a very strange behaviour with windows networking.
I have an HTPC, which I access from my Linux desktop. It runs Windows 7. Network is GigaBit, routed by a Linksys WRT320N running latest DD-WRT. I setup an automount path under /net/htpc with CIFS filesystem. I can also access the drives in HTPC via gvfs from Gnome DE.

Quoted 7 lines
Quoted 5 lines

Problem: speed is _VERY_ different.
R/W through the automount point (CIFS fs), is roughly 7-8 MB/s (ridiculous for gigabit speeds).
R/W through Gnome/gvfs (kind of fuse) goes around 35 - 40 MB/s (400 Mbits/s), that look much better.

What is the kernel version In which you are seeing the problem? I'm not

Its just new 2.6.36 final, mandriva distro kernel.

Quoted 1 line

seeing this on 2.6.36-rc4. Do you have a wired interface as well? Are

Yes, boxes are all wired to the 320N router, at gigabit speeds. Just to double check, I also probed with my netbook with 100Mb ehternet. The effect is less obvious, but writing to the Win7 box through the cifs automount path goes at 3.5-4.0 Mb/s, and with gvfs it reaches 7.5 Mb/s.

Quoted 2 lines

you sure that both are using the same network path?

If you mean hardware network path (cabling, router), yes. For file paths, CIFS fs is under /net/htpc/media, and gvfs uses a mount point on '~/.gvfs/media on htpc'.

And here comes the weird part. To rule out the desktop environment factor, I tried a dd in both paths. On /net, I get 16.6 MB/s, and on ~/.gvfs/xxx, I just get 5.5 MB/s. I really don't understand anything... things get reversed depending if you use a gui or commandline.

Weird, really weird...

Signature · 2 lines
J.A. Magallon <jamagallon()ono!com>     \               Software is like sex:
                                         \         It's better when it's free
#4 lore raw
Quoted 37 lines · J.A. Magallón
Quoted 10 lines · Suresh Jayaraman

(Cc [email protected])

Quoted 6 lines · J.A. Magallón

I have a very strange behaviour with windows networking.
I have an HTPC, which I access from my Linux desktop. It runs Windows 7. Network is GigaBit, routed by a Linksys WRT320N running latest DD-WRT. I setup an automount path under /net/htpc with CIFS filesystem. I can also access the drives in HTPC via gvfs from Gnome DE.

Quoted 7 lines
Quoted 5 lines

Problem: speed is _VERY_ different.
R/W through the automount point (CIFS fs), is roughly 7-8 MB/s (ridiculous for gigabit speeds).
R/W through Gnome/gvfs (kind of fuse) goes around 35 - 40 MB/s (400 Mbits/s), that look much better.

What is the kernel version In which you are seeing the problem? I'm not

Its just new 2.6.36 final, mandriva distro kernel.

Quoted 1 line

seeing this on 2.6.36-rc4. Do you have a wired interface as well? Are

Yes, boxes are all wired to the 320N router, at gigabit speeds. Just to double check, I also probed with my netbook with 100Mb ehternet. The effect is less obvious, but writing to the Win7 box through the cifs automount path goes at 3.5-4.0 Mb/s, and with gvfs it reaches 7.5 Mb/s.

Quoted 2 lines

you sure that both are using the same network path?

If you mean hardware network path (cabling, router), yes. For file paths, CIFS fs is under /net/htpc/media, and gvfs uses a mount point on '~/.gvfs/media on htpc'.

Quoted 7 lines

And here comes the weird part. To rule out the desktop environment factor, I tried a dd in both paths. On /net, I get 16.6 MB/s, and on ~/.gvfs/xxx, I just get 5.5 MB/s. I really don't understand anything... things get reversed depending if you use a gui or commandline.

Weird, really weird...

Well, to sort things out, I did some more tests (this does not mean things get any closer to be clear for me at least...).

Network card state in linux side:

werewolf:/etc/autofs# ethtool eth0
Settings for eth0:
    Supported ports: [ TP ]
    Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Supports auto-negotiation: Yes
    Advertised link modes:  10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Advertised auto-negotiation: Yes
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: on
    Supports Wake-on: pg
    Wake-on: d
    Current message level: 0x00000000 (0)
    Link detected: yes

Raw iperf behaviour:

lnx -> win: [  3]  0.0-10.0 sec    988 MBytes    828 Mbits/sec
win -> lnx: [  4]  0.0-10.0 sec    333 MBytes    279 Mbits/sec

Don't know why the difference, but anyways worst speed is 280 Mb/s.

auto.master:
/net/htpc file:/etc/autofs/auto.htpc
auto.htpc:

	media -fstype=cifs,credentials=/etc/autofs/credentials/htpc,iocharset=utf8,uid=3001,gid=3000 ://htpc/media

Times for gnome-ui-copy are approximate, speed is what the copy window says.

Case A: automount via cifs.
/proc/mounts:

	/etc/autofs/auto.htpc /net/htpc autofs rw,relatime,fd=6,pgrp=14361,timeout=300,minproto=5,maxproto=5,indirect 0 0
	//htpc/media/ /net/htpc/media cifs rw,relatime,unc=\\htpc\media,username=media,uid=0,noforceuid,gid=0,noforcegid,addr=192.168.1.20,file_mode=0755,dir_mode=0755,serverino,rsize=130048,wsize=57344 0 0

- dd if=~/file of=file.dd bs=1M
292538368 bytes (293 MB) copied, 8.44771 s, 34.6 MB/s
- time cp ~/file file.cp: real 0m7.429s
- gnome UI copy: 39s, 7.8 MB/s

Case B: network mount via nautilus.
/proc/mounts:

	/etc/autofs/auto.htpc /net/htpc autofs rw,relatime,fd=6,pgrp=14361,timeout=300,minproto=5,maxproto=5,indirect 0 0
	gvfs-fuse-daemon /home/magallon/.gvfs fuse.gvfs-fuse-daemon rw,nosuid,nodev,relatime,user_id=3001,group_id=3000 0 0

- dd if=~/file of=file.dd bs=1M
292538368 bytes (293 MB) copied, 50.4364 s, 5.8 MB/s
- time cp ~/file file.cp: real 0m50.151s
- gnome UI copy: 8s, 33.2 MB/s

And the best of all:

werewolf:~> time smbclient -U media%XXXXX //htpc/media -c 'put file' Domain=[HOME] OS=[Windows 7 Enterprise 7600] Server=[Windows 7 Enterprise 6.1] putting file file as \file (66345.1 kb/s) (average 66345.1 kb/s)

real 0m4.318s
user 0m0.466s
sys 0m0.553s

Can anybody tell me whats going here...??
It looks just like gui tools behave just in reverse of cli, and that a plain smbclient is faster that cifs. I'm really confused...

Signature · 2 lines
J.A. Magallon <jamagallon()ono!com>     \               Software is like sex:
                                         \         It's better when it's free
#5 lore raw
Quoted 15 lines · J.A. Magallón
Quoted 5 lines · J.A. Magall�n

If you mean hardware network path (cabling, router), yes. For file paths, CIFS fs is under /net/htpc/media, and gvfs uses a mount point on '~/.gvfs/media on htpc'.

Quoted 7 lines

And here comes the weird part. To rule out the desktop environment factor, I tried a dd in both paths. On /net, I get 16.6 MB/s, and on ~/.gvfs/xxx, I just get 5.5 MB/s. I really don't understand anything... things get reversed depending if you use a gui or commandline.

Weird, really weird...

Quoted 7 lines

Well, to sort things out, I did some more tests (this does not mean things get any closer to be clear for me at least...).

Can anybody tell me whats going here...??
It looks just like gui tools behave just in reverse of cli, and that a plain smbclient is faster that cifs. I'm really confused...

Could you try mounting from commandline with something like
mount -t cifs //server/share /mnt/point -o user=user1
(commenting out the fstab entry) and try copying the file?

Signature · 1 line
Suresh Jayaraman
#6 lore raw
Quoted 19 lines · Suresh Jayaraman
Quoted 18 lines · J.A. Magallón
Quoted 5 lines · J.A. Magall�n

If you mean hardware network path (cabling, router), yes. For file paths, CIFS fs is under /net/htpc/media, and gvfs uses a mount point on '~/.gvfs/media on htpc'.

Quoted 7 lines

And here comes the weird part. To rule out the desktop environment factor, I tried a dd in both paths. On /net, I get 16.6 MB/s, and on ~/.gvfs/xxx, I just get 5.5 MB/s. I really don't understand anything... things get reversed depending if you use a gui or commandline. update
Weird, really weird...

Well, to sort things out, I did some more tests (this does not mean things get any closer to be clear for me at least...).

Quoted 10 lines
Quoted 4 lines

Can anybody tell me whats going here...??
It looks just like gui tools behave just in reverse of cli, and that a plain smbclient is faster that cifs. I'm really confused...

Could you try mounting from commandline with something like
mount -t cifs //server/share /mnt/point -o user=user1
(commenting out the fstab entry) and try copying the file?

Same results, cli copy takes about 7.5 seconds (thats around 40MB/s), and gui copy says it runs at 7.8 MB/s.
This could be the normal behaviour I could expect, gui is slower as it has to update progress bars, move flying sheets, or even does a sync from time to time.

The misbehaviour I see is the other way aroung, FUSE/GVFS is fast from GUI and slow from CLI. And, the best, why are there any difference ???

Signature · 2 lines
J.A. Magallon <jamagallon()ono!com>     \               Software is like sex:
                                         \         It's better when it's free
#7 lore raw
Quoted 32 lines · J.A. Magallón
Quoted 19 lines · Suresh Jayaraman
Quoted 18 lines · J.A. Magallón
Quoted 5 lines · J.A. Magall�n

If you mean hardware network path (cabling, router), yes. For file paths, CIFS fs is under /net/htpc/media, and gvfs uses a mount point on '~/.gvfs/media on htpc'.

Quoted 7 lines

And here comes the weird part. To rule out the desktop environment factor, I tried a dd in both paths. On /net, I get 16.6 MB/s, and on ~/.gvfs/xxx, I just get 5.5 MB/s. I really don't understand anything... things get reversed depending if you use a gui or commandline. update
Weird, really weird...

Well, to sort things out, I did some more tests (this does not mean things get any closer to be clear for me at least...).

Quoted 10 lines
Quoted 4 lines

Can anybody tell me whats going here...??
It looks just like gui tools behave just in reverse of cli, and that a plain smbclient is faster that cifs. I'm really confused...

Could you try mounting from commandline with something like mount -t cifs //server/share /mnt/point -o user=user1 (commenting out the fstab entry) and try copying the file?

Quoted 8 lines

Same results, cli copy takes about 7.5 seconds (thats around 40MB/s), and gui copy says it runs at 7.8 MB/s.
This could be the normal behaviour I could expect, gui is slower as it has to update progress bars, move flying sheets, or even does a sync from time to time.

The misbehaviour I see is the other way aroung, FUSE/GVFS is fast from GUI and slow from CLI. And, the best, why are there any difference ???

The obvious thing to check is actual cifs write sizes on the network (with wireshark e.g.)
and how much parallelism. We know cifs kernel client will send 56K
writes, but only one at
a time for any particular file. (thanks to Pavel the smb2 kernel client will send more
in parallel and get about two to three times better performance). cifs kernel
client can send less than 56K if the dirty pages are scattered of course (writepages won't be called with contiguous dirty pages) and cifs client can send more than 56K when mounting with "directio" cifs mount option but only if the application tries to write more than 56K for each user space write operation (mounting with directio might be a nice comparison point).

Signature · 3 lines
Thanks,

Steve