Skip to contentSkip to conversation

mmap over nfs leads to excessive system load

DISCUSSION Kenny Simpson 1 participant 1 message Nov 8, 2005

lore
Share & tools
lore
#0 lore raw

I seem to have hit the same issue as Fabio Brugnara
http://www.ussg.iu.edu/hypermail/linux/kernel/0505.0/1061.html
but although he said the problem went away for him in 2.6.11, I saw the problem in 2.6.8, and still see it in 2.6.14.

When using a sliding mmap window over a file on nfs, the system time increases with time, and the throughput drops.
Using the same program on a local file I get about 30MB/s with very little system time... just a bunch of iowait.

I am using a gigabit connection to a netapp, and initially get 30+MB/s, but after about 30 seconds, this decays to only about 8MB/s with the system time close to 80%.

I did a quick Oprofile run, and here is the top of the list:

1739810 16.8926 vmlinux-2.6.14 vmlinux-2.6.14
find_get_pages_tag
1305104 12.6718 vmlinux-2.6.14 vmlinux-2.6.14
mpage_writepages

1041147  10.1089  vmlinux-2.6.14           vmlinux-2.6.14           unlock_page
998692    9.6967  vmlinux-2.6.14           vmlinux-2.6.14          

clear_page_dirty_for_io
966458 9.3838 vmlinux-2.6.14 vmlinux-2.6.14
release_pages
674434 6.5484 vmlinux-2.6.14 vmlinux-2.6.14
pci_conf1_write
486345 4.7221 vmlinux-2.6.14 vmlinux-2.6.14
__lookup_tag
399627 3.8801 vmlinux-2.6.14 vmlinux-2.6.14
page_waitqueue
134250 1.3035 vmlinux-2.6.14 vmlinux-2.6.14
_spin_lock_irqsave
130594 1.2680 vmlinux-2.6.14 vmlinux-2.6.14
_read_lock_irqsave

I'm attaching the config.
Other details:
Dual Xeon 2.66 w/ 2GB RAM
Debian sarge + 2.6.14 kernel + glibc 2.3.5
nvidia drivers
broadcom gigabit driver (8.2.18)

thanks,
-Kenny

__________________________________
Yahoo! FareChase: Search multiple travel sites in one click.
http://farechase.yahoo.com