Skip to contentSkip to conversation

unresponsiveness on linux desktop during file copy

DISCUSSION Satish Eerpini 9 participants 33 messages May 15, 2009 – May 17, 2009 2 messages not in the archive

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

Hello everyone,

I am trying to figure out why the Linux desktop becomes overly unresponsive whenever there is a file copy going on, this happens to a large extent with GUI based file copying, and some lesser, but with command line copying too.
does it have anything to do with the kernel ??,
if it does help, then I am running Fedora 10 on a Hp nx 7400, Centrino Duo processor.

Cheers
Satish

Signature · 1 line
http://satish.playdrupal.com
#1 lore raw
Quoted 10 lines · Satish Eerpini

Hello everyone,

I am trying to figure out why the Linux desktop becomes overly unresponsive whenever there is a file copy going on, this happens to a large extent with GUI based file copying, and some lesser, but with command line copying too.
does it have anything to do with the kernel ??,
if it does help, then I am running Fedora 10 on a Hp nx 7400, Centrino Duo processor.

Quoted 3 lines

Cheers
Satish
--

Quickly, is the system getting good hard drive performance overall?

hdparm -tT /dev/sda

or whatever device you are using. Maybe it's stuck in some sort of programmed IO or something very old?

- Mark

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw
Quoted 6 lines

I presume that if you start your machine and don't start X at all, never run Gnome or whatever, that you get similar numbers in your console? Possibly the desktop is using a lot of CPU but I doubt it. I've never seen that. Every time I've had problems like this it's been a newish chipset and the driver for the hard drive interface isn't up to snuff yet.

yeah I get similar numbers even in the absence of the desktop manager , when I tried to copy from a virtual console.
the stats look something like this :

Linux 2.6.29.1 (satish) 05/16/2009

08:01:49 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
08:01:52 AM     all      3.68      0.00      6.72     41.60      0.00     48.00
08:01:55 AM     all      7.10      0.00     15.00     35.81      0.00     42.10
08:01:58 AM     all     10.45      0.00     22.03     51.29      0.00     16.24
08:02:01 AM     all      5.37      0.00      9.12     72.48      0.00     13.03
08:02:04 AM     all      7.25      0.00     12.40     37.04      0.00     43.32
Average:        all      6.77      0.00     13.06     47.58      0.00     32.59
Quoted 1 line

So, maybe something now about the actual hardware? lspci, lsmod, uname

uname -a :
Linux satish 2.6.29.1 #1 SMP Tue Apr 21 01:17:09 IST 2009 i686 i686 i386 GNU/Linux

lspci
00:00.0 Host bridge: Intel Corporation Mobile 945GM/PM/GMS, 943/940GML and 945GT Express Memory Controller Hub (rev 03)
00:02.0 VGA compatible controller: Intel Corporation Mobile 945GM/GMS, 943/940GML Express Integrated Graphics Controller (rev 03)
00:02.1 Display controller: Intel Corporation Mobile 945GM/GMS/GME, 943/940GML Express Integrated Graphics Controller (rev 03)
00:1b.0 Audio device: Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller (rev 01)
00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 01)
00:1c.1 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 2 (rev 01)
00:1c.3 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 4 (rev 01)
00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 (rev 01)
00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #2 (rev 01)
00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #3 (rev 01)
00:1d.3 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #4 (rev 01)
00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller (rev 01)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev e1) 00:1f.0 ISA bridge: Intel Corporation 82801GBM (ICH7-M) LPC Interface Bridge (rev 01)
00:1f.2 IDE interface: Intel Corporation 82801GBM/GHM (ICH7 Family) SATA IDE Controller (rev 01)
02:06.0 CardBus bridge: Texas Instruments PCIxx12 Cardbus Controller 02:06.1 FireWire (IEEE 1394): Texas Instruments PCIxx12 OHCI Compliant IEEE 1394 Host Controller
02:0e.0 Ethernet controller: Broadcom Corporation BCM4401-B0 100Base-TX (rev 02) 10:00.0 Network controller: Intel Corporation PRO/Wireless 3945ABG [Golan] Network Connection (rev 02)

(sorry , I did not know what to filter out)

Quoted 2 lines

-a. This is your main drive, correct? The driver is built in and not modular, or is it a second drive?

does it make a difference if the driver is built in or a module ??

Satish

Signature · 1 line
http://satish.playdrupal.com
#5 lore raw

On Fri, May 15, 2009 at 7:38 PM, Satish Eerpini <[email protected]> wrote:
<SNIP>

Quoted 3 lines

yeah I get similar numbers even in the absence of the desktop manager , when I tried to copy from  a virtual console.
the stats look something like this :

<SNIP>

Quoted 2 lines

00:1f.2 IDE interface: Intel Corporation 82801GBM/GHM (ICH7 Family) SATA IDE Controller (rev 01)

<SNIP>

I googled on "82801GBM/GHM slow" and found a lot to read.

Is DMA for this drive enabled? It seems that some others didn't have it enabled by default, or maybe at all. I didn't real the threads very far.

hdparm -d1 /dev/sda

I think hdparm works for most SATA drivers. man sata to see the other things you can check in terms of configuration.

Clearly, since you see the same thing in a console with X never having been started says it's not a GUI issue.

Hope this helps,
Mark

#6 lore raw
Quoted 1 line

I googled on "82801GBM/GHM slow" and found a lot to read.

will read through that ,...

Quoted 3 lines

Is DMA for this drive enabled? It seems that some others didn't have it enabled by default, or maybe at all. I didn't real the threads very far.

the udma5 mode seems to be enabled , though I can't really make any sense out of it:

hdparm -i /dev/sda

/dev/sda:

Model=TOSHIBA MK1234GSX , FwRev=AH001H ,
SerialNo= 372HFDVGS
Config={ Fixed }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4 BuffType=unknown, BuffSize=0kB, MaxMultSect=16, MultSect=?16? CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=234441648 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120} PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: sdma0 sdma1 sdma2 mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5 AdvancedPM=yes: unknown setting WriteCache=enabled Drive conforms to: Unspecified: ATA/ATAPI-3,4,5,6,7

* signifies the current active mode

Quoted 1 line

hdparm -d1 /dev/sda

got the following error when I tried setting the mode as above, I think along with -d1 we need to give some "X" argument( or so reads the man page)..

hdparm -d1 /dev/sda

/dev/sda:
setting using_dma to 1 (on)
HDIO_SET_DMA failed: Inappropriate ioctl for device HDIO_GET_DMA failed: Inappropriate ioctl for device

Quoted 2 lines

Clearly, since you see the same thing in a console with X never having been started says it's not a GUI issue.

yes I suppose, but any idea how I figure out what exactly is causing the IOWAIT, .. and if there are any patches, let me know, I can test them .

Cheers
Satish

Signature · 1 line
http://satish.playdrupal.com
#7 lore raw
Quoted 46 lines · Satish Eerpini
Quoted 1 line

I googled on "82801GBM/GHM slow" and found a lot to read.

will read through that ,...

Quoted 3 lines

Is DMA for this drive enabled? It seems that some others didn't have it enabled by default, or maybe at all. I didn't real the threads very far.

the udma5 mode seems to be enabled , though I can't really make any sense out of it:

hdparm -i /dev/sda

/dev/sda:

 Model=TOSHIBA MK1234GSX                       , FwRev=AH001H  , SerialNo=           372HFDVGS
 Config={ Fixed }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4 BuffType=unknown, BuffSize=0kB, MaxMultSect=16, MultSect=?16? CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=234441648 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120} PIO modes:  pio0 pio1 pio2 pio3 pio4
 DMA modes:  sdma0 sdma1 sdma2 mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5 AdvancedPM=yes: unknown setting WriteCache=enabled Drive conforms to: Unspecified:  ATA/ATAPI-3,4,5,6,7

 * signifies the current active mode

Quoted 1 line

hdparm -d1 /dev/sda

got the following error when I tried setting the mode as above, I think along with -d1 we need to give some "X" argument( or so reads the man page)..

 hdparm -d1 /dev/sda

/dev/sda:
 setting using_dma to 1 (on)
 HDIO_SET_DMA failed: Inappropriate ioctl for device HDIO_GET_DMA failed: Inappropriate ioctl for device

Quoted 2 lines

Clearly, since you see the same thing in a console with X never having been started says it's not a GUI issue.

Quoted 6 lines

yes I suppose, but any idea how I figure out what exactly is causing the IOWAIT, .. and if there are any patches, let me know, I can test them .

Cheers
Satish

OK, the solution for this is probably beyond my expertise but this looks identical to when I got a Compaq laptop and the ATI chipset's DMA driver wasn't working right. hdparm said I was using udma but I got error messages like you are when I tried to enable DMA.

Do you believe this was working better on an earlier kernel? If so you might drop back and see if these commands work better. If so that would suggest a regression of some type.

Hopefully someone here can direct you to the right person that supports this.

- Mark

#8 lore raw
Quoted 3 lines

Do you believe this was working better on an earlier kernel? If so you might drop back and see if these commands work better. If so that would suggest a regression of some type.

I remember good performance with 2.6.26.x series, ... but I could not try that out, ,... things on a 2.6.27.23 kernel seem no better :

Linux 2.6.27.23 (satish) 05/16/2009

12:11:18 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:11:21 PM     all      0.00      0.00      3.92     75.65      0.00     20.42
12:11:24 PM     all      0.00      0.00      3.55     67.85      0.00     28.59
12:11:27 PM     all      0.16      0.00      4.51     45.09      0.00     50.24
12:11:30 PM     all      0.00      0.00      3.59     44.77      0.00     51.63
12:11:33 PM     all      0.16      0.00      5.25     82.51      0.00     12.08
Average:        all      0.06      0.00      4.17     63.24      0.00     32.53

the average disk speed was about 10.5 MB/s, indeed the average IOWAIT is more than that in the latest kernel, I will see if I can test it on a 2.6.26.x kernel or earlier one and send you guys the stats.

Cheers
satish

Signature · 1 line
http://satish.playdrupal.com
#9 lore raw
Quoted 7 lines · Satish Eerpini
Quoted 3 lines

Do you believe this was working better on an earlier kernel? If so you might drop back and see if these commands work better. If so that would suggest a regression of some type.

I remember good performance with 2.6.26.x series, ... but I could not try that out, ,... things on a 2.6.27.23 kernel seem no better :

I think it's critically important that you find a kernel that has good performance. Without it we're trusting a memory. With it developers can bisect and find the solution.

- Mark

#10 lore raw
In reply to Satish Eerpini
Quoted 17 lines · Satish Eerpini
Quoted 3 lines

Do you believe this was working better on an earlier kernel? If so you might drop back and see if these commands work better. If so that would suggest a regression of some type.

I remember good performance with 2.6.26.x series, ... but I could not try that out, ,... things on a 2.6.27.23 kernel seem no better :

Linux 2.6.27.23 (satish) 05/16/2009

12:11:18 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:11:21 PM     all      0.00      0.00      3.92     75.65      0.00     20.42
12:11:24 PM     all      0.00      0.00      3.55     67.85      0.00     28.59
12:11:27 PM     all      0.16      0.00      4.51     45.09      0.00     50.24
12:11:30 PM     all      0.00      0.00      3.59     44.77      0.00     51.63
12:11:33 PM     all      0.16      0.00      5.25     82.51      0.00     12.08
Average:        all      0.06      0.00      4.17     63.24      0.00     32.53
Quoted 3 lines

the average disk speed was about 10.5 MB/s, indeed the average IOWAIT is more than that in the latest kernel, I will see if I can test it on a 2.6.26.x kernel or earlier one and send you guys the stats.

High iowait during a file copy with no other activity is entirely normal. If you have a core that has nothing to do but wait for IO to complete, you'll get iowait time. This isn't indicating a problem.

#11 lore raw
Quoted 3 lines

High iowait during a file copy with no other activity is entirely normal. If you have a core that has nothing to do but wait for IO to complete, you'll get iowait time. This isn't indicating a problem.

then the responsiveness seemed to be better with Wu Fengguang's patches applied, or when I decreased the swappiness by decreasing the value in /proc/sys/vm/swappiness . So I think I will stick with that, and thanks everyone for the help.

thanks
Satish

Signature · 1 line
http://satish.playdrupal.com
#12 lore raw
In reply to Mark Knecht
Quoted 12 lines · Mark Knecht

On Fri, May 15, 2009 at 7:38 PM, Satish Eerpini <[email protected]> wrote:
<SNIP>

Quoted 3 lines

yeah I get similar numbers even in the absence of the desktop manager , when I tried to copy from a virtual console.
the stats look something like this :

<SNIP>

Quoted 2 lines

00:1f.2 IDE interface: Intel Corporation 82801GBM/GHM (ICH7 Family) SATA IDE Controller (rev 01)

<SNIP>

I googled on "82801GBM/GHM slow" and found a lot to read.

Quoted 8 lines

Is DMA for this drive enabled? It seems that some others didn't have it enabled by default, or maybe at all. I didn't real the threads very far.

hdparm -d1 /dev/sda

I think hdparm works for most SATA drivers. man sata to see the other things you can check in terms of configuration.

Switching on DMA manually isn't possible with libata through hdparm currently. However, this shouldn't be required with libata.

Quoted 6 lines

Clearly, since you see the same thing in a console with X never having been started says it's not a GUI issue.

Hope this helps,
Mark

#13 lore raw
In reply to Satish Eerpini
Quoted 9 lines · Satish Eerpini

Hello everyone,

I am trying to figure out why the Linux desktop becomes overly unresponsive whenever there is a file copy going on, this happens to a large extent with GUI based file copying, and some lesser, but with command line copying too.
does it have anything to do with the kernel ??,
if it does help, then I am running Fedora 10 on a Hp nx 7400, Centrino Duo processor.

Which kernel version? If it's something recent, your programs may be getting swapped out due to a flaw in the latest kernels, where it accidentally prefers caching file data over mapped pages. If that's the case, Rik (cc:d) is working on patches to address the issue, and may appreciate another tester.

#14 lore raw
Quoted 5 lines · Ray Lee

Which kernel version? If it's something recent, your programs may be getting swapped out due to a flaw in the latest kernels, where it accidentally prefers caching file data over mapped pages. If that's the case, Rik (cc:d) is working on patches to address the issue, and may appreciate another tester.

Please keep me in the loop. I'd be interested if those patches work and what effect they have.

#15 lore raw
Quoted 10 lines · Christoph Lameter
Quoted 5 lines · Ray Lee

Which kernel version? If it's something recent, your programs may be getting swapped out due to a flaw in the latest kernels, where it accidentally prefers caching file data over mapped pages. If that's the case, Rik (cc:d) is working on patches to address the issue, and may appreciate another tester.

Please keep me in the loop. I'd be interested if those patches work and what effect they have.

Whoops, looks like Wu Fengguang (cc:d), not Rik, has been working on those patches. See the messages starting with "[PATCH 0/3] make mapped executable pages the first class citizen" [
http://lkml.org/lkml/2009/5/16/23 ]

Wu: The original reporter in this thread, Satish Eerpini, is reporting interactivity problems that sound very much like mapped executables are getting paged out while doing large copies. I suggested that perhaps he'd like to try out your patches to see if it addresses his problem. His original message is reproduced below. Later in the thread, he said 2.6.26 was fine, 2.6.27 & .28 were not.

Quoted 14 lines

Hello everyone,

I am trying to figure out why the Linux desktop becomes overly unresponsive whenever there is a file copy going on, this happens to a large extent with GUI based file copying, and some lesser, but with command line copying too.
does it have anything to do with the kernel ??,
if it does help, then I am running Fedora 10 on a Hp nx 7400, Centrino Duo processor.

Cheers
Satish
--
http://satish.playdrupal.com

#16 lore raw
Quoted 9 lines · Ray Lee
Quoted 7 lines · Christoph Lameter
Quoted 5 lines · Ray Lee

Which kernel version? If it's something recent, your programs may be getting swapped out due to a flaw in the latest kernels, where it accidentally prefers caching file data over mapped pages. If that's the case, Rik (cc:d) is working on patches to address the issue, and may appreciate another tester.

Quoted 7 lines
Quoted 2 lines

Please keep me in the loop. I'd be interested if those patches work and what effect they have.

Whoops, looks like Wu Fengguang (cc:d), not Rik, has been working on those patches. See the messages starting with "[PATCH 0/3] make mapped executable pages the first class citizen" [
http://lkml.org/lkml/2009/5/16/23 ]

We both have. Wu's patches build on a patch of mine, which Andrew has already merged into -mm.

http://lkml.org/lkml/2009/4/29/489

Signature · 1 line
All rights reversed.
#17 lore raw
Quoted 18 lines · Rik van Riel
Quoted 9 lines · Ray Lee
Quoted 7 lines · Christoph Lameter
Quoted 5 lines · Ray Lee

Which kernel version? If it's something recent, your programs may be getting swapped out due to a flaw in the latest kernels, where it accidentally prefers caching file data over mapped pages. If that's the case, Rik (cc:d) is working on patches to address the issue, and may appreciate another tester.

Quoted 7 lines
Quoted 2 lines

Please keep me in the loop. I'd be interested if those patches work and what effect they have.

Whoops, looks like Wu Fengguang (cc:d), not Rik, has been working on those patches. See the messages starting with "[PATCH 0/3] make mapped executable pages the first class citizen" [
http://lkml.org/lkml/2009/5/16/23 ]

Quoted 4 lines

We both have. Wu's patches build on a patch of mine, which Andrew has already merged into -mm.

http://lkml.org/lkml/2009/4/29/489

Yes, better to apply Rik and mine patches together to get the best cache behavior. Rik's patch is crucial. They are both small patches, so can be backported trivially. Satish, what's your kernel version (or, do you want to try a newer kernel)? And the content of /proc/meminfo?

Thanks,
Fengguang

#18 lore raw
Quoted 5 lines

Yes, better to apply Rik and mine patches together to get the best cache behavior. Rik's patch is crucial. They are both small patches, so can be backported trivially. Satish, what's your kernel version (or, do you want to try a newer kernel)? And the content of /proc/meminfo?

I am running 2.6.29.1 right now , just tell me which kernel to try, or to which kernel I have to apply the patches and try them, I will test the patches and let you know if the situation improves.
and the contents of /proc/meminfo are :

MemTotal:        2585724 kB
MemFree:          900524 kB
Buffers:          197504 kB
Cached:          1080332 kB
SwapCached:            0 kB
Active:           639544 kB
Inactive:         933068 kB
Active(anon):     307100 kB
Inactive(anon):        0 kB
Active(file):     332444 kB
Inactive(file):   933068 kB
Unevictable:          16 kB
Mlocked:              16 kB
HighTotal:       1707848 kB
HighFree:         321868 kB
LowTotal:         877876 kB
LowFree:          578656 kB
SwapTotal:       3526256 kB
SwapFree:        3526256 kB
Dirty:              1000 kB
Writeback:             0 kB
AnonPages:        294788 kB
Mapped:            80084 kB
Slab:              89692 kB
SReclaimable:      71436 kB
SUnreclaim:        18256 kB
PageTables:         6008 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     4819116 kB
Committed_AS:     796332 kB
VmallocTotal:     122880 kB
VmallocUsed:        8240 kB
VmallocChunk:      96952 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       4096 kB
DirectMap4k:       20472 kB
DirectMap4M:      884736 kB

Cheers
Satish

Signature · 1 line
http://satish.playdrupal.com
#19 lore raw
Quoted 10 lines · Satish Eerpini
Quoted 5 lines

Yes, better to apply Rik and mine patches together to get the best cache behavior. Rik's patch is crucial. They are both small patches, so can be backported trivially. Satish, what's your kernel version (or, do you want to try a newer kernel)? And the content of /proc/meminfo?

I am running 2.6.29.1 right now , just tell me which kernel to try, or to which kernel I have to apply the patches and try them, I will test the patches and let you know if the situation improves.
and the contents of /proc/meminfo are :

Satish, so you have 2.5GB memory and 80MB mapped pages, 33MB active desktop files to protect.

I've prepared a rolled up patch for you, run time tested. It should protect all of the above pages from being evicted by large file copies.

The attached patch is for 2.6.29.

Thanks,
Fengguang

Quoted 43 lines
MemTotal:        2585724 kB
MemFree:          900524 kB
Buffers:          197504 kB
Cached:          1080332 kB
SwapCached:            0 kB
Active:           639544 kB
Inactive:         933068 kB
Active(anon):     307100 kB
Inactive(anon):        0 kB
Active(file):     332444 kB
Inactive(file):   933068 kB
Unevictable:          16 kB
Mlocked:              16 kB
HighTotal:       1707848 kB
HighFree:         321868 kB
LowTotal:         877876 kB
LowFree:          578656 kB
SwapTotal:       3526256 kB
SwapFree:        3526256 kB
Dirty:              1000 kB
Writeback:             0 kB
AnonPages:        294788 kB
Mapped:            80084 kB
Slab:              89692 kB
SReclaimable:      71436 kB
SUnreclaim:        18256 kB
PageTables:         6008 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     4819116 kB
Committed_AS:     796332 kB
VmallocTotal:     122880 kB
VmallocUsed:        8240 kB
VmallocChunk:      96952 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       4096 kB
DirectMap4k:       20472 kB
DirectMap4M:      884736 kB
Quoted 4 lines

Cheers
Satish

Signature · 1 line
http://satish.playdrupal.com
mm/vmscan.c +88 -22
@@ -581,6 +581,7 @@ static unsigned long shrink_page_list(st
 	struct pagevec freed_pvec;
 	int pgactivate = 0;
 	unsigned long nr_reclaimed = 0;
+	unsigned long vm_flags;
 
 	cond_resched();
 
@@ -631,7 +632,8 @@ static unsigned long shrink_page_list(st
 				goto keep_locked;
 		}
 
-		referenced = page_referenced(page, 1, sc->mem_cgroup);
+		referenced = page_referenced(page, 1,
+						sc->mem_cgroup, &vm_flags);
 		/* In active use or really unfreeable?  Activate it. */
 		if (sc->order <= PAGE_ALLOC_COSTLY_ORDER &&
 					referenced && page_mapping_inuse(page))
@@ -1210,9 +1212,10 @@ static void shrink_active_list(unsigned 
 			struct scan_control *sc, int priority, int file)
 {
 	unsigned long pgmoved;
-	int pgdeactivate = 0;
 	unsigned long pgscanned;
+	unsigned long vm_flags;
 	LIST_HEAD(l_hold);	/* The pages which were snipped off */
+	LIST_HEAD(l_active);
 	LIST_HEAD(l_inactive);
 	struct page *page;
 	struct pagevec pvec;
@@ -1239,7 +1242,7 @@ static void shrink_active_list(unsigned 
 		__mod_zone_page_state(zone, NR_ACTIVE_ANON, -pgmoved);
 	spin_unlock_irq(&zone->lru_lock);
 
-	pgmoved = 0;
+	pgmoved = 0;  /* count referenced (mapping) mapped pages */
 	while (!list_empty(&l_hold)) {
 		cond_resched();
 		page = lru_to_page(&l_hold);
@@ -1252,28 +1255,41 @@ static void shrink_active_list(unsigned 
 
 		/* page_referenced clears PageReferenced */
 		if (page_mapping_inuse(page) &&
-		    page_referenced(page, 0, sc->mem_cgroup))
+		    page_referenced(page, 0, sc->mem_cgroup, &vm_flags)) {
 			pgmoved++;
+			/*
+			 * Identify referenced, file-backed active pages and
+			 * give them one more trip around the active list. So
+			 * that executable code get better chances to stay in
+			 * memory under moderate memory pressure.  Anon pages
+			 * are ignored, since JVM can create lots of anon
+			 * VM_EXEC pages.
+			 */
+			if ((vm_flags & VM_EXEC) && !PageAnon(page)) {
+				list_add(&page->lru, &l_active);
+				continue;
+			}
+		}
 
 		list_add(&page->lru, &l_inactive);
 	}
 
 	/*
-	 * Move the pages to the [file or anon] inactive list.
+	 * Move pages back to the lru list.
 	 */
 	pagevec_init(&pvec, 1);
-	lru = LRU_BASE + file * LRU_FILE;
 
 	spin_lock_irq(&zone->lru_lock);
 	/*
-	 * Count referenced pages from currently used mappings as
-	 * rotated, even though they are moved to the inactive list.
-	 * This helps balance scan pressure between file and anonymous
-	 * pages in get_scan_ratio.
+	 * Count referenced pages from currently used mappings as rotated,
+	 * even though only some of them are actually re-activated.  This
+	 * helps balance scan pressure between file and anonymous pages in
+	 * get_scan_ratio.
 	 */
 	reclaim_stat->recent_rotated[!!file] += pgmoved;
 
-	pgmoved = 0;
+	pgmoved = 0;  /* count pages moved to inactive list */
+	lru = LRU_BASE + file * LRU_FILE;
 	while (!list_empty(&l_inactive)) {
 		page = lru_to_page(&l_inactive);
 		prefetchw_prev_lru_page(page, &l_inactive, flags);
@@ -1286,10 +1302,7 @@ static void shrink_active_list(unsigned 
 		mem_cgroup_add_lru_list(page, lru);
 		pgmoved++;
 		if (!pagevec_add(&pvec, page)) {
-			__mod_zone_page_state(zone, NR_LRU_BASE + lru, pgmoved);
 			spin_unlock_irq(&zone->lru_lock);
-			pgdeactivate += pgmoved;
-			pgmoved = 0;
 			if (buffer_heads_over_limit)
 				pagevec_strip(&pvec);
 			__pagevec_release(&pvec);
@@ -1297,14 +1310,31 @@ static void shrink_active_list(unsigned 
 		}
 	}
 	__mod_zone_page_state(zone, NR_LRU_BASE + lru, pgmoved);
-	pgdeactivate += pgmoved;
-	if (buffer_heads_over_limit) {
-		spin_unlock_irq(&zone->lru_lock);
-		pagevec_strip(&pvec);
-		spin_lock_irq(&zone->lru_lock);
-	}
 	__count_zone_vm_events(PGREFILL, zone, pgscanned);
-	__count_vm_events(PGDEACTIVATE, pgdeactivate);
+	__count_vm_events(PGDEACTIVATE, pgmoved);
+
+	pgmoved = 0;  /* count pages moved back to active list */
+	lru = LRU_ACTIVE + file * LRU_FILE;
+	while (!list_empty(&l_active)) {
+		page = lru_to_page(&l_active);
+		prefetchw_prev_lru_page(page, &l_active, flags);
+		VM_BUG_ON(PageLRU(page));
+		SetPageLRU(page);
+		VM_BUG_ON(!PageActive(page));
+
+		list_move(&page->lru, &zone->lru[lru].list);
+		mem_cgroup_add_lru_list(page, lru);
+		pgmoved++;
+		if (!pagevec_add(&pvec, page)) {
+			spin_unlock_irq(&zone->lru_lock);
+			if (buffer_heads_over_limit)
+				pagevec_strip(&pvec);
+			__pagevec_release(&pvec);
+			spin_lock_irq(&zone->lru_lock);
+		}
+	}
+	__mod_zone_page_state(zone, NR_LRU_BASE + lru, pgmoved);
+
 	spin_unlock_irq(&zone->lru_lock);
 	if (vm_swap_full())
 		pagevec_swap_free(&pvec);
@@ -1344,12 +1374,48 @@ static int inactive_anon_is_low(struct z
 	return low;
 }
 
+static int inactive_file_is_low_global(struct zone *zone)
+{
+	unsigned long active, inactive;
+
+	active = zone_page_state(zone, NR_ACTIVE_FILE);
+	inactive = zone_page_state(zone, NR_INACTIVE_FILE);
+
+	return (active > inactive);
+}
+
+/**
+ * inactive_file_is_low - check if file pages need to be deactivated
+ * @zone: zone to check
+ * @sc:   scan control of this context
+ *
+ * When the system is doing streaming IO, memory pressure here
+ * ensures that active file pages get deactivated, until more
+ * than half of the file pages are on the inactive list.
+ *
+ * Once we get to that situation, protect the system's working
+ * set from being evicted by disabling active file page aging.
+ *
+ * This uses a different ratio than the anonymous pages, because
+ * the page cache uses a use-once replacement algorithm.
+ */
+static int inactive_file_is_low(struct zone *zone, struct scan_control *sc)
+{
+	int low;
+
+	if (scanning_global_lru(sc))
+		low = inactive_file_is_low_global(zone);
+	else
+		low = mem_cgroup_inactive_file_is_low(sc->mem_cgroup);
+	return low;
+}
+
 static unsigned long shrink_list(enum lru_list lru, unsigned long nr_to_scan,
 	struct zone *zone, struct scan_control *sc, int priority)
 {
 	int file = is_file_lru(lru);
 
-	if (lru == LRU_ACTIVE_FILE) {
+	if (lru == LRU_ACTIVE_FILE && inactive_file_is_low(zone, sc)) {
 		shrink_active_list(nr_to_scan, zone, sc, priority, file);
 		return 0;
 	}
include/linux/memcontrol.h +7 -0
@@ -97,6 +97,7 @@ extern void mem_cgroup_note_reclaim_prio
 extern void mem_cgroup_record_reclaim_priority(struct mem_cgroup *mem,
 							int priority);
 int mem_cgroup_inactive_anon_is_low(struct mem_cgroup *memcg);
+int mem_cgroup_inactive_file_is_low(struct mem_cgroup *memcg);
 unsigned long mem_cgroup_zone_nr_pages(struct mem_cgroup *memcg,
 				       struct zone *zone,
 				       enum lru_list lru);
@@ -250,6 +251,12 @@ mem_cgroup_inactive_anon_is_low(struct m
 	return 1;
 }
 
+static inline int
+mem_cgroup_inactive_file_is_low(struct mem_cgroup *memcg)
+{
+	return 1;
+}
+
 static inline unsigned long
 mem_cgroup_zone_nr_pages(struct mem_cgroup *memcg, struct zone *zone,
 			 enum lru_list lru)
mm/memcontrol.c +11 -0
@@ -536,6 +536,17 @@ int mem_cgroup_inactive_anon_is_low(stru
 	return 0;
 }
 
+int mem_cgroup_inactive_file_is_low(struct mem_cgroup *memcg)
+{
+	unsigned long active;
+	unsigned long inactive;
+
+	inactive = mem_cgroup_get_local_zonestat(memcg, LRU_INACTIVE_FILE);
+	active = mem_cgroup_get_local_zonestat(memcg, LRU_ACTIVE_FILE);
+
+	return (active > inactive);
+}
+
 unsigned long mem_cgroup_zone_nr_pages(struct mem_cgroup *memcg,
 				       struct zone *zone,
 				       enum lru_list lru)
include/linux/rmap.h +3 -2
@@ -83,7 +83,8 @@ static inline void page_dup_rmap(struct 
 /*
  * Called from mm/vmscan.c to handle paging out
  */
-int page_referenced(struct page *, int is_locked, struct mem_cgroup *cnt);
+int page_referenced(struct page *, int is_locked,
+			struct mem_cgroup *cnt, unsigned long *vm_flags);
 int try_to_unmap(struct page *, int ignore_refs);
 
 /*
@@ -124,7 +125,7 @@ static inline int try_to_munlock(struct 
 #define anon_vma_prepare(vma)	(0)
 #define anon_vma_link(vma)	do {} while (0)
 
-#define page_referenced(page,l,cnt) TestClearPageReferenced(page)
+#define page_referenced(page, locked, cnt, flags) TestClearPageReferenced(page)
 #define try_to_unmap(page, refs) SWAP_FAIL
 
 static inline int page_mkclean(struct page *page)
mm/rmap.c +26 -11
@@ -333,7 +333,9 @@ static int page_mapped_in_vma(struct pag
  * repeatedly from either page_referenced_anon or page_referenced_file.
  */
 static int page_referenced_one(struct page *page,
-	struct vm_area_struct *vma, unsigned int *mapcount)
+			       struct vm_area_struct *vma,
+			       unsigned int *mapcount,
+			       unsigned long *vm_flags)
 {
 	struct mm_struct *mm = vma->vm_mm;
 	unsigned long address;
@@ -381,11 +383,14 @@ out_unmap:
 	(*mapcount)--;
 	pte_unmap_unlock(pte, ptl);
 out:
+	if (referenced)
+		*vm_flags |= vma->vm_flags;
 	return referenced;
 }
 
 static int page_referenced_anon(struct page *page,
-				struct mem_cgroup *mem_cont)
+				struct mem_cgroup *mem_cont,
+				unsigned long *vm_flags)
 {
 	unsigned int mapcount;
 	struct anon_vma *anon_vma;
@@ -405,7 +410,8 @@ static int page_referenced_anon(struct p
 		 */
 		if (mem_cont && !mm_match_cgroup(vma->vm_mm, mem_cont))
 			continue;
-		referenced += page_referenced_one(page, vma, &mapcount);
+		referenced += page_referenced_one(page, vma,
+						  &mapcount, vm_flags);
 		if (!mapcount)
 			break;
 	}
@@ -418,6 +424,7 @@ static int page_referenced_anon(struct p
  * page_referenced_file - referenced check for object-based rmap
  * @page: the page we're checking references on.
  * @mem_cont: target memory controller
+ * @vm_flags: collect encountered vma->vm_flags
  *
  * For an object-based mapped page, find all the places it is mapped and
  * check/clear the referenced flag.  This is done by following the page->mapping
@@ -427,7 +434,8 @@ static int page_referenced_anon(struct p
  * This function is only called from page_referenced for object-based pages.
  */
 static int page_referenced_file(struct page *page,
-				struct mem_cgroup *mem_cont)
+				struct mem_cgroup *mem_cont,
+				unsigned long *vm_flags)
 {
 	unsigned int mapcount;
 	struct address_space *mapping = page->mapping;
@@ -467,7 +475,8 @@ static int page_referenced_file(struct p
 		 */
 		if (mem_cont && !mm_match_cgroup(vma->vm_mm, mem_cont))
 			continue;
-		referenced += page_referenced_one(page, vma, &mapcount);
+		referenced += page_referenced_one(page, vma,
+						  &mapcount, vm_flags);
 		if (!mapcount)
 			break;
 	}
@@ -481,29 +490,35 @@ static int page_referenced_file(struct p
  * @page: the page to test
  * @is_locked: caller holds lock on the page
  * @mem_cont: target memory controller
+ * @vm_flags: collect encountered vma->vm_flags
  *
  * Quick test_and_clear_referenced for all mappings to a page,
  * returns the number of ptes which referenced the page.
  */
-int page_referenced(struct page *page, int is_locked,
-			struct mem_cgroup *mem_cont)
+int page_referenced(struct page *page,
+		    int is_locked,
+		    struct mem_cgroup *mem_cont,
+		    unsigned long *vm_flags)
 {
 	int referenced = 0;
 
 	if (TestClearPageReferenced(page))
 		referenced++;
 
+	*vm_flags = 0;
 	if (page_mapped(page) && page->mapping) {
 		if (PageAnon(page))
-			referenced += page_referenced_anon(page, mem_cont);
+			referenced += page_referenced_anon(page, mem_cont,
+								vm_flags);
 		else if (is_locked)
-			referenced += page_referenced_file(page, mem_cont);
+			referenced += page_referenced_file(page, mem_cont,
+								vm_flags);
 		else if (!trylock_page(page))
 			referenced++;
 		else {
 			if (page->mapping)
-				referenced +=
-					page_referenced_file(page, mem_cont);
+				referenced += page_referenced_file(page,
+							mem_cont, vm_flags);
 			unlock_page(page);
 		}
 	}
#20 lore raw
Quoted 4 lines

I've prepared a rolled up patch for you, run time tested.  It should protect all of the above pages from being evicted by large file copies.

The attached patch is for 2.6.29.

I applied the patch , but the number don seem to show much difference , following are the statistics with the patched kernel, could something else be causing the huge iowait, :

Linux 2.6.29 (satish) 05/17/2009

09:57:14 AM CPU %user %nice %system %iowait %steal %idle
09:57:18 AM all 12.93 0.00 10.25 33.44
0.00 43.38
09:57:21 AM all 13.32 0.00 28.09 24.40
0.00 34.19
09:57:24 AM all 10.79 0.00 4.76 75.56
0.00 8.89
09:57:26 AM all 11.46 0.00 8.01 35.64
0.00 44.90
09:57:30 AM all 11.68 0.00 8.88 35.05
0.00 44.39
Average: all 12.03 0.00 11.94 40.81
0.00 35.22

and

hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 1332 MB in 2.00 seconds = 666.35 MB/sec
Timing buffered disk reads: 24 MB in 3.32 seconds = 7.23 MB/sec

,

Cheers
Satish

Signature · 1 line
http://satish.playdrupal.com
#21 lore raw
Quoted 10 lines · On Sun, 17 May 2009 10:00:28 +0530 Satish Eerpini <[email protected]> wrote:
Quoted 5 lines

I've prepared a rolled up patch for you, run time tested.  It should protect all of the above pages from being evicted by large file copies.

The attached patch is for 2.6.29.

I applied the patch , but the number don seem to show much difference , following are the statistics with the patched kernel, could something else be causing the huge iowait, :

for i in pidof kjournald ; do ionice -c1 -p $i ; done

is another thing to try to fix the responsiveness...

Signature · 3 lines
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
#22 lore raw
In reply to Satish Eerpini
Quoted 6 lines · Satish Eerpini
Quoted 4 lines

I've prepared a rolled up patch for you, run time tested.  It should protect all of the above pages from being evicted by large file copies.

The attached patch is for 2.6.29.

I applied the patch , but the number don seem to show much difference

The numbers listed in your email? No the most relevant numbers are memory and disk ones.

For my part, the number of mapped pages keeps stable when there is an ongoing background file copy:

% ls -l /b/sparse
-rw-r--r-- 1 root root 98T 2009-04-29 22:38 /b/sparse
% cp /b/sparse /dev/null&
% for i in seq 1 100; do grep Mapped /proc/meminfo; sleep 1; done

Mapped:            22764 kB
Mapped:            22796 kB
Mapped:            22756 kB
Mapped:            22716 kB
Mapped:            22800 kB
Mapped:            22788 kB
Mapped:            22804 kB
Mapped:            22808 kB
Mapped:            22748 kB
Mapped:            22708 kB
Mapped:            22916 kB
Mapped:            22944 kB
Mapped:            22944 kB
Mapped:            22944 kB
Mapped:            22944 kB
Mapped:            22944 kB
Mapped:            22832 kB
Mapped:            22812 kB
Mapped:            22812 kB
Mapped:            22792 kB
Mapped:            22772 kB
Mapped:            22860 kB
Mapped:            22860 kB
Mapped:            22748 kB
Mapped:            22808 kB
Mapped:            22868 kB
Mapped:            22956 kB
Mapped:            22956 kB
Mapped:            22832 kB
Mapped:            22832 kB
Mapped:            22980 kB
Mapped:            22980 kB
Mapped:            22980 kB
Mapped:            22980 kB
Mapped:            22872 kB
Mapped:            22872 kB
Mapped:            22900 kB
Mapped:            22792 kB
Mapped:            22920 kB
Mapped:            22812 kB
Mapped:            22812 kB
Mapped:            22752 kB
Mapped:            22864 kB
Mapped:            22972 kB
Mapped:            22860 kB
Mapped:            22796 kB
Mapped:            22900 kB
Mapped:            22888 kB
Mapped:            22888 kB
Mapped:            22888 kB
Mapped:            22764 kB
Quoted 5 lines

, following are the statistics with the patched kernel, could something else be causing the huge iowait, :

Linux 2.6.29 (satish) 05/17/2009

Quoted 7 lines
09:57:14 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
09:57:18 AM     all     12.93      0.00     10.25     33.44      0.00     43.38
09:57:21 AM     all     13.32      0.00     28.09     24.40      0.00     34.19
09:57:24 AM     all     10.79      0.00      4.76     75.56      0.00      8.89
09:57:26 AM     all     11.46      0.00      8.01     35.64      0.00     44.90
09:57:30 AM     all     11.68      0.00      8.88     35.05      0.00     44.39
Average:        all     12.03      0.00     11.94     40.81      0.00     35.22

The iowait is high. Do you have "iostat -x 5' numbers?

Quoted 7 lines

and

hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 1332 MB in 2.00 seconds = 666.35 MB/sec
Timing buffered disk reads: 24 MB in 3.32 seconds = 7.23 MB/sec

That's pretty slow numbers. On my laptop:

# hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 10266 MB in 1.98 seconds = 5188.46 MB/sec
Timing buffered disk reads: 138 MB in 3.01 seconds = 45.90 MB/sec

Thanks,
Fengguang

#23 lore raw
Quoted 2 lines

The numbers listed in your email? No the most relevant numbers are memory and disk ones.

the numbers I earlier mentioned were the output of : "sar" for the number of mapped pages stays almost constant on my lappy too :

Mapped:            70840 kB
Mapped:            70860 kB
Mapped:            70736 kB
Mapped:            70860 kB
Mapped:            70736 kB
Mapped:            70860 kB
Mapped:            70860 kB
Mapped:            70736 kB
Mapped:            70736 kB
Mapped:            70736 kB
Mapped:            70860 kB
Mapped:            70736 kB
Mapped:            70736 kB
Mapped:            70860 kB
Mapped:            70860 kB
Mapped:            70736 kB
Mapped:            70736 kB
Mapped:            70860 kB
Mapped:            70860 kB
Mapped:            70736 kB
Mapped:            70736 kB
Mapped:            70736 kB
Mapped:            70736 kB
Mapped:            70736 kB
Mapped:            70860 kB
Mapped:            70860 kB
Mapped:            70736 kB
Mapped:            70860 kB
Mapped:            70736 kB
Mapped:            70724 kB
Mapped:            70860 kB
Mapped:            70860 kB
Mapped:            70736 kB
Mapped:            70736 kB

Satish

Signature · 1 line
http://satish.playdrupal.com
#24 lore raw
Quoted 9 lines · Satish Eerpini
Quoted 2 lines

The numbers listed in your email? No the most relevant numbers are memory and disk ones.

the numbers I earlier mentioned were the output of : "sar" for the number of mapped pages stays almost constant on my lappy too :

Mapped:            70840 kB
Mapped:            70860 kB
Mapped:            70736 kB
Mapped:            70860 kB

Hmm, how do you define/feel the unresponsiveness? Is it related to the currently *running* applications, or opening a *new* application?

Thanks,
Fengguang

#25 lore raw
Quoted 2 lines

Hmm, how do you define/feel the unresponsiveness?  Is it related to the currently *running* applications, or opening a *new* application?

the unresponsiveness is for both the currently running applications and for opening new applications, here are some pointers :

--> during normal operation, swtiching between windows( alt+tab) is simultaneous, while during a file copy it takes around 1 second, which I think is quite a lag.
--> Opening a simple application like gnome-terminal takes a second or more for sure when there is a file copy going on.
-->Also a particular window after gaining focus takes a second or so, to resume what it was doing, .. so if I am copying a file in a session of gnome-terminal and open another session of gnome-terminal and type in "clear" it takes around 2 seconds before the screen is cleared on the second session,
this is the case with almost every application and worser with big ones like firefox and office. This was the kind of unresponsiveness I was talking about.

Cheers
Satish

Signature · 1 line
http://satish.playdrupal.com
#26 lore raw
Quoted 6 lines · Satish Eerpini
Quoted 2 lines

Hmm, how do you define/feel the unresponsiveness?  Is it related to the currently *running* applications, or opening a *new* application?

the unresponsiveness is for both the currently running applications and for opening new applications, here are some pointers :

Quoted 10 lines

--> during normal operation, swtiching between windows( alt+tab) is simultaneous, while during a file copy it takes around 1 second, which I think is quite a lag.
--> Opening a simple application like gnome-terminal takes a second or more for sure when there is a file copy going on.
-->Also a particular window after gaining focus takes a second or so, to resume what it was doing, .. so if I am copying a file in a session of gnome-terminal and open another session of gnome-terminal and type in "clear" it takes around 2 seconds before the screen is cleared on the second session,

The clear command can be a big clue. Can you try these commands in the terminal and show us the two log files?

strace -o all.log -T clear
strace -o sum.log -c clear

Quoted 8 lines

this is the case with almost every application and worser with big ones like firefox and office. This was the kind of unresponsiveness I was talking about.

Cheers
Satish

Signature · 1 line
http://satish.playdrupal.com
#27 lore raw
Quoted 6 lines

The clear command can be a big clue. Can you try these commands in the terminal and show us the two log files?

       strace -o all.log -T clear
       strace -o sum.log -c clear

ok here go the logs :

"all.log"
execve("/usr/bin/clear", ["clear"], [/* 47 vars */]) = 0 <0.000432>
brk(0) = 0x981e000 <0.000018>
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or
directory) <0.000026>
open("/etc/ld.so.cache", O_RDONLY) = 3 <0.000027>
fstat64(3, {st_mode=S_IFREG|0644, st_size=140850, ...}) = 0 <0.000019> mmap2(NULL, 140850, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f01000 <0.000023>
close(3) = 0 <0.000018>
open("/lib/libtinfo.so.5", O_RDONLY) = 3 <0.000043>
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\332\355\0074\0\0\0D"...,
512) = 512 <0.000028>
fstat64(3, {st_mode=S_IFREG|0755, st_size=98036, ...}) = 0 <0.000020> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7f00000 <0.000021>
mmap2(0x7ed8000, 99928, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ed8000 <0.000023> mmap2(0x7eee000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15) = 0x7eee000 <0.000030>
close(3) = 0 <0.000018>
open("/lib/libc.so.6", O_RDONLY) = 3 <0.000029>
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@xk\0004\0\0\0P"...,
512) = 512 <0.000021>
fstat64(3, {st_mode=S_IFREG|0755, st_size=1809672, ...}) = 0 <0.000019> mmap2(0x6a1000, 1521232, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x6a1000 <0.000022>
mmap2(0x80f000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16e) = 0x80f000 <0.000032> mmap2(0x812000, 9808, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x812000 <0.000026>
close(3) = 0 <0.000018>
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7eff000 <0.000024>
set_thread_area({entry_number:-1 -> 6, base_addr:0xb7eff6c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 <0.000019>

mprotect(0x80f000, 8192, PROT_READ)     = 0 <0.000026>
mprotect(0x69d000, 4096, PROT_READ)     = 0 <0.000023>
munmap(0xb7f01000, 140850)              = 0 <0.000032>
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo
...}) = 0 <0.000023>
brk(0)                                  = 0x981e000 <0.000019>
brk(0x983f000)                          = 0x983f000 <0.000022>
stat64("/root/.terminfo", 0xbfd20f24)   = -1 ENOENT (No such file or

directory) <0.000027>
stat64("/etc/terminfo", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000025>
access("/etc/terminfo/x/xterm", R_OK) = -1 ENOENT (No such file or
directory) <0.000026>
stat64("/usr/share/terminfo", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000026>
access("/usr/share/terminfo/x/xterm", R_OK) = 0 <0.000028> open("/usr/share/terminfo/x/xterm", O_RDONLY|O_LARGEFILE) = 3 <0.000031> read(3, "\32\0010\0&\0\17\0\235\1F\5xterm|xterm terminal "..., 4097) = 3220 <0.000035>
close(3) = 0 <0.000022>
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 <0.000025>
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 <0.000020>
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 <0.000021>
ioctl(1, TIOCGWINSZ, {ws_row=24, ws_col=80, ws_xpixel=0, ws_ypixel=0}) = 0 <0.000020>
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0 <0.000020> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7f23000 <0.000025>
write(1, "\33[H\33[2J"..., 7) = 7 <0.000027>
exit_group(0) = ?

"sum.log"

% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------

 93.27    0.000485         485         1           execve
  6.73    0.000035           7         5           ioctl
  0.00    0.000000           0         3           read
  0.00    0.000000           0         1           write
  0.00    0.000000           0         4           open
  0.00    0.000000           0         4           close
  0.00    0.000000           0         3         2 access
  0.00    0.000000           0         3           brk
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         2           mprotect
  0.00    0.000000           0         9           mmap2
  0.00    0.000000           0         3         1 stat64
  0.00    0.000000           0         4           fstat64
  0.00    0.000000           0         1           set_thread_area

------ ----------- ----------- --------- --------- ----------------
100.00 0.000520 44 3 total

I have also attached the logs, in case that is more useful .

Cheers
Satish

Signature · 1 line
http://satish.playdrupal.com

execve("/usr/bin/clear", ["clear"], [/* 47 vars */]) = 0 <0.000432>

brk(0)                                  = 0x981e000 <0.000018>
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory) <0.000026>
open("/etc/ld.so.cache", O_RDONLY)      = 3 <0.000027>

fstat64(3, {st_mode=S_IFREG|0644, st_size=140850, ...}) = 0 <0.000019>
mmap2(NULL, 140850, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f01000 <0.000023>
close(3) = 0 <0.000018>
open("/lib/libtinfo.so.5", O_RDONLY) = 3 <0.000043>
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\332\355\0074\0\0\0D"..., 512) = 512 <0.000028>
fstat64(3, {st_mode=S_IFREG|0755, st_size=98036, ...}) = 0 <0.000020>
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f00000 <0.000021>
mmap2(0x7ed8000, 99928, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ed8000 <0.000023>
mmap2(0x7eee000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15) = 0x7eee000 <0.000030>
close(3) = 0 <0.000018>
open("/lib/libc.so.6", O_RDONLY) = 3 <0.000029>
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@xk\0004\0\0\0P"..., 512) = 512 <0.000021>
fstat64(3, {st_mode=S_IFREG|0755, st_size=1809672, ...}) = 0 <0.000019>
mmap2(0x6a1000, 1521232, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x6a1000 <0.000022>
mmap2(0x80f000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16e) = 0x80f000 <0.000032>
mmap2(0x812000, 9808, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x812000 <0.000026>
close(3) = 0 <0.000018>
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7eff000 <0.000024>
set_thread_area({entry_number:-1 -> 6, base_addr:0xb7eff6c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 <0.000019>

mprotect(0x80f000, 8192, PROT_READ)     = 0 <0.000026>
mprotect(0x69d000, 4096, PROT_READ)     = 0 <0.000023>
munmap(0xb7f01000, 140850)              = 0 <0.000032>
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 <0.000023>
brk(0)                                  = 0x981e000 <0.000019>
brk(0x983f000)                          = 0x983f000 <0.000022>
stat64("/root/.terminfo", 0xbfd20f24)   = -1 ENOENT (No such file or directory) <0.000027>
stat64("/etc/terminfo", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000025>
access("/etc/terminfo/x/xterm", R_OK)   = -1 ENOENT (No such file or directory) <0.000026>

stat64("/usr/share/terminfo", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 <0.000026> access("/usr/share/terminfo/x/xterm", R_OK) = 0 <0.000028> open("/usr/share/terminfo/x/xterm", O_RDONLY|O_LARGEFILE) = 3 <0.000031>
read(3, "\32\0010\0&\0\17\0\235\1F\5xterm|xterm terminal "..., 4097) = 3220 <0.000035>
close(3) = 0 <0.000022>
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 <0.000025> ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 <0.000020> ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 <0.000021> ioctl(1, TIOCGWINSZ, {ws_row=24, ws_col=80, ws_xpixel=0, ws_ypixel=0}) = 0 <0.000020> fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0 <0.000020> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f23000 <0.000025>
write(1, "\33[H\33[2J"..., 7) = 7 <0.000027>
exit_group(0) = ?

% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------

 93.27    0.000485         485         1           execve
  6.73    0.000035           7         5           ioctl
  0.00    0.000000           0         3           read
  0.00    0.000000           0         1           write
  0.00    0.000000           0         4           open
  0.00    0.000000           0         4           close
  0.00    0.000000           0         3         2 access
  0.00    0.000000           0         3           brk
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         2           mprotect
  0.00    0.000000           0         9           mmap2
  0.00    0.000000           0         3         1 stat64
  0.00    0.000000           0         4           fstat64
  0.00    0.000000           0         1           set_thread_area

------ ----------- ----------- --------- --------- ----------------
100.00 0.000520 44 3 total

#28 lore raw
Quoted 8 lines · Satish Eerpini
Quoted 6 lines

The clear command can be a big clue. Can you try these commands in the terminal and show us the two log files?

       strace -o all.log -T clear
       strace -o sum.log -c clear

ok here go the logs :

It looks that the "clear" command runs pretty fast in itself. To be sure, can you run this?

strace -o log -tt clear

Or maybe the time is spent in gnome-terminal or bash? What if you change them to xterm and dash?

Quoted 30 lines

"sum.log"

% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------

 93.27    0.000485         485         1           execve
  6.73    0.000035           7         5           ioctl
  0.00    0.000000           0         3           read
  0.00    0.000000           0         1           write
  0.00    0.000000           0         4           open
  0.00    0.000000           0         4           close
  0.00    0.000000           0         3         2 access
  0.00    0.000000           0         3           brk
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         2           mprotect
  0.00    0.000000           0         9           mmap2
  0.00    0.000000           0         3         1 stat64
  0.00    0.000000           0         4           fstat64
  0.00    0.000000           0         1           set_thread_area

------ ----------- ----------- --------- --------- ----------------
100.00 0.000520 44 3 total

I have also attached the logs, in case that is more useful .

Cheers
Satish

Signature · 1 line
http://satish.playdrupal.com
#29 lore raw
Quoted 3 lines

Or maybe the time is spent in gnome-terminal or bash? What if you change them to xterm and dash?

ok here you go , I ran "strace -o log -tt clear " on xterm and dash :

"log "

18:31:02.162301 execve("/usr/bin/clear", ["clear"], [/* 49 vars */]) = 0
18:31:02.163558 brk(0) = 0x943b000
18:31:02.163668 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
18:31:02.163779 open("/etc/ld.so.cache", O_RDONLY) = 3 18:31:02.163858 fstat64(3, {st_mode=S_IFREG|0644, st_size=140850, ...}) = 0 18:31:02.163975 mmap2(NULL, 140850, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f16000
18:31:02.164036 close(3) = 0
18:31:02.164122 open("/lib/libtinfo.so.5", O_RDONLY) = 3 18:31:02.164206 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\332\355\0074\0\0\0D"...,
512) = 512
18:31:02.164303 fstat64(3, {st_mode=S_IFREG|0755, st_size=98036, ...}) = 0 18:31:02.164411 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f15000
18:31:02.164481 mmap2(0x7ed8000, 99928, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ed8000
18:31:02.164545 mmap2(0x7eee000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15) = 0x7eee000
18:31:02.164620 close(3) = 0
18:31:02.164693 open("/lib/libc.so.6", O_RDONLY) = 3 18:31:02.164777 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@xk\0004\0\0\0P"...,
512) = 512
18:31:02.164862 fstat64(3, {st_mode=S_IFREG|0755, st_size=1809672, ...}) = 0 18:31:02.164972 mmap2(0x6a1000, 1521232, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x6a1000
18:31:02.165035 mmap2(0x80f000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16e) = 0x80f000 18:31:02.165123 mmap2(0x812000, 9808, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x812000
18:31:02.165187 close(3) = 0
18:31:02.165265 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f14000
18:31:02.165327 set_thread_area({entry_number:-1 -> 6, base_addr:0xb7f146c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 18:31:02.165412 mprotect(0x80f000, 8192, PROT_READ) = 0 18:31:02.165472 mprotect(0x69d000, 4096, PROT_READ) = 0 18:31:02.165532 munmap(0xb7f16000, 140850) = 0
18:31:02.165635 ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
18:31:02.165816 brk(0) = 0x943b000
18:31:02.165870 brk(0x945c000) = 0x945c000
18:31:02.165966 stat64("/root/.terminfo", 0xbfc37e14) = -1 ENOENT (No such file or directory)
18:31:02.166040 stat64("/etc/terminfo", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
18:31:02.166181 access("/etc/terminfo/x/xterm", R_OK) = -1 ENOENT (No such file or directory)
18:31:02.166256 stat64("/usr/share/terminfo", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
18:31:02.166379 access("/usr/share/terminfo/x/xterm", R_OK) = 0 18:31:02.166454 open("/usr/share/terminfo/x/xterm", O_RDONLY|O_LARGEFILE) = 3 18:31:02.166531 read(3, "\32\0010\0&\0\17\0\235\1F\5xterm|xterm terminal "..., 4097) = 3220
18:31:02.166646 close(3) = 0
18:31:02.166709 ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
18:31:02.166788 ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
18:31:02.166866 ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
18:31:02.166941 ioctl(1, TIOCGWINSZ, {ws_row=24, ws_col=80, ws_xpixel=499, ws_ypixel=316}) = 0
18:31:02.167010 fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
18:31:02.167134 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f38000
18:31:02.167205 write(1, "\33[H\33[2J"..., 7) = 7
18:31:02.167613 exit_group(0) = ?

Satish

Signature · 1 line
http://satish.playdrupal.com
#30 lore raw
Quoted 6 lines · Satish Eerpini
Quoted 3 lines

Or maybe the time is spent in gnome-terminal or bash? What if you change them to xterm and dash?

ok here you go , I ran "strace -o log -tt clear " on xterm and dash :

"log "

Again, it's pretty fast:

begin with:

Quoted 1 line

18:31:02.162301 execve("/usr/bin/clear", ["clear"], [/* 49 vars */]) = 0

end with:

Quoted 1 line

18:31:02.167613 exit_group(0) = ?

So the elapsed time for "clear" is about 5ms.

#31 lore raw
Quoted 5 lines
Quoted 4 lines
Quoted 1 line

ok here you go , I ran "strace -o log -tt clear " on xterm and dash :

"log "

Quoted 8 lines

Again, it's pretty fast:
begin with:

Quoted 1 line

18:31:02.162301 execve("/usr/bin/clear", ["clear"], [/* 49 vars */]) = 0

end with:

Quoted 1 line

18:31:02.167613 exit_group(0)           = ?

So the elapsed time for "clear" is about 5ms.

right, I could not generate the lag in this case, as I said , I am not able to generate the same unresponsiveness deterministically ,
.......

Satish

Signature · 1 line
http://satish.playdrupal.com
#32 lore raw
In reply to Wu Fengguang
Quoted 93 lines · you
Quoted 6 lines · Satish Eerpini
Quoted 4 lines

I've prepared a rolled up patch for you, run time tested.  It should protect all of the above pages from being evicted by large file copies.

The attached patch is for 2.6.29.

I applied the patch , but the number don seem to show much difference

The numbers listed in your email? No the most relevant numbers are memory and disk ones.

For my part, the number of mapped pages keeps stable when there is an ongoing background file copy:

% ls -l /b/sparse
-rw-r--r-- 1 root root 98T 2009-04-29 22:38 /b/sparse
% cp /b/sparse /dev/null&
% for i in seq 1 100; do grep Mapped /proc/meminfo; sleep 1; done

Mapped:            22764 kB
Mapped:            22796 kB
Mapped:            22756 kB
Mapped:            22716 kB
Mapped:            22800 kB
Mapped:            22788 kB
Mapped:            22804 kB
Mapped:            22808 kB
Mapped:            22748 kB
Mapped:            22708 kB
Mapped:            22916 kB
Mapped:            22944 kB
Mapped:            22944 kB
Mapped:            22944 kB
Mapped:            22944 kB
Mapped:            22944 kB
Mapped:            22832 kB
Mapped:            22812 kB
Mapped:            22812 kB
Mapped:            22792 kB
Mapped:            22772 kB
Mapped:            22860 kB
Mapped:            22860 kB
Mapped:            22748 kB
Mapped:            22808 kB
Mapped:            22868 kB
Mapped:            22956 kB
Mapped:            22956 kB
Mapped:            22832 kB
Mapped:            22832 kB
Mapped:            22980 kB
Mapped:            22980 kB
Mapped:            22980 kB
Mapped:            22980 kB
Mapped:            22872 kB
Mapped:            22872 kB
Mapped:            22900 kB
Mapped:            22792 kB
Mapped:            22920 kB
Mapped:            22812 kB
Mapped:            22812 kB
Mapped:            22752 kB
Mapped:            22864 kB
Mapped:            22972 kB
Mapped:            22860 kB
Mapped:            22796 kB
Mapped:            22900 kB
Mapped:            22888 kB
Mapped:            22888 kB
Mapped:            22888 kB
Mapped:            22764 kB
Quoted 5 lines

, following are the statistics with the patched kernel, could something else be causing the huge iowait, :

Linux 2.6.29 (satish) 05/17/2009

Quoted 7 lines
09:57:14 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
09:57:18 AM     all     12.93      0.00     10.25     33.44      0.00     43.38
09:57:21 AM     all     13.32      0.00     28.09     24.40      0.00     34.19
09:57:24 AM     all     10.79      0.00      4.76     75.56      0.00      8.89
09:57:26 AM     all     11.46      0.00      8.01     35.64      0.00     44.90
09:57:30 AM     all     11.68      0.00      8.88     35.05      0.00     44.39
Average:        all     12.03      0.00     11.94     40.81      0.00     35.22

The iowait is high. Do you have "iostat -x 5' numbers?

Quoted 7 lines

and

hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 1332 MB in 2.00 seconds = 666.35 MB/sec
Timing buffered disk reads: 24 MB in 3.32 seconds = 7.23 MB/sec

Quoted 8 lines

That's pretty slow numbers. On my laptop:

# hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 10266 MB in 1.98 seconds = 5188.46 MB/sec
Timing buffered disk reads: 138 MB in 3.01 seconds = 45.90 MB/sec

I see (on 2.6.30-rc6, no patches):

# hdparm -tT /dev/sda (The internal, low-power, low performance hdd):

/dev/sda:
Timing cached reads: 492 MB in 2.00 seconds = 245.83 MB/sec
Timing buffered disk reads: 74 MB in 3.05 seconds = 24.26 MB/sec

# hdparm -tT /dev/sdb (A Class-6, SDHC card):

/dev/sdb:
Timing cached reads: 472 MB in 2.01 seconds = 235.28 MB/sec
Timing buffered disk reads: 40 MB in 3.13 seconds = 12.79 MB/sec

And the only thing going on -
(once you deduct the effect of getting this output) - nothing:

top - 08:09:25 up 9 min, 1 user, load average: 0.24, 0.40, 0.25

Tasks:  86 total,   1 running,  85 sleeping,   0 stopped,   0 zombie
Cpu(s):  2.9%us,  4.2%sy,  0.8%ni, 59.9%id, 32.2%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:    447040k total,   242996k used,   204044k free,    73516k buffers
Swap:  2199324k total,        0k used,  2199324k free,    79868k cached
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 3359 root      20   0  2444 1048  824 R  3.8  0.2   0:00.04 top
 3360 root      20   0  3376  888  744 S  1.9  0.2   0:00.01 less
    1 root      20   0  3084 1884  564 S  0.0  0.4   0:01.59 init
    2 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kthreadd

Mike

Quoted 9 lines

Thanks,
Fengguang
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#33 lore raw
In reply to Ray Lee
Quoted 5 lines

Which kernel version? If it's something recent, your programs may be getting swapped out due to a flaw in the latest kernels, where it accidentally prefers caching file data over mapped pages. If that's the case, Rik (cc:d) is working on patches to address the issue, and may appreciate another tester.

this may be the reason, I have seen this lag only with kernel > 2.6.28, currently I am running vanilla kernel 2.6.29.1 .

Satish

Signature · 1 line
http://satish.playdrupal.com
#34 lore raw
In reply to Ray Lee
Quoted 5 lines

Which kernel version? If it's something recent, your programs may be getting swapped out due to a flaw in the latest kernels, where it accidentally prefers caching file data over mapped pages. If that's the case, Rik (cc:d) is working on patches to address the issue, and may appreciate another tester.

well I went through http://kerneltrap.org/node/3000 on kerneltrap , which I think is what you are pointing out here. Well I checked the "swappiness" parameter on my system and it is set to 60 , and bravely I changed it to '0', while the system appeared to have become a lot more responsive, I see no difference in the statistics:

Linux 2.6.29.1 (satish) 05/16/2009

07:49:43 PM CPU %user %nice %system %iowait %steal %idle
07:49:46 PM all 7.30 0.00 8.25 40.00

    0.00     44.44
07:49:49 PM     all      7.21      0.00            9.40          36.36
     0.00     47.02
07:49:52 PM     all      8.62      0.00            9.40          34.80
     0.00     47.18
07:49:55 PM     all      7.26      0.00            8.36          37.07
     0.00     47.32
07:49:58 PM     all      7.89      0.00           9.15           36.59
     0.00     46.37

Average: all 7.66 0.00 8.92
36.96 0.00 46.47

so there is still a huge IOWAIT , ..

Cheers
Satish

Signature · 1 line
http://satish.playdrupal.com