Optimization for use-once pages
Subsystems (4)
[ CC: to linux-kernel added back since the topic isn't flammable ;) ]
Quoted 8 lines · Stuart MacDonald
From: "Rik van Riel" <[email protected]>
Quoted 3 lines
If a page gets 2 1-byte reads in a microsecond, with this patch it would get promoted to the active list, even though it's really only used once.
I hate to bother you directly, but I don't wish to start a flame war on lkml. How exactly would you explain two accesses as being "used once"?
Because they occur in a very short interval, an interval MUCH shorter than the time scale in which the VM subsystem looks at referenced bits, etc...
Generally a CPU doesn't read more than one cache line at a time, so I guess all "single references" are in reality multiple accesses very shortly following each other.
This seems to be generally accepted theory and practice in the field of page replacement.
regards,
Rik
--
Executive summary of a recent Microsoft press release:
"we are concerned about the GNU General Public License (GPL)"
http://www.surriel.com/
From: "Rik van Riel" <[email protected]>
Quoted 3 lines
Because they occur in a very short interval, an interval MUCH shorter than the time scale in which the VM subsystem looks at referenced bits, etc...
So what you're sayng is that any number of accesses, as long as they all occur within the interval < VM subsystem scanning interval, are all counted as one?
Quoted 2 lines
This seems to be generally accepted theory and practice in the field of page replacement.
Okay, just seems odd to me, but IANAVMGuru.
..Stu
Quoted 7 lines · Rik van Riel
Quoted 3 lines
I hate to bother you directly, but I don't wish to start a flame war on lkml. How exactly would you explain two accesses as being "used once"?
Because they occur in a very short interval, an interval MUCH shorter than the time scale in which the VM subsystem looks at referenced bits, etc...
Would mmap() then a for(;;) loop over the range be an example of such a use?
mrc
Signature · 3 lines
Mike Castle [email protected] www.netcom.com/~dalgoda/ We are all of us living in the shadow of Manhattan. -- Watchmen fatal ("You are in a maze of twisty compiler features, all different"); -- gcc
Quoted 8 lines · Stuart MacDonald
From: "Rik van Riel" <[email protected]>
Quoted 3 lines
Because they occur in a very short interval, an interval MUCH shorter than the time scale in which the VM subsystem looks at referenced bits, etc...
So what you're sayng is that any number of accesses, as long as they all occur within the interval < VM subsystem scanning interval, are all counted as one?
Actually, the length of this interval could be even smaller and is often a point of furious debating.
The 2Q algorithm seems to have solved this problem by not using an interval, but a FIFO queue of small, fixed length.
Quoted 4 lines
Quoted 2 lines
This seems to be generally accepted theory and practice in the field of page replacement.
Okay, just seems odd to me, but IANAVMGuru.
Seems odd at first glance, true.
Let me give you an example:
- sequential access of a file
- script reads the file in 80-byte segments (parsing some arcane data structure)
- these segments are accessed in rapid succession
- each 80-byte segment is accessed ONCE
In this case, even though the data is accessed only once, each page is touched PAGE_SIZE/80 times, with one 80-byte read() each time.
I hope this example gives you some holding point to make it easier and grasp this - somewhat counter intuitive - concept.
regards,
Rik
--
Executive summary of a recent Microsoft press release:
"we are concerned about the GNU General Public License (GPL)"
http://www.surriel.com/
Quoted 11 lines · Mike Castle
Quoted 7 lines · Rik van Riel
Quoted 3 lines
I hate to bother you directly, but I don't wish to start a flame war on lkml. How exactly would you explain two accesses as being "used once"?
Because they occur in a very short interval, an interval MUCH shorter than the time scale in which the VM subsystem looks at referenced bits, etc...
Would
mmap()then a for(;;) loop over the range be an example of such a use?
Yes. The problem here, however, would be that we'll only find the referenced bit in the page table some time later.
This is another reason for having a "new" queue like 2Q's A1in queue. It means we can both count all accesses in the first short period as one access AND we can avoid actually doing anything special as these accesses happen....
regards,
Rik
--
Executive summary of a recent Microsoft press release:
"we are concerned about the GNU General Public License (GPL)"
http://www.surriel.com/
From: "Rik van Riel" <[email protected]>
Quoted 2 lines
Actually, the length of this interval could be even smaller and is often a point of furious debating.
Which is why I was avoiding flames earlier; I sorta figured this might be a hot issue.
Quoted 11 lines
Let me give you an example:
- sequential access of a file
- script reads the file in 80-byte segments (parsing some arcane data structure)
- these segments are accessed in rapid succession
- each 80-byte segment is accessed ONCEIn this case, even though the data is accessed only once, each page is touched PAGE_SIZE/80 times, with one 80-byte
read()each time.
I'd figured that sort of scenario was the basis for counting them all as one. What about
- random access of same file
- script reads one arcane 80 byte struct
- script updates that struct say PAGE_SIZE/80
times, with one 80-byte write() each time
If they're all counted as one, would the page age correctly, if the script happens to take a few seconds break between another flurry of all-as-one updating?
..Stu
Quoted 11 lines · Rik van Riel
Let me give you an example:
- sequential access of a file
- script reads the file in 80-byte segments
(parsing some arcane data structure)
- these segments are accessed in rapid succession
- each 80-byte segment is accessed ONCEIn this case, even though the data is accessed only once, each page is touched PAGE_SIZE/80 times, with one 80-byte
read()each time.
Of course, such a program is poorly written anyway. That many individual system calls! At least rewrite to use fread(). ;->
Change the example to using mmap(), and one feels less likely throttle the programmer.
mrc
Signature · 3 lines
Mike Castle [email protected] www.netcom.com/~dalgoda/ We are all of us living in the shadow of Manhattan. -- Watchmen fatal ("You are in a maze of twisty compiler features, all different"); -- gcc
Quoted 5 lines · Mike Castle
Quoted 1 line · Rik van Riel
Let me give you an example:
Of course, such a program is poorly written anyway. That many individual system calls!
That's not the point I was trying to make ;))
Rik
--
Executive summary of a recent Microsoft press release:
"we are concerned about the GNU General Public License (GPL)"
http://www.surriel.com/
adaptec aha152x.o stops working with AVA-1505 on 2.4.7
I compiled aha152x as a module and usually have it load at boot. However, with kernel 2.4.7, the machine hangs in some in-between state when the module is loaded. No oops or panic is printed -- the machine simply hangs and does nothing. The keyboard is responsive however but it appears as though nothing gets through to the kernel.
Furthermore, nothing is printed in the logs about any such error. I'm wondering if this is somehow related to the recent SCSI layer cleanups?
The machine is an SMP Pentium III with 640 megabytes of RAM. If you need any more details, e.g. .config, lspci, etc. let me know and I'll provide them. For the moment, I have removed the controller card from the machine simply because it was pissing me off to no extent and I didn't want to deal with it any longer. Fortunately, the only device connected to it is an optical disk drive that gets very infrequent use, so I can live without it, for a while.
LMK what you come up with. TIA,
Kelsey Hudson [email protected]
Software Engineer
Compendium Technologies, Inc (619) 725-0771
---------------------------------------------------------------------------
Patch for better ip_dynaddr for 2.2.19
In reply to Rik van RielI'm sorry that it is for old 2.2.x but I have 2.2.19 router as MASQ box for our microwave line. I setup automatic backup via ISDN but I find that masq connections stops working when ISDN gets dialed.
It is because routing is changed and maddr field is bad in ip_masq struct.
There is existing solution: sysctl_ip_dynaddr. But it replaces maddr only on connections where no reply was recieved from outside. It is true for machine connected via diald for example but when changing LIVE route it simply doesn't work.
I patched 3 files with very short patch. It allows you to add 4 to sysctl_ip_dynaddr flag and it skips no-reply test.
It works well now.
Patched version is bacward compatible and there is no speed penalty (well almost).
What about 2.2.20 inclusion ? It this correct place to send patch to ?
devik
--------
net/ipv4/ip_masq.c +7 -6
@@ -50,6 +50,7 @@* Kai Bankett : do not toss other IP protos in proto_doff() * Dan Kegel : pointed correct NAT behavior for UDP streams * Julian Anastasov : use daddr and dport as hash keys +* Martin Devera : extended sysctl_ip_dynaddr functionality * */
@@ -1236,9 +1237,9 @@* in this tunnel and routing iface address has changed... * "You are welcome, diald". */ -if ( sysctl_ip_dynaddr && ms->flags &IP_MASQ_F_NO_REPLY && maddr != ms->maddr) { - -if (sysctl_ip_dynaddr >1 ) { +if ( sysctl_ip_dynaddr && (sysctl_ip_dynaddr &4 || + ms->flags &IP_MASQ_F_NO_REPLY ) && maddr != ms->maddr) { +if ((sysctl_ip_dynaddr &3 ) >1 ) {IP_MASQ_INFO ("ip_fw_masquerade(): change masq.addr from %d.%d.%d.%d to %d.%d.%d.%d\n" ,NIPQUAD (ms->maddr),NIPQUAD (maddr)); }
@@ -1527,9 +1528,9 @@* in this tunnel and routing iface address has changed... * "You are welcome, diald". */ -if ( sysctl_ip_dynaddr && ms->flags &IP_MASQ_F_NO_REPLY && maddr != ms->maddr) { - -if (sysctl_ip_dynaddr >1 ) { +if ( sysctl_ip_dynaddr && (sysctl_ip_dynaddr &4 || + ms->flags &IP_MASQ_F_NO_REPLY ) && maddr != ms->maddr) { +if ((sysctl_ip_dynaddr &3 ) >1 ) {IP_MASQ_INFO ("ip_fw_masq_icmp(): change masq.addr %d.%d.%d.%d to %d.%d.%d.%d" ,NIPQUAD (ms->maddr),NIPQUAD (maddr)); }
net/ipv4/tcp_ipv4.c +1 -1
@@ -1889,7 +1889,7 @@ }if (new_saddr != sk->saddr) { -if (sysctl_ip_dynaddr >1 ) { +if ((sysctl_ip_dynaddr &3 ) >1 ) { printk(KERN_INFO "tcp_v4_rebuild_header(): shifting sk->saddr " "from %d.%d.%d.%d to %d.%d.%d.%d\n", NIPQUAD(sk->saddr),
Documentation/networking/ip_dynaddr.txt +6 -0
@@ -23,6 +23,12 @@ # echo 2 > /proc/sys/net/ipv4/ip_dynaddr To disable (default) # echo 0 > /proc/sys/net/ipv4/ip_dynaddr + To always rewrite MASQ connections instead only when no reply + packet is recieved (needed to be able to change routing, for + example when doing backup-line) add 4 to flag, like: + # echo 5 > /proc/sys/net/ipv4/ip_dynaddr + or + # echo 6 > /proc/sys/net/ipv4/ip_dynaddr Enjoy!
---------------
Question about termios parameters
This may not be the best place to ask this, but I've done the research,
can't come up with an answer, and don't know a better group of people to ask.
I have an embedded Linux device (2.2.12 kernel, BlueCat distro) that uses a
serial port for the console. When the box comes up, rc.sysinit starts the application as a detached process (my_program&). When the program spits out periodic status reports, the \n is not being mapped to CR-LF (i.e., I'm getting only linefeeds).
Once you log on to the box (via login, into bash), the output becomes
correctly cooked.
I've tried twiddling termios parameters for OPOST and ONLCR, but it has no
effect. Trying to have the application run "stty -a" via a system() call reports an error regarding it can't get the parameters for stdin.
What parameters are required to be set for a detached process started via
init to correctly have it's output mapped from \n to CR-NL?
--John