User applications
I have a few questions about user applications.
When a new user process is started will its user space be cleared by the kernel or is there a potential leak from an older user process ? What about the registers values, are they cleared for each new user application or will it simply contain the current value it got when the user application is started ?
How can you flush the data and instruction cashes from a user application ?
/Carsten
--
_ _ ____ ___ Carsten Langgaard Mailto:[email protected] |\ /|||___)(___ MIPS Denmark Direct: +45 4486 5527 | \/ ||| ____) Lautrupvang 4B Switch: +45 4486 5555 TECHNOLOGIES 2750 Ballerup Fax...: +45 4486 5556 Denmark http://www.mips.com
Quoted 4 lines · Carsten Langgaard
I have a few questions about user applications.
When a new user process is started will its user space be cleared by the kernel or is there a potential leak from an older user process ?
Usually it is defied by the loader. If the data section contents is set to LOAD, then the contents of the section will be loaded from disk (no leak), if not -- whatever values left i nmemory will be there, or exactly, the virtual page of some other proccess that was swapped out or ended.
Quoted 3 lines
What about the registers values, are they cleared for each new user application or will it simply contain the current value it got when the user application is started ?
It depends on the context switch algorithm of the processor, I think.
Quoted 3 lines
How can you flush the data and instruction cashes from a user application ?
As far as I understand, ASID must take care of it. It contains unique IDs per process virtual space, so that even
though virtual addresses may be found in TLB, their ASID will be different, causing TLB miss and probably page fault.
Quoted 9 lines
/Carsten
--
_ _ ____ ___ Carsten Langgaard Mailto:[email protected] |\ /|||___)(___ MIPS Denmark Direct: +45 4486 5527 | \/ ||| ____) Lautrupvang 4B Switch: +45 4486 5555 TECHNOLOGIES 2750 Ballerup Fax...: +45 4486 5556 Denmark http://www.mips.com
Michael.
Quoted 18 lines · Michael Shmulevich
Quoted 4 lines · Carsten Langgaard
I have a few questions about user applications.
When a new user process is started will its user space be cleared by the kernel or is there a potential leak from an older user process ?
Usually it is defied by the loader. If the data section contents is set to LOAD, then the contents of the section will be loaded from disk (no leak), if not -- whatever values left i nmemory will be there, or exactly, the virtual page of some other proccess that was swapped out or ended.
Quoted 3 lines
What about the registers values, are they cleared for each new user application or will it simply contain the current value it got when the user application is started ?
It depends on the context switch algorithm of the processor, I think.
Quoted 9 lines
Quoted 3 lines
How can you flush the data and instruction cashes from a user application ?
As far as I understand, ASID must take care of it. It contains unique IDs per process virtual space, so that even
though virtual addresses may be found in TLB, their ASID will be different, causing TLB miss and probably page fault.
My problem is that I want to make self-modified code, so I need to flush both the instruction and data cache.
Quoted 4 lines
Quoted 4 lines
/Carsten
Quoted 8 lines
Quoted 6 lines
--
_ _ ____ ___ Carsten Langgaard Mailto:[email protected] |\ /|||___)(___ MIPS Denmark Direct: +45 4486 5527 | \/ ||| ____) Lautrupvang 4B Switch: +45 4486 5555 TECHNOLOGIES 2750 Ballerup Fax...: +45 4486 5556Denmark http://www.mips.com
Michael.
--
_ _ ____ ___ Carsten Langgaard Mailto:[email protected] |\ /|||___)(___ MIPS Denmark Direct: +45 4486 5527 | \/ ||| ____) Lautrupvang 4B Switch: +45 4486 5555 TECHNOLOGIES 2750 Ballerup Fax...: +45 4486 5556 Denmark http://www.mips.com
Quoted 7 lines
Quoted 2 lines
When a new user process is started will its user space be cleared by the kernel or is there a potential leak from an older user process ?
Usually it is defied by the loader. If the data section contents is set to LOAD, then the contents of the section will be loaded from disk (no leak), if not -- whatever values left i nmemory will be there, or exactly, the virtual page of some other proccess that was swapped out or ended.
Note that what you are describing here is the "exec()" behavior. I believe Carsten was talking about what happens on a "fork()".
Quoted 5 lines
Quoted 3 lines
What about the registers values, are they cleared for each new user application or will it simply contain the current value it got when the user application is started ?
It depends on the context switch algorithm of the processor, I think.
On a fork() (or presumably clone()) operation, the set of registers is copied. Loading a new program ("exec()") should set up the registers that point to the base of the new stack, the environment, etc. Historically, it's up to the runtime startup code ("crt0" in old Unix systems) to do any other register initialization.
Quoted 7 lines
Quoted 3 lines
How can you flush the data and instruction cashes from a user application ?
As far as I understand, ASID must take care of it. It contains unique IDs per process virtual space, so that even
though virtual addresses may be found in TLB, their ASID will be
different,
Quoted 1 line
causing TLB miss and probably page fault.
That won't necessarily affect the caches, though. While it would be possible to do so, I don't believe any existing MIPS implementations include ASID in the cache tags. Hits are determined by an address match, period.
Back in the Ancient Old Days of System V, every architecture had an architecture-specific system call entry, the first parameter of which expressed what needed to be done. Do we have
such a thing in Linux? That would be the logical place to
things like cache flush and the atomic operations that were being discussed here a couple of weeks ago.
Regards,
Kevin K.
Quoted 38 lines · Kevin D. Kissell
Quoted 7 lines
Quoted 2 lines
When a new user process is started will its user space be cleared by the kernel or is there a potential leak from an older user process ?
Usually it is defied by the loader. If the data section contents is set to LOAD, then the contents of the section will be loaded from disk (no leak), if not -- whatever values left i nmemory will be there, or exactly, the virtual page of some other proccess that was swapped out or ended.
Note that what you are describing here is the "
exec()" behavior. I believe Carsten was talking about what happens on a "fork()".Quoted 5 lines
Quoted 3 lines
What about the registers values, are they cleared for each new user application or will it simply contain the current value it got when the user application is started ?
It depends on the context switch algorithm of the processor, I think.
On a
fork()(or presumablyclone()) operation, the set of registers is copied. Loading a new program ("exec()") should set up the registers that point to the base of the new stack, the environment, etc. Historically, it's up to the runtime startup code ("crt0" in old Unix systems) to do any other register initialization.Quoted 7 lines
Quoted 3 lines
How can you flush the data and instruction cashes from a user application ?
As far as I understand, ASID must take care of it. It contains unique IDs per process virtual space, so that even
though virtual addresses may be found in TLB, their ASID will bedifferent,
Quoted 1 line
causing TLB miss and probably page fault.
That won't necessarily affect the caches, though. While it would be possible to do so, I don't believe any existing MIPS implementations include ASID in the cache tags. Hits are determined by an address match, period.
Quoted 6 lines
Back in the Ancient Old Days of System V, every architecture had an architecture-specific system call entry, the first parameter of which expressed what needed to be done. Do we have
such a thing in Linux? That would be the logical place to
things like cache flush and the atomic operations that were being discussed here a couple of weeks ago.
I think I just found it.
The system call is sysmips(FLUSH_CACHE).
Quoted 4 lines
Regards,
Kevin K.
--
_ _ ____ ___ Carsten Langgaard Mailto:[email protected] |\ /|||___)(___ MIPS Denmark Direct: +45 4486 5527 | \/ ||| ____) Lautrupvang 4B Switch: +45 4486 5555 TECHNOLOGIES 2750 Ballerup Fax...: +45 4486 5556 Denmark http://www.mips.com
Quoted 2 lines · Carsten Langgaard
I think I just found it.
The system call issysmips(FLUSH_CACHE).
Don't. Sysmips(FLUSH_CACHE, ...) only allows very coarse flush operation, that is flushing all caches. The whole sysmips(2) call exists in Linux only as a stone age compatibility thing.
Ralf
Quoted 6 lines · Kevin D. Kissell
Back in the Ancient Old Days of System V, every architecture had an architecture-specific system call entry, the first parameter of which expressed what needed to be done. Do we have
such a thing in Linux? That would be the logical place to
things like cache flush and the atomic operations that were being discussed here a couple of weeks ago.
The only case caches need to be synchronized is modifying some code. The
ptrace syscall does it automatically for text writes -- it's needed and used by gdb to set breakpoints, for example. For other code there is cacheflush() which allows you to flush a cache range relevant to a given virtual address (I see it's not implemented very well at the moment).
Obviously, you don't want to allow unprivileged users to flush caches as
a whole as it could lead to a DoS.
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
----- Original Message -----
From: "Maciej W. Rozycki" <[email protected]>
To: "Kevin D. Kissell" <[email protected]>
Cc: <[email protected]>; "Carsten Langgaard" <[email protected]>; "Michael Shmulevich" <[email protected]>
Sent: Monday, January 08, 2001 4:07 PM
Subject: Re: User applications
Quoted 10 lines · Kevin D. Kissell
Quoted 7 lines
Back in the Ancient Old Days of System V, every architecture had an architecture-specific system call entry, the first parameter of which expressed what needed to be done. Do we have
such a thing in Linux? That would be the logical place to
things like cache flush and the atomic operations that were being discussed here a couple of weeks ago.The only case caches need to be synchronized is modifying some code.
Which just happens to be the case under discussion here.
Quoted 1 line
The ptrace syscall does it automatically for text writes -- it's needed
and
Quoted 6 lines
used by gdb to set breakpoints, for example. For other code there is
cacheflush()which allows you to flush a cache range relevant to a given virtual address (I see it's not implemented very well at the moment).Obviously, you don't want to allow unprivileged users to flush caches as
a whole as it could lead to a DoS.
By that logic, we should not allow users to allocate more virtual memory than there is physical memory in the system! A pathological swap program is arguably far a far worse denial of service attack than flushing the caches - so long as by "flush" we mean invalidate with writeback (on copyback caches), of course.
Quoted 6 lines · Kevin D. Kissell
Quoted 2 lines
Obviously, you don't want to allow unprivileged users to flush caches as
a whole as it could lead to a DoS.By that logic, we should not allow users to allocate more virtual memory than there is physical memory in the system! A pathological swap program is arguably far a far worse denial of service attack
There are limits -- see `info setrlimit'. There is no way to prevent a
program from executing:
while (1) flush_cache_all();
though but the system's performance would suffer much. Remember there is real world out there...
Which means sysmips(FLUSH_CACHE, ...) needs to be fixed or removed.
Quoted 2 lines
than flushing the caches - so long as by "flush" we mean invalidate with writeback (on copyback caches), of course.
What's wrong with cacheflush(addr, count, which) that actually checks if
<addr; addr+count> lies within the caller's address space before performing the flush and returns -EPERM otherwise? It would make the caller crawl like a turtle if it wished to but it would leave other processes alone.
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
Quoted 8 lines · Maciej W. Rozycki
Quoted 2 lines
than flushing the caches - so long as by "flush" we mean invalidate with writeback (on copyback caches), of course.
What's wrong with
cacheflush(addr, count, which)that actually checks if
<addr; addr+count> lies within the caller's address space before performing the flush and returns-EPERMotherwise? It would make the caller crawl like a turtle if it wished to but it would leave other processes alone.
cacheflush(2) actually is supposed to handle things that way.
Ralf
Quoted 7 lines · Ralf Baechle
Quoted 5 lines
What's wrong with
cacheflush(addr, count, which)that actually checks if
<addr; addr+count> lies within the caller's address space before performing the flush and returns-EPERMotherwise? It would make the caller crawl like a turtle if it wished to but it would leave other processes alone.cacheflush(2) actually is supposed to handle things that way.
Didn't I write it clearly-enough? ;-)
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
Quoted 9 lines · Maciej W. Rozycki
Quoted 7 lines
Quoted 5 lines
What's wrong with
cacheflush(addr, count, which)that actually checks if
<addr; addr+count> lies within the caller's address space before performing the flush and returns-EPERMotherwise? It would make the caller crawl like a turtle if it wished to but it would leave other processes alone.cacheflush(2) actually is supposed to handle things that way.
Didn't I write it clearly-enough? ;-)
return -EOVERBLOODINCAFFEIN;
Ralf
Quoted 8 lines · Maciej W. Rozycki
The only case caches need to be synchronized is modifying some code. The
ptrace syscall does it automatically for text writes -- it's needed and used by gdb to set breakpoints, for example. For other code there iscacheflush()which allows you to flush a cache range relevant to a given virtual address (I see it's not implemented very well at the moment).Obviously, you don't want to allow unprivileged users to flush caches as
a whole as it could lead to a DoS.
You obviously want to allow partial cache flushes or trampolines don't work and flushing the entire cache can be constructed from that.
Ralf
Quoted 8 lines · Ralf Baechle
Quoted 6 lines · Maciej W. Rozycki
The only case caches need to be synchronized is modifying some code. The
ptrace syscall does it automatically for text writes -- it's needed and used by gdb to set breakpoints, for example. For other code there iscacheflush()which allows you to flush a cache range relevant to a given virtual address (I see it's not implemented very well at the moment).
Quoted 6 lines
Quoted 2 lines
Obviously, you don't want to allow unprivileged users to flush caches as
a whole as it could lead to a DoS.You obviously want to allow partial cache flushes or trampolines don't work and flushing the entire cache can be constructed from that.
What we need is a mechanism to partial invalidate the I-cache and a mechanism to write-back and/or invalidate the D-cache.
/Carsten
--
_ _ ____ ___ Carsten Langgaard Mailto:[email protected] |\ /|||___)(___ MIPS Denmark Direct: +45 4486 5527 | \/ ||| ____) Lautrupvang 4B Switch: +45 4486 5555 TECHNOLOGIES 2750 Ballerup Fax...: +45 4486 5556 Denmark http://www.mips.com
Quoted 2 lines · Carsten Langgaard
What we need is a mechanism to partial invalidate the I-cache and a mechanism to write-back and/or invalidate the D-cache.
There is this nice little man page which should even be installed on your Linux/Inhell box:
CACHEFLUSH(2) Linux Programmer's Manual CACHEFLUSH(2)
NAME
cacheflush - flush contents of instruction and/or data cache
SYNOPSIS
#include <asm/cachectl.h>
int cacheflush(char *addr, int nbytes, int cache);DESCRIPTION
cacheflush flushes contents of indicated cache(s) for user addresses in the range addr to (addr+nbytes-1). Cache may be one of:
ICACHE Flush the instruction cache.
DCACHE Write back to memory and invalidate the affected
valid cache lines.
BCACHE Same as (ICACHE|DCACHE).
RETURN VALUE
cacheflush returns 0 on success or -1 on error. If errors are detected, errno will indicate the error.
ERRORS
EINVAL cache parameter is not one of ICACHE, DCACHE, or
BCACHE.
EFAULT Some or all of the address range addr to
(addr+nbytes-1) is not accessible.
BUGS
The current implementation ignores the addr and nbytes
parameters. Therefore always the whole cache is flushed.
NOTE
This system call is only available on MIPS based systems. It should not be used in programs intended to be portable.
Linux 2.0.32 27 June 95 1
Exactly that I need, but I don't think it is implemented properly for mips. It simply flushes all the caches, no matter what options you gives it.
/Carsten
Quoted 46 lines · Ralf Baechle
Quoted 2 lines · Carsten Langgaard
What we need is a mechanism to partial invalidate the I-cache and a mechanism to write-back and/or invalidate the D-cache.
There is this nice little man page which should even be installed on your Linux/Inhell box:
CACHEFLUSH(2) Linux Programmer's Manual CACHEFLUSH(2)
NAME
cacheflush - flush contents of instruction and/or data cacheSYNOPSIS
#include <asm/cachectl.h> int cacheflush(char *addr, int nbytes, int cache);DESCRIPTION
cacheflush flushes contents of indicated cache(s) for user addresses in the range addr to (addr+nbytes-1). Cache may be one of:ICACHE Flush the instruction cache.
DCACHE Write back to memory and invalidate the affected
valid cache lines.BCACHE Same as (ICACHE|DCACHE).
RETURN VALUE
cacheflush returns 0 on success or -1 on error. If errors are detected, errno will indicate the error.ERRORS
EINVAL cache parameter is not one of ICACHE, DCACHE, or
BCACHE.EFAULT Some or all of the address range addr to
(addr+nbytes-1) is not accessible.BUGS
The current implementation ignores the addr and nbytes
parameters. Therefore always the whole cache is flushed.
Quoted 5 lines
NOTE
This system call is only available on MIPS based systems. It should not be used in programs intended to be portable.Linux 2.0.32 27 June 95 1
--
_ _ ____ ___ Carsten Langgaard Mailto:[email protected] |\ /|||___)(___ MIPS Denmark Direct: +45 4486 5527 | \/ ||| ____) Lautrupvang 4B Switch: +45 4486 5555 TECHNOLOGIES 2750 Ballerup Fax...: +45 4486 5556 Denmark http://www.mips.com
Quoted 2 lines · Carsten Langgaard
Exactly that I need, but I don't think it is implemented properly for mips. It simply flushes all the caches, no matter what options you gives it.
Yep, it's just a limitation of the implementation. But it's the right
function and now that I know of the problem it goes to my to do list (but processing of the list is not that fast these days, so anyone feel free to do it oneself).
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
Quoted 2 lines · Carsten Langgaard
What we need is a mechanism to partial invalidate the I-cache and a mechanism to write-back and/or invalidate the D-cache.
What's wrong with the already mentioned cachectl?
$ mipsel-linux-objdump -T /usr/mipsel-linux/lib/libc-2.2.so | grep cachectl 00000000600ca0a0 w DF *ABS* 0000000000000000 GLIBC_2.0 cachectl $ ls /usr/mipsel-linux/include/sys/cachectl.h /usr/mipsel-linux/include/sys/cachectl.h
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
Quoted 9 lines · Maciej W. Rozycki
Quoted 2 lines
What we need is a mechanism to partial invalidate the I-cache and a mechanism to write-back and/or invalidate the D-cache.
What's wrong with the already mentioned cachectl?
$ mipsel-linux-objdump -T /usr/mipsel-linux/lib/libc-2.2.so | grep cachectl 00000000600ca0a0 w DF *ABS* 0000000000000000 GLIBC_2.0 cachectl $ ls /usr/mipsel-linux/include/sys/cachectl.h /usr/mipsel-linux/include/sys/cachectl.h
cachectl(2) is a syscall that is manipulates the cachability of a memory area. And not yet implemented ...
Ralf
Quoted 7 lines · Ralf Baechle
Quoted 4 lines
$ mipsel-linux-objdump -T /usr/mipsel-linux/lib/libc-2.2.so | grep cachectl 00000000600ca0a0 w DF *ABS* 0000000000000000 GLIBC_2.0 cachectl $ ls /usr/mipsel-linux/include/sys/cachectl.h /usr/mipsel-linux/include/sys/cachectl.hcachectl(2) is a syscall that is manipulates the cachability of a memory area. And not yet implemented ...
s/cachectl$/cacheflush/, of course (but the header is still valid).
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
As a side question, I would like to to know why exactly the CPU cache operations are
promoted to the syscall status? What is the situation that a user in its program would like
to call cacheflush() ? Unless, of course, he is doing DoS.
I can understand why we need this in kernel, for context switch, for example, but as a syscall?...
Michael.
Quoted 16 lines · Maciej W. Rozycki
Quoted 7 lines · Ralf Baechle
Quoted 4 lines
$ mipsel-linux-objdump -T /usr/mipsel-linux/lib/libc-2.2.so | grep cachectl 00000000600ca0a0 w DF *ABS* 0000000000000000 GLIBC_2.0 cachectl $ ls /usr/mipsel-linux/include/sys/cachectl.h /usr/mipsel-linux/include/sys/cachectl.hcachectl(2) is a syscall that is manipulates the cachability of a memory area. And not yet implemented ...
s/cachectl$/cacheflush/, of course (but the header is still valid).
--
(quoted diff) +3
-0+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
Quoted 3 lines
promoted to the syscall status? What is the situation that a user in its program would like
to callcacheflush()? Unless, of course, he is doing DoS.
A cache flush is not a denial of service attack. Its no less effective than a 1Mb memcpy
Quoted 2 lines
I can understand why we need this in kernel, for context switch, for example, but as a syscall?...
Self modifying code, dynamic compilation, glibc trampolines
Quoted 8 lines · Michael Shmulevich
As a side question, I would like to to know why exactly the CPU cache operations are
promoted to the syscall status? What is the situation that a user in its program would like
to callcacheflush()? Unless, of course, he is doing DoS.I can understand why we need this in kernel, for context switch, for example, but as a syscall?...
For trampolines. These are small pieces of code created on the stack, and require flushing of the caches before they are excuted.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven ------------- Sony Software Development Center Europe (SDCE) [email protected] ------------------- Sint-Stevens-Woluwestraat 55 Voice +32-2-7248626 Fax +32-2-7262686 ---------------- B-1130 Brussels, Belgium
Quoted 5 lines · Michael Shmulevich
As a side question, I would like to to know why exactly the CPU cache operations are
promoted to the syscall status? What is the situation that a user in its program would like
to callcacheflush()? Unless, of course, he is doing DoS.
Any software that modifes text needs it. For example the dynamic linker
or libdl.
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
Quoted 2 lines · Ralf Baechle
You obviously want to allow partial cache flushes or trampolines don't work and flushing the entire cache can be constructed from that.
It depends on the implementation. The current one obviously allows it.
;-)
I'm writing of the design.
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
Quoted 7 lines · Michael Shmulevich
Quoted 2 lines
When a new user process is started will its user space be cleared by the kernel or is there a potential leak from an older user process ?
Usually it is defied by the loader. If the data section contents is set to LOAD, then the contents of the section will be loaded from disk (no leak), if not -- whatever values left i nmemory will be there, or exactly, the virtual page of some other proccess that was swapped out or ended.
What!??? I'm assume you are writing about executing a new program and
not forking a new process here. In the latter case no memory is changed. When you exec a new program, any allocated memory is cleared by the kernel before returning to the user space. It would be a huge security hole otherwise.
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
Quoted 2 lines · Carsten Langgaard
When a new user process is started will its user space be cleared by the kernel or is there a potential leak from an older user process ?
A new process is started by the clone(2) or fork(2) syscalls. Module the options that can be passed to clone(2) the two only create an identical copy of the invoking process, so they're designed to leak information by design ;-)
execve(2) replaces the existing mappings with a new process image loaded from files plus a newly created stack area. No old mappings survive, so there in memory there is no information leak.
Quoted 3 lines
What about the registers values, are they cleared for each new user application or will it simply contain the current value it got when the user application is started ?
We make no attempt at the integer registers for a new process, so some information might be leaked in registers. All the callee saved registers will be passed unchanged to the child process; the caller saved registers except those that are used as syscall return values will return random garbage. Floating point registers will be cleared with SNANs as soon as the process is attempting to use a FPU for the first time, that is we won't leak information via fpu registers.
(Ooops, we're not Orange Book B1 compliant, how sad ;-)
Quoted 1 line
How can you flush the data and instruction cashes from a user application ?
cacheflush(2). See man page.
Ralf