FP in kernelspace
Hello,
I have a driver written for 2.4 + RT patches with FP support. I want it to work in 2.6. How to implement FP? Has anybody developped some "protocol" between KS and US yet? If not, could somebody point me, how to do it the best -- with low latency.
The device doesn't generate irqs *), I need to quickly respond to timer call, because interval between two posts of data to the device has to be equal as much as possible (BTW is there any way how to gain up to 5000Hz).
I've one idea: have a thread with RT priority and wake the app in US waiting in read of character device when timer ticks, post a struct with 2 floats and operation and wait in write for the result. App computes, writes the result, we are woken and can post it to the device. But I'm afraid it would be tooo slow.
*) I don't know how to persuade it (standard PLX chip with unknown piece of logic behind) to generate, because official driver is closed and _very_ expensive. Old (2.4) driver was implemented with RT thread and timer, where FP is implemented within RT and computed directly in KS.
So 2 questions are:
1) howto FP in kernel
2) howto precise timer (may mingo RT patches help?)
3) any way to have faster ticks (up to 5000Hz)?
Any suggestions, please?
thanks,
Signature · 4 lines
<a href="http://www.fi.muni.cz/~xslaby/">Jiri Slaby</a> faculty of informatics, masaryk university, brno, cz e-mail: jirislaby gmail com, gpg pubkey fingerprint: B674 9967 0407 CE62 ACC8 22A0 32CC 55C3 39D4 7A7E
Quoted 3 lines · Jiri Slaby
Hello,
I have a driver written for 2.4 + RT patches with FP support. I want it to work in 2.6. How to implement FP? Has anybody developped some "protocol" between KS and US yet? If not, could somebody point me, how to do it the best -- with low latency.
One more thing, it can't be table-precomputed, positions are read from the device, needs to be recomputed and pushed back.
regards,
Signature · 4 lines
<a href="http://www.fi.muni.cz/~xslaby/">Jiri Slaby</a> faculty of informatics, masaryk university, brno, cz e-mail: jirislaby gmail com, gpg pubkey fingerprint: B674 9967 0407 CE62 ACC8 22A0 32CC 55C3 39D4 7A7E
Quoted 13 lines · Jiri Slaby
I have a driver written for 2.4 + RT patches with FP support. I want it to work
in 2.6. How to implement FP? Has anybody developped some "protocol" between KS
and US yet? If not, could somebody point me, how to do it the best -- with low
latency.
The device doesn't generate irqs *), I need to quickly respond to timer call,
because interval between two posts of data to the device has to be equal as much
as possible (BTW is there any way how to gain up to 5000Hz).
Quoted 8 lines
I've one idea: have a thread with RT priority and wake the app in US waiting in
read of character device when timer ticks, post a struct with 2 floats and
operation and wait in write for the result. App computes, writes the result, we
are woken and can post it to the device. But I'm afraid it would be tooo slow.
real floating point in the kernel is basically no-go; especially if you enable preemption. The entire floating point exception, context save and restore engine is designed with the assumption of no kernel space FPU usage... and there are a LOT of nasty corner cases to deal with... and so far the kernel decided to cheap out on many of those by just not allowing kernel FPU use (mmx is easier so that can be dealt with).
Volume 3 of the Intel architecture manuals are a good start if you want to know what some of these corner cases are...
Signature · 1 line
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Quoted 15 lines · Jiri Slaby
Hello,
I have a driver written for 2.4 + RT patches with FP support. I want it to work in 2.6. How to implement FP? Has anybody developped some "protocol" between KS and US yet? If not, could somebody point me, how to do it the best -- with low latency.
The device doesn't generate irqs *), I need to quickly respond to timer call, because interval between two posts of data to the device has to be equal as much as possible (BTW is there any way how to gain up to 5000Hz).
I've one idea: have a thread with RT priority and wake the app in US waiting in read of character device when timer ticks, post a struct with 2 floats and operation and wait in write for the result. App computes, writes the result, we are woken and can post it to the device. But I'm afraid it would be tooo slow.
Quoted 8 lines
*) I don't know how to persuade it (standard PLX chip with unknown piece of logic behind) to generate, because official driver is closed and _very_ expensive. Old (2.4) driver was implemented with RT thread and timer, where FP is implemented within RT and computed directly in KS.
So 2 questions are:
1) howto FP in kernel
kernel_fpu_begin();
c = d * 3.14;kernel_fpu_end();
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 8 lines
Quoted 4 lines
So 2 questions are:
1) howto FP in kernel
kernel_fpu_begin();
c = d * 3.14;kernel_fpu_end();
unfortunately this only works for MMX not for real fpu (due to exception handling uglies)
Signature · 1 line
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Quoted 6 lines · Arjan van de Ven
Quoted 6 lines
Quoted 3 lines
So 2 questions are:
1) howto FP in kernel
kernel_fpu_begin();
c = d * 3.14;kernel_fpu_end();
Yup, I know about this possibility, but this is only x86 specific?!
Quoted 2 lines
unfortunately this only works for MMX not for real fpu (due to exception handling uglies)
concludes it's not multiplatform at all... For that reasen I (maybe) want some "protocol" for communication with US, where I can easily compute it.
Another way could be rtai (there is FP implemented IIRC), but it means having out-of-kernel driver.
thanks,
Signature · 4 lines
<a href="http://www.fi.muni.cz/~xslaby/">Jiri Slaby</a> faculty of informatics, masaryk university, brno, cz e-mail: jirislaby gmail com, gpg pubkey fingerprint: B674 9967 0407 CE62 ACC8 22A0 32CC 55C3 39D4 7A7E
Quoted 8 lines · Jiri Slaby
Quoted 6 lines · Arjan van de Ven
Quoted 6 lines
Quoted 3 lines
So 2 questions are:
1) howto FP in kernel
kernel_fpu_begin();
c = d * 3.14;kernel_fpu_end();
Quoted 7 lines
Yup, I know about this possibility, but this is only x86 specific?!
Quoted 2 lines
unfortunately this only works for MMX not for real fpu (due to exception handling uglies)
concludes it's not multiplatform at all... For that reasen I (maybe) want some "protocol" for communication with US, where I can easily compute it.]
easiest might be a shared mmap ring buffer (well 2 of these actually), kernel puts data in buffer, updates ring pointer (and wakes userspace if needed). Userspace notices data in ring, does math, puts data back in the 2nd ringbuffer and updates that ring pointer. Then maybe does a syscall to 'wake' the kernel. Kernel then sees data in the 2nd ringbuffer and gets it. In addition to the wakeups I mentioned you can also poll for the buffers; the wakeups are just there for extra safety.
Quoted 4 lines · Arjan van de Ven
Quoted 4 lines
Quoted 4 lines
So 2 questions are:
1) howto FP in kernel
Quoted 8 lines
Quoted 4 lines
kernel_fpu_begin();
c = d * 3.14;kernel_fpu_end();unfortunately this only works for MMX not for real fpu (due to exception handling uglies)
Perhaps there should be a comment to that effect? Neither the code nor Documentation/preemt-locking (which mentions the fpu) says anything about this little fact.
It's also broken for x86-64, which uses sse for floating point, not the x87 fpu.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 8 lines · Jiri Slaby
Quoted 2 lines
unfortunately this only works for MMX not for real fpu (due to
exception
Quoted 1 line
handling uglies)
concludes it's not multiplatform at all... For that reasen I (maybe) want some "protocol" for communication with US, where I can easily compute it.
Well, usually such a protocol is by means of read(2) and write(2) to a character device provided by your driver: userspace reads some data, does the fp ops, and writes it back. If you want realtime you'll have to use a realtime thread with the appropriate priority.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 5 lines · Jiri Slaby
I have a driver written for 2.4 + RT patches with FP support. I want it to work in 2.6. How to implement FP? Has anybody developped some "protocol" between KS and US yet? If not, could somebody point me, how to do it the best -- with low latency.
The device doesn't generate irqs
Doesn't sound like something that should be supported for mainline then.
Quoted 2 lines
So 2 questions are:
1) howto FP in kernel
You can use kernel_fpu_begin()/_end() around the FP code in the kernel, but the real time people will probably hate you for it because it disables preemption.
Quoted 1 line
3) any way to have faster ticks (up to 5000Hz)?
There are plenty of interval timers in various hardware in a typical system that can be used. e.g. RTC is a common choice. Drawback is that you might conflict with other users, especially in user space.
But please don't try to submit such a hack.
-Andi
Quoted 3 lines · Arjan van de Ven
unfortunately this only works for MMX not for real fpu (due to exception handling uglies)
He can always disable all FPU exceptions in his code or make sure there aren't any.
-Andi
Quoted 3 lines · Avi Kivity
It's also broken for x86-64, which uses sse for floating point, not the x87 fpu.
Sorry, that doesn't make sense.
-Andi
Quoted 12 lines · Andi Kleen
Quoted 3 lines · Avi Kivity
It's also broken for x86-64, which uses sse for floating point, not the x87 fpu.
Sorry, that doesn't make sense.
Quoted 3 lines
kernel_fpu_begin();
c = d * 3.14;kernel_fpu_end();
Cannot work on x86-64, even disregarding fp exceptions, because kernel_fpu_begin() doesn't save the sse state which is used by fp math.
No?
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Quoted 5 lines
Cannot work on x86-64, even disregarding fp exceptions, because
kernel_fpu_begin()doesn't save the sse state which is used by fp math.No?
No. You can - if you want - exclusively use x87 math on x64 (at least for userspace) and don't care about SSE. Otherwise, GCC having a -mfpmath=387 would be pretty useless.
No?
Jan Engelhardt
Quoted 3 lines
kernel_fpu_begin();
c = d * 3.14;kernel_fpu_end();
static inline void kernel_fpu_begin() {
...
preempt_disable();
...
}Jan Engelhardt
Quoted 6 lines
Cannot work on x86-64, even disregarding fp exceptions, because
kernel_fpu_begin()doesn't save the sse state which is used by fp math.No?
It does - FXSAVE saves everything.
BTW you can use x87 on x86-64 too, you just need to use long double.
-Andi
Quoted 11 lines · Jan Engelhardt
Quoted 4 lines
kernel_fpu_begin();
c = d * 3.14;kernel_fpu_end();static inline void kernel_fpu_begin() { ... preempt_disable(); ... }
Is the kernel allowed to clobber userspace's sse registers?
What about interrupt code?
xor.h at least appears to save the sse state before use.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Quoted 9 lines · Andi Kleen
Quoted 6 lines
Cannot work on x86-64, even disregarding fp exceptions, because
kernel_fpu_begin()doesn't save the sse state which is used by fp math.No?
It does - FXSAVE saves everything.
I see. Thanks for clearing it up.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Quoted 11 lines
Quoted 10 lines
Quoted 3 lines
kernel_fpu_begin();
c = d * 3.14;kernel_fpu_end();static inline void
kernel_fpu_begin(){
...preempt_disable();
...
}Is the kernel allowed to clobber userspace's sse registers?
As long as you save and restore it properly, and make it look like to all other threads that nothing happened, you are (I hope) free to do anything.
Quoted 1 line
What about interrupt code?
You do not want to go there...
Quoted 5 lines
xor.hat least appears to save the sse state before use.Signature · 2 lines
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Follow that advice, don't do FP. But OTOH......<long dots> there is already an SSE player in the kernel: drivers/md/raid6sse*.c
Jan Engelhardt
Quoted 14 lines · Jiri Slaby
Hello,
I have a driver written for 2.4 + RT patches with FP support. I want it to work in 2.6. How to implement FP? Has anybody developped some "protocol" between KS and US yet? If not, could somebody point me, how to do it the best -- with low latency.
The device doesn't generate irqs *), I need to quickly respond to timer call, because interval between two posts of data to the device has to be equal as much as possible (BTW is there any way how to gain up to 5000Hz).
I've one idea: have a thread with RT priority and wake the app in US waiting in read of character device when timer ticks, post a struct with 2 floats and operation and wait in write for the result. App computes, writes the result, we are woken and can post it to the device. But I'm afraid it would be tooo slow.
Quoted 8 lines
*) I don't know how to persuade it (standard PLX chip with unknown piece of logic behind) to generate, because official driver is closed and _very_ expensive. Old (2.4) driver was implemented with RT thread and timer, where FP is implemented within RT and computed directly in KS.
So 2 questions are:
1) howto FP in kernel
2) howto precise timer (may mingo RT patches help?)
Well Ingo's RT patch set has the high resolution timers developed by Thomas Gleixner, which may help you here.
Quoted 1 line
3) any way to have faster ticks (up to 5000Hz)?
Why do you need faster ticks? The high res timers are done in nano secs, and the resolution is up to the hardware.
Quoted 2 lines
Any suggestions, please?
-- Steve