desktop and multimedia as an afterthought?
Quoted 3 lines · Florin Andrei
I could be wrong, but it seems to me that at least a part of the kernel development team has the desktop and multimedia issues very low on the priority list.
You're right, but you imply that this is bad. It would only be bad if _all_ of the kernel development team had the desktop and multimedia issues very low on the priority list. It would be far worse if everyone worked on multimedia issues.
Quoted 7 lines
The CK patches floated around as separate patches for a long time, even though they brought significant improvements to the kernel w.r.t. desktop and media.
Now the stock 2.6 kernel has a pretty bad problem with the latency. Also, there seems to be a reluctance in accepting the autoregulated swappiness patch, even though it markedly improves the overall behaviour of the system as a desktop.
Reluctance is good. Reluctance keeps out bloat. Reluctance supplies time to investigate alternate ideas that might be better.
Quoted 4 lines
The CK patches have devouted "cult followers".
Projects such as PlanetCCRMA, which attempt to build a multimedia-ready Linux distribution, are running kernels patched with the CK patches by default.
This is expected. It's how proposed features get widespread testing.
Quoted 3 lines
On the Linux multimedia mailing lists and forums, one can often hear advices in the vein of "use such-and-such patch if you want your system to do any serious multimedia work, the vanilla kernel sucks."
...
Quoted 2 lines
I mean, go read the forums. No one in the multimedia community uses the vanilla kernel. They could be all wrong, sure, but there's lots of them.
It's too bad that the multimedia community didn't participate much during the 2.5.xx development leading up to 2.6.0. If they had done so, the situation might be different today. Fortunately, fixing up the multimedia problems isn't too risky to do during the stable 2.6.xx series.
Quoted 3 lines
It seems like there's a split-brain situation within the Linux community, with the core kernel developers sitting on one side of the rift, and the multimedia people on the other side.
Nah. People do forget about problems if nobody is reporting them. Since the problem reports came in late, you get to be last.
Quoted 5 lines
Now, i remember someone saying, a while ago, that the server stuff is pretty much done, and the interesting things are going to happen on the desktop. That sounds plausible, but if the kernel has poor support for typical desktop scenarios, how are those big desktop improvements going to take place?
Much of that is about hot-plug, laptops going to sleep, busy disks being ripped from the machine, support for recent video card features, and so on. Multimedia is only a small part of the big picture, even for desktop usage.
Quoted 5 lines
It's too bad that the multimedia community didn't participate much during the 2.5.xx development leading up to 2.6.0. If they had done so, the situation might be different today. Fortunately, fixing up the multimedia problems isn't too risky to do during the stable 2.6.xx series.
I regret that this description is persisting here. "We" (the audio developer community) did not participate because it was made clear that our needs were not going to be considered. We were told that the preemption patch was sufficient to provide "low latency", and that rescheduling points dotted all over the place was bad engineering (probably true). With this as the pre-rendered verdict, there's not a lot of point in dedicating time to tracking a situation that clearly is not going to work.
The kernel is not going to provide adequate latency for multimedia needs without either (1) latency issues being front and center in every kernel developer's mind, which seems unlikely and/or (2) conditional rescheduling points added to the kernel, which appears to require non-mainstreamed patches.
--p
Quoted 15 lines · Paul Davis
Quoted 5 lines
It's too bad that the multimedia community didn't participate much during the 2.5.xx development leading up to 2.6.0. If they had done so, the situation might be different today. Fortunately, fixing up the multimedia problems isn't too risky to do during the stable 2.6.xx series.
I regret that this description is persisting here. "We" (the audio developer community) did not participate because it was made clear that our needs were not going to be considered. We were told that the preemption patch was sufficient to provide "low latency", and that rescheduling points dotted all over the place was bad engineering (probably true). With this as the pre-rendered verdict, there's not a lot of point in dedicating time to tracking a situation that clearly is not going to work.
Quoted 5 lines
The kernel is not going to provide adequate latency for multimedia needs without either (1) latency issues being front and center in every kernel developer's mind, which seems unlikely and/or (2) conditional rescheduling points added to the kernel, which appears to require non-mainstreamed patches.
Please dont start a low level flamewar over this. Latency is firmly on the agenda and under consideration for the mainline kernel now.
There is nothing wrong with using a dedicated alternative patchset for specific tasks, as long as any lessons learnt from it are also taken into consideration for mainline. Mainline kernels must have (high gain)/(low risk) ratio changes only.
Rather than just saying that the desktop and multimedia is not considered it would be more helpful to say what helps where and why in the public forum of the main kernel mailing list. Off list discussion can go completely unnoticed (even I wasn't aware of my patchset being used and quoted!)
Cheers,
Con
Quoted 7 lines · Paul Davis
Quoted 6 lines
It's too bad that the multimedia community didn't participate much during the 2.5.xx development leading up to 2.6.0. If they had done so, the situation might be different today. Fortunately, fixing up the multimedia problems isn't too risky to do during the stable 2.6.xx series.
Quoted 8 lines
I regret that this description is persisting here. "We" (the audio developer community) did not participate because it was made clear that our needs were not going to be considered. We were told that the preemption patch was sufficient to provide "low latency", and that rescheduling points dotted all over the place was bad engineering (probably true). With this as the pre-rendered verdict, there's not a lot of point in dedicating time to tracking a situation that clearly is not going to work.
No, this is wrong. 2.6+preempt can satisfy your latency requirements without any scheduling points. All it requires is that the long-held locks be addressed. I've already done a metric ton of work in that area (notably removal of the buffer_head LRUs and rewriting the truncate code) but more apparently remains to be done. We know that reiserfs has problems.
But what can I do? I set up a preempt-on-ext3 test box, thrash the crap out of it and see 300 usecs worst-case latency. So I am left empty-handed, wondering what on earth is happening out there.
I am deeply skeptical about claims that autoregulated swappiness can make any difference.
I am deeply skeptical about claims that CPU scheduler changes make any difference. A scheduler change shouldn't improve responsiveness of !SCHED_OTHER tasks at all, so perhaps there are application priority inversion problems, or applications aren't setting SCHED_FIFO/RR correctly. I do not know.
I am also fairly skeptical about claims that voluntary-preempt helps, because it only pops a couple of locks, and I doubt that testers are hitting the code paths which those changes address anyway.
So Something Is Up, and I don't know what it is.
Please double-check that there are no priority inversion problems and that the application is correctly setting the scheduling policy and that it is mlocking everything appropriately.
And please ensure that people are setting xrun_debug, and are sending reports.
Quoted 6 lines
The kernel is not going to provide adequate latency for multimedia needs without either (1) latency issues being front and center in every kernel developer's mind, which seems unlikely and/or (2) conditional rescheduling points added to the kernel, which appears to require non-mainstreamed patches.
Nope, the conditional rescheduling points provide zero benefit on a preemptible kernel.
Something weird is happening, I don't know what it is, I cannot reproduce it and I need help understanding what it is, OK? The sooner we can do that, the sooner it gets fixed up.
Quoted 2 lines
Please dont start a low level flamewar over this. Latency is firmly on the agenda and under consideration for the mainline kernel now.
Yes, and I apologize for that intemperate post from me. Quite unnecessary and utterly unhelpful. It just irks me to be told "you're late to the party" when we were told "the party can happen without you". Nothing more than that. Its great to see that people *do* care about this stuff. Sorry for the noise.
--p
Quoted 3 lines · Andrew Morton
Something weird is happening, I don't know what it is, I cannot reproduce it and I need help understanding what it is, OK? The sooner we can do that, the sooner it gets fixed up.
Sounds fair to me.
I have no time whatsoever for such things, but if i do get time, i'll do some tests and post results. I'll lurk in the background meanwhile, to figure out more precisely just what kind of tests you guys want, what kind of debugging to turn on, etc.
Thank you,
Signature · 3 lines
Florin Andrei http://florin.myip.org/
Quoted 1 line · Con Kolivas
(even I wasn't aware of my patchset being used and quoted!)
Well, no one's using your patch except the entire Linux sound/music community :-) the people around the linux-audio-user mailing list, the users of the PlanetCCRMA distro, etc.
I suspect also that the video guys are also quite familiar with your work, at least those who do a lot of analog video capture (and maybe digital as well).
Signature · 3 lines
Florin Andrei http://florin.myip.org/
At Mon, 12 Jul 2004 17:24:58 -0700,
Quoted 6 lines · Andrew Morton
I am deeply skeptical about claims that CPU scheduler changes make any difference. A scheduler change shouldn't improve responsiveness of !SCHED_OTHER tasks at all, so perhaps there are application priority inversion problems, or applications aren't setting SCHED_FIFO/RR correctly. I do not know.
Regarding the JACK problem, it seems that the incompatibility with NPTL (SCHED_INHERIT is default) did wrong.
Taking a look through the thread, I feel that very different topics are argued in the single "desktop" problem, namely, the interactivity and the latency. The latter, the problem of real-time audio (e.g. JACK), must be irrelevant with the CPU scheduler. It can be fixed by detecting the too long critical sections, but the fix won't always improve the interactivity.
OTOH, the interactivity can be, and should be improved somehow with tuning of CPU scheduler. However, even about this word, we discuss totally different meanings. For example, the GUI response and the fluent audio/video playback. The improvement of the former doesn't imply the improvement of the latter (often contradictorily)...
--
Takashi Iwai <[email protected]> ALSA Developer - www.alsa-project.org
Quoted 4 lines
And please ensure that people are setting
xrun_debug, and are sending reports.
Hi,
On my system xruns seem related to the keyboard. I get xruns on ~8.079 seconds boundaries when the keyboard is in use, regardless of the load. My usual test is running jack with 2 periods of 64 samples and no client, and keep a key pressed. Those latencytest graphs give an idea of the problem : http://www.undata.org/~thomas/latencytest/index.html
Here are the xrun_debug reports :
For the intel8x0 :
XRUN: pcmC1D0p
Stack pointer is garbage, not printing trace
Unexpected hw_pointer value [1] (stream = 1, delta: -16, max jitter = 64): wrong interrupt acknowledge?
[<c0105f3e>] dump_stack+0x1e/0x30 [<c033240d>] snd_pcm_period_elapsed+0x1cd/0x420 [<c03578cc>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107824>] do_IRQ+0x194/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107754>] do_IRQ+0xc4/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0108126>] do_softirq+0x46/0x60 [<c01077d9>] do_IRQ+0x149/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01030f4>] cpu_idle+0x34/0x40 [<c053c809>] start_kernel+0x169/0x190 [<c010019f>] 0xc010019f ======================= [<c0105f3e>] dump_stack+0x1e/0x30 [<c033240d>] snd_pcm_period_elapsed+0x1cd/0x420 [<c03578cc>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107824>] do_IRQ+0x194/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107754>] do_IRQ+0xc4/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0108126>] do_softirq+0x46/0x60 [<c01077d9>] do_IRQ+0x149/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01030f4>] cpu_idle+0x34/0x40 [<c053c809>] start_kernel+0x169/0x190 [<c010019f>] 0xc010019f ======================= [<c0105f3e>] dump_stack+0x1e/0x30 [<c033240d>] snd_pcm_period_elapsed+0x1cd/0x420 [<c03578cc>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107824>] do_IRQ+0x194/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107754>] do_IRQ+0xc4/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0108126>] do_softirq+0x46/0x60 [<c01077d9>] do_IRQ+0x149/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01030f4>] cpu_idle+0x34/0x40 [<c053c809>] start_kernel+0x169/0x190 [<c010019f>] 0xc010019f
For the hdsp:
XRUN: pcmC2D0c
Stack pointer is garbage, not printing trace
XRUN: pcmC2D0c
[<c0105f3e>] dump_stack+0x1e/0x30 [<c0332521>] snd_pcm_period_elapsed+0x2e1/0x420 [<c0368e94>] snd_hdsp_interrupt+0x174/0x180 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107824>] do_IRQ+0x194/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107754>] do_IRQ+0xc4/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0108126>] do_softirq+0x46/0x60 [<c01077d9>] do_IRQ+0x149/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01030f4>] cpu_idle+0x34/0x40 [<c053c809>] start_kernel+0x169/0x190 [<c010019f>] 0xc010019f ======================= [<c0105f3e>] dump_stack+0x1e/0x30 [<c0332521>] snd_pcm_period_elapsed+0x2e1/0x420 [<c0368e94>] snd_hdsp_interrupt+0x174/0x180 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107824>] do_IRQ+0x194/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107754>] do_IRQ+0xc4/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0108126>] do_softirq+0x46/0x60 [<c01077d9>] do_IRQ+0x149/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01030f4>] cpu_idle+0x34/0x40 [<c053c809>] start_kernel+0x169/0x190 [<c010019f>] 0xc010019f ======================= [<c0105f3e>] dump_stack+0x1e/0x30 [<c0332521>] snd_pcm_period_elapsed+0x2e1/0x420 [<c0368e94>] snd_hdsp_interrupt+0x174/0x180 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107824>] do_IRQ+0x194/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107754>] do_IRQ+0xc4/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0108126>] do_softirq+0x46/0x60 [<c01077d9>] do_IRQ+0x149/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01030f4>] cpu_idle+0x34/0x40 [<c053c809>] start_kernel+0x169/0x190 [<c010019f>] 0xc010019f
Thomas
Quoted 8 lines · Thomas Charbonnel
On my system xruns seem related to the keyboard. I get xruns on ~8.079
seconds boundaries when the keyboard is in use, regardless of the load. My usual test is running jack with 2 periods of 64 samples and no client, and keep a key pressed. Those latencytest graphs give an idea of the problem : http://www.undata.org/~thomas/latencytest/index.htmlHere are the
xrun_debugreports :
OK, thanks.
Stack tracing seems a bit broken with 4k stacks. Can you disable CONFIG_4KSTACKS for future testing?
Quoted 33 lines
For the intel8x0 :
XRUN: pcmC1D0p
Stack pointer is garbage, not printing trace
Unexpectedhw_pointervalue [1] (stream = 1, delta: -16, max jitter = 64): wrong interrupt acknowledge?[<c0105f3e>] dump_stack+0x1e/0x30 [<c033240d>] snd_pcm_period_elapsed+0x1cd/0x420 [<c03578cc>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107824>] do_IRQ+0x194/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107754>] do_IRQ+0xc4/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0108126>] do_softirq+0x46/0x60 [<c01077d9>] do_IRQ+0x149/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01030f4>] cpu_idle+0x34/0x40 [<c053c809>] start_kernel+0x169/0x190 [<c010019f>] 0xc010019f ======================= [<c0105f3e>] dump_stack+0x1e/0x30 [<c033240d>] snd_pcm_period_elapsed+0x1cd/0x420 [<c03578cc>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107824>] do_IRQ+0x194/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107754>] do_IRQ+0xc4/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0108126>] do_softirq+0x46/0x60 [<c01077d9>] do_IRQ+0x149/0x1b0 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01030f4>] cpu_idle+0x34/0x40 [<c053c809>] start_kernel+0x169/0x190
Quoted 3 lines · Andrew Morton
Stack tracing seems a bit broken with 4k stacks. Can you disable
CONFIG_4KSTACKSfor future testing?
Now I get :
for the intel8x0 :
XRUN: pcmC1D0p
[<c0105f3e>] dump_stack+0x1e/0x30 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0146728>] do_anonymous_page+0x118/0x190 [<c01467ff>] do_no_page+0x5f/0x300 [<c0146c92>] handle_mm_fault+0xe2/0x190 [<c01456a0>] get_user_pages+0x150/0x380 [<c0146e07>] make_pages_present+0x87/0xb0 [<c01472a9>] mlock_fixup+0xa9/0xc0 [<c01475ef>] do_mlockall+0x7f/0xa0 [<c01476cb>] sys_mlockall+0xbb/0xc0 [<c0105157>] syscall_call+0x7/0xb Unexpected hw_pointer value [1] (stream = 1, delta: -50, max jitter = 64): wrong interrupt acknowledge? [<c0105f3e>] dump_stack+0x1e/0x30 [<c033244d>] snd_pcm_period_elapsed+0x1cd/0x420 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01201ab>] do_softirq+0x2b/0x30 [<c0107798>] do_IRQ+0x108/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 Unexpected hw_pointer value [1] (stream = 0, delta: -4, max jitter = 64): wrong interrupt acknowledge? [<c0105f3e>] dump_stack+0x1e/0x30 [<c033244d>] snd_pcm_period_elapsed+0x1cd/0x420 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01201ab>] do_softirq+0x2b/0x30 [<c0107798>] do_IRQ+0x108/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 XRUN: pcmC1D0p [<c0105f3e>] dump_stack+0x1e/0x30 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 Unexpected hw_pointer value [1] (stream = 1, delta: -22, max jitter = 64): wrong interrupt acknowledge? [<c0105f3e>] dump_stack+0x1e/0x30 [<c033244d>] snd_pcm_period_elapsed+0x1cd/0x420 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01201ab>] do_softirq+0x2b/0x30 [<c0107798>] do_IRQ+0x108/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01030f4>] cpu_idle+0x34/0x40 [<c053e809>] start_kernel+0x169/0x190 [<c010019f>] 0xc010019f XRUN: pcmC1D0p [<c0105f3e>] dump_stack+0x1e/0x30 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 XRUN: pcmC1D0p [<c0105f3e>] dump_stack+0x1e/0x30 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20
for the hdsp:
XRUN: pcmC2D0c
[<c0105f3e>] dump_stack+0x1e/0x30 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420 [<c0368ed4>] snd_hdsp_interrupt+0x174/0x180 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01467ff>] do_no_page+0x5f/0x300 [<c0146c92>] handle_mm_fault+0xe2/0x190 [<c01456a0>] get_user_pages+0x150/0x380 [<c0146e07>] make_pages_present+0x87/0xb0 [<c01487f7>] do_mmap_pgoff+0x467/0x6c0 [<c010ba10>] sys_mmap2+0xa0/0xe0 [<c0105157>] syscall_call+0x7/0xb XRUN: pcmC2D0c [<c0105f3e>] dump_stack+0x1e/0x30 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420 [<c0368ed4>] snd_hdsp_interrupt+0x174/0x180 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0145633>] get_user_pages+0xe3/0x380 [<c0146e07>] make_pages_present+0x87/0xb0 [<c01487f7>] do_mmap_pgoff+0x467/0x6c0 [<c010ba10>] sys_mmap2+0xa0/0xe0 [<c0105157>] syscall_call+0x7/0xb XRUN: pcmC2D0c [<c0105f3e>] dump_stack+0x1e/0x30 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420 [<c0368ed4>] snd_hdsp_interrupt+0x174/0x180 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01201ab>] do_softirq+0x2b/0x30 [<c0107798>] do_IRQ+0x108/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c01030f4>] cpu_idle+0x34/0x40 [<c053e809>] start_kernel+0x169/0x190 [<c010019f>] 0xc010019f XRUN: pcmC2D0c [<c0105f3e>] dump_stack+0x1e/0x30 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420 [<c0368ed4>] snd_hdsp_interrupt+0x174/0x180 [<c010739b>] handle_IRQ_event+0x3b/0x70 [<c0107726>] do_IRQ+0x96/0x150 [<c0105ac4>] common_interrupt+0x18/0x20 [<c0144fc5>] zap_pte_range+0x145/0x240 [<c0145125>] zap_pmd_range+0x65/0x90 [<c01451a5>] unmap_page_range+0x55/0x80 [<c01452e4>] unmap_vmas+0x114/0x1e0 [<c0149845>] exit_mmap+0x85/0x170 [<c0119ebf>] mmput+0x6f/0xb0 [<c011e6c4>] do_exit+0x114/0x470 [<c011eaca>] do_group_exit+0x3a/0xb0 [<c012700e>] get_signal_to_deliver+0x25e/0x370 [<c0104eea>] do_signal+0x6a/0x100 [<c0104fb9>] do_notify_resume+0x39/0x3c [<c01051a2>] work_notifysig+0x13/0x15
Thomas
Quoted 27 lines · Andrew Morton
Quoted 7 lines · Paul Davis
Quoted 6 lines
It's too bad that the multimedia community didn't participate much during the 2.5.xx development leading up to 2.6.0. If they had done so, the situation might be different today. Fortunately, fixing up the multimedia problems isn't too risky to do during the stable 2.6.xx series.
Quoted 8 lines
I regret that this description is persisting here. "We" (the audio developer community) did not participate because it was made clear that our needs were not going to be considered. We were told that the preemption patch was sufficient to provide "low latency", and that rescheduling points dotted all over the place was bad engineering (probably true). With this as the pre-rendered verdict, there's not a lot of point in dedicating time to tracking a situation that clearly is not going to work.
No, this is wrong. 2.6+preempt can satisfy your latency requirements without any scheduling points. All it requires is that the long-held locks be addressed. I've already done a metric ton of work in that area (notably removal of the
buffer_headLRUs and rewriting the truncate code) but more apparently remains to be done. We know that reiserfs has problems.But what can I do? I set up a preempt-on-ext3 test box, thrash the crap out of it and see 300 usecs worst-case latency. So I am left empty-handed, wondering what on earth is happening out there.
Quoted 8 lines
I am deeply skeptical about claims that autoregulated swappiness can make any difference.
I am deeply skeptical about claims that CPU scheduler changes make any difference. A scheduler change shouldn't improve responsiveness of !SCHED_OTHER tasks at all, so perhaps there are application priority inversion problems, or applications aren't setting SCHED_FIFO/RR correctly. I do not know.
sorry to interrupt, and i dont know if i get this right, but i have been using nick piggins patch for quite some time, and it really does magic :)
Quoted 33 lines
I am also fairly skeptical about claims that voluntary-preempt helps, because it only pops a couple of locks, and I doubt that testers are hitting the code paths which those changes address anyway.
So Something Is Up, and I don't know what it is.
Please double-check that there are no priority inversion problems and that the application is correctly setting the scheduling policy and that it is mlocking everything appropriately.
And please ensure that people are setting
xrun_debug, and are sending reports.Quoted 6 lines
The kernel is not going to provide adequate latency for multimedia needs without either (1) latency issues being front and center in every kernel developer's mind, which seems unlikely and/or (2) conditional rescheduling points added to the kernel, which appears to require non-mainstreamed patches.
Nope, the conditional rescheduling points provide zero benefit on a preemptible kernel.
Something weird is happening, I don't know what it is, I cannot reproduce it and I need help understanding what it is, OK? The sooner we can do that, the sooner it gets fixed up.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[...]
Quoted 3 lines
Please double-check that there are no priority inversion problems and that the application is correctly setting the scheduling policy and that it is mlocking everything appropriately.
I don't think it is currently possible to have cooperating threads with different priorities without priority inversion when using a mutex to serialize access to shared data; and using a mutex is in fact the only portable way to do that...
Thus, the fact that Linux does not support protocols to prevent priority inversion (please correct me if I am wrong) kind of suggests that supporting realtime applications is not considered very important.
It is often heard in the Linux audio community that mutexes are not realtime safe and a lock-free ringbuffer should be used instead. Using such a lock-free ringbuffer requires non-standard atomic integer operations and does not guarantee memory synchronization (and should probably not perform significantly better than a decent mutex implementation) and is thus not portable.
--ms
Quoted 3 lines
Thus, the fact that Linux does not support protocols to prevent priority inversion (please correct me if I am wrong) kind of suggests that supporting realtime applications is not considered very important.
we went through this (you and i in particular) right here on LAD a year or so ago. while i might agree with you about the priority given to RT-ish apps, my recollection of the end of that discussion is that priority inheritance is neither necessary nor sufficient to allow adequate RT performance. priority inversion generally can be factored out through application redesign, and the protocols i've seen to address it are not useful for RT purposes - they just help deadlock.
--p
Quoted 5 lines · Martijn Sipkema
[...]
Quoted 3 lines
Please double-check that there are no priority inversion problems and that the application is correctly setting the scheduling policy and that it is mlocking everything appropriately.
Quoted 8 lines
I don't think it is currently possible to have cooperating threads with different priorities without priority inversion when using a mutex to serialize access to shared data; and using a mutex is in fact the only portable way to do that...
Thus, the fact that Linux does not support protocols to prevent priority inversion (please correct me if I am wrong) kind of suggests that supporting realtime applications is not considered very important.
Any use of an explicit or implied blocking mutex across threads with differing priorities can results in priority inversion problems. The real problem, however, is contention. If you get rid of the contention in a certain critical section, you then also get rid of latency in the system. They are one and the same problem.
Quoted 6 lines
It is often heard in the Linux audio community that mutexes are not realtime safe and a lock-free ringbuffer should be used instead. Using such a lock-free ringbuffer requires non-standard atomic integer operations and does not guarantee memory synchronization (and should probably not perform significantly better than a decent mutex implementation) and is thus not portable.
It's to decouple the system from various time related problems with jitter. It's critical to use this since the nature of Linus is so temporally coarse that these techniques must be used to "smooth" over latency problems in the Linux kernel.
I personally would love to see these audio applications run on a first-class basis under Linux. Unfortunately, that won't happen until it gets near real time support prevasively through the kernel just like in SGI's IRIX. Multimedia applications really need to be under a hard real time system with special scheduler support so that CPU resources, IO channels can be throttled.
The techniques Linux media folks are using now are basically a coarse hack to get things basically working. This won't change unless some fundamental concurrency issues (moving to a preemptive kernel with interrupt threads, etc..) change in Linux. Scattering preemption points manually over 2.6 is starting to look unmanable from all of the stack traces I've been reading in these latency related threads.
That's all. :)
bill
Quoted 32 lines · Bill Huey
Quoted 5 lines · Martijn Sipkema
[...]
Quoted 3 lines
Please double-check that there are no priority inversion problems and that the application is correctly setting the scheduling policy and that it is mlocking everything appropriately.
Quoted 8 lines
I don't think it is currently possible to have cooperating threads with different priorities without priority inversion when using a mutex to serialize access to shared data; and using a mutex is in fact the only portable way to do that...
Thus, the fact that Linux does not support protocols to prevent priority inversion (please correct me if I am wrong) kind of suggests that supporting realtime applications is not considered very important.
Any use of an explicit or implied blocking mutex across threads with differing priorities can results in priority inversion problems. The real problem, however, is contention. If you get rid of the contention in a certain critical section, you then also get rid of latency in the system. They are one and the same problem.
Quoted 6 lines
It is often heard in the Linux audio community that mutexes are not realtime safe and a lock-free ringbuffer should be used instead. Using such a lock-free ringbuffer requires non-standard atomic integer operations and does not guarantee memory synchronization (and should probably not perform significantly better than a decent mutex implementation) and is thus not portable.
It's to decouple the system from various time related problems with jitter. It's critical to use this since the nature of Linus is so temporally coarse that these techniques must be used to "smooth" over latency problems in the Linux kernel.
Quoted 6 lines
I personally would love to see these audio applications run on a first-class basis under Linux. Unfortunately, that won't happen until it gets near real time support prevasively through the kernel just like in SGI's IRIX. Multimedia applications really need to be under a hard real time system with special scheduler support so that CPU resources, IO channels can be throttled.
I don't think invoking IRIX is going to get us a lot of sympathy on LKML. It is widely reviled. BEOS is probably a better example.
Just my $0.02.
Lee
From: "Paul Davis" <[email protected]>
Quoted 9 lines
Quoted 3 lines
Thus, the fact that Linux does not support protocols to prevent priority inversion (please correct me if I am wrong) kind of suggests that supporting realtime applications is not considered very important.
we went through this (you and i in particular) right here on LAD a year or so ago. while i might agree with you about the priority given to RT-ish apps, my recollection of the end of that discussion is that priority inheritance is neither necessary nor sufficient to allow adequate RT performance.
I don't recall that that is what was concluded.
Priority inheritance or some other protocol for priority inversion _is_ needed for realtime applications that have threads with different priorities accessing common data. One could increase the priority of the low priority thread before taking the mutex and release it afterwards (as in priority ceiling), but I doubt that's optimal.
Quoted 3 lines
priority inversion generally can be factored
out through application redesign, and the protocols i've seen to address it are not useful for RT purposes - they just help deadlock.
For cases where some form of message passing does not work you will need shared data with a mutex to serialize access and you _will_ need to prevent priority inversion. Mutexes are part of the POSIX realtime threading extensions; you can use a semaphore when priority inversion is not an issue.
IMHO it is the lack of a mutex implementation with priority ceiling or inheritance and the stories about relying on either being a design problem that have caused the Linux audio community to not use mutexes and declare them non-RT safe while in fact they are required according to POSIX to synchronize memory between cooperating threads.
--ms
From: "Bill Huey (hui)" <[email protected]>
Quoted 11 lines · Martijn Sipkema
Quoted 10 lines
[...]
Quoted 3 lines
Please double-check that there are no priority inversion problems and that the application is correctly setting the scheduling policy and that it is mlocking everything appropriately.
I don't think it is currently possible to have cooperating threads with different priorities without priority inversion when using a mutex to serialize access to shared data; and using a mutex is in fact the only portable way to do that...
Quoted 8 lines
Quoted 3 lines
Thus, the fact that Linux does not support protocols to prevent priority inversion (please correct me if I am wrong) kind of suggests that supporting realtime applications is not considered very important.
Any use of an explicit or implied blocking mutex across threads with differing priorities can results in priority inversion problems. The real problem, however, is contention. If you get rid of the contention in a certain critical section, you then also get rid of latency in the system. They are one and the same problem.
The worst case latency is the one that counts and that is the contended case. If you could guarantee no contention then the worst case latency would be the very fast uncontended case, but I doubt there are many (any?) examples of this in practice. There are valid uses of mutexes with priority inheritance/ceiling protocol; the poeple making the POSIX standard aren't stupid...
Quoted 11 lines
Quoted 6 lines
It is often heard in the Linux audio community that mutexes are not realtime safe and a lock-free ringbuffer should be used instead. Using such a lock-free ringbuffer requires non-standard atomic integer operations and does not guarantee memory synchronization (and should probably not perform significantly better than a decent mutex implementation) and is thus not portable.
It's to decouple the system from various time related problems with jitter. It's critical to use this since the nature of Linus is so temporally coarse that these techniques must be used to "smooth" over latency problems in the Linux kernel.
Either use mutexes or POSIX message queues... the latter also are not intended for realtime use under Linux (though they are meant for it in POSIX), since they don't allocate memory on creation.
Quoted 6 lines
I personally would love to see these audio applications run on a first-class basis under Linux. Unfortunately, that won't happen until it gets near real time support prevasively through the kernel just like in SGI's IRIX. Multimedia applications really need to be under a hard real time system with special scheduler support so that CPU resources, IO channels can be throttled.
Quoted 6 lines
The techniques Linux media folks are using now are basically a coarse hack to get things basically working. This won't change unless some fundamental concurrency issues (moving to a preemptive kernel with interrupt threads, etc..) change in Linux. Scattering preemption points manually over 2.6 is starting to look unmanable from all of the stack traces I've been reading in these latency related threads.
Improving the mutex and mqueue implementations to better support realtime use would be a significant improvement I think, making Linux quite suitable for realtime audio use.
--ms
On Tue, Jul 13, 2004 at 11:44:59PM +0100, Martijn Sipkema wrote:
[...]
Quoted 5 lines
The worst case latency is the one that counts and that is the contended case. If you could guarantee no contention then the worst case latency would be the very fast uncontended case, but I doubt there are many (any?) examples of this in practice. There are valid uses of mutexes with priority inheritance/ceiling protocol; the poeple making the POSIX standard aren't stupid...
There are cases where you have to use priority inheritance, but the schemes that are typically used either have a kind of exhaustive analysis backing it or uses a simple late detection scheme. In a general purpose OS, the latter is useful for various kind of overload cases. But if your system is constantly using that specific case, then it's a sign the contention in the kernel must *also* be a problem under SMP conditions. The constant use of priority inheritance overloads the scheduler, puts pressure on the cache and other negative things that hurt CPU local performance of the system.
The reason why I mention this is because of Linux's hand-crafted nature of dealing with this. These are basically contention problems expressed in a different manner. The traditional Linux method is the correct method of deal with this in a general purpose OS. This also applies to application structure as well. The use of these mechanisms need to be thought out before application.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
It is often heard in the Linux audio community that mutexes are not realtime safe and a lock-free ringbuffer should be used instead. Using such a lock-free ringbuffer requires non-standard atomic integer operations and does not guarantee memory synchronization (and should probably not perform significantly better than a decent mutex implementation) and is thus not portable.
Quoted 8 lines
Quoted 4 lines
It's to decouple the system from various time related problems with jitter. It's critical to use this since the nature of Linus is so temporally coarse that these techniques must be used to "smooth" over latency problems in the Linux kernel.
Either use mutexes or POSIX message queues... the latter also are not intended for realtime use under Linux (though they are meant for it in POSIX), since they don't allocate memory on creation.
The nature these kind of applications push into a very demanding space where typical methodologies surrounding the use of threads goes out the window. Pushing both the IO and CPU resources of a kernel is the common case and often you have to roll your own APIs, synchronization mechanisms to deal with these problem. Simple Posix API and traditional mutexes are a bit too narrow in scope to solve these cross system concurrency problems. It's not trivial stuff at all and can span from loosely to tightly coupled systems, yes, all for pro-audio/video.
Posix and friends in these cases simply aren't good enough to cut it.
Quoted 13 lines
Quoted 6 lines
I personally would love to see these audio applications run on a first-class basis under Linux. Unfortunately, that won't happen until it gets near real time support prevasively through the kernel just like in SGI's IRIX. Multimedia applications really need to be under a hard real time system with special scheduler support so that CPU resources, IO channels can be throttled.
Quoted 6 lines
The techniques Linux media folks are using now are basically a coarse hack to get things basically working. This won't change unless some fundamental concurrency issues (moving to a preemptive kernel with interrupt threads, etc..) change in Linux. Scattering preemption points manually over 2.6 is starting to look unmanable from all of the stack traces I've been reading in these latency related threads.
Quoted 3 lines
Improving the mutex and mqueue implementations to better support realtime use would be a significant improvement I think, making Linux quite suitable for realtime audio use.
Yes and no. This really all really needs to be hard real time in the long run. With things like Tivo and other embedded application putting high demand on kernel resources this kind of stuff needs to go into the Linux kernel sooner rather than later so that Linux can keep up with current industry tracks.
That's my two cents. :)
bill
Quoted 6 lines · Martijn Sipkema
IMHO it is the lack of a mutex implementation with priority ceiling or inheritance and the stories about relying on either being a design problem that have caused the Linux audio community to not use mutexes and declare them non-RT safe while in fact they are required according to POSIX to synchronize memory between cooperating threads.
Does someone have an authoritive answer to the following question:
Will using try_lock() on a mutex ever block the caller ?
AFAIK it will not. If this is true I don't see the point of a lock-free ring buffer at all. You will need some way to wake up the non-RT thread anyway, in case it went to sleep when finding and empty ring buffer. This same synchronisation method can be used to share the state of the buffer between two threads.
For example if you use a counting semaphore (built using a condition variable and a mutex), the RT thread would increment the sema for every N samples it adds to a circular buffer, and the consumer will decrement it for every N samples it reads. Then the state of the sema at all times reflects the number of samples in the buffer.
If ever the V operation in the RT thread fails (unlikely, but possible since it has to use try_lock()), it will remember this and increment by one more the next time.
The point is that there is no need to use lock-free techniques to maintain a shared sample count - it's already provided by the sync mechanism which you need anayway. I've been using this method for years in critical apps, and never seen it fail.
Signature · 1 line
FA
From: "Bill Huey (hui)" <[email protected]>
Quoted 16 lines · Martijn Sipkema
[...]
Quoted 5 lines
The worst case latency is the one that counts and that is the contended case. If you could guarantee no contention then the worst case latency would be the very fast uncontended case, but I doubt there are many (any?) examples of this in practice. There are valid uses of mutexes with priority inheritance/ceiling protocol; the poeple making the POSIX standard aren't stupid...
There are cases where you have to use priority inheritance, but the schemes that are typically used either have a kind of exhaustive analysis backing it or uses a simple late detection scheme. In a general purpose OS, the latter is useful for various kind of overload cases. But if your system is constantly using that specific case, then it's a sign the contention in the kernel must *also* be a problem under SMP conditions. The constant use of priority inheritance overloads the scheduler, puts pressure on the cache and other negative things that hurt CPU local performance of the system.
Quoted 5 lines
The reason why I mention this is because of Linux's hand-crafted nature of dealing with this. These are basically contention problems expressed in a different manner. The traditional Linux method is the correct method of deal with this in a general purpose OS. This also applies to application structure as well. The use of these mechanisms need to be thought out before application.
To be honest, I don't understand a word of what you are saying here. Could you give an example of a ``contention problem'' and how it should be solved?
Quoted 24 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
It is often heard in the Linux audio community that mutexes are not realtime safe and a lock-free ringbuffer should be used instead. Using such a lock-free ringbuffer requires non-standard atomic integer operations and does not guarantee memory synchronization (and should probably not perform significantly better than a decent mutex implementation) and is thus not portable.
Quoted 8 lines
Quoted 4 lines
It's to decouple the system from various time related problems with jitter. It's critical to use this since the nature of Linus is so temporally coarse that these techniques must be used to "smooth" over latency problems in the Linux kernel.
Either use mutexes or POSIX message queues... the latter also are not intended for realtime use under Linux (though they are meant for it in POSIX), since they don't allocate memory on creation.
The nature these kind of applications push into a very demanding space where typical methodologies surrounding the use of threads goes out the window. Pushing both the IO and CPU resources of a kernel is the common case and often you have to roll your own APIs, synchronization mechanisms to deal with these problem. Simple Posix API and traditional mutexes are a bit too narrow in scope to solve these cross system concurrency problems. It's not trivial stuff at all and can span from loosely to tightly coupled systems, yes, all for pro-audio/video.
Quoted 1 line
Posix and friends in these cases simply aren't good enough to cut it.
I find this a little abstract. Sure, there might be areas where POSIX doesn't supply all the needed tools, e.g. one might want some scheduling policy especially for audio, but to say that POSIX isn't good enough without providing much explanation...
--ms