rtc: lost some interrupts at 256Hz
When you google for such messages, you can find a lot of people asking, but nobody seems to have an answer. That's why I ask this list, where the Godfathers Of Linux reside, and maybe someone hears my prayer and could explain us sheep what you should do in such a case. Increase the HZ from 250 to 1000, or decrease to 100? Or maybe setting the preemption model from server to voluntary or preemptible? Or is that whining to be ignored, and if yes, what is this message for at all?
Please give us wisdom, and we will spread your word. Amen.
Answers please per PM, I'm not on this list.
mfg zmi *or could you ask in a nicer way?*
Signature · 5 lines
// Michael Monnerie, Ing.BSc ----- http://it-management.at // Tel: 0660/4156531 .network.your.ideas. // PGP Key: "lynx -source http://zmi.at/zmi3.asc | gpg --import" // Fingerprint: 44A3 C1EC B71E C71A B4C2 9AA6 C818 847C 55CB A4EE // Keyserver: www.keyserver.net Key-ID: 0x55CBA4EE
Quoted 8 lines · Michael Monnerie
When you google for such messages, you can find a lot of people asking, but nobody seems to have an answer. That's why I ask this list, where the Godfathers Of Linux reside, and maybe someone hears my prayer and could explain us sheep what you should do in such a case. Increase the HZ from 250 to 1000, or decrease to 100? Or maybe setting the preemption model from server to voluntary or preemptible? Or is that whining to be ignored, and if yes, what is this message for at all?
Quoted 4 lines
Please give us wisdom, and we will spread your word. Amen.
Answers please per PM, I'm not on this list.
If you are losing interrupts at 256 Hz, you have either/or:
(1) Some very BAD driver that is disabling interrupts for way too long.
(2) Some very slow CPU (like 40 Mhz) that is being overwhelmed by a lot of network interrupt activity.
The last time I measured, with a 400 MHz, '486 machine, Linux could handle 50,000 interrupts per second off the printer port, with the lowest priority interrupt (IRQ7). The work, within the interrupt was to toggle a bit in memory and write it back to the printer data port.
This was used to show latency (with a scope) and log any missed interrupts of which there were none.
Cheers,
Dick Johnson
Penguin : Linux version 2.6.16.4 on an i686 machine (5592.89 BogoMips). Warning : 98.36% of all statistics are fiction, book release in April. _
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
Quoted 8 lines · Michael Monnerie
When you google for such messages, you can find a lot of people asking, but nobody seems to have an answer. That's why I ask this list, where the Godfathers Of Linux reside, and maybe someone hears my prayer and could explain us sheep what you should do in such a case. Increase the HZ from 250 to 1000, or decrease to 100? Or maybe setting the preemption model from server to voluntary or preemptible? Or is that whining to be ignored, and if yes, what is this message for at all?
Quoted 5 lines
Please give us wisdom, and we will spread your word. Amen.
Answers please per PM, I'm not on this list.
mfg zmi *or could you ask in a nicer way?*
Changing the preemption model to voluntary or full preemption could certainly help. What app is using the RTC, mplayer?
Lee
Quoted 2 lines · On Thu, 20 Apr 2006 17:03:02 -0400 Lee Revell <[email protected]> wrote:
Changing the preemption model to voluntary or full preemption could certainly help. What app is using the RTC, mplayer?
I wonder what the "lost ticks" message really means. That the tick wasn't consumed by a read on the device file? If so, setting niceness or rt prio for the consuming task might help.
Or does it mean that the kernel failed in some way keeping up with the rtc?
Flo
Signature · 2 lines
Palimm Palimm! http://tapas.affenbande.org
Quoted 18 lines · linux-os (Dick Johnson)
Quoted 11 lines · Michael Monnerie
When you google for such messages, you can find a lot of people asking, but nobody seems to have an answer. That's why I ask this list, where the Godfathers Of Linux reside, and maybe someone hears my prayer and could explain us sheep what you should do in such a case. Increase the HZ from 250 to 1000, or decrease to 100? Or maybe setting the preemption model from server to voluntary or preemptible? Or is that whining to be ignored, and if yes, what is this message for at all?
Please give us wisdom, and we will spread your word. Amen.
Answers please per PM, I'm not on this list.
If you are losing interrupts at 256 Hz, you have either/or:
(1) Some very BAD driver that is disabling interrupts for way too long.
(2) Some very slow CPU (like 40 Mhz) that is being overwhelmed by a lot of network interrupt activity.
Quoted 3 lines · Lee Revell
Date: Thu, 20 Apr 2006 17:03:02 -0400
Changing the preemption model to voluntary or full preemption could certainly help. What app is using the RTC, mplayer?
I see this message too. (And I've got some more details).
Whenever I switch from console to X, I get that message. I presume it is due the binary nvidia blob I have loaded, because when switching to X, even the sound gets distorted for a microsecond (the well-known the soundcard replays the sound buffer - no new data transferred to it). Yeah, looks like something's keeping IRQ disabled for quite a while. (My normal freq is 100 Hz + no preemption, if that makes any difference.)
I have not yet seen this on the free nv driver for Xorg or the (free) ati/Xorg. Same configuration (100 Hz, no preempt), though!
Does not really matter what userspace app is running. If it's mplayer the message is "rtc: lost some interrupts at 1024Hz.", and if it's VMware it's whatever the Guest OS requires, mostly 2000 or 200 Hz.
Jan Engelhardt
Quoted 3 lines · Jan Engelhardt
Does not really matter what userspace app is running. If it's mplayer the message is "rtc: lost some interrupts at 1024Hz.", and if it's VMware it's whatever the Guest OS requires, mostly 2000 or 200 Hz.
I asked about the app because I was wondering whether he had a server or a desktop.
Lee
Quoted 2 lines · On Freitag, 21. April 2006 00:22 Lee Revell wrote:
I asked about the app because I was wondering whether he had a server or a desktop.
It's all servers. The Hz rate can change over time:
Mar 3 02:04:41 baum kernel: rtc: lost some interrupts at 256Hz. Mar 3 02:04:53 baum kernel: rtc: lost some interrupts at 256Hz. Mar 3 02:04:55 baum kernel: rtc: lost some interrupts at 256Hz. Mar 3 02:38:19 baum kernel: rtc: lost some interrupts at 256Hz. Mar 3 02:42:22 baum kernel: rtc: lost some interrupts at 256Hz. Mar 16 15:53:50 baum kernel: rtc: lost some interrupts at 2048Hz. Mar 16 15:54:01 baum kernel: rtc: lost some interrupts at 2048Hz. Mar 16 16:29:33 baum kernel: rtc: lost some interrupts at 2048Hz. Mar 29 23:01:36 baum kernel: warning: many lost ticks.
Interesting may be the fact that VMware runs on at least some hosts. I didn't check if all servers have it, but I found a good article on vmware support:
So as a summary:
- Lee Revell said changing preemption model from server to voluntary or preemption could help
- increasing HZ on the vm-host, decreasing in the vm-clients should help, according to VMware support
It would be good if this information is spread on the support lists, I saw that question a lot. Also, a short text under Documentation in the kernel source could be nice, explaining why this happens and what to do then.
I will try the suggestion from the vmware support link I posted above, and will report back.
mfg zmi
Signature · 5 lines
// Michael Monnerie, Ing.BSc ----- http://it-management.at // Tel: 0660/4156531 .network.your.ideas. // PGP Key: "lynx -source http://zmi.at/zmi3.asc | gpg --import" // Fingerprint: 44A3 C1EC B71E C71A B4C2 9AA6 C818 847C 55CB A4EE // Keyserver: www.keyserver.net Key-ID: 0x55CBA4EE
Quoted 3 lines · On Freitag, 21. April 2006 00:18 Jan Engelhardt wrote:
Does not really matter what userspace app is running. If it's mplayer the message is "rtc: lost some interrupts at 1024Hz.", and if it's VMware it's whatever the Guest OS requires, mostly 2000 or 200 Hz.
The tipps from vmware support at
http://www.vmware.com/support/kb/enduser/std_adp.php?p_sid=tthOqz5i&p_lva=&p_faqid=1420&p_created=1093994398&p_sp=cF9zcmNoPTEmcF9ncmlkc29ydD0mcF9yb3dfY250PTE3JnBfc2VhcmNoX3RleHQ9bG9zdCBzb21lIGludGVycnJ1cHRzJnBfc2VhcmNoX3R5cGU9NyZwX3Byb2RfbHZsMT1_YW55fiZwX3Byb2RfbHZsMj1_YW55fiZwX3NvcnRfYnk9ZGZsdCZwX3BhZ2U9MQ**&p_li=
seem to help, didn't get a "lost some interrupts" warning during the last 2 days. But, I got one "many lost ticks" warning:
Apr 23 23:06:51 baum kernel: warning: many lost ticks. Apr 23 23:06:51 baum kernel: Your time source seems to be instable or some driver is hogging interupts Apr 23 23:06:51 baum kernel: rip __do_softirq+0x4f/0xd0
What I did was
- setup VM host OS to 1000HZ, preemptible kernel
- VM clients use 100HZ, and kernel options "clock=pit nosmp noapic nolapic"
- VM client kernels are all non-SMP compiled now.
Still, some kind of documentation in the Linux source tree would be nice, explaining what that message means and how to fix it.
mfg zmi
Signature · 5 lines
// Michael Monnerie, Ing.BSc ----- http://it-management.at // Tel: 0660/4156531 .network.your.ideas. // PGP Key: "lynx -source http://zmi.at/zmi3.asc | gpg --import" // Fingerprint: 44A3 C1EC B71E C71A B4C2 9AA6 C818 847C 55CB A4EE // Keyserver: www.keyserver.net Key-ID: 0x55CBA4EE
Quoted 3 lines · Michael Monnerie
Still, some kind of documentation in the Linux source tree would be nice, explaining what that message means and how to fix it.
It means exactly what the printk says:
"Your time source seems to be instable or
some driver is hogging interupts"
Lee