preempt-timing-2.6.8-rc1
Subsystems (12)
Re: [patch] voluntary-preempt-2.6.8.1-P4
In reply to Ingo MolnarQuoted 1 line · Lee Revell
Quoted 1 line
Any comments on the
unmap_vmasissue?
Quoted 2 lines · Ingo Molnar
wli indicated he's working on the pagetable zapping critical section issue - wli?
What I have is meant to remove clear_page_tables() by incrementally pruning at munmap() -time. As far as unmap_vmas() goes, it's not directly pertinent. It rather affects (and is meant to remove) clear_page_tables().
-- wli
Re: [patch] voluntary-preempt-2.6.8.1-P4
In reply to Ingo MolnarQuoted 4 lines · On Thu, 19 Aug 2004 09:32:47 +0200 Ingo Molnar <[email protected]> wrote:
i've uploaded the -P4 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P4
Hi, this patch doesn't really apply cleanly. Do i need to worry about the "fuzz" and the offsets?
flo
patch -p1 --dry-run < /home/tapas/downloads/voluntary-preempt-2.6.8.1-P4 patching file arch/i386/kernel/entry.S
patching file arch/i386/kernel/i386_ksyms.c
patching file arch/i386/kernel/i8259.c
patching file arch/i386/kernel/io_apic.c
patching file arch/i386/kernel/irq.c
Hunk #12 succeeded at 656 with fuzz 2.
patching file arch/i386/kernel/apic.c
patching file arch/i386/kernel/smp.c
patching file arch/i386/kernel/traps.c
patching file arch/i386/mm/fault.c
patching file arch/i386/Kconfig
patching file arch/i386/lib/usercopy.c
patching file arch/i386/mach-visws/setup.c
patching file arch/i386/mach-visws/visws_apic.c
patching file arch/i386/mach-voyager/setup.c
patching file arch/i386/boot/compressed/misc.c
patching file arch/i386/mach-default/setup.c
patching file include/asm-ppc64/uaccess.h
patching file include/asm-m68k/uaccess.h
patching file include/asm-parisc/uaccess.h
patching file include/asm-mips/uaccess.h
patching file include/asm-ppc/uaccess.h
patching file include/asm-x86_64/uaccess.h
patching file include/asm-ia64/uaccess.h
patching file include/linux/irq.h
patching file include/linux/sched.h
patching file include/linux/blkdev.h
patching file include/linux/buffer_head.h
patching file include/linux/interrupt.h
patching file include/linux/kernel.h
patching file include/linux/kernel_stat.h
patching file include/linux/pagemap.h
patching file include/linux/preempt.h
patching file include/linux/sysctl.h
patching file include/linux/writeback.h
patching file include/asm-m68knommu/uaccess.h
patching file include/asm-um/uaccess.h
patching file include/asm-arm/uaccess.h
patching file include/asm-sparc/uaccess.h
patching file include/asm-alpha/uaccess.h
patching file include/asm-h8300/uaccess.h
patching file include/asm-i386/irq.h
patching file include/asm-i386/checksum.h
patching file include/asm-i386/hardirq.h
patching file include/asm-i386/hw_irq.h
patching file include/asm-i386/io_apic.h
patching file include/asm-i386/signal.h
patching file include/asm-i386/uaccess.h
patching file include/asm-i386/system.h
patching file include/asm-s390/uaccess.h
patching file include/asm-cris/uaccess.h
patching file include/asm-sparc64/uaccess.h
patching file include/asm-arm26/uaccess.h
patching file include/asm-v850/uaccess.h
patching file include/asm-sh/uaccess.h
patching file include/asm-sh64/uaccess.h
patching file drivers/char/tty_io.c
patching file drivers/char/random.c
Hunk #1 succeeded at 947 (offset 5 lines).
patching file drivers/block/ll_rw_blk.c
Hunk #3 succeeded at 2400 (offset 1 line).
Hunk #4 succeeded at 2448 (offset 1 line).
Hunk #5 succeeded at 3065 (offset 3 lines).
Hunk #6 succeeded at 3132 (offset 3 lines).
patching file drivers/input/keyboard/atkbd.c
patching file drivers/input/mouse/psmouse-base.c
patching file drivers/ide/ide-io.c
patching file drivers/md/dm-table.c
patching file drivers/md/linear.c
patching file drivers/md/multipath.c
patching file drivers/md/raid0.c
patching file drivers/md/raid1.c
patching file fs/ext3/ialloc.c
patching file fs/ext3/inode.c
patching file fs/ext3/namei.c
patching file fs/ext3/super.c
patching file fs/proc/proc_misc.c
patching file fs/proc/task_mmu.c
patching file fs/jbd/checkpoint.c
patching file fs/jbd/commit.c
patching file fs/jbd/revoke.c
patching file fs/buffer.c
patching file fs/dcache.c
patching file fs/exec.c
patching file fs/file_table.c
patching file fs/fs-writeback.c
patching file fs/inode.c
patching file fs/select.c
patching file net/ipv4/route.c
patching file net/ipv4/tcp_minisocks.c
patching file net/core/dev.c
patching file net/core/sock.c
patching file kernel/Makefile
patching file kernel/exit.c
patching file kernel/hardirq.c
patching file kernel/kthread.c
patching file kernel/printk.c
patching file kernel/profile.c
patching file kernel/rcupdate.c
patching file kernel/sched.c
patching file kernel/softirq.c
patching file kernel/sysctl.c
patching file kernel/timer.c
patching file kernel/latency.c
patching file kernel/kallsyms.c
patching file mm/filemap.c
patching file mm/memory.c
patching file mm/mempool.c
patching file mm/mmap.c
patching file mm/slab.c
patching file mm/vmscan.c
patching file mm/msync.c
patching file init/Kconfig
patching file init/main.c
patching file Makefile
Re: [patch] voluntary-preempt-2.6.8.1-P4
Quoted 11 lines · Florian Schmidt
Quoted 4 lines · On Thu, 19 Aug 2004 09:32:47 +0200 Ingo Molnar <[email protected]> wrote:
i've uploaded the -P4 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P4
Hi, this patch doesn't really apply cleanly. Do i need to worry about the "fuzz" and the offsets?
Does not seem to hurt anything. Ingo probably has some differences in his tree, the 'fuzz' just means that patch was smart enough to figure out that they don't matter.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P4
Quoted 8 lines · Lee Revell
Quoted 4 lines
Hi, this patch doesn't really apply cleanly. Do i need to worry about the "fuzz" and the offsets?
Does not seem to hurt anything. Ingo probably has some differences in his tree, the 'fuzz' just means that patch was smart enough to figure out that they don't matter.
yeah, it shouldnt be an issue, until there's no reject. (I'll rebuild my tree because i thought i have a vanilla tree, but apparently not.)
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P4
In reply to Ingo MolnarQuoted 9 lines · --- Ingo Molnar
i've uploaded the -P4 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P4
the main change is a more robust latency tracer - the previous one was not 100% correct for interrupts. People who have exprienced those weird ~1msec latencies in the idle task please re-check and re-post latency traces.
I receive the following errors on compile, after make has gotten through building all config options:
-- SNIP --
GEN .version CHK include/linux/compile.h UPD include/linux/compile.h CC init/version.o LD init/built-in.o LD .tmp_vmlinux1
arch/i386/kernel/built-in.o(.text+0x398e): In function `do_nmi':
: undefined reference to __trace' arch/i386/kernel/built-in.o(.text+0x4101): In function do_IRQ':
: undefined reference to __trace' arch/i386/kernel/built-in.o(.text+0xe1e9): In function smp_apic_timer_interrupt':
: undefined reference to __trace' arch/i386/mm/built-in.o(.text+0x71b): In function do_page_fault':
: undefined reference to `__trace'
make: *** [.tmp_vmlinux1] Error 1
-P2 and -P3 gave me the same errors except for `smp_apic_timer_interrupt'.
I have tried my .config, default, allyes, and allno, and I get the same result regardless of config. Tree is fresh, patch applied with -Np1 <../[patchname]. I also get fuzz and offsets, but they don't seem to bear on the problem directly. I'm currently running -P1, which doesn't have this problem. gcc is 3.3.4.
With thanks for a smoother system feel,
Matthew Frost
artusemrys at sbcglobal dot net
Re: [patch] voluntary-preempt-2.6.8.1-P4
In reply to Ingo MolnarWhen logging into my Fedora Core 2 box on the console (normal vga text console), I get the following latency trace:
(preempt=1 voluntary-preempt=3 elevator=cfq)
preemption latency trace v1.0.1
-------------------------------
latency: 3568 us, entries: 137 (137)
-----------------
| task: setfont/2209, uid:500 nice:0 policy:0 rt_prio:0
----------------- => started at: voluntary_resched+0x31/0x60
=> ended at: voluntary_resched+0x31/0x60
=======>
0.000ms (+0.000ms): touch_preempt_timing (voluntary_resched) 0.000ms (+0.000ms): vgacon_font_set (con_font_set) 0.000ms (+0.000ms): vgacon_do_font_op (vgacon_font_set) 0.057ms (+0.056ms): smp_apic_timer_interrupt (vgacon_do_font_op) 0.057ms (+0.000ms): profile_hook (smp_apic_timer_interrupt) 0.057ms (+0.000ms): notifier_call_chain (profile_hook) 0.120ms (+0.063ms): do_IRQ (vgacon_do_font_op)
0.121ms (+0.000ms): ack_edge_ioapic_irq (do_IRQ) 0.121ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 0.121ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 0.121ms (+0.000ms): timer_interrupt (generic_handle_IRQ_event) 0.121ms (+0.000ms): mark_offset_pmtmr (timer_interrupt) 0.128ms (+0.006ms): do_timer (timer_interrupt)
0.128ms (+0.000ms): update_process_times (do_timer) 0.128ms (+0.000ms): update_one_process (update_process_times) 0.128ms (+0.000ms): run_local_timers (update_process_times) 0.128ms (+0.000ms): raise_softirq (update_process_times) 0.128ms (+0.000ms): scheduler_tick (update_process_times) 0.128ms (+0.000ms): sched_clock (scheduler_tick) 0.128ms (+0.000ms): rcu_check_callbacks (scheduler_tick) 0.128ms (+0.000ms): idle_cpu (rcu_check_callbacks) 0.129ms (+0.000ms): __tasklet_schedule (scheduler_tick) 0.129ms (+0.000ms): update_wall_time (do_timer) 0.129ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 0.129ms (+0.000ms): generic_note_interrupt (do_IRQ) 0.129ms (+0.000ms): end_edge_ioapic_irq (do_IRQ) 0.130ms (+0.000ms): do_softirq (do_IRQ)
0.130ms (+0.000ms): __do_softirq (do_softirq)
0.130ms (+0.000ms): wake_up_process (do_softirq) 0.130ms (+0.000ms): try_to_wake_up (wake_up_process) 0.130ms (+0.000ms): task_rq_lock (try_to_wake_up) 0.130ms (+0.000ms): activate_task (try_to_wake_up) 0.130ms (+0.000ms): sched_clock (activate_task) 0.130ms (+0.000ms): recalc_task_prio (activate_task) 0.130ms (+0.000ms): effective_prio (recalc_task_prio) 0.130ms (+0.000ms): enqueue_task (activate_task) 1.058ms (+0.927ms): smp_apic_timer_interrupt (vgacon_do_font_op) 1.058ms (+0.000ms): profile_hook (smp_apic_timer_interrupt) 1.058ms (+0.000ms): notifier_call_chain (profile_hook) 1.058ms (+0.000ms): preempt_schedule (smp_apic_timer_interrupt) 1.058ms (+0.000ms): do_softirq (smp_apic_timer_interrupt) 1.058ms (+0.000ms): __do_softirq (do_softirq)
1.122ms (+0.063ms): do_IRQ (vgacon_do_font_op)
1.122ms (+0.000ms): ack_edge_ioapic_irq (do_IRQ) 1.122ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 1.122ms (+0.000ms): preempt_schedule (do_IRQ)
1.122ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 1.122ms (+0.000ms): timer_interrupt (generic_handle_IRQ_event) 1.123ms (+0.000ms): mark_offset_pmtmr (timer_interrupt) 1.129ms (+0.006ms): do_timer (timer_interrupt)
1.129ms (+0.000ms): update_process_times (do_timer) 1.129ms (+0.000ms): update_one_process (update_process_times) 1.129ms (+0.000ms): run_local_timers (update_process_times) 1.129ms (+0.000ms): raise_softirq (update_process_times) 1.129ms (+0.000ms): scheduler_tick (update_process_times) 1.129ms (+0.000ms): sched_clock (scheduler_tick) 1.129ms (+0.000ms): rcu_check_callbacks (scheduler_tick) 1.129ms (+0.000ms): idle_cpu (rcu_check_callbacks) 1.130ms (+0.000ms): update_wall_time (do_timer) 1.130ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 1.130ms (+0.000ms): generic_note_interrupt (do_IRQ) 1.130ms (+0.000ms): end_edge_ioapic_irq (do_IRQ) 1.130ms (+0.000ms): preempt_schedule (do_IRQ)
1.130ms (+0.000ms): do_softirq (do_IRQ)
1.130ms (+0.000ms): __do_softirq (do_softirq)
2.059ms (+0.928ms): smp_apic_timer_interrupt (vgacon_do_font_op) 2.059ms (+0.000ms): profile_hook (smp_apic_timer_interrupt) 2.059ms (+0.000ms): notifier_call_chain (profile_hook) 2.059ms (+0.000ms): preempt_schedule (smp_apic_timer_interrupt) 2.059ms (+0.000ms): do_softirq (smp_apic_timer_interrupt) 2.059ms (+0.000ms): __do_softirq (do_softirq)
2.123ms (+0.063ms): do_IRQ (vgacon_do_font_op)
2.123ms (+0.000ms): ack_edge_ioapic_irq (do_IRQ) 2.123ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 2.123ms (+0.000ms): preempt_schedule (do_IRQ)
2.123ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 2.123ms (+0.000ms): timer_interrupt (generic_handle_IRQ_event) 2.123ms (+0.000ms): mark_offset_pmtmr (timer_interrupt) 2.130ms (+0.006ms): do_timer (timer_interrupt)
2.130ms (+0.000ms): update_process_times (do_timer) 2.130ms (+0.000ms): update_one_process (update_process_times) 2.130ms (+0.000ms): run_local_timers (update_process_times) 2.130ms (+0.000ms): raise_softirq (update_process_times) 2.130ms (+0.000ms): scheduler_tick (update_process_times) 2.130ms (+0.000ms): sched_clock (scheduler_tick) 2.130ms (+0.000ms): rcu_check_callbacks (scheduler_tick) 2.131ms (+0.000ms): idle_cpu (rcu_check_callbacks) 2.131ms (+0.000ms): dequeue_task (scheduler_tick) 2.131ms (+0.000ms): effective_prio (scheduler_tick) 2.131ms (+0.000ms): task_timeslice (scheduler_tick) 2.131ms (+0.000ms): enqueue_task (scheduler_tick) 2.131ms (+0.000ms): update_wall_time (do_timer) 2.131ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 2.131ms (+0.000ms): generic_note_interrupt (do_IRQ) 2.132ms (+0.000ms): end_edge_ioapic_irq (do_IRQ) 2.132ms (+0.000ms): preempt_schedule (do_IRQ)
2.132ms (+0.000ms): do_softirq (do_IRQ)
2.132ms (+0.000ms): __do_softirq (do_softirq)
3.060ms (+0.927ms): smp_apic_timer_interrupt (vgacon_do_font_op) 3.060ms (+0.000ms): profile_hook (smp_apic_timer_interrupt) 3.060ms (+0.000ms): notifier_call_chain (profile_hook) 3.060ms (+0.000ms): preempt_schedule (smp_apic_timer_interrupt) 3.060ms (+0.000ms): do_softirq (smp_apic_timer_interrupt) 3.060ms (+0.000ms): __do_softirq (do_softirq)
3.124ms (+0.063ms): do_IRQ (vgacon_do_font_op)
3.124ms (+0.000ms): ack_edge_ioapic_irq (do_IRQ) 3.124ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 3.124ms (+0.000ms): preempt_schedule (do_IRQ)
3.124ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 3.124ms (+0.000ms): timer_interrupt (generic_handle_IRQ_event) 3.124ms (+0.000ms): mark_offset_pmtmr (timer_interrupt) 3.131ms (+0.006ms): do_timer (timer_interrupt)
3.131ms (+0.000ms): update_process_times (do_timer) 3.131ms (+0.000ms): update_one_process (update_process_times) 3.131ms (+0.000ms): run_local_timers (update_process_times) 3.131ms (+0.000ms): raise_softirq (update_process_times) 3.131ms (+0.000ms): scheduler_tick (update_process_times) 3.131ms (+0.000ms): sched_clock (scheduler_tick) 3.131ms (+0.000ms): rcu_check_callbacks (scheduler_tick) 3.131ms (+0.000ms): idle_cpu (rcu_check_callbacks) 3.131ms (+0.000ms): update_wall_time (do_timer) 3.131ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 3.132ms (+0.000ms): generic_note_interrupt (do_IRQ) 3.132ms (+0.000ms): end_edge_ioapic_irq (do_IRQ) 3.132ms (+0.000ms): preempt_schedule (do_IRQ)
3.132ms (+0.000ms): do_softirq (do_IRQ)
3.132ms (+0.000ms): __do_softirq (do_softirq)
3.485ms (+0.353ms): preempt_schedule (vgacon_do_font_op) 3.486ms (+0.000ms): vgacon_adjust_height (vgacon_font_set) 3.486ms (+0.000ms): release_console_sem (con_font_set) 3.486ms (+0.000ms): preempt_schedule (release_console_sem) 3.486ms (+0.000ms): kfree (con_font_set)
3.487ms (+0.000ms): copy_to_user (vt_ioctl)
3.487ms (+0.000ms): __might_sleep (copy_to_user) 3.487ms (+0.000ms): voluntary_resched (copy_to_user) 3.487ms (+0.000ms): __might_sleep (voluntary_resched) 3.487ms (+0.000ms): touch_preempt_timing (voluntary_resched)
$ cat /proc/interrupts
CPU0
0: 1286100 IO-APIC-edge timer
1: 4572 IO-APIC-edge i8042
8: 1 IO-APIC-edge rtc
9: 10 IO-APIC-level acpi
12: 86351 IO-APIC-edge i8042
14: 14089 IO-APIC-edge ide0
15: 477 IO-APIC-edge ide1
19: 13568 IO-APIC-level eth0
20: 94 IO-APIC-level ehci_hcd, ohci_hcd
21: 0 IO-APIC-level ohci_hcd
22: 593 IO-APIC-level NVidia nForce3
NMI: 0
LOC: 1286057
ERR: 0
MIS: 0Machine is an ACER Aspire 1510 notebook
(AMD64 processor running a 32-bit kernel)
Re: [patch] voluntary-preempt-2.6.8.1-P4
The following latency trace is generated each time the sound driver is opened by an application on my box.
# lspci -s 00:06.0 00:06.0 Multimedia audio controller: nVidia Corporation nForce3 Audio (rev a2)
Audio driver:
# lsmod|grep snd
snd_seq 51856 0 snd_intel8x0 30600 0 snd_ac97_codec 66820 1 snd_intel8x0 snd_pcm 88264 1 snd_intel8x0 snd_timer 22596 2 snd_seq,snd_pcm snd_page_alloc 9288 2 snd_intel8x0,snd_pcm snd_mpu401_uart 6656 1 snd_intel8x0 snd_rawmidi 21156 1 snd_mpu401_uart snd_seq_device 7176 2 snd_seq,snd_rawmidi snd 48804 8 snd_seq,snd_intel8x0,snd_ac97_codec,snd_pcm,snd_timer,snd_mpu401_uart,snd_rawmidi,snd_seq_device soundcore 7840 1 snd
preemption latency trace v1.0.1
-------------------------------
latency: 50752 us, entries: 267 (267)
-----------------
| task: artsd/2502, uid:500 nice:0 policy:0 rt_prio:0
----------------- => started at: voluntary_resched+0x31/0x60
=> ended at: sys_ioctl+0xf4/0x2b0
=======>
0.000ms (+0.000ms): touch_preempt_timing (voluntary_resched) 0.000ms (+0.000ms): snd_power_wait (snd_pcm_prepare)
0.000ms (+0.000ms): snd_pcm_action_lock_irq (snd_pcm_prepare) 0.001ms (+0.000ms): snd_pcm_action_single (snd_pcm_action_lock_irq) 0.001ms (+0.000ms): snd_pcm_pre_prepare (snd_pcm_action_single) 0.001ms (+0.000ms): snd_pcm_do_prepare (snd_pcm_action_single) 0.001ms (+0.000ms): snd_intel8x0_pcm_prepare (snd_pcm_do_prepare) 0.002ms (+0.000ms): snd_intel8x0_setup_pcm_out (snd_intel8x0_pcm_prepare) 0.002ms (+0.000ms): igetdword (snd_intel8x0_setup_pcm_out) 0.002ms (+0.000ms): iputdword (snd_intel8x0_setup_pcm_out) 0.003ms (+0.000ms): __const_udelay (snd_intel8x0_setup_pcm_out) 0.003ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 0.003ms (+0.000ms): delay_pmtmr (__delay)
0.995ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 0.995ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 0.995ms (+0.000ms): delay_pmtmr (__delay)
1.986ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 1.986ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 1.986ms (+0.000ms): delay_pmtmr (__delay)
2.978ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 2.978ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 2.978ms (+0.000ms): delay_pmtmr (__delay)
3.969ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 3.969ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 3.970ms (+0.000ms): delay_pmtmr (__delay)
4.961ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 4.961ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 4.961ms (+0.000ms): delay_pmtmr (__delay)
5.953ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 5.953ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 5.953ms (+0.000ms): delay_pmtmr (__delay)
6.944ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 6.944ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 6.944ms (+0.000ms): delay_pmtmr (__delay)
7.936ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 7.936ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 7.936ms (+0.000ms): delay_pmtmr (__delay)
8.927ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 8.927ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 8.927ms (+0.000ms): delay_pmtmr (__delay)
9.919ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 9.919ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 9.919ms (+0.000ms): delay_pmtmr (__delay)
10.910ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 10.910ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 10.911ms (+0.000ms): delay_pmtmr (__delay)
11.902ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 11.902ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 11.902ms (+0.000ms): delay_pmtmr (__delay)
12.893ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 12.894ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 12.894ms (+0.000ms): delay_pmtmr (__delay)
13.885ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 13.885ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 13.885ms (+0.000ms): delay_pmtmr (__delay)
14.877ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 14.877ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 14.877ms (+0.000ms): delay_pmtmr (__delay)
15.868ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 15.868ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 15.868ms (+0.000ms): delay_pmtmr (__delay)
16.860ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 16.860ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 16.860ms (+0.000ms): delay_pmtmr (__delay)
17.851ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 17.851ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 17.851ms (+0.000ms): delay_pmtmr (__delay)
18.843ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 18.843ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 18.843ms (+0.000ms): delay_pmtmr (__delay)
19.834ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 19.835ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 19.835ms (+0.000ms): delay_pmtmr (__delay)
20.826ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 20.826ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 20.826ms (+0.000ms): delay_pmtmr (__delay)
21.818ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 21.818ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 21.818ms (+0.000ms): delay_pmtmr (__delay)
22.809ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 22.809ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 22.809ms (+0.000ms): delay_pmtmr (__delay)
23.801ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 23.801ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 23.801ms (+0.000ms): delay_pmtmr (__delay)
24.792ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 24.792ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 24.792ms (+0.000ms): delay_pmtmr (__delay)
25.784ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 25.784ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 25.784ms (+0.000ms): delay_pmtmr (__delay)
26.775ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 26.775ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 26.776ms (+0.000ms): delay_pmtmr (__delay)
27.767ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 27.767ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 27.767ms (+0.000ms): delay_pmtmr (__delay)
28.759ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 28.759ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 28.759ms (+0.000ms): delay_pmtmr (__delay)
29.750ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 29.750ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 29.750ms (+0.000ms): delay_pmtmr (__delay)
30.742ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 30.742ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 30.742ms (+0.000ms): delay_pmtmr (__delay)
31.733ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 31.733ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 31.733ms (+0.000ms): delay_pmtmr (__delay)
32.725ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 32.725ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 32.725ms (+0.000ms): delay_pmtmr (__delay)
33.716ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 33.716ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 33.717ms (+0.000ms): delay_pmtmr (__delay)
34.708ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 34.708ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 34.708ms (+0.000ms): delay_pmtmr (__delay)
35.700ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 35.700ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 35.700ms (+0.000ms): delay_pmtmr (__delay)
36.691ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 36.691ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 36.691ms (+0.000ms): delay_pmtmr (__delay)
37.683ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 37.683ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 37.683ms (+0.000ms): delay_pmtmr (__delay)
38.674ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 38.674ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 38.674ms (+0.000ms): delay_pmtmr (__delay)
39.666ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 39.666ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 39.666ms (+0.000ms): delay_pmtmr (__delay)
40.657ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 40.657ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 40.657ms (+0.000ms): delay_pmtmr (__delay)
41.649ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 41.649ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 41.649ms (+0.000ms): delay_pmtmr (__delay)
42.640ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 42.641ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 42.641ms (+0.000ms): delay_pmtmr (__delay)
43.632ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 43.632ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 43.632ms (+0.000ms): delay_pmtmr (__delay)
44.624ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 44.624ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 44.624ms (+0.000ms): delay_pmtmr (__delay)
45.615ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 45.615ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 45.615ms (+0.000ms): delay_pmtmr (__delay)
46.607ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 46.607ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 46.607ms (+0.000ms): delay_pmtmr (__delay)
47.598ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 47.598ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 47.598ms (+0.000ms): delay_pmtmr (__delay)
48.590ms (+0.991ms): __const_udelay (snd_intel8x0_setup_pcm_out) 48.590ms (+0.000ms): __delay (snd_intel8x0_setup_pcm_out) 48.590ms (+0.000ms): delay_pmtmr (__delay)
49.581ms (+0.991ms): iputdword (snd_intel8x0_setup_pcm_out) 49.582ms (+0.000ms): snd_intel8x0_setup_periods (snd_intel8x0_pcm_prepare) 49.582ms (+0.000ms): iputdword (snd_intel8x0_setup_periods) 49.585ms (+0.002ms): iputbyte (snd_intel8x0_setup_periods) 49.585ms (+0.000ms): iputbyte (snd_intel8x0_setup_periods) 49.586ms (+0.000ms): iputbyte (snd_intel8x0_setup_periods) 49.586ms (+0.000ms): snd_pcm_do_reset (snd_pcm_action_single) 49.586ms (+0.000ms): snd_pcm_lib_ioctl (snd_pcm_do_reset) 49.586ms (+0.000ms): snd_pcm_lib_ioctl_reset (snd_pcm_do_reset) 49.587ms (+0.000ms): snd_pcm_post_prepare (snd_pcm_action_single) 49.587ms (+0.000ms): smp_apic_timer_interrupt (snd_pcm_action_lock_irq) 49.587ms (+0.000ms): profile_hook (smp_apic_timer_interrupt) 49.588ms (+0.000ms): notifier_call_chain (profile_hook) 49.588ms (+0.000ms): do_IRQ (snd_pcm_action_lock_irq)
49.589ms (+0.000ms): ack_edge_ioapic_irq (do_IRQ)
49.589ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 49.589ms (+0.000ms): wake_up_process (generic_redirect_hardirq) 49.589ms (+0.000ms): try_to_wake_up (wake_up_process)
49.589ms (+0.000ms): task_rq_lock (try_to_wake_up)
49.589ms (+0.000ms): activate_task (try_to_wake_up)
49.589ms (+0.000ms): sched_clock (activate_task)
49.590ms (+0.000ms): recalc_task_prio (activate_task)
49.590ms (+0.000ms): effective_prio (recalc_task_prio) 49.590ms (+0.000ms): enqueue_task (activate_task)
49.590ms (+0.000ms): preempt_schedule (try_to_wake_up) 49.590ms (+0.000ms): preempt_schedule (do_IRQ)
49.591ms (+0.000ms): do_IRQ (snd_pcm_action_lock_irq)
49.591ms (+0.000ms): ack_edge_ioapic_irq (do_IRQ)
49.591ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 49.591ms (+0.000ms): preempt_schedule (do_IRQ)
49.591ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 49.591ms (+0.000ms): timer_interrupt (generic_handle_IRQ_event) 49.592ms (+0.000ms): mark_offset_pmtmr (timer_interrupt) 49.598ms (+0.006ms): do_timer (timer_interrupt)
49.598ms (+0.000ms): update_process_times (do_timer)
49.598ms (+0.000ms): update_one_process (update_process_times) 49.598ms (+0.000ms): run_local_timers (update_process_times) 49.598ms (+0.000ms): raise_softirq (update_process_times) 49.598ms (+0.000ms): scheduler_tick (update_process_times) 49.598ms (+0.000ms): sched_clock (scheduler_tick)
49.599ms (+0.000ms): task_timeslice (scheduler_tick)
49.599ms (+0.000ms): update_wall_time (do_timer)
49.599ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.599ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.599ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.599ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.599ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.599ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.600ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.601ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.602ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.603ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.603ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.603ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.603ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.603ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.603ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.603ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.603ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.603ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 49.603ms (+0.000ms): generic_note_interrupt (do_IRQ)
49.604ms (+0.000ms): end_edge_ioapic_irq (do_IRQ)
49.604ms (+0.000ms): preempt_schedule (do_IRQ)
49.604ms (+0.000ms): do_softirq (do_IRQ)
49.604ms (+0.000ms): __do_softirq (do_softirq)
49.604ms (+0.000ms): wake_up_process (do_softirq)
49.604ms (+0.000ms): try_to_wake_up (wake_up_process)
49.604ms (+0.000ms): task_rq_lock (try_to_wake_up)
49.604ms (+0.000ms): activate_task (try_to_wake_up)
49.604ms (+0.000ms): sched_clock (activate_task)
49.604ms (+0.000ms): recalc_task_prio (activate_task)
49.605ms (+0.000ms): effective_prio (recalc_task_prio) 49.605ms (+0.000ms): enqueue_task (activate_task)
49.605ms (+0.000ms): preempt_schedule (snd_pcm_action_lock_irq) 49.605ms (+0.000ms): sub_preempt_count (sys_ioctl)
sound/pci/intel8x0.c contains:
if (chip->device_type == DEVICE_NFORCE) {
/* reset to 2ch once to keep the 6 channel data in alignment,
* to start from Front Left always
*/
iputdword(chip, ICHREG(GLOB_CNT), (cnt & 0xcfffff));
mdelay(50); /* grrr... */
} else if (chip->device_type == DEVICE_INTEL_ICH4) {So I guess it's that mdelay(50) which is generating these (each time the device is opened).
Re: [patch] voluntary-preempt-2.6.8.1-P4
Quoted 3 lines · On Thu, 2004-08-19 at 16:37, [email protected] wrote:
The following latency trace is generated each time the sound driver is opened by an application on my box.
This is a pretty big trace. Please try to trim these, especially if a few lines repeat hundreds of times (common).
The comment seems to imply that the author didn't like the mdelay but it didn't work otherwise. What happens if you get rid of the mdelay?
Lee
PATCH 1/1 intel8x0 latency fix
In reply to [email protected]Quoted 4 lines · [email protected] writes:
# lspci -s 00:06.0 00:06.0 Multimedia audio controller: nVidia Corporation nForce3 Audio (rev a2)latency: 50752 us, entries: 267 (267)
Following patch fixes it for 2 channel devices (like my notebook). Although I'm not sure if this is all that useful... if there are workloads other than audio work that need low latency, then this might be useful.
Without this patch, my machine was generating the latency each time artsd (KDE) re-opened the audio device.
sound/pci/intel8x0.c +4 -2
@@ -1021,8 +1021,10 @@/* reset to 2ch once to keep the 6 channel data in alignment, * to start from Front Left always */ -iputdword (chip,ICHREG (GLOB_CNT ), (cnt &0xcfffff )); -mdelay (50 );/* grrr... */ +if (chip->multi4 || chip->multi6) { +iputdword (chip,ICHREG (GLOB_CNT ), (cnt &0xcfffff )); +mdelay (50 );/* grrr... */ + } }else if (chip->device_type ==DEVICE_INTEL_ICH4 ) {if (sample_bits >16 ) cnt |=ICH_PCM_20BIT ;
Re: [patch] intel8x0 latency fix
At Fri, 20 Aug 2004 18:19:12 +0200,
Quoted 6 lines · [email protected] wrote:
Quoted 3 lines · [email protected] writes:
# lspci -s 00:06.0 00:06.0 Multimedia audio controller: nVidia Corporation nForce3 Audio (rev a2)
Quoted 7 lines
Quoted 1 line
latency: 50752 us, entries: 267 (267)
Following patch fixes it for 2 channel devices (like my notebook). Although I'm not sure if this is all that useful... if there are workloads other than audio work that need low latency, then this might be useful.
In the recent version (linux-sound tree), we can use msleep() (that is more latency friendely) in that callback. The following patch was applied to the current version.
Takashi
sound/pci/intel8x0.c +4 -2
@@ -1024,8 +1024,10 @@/* reset to 2ch once to keep the 6 channel data in alignment, * to start from Front Left always */ -iputdword (chip,ICHREG (GLOB_CNT ), (cnt &0xcfffff )); -mdelay (50 );/* grrr... */ +if (cnt &ICH_PCM_246_MASK ) { +iputdword (chip,ICHREG (GLOB_CNT ), cnt & ~ICH_PCM_246_MASK ); +msleep (50 ); + } }else if (chip->device_type ==DEVICE_INTEL_ICH4 ) {if (sample_bits >16 ) cnt |=ICH_PCM_20BIT ;
Re: [patch] voluntary-preempt-2.6.8.1-P4
In reply to Ingo MolnarQuoted 3 lines · Ingo Molnar
i've uploaded the -P4 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P4
I think I am seeing those weird ~1 ms latencies still (actually I am not sure I saw before now):
This was the result of doing `make -j12' with a large C++ program (I know, I wanted to see what would happen). This gradually slowed the machine to a crawl - for a while I could watch the /proc/latency_trace fiels and see the latency gradually climbing. Eventually tha machine ground almost to a halt, with lots of disk activity., and trying to switch from X to a console window resulted in a blank screen for 2-3 minutes. I did eventually get a text console and was able to kill all the cc1plus processes, and the machine went back to normal.
The link above is the highest latency recorded during this episode.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P4
Quoted 6 lines · Lee Revell
I think I am seeing those weird ~1 ms latencies still (actually I am not sure I saw before now):
this is a 9 msec latency in fact. I know what's going on - it's the get_swap_page() locking - i looked at it once already but there's no immediate silver arrow. As we allocate more and more swap entries the longer the scan gets.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P4
Quoted 7 lines · Ingo Molnar
Quoted 5 lines · Lee Revell
I think I am seeing those weird ~1 ms latencies still (actually I am not sure I saw before now):
Quoted 7 lines
Quoted 1 line
this is a 9 msec latency in fact. I know what's going on - it's the
get_swap_page()locking - i looked at it once already but there's no immediate silver arrow. As we allocate more and more swap entries the longer the scan gets.
This is an extreme load situation, so I don't think it will be a real-world problem. I have not seen it under any normal workload.
What about this one:
This appears during normal use.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P4
Quoted 2 lines · Lee Revell
This is an extreme load situation, so I don't think it will be a real-world problem. I have not seen it under any normal workload.
well, 9 msecs is still not nice. I've been able to trigger larger than 10msec latencies too on a 2 GHz box.
Quoted 5 lines
What about this one:
This appears during normal use.
hm, tcp_collapse() in net/ipv4/tcp_input.c. Could you try to just return from that function? Collapsing skbs of a given socket is not a necessary functionality (it is only a 'nice' thing to have in OOM situations) and it indeed can introduce quite high latencies.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P4
Quoted 8 lines · Ingo Molnar
Quoted 2 lines · Lee Revell
This is an extreme load situation, so I don't think it will be a real-world problem. I have not seen it under any normal workload.
well, 9 msecs is still not nice. I've been able to trigger larger than 10msec latencies too on a 2 GHz box.
If this is the case then should a make -j12 have ground the machine to a halt the way it did?
Quoted 5 lines
hm,
tcp_collapse()innet/ipv4/tcp_input.c. Could you try to just return from that function? Collapsing skbs of a given socket is not a necessary functionality (it is only a 'nice' thing to have in OOM situations) and it indeed can introduce quite high latencies.
OK, will test this.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P4
Quoted 5 lines · Lee Revell
Quoted 2 lines
well, 9 msecs is still not nice. I've been able to trigger larger than 10msec latencies too on a 2 GHz box.
If this is the case then should a make -j12 have ground the machine to a halt the way it did?
well, does it slow down that much with tracing and voluntary-preempt disabled too?
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P4
Quoted 11 lines · Ingo Molnar
Quoted 5 lines · Lee Revell
Quoted 2 lines
well, 9 msecs is still not nice. I've been able to trigger larger than 10msec latencies too on a 2 GHz box.
If this is the case then should a make -j12 have ground the machine to a halt the way it did?
well, does it slow down that much with tracing and voluntary-preempt disabled too?
Yup. turns out, this is enough to use all available RAM and swap space (1G total). I probably went OOM.
Good stress test though.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P4
In reply to Ingo MolnarQuoted 8 lines · Ingo Molnar
Quoted 2 lines · Lee Revell
This is an extreme load situation, so I don't think it will be a real-world problem. I have not seen it under any normal workload.
well, 9 msecs is still not nice. I've been able to trigger larger than 10msec latencies too on a 2 GHz box.
Here is another one:
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P4
In reply to Ingo MolnarQuoted 14 lines · Ingo Molnar
Quoted 2 lines · Lee Revell
This is an extreme load situation, so I don't think it will be a real-world problem. I have not seen it under any normal workload.
well, 9 msecs is still not nice. I've been able to trigger larger than 10msec latencies too on a 2 GHz box.
Quoted 5 lines
What about this one:
This appears during normal use.
Here is another:
Lee
PATCH 1/1 voluntary-preempt-2.6.8.1-P5
In reply to Ingo Molnari've uploaded the -P5 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P5
Changes since -P4:
- increase PERCPU_ENOUGH_ROOM to avoid percpu overflow on SMP. (Mark H Johnson)
- reduce ZAP_BLOCK_SIZE to 16 when vp != 0. This pushes the exit latency down to below 100 usecs on Lee Revell's box.
- added a preempt_count field to /proc/latency_trace. This makes it easier to spot IRQ contexts and generally it gives a nice overview of how the preemption depth changes. It should also help us debug those 900usec weirdnesses related to cpu_idle. (if they still occur)
- made the tcp packet-queue collapsing dependent on VOLUNTARY_PREEMPT.
- fixed 10-20 msec latencies triggered by 'netstat', which occur when there are lots of sockets on a box.
- rediffed against 2.6.8.1 for the patch to apply without fuzz
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P5
Quoted 4 lines · Ingo Molnar
i've uploaded the -P5 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P5
<snip>
I have been running the voluntary-preempt patches on one of my slower
(450) servers at home. The question is would latency traces from such a slow system be useful to you in this testing. About the most load that gets generated on this system, usually, is compiling the kernel or a very large app. that I do development on. What I tend to do to load the system is just run the stress-kernel suite and sometimes Andrew's amlat program to provide RT scheduling pressure.
If this would be useful, let me know as I have an interest in seeing latencies reduced as much as possible. I am building with the -P5 patch right now. If it would be more useful to try this on a faster system or to stess the system in a different manner, I could do that as well.
kr
Re: [patch] voluntary-preempt-2.6.8.1-P5
Quoted 15 lines · K.R. Foley
Quoted 4 lines · Ingo Molnar
i've uploaded the -P5 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P5
<snip>
I have been running the voluntary-preempt patches on one of my slower
(450) servers at home. The question is would latency traces from such a slow system be useful to you in this testing. About the most load that gets generated on this system, usually, is compiling the kernel or a very large app. that I do development on. What I tend to do to load the system is just run the stress-kernel suite and sometimes Andrew's amlat program to provide RT scheduling pressure.
Quoted 5 lines
If this would be useful, let me know as I have an interest in seeing latencies reduced as much as possible. I am building with the -P5 patch right now. If it would be more useful to try this on a faster system or to stess the system in a different manner, I could do that as well.
sure, all traces are interesting! On your system the latencies will be a bit larger than on others, but that's all - the maximum latency points should be the same (or similar) critical sections. So it's just as useful as the other traces.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P5
In reply to Ingo MolnarQuoted 4 lines · Ingo Molnar
i've uploaded the -P5 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P5
Here is a latency trace generated by the NFS compile in the stress-kernel. It is ~1.7 ms which doesn't really surprise me for the NFS compile. I am going to take the NFS compile out of the test now and see what else falls out.
If it would be useful for me to provide additional information with this, please let me know.
http://www.cybsft.com/testresults/2.6.8.1-P5/latency_trace1.txt
kr
Re: [patch] voluntary-preempt-2.6.8.1-P5
Quoted 4 lines · K.R. Foley
Here is a latency trace generated by the NFS compile in the stress-kernel. It is ~1.7 ms which doesn't really surprise me for the NFS compile. I am going to take the NFS compile out of the test now and see what else falls out.
NFS uses lock_kernel() for most of its locking. This means that the sharing and locking rules and assumptions are not clear (it's all within this opaque lock). I tried to figure out whether it would be safe to do a lock-break in nfs_cached_lookup() but it's quite hard ... Please bug the NFS-client maintainer(s) about this. To break this latency we'd have to do a lock-break in fs/nfs/dir.c:nfs_cached_lookup()'s main loop:
for(;(page = find_get_page(dir->i_mapping, desc.page_index)); desc.page_index++) {
res = -EIO;you could try this at the end of the loop:
if (res == 0)
goto out_found;
if (res != -EAGAIN)
break;
+ cond_resched();
}but i cannot guarantee that this is correct ... It might seem to work now but break later under load, etc. It's the NFS maintainer's call.
Quoted 2 lines
If it would be useful for me to provide additional information with this, please let me know.
reports like this are perfect for most latencies. I tried to include everything that is needed normally in /proc/latency_trace itself.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P5
In reply to K.R. FoleyQuoted 11 lines · K.R. Foley
Quoted 4 lines · Ingo Molnar
i've uploaded the -P5 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P5
Here is a latency trace generated by the NFS compile in the stress-kernel. It is ~1.7 ms which doesn't really surprise me for the NFS compile. I am going to take the NFS compile out of the test now and see what else falls out.
Quoted 5 lines
If it would be useful for me to provide additional information with this, please let me know.
http://www.cybsft.com/testresults/2.6.8.1-P5/latency_trace1.txt
Ugh, XDR rears its ugly head...
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P5
In reply to K.R. FoleyQuoted 11 lines · K. R. Foley
Quoted 4 lines · Ingo Molnar
i've uploaded the -P5 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P5
Here is a latency trace generated by the NFS compile in the stress-kernel. It is ~1.7 ms which doesn't really surprise me for the NFS compile. I am going to take the NFS compile out of the test now and see what else falls out.
Quoted 4 lines
If it would be useful for me to provide additional information with this, please let me know.
http://www.cybsft.com/testresults/2.6.8.1-P5/latency_trace1.txt
You don't need to do a NFS compilation to trigger it. just to a "find" on a huge NFS-mounted tree and you will see it.
Another candidate for cond_resched() is the loop in nfs_readdir(). It seems safe to do so be cause it calls readdir_search_pagecache() which would call schedule().
Regards
Signature · 1 line
Linh Dang
PATCH 1/1 voluntary-preempt-2.6.8.1-P6
In reply to Ingo Molnari've uploaded the -P6 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P6
i'm releasing another patch today because the mystic 900 usec latency seems to be nailed finally, with the help of Mark H Johnson's traces:
there were two places that set ->preempt_count back to 0 without the tracer noticing it: preempt_schedule() and entry.S's return path.
other changes since -P5:
- generic kernel fix: do not ignore idle=poll on non-P4 CPUs. (Mark H Johnson)
- fix the __trace link bug reported by Martin Rumori. All combinations of CONFIG_PREEMPT_TIMING & CONFIG_LATENCY_TRACE should work now.
- make kernel_preemption=1 the default. Most people use this anyway.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P6
Quoted 4 lines · Ingo Molnar
i've uploaded the -P6 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P6
Traces here (I have given up on giving them descriptive names). Max is about 120 usec:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P6
In reply to Ingo MolnarQuoted 4 lines · Ingo Molnar
i've uploaded the -P6 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P6
What is the number in the left column?
04000000 0.000ms (+0.000ms): preempt_schedule (unix_stream_sendmsg) 04000000 0.000ms (+0.000ms): schedule (preempt_schedule) 04000001 0.001ms (+0.000ms): sched_clock (schedule) 04000002 0.002ms (+0.001ms): dequeue_task (schedule)
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P6
In reply to Ingo MolnarQuoted 4 lines · Ingo Molnar
i've uploaded the -P6 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P6
Here's a 171 usec latency from ext3_free_blocks:
preemption latency trace v1.0.1
-------------------------------
latency: 171 us, entries: 2 (2)
-----------------
| task: evolution/863, uid:1000 nice:0 policy:0 rt_prio:0
----------------- => started at: ext3_free_blocks+0x1d0/0x4b0
=> ended at: ext3_free_blocks+0x229/0x4b0
=======>
00000001 0.000ms (+0.000ms): ext3_free_blocks (ext3_free_data) 00000001 0.167ms (+0.167ms): sub_preempt_count (ext3_free_blocks)
Also I have noticed a pattern with the XFree86 schedule() latencies, they all have a section like this:
04000002 0.003ms (+0.000ms): effective_prio (recalc_task_prio) 04000002 0.003ms (+0.000ms): enqueue_task (schedule) 00000002 0.006ms (+0.003ms): __switch_to (schedule) 00000002 0.088ms (+0.082ms): finish_task_switch (schedule) 00010002 0.090ms (+0.001ms): do_IRQ (finish_task_switch) 00010003 0.091ms (+0.000ms): mask_and_ack_8259A (do_IRQ)
04000002 0.002ms (+0.000ms): effective_prio (recalc_task_prio) 04000002 0.002ms (+0.000ms): enqueue_task (schedule) 00000002 0.005ms (+0.002ms): __switch_to (schedule) 00000002 0.067ms (+0.062ms): finish_task_switch (schedule) 00010002 0.068ms (+0.001ms): do_IRQ (finish_task_switch) 00010003 0.069ms (+0.000ms): mask_and_ack_8259A (do_IRQ)
I presume the 04000002 -> 00000002 is some interrupt being unmasked (or interrupts being globally enabled), then there's a 60-80 usec latency in schedule().
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P6
Quoted 9 lines · Lee Revell
Also I have noticed a pattern with the XFree86
schedule()latencies, they all have a section like this:04000002 0.003ms (+0.000ms):
effective_prio(recalc_task_prio) 04000002 0.003ms (+0.000ms):enqueue_task(schedule) 00000002 0.006ms (+0.003ms):__switch_to(schedule) 00000002 0.088ms (+0.082ms):finish_task_switch(schedule) 00010002 0.090ms (+0.001ms):do_IRQ(finish_task_switch)
Quoted 3 lines
I presume the 04000002 -> 00000002 is some interrupt being unmasked (or interrupts being globally enabled), then there's a 60-80 usec latency in
schedule().
0x04000000 is PREEMPT_ACTIVE - which is just a bit we set to make sure we dont try to preempt recursively.
but the XFree86 latency is interesting indeed. It could be the effect of the now-enlarged ioperm() bitmap! 80 usecs is excessive.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P6
Quoted 2 lines · Ingo Molnar
but the XFree86 latency is interesting indeed. It could be the effect of the now-enlarged
ioperm()bitmap! 80 usecs is excessive.
yeah, that's the reason. The quick fix is to reduce IO_BITMAP_BITS to 1024 in include/asm-i386/processor.h. I'm working on a real fix.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P6
In reply to Lee RevellQuoted 5 lines · Lee Revell
Here's a 171 usec latency from
ext3_free_blocks:preemption latency trace v1.0.1
-------------------------------
latency: 171 us, entries: 2 (2)
Quoted 8 lines
----------------- | task: evolution/863, uid:1000 nice:0 policy:0 rt_prio:0 -----------------=> started at:
ext3_free_blocks+0x1d0/0x4b0
=> ended at:ext3_free_blocks+0x229/0x4b0
=======>
00000001 0.000ms (+0.000ms):ext3_free_blocks(ext3_free_data) 00000001 0.167ms (+0.167ms):sub_preempt_count(ext3_free_blocks)
ok, i broke this lock in my tree, will show up in -P7.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P6
In reply to Ingo Molnarpreemption latency trace v1.0.1
-------------------------------
latency: 408 us, entries: 2273 (2273)
-----------------
| task: ksoftirqd/0/2, uid:0 nice:-10 policy:0 rt_prio:0
----------------- => started at: netif_receive_skb+0x74/0x1f0
=> ended at: netif_receive_skb+0x171/0x1f0
=======>
This one is hard to trim to post here, so full trace is posted on:
http://users.telenet.be/kvogel/network.txt
.config at:
http://users.telenet.be/kvogel/config.txt
PATCH 1/1 voluntary-preempt-2.6.8.1-P7
In reply to Ingo Molnari've uploaded the -P7 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P7
Changes since -P6:
- fixed the XFree86/X.org context-switch latency. (let me know if you see any weirdness like X not starting up while it did before.)
- halved the pagevec size, to reduce the radix gang-lookup costs.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P7
Quoted 11 lines · Ingo Molnar
i've uploaded the -P7 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P7
Changes since -P6:
- fixed the XFree86/X.org context-switch latency. (let me know if you see any weirdness like X not starting up while it did before.)
- halved the pagevec size, to reduce the radix gang-lookup costs.
Great, this is a significant improvement. Most of the worst case latencies (~150 usec) seem related to the TCP stack now, and a minor one (51 usec) in the ext3 journaling:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P7
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P7
Quoted 15 lines · Lee Revell
Quoted 11 lines · Ingo Molnar
i've uploaded the -P7 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P7
Changes since -P6:
- fixed the XFree86/X.org context-switch latency. (let me know if you see any weirdness like X not starting up while it did before.)
- halved the pagevec size, to reduce the radix gang-lookup costs.
Quoted 8 lines
Great, this is a significant improvement. Most of the worst case latencies (~150 usec) seem related to the TCP stack now, and a minor one (51 usec) in the ext3 journaling:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P7
Lee
I just posted a similar trace of ~4141 usec from P6 here:
http://www.cybsft.com/testresults/2.6.8.1-P6/latency-trace1.txt
This was part of a run from yesterday evening. After just rebooting, I too am seeing several of these. With the system not being hammered by the stress tests though, the max is only up to ~103 usec. I will be updating to P7 shortly.
kr
Re: [patch] voluntary-preempt-2.6.8.1-P7
Quoted 4 lines · K.R. Foley
I just posted a similar trace of ~4141 usec from P6 here:
http://www.cybsft.com/testresults/2.6.8.1-P6/latency-trace1.txt
This looks wrong:
00000003 0.008ms (+0.001ms): dummy_socket_sock_rcv_skb (tcp_v4_rcv) 00000004 0.008ms (+0.000ms): tcp_v4_do_rcv (tcp_v4_rcv) 00000004 0.009ms (+0.000ms): tcp_rcv_established (tcp_v4_do_rcv) 00010004 3.998ms (+3.989ms): do_IRQ (tcp_rcv_established) 00010005 3.999ms (+0.000ms): mask_and_ack_8259A (do_IRQ) 00010005 4.001ms (+0.002ms): generic_redirect_hardirq (do_IRQ) 00010004 4.002ms (+0.000ms): generic_handle_IRQ_event (do_IRQ)
Probably a false positive, Ingo would know better. What kind of stress testing were you doing?
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P7
Quoted 18 lines · Lee Revell
Quoted 4 lines · K.R. Foley
I just posted a similar trace of ~4141 usec from P6 here:
http://www.cybsft.com/testresults/2.6.8.1-P6/latency-trace1.txt
This looks wrong:
00000003 0.008ms (+0.001ms):
dummy_socket_sock_rcv_skb(tcp_v4_rcv) 00000004 0.008ms (+0.000ms):tcp_v4_do_rcv(tcp_v4_rcv) 00000004 0.009ms (+0.000ms):tcp_rcv_established(tcp_v4_do_rcv) 00010004 3.998ms (+3.989ms):do_IRQ(tcp_rcv_established) 00010005 3.999ms (+0.000ms):mask_and_ack_8259A(do_IRQ) 00010005 4.001ms (+0.002ms):generic_redirect_hardirq(do_IRQ) 00010004 4.002ms (+0.000ms):generic_handle_IRQ_event(do_IRQ)
Quoted 5 lines
Probably a false positive, Ingo would know better. What kind of stress testing were you doing?
Lee
This is while running the stress-kernel suite. I don't know about it being a false possitive, it very well may be. Looking back through the logs though I am not sure this is a valid latency anyway. This trace was from 06:37 this morning. About 19:54 last night I got an oops in kswapd and this morning around 10:30ish I had stuff getting killed right and left by oom. So, I am thinking that this probably is not a very reliable test.
kr
Re: [patch] voluntary-preempt-2.6.8.1-P7
In reply to Lee RevellQuoted 16 lines · Lee Revell
Quoted 4 lines · K.R. Foley
I just posted a similar trace of ~4141 usec from P6 here:
http://www.cybsft.com/testresults/2.6.8.1-P6/latency-trace1.txt
This looks wrong:
00000003 0.008ms (+0.001ms):
dummy_socket_sock_rcv_skb(tcp_v4_rcv) 00000004 0.008ms (+0.000ms):tcp_v4_do_rcv(tcp_v4_rcv) 00000004 0.009ms (+0.000ms):tcp_rcv_established(tcp_v4_do_rcv) 00010004 3.998ms (+3.989ms):do_IRQ(tcp_rcv_established) 00010005 3.999ms (+0.000ms):mask_and_ack_8259A(do_IRQ) 00010005 4.001ms (+0.002ms):generic_redirect_hardirq(do_IRQ) 00010004 4.002ms (+0.000ms):generic_handle_IRQ_event(do_IRQ)
Quoted 2 lines
Probably a false positive, Ingo would know better. What kind of stress testing were you doing?
indeed this looks suspect. Is this an SMP system?
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P7
Quoted 20 lines · Ingo Molnar
Quoted 17 lines · Lee Revell
Quoted 4 lines · K.R. Foley
I just posted a similar trace of ~4141 usec from P6 here:
http://www.cybsft.com/testresults/2.6.8.1-P6/latency-trace1.txt
This looks wrong:
00000003 0.008ms (+0.001ms):
dummy_socket_sock_rcv_skb(tcp_v4_rcv) 00000004 0.008ms (+0.000ms):tcp_v4_do_rcv(tcp_v4_rcv) 00000004 0.009ms (+0.000ms):tcp_rcv_established(tcp_v4_do_rcv) 00010004 3.998ms (+3.989ms):do_IRQ(tcp_rcv_established) 00010005 3.999ms (+0.000ms):mask_and_ack_8259A(do_IRQ) 00010005 4.001ms (+0.002ms):generic_redirect_hardirq(do_IRQ) 00010004 4.002ms (+0.000ms):generic_handle_IRQ_event(do_IRQ)
Quoted 8 lines
Quoted 2 lines
Probably a false positive, Ingo would know better. What kind of stress testing were you doing?
indeed this looks suspect. Is this an SMP system?
Ingo
Actually no. It is an SMP ready system, but with a single PII 450. As I responded to Lee's response, I am not sure that I completely trust the results of this trace anyway.
I would like to know why you guys think this may be a false positive. Is it just the extremely long latency? Or is there something else that makes it look suspect?
By the way I just posted two more traces, one that I caught last night and one from this morning:
This is another one similar to the last but much more reasonably latency:
http://www.cybsft.com/testresults/2.6.8.1-P7/2.6.8.1-P7-1.txt
And this one this morning appears to be from updatedb running, while the
tests were running. It's worth noting that this one appears to have
happened about the same time today that the other ~4100+ one happened yesterday. Also worth noting is that the system was probably swapping pretty good when this occurred.
http://www.cybsft.com/testresults/2.6.8.1-P7/2.6.8.1-P7-2.txt
kr
Re: [patch] voluntary-preempt-2.6.8.1-P7
In reply to Ingo MolnarQuoted 3 lines · Ingo Molnar
i've uploaded the -P7 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P7
Here's a pretty bad one, over 500 usecs in rt_garbage_collect:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P7#/var/www/2.6.8.1-P7/trace7.txt
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P7
Quoted 7 lines · Lee Revell
Quoted 3 lines · Ingo Molnar
i've uploaded the -P7 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P7
Here's a pretty bad one, over 500 usecs in
rt_garbage_collect:
Grr, apt-get upgrade (debian unstable) broke php on my web server. subsequent apt-get upgrades have not fixed it, and I have not had time to file a bug report. Traces can be found here:
http://krustophenia.net/2.6.8.1-P7
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P7
Quoted 3 lines · Lee Revell
to file a bug report. Traces can be found here:
re the skb latencies: Mark H Johnson managed to reduce net-input latencies by decreasing /proc/sys/net/core/netdev_max_backlog to 8 - could you try this, does it help? (you could try an even more extreme setting like 4.)
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P7
Quoted 11 lines · Ingo Molnar
Quoted 3 lines · Lee Revell
to file a bug report. Traces can be found here:
re the skb latencies: Mark H Johnson managed to reduce net-input latencies by decreasing
/proc/sys/net/core/netdev_max_backlogto 8 - could you try this, does it help? (you could try an even more extreme setting like 4.)
Spoke too soon with my last report. I am actually still seeing latencies close to 200 usecs in netdev_process_backlog. Reducing netdev_max_backlog to 4, then 2 did not seem to help.
netdev_process_backlog = 4:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P8#/var/www/2.6.8.1-P8/trace9.txt
netdev_process_backlog = 2:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P8#/var/www/2.6.8.1-P8/trace12.txt
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P7
In reply to Lee RevellQuoted 9 lines · Lee Revell
Quoted 3 lines · Ingo Molnar
i've uploaded the -P7 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P7
Here's a pretty bad one, over 500 usecs in
rt_garbage_collect:http://krustophenia.net/testresults.php?dataset=2.6.8.1-P7#/var/www/2.6.8.1-P7/trace7.txt
Thanks to everyone who pointed out that PHP had broken on my web server. It's fixed, all the links in these threads should work now.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P7
In reply to Ingo MolnarQuoted 10 lines · Ingo Molnar
i've uploaded the -P7 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P7
Changes since -P6:
- fixed the XFree86/X.org context-switch latency. (let me know if you see any weirdness like X not starting up while it did before.)
- halved the pagevec size, to reduce the radix gang-lookup costs.
It does not compile on x86-64. I get this:
CC arch/x86_64/ia32/syscall32.okernel/hardirq.c: In function `recalculate_desc_flags':kernel/hardirq.c:309: error: `SA_NODELAY' undeclared (first use in this function)kernel/hardirq.c:309: error: (Each undeclared identifier is reported only once kernel/hardirq.c:309: error: for each function it appears in.) kernel/hardirq.c: In function `generic_setup_irq':kernel/hardirq.c:339: error: `SA_NODELAY' undeclared (first use in this function)kernel/hardirq.c: In function `threaded_read_proc':kernel/hardirq.c:611: error: `SA_NODELAY' undeclared (first use in this function)kernel/hardirq.c: In function `threaded_write_proc':kernel/hardirq.c:629: error: `SA_NODELAY' undeclared (first use in this function)
make[1]: *** [kernel/hardirq.o] Error 1
make: *** [kernel] Error 2
The kernel config is attached.
Greetings,
Signature · 5 lines
Rafael J. Wysocki ---------------------------- For a successful technology, reality must take precedence over public relations, for nature cannot be fooled. -- Richard P. Feynman
#
# Automatically generated make config: don't edit
#CONFIG_X86_64=yCONFIG_64BIT=yCONFIG_X86=yCONFIG_MMU=yCONFIG_RWSEM_GENERIC_SPINLOCK=yCONFIG_X86_CMPXCHG=yCONFIG_EARLY_PRINTK=yCONFIG_HPET_TIMER=yCONFIG_HPET_EMULATE_RTC=yCONFIG_GENERIC_ISA_DMA=y
#
# Code maturity level options
#CONFIG_EXPERIMENTAL=yCONFIG_CLEAN_COMPILE=y
#
# General setup
#CONFIG_SWAP=yCONFIG_SYSVIPC=yCONFIG_POSIX_MQUEUE=yCONFIG_BSD_PROCESS_ACCT=yCONFIG_BSD_PROCESS_ACCT_V3=yCONFIG_SYSCTL=yCONFIG_AUDIT=yCONFIG_AUDITSYSCALL=yCONFIG_LOG_BUF_SHIFT=15CONFIG_HOTPLUG=y
# CONFIG_IKCONFIG is not set
# CONFIG_EMBEDDED is not setCONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not setCONFIG_FUTEX=yCONFIG_EPOLL=yCONFIG_IOSCHED_NOOP=yCONFIG_IOSCHED_AS=yCONFIG_IOSCHED_DEADLINE=yCONFIG_IOSCHED_CFQ=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
#
# Loadable module support
#CONFIG_MODULES=yCONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not setCONFIG_OBSOLETE_MODPARM=yCONFIG_MODVERSIONS=yCONFIG_KMOD=yCONFIG_STOP_MACHINE=y
#
# Processor type and features
#CONFIG_MK8=y
# CONFIG_MPSC is not set
# CONFIG_GENERIC_CPU is not setCONFIG_X86_L1_CACHE_BYTES=64CONFIG_X86_L1_CACHE_SHIFT=6CONFIG_X86_TSC=yCONFIG_X86_GOOD_APIC=y
# CONFIG_MICROCODE is not setCONFIG_X86_MSR=yCONFIG_X86_CPUID=yCONFIG_X86_IO_APIC=yCONFIG_X86_LOCAL_APIC=yCONFIG_MTRR=yCONFIG_SMP=y
# CONFIG_PREEMPT is not set
# CONFIG_SCHED_SMT is not setCONFIG_K8_NUMA=yCONFIG_DISCONTIGMEM=yCONFIG_NUMA=yCONFIG_HAVE_DEC_LOCK=yCONFIG_NR_CPUS=2CONFIG_GART_IOMMU=yCONFIG_SWIOTLB=yCONFIG_X86_MCE=y
#
# Power management options
#CONFIG_PM=y
# CONFIG_SOFTWARE_SUSPEND is not set
#
# ACPI (Advanced Configuration and Power Interface) Support #CONFIG_ACPI=yCONFIG_ACPI_BOOT=yCONFIG_ACPI_INTERPRETER=y
# CONFIG_ACPI_SLEEP is not setCONFIG_ACPI_AC=y
# CONFIG_ACPI_BATTERY is not set CONFIG_ACPI_BUTTON=yCONFIG_ACPI_FAN=yCONFIG_ACPI_PROCESSOR=yCONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_ASUS is not set
# CONFIG_ACPI_TOSHIBA is not set
# CONFIG_ACPI_DEBUG is not setCONFIG_ACPI_BUS=yCONFIG_ACPI_EC=yCONFIG_ACPI_POWER=yCONFIG_ACPI_PCI=yCONFIG_ACPI_SYSTEM=y
#
# CPU Frequency scaling
#CONFIG_CPU_FREQ=y
# CONFIG_CPU_FREQ_PROC_INTF is not set CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_GOV_POWERSAVE is not set # CONFIG_CPU_FREQ_GOV_USERSPACE is not set CONFIG_CPU_FREQ_TABLE=y
#
# CPUFreq processor drivers
#CONFIG_X86_POWERNOW_K8=m
#
# Bus options (PCI etc.)
#CONFIG_PCI=yCONFIG_PCI_DIRECT=yCONFIG_PCI_MMCONFIG=y
# CONFIG_PCI_MSI is not setCONFIG_PCI_LEGACY_PROC=yCONFIG_PCI_NAMES=y
#
# PCMCIA/CardBus support
#
# CONFIG_PCMCIA is not set
#
# PCI Hotplug Support
#
# CONFIG_HOTPLUG_PCI is not set
#
# Executable file formats / Emulations
#CONFIG_BINFMT_ELF=yCONFIG_BINFMT_MISC=mCONFIG_IA32_EMULATION=yCONFIG_IA32_AOUT=yCONFIG_COMPAT=yCONFIG_SYSVIPC_COMPAT=yCONFIG_UID16=y
#
# Device Drivers
#
#
# Generic Driver Options
#CONFIG_STANDALONE=yCONFIG_PREVENT_FIRMWARE_BUILD=y
# CONFIG_FW_LOADER is not set
# CONFIG_DEBUG_DRIVER is not set
#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set
#
# Parallel port support
#CONFIG_PARPORT=mCONFIG_PARPORT_PC=mCONFIG_PARPORT_PC_CML1=m
# CONFIG_PARPORT_SERIAL is not setCONFIG_PARPORT_PC_FIFO=yCONFIG_PARPORT_PC_SUPERIO=yCONFIG_PARPORT_OTHER=yCONFIG_PARPORT_1284=y
#
# Plug and Play support
#
#
# Block devices
#CONFIG_BLK_DEV_FD=m
# CONFIG_PARIDE is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not setCONFIG_BLK_DEV_LOOP=yCONFIG_BLK_DEV_CRYPTOLOOP=m
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_RAM is not setCONFIG_LBD=y
#
# ATA/ATAPI/MFM/RLL support
#CONFIG_IDE=yCONFIG_BLK_DEV_IDE=y
#
# Please see Documentation/ide.txt for help/info on IDE drives #
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set CONFIG_BLK_DEV_IDEDISK=yCONFIG_IDEDISK_MULTI_MODE=yCONFIG_BLK_DEV_IDECD=m
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set CONFIG_BLK_DEV_IDESCSI=m
# CONFIG_IDE_TASK_IOCTL is not set
# CONFIG_IDE_TASKFILE_IO is not set
#
# IDE chipset support/bugfixes
#CONFIG_IDE_GENERIC=y
# CONFIG_BLK_DEV_CMD640 is not setCONFIG_BLK_DEV_IDEPCI=yCONFIG_IDEPCI_SHARE_IRQ=yCONFIG_BLK_DEV_OFFBOARD=yCONFIG_BLK_DEV_GENERIC=y
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not setCONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not setCONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not setCONFIG_BLK_DEV_ADMA=y
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not setCONFIG_BLK_DEV_AMD74XX=y
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
# CONFIG_BLK_DEV_PIIX is not set
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
# CONFIG_BLK_DEV_VIA82CXXX is not set
# CONFIG_IDE_ARM is not setCONFIG_BLK_DEV_IDEDMA=yCONFIG_IDEDMA_IVB=yCONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set
#
# SCSI device support
#CONFIG_SCSI=yCONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not setCONFIG_BLK_DEV_SR=mCONFIG_BLK_DEV_SR_VENDOR=yCONFIG_CHR_DEV_SG=m
#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs #
# CONFIG_SCSI_MULTI_LUN is not set CONFIG_SCSI_CONSTANTS=yCONFIG_SCSI_LOGGING=y
#
# SCSI Transport Attributes
#CONFIG_SCSI_SPI_ATTRS=y
# CONFIG_SCSI_FC_ATTRS is not set
#
# SCSI low-level drivers
#CONFIG_BLK_DEV_3W_XXXX_RAID=y
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_MEGARAID is not setCONFIG_SCSI_SATA=y
# CONFIG_SCSI_SATA_SVW is not set
# CONFIG_SCSI_ATA_PIIX is not set
# CONFIG_SCSI_SATA_NV is not set
# CONFIG_SCSI_SATA_PROMISE is not set
# CONFIG_SCSI_SATA_SX4 is not setCONFIG_SCSI_SATA_SIL=y
# CONFIG_SCSI_SATA_SIS is not set
# CONFIG_SCSI_SATA_VIA is not set
# CONFIG_SCSI_SATA_VITESSE is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_EATA_PIO is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_PPA is not set
# CONFIG_SCSI_IMM is not setCONFIG_SCSI_SYM53C8XX_2=yCONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
# CONFIG_SCSI_SYM53C8XX_IOMAPPED is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_QLOGIC_ISP is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not setCONFIG_SCSI_QLA2XXX=y
# CONFIG_SCSI_QLA21XX is not set
# CONFIG_SCSI_QLA22XX is not set
# CONFIG_SCSI_QLA2300 is not set
# CONFIG_SCSI_QLA2322 is not set
# CONFIG_SCSI_QLA6312 is not set
# CONFIG_SCSI_QLA6322 is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not setCONFIG_SCSI_DEBUG=m
#
# Multi-device support (RAID and LVM)
#CONFIG_MD=yCONFIG_BLK_DEV_MD=y
# CONFIG_MD_LINEAR is not set
# CONFIG_MD_RAID0 is not setCONFIG_MD_RAID1=yCONFIG_MD_RAID5=y
# CONFIG_MD_RAID6 is not setCONFIG_MD_MULTIPATH=mCONFIG_BLK_DEV_DM=mCONFIG_DM_CRYPT=mCONFIG_DM_SNAPSHOT=mCONFIG_DM_MIRROR=mCONFIG_DM_ZERO=m
#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
#
# IEEE 1394 (FireWire) support
#CONFIG_IEEE1394=m
#
# Subsystem Options
#
# CONFIG_IEEE1394_VERBOSEDEBUG is not set
# CONFIG_IEEE1394_OUI_DB is not setCONFIG_IEEE1394_EXTRA_CONFIG_ROMS=yCONFIG_IEEE1394_CONFIG_ROM_IP1394=y
#
# Device Drivers
#
#
# Texas Instruments PCILynx requires I2C
#CONFIG_IEEE1394_OHCI1394=m
#
# Protocol Drivers
#CONFIG_IEEE1394_VIDEO1394=mCONFIG_IEEE1394_SBP2=m
# CONFIG_IEEE1394_SBP2_PHYS_DMA is not setCONFIG_IEEE1394_ETH1394=mCONFIG_IEEE1394_DV1394=mCONFIG_IEEE1394_RAWIO=mCONFIG_IEEE1394_CMP=mCONFIG_IEEE1394_AMDTP=m
#
# I2O device support
#
# CONFIG_I2O is not set
#
# Networking support
#CONFIG_NET=y
#
# Networking options
#CONFIG_PACKET=mCONFIG_PACKET_MMAP=y
# CONFIG_NETLINK_DEV is not setCONFIG_UNIX=yCONFIG_NET_KEY=yCONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_ARPD is not setCONFIG_SYN_COOKIES=yCONFIG_INET_AH=yCONFIG_INET_ESP=yCONFIG_INET_IPCOMP=y
#
# IP: Virtual Server Configuration
#
# CONFIG_IP_VS is not set
# CONFIG_IPV6 is not setCONFIG_NETFILTER=yCONFIG_NETFILTER_DEBUG=y
#
# IP: Netfilter Configuration
#CONFIG_IP_NF_CONNTRACK=yCONFIG_IP_NF_FTP=yCONFIG_IP_NF_IRC=mCONFIG_IP_NF_TFTP=mCONFIG_IP_NF_AMANDA=mCONFIG_IP_NF_QUEUE=mCONFIG_IP_NF_IPTABLES=mCONFIG_IP_NF_MATCH_LIMIT=mCONFIG_IP_NF_MATCH_IPRANGE=mCONFIG_IP_NF_MATCH_MAC=mCONFIG_IP_NF_MATCH_PKTTYPE=mCONFIG_IP_NF_MATCH_MARK=mCONFIG_IP_NF_MATCH_MULTIPORT=mCONFIG_IP_NF_MATCH_TOS=mCONFIG_IP_NF_MATCH_RECENT=mCONFIG_IP_NF_MATCH_ECN=mCONFIG_IP_NF_MATCH_DSCP=mCONFIG_IP_NF_MATCH_AH_ESP=mCONFIG_IP_NF_MATCH_LENGTH=mCONFIG_IP_NF_MATCH_TTL=mCONFIG_IP_NF_MATCH_TCPMSS=mCONFIG_IP_NF_MATCH_HELPER=mCONFIG_IP_NF_MATCH_STATE=mCONFIG_IP_NF_MATCH_CONNTRACK=mCONFIG_IP_NF_MATCH_OWNER=mCONFIG_IP_NF_FILTER=mCONFIG_IP_NF_TARGET_REJECT=mCONFIG_IP_NF_NAT=mCONFIG_IP_NF_NAT_NEEDED=yCONFIG_IP_NF_TARGET_MASQUERADE=mCONFIG_IP_NF_TARGET_REDIRECT=mCONFIG_IP_NF_TARGET_NETMAP=mCONFIG_IP_NF_TARGET_SAME=mCONFIG_IP_NF_NAT_LOCAL=yCONFIG_IP_NF_NAT_SNMP_BASIC=mCONFIG_IP_NF_NAT_IRC=mCONFIG_IP_NF_NAT_FTP=mCONFIG_IP_NF_NAT_TFTP=mCONFIG_IP_NF_NAT_AMANDA=mCONFIG_IP_NF_MANGLE=mCONFIG_IP_NF_TARGET_TOS=mCONFIG_IP_NF_TARGET_ECN=mCONFIG_IP_NF_TARGET_DSCP=mCONFIG_IP_NF_TARGET_MARK=mCONFIG_IP_NF_TARGET_CLASSIFY=mCONFIG_IP_NF_TARGET_LOG=mCONFIG_IP_NF_TARGET_ULOG=mCONFIG_IP_NF_TARGET_TCPMSS=mCONFIG_IP_NF_ARPTABLES=mCONFIG_IP_NF_ARPFILTER=mCONFIG_IP_NF_ARP_MANGLE=m
# CONFIG_IP_NF_RAW is not set
# CONFIG_IP_NF_MATCH_ADDRTYPE is not set
# CONFIG_IP_NF_MATCH_REALM is not setCONFIG_XFRM=yCONFIG_XFRM_USER=m
#
# SCTP Configuration (EXPERIMENTAL)
#CONFIG_IP_SCTP=mCONFIG_SCTP_DBG_MSG=yCONFIG_SCTP_DBG_OBJCNT=yCONFIG_SCTP_HMAC_NONE=y
# CONFIG_SCTP_HMAC_SHA1 is not set
# CONFIG_SCTP_HMAC_MD5 is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_HW_FLOWCONTROL is not set
#
# QoS and/or fair queueing
#
# CONFIG_NET_SCHED is not set
# CONFIG_NET_CLS_ROUTE is not set
#
# Network testing
#CONFIG_NET_PKTGEN=m
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not setCONFIG_NETDEVICES=y
# CONFIG_DUMMY is not set
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
#
# ARCnet devices
#
# CONFIG_ARCNET is not set
#
# Ethernet (10 or 100Mbit)
#CONFIG_NET_ETHERNET=yCONFIG_MII=m
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not setCONFIG_NET_VENDOR_3COM=yCONFIG_VORTEX=m
# CONFIG_TYPHOON is not set
#
# Tulip family network device support
#
# CONFIG_NET_TULIP is not set
# CONFIG_HP100 is not setCONFIG_NET_PCI=y
# CONFIG_PCNET32 is not setCONFIG_AMD8111_ETH=m
# CONFIG_AMD8111E_NAPI is not set
# CONFIG_ADAPTEC_STARFIRE is not setCONFIG_B44=m
# CONFIG_FORCEDETH is not set
# CONFIG_DGRS is not set
# CONFIG_EEPRO100 is not setCONFIG_E100=m
# CONFIG_E100_NAPI is not set
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
# CONFIG_NE2K_PCI is not set
# CONFIG_8139CP is not setCONFIG_8139TOO=m
# CONFIG_8139TOO_PIO is not set
# CONFIG_8139TOO_TUNE_TWISTER is not set
# CONFIG_8139TOO_8129 is not set
# CONFIG_8139_OLD_RX_RESET is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_VIA_RHINE is not set
# CONFIG_VIA_VELOCITY is not set
#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SK98LIN is not setCONFIG_TIGON3=m
#
# Ethernet (10000 Mbit)
#
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set
#
# Token Ring devices
#
# CONFIG_TR is not set
#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set
#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not setCONFIG_PPP=m
# CONFIG_PPP_MULTILINK is not setCONFIG_PPP_FILTER=yCONFIG_PPP_ASYNC=mCONFIG_PPP_SYNC_TTY=mCONFIG_PPP_DEFLATE=mCONFIG_PPP_BSDCOMP=mCONFIG_PPPOE=m
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not setCONFIG_SHAPER=m
# CONFIG_NETCONSOLE is not set
#
# ISDN subsystem
#
# CONFIG_ISDN is not set
#
# Telephony Support
#
# CONFIG_PHONE is not set
#
# Input device support
#CONFIG_INPUT=y
#
# Userland interfaces
#CONFIG_INPUT_MOUSEDEV=yCONFIG_INPUT_MOUSEDEV_PSAUX=yCONFIG_INPUT_MOUSEDEV_SCREEN_X=1280CONFIG_INPUT_MOUSEDEV_SCREEN_Y=1024CONFIG_INPUT_JOYDEV=m
# CONFIG_INPUT_TSDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set
#
# Input I/O drivers
#CONFIG_GAMEPORT=mCONFIG_SOUND_GAMEPORT=m
# CONFIG_GAMEPORT_NS558 is not set
# CONFIG_GAMEPORT_L4 is not set
# CONFIG_GAMEPORT_EMU10K1 is not set
# CONFIG_GAMEPORT_VORTEX is not set
# CONFIG_GAMEPORT_FM801 is not set
# CONFIG_GAMEPORT_CS461x is not setCONFIG_SERIO=yCONFIG_SERIO_I8042=yCONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not setCONFIG_SERIO_PCIPS2=m
#
# Input Device Drivers
#CONFIG_INPUT_KEYBOARD=yCONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not setCONFIG_INPUT_MOUSE=yCONFIG_MOUSE_PS2=y
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set
#
# Character devices
#CONFIG_VT=yCONFIG_VT_CONSOLE=yCONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set
#
# Serial drivers
#CONFIG_SERIAL_8250=yCONFIG_SERIAL_8250_CONSOLE=yCONFIG_SERIAL_8250_ACPI=yCONFIG_SERIAL_8250_NR_UARTS=4CONFIG_SERIAL_8250_EXTENDED=y
# CONFIG_SERIAL_8250_MANY_PORTS is not setCONFIG_SERIAL_8250_SHARE_IRQ=y
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
# CONFIG_SERIAL_8250_MULTIPORT is not set
# CONFIG_SERIAL_8250_RSA is not set
#
# Non-8250 serial port support
#CONFIG_SERIAL_CORE=yCONFIG_SERIAL_CORE_CONSOLE=yCONFIG_UNIX98_PTYS=yCONFIG_LEGACY_PTYS=yCONFIG_LEGACY_PTY_COUNT=256CONFIG_PRINTER=m
# CONFIG_LP_CONSOLE is not set
# CONFIG_PPDEV is not set
# CONFIG_TIPAR is not set
# CONFIG_QIC02_TAPE is not set
#
# IPMI
#CONFIG_IPMI_HANDLER=mCONFIG_IPMI_PANIC_EVENT=y
# CONFIG_IPMI_PANIC_STRING is not setCONFIG_IPMI_DEVICE_INTERFACE=m
# CONFIG_IPMI_SI is not setCONFIG_IPMI_WATCHDOG=m
#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not setCONFIG_HW_RANDOM=yCONFIG_NVRAM=mCONFIG_RTC=yCONFIG_DTLK=m
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
#
# Ftape, the floppy tape device driver
#CONFIG_AGP=yCONFIG_AGP_AMD64=y
# CONFIG_AGP_INTEL_MCH is not setCONFIG_DRM=y
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_GAMMA is not set
# CONFIG_DRM_R128 is not set
# CONFIG_DRM_RADEON is not set
# CONFIG_DRM_SIS is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HPET is not setCONFIG_HANGCHECK_TIMER=m
#
# I2C support
#
# CONFIG_I2C is not set
#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set
#
# Misc devices
#
# CONFIG_IBM_ASM is not set
#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set
#
# Graphics support
#CONFIG_FB=y
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not setCONFIG_FB_VESA=yCONFIG_VIDEO_SELECT=y
# CONFIG_FB_HGA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON_OLD is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_VIRTUAL is not set
#
# Console display driver support
#CONFIG_VGA_CONSOLE=y
# CONFIG_MDA_CONSOLE is not setCONFIG_DUMMY_CONSOLE=yCONFIG_FRAMEBUFFER_CONSOLE=yCONFIG_FONTS=y
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
# CONFIG_FONT_6x11 is not set
# CONFIG_FONT_PEARL_8x8 is not set
# CONFIG_FONT_ACORN_8x8 is not set
# CONFIG_FONT_MINI_4x6 is not set
# CONFIG_FONT_SUN8x16 is not set
# CONFIG_FONT_SUN12x22 is not set
#
# Logo configuration
#CONFIG_LOGO=y
# CONFIG_LOGO_LINUX_MONO is not set
# CONFIG_LOGO_LINUX_VGA16 is not setCONFIG_LOGO_LINUX_CLUT224=y
#
# Sound
#CONFIG_SOUND=m
#
# Advanced Linux Sound Architecture
#CONFIG_SND=mCONFIG_SND_TIMER=mCONFIG_SND_PCM=mCONFIG_SND_HWDEP=mCONFIG_SND_RAWMIDI=mCONFIG_SND_SEQUENCER=mCONFIG_SND_SEQ_DUMMY=mCONFIG_SND_OSSEMUL=yCONFIG_SND_MIXER_OSS=mCONFIG_SND_PCM_OSS=mCONFIG_SND_SEQUENCER_OSS=yCONFIG_SND_BIT32_EMUL=mCONFIG_SND_RTCTIMER=mCONFIG_SND_VERBOSE_PRINTK=yCONFIG_SND_DEBUG=yCONFIG_SND_DEBUG_MEMORY=yCONFIG_SND_DEBUG_DETECT=y
#
# Generic devices
#CONFIG_SND_MPU401_UART=mCONFIG_SND_DUMMY=mCONFIG_SND_VIRMIDI=mCONFIG_SND_MTPAV=mCONFIG_SND_SERIAL_U16550=mCONFIG_SND_MPU401=m
#
# PCI devices
#CONFIG_SND_AC97_CODEC=m
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_HDSP is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_YMFPCI is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_FM801 is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not setCONFIG_SND_INTEL8X0=m
# CONFIG_SND_INTEL8X0M is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VX222 is not set
#
# ALSA USB devices
#
# CONFIG_SND_USB_AUDIO is not set
#
# Open Sound System
#CONFIG_SOUND_PRIME=m
# CONFIG_SOUND_BT878 is not set
# CONFIG_SOUND_CMPCI is not set
# CONFIG_SOUND_EMU10K1 is not set
# CONFIG_SOUND_FUSION is not set
# CONFIG_SOUND_CS4281 is not set
# CONFIG_SOUND_ES1370 is not set
# CONFIG_SOUND_ES1371 is not set
# CONFIG_SOUND_ESSSOLO1 is not set
# CONFIG_SOUND_MAESTRO is not set
# CONFIG_SOUND_MAESTRO3 is not setCONFIG_SOUND_ICH=m
# CONFIG_SOUND_SONICVIBES is not set
# CONFIG_SOUND_TRIDENT is not set
# CONFIG_SOUND_MSNDCLAS is not set
# CONFIG_SOUND_MSNDPIN is not set
# CONFIG_SOUND_VIA82CXXX is not set
# CONFIG_SOUND_OSS is not set
# CONFIG_SOUND_ALI5455 is not set
# CONFIG_SOUND_FORTE is not set
# CONFIG_SOUND_RME96XX is not set
# CONFIG_SOUND_AD1980 is not set
#
# USB support
#CONFIG_USB=m
# CONFIG_USB_DEBUG is not set
#
# Miscellaneous USB options
#CONFIG_USB_DEVICEFS=yCONFIG_USB_BANDWIDTH=yCONFIG_USB_DYNAMIC_MINORS=y
#
# USB Host Controller Drivers
#CONFIG_USB_EHCI_HCD=m
# CONFIG_USB_EHCI_SPLIT_ISO is not set
# CONFIG_USB_EHCI_ROOT_HUB_TT is not setCONFIG_USB_OHCI_HCD=m
# CONFIG_USB_UHCI_HCD is not set
#
# USB Device Class drivers
#CONFIG_USB_AUDIO=m
# CONFIG_USB_BLUETOOTH_TTY is not set
# CONFIG_USB_MIDI is not set
# CONFIG_USB_ACM is not setCONFIG_USB_PRINTER=mCONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_RW_DETECT is not setCONFIG_USB_STORAGE_DATAFAB=yCONFIG_USB_STORAGE_FREECOM=yCONFIG_USB_STORAGE_ISD200=yCONFIG_USB_STORAGE_DPCM=y
CONFIG_USB_STORAGE_HP8200e=yCONFIG_USB_STORAGE_SDDR09=yCONFIG_USB_STORAGE_SDDR55=yCONFIG_USB_STORAGE_JUMPSHOT=y
#
# USB Human Interface Devices (HID)
#CONFIG_USB_HID=mCONFIG_USB_HIDINPUT=y
# CONFIG_HID_FF is not setCONFIG_USB_HIDDEV=y
#
# USB HID Boot Protocol drivers
#
# CONFIG_USB_KBD is not set
# CONFIG_USB_MOUSE is not set
# CONFIG_USB_AIPTEK is not set
# CONFIG_USB_WACOM is not set
# CONFIG_USB_KBTAB is not set
# CONFIG_USB_POWERMATE is not set
# CONFIG_USB_MTOUCH is not set
# CONFIG_USB_EGALAX is not set
# CONFIG_USB_XPAD is not set
# CONFIG_USB_ATI_REMOTE is not set
#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not setCONFIG_USB_HPUSBSCSI=m
#
# USB Multimedia devices
#
# CONFIG_USB_DABUSB is not set
#
# Video4Linux support is needed for USB Multimedia device support #
#
# USB Network adaptors
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET is not set
#
# USB port drivers
#
# CONFIG_USB_USS720 is not set
#
# USB Serial Converter support
#CONFIG_USB_SERIAL=mCONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_BELKIN is not set
# CONFIG_USB_SERIAL_DIGI_ACCELEPORT is not set
# CONFIG_USB_SERIAL_EMPEG is not set
# CONFIG_USB_SERIAL_FTDI_SIO is not set
# CONFIG_USB_SERIAL_VISOR is not set
# CONFIG_USB_SERIAL_IPAQ is not set
# CONFIG_USB_SERIAL_IR is not set
# CONFIG_USB_SERIAL_EDGEPORT is not set
# CONFIG_USB_SERIAL_EDGEPORT_TI is not set
# CONFIG_USB_SERIAL_KEYSPAN_PDA is not set
# CONFIG_USB_SERIAL_KEYSPAN is not set
# CONFIG_USB_SERIAL_KLSI is not set
# CONFIG_USB_SERIAL_KOBIL_SCT is not set
# CONFIG_USB_SERIAL_MCT_U232 is not setCONFIG_USB_SERIAL_PL2303=m
# CONFIG_USB_SERIAL_SAFE is not set
# CONFIG_USB_SERIAL_CYBERJACK is not set
# CONFIG_USB_SERIAL_XIRCOM is not set
# CONFIG_USB_SERIAL_OMNINET is not set
#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_TIGL is not set
# CONFIG_USB_AUERSWALD is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_PHIDGETSERVO is not setCONFIG_USB_TEST=m
#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set
#
# Firmware Drivers
#
# CONFIG_EDD is not set
#
# File systems
#CONFIG_EXT2_FS=yCONFIG_EXT2_FS_XATTR=yCONFIG_EXT2_FS_POSIX_ACL=yCONFIG_EXT2_FS_SECURITY=yCONFIG_EXT3_FS=yCONFIG_EXT3_FS_XATTR=yCONFIG_EXT3_FS_POSIX_ACL=yCONFIG_EXT3_FS_SECURITY=yCONFIG_JBD=y
# CONFIG_JBD_DEBUG is not setCONFIG_FS_MBCACHE=yCONFIG_REISERFS_FS=y
# CONFIG_REISERFS_CHECK is not setCONFIG_REISERFS_PROC_INFO=yCONFIG_REISERFS_FS_XATTR=yCONFIG_REISERFS_FS_POSIX_ACL=yCONFIG_REISERFS_FS_SECURITY=y
# CONFIG_JFS_FS is not setCONFIG_FS_POSIX_ACL=y
# CONFIG_XFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not setCONFIG_QUOTA=yCONFIG_QFMT_V1=yCONFIG_QFMT_V2=yCONFIG_QUOTACTL=yCONFIG_AUTOFS_FS=mCONFIG_AUTOFS4_FS=m
#
# CD-ROM/DVD Filesystems
#CONFIG_ISO9660_FS=mCONFIG_JOLIET=yCONFIG_ZISOFS=yCONFIG_ZISOFS_FS=mCONFIG_UDF_FS=mCONFIG_UDF_NLS=y
#
# DOS/FAT/NT Filesystems
#CONFIG_FAT_FS=mCONFIG_MSDOS_FS=mCONFIG_VFAT_FS=mCONFIG_FAT_DEFAULT_CODEPAGE=437CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_NTFS_FS is not set
#
# Pseudo filesystems
#CONFIG_PROC_FS=yCONFIG_PROC_KCORE=yCONFIG_SYSFS=y
# CONFIG_DEVFS_FS is not setCONFIG_DEVPTS_FS_XATTR=yCONFIG_DEVPTS_FS_SECURITY=yCONFIG_TMPFS=yCONFIG_HUGETLBFS=yCONFIG_HUGETLB_PAGE=yCONFIG_RAMFS=y
#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
#
# Network File Systems
#CONFIG_NFS_FS=mCONFIG_NFS_V3=yCONFIG_NFS_V4=yCONFIG_NFS_DIRECTIO=yCONFIG_NFSD=mCONFIG_NFSD_V3=yCONFIG_NFSD_V4=yCONFIG_NFSD_TCP=yCONFIG_LOCKD=mCONFIG_LOCKD_V4=yCONFIG_EXPORTFS=mCONFIG_SUNRPC=mCONFIG_SUNRPC_GSS=mCONFIG_RPCSEC_GSS_KRB5=mCONFIG_SMB_FS=mCONFIG_SMB_NLS_DEFAULT=yCONFIG_SMB_NLS_REMOTE="cp852"CONFIG_CIFS=mCONFIG_CIFS_STATS=y
# CONFIG_CIFS_XATTR is not setCONFIG_CIFS_POSIX=y
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not setCONFIG_MSDOS_PARTITION=y
#
# Native Language Support
#CONFIG_NLS=yCONFIG_NLS_DEFAULT="iso8859-2"CONFIG_NLS_CODEPAGE_437=m
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not setCONFIG_NLS_CODEPAGE_852=m
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not setCONFIG_NLS_CODEPAGE_1250=mCONFIG_NLS_CODEPAGE_1251=m
# CONFIG_NLS_ASCII is not setCONFIG_NLS_ISO8859_1=mCONFIG_NLS_ISO8859_2=y
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not setCONFIG_NLS_ISO8859_15=m
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not setCONFIG_NLS_UTF8=y
#
# Profiling support
#CONFIG_PROFILING=yCONFIG_OPROFILE=m
#
# Kernel hacking
#CONFIG_DEBUG_KERNEL=yCONFIG_DEBUG_SLAB=yCONFIG_MAGIC_SYSRQ=y
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_INIT_DEBUG is not set
# CONFIG_DEBUG_INFO is not set
# CONFIG_FRAME_POINTER is not set
# CONFIG_IOMMU_DEBUG is not set
#
# Security options
#CONFIG_SECURITY=yCONFIG_SECURITY_NETWORK=yCONFIG_SECURITY_CAPABILITIES=yCONFIG_SECURITY_ROOTPLUG=mCONFIG_SECURITY_SELINUX=yCONFIG_SECURITY_SELINUX_BOOTPARAM=y
# CONFIG_SECURITY_SELINUX_DISABLE is not setCONFIG_SECURITY_SELINUX_DEVELOP=yCONFIG_SECURITY_SELINUX_MLS=y
#
# Cryptographic options
#CONFIG_CRYPTO=yCONFIG_CRYPTO_HMAC=yCONFIG_CRYPTO_NULL=mCONFIG_CRYPTO_MD4=mCONFIG_CRYPTO_MD5=yCONFIG_CRYPTO_SHA1=yCONFIG_CRYPTO_SHA256=mCONFIG_CRYPTO_SHA512=mCONFIG_CRYPTO_DES=yCONFIG_CRYPTO_BLOWFISH=mCONFIG_CRYPTO_TWOFISH=mCONFIG_CRYPTO_SERPENT=mCONFIG_CRYPTO_AES=mCONFIG_CRYPTO_CAST5=mCONFIG_CRYPTO_CAST6=mCONFIG_CRYPTO_TEA=mCONFIG_CRYPTO_ARC4=mCONFIG_CRYPTO_KHAZAD=mCONFIG_CRYPTO_DEFLATE=yCONFIG_CRYPTO_MICHAEL_MIC=mCONFIG_CRYPTO_CRC32C=mCONFIG_CRYPTO_TEST=m
#
# Library routines
#CONFIG_CRC_CCITT=mCONFIG_CRC32=mCONFIG_LIBCRC32C=mCONFIG_ZLIB_INFLATE=yCONFIG_ZLIB_DEFLATE=y
Re: [patch] voluntary-preempt-2.6.8.1-P7
In reply to Ingo MolnarQuoted 4 lines · Ingo Molnar
i've uploaded the -P7 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P7
Is this one for real, or another false positive? A ~4119 usec latency in the scheduler?
http://www.cybsft.com/testresults/2.6.8.1-P7/2.6.8.1-P7-5.txt
kr
PATCH 1/1 voluntary-preempt-2.6.8.1-P8
In reply to Ingo Molnari've uploaded the -P8 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P8
Changes since -P8:
- fixes the DRI/DRM latency in radeon (and other drivers). The concept was investigated/tested by Dave Airlie.
- reduce netdev_max_backlog to 8 (Mark H Johnson)
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P8
Quoted 11 lines · Ingo Molnar
i've uploaded the -P8 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P8
Changes since -P8:
- fixes the DRI/DRM latency in radeon (and other drivers). The concept was investigated/tested by Dave Airlie.
- reduce
netdev_max_backlogto 8 (Mark H Johnson)
Should this fix the 500+ usec latency I saw in rt_garbage_collect? This one took a while to occur (overnight).
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P8
Quoted 7 lines · Lee Revell
Quoted 6 lines · Ingo Molnar
i've uploaded the -P8 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P8
Changes since -P8:
Quoted 8 lines
Quoted 5 lines
- fixes the DRI/DRM latency in radeon (and other drivers). The concept was investigated/tested by Dave Airlie.
- reduce
netdev_max_backlogto 8 (Mark H Johnson)Should this fix the 500+ usec latency I saw in
rt_garbage_collect? This one took a while to occur (overnight).
i dont think it will. Does the patch below help?
Ingo
ipv4/route.c renamed from ipv4/route.c.orig +1 -1
@@ -738,7 +738,7 @@ static int rt_garbage_collect(void)if (atomic_read (&ipv4_dst_ops.entries) < ip_rt_max_size) goto out; - }while (!in_softirq () &&time_before_eq (jiffies, now)); + }while (!in_softirq () &&time_before_eq (jiffies, now) && !need_resched ());if (atomic_read (&ipv4_dst_ops.entries) < ip_rt_max_size) goto out;
Re: [patch] voluntary-preempt-2.6.8.1-P8
Quoted 10 lines · Ingo Molnar
Quoted 6 lines · Lee Revell
Quoted 3 lines · Ingo Molnar
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P8
Should this fix the 500+ usec latency I saw in
rt_garbage_collect? This one took a while to occur (overnight).i dont think it will. Does the patch below help?
Probably. This one is pretty rare, it might take a day to recur, and it's pretty clear the patch will fix the problem, so I recommend adding it to P9.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P8
In reply to Ingo MolnarQuoted 12 lines · Ingo Molnar
Quoted 2 lines · Lee Revell
Should this fix the 500+ usec latency I saw in
rt_garbage_collect? This one took a while to occur (overnight).i dont think it will. Does the patch below help?
Ingo
ipv4/route.c static int rt_garbage_collect(void) · line 744
Quoted 8 lines
ipv4/route.c renamed from ipv4/route.c.orig +1
-1@@ -738,7 +738,7 @@ static int rt_garbage_collect(void)if (atomic_read (&ipv4_dst_ops.entries) < ip_rt_max_size) goto out; - }while (!in_softirq () &&time_before_eq (jiffies, now)); + }while (!in_softirq () &&time_before_eq (jiffies, now) && !need_resched ());if (atomic_read (&ipv4_dst_ops.entries) < ip_rt_max_size) goto out;
Nope, the above does not actually fix it. I got a 716 usec latency in rt_garbage_collect:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P9#/var/www/2.6.8.1-P9/trace12.txt
I believe this is associated with heavy route cache activity, I did not see this one again until I left a gnutella client running overnight.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P8
In reply to Ingo MolnarQuoted 10 lines · Ingo Molnar
i've uploaded the -P8 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P8
Changes since -P8:
- fixes the DRI/DRM latency in radeon (and other drivers). The concept was investigated/tested by Dave Airlie.
- reduce
netdev_max_backlogto 8 (Mark H Johnson)
With -P8 I have not seen a latency over 100 usecs yet. The most common latencies are those involving zap_pte_range, journal_get_undo_access and netdev_process_backlog (even with netdev_max_backlog set to 4).
Traces:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P8
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P8
In reply to Ingo MolnarQuoted 2 lines · Ingo Molnar
- reduce
netdev_max_backlogto 8 (Mark H Johnson)
On my system this setting has absolutely no effect on the skb related latencies. I tested setting netdev_max_backlog to every power of two between 1 and 128, and regardless of this setting, I can produce a 450-600 usec latency with:
ping -s 65507 -f $DEFAULT_GATEWAY
Looks like skb_checksum is the problem. Here is one of the traces:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P8#/var/www/2.6.8.1-P8/trace14.txt
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P8
Quoted 13 lines · Lee Revell
Quoted 2 lines · Ingo Molnar
- reduce
netdev_max_backlogto 8 (Mark H Johnson)On my system this setting has absolutely no effect on the skb related
latencies. I tested settingnetdev_max_backlogto every power of two between 1 and 128, and regardless of this setting, I can produce a 450-600 usec latency with:ping -s 65507 -f $DEFAULT_GATEWAY
Quoted 7 lines
Looks like
skb_checksumis the problem. Here is one of the traces:http://krustophenia.net/testresults.php?dataset=2.6.8.1-P8#/var/www/2.6.8.1-P8/trace14.txt
Lee
-
Setting netdev_max_backlog doesn't seem to have any effect for me either.
I get a similar latency when I try to reproduce your result above, ~612 usec. However, my trace is very different than yours:
http://www.cybsft.com/testresults/2.6.8.1-P8/latency_trace3.txt
In fact most any network activity, with the POSSIBLE exception of already open connections, seem to be able to trigger higher latencies. As you can see here:
http://www.cybsft.com/testresults/2.6.8.1-P8/
kr
Re: [patch] voluntary-preempt-2.6.8.1-P8
Quoted 21 lines · K.R. Foley
Quoted 20 lines · Lee Revell
Quoted 2 lines · Ingo Molnar
- reduce
netdev_max_backlogto 8 (Mark H Johnson)On my system this setting has absolutely no effect on the skb related latencies. I tested setting
netdev_max_backlogto every power of two between 1 and 128, and regardless of this setting, I can produce a 450-600 usec latency with:ping -s 65507 -f $DEFAULT_GATEWAY
Looks like
skb_checksumis the problem. Here is one of the traces:http://krustophenia.net/testresults.php?dataset=2.6.8.1-P8#/var/www/2.6.8.1-P8/trace14.txt
Lee
Quoted 9 lines
Quoted 1 line
-
Setting
netdev_max_backlogdoesn't seem to have any effect for me either.I get a similar latency when I try to reproduce your result above, ~612 usec. However, my trace is very different than yours:
http://www.cybsft.com/testresults/2.6.8.1-P8/latency_trace3.txt
I have some that are similar to this. Looks like many of the differences are due to different network drivers (e100 vs via-rhine).
Quoted 6 lines
In fact most any network activity, with the POSSIBLE exception of already open connections, seem to be able to trigger higher latencies. As you can see here:
Looks like except for the one above, all of these involve the extract_entropy issue. I am using a hack to disable the random driver to avoid these latencies.
There are a few other threads going on discussing a real solution to the extract_entropy issue. However, none of the solutions that has been posted yet fix the problem.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P8
In reply to K.R. FoleyQuoted 1 line · K.R. Foley
http://www.cybsft.com/testresults/2.6.8.1-P8/latency_trace3.txt
this trace shows a TX processing latency of 45 frames - please try 'ifconfig eth0 txqueuelen 8', does it help?
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P8
Quoted 11 lines · Ingo Molnar
Quoted 1 line · K.R. Foley
http://www.cybsft.com/testresults/2.6.8.1-P8/latency_trace3.txt
this trace shows a TX processing latency of 45 frames - please try 'ifconfig eth0 txqueuelen 8', does it help?
Ingo
There's no difference that jumps out at me. Here are a couple more with txqueuelen=8
http://www.cybsft.com/testresults/2.6.8.1-P8/latency_trace7.txt
http://www.cybsft.com/testresults/2.6.8.1-P8/latency_trace8.txt
kr
Re: [patch] voluntary-preempt-2.6.8.1-P8
In reply to Lee RevellQuoted 4 lines · Lee Revell
Quoted 1 line
- reduce
netdev_max_backlogto 8 (Mark H Johnson)On my system this setting has absolutely no effect on the skb related latencies. [...]
it has an effect on input queue length. Output queue lengths can be reduced via 'ifconfig eth0 txqueuelen 8'.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P8
Quoted 10 lines · Ingo Molnar
Quoted 4 lines · Lee Revell
Quoted 1 line
- reduce
netdev_max_backlogto 8 (Mark H Johnson)On my system this setting has absolutely no effect on the skb related latencies. [...]
it has an effect on input queue length. Output queue lengths can be reduced via 'ifconfig eth0 txqueuelen 8'.
OK. With both of these set to 4, the largest latency I was able to generate by ping flooding was 253 usecs.
Is there any particular reason one of these parameters is set via a sysctl/proc entry, and one by ifconfig?
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P8
Quoted 8 lines · Lee Revell
Quoted 2 lines
it has an effect on input queue length. Output queue lengths can be reduced via 'ifconfig eth0 txqueuelen 8'.
OK. With both of these set to 4, the largest latency I was able to generate by ping flooding was 253 usecs.
Is there any particular reason one of these parameters is set via a sysctl/proc entry, and one by ifconfig?
well, the input packet backlog is global, the output one is local. No good reason otherwise.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P8
In reply to Lee RevellQuoted 10 lines · Lee Revell
Quoted 8 lines · Ingo Molnar
Quoted 4 lines · Lee Revell
Quoted 1 line
- reduce
netdev_max_backlogto 8 (Mark H Johnson)On my system this setting has absolutely no effect on the skb related latencies. [...]
Quoted 8 lines
Quoted 3 lines
it has an effect on input queue length. Output queue lengths can be reduced via 'ifconfig eth0 txqueuelen 8'.
OK. With both of these set to 4, the largest latency I was able to generate by ping flooding was 253 usecs.
Even with both of these set to 4, I still get similar results ~647 usec :(
<snip>
Re: [patch] voluntary-preempt-2.6.8.1-P8
Quoted 16 lines · K.R. Foley
Quoted 15 lines · Lee Revell
Quoted 11 lines · Ingo Molnar
Quoted 4 lines · Lee Revell
Quoted 1 line
- reduce
netdev_max_backlogto 8 (Mark H Johnson)On my system this setting has absolutely no effect on the skb related latencies. [...]
it has an effect on input queue length. Output queue lengths can be reduced via 'ifconfig eth0 txqueuelen 8'.
Quoted 6 lines
Quoted 3 lines
OK. With both of these set to 4, the largest latency I was able to generate by ping flooding was 253 usecs.
Even with both of these set to 4, I still get similar results ~647 usec :(
I am able to generate a 1012 usec latency by flood pinging the broadcast address. These are pretty pathological cases, but if we are going for bounded latency it seems like they should be addressed.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P8
Lee Revell wrote:
<snip>
Quoted 7 lines
I am able to generate a 1012 usec latency by flood pinging the broadcast address. These are pretty pathological cases, but if we are going for bounded latency it seems like they should be addressed.
Lee
I agree, if that is possible. Whether it's possible in all cases is not something I am really qualified to answer.
kr
Re: [patch] voluntary-preempt-2.6.8.1-P7
In reply to Ingo MolnarQuoted 4 lines · Ingo Molnar
i've uploaded the -P7 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P7
Here are a couple more traces in the ~200 - ~300 usec range:
http://www.cybsft.com/testresults/2.6.8.1-P7/2.6.8.1-P7-4.txt
http://www.cybsft.com/testresults/2.6.8.1-P7/2.6.8.1-P7-10.txt
Also, can someone tell me why I often see traces in the 4 ms range with
3 - 4 ms being reported by a single routine?
kr
Re: [patch] voluntary-preempt-2.6.8.1-P6 -- False positive?!
In reply to Ingo MolnarI'm not sure if the following is a false positive or not:
preemption latency trace v1.0.1
-------------------------------
latency: 3022 us, entries: 15 (15)
-----------------
| task: swapper/0, uid:0 nice:0 policy:0 rt_prio:0
----------------- => started at: do_IRQ+0x27/0x230
=> ended at: do_IRQ+0x1a3/0x230
=======>
00010000 0.000ms (+0.000ms): do_IRQ (common_interrupt) 00010000 0.000ms (+0.000ms): do_IRQ (acpi_processor_idle) 00010001 0.000ms (+0.000ms): ack_edge_ioapic_irq (do_IRQ) 00010001 0.000ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 00010001 2.953ms (+2.952ms): wake_up_process (generic_redirect_hardirq) 00010001 2.953ms (+0.000ms): try_to_wake_up (wake_up_process) 00010001 2.953ms (+0.000ms): task_rq_lock (try_to_wake_up) 00010002 2.953ms (+0.000ms): activate_task (try_to_wake_up) 00010002 2.953ms (+0.000ms): sched_clock (activate_task) 00010002 2.953ms (+0.000ms): recalc_task_prio (activate_task) 00010002 2.953ms (+0.000ms): effective_prio (recalc_task_prio) 00010002 2.953ms (+0.000ms): enqueue_task (activate_task) 00010001 2.954ms (+0.000ms): preempt_schedule (try_to_wake_up) 00010000 2.954ms (+0.000ms): preempt_schedule (do_IRQ) 00000001 2.954ms (+0.000ms): sub_preempt_count (do_IRQ)
The following looks similar:
latency: 517 us, entries: 29 (29)
-----------------
| task: swapper/0, uid:0 nice:0 policy:0 rt_prio:0
----------------- => started at: do_IRQ+0x27/0x230
=> ended at: do_IRQ+0x1a3/0x230
=======>
00010000 0.000ms (+0.000ms): do_IRQ (common_interrupt) 00010000 0.000ms (+0.000ms): do_IRQ (acpi_processor_idle) 00010001 0.000ms (+0.000ms): ack_edge_ioapic_irq (do_IRQ) 00010001 0.000ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 00010000 0.000ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 00010000 0.000ms (+0.000ms): timer_interrupt (generic_handle_IRQ_event) 00010001 0.000ms (+0.000ms): mark_offset_tsc (timer_interrupt) 00010001 0.009ms (+0.008ms): do_timer (timer_interrupt) 00010001 0.503ms (+0.493ms): update_process_times (do_timer) 00010001 0.503ms (+0.000ms): update_one_process (update_process_times) 00010001 0.503ms (+0.000ms): run_local_timers (update_process_times) 00010001 0.503ms (+0.000ms): raise_softirq (update_process_times) 00010001 0.503ms (+0.000ms): scheduler_tick (update_process_times) 00010001 0.504ms (+0.000ms): sched_clock (scheduler_tick) 00010001 0.504ms (+0.000ms): update_wall_time (do_timer) 00010001 0.504ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 00010001 0.504ms (+0.000ms): generic_note_interrupt (do_IRQ) 00010001 0.504ms (+0.000ms): end_edge_ioapic_irq (do_IRQ) 00000001 0.504ms (+0.000ms): do_softirq (do_IRQ)
00000100 0.505ms (+0.000ms): __do_softirq (do_softirq) 00000100 0.505ms (+0.000ms): wake_up_process (do_softirq) 00000100 0.505ms (+0.000ms): try_to_wake_up (wake_up_process) 00000100 0.505ms (+0.000ms): task_rq_lock (try_to_wake_up) 00000101 0.505ms (+0.000ms): activate_task (try_to_wake_up) 00000101 0.505ms (+0.000ms): sched_clock (activate_task) 00000101 0.505ms (+0.000ms): recalc_task_prio (activate_task) 00000101 0.505ms (+0.000ms): effective_prio (recalc_task_prio) 00000101 0.505ms (+0.000ms): enqueue_task (activate_task) 00000001 0.506ms (+0.000ms): sub_preempt_count (do_IRQ)
Re: [patch] voluntary-preempt-2.6.8.1-P6 -- False positive?!
I think this may have been my own mistake. In between all the kernel compiles, I seem to have managed to do the following:
# CONFIG_X86_PM_TIMER is not set
Which was probably the cause. (was getting other side effects too, like keys stuck in repeat, 1-2 second freezes etc etc)
Re: [patch] voluntary-preempt-2.6.8.1-P5
In reply to Ingo MolnarQuoted 9 lines · Ingo Molnar
i've uploaded the -P5 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P5
- added a
preempt_countfield to/proc/latency_trace. This makes it easier to spot IRQ contexts and generally it gives a nice overview of how the preemption depth changes. It should also help us debug those 900usec weirdnesses related tocpu_idle. (if they still occur)
Ok, disregard my question about what this field was.
Quoted 2 lines
- made the tcp packet-queue collapsing dependent on
VOLUNTARY_PREEMPT.
I am seeing ~180usec latencies related to netif_receive_skb:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6#/var/www/2.6.8.1-P6/trace5.txt
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P5
In reply to Ingo MolnarQuoted 3 lines · Ingo Molnar
- reduce
ZAP_BLOCK_SIZEto 16 when vp != 0. This pushes the exit latency down to below 100 usecs on Lee Revell's box.
Almost:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6#/var/www/2.6.8.1-P6/trace8.txt
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P5
Quoted 9 lines · Lee Revell
Quoted 3 lines · Ingo Molnar
- reduce
ZAP_BLOCK_SIZEto 16 when vp != 0. This pushes the exit latency down to below 100 usecs on Lee Revell's box.Almost:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6#/var/www/2.6.8.1-P6/trace8.txt
i suspect if you turn off tracing (or disable it in the kernel completely) then you'd see below-100 usec latencies here. A trace entry costs ~0.3 usecs on your box, so this 50-entry trace has roughly 15 usecs of tracing overhead :-)
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P5
Quoted 8 lines · Ingo Molnar
Quoted 6 lines · Lee Revell
Quoted 3 lines · Ingo Molnar
- reduce
ZAP_BLOCK_SIZEto 16 when vp != 0. This pushes the exit latency down to below 100 usecs on Lee Revell's box.
Quoted 9 lines
Quoted 3 lines
Almost:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6#/var/www/2.6.8.1-P6/trace8.txt
i suspect if you turn off tracing (or disable it in the kernel completely) then you'd see below-100 usec latencies here. A trace entry costs ~0.3 usecs on your box, so this 50-entry trace has roughly 15 usecs of tracing overhead :-)
FWIW, I did see one of these over 200 usecs.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P5
Quoted 9 lines · Lee Revell
Quoted 7 lines
Quoted 1 line
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6#/var/www/2.6.8.1-P6/trace8.txt
i suspect if you turn off tracing (or disable it in the kernel completely) then you'd see below-100 usec latencies here. A trace entry costs ~0.3 usecs on your box, so this 50-entry trace has roughly 15 usecs of tracing overhead :-)
FWIW, I did see one of these over 200 usecs.
which trace is this?
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P5
Quoted 10 lines · Ingo Molnar
Quoted 8 lines · Lee Revell
Quoted 7 lines
Quoted 1 line
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6#/var/www/2.6.8.1-P6/trace8.txt
i suspect if you turn off tracing (or disable it in the kernel completely) then you'd see below-100 usec latencies here. A trace entry costs ~0.3 usecs on your box, so this 50-entry trace has roughly 15 usecs of tracing overhead :-)
Quoted 4 lines
Quoted 1 line
FWIW, I did see one of these over 200 usecs.
which trace is this?
I don't think I was able to save this one. I will post it if I see it again.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P5
In reply to Ingo MolnarQuoted 10 lines · Ingo Molnar
Quoted 8 lines · Lee Revell
Quoted 7 lines
Quoted 1 line
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6#/var/www/2.6.8.1-P6/trace8.txt
i suspect if you turn off tracing (or disable it in the kernel completely) then you'd see below-100 usec latencies here. A trace entry costs ~0.3 usecs on your box, so this 50-entry trace has roughly 15 usecs of tracing overhead :-)
Quoted 4 lines
Quoted 1 line
FWIW, I did see one of these over 200 usecs.
which trace is this?
Sorry, I was thinking of this one:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6#/var/www/2.6.8.1-P6/trace10.txt
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P5
Quoted 3 lines · Lee Revell
Sorry, I was thinking of this one:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6#/var/www/2.6.8.1-P6/trace10.txt
ok, reduced the zap block size to 8 pages a time. It should not be a big issue to have this at a low value because we dont retry anything and we do a full TLB flush only once. (subsequent flushes are caught by the tlb->need_flush optimization.)
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P5
Quoted 11 lines · Ingo Molnar
Quoted 3 lines · Lee Revell
Sorry, I was thinking of this one:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P6#/var/www/2.6.8.1-P6/trace10.txt
ok, reduced the zap block size to 8 pages a time. It should not be a big issue to have this at a low value because we dont retry anything and we do a full TLB flush only once. (subsequent flushes are caught by the
tlb->need_flushoptimization.)
This one is interesting. What is going on here?
preemption latency trace v1.0.1
-------------------------------
latency: 146 us, entries: 45 (45)
-----------------
| task: pdflush/33, uid:0 nice:0 policy:0 rt_prio:0
----------------- => started at: find_get_pages_tag+0x19/0x90
=> ended at: find_get_pages_tag+0x61/0x90
=======>
00000001 0.000ms (+0.000ms): find_get_pages_tag (pagevec_lookup_tag) 00000001 0.000ms (+0.000ms): radix_tree_gang_lookup_tag (find_get_pages_tag) 00000001 0.001ms (+0.000ms): __lookup_tag (radix_tree_gang_lookup_tag) 00000001 0.007ms (+0.005ms): __lookup_tag (radix_tree_gang_lookup_tag)
[ 20-30 more of these ]
00000001 0.119ms (+0.003ms): __lookup_tag (radix_tree_gang_lookup_tag) 00000001 0.125ms (+0.005ms): __lookup_tag (radix_tree_gang_lookup_tag) 00010001 0.132ms (+0.007ms): do_IRQ (find_get_pages_tag) 00010002 0.133ms (+0.000ms): mask_and_ack_8259A (do_IRQ) 00010002 0.137ms (+0.004ms): generic_redirect_hardirq (do_IRQ) 00010002 0.138ms (+0.000ms): wake_up_process (generic_redirect_hardirq) 00010002 0.138ms (+0.000ms): try_to_wake_up (wake_up_process) 00010002 0.138ms (+0.000ms): task_rq_lock (try_to_wake_up) 00010003 0.139ms (+0.000ms): activate_task (try_to_wake_up) 00010003 0.139ms (+0.000ms): sched_clock (activate_task) 00010003 0.140ms (+0.000ms): recalc_task_prio (activate_task) 00010003 0.141ms (+0.000ms): effective_prio (recalc_task_prio) 00010003 0.141ms (+0.000ms): enqueue_task (activate_task) 00010002 0.141ms (+0.000ms): preempt_schedule (try_to_wake_up) 00010001 0.142ms (+0.000ms): preempt_schedule (do_IRQ) 00000001 0.143ms (+0.001ms): sub_preempt_count (find_get_pages_tag)
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P5
Quoted 17 lines · Lee Revell
This one is interesting. What is going on here?
preemption latency trace v1.0.1
-------------------------------
latency: 146 us, entries: 45 (45)----------------- | task: pdflush/33, uid:0 nice:0 policy:0 rt_prio:0 -----------------=> started at:
find_get_pages_tag+0x19/0x90
=> ended at:find_get_pages_tag+0x61/0x90
=======>
00000001 0.000ms (+0.000ms):find_get_pages_tag(pagevec_lookup_tag) 00000001 0.000ms (+0.000ms):radix_tree_gang_lookup_tag(find_get_pages_tag) 00000001 0.001ms (+0.000ms):__lookup_tag(radix_tree_gang_lookup_tag) 00000001 0.007ms (+0.005ms):__lookup_tag(radix_tree_gang_lookup_tag)
Quoted 1 line
[ 20-30 more of these ]
this is the pagevec code, triggering the radix tree multi-entry-lookup code. Could you try to reduce PAGEVEC_SIZE from 16 to 8 (or 4) in include/linux/pagevec.h - does this reduce these particular latencies?
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P5
In reply to Ingo MolnarI'm regularly seeing acpi_ec_read latencies. (e.g. when closing the lid of my notebook, when the processor cooler activates, ...)
I'm guessing there isn't much that can be done about this, right?!
preemption latency trace v1.0.1
-------------------------------
latency: 1151 us, entries: 94 (94)
-----------------
| task: kacpid/6, uid:0 nice:-10 policy:0 rt_prio:0
----------------- => started at: acpi_ec_read+0x6b/0xf7
=> ended at: acpi_ec_read+0xc8/0xf7
=======>
00000001 0.000ms (+0.000ms): acpi_ec_read (acpi_ec_space_handler) 00000001 0.000ms (+0.000ms): acpi_hw_low_level_write (acpi_ec_read) 00000001 0.000ms (+0.000ms): acpi_os_write_port (acpi_hw_low_level_write) 00000001 0.001ms (+0.001ms): acpi_ec_wait (acpi_ec_read) 00000001 0.001ms (+0.000ms): acpi_hw_low_level_read (acpi_ec_wait) 00000001 0.001ms (+0.000ms): acpi_os_read_port (acpi_hw_low_level_read) 00000001 0.003ms (+0.001ms): __const_udelay (acpi_ec_wait) 00000001 0.003ms (+0.000ms): __delay (acpi_ec_wait)
00000001 0.003ms (+0.000ms): delay_tsc (__delay)
[...]
00000001 1.113ms (+0.001ms): smp_apic_timer_interrupt (acpi_ec_read) 00010001 1.113ms (+0.000ms): profile_hook (smp_apic_timer_interrupt) 00010002 1.113ms (+0.000ms): notifier_call_chain (profile_hook) 00000002 1.113ms (+0.000ms): do_softirq (smp_apic_timer_interrupt) 00000100 1.113ms (+0.000ms): __do_softirq (do_softirq) 00010001 1.113ms (+0.000ms): do_IRQ (acpi_ec_read)
00010002 1.113ms (+0.000ms): ack_edge_ioapic_irq (do_IRQ) 00010002 1.113ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 00010000 1.113ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 00010000 1.113ms (+0.000ms): timer_interrupt (generic_handle_IRQ_event) 00010001 1.113ms (+0.000ms): mark_offset_tsc (timer_interrupt) 00010001 1.123ms (+0.009ms): do_timer (timer_interrupt) 00010001 1.123ms (+0.000ms): update_process_times (do_timer) 00010001 1.123ms (+0.000ms): update_one_process (update_process_times) 00010001 1.123ms (+0.000ms): run_local_timers (update_process_times) 00010001 1.123ms (+0.000ms): raise_softirq (update_process_times) 00010001 1.123ms (+0.000ms): scheduler_tick (update_process_times) 00010001 1.123ms (+0.000ms): sched_clock (scheduler_tick) 00010002 1.123ms (+0.000ms): task_timeslice (scheduler_tick) 00010001 1.123ms (+0.000ms): update_wall_time (do_timer) 00010001 1.123ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 00010002 1.123ms (+0.000ms): generic_note_interrupt (do_IRQ) 00010002 1.123ms (+0.000ms): end_edge_ioapic_irq (do_IRQ) 00000002 1.123ms (+0.000ms): do_softirq (do_IRQ)
00000100 1.123ms (+0.000ms): __do_softirq (do_softirq) 00000001 1.124ms (+0.000ms): sub_preempt_count (acpi_ec_read)
Full trace output available on:
http://users.telenet.be/kvogel/ec-latency.txt
Culprit is the following code in drivers/acpi/ec.c function acpi_ec_wait (edited for mail):
----
#define ACPI_EC_UDELAY 100 /* Poll @ 100us increments */ #define ACPI_EC_UDELAY_COUNT 1000 /* Wait 10ms max. during EC ops */
u32 i = ACPI_EC_UDELAY_COUNT;
do {
acpi_hw_low_level_read(8, &acpi_ec_status, &ec->status_addr);
if (acpi_ec_status & ACPI_EC_FLAG_OBF)
return 0;
udelay(ACPI_EC_UDELAY);
} while (--i>0);----
NOTE: isn't the comment on the ACPI_EC_UDELAY_COUNT wrong?! Or is my arithmetic not what it used to be. (1000 loops of 100us = 100ms instead of 10ms, no?)
Re: [patch] voluntary-preempt-2.6.8.1-P2
In reply to Ingo MolnarQuoted 6 lines · Ingo Molnar
here's -P2:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P2
Changes since -P1:
Quoted 7 lines
- trace interrupted kernel code (via hardirqs, NMIs and pagefaults)
- yet another shot at trying to fix the IO-APIC/USB issues.
- mcount speedups - tracing should be faster
Ingo
I think I stumbled across some bugs/false positives.
Those tests were run with acpi=off, so that this specific issue doesn't interfere. voluntary_preemption is set to 3 throughout.
The first problem was already reported by Lee Revell. Creating a new tab in gnome-terminal gives :
[...]
0.064ms (+0.000ms): preempt_schedule (try_to_wake_up) 0.065ms (+0.000ms): preempt_schedule (copy_page_range) 0.065ms (+0.000ms): preempt_schedule (copy_page_range)
[... plenty of preempt_schedule (copy_page_range) skipped ...]
0.202ms (+0.000ms): preempt_schedule (copy_page_range) 0.202ms (+0.000ms): preempt_schedule (copy_page_range) 0.202ms (+0.000ms): check_preempt_timing (touch_preempt_timing)
This is induced by kernel_preemption=0 and disappears with kernel_preemption=1. It seems to be a side-effect of the current design. Is there a way to avoid this ?
The second one still involves creating a new tab in gnome-terminal, but with kernel_preemption=1 :
preemption latency trace v1.0
-----------------------------
latency: 130 us, entries: 6 (6)
process: gnome-terminal/14381, uid: 1000
nice: 0, policy: 0, rt_priority: 0
=======>
0.000ms (+0.000ms): __vma_link_rb (copy_mm)
0.000ms (+0.000ms): rb_insert_color (copy_mm)
0.000ms (+0.000ms): __rb_rotate_left (rb_insert_color) 0.000ms (+0.000ms): copy_page_range (copy_mm)
0.000ms (+0.000ms): pte_alloc_map (copy_page_range) 0.127ms (+0.126ms): check_preempt_timing (touch_preempt_timing)
When entering check_preempt_timing, preempt_thresh was 0, and preempt_max_latency had been freshly reset to 100. It should have triggered this code :
if (latency < preempt_max_latency) goto out;
but for some reason it didn't (or there is a problem in the tracing code, not showing events that would have increased 'latency').
Thomas
Re: [patch] voluntary-preempt-2.6.8.1-P2
Quoted 9 lines · Thomas Charbonnel
When entering
check_preempt_timing,preempt_threshwas 0, andpreempt_max_latencyhad been freshly reset to 100. It should have triggered this code :if (latency < preempt_max_latency) goto out;but for some reason it didn't (or there is a problem in the tracing code, not showing events that would have increased 'latency').
there is one case where we could 'miss' a new latency: when /proc/latency_trace is accessed. For the duration of /proc/latency_trace access, the updating of the max latency is stopped:
if (down_trylock(&max_mutex))
goto out;this is not really a practical problem and fixing it would be quite complex.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P2
In reply to Ingo MolnarQuoted 6 lines · Ingo Molnar
here's -P2:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P2
Changes since -P1:
Quoted 8 lines
- trace interrupted kernel code (via hardirqs, NMIs and pagefaults)
- yet another shot at trying to fix the IO-APIC/USB issues.
- mcount speedups - tracing should be faster
Ingo
-
The next problem I have relates to irq sharing.
On my laptop I can't avoid it :
10: 1070631 XT-PIC yenta, yenta, uhci_hcd, Intel
82801CA-ICH3, hdsp, eth0
If I set the sound card's interrupt to be non threaded, then I get a rather long non preemptible section :
http://www.undata.org/~thomas/irq_sharing.trace
As a side note, and this has already been reported here several times, the SA_INTERRUPT flag set notably by the sound card drivers handlers is not honored on current kernels if the device is not the first one to be registered. A simple fix would be to add SA_INTERRUPT handlers at the beginning instead of the end of the irq queue in setup_irq.
Similarly, when using SA_SAMPLE_RANDOM, all devices on the given irq contribute to the entropy, even those that have a predictable interrupt rate (e.g. sound cards), and/or for which the number of interrupts could outweight the number of interrupts of the original SA_SAMPLE_RANDOM driver.
Thomas
Re: [patch] voluntary-preempt-2.6.8.1-P2
At Wed, 18 Aug 2004 14:22:07 +0200,
Quoted 7 lines · Thomas Charbonnel
As a side note, and this has already been reported here several times, the
SA_INTERRUPTflag set notably by the sound card drivers handlers is not honored on current kernels if the device is not the first one to be registered. A simple fix would be to addSA_INTERRUPThandlers at the beginning instead of the end of the irq queue insetup_irq.
Quoted 5 lines
Similarly, when using
SA_SAMPLE_RANDOM, all devices on the given irq contribute to the entropy, even those that have a predictable interrupt rate (e.g. sound cards), and/or for which the number of interrupts could outweight the number of interrupts of the originalSA_SAMPLE_RANDOMdriver.
Hmm, something like this?
Takashi
arch/i386/kernel/irq.c +13 -6
@@ -219,15 +219,22 @@ inline void synchronize_irq(unsigned int asmlinkage int handle_IRQ_event(unsigned int irq,struct pt_regs *regs,struct irqaction *action) { -int status =1 ;/* Force the "do bottom halves" bit */ -int retval =0 ; - -if (!(action->flags &SA_INTERRUPT )) +int status; +int ret, retval =0 ; +struct irqaction *ac; + + status =0 ; +for (ac = action; ac; ac = ac->next) + status |= ac->flags; +if (!(status &SA_INTERRUPT ))local_irq_enable (); + status =0 ;do { - status |= action->flags; - retval |= action->handler (irq, action->dev_id, regs); + ret = action->handler (irq, action->dev_id, regs); +if (ret) + status |= action->flags; + retval |= ret; action = action->next; }while (action);if (status &SA_SAMPLE_RANDOM )
Re: [patch] voluntary-preempt-2.6.8.1-P2
Quoted 41 lines · Takashi Iwai
At Wed, 18 Aug 2004 14:22:07 +0200,
Quoted 7 lines · Thomas Charbonnel
As a side note, and this has already been reported here several times, the
SA_INTERRUPTflag set notably by the sound card drivers handlers is not honored on current kernels if the device is not the first one to be registered. A simple fix would be to addSA_INTERRUPThandlers at the beginning instead of the end of the irq queue insetup_irq.Quoted 5 lines
Similarly, when using
SA_SAMPLE_RANDOM, all devices on the given irq contribute to the entropy, even those that have a predictable interrupt rate (e.g. sound cards), and/or for which the number of interrupts could outweight the number of interrupts of the originalSA_SAMPLE_RANDOMdriver.Hmm, something like this?
Takashi
arch/i386/kernel/irq.c +9
-4@@ -219,15 +219,22 @@ inline void synchronize_irq(unsigned int asmlinkage int handle_IRQ_event(unsigned int irq,struct pt_regs *regs,struct irqaction *action) { -int status =1 ;/* Force the "do bottom halves" bit */ -int retval =0 ; - -if (!(action->flags &SA_INTERRUPT )) +int status; +int ret, retval =0 ; +struct irqaction *ac; + + status =0 ; +for (ac = action; ac; ac = ac->next) + status |= ac->flags; +if (!(status &SA_INTERRUPT ))local_irq_enable (); + status =0 ;
arch/i386/kernel/irq.c inline void synchronize_irq(unsigned int · line 240
Quoted 10 lines
arch/i386/kernel/irq.c +4
-2do { - status |= action->flags; - retval |= action->handler (irq, action->dev_id, regs); + ret = action->handler (irq, action->dev_id, regs); +if (ret) + status |= action->flags; + retval |= ret; action = action->next; }while (action);if (status &SA_SAMPLE_RANDOM )
I was thinking more of something like this :
irq.c 2004-08-18 17:23:18.011059064 +0200 renamed from irq.c.orig 2004-08-16 14:26:34.000000000 +0200 +10 -5
@@ -955,11 +955,16 @@ return -EBUSY; } - /* add new interrupt at end of irq queue */ - do { - p = &old->next; - old = *p; - } while (old); + if (new->flags & SA_INTERRUPT) + /* add interrupt at the start of the queue */ + new->next = old; + else + /* add new interrupt at end of irq queue */ + do { + p = &old->next; + old = *p; + } while (old); + shared = 1; }
Thomas
Re: [patch] voluntary-preempt-2.6.8.1-P2
At Wed, 18 Aug 2004 17:26:16 +0200,
Quoted 16 lines · Thomas Charbonnel
I was thinking more of something like this :
irq.c 2004-08-18 17:23:18.011059064 +0200 renamed from irq.c.orig 2004-08-16 14:26:34.000000000 +0200 +2
-5@@ -955,11 +955,16 @@ return -EBUSY; } - /* add new interrupt at end of irq queue */ - do { - p = &old->next; - old = *p; - } while (old); + if (new->flags & SA_INTERRUPT) + /* add interrupt at the start of the queue */
irq.c 2004-08-18 17:23:18.011059064 +0200 line 969
Quoted 10 lines
irq.c 2004-08-18 17:23:18.011059064 +0200 renamed from irq.c.orig 2004-08-16 14:26:34.000000000 +0200 +8
-0+ new->next = old; + else + /* add new interrupt at end of irq queue */ + do { + p = &old->next; + old = *p; + } while (old); + shared = 1; }
Ok, how about this one? Together with your patch, the irq is turned on for handlers without SA_INTERRUPT.
Takashi
arch/i386/kernel/irq.c +25 -12
@@ -219,15 +219,23 @@ inline void synchronize_irq(unsigned int asmlinkage int handle_IRQ_event(unsigned int irq,struct pt_regs *regs,struct irqaction *action) { -int status =1 ;/* Force the "do bottom halves" bit */ -int retval =0 ; - -if (!(action->flags &SA_INTERRUPT )) -local_irq_enable (); +int status; +int ret, retval =0 ; +int irq_off =1 ; + status =0 ;do { - status |= action->flags; - retval |= action->handler (irq, action->dev_id, regs); +/* Assume that all SA_INTERRUPT handlers are at the head +* of the irq queue +*/ +if (irq_off && ! (action->flags &SA_INTERRUPT )) { +local_irq_enable (); + irq_off =0 ; + } + ret = action->handler (irq, action->dev_id, regs); +if (ret) + status |= action->flags; + retval |= ret; action = action->next; }while (action);if (status &SA_SAMPLE_RANDOM )
@@ -955,11 +963,16 @@ int setup_irq(unsigned int irq, struct ireturn -EBUSY ; } -/* add new interrupt at end of irq queue */ -do { - p = &old->next; - old = *p; - }while (old); +if (new->flags &SA_INTERRUPT ) +/* add interrupt at the start of the queue */ + new->next = old; +else +/* add new interrupt at end of irq queue */ +do { + p = &old->next; + old = *p; + }while (old); + shared =1 ; }
Re: [patch] voluntary-preempt-2.6.8.1-P2
Takashi Iwai wrote :
[...]
Quoted 51 lines
Ok, how about this one? Together with your patch, the irq is turned on for handlers without
SA_INTERRUPT.Takashi
arch/i386/kernel/irq.c +18
-12@@ -219,15 +219,23 @@ inline void synchronize_irq(unsigned int asmlinkage int handle_IRQ_event(unsigned int irq,struct pt_regs *regs,struct irqaction *action) { -int status =1 ;/* Force the "do bottom halves" bit */ -int retval =0 ; - -if (!(action->flags &SA_INTERRUPT )) -local_irq_enable (); +int status; +int ret, retval =0 ; +int irq_off =1 ; + status =0 ;do { - status |= action->flags; - retval |= action->handler (irq, action->dev_id, regs); +/* Assume that all SA_INTERRUPT handlers are at the head +* of the irq queue +*/ +if (irq_off && ! (action->flags &SA_INTERRUPT )) { +local_irq_enable (); + irq_off =0 ; + } + ret = action->handler (irq, action->dev_id, regs); +if (ret) + status |= action->flags; + retval |= ret; action = action->next; }while (action);if (status &SA_SAMPLE_RANDOM )@@ -955,11 +963,16 @@ int setup_irq(unsigned int irq, struct ireturn -EBUSY ; } -/* add new interrupt at end of irq queue */ -do { - p = &old->next; - old = *p; - }while (old); +if (new->flags &SA_INTERRUPT ) +/* add interrupt at the start of the queue */ + new->next = old;
arch/i386/kernel/irq.c int setup_irq(unsigned int irq, struct i · line 978
Quoted 10 lines
arch/i386/kernel/irq.c +7
-0+else +/* add new interrupt at end of irq queue */ +do { + p = &old->next; + old = *p; + }while (old); + shared =1 ; }
You're right, of course, my mistake. It looks good now.
Thomas
Re: [patch] voluntary-preempt-2.6.8.1-P2
In reply to Thomas CharbonnelQuoted 7 lines · Thomas Charbonnel
The next problem I have relates to irq sharing.
On my laptop I can't avoid it :
10: 1070631 XT-PIC yenta, yenta,uhci_hcd, Intel
82801CA-ICH3, hdsp, eth0
If I set the sound card's interrupt to be non threaded, then I get a rather long non preemptible section :
http://www.undata.org/~thomas/irq_sharing.trace
i'm not sure the IRQ sharing problem can be solved.
we could execute certain handlers immediately, and defer others to an IRQ thread. But when we defer an IRQ we must keep the IRQ masked - which prevents further interrupts (possibly from a high-prio non-threaded handler) to be executed. So we'd see similar (or in fact worse, due to the redirection cost) latencies than with the current 'all or nothing' approach.
now in theory we only have to keep the IRQ line masked for level triggered interrupts (most APIC interrupts are level-triggered). Edge-triggered interrupts (such as the XT-PIC ones you have) could be acked immediately. I'll try to do something later, but right now there are still some IRQ problems (USB issues, PS2 mouse/keyboard issues) so i'd not like to complicate the design just yet.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P1
In reply to Ingo MolnarQuoted 16 lines · Ingo Molnar
Quoted 11 lines · Thomas Charbonnel
Tested P1. The biggest offender on my system is still reiserfs'
search_by_key. The trace is made of a very long loop of :
0.016ms (+0.000ms):reiserfs_in_journal(scan_bitmap_block) 0.016ms (+0.000ms):find_next_zero_bit(scan_bitmap_block) (...)
0.977ms (+0.000ms):reiserfs_in_journal(scan_bitmap_block) 0.977ms (+0.000ms):find_next_zero_bit(scan_bitmap_block) with interrupts showing up in the trace from time to time. Do you have plans to fix this, or should I switch to ext3 ?
Quoted 5 lines
i took a quick look and the reiserfs locking rules in that place do not seem to be easily fixable - this is the tree-lookup code which i suspect cannot be preempted. The reiser journalling code also makes use of the big kernel lock. I'd suggest reporting this to the reiserfs folks
the fix to reiserfs doing single threaded balancing/searching is called reiser4;-) It was not a trivial fix. It is however released.
Quoted 40 lines
and
(if it's not too much effort to migrate) use ext3 meanwhile.Quoted 11 lines
but the weird thing is that the latency sum goes way above those 100us, the culprit being
do_IRQ, regularly chewing up to 1ms !0.011ms (+0.000ms):
enable_8259A_irq(startup_8259A_irq) 0.954ms (+0.943ms):do_IRQ(common_interrupt)weird. This has the looks of a preempt-timing bug (we get a timer IRQ every 1 msec) - but there should be no preempt-timing when we are in the idle task (swapper).
Quoted 3 lines
i'll upload -P2 in a couple of minutes, it will trace the code that
do_IRQ()interrupted too - that would be quite useful in your case.Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [patch] voluntary-preempt-2.6.8.1-P1
Quoted 10 lines · Hans Reiser
Quoted 6 lines · Ingo Molnar
i took a quick look and the reiserfs locking rules in that place do not seem to be easily fixable - this is the tree-lookup code which i suspect cannot be preempted. The reiser journalling code also makes use of the big kernel lock. I'd suggest reporting this to the reiserfs folks
the fix to reiserfs doing single threaded balancing/searching is called reiser4;-) It was not a trivial fix. It is however released.
When will reiser4 will be in the stock kernel?
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P1
Quoted 9 lines · Lee Revell
When will reiser4 will be in the stock kernel?
Lee
I hope for 3-6 weeks from now, but it is not my decision.....
We are going into -mm this week.
Re: [patch] voluntary-preempt-2.6.8.1-P1
In reply to Ingo MolnarQuoted 11 lines · On Mon, 16 Aug 2004 06:05:15 +0200 Ingo Molnar <[email protected]> wrote:
Quoted 2 lines · Lee Revell
Anyway, the change to
sched.cfixes the mlockall bug, it works perfectly now. Thanks!great! This fix also means that we've got one more lock-break in the ext3 journalling code and one more lock-break in
dcache.c. I've released-P1 with the fix included:http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8.1-P1
Hi,
i don't know if this was mentioned before, but i sometimes see traces like this where half the entries are "preempt_schedule (copy_page_range)". I just wanted to ask if this is normal and expected behaviour.
preemption latency trace v1.0
-----------------------------
latency: 162 us, entries: 361 (361)
process: qjackctl/828, uid: 1000
nice: 0, policy: 0, rt_priority: 0
=======>
0.000ms (+0.000ms): __vma_link_rb (copy_mm)
0.000ms (+0.000ms): rb_insert_color (copy_mm)
0.000ms (+0.000ms): __rb_rotate_left (rb_insert_color) 0.000ms (+0.000ms): copy_page_range (copy_mm)
0.000ms (+0.000ms): pte_alloc_map (copy_page_range) 0.053ms (+0.053ms): do_IRQ (common_interrupt)
0.054ms (+0.000ms): mask_and_ack_8259A (do_IRQ) 0.056ms (+0.001ms): generic_redirect_hardirq (do_IRQ) 0.056ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 0.056ms (+0.000ms): timer_interrupt (generic_handle_IRQ_event) 0.057ms (+0.000ms): mark_offset_tsc (timer_interrupt) 0.064ms (+0.006ms): do_timer (timer_interrupt)
0.064ms (+0.000ms): update_process_times (do_timer) 0.064ms (+0.000ms): update_one_process (update_process_times) 0.064ms (+0.000ms): run_local_timers (update_process_times) 0.064ms (+0.000ms): raise_softirq (update_process_times) 0.065ms (+0.000ms): scheduler_tick (update_process_times) 0.065ms (+0.000ms): sched_clock (scheduler_tick) 0.066ms (+0.001ms): task_timeslice (scheduler_tick) 0.066ms (+0.000ms): update_wall_time (do_timer) 0.066ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 0.068ms (+0.001ms): generic_note_interrupt (do_IRQ) 0.068ms (+0.000ms): end_8259A_irq (do_IRQ)
0.069ms (+0.000ms): enable_8259A_irq (do_IRQ)
0.070ms (+0.000ms): do_softirq (do_IRQ)
0.070ms (+0.000ms): __do_softirq (do_softirq)
0.070ms (+0.000ms): wake_up_process (do_softirq) 0.070ms (+0.000ms): try_to_wake_up (wake_up_process) 0.070ms (+0.000ms): task_rq_lock (try_to_wake_up) 0.071ms (+0.000ms): activate_task (try_to_wake_up) 0.071ms (+0.000ms): sched_clock (activate_task) 0.071ms (+0.000ms): recalc_task_prio (activate_task) 0.071ms (+0.000ms): effective_prio (recalc_task_prio) 0.071ms (+0.000ms): enqueue_task (activate_task) 0.072ms (+0.000ms): preempt_schedule (try_to_wake_up) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.074ms (+0.000ms): preempt_schedule (copy_page_range) 0.074ms (+0.000ms): preempt_schedule (copy_page_range) 0.074ms (+0.000ms): preempt_schedule (copy_page_range) 0.074ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.086ms (+0.000ms): preempt_schedule (copy_page_range) 0.086ms (+0.000ms): preempt_schedule (copy_page_range) 0.086ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.092ms (+0.000ms): preempt_schedule (copy_page_range) 0.092ms (+0.000ms): preempt_schedule (copy_page_range) 0.092ms (+0.000ms): preempt_schedule (copy_page_range) 0.092ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.094ms (+0.000ms): preempt_schedule (copy_page_range) 0.094ms (+0.000ms): preempt_schedule (copy_page_range) 0.094ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.105ms (+0.000ms): preempt_schedule (copy_page_range) 0.105ms (+0.000ms): preempt_schedule (copy_page_range) 0.105ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.109ms (+0.000ms): preempt_schedule (copy_page_range) 0.109ms (+0.000ms): preempt_schedule (copy_page_range) 0.109ms (+0.000ms): preempt_schedule (copy_page_range) 0.109ms (+0.000ms): preempt_schedule (copy_page_range) 0.110ms (+0.000ms): preempt_schedule (copy_page_range) 0.110ms (+0.000ms): preempt_schedule (copy_page_range) 0.110ms (+0.000ms): preempt_schedule (copy_page_range) 0.110ms (+0.000ms): preempt_schedule (copy_page_range) 0.110ms (+0.000ms): preempt_schedule (copy_page_range) 0.111ms (+0.000ms): preempt_schedule (copy_page_range) 0.111ms (+0.000ms): preempt_schedule (copy_page_range) 0.111ms (+0.000ms): preempt_schedule (copy_page_range) 0.111ms (+0.000ms): preempt_schedule (copy_page_range) 0.112ms (+0.000ms): preempt_schedule (copy_page_range) 0.112ms (+0.000ms): preempt_schedule (copy_page_range) 0.112ms (+0.000ms): preempt_schedule (copy_page_range) 0.112ms (+0.000ms): preempt_schedule (copy_page_range) 0.113ms (+0.000ms): preempt_schedule (copy_page_range) 0.113ms (+0.000ms): preempt_schedule (copy_page_range) 0.113ms (+0.000ms): preempt_schedule (copy_page_range) 0.114ms (+0.000ms): preempt_schedule (copy_page_range) 0.114ms (+0.000ms): preempt_schedule (copy_page_range) 0.114ms (+0.000ms): preempt_schedule (copy_page_range) 0.114ms (+0.000ms): preempt_schedule (copy_page_range) 0.114ms (+0.000ms): preempt_schedule (copy_page_range) 0.115ms (+0.000ms): preempt_schedule (copy_page_range) 0.115ms (+0.000ms): preempt_schedule (copy_page_range) 0.115ms (+0.000ms): preempt_schedule (copy_page_range) 0.116ms (+0.000ms): preempt_schedule (copy_page_range) 0.116ms (+0.000ms): preempt_schedule (copy_page_range) 0.116ms (+0.000ms): preempt_schedule (copy_page_range) 0.117ms (+0.000ms): preempt_schedule (copy_page_range) 0.117ms (+0.000ms): preempt_schedule (copy_page_range) 0.117ms (+0.000ms): preempt_schedule (copy_page_range) 0.117ms (+0.000ms): preempt_schedule (copy_page_range) 0.117ms (+0.000ms): preempt_schedule (copy_page_range) 0.117ms (+0.000ms): preempt_schedule (copy_page_range) 0.118ms (+0.000ms): preempt_schedule (copy_page_range) 0.118ms (+0.000ms): preempt_schedule (copy_page_range) 0.118ms (+0.000ms): preempt_schedule (copy_page_range) 0.118ms (+0.000ms): preempt_schedule (copy_page_range) 0.119ms (+0.000ms): preempt_schedule (copy_page_range) 0.119ms (+0.000ms): preempt_schedule (copy_page_range) 0.119ms (+0.000ms): preempt_schedule (copy_page_range) 0.120ms (+0.000ms): preempt_schedule (copy_page_range) 0.120ms (+0.000ms): preempt_schedule (copy_page_range) 0.120ms (+0.000ms): preempt_schedule (copy_page_range) 0.120ms (+0.000ms): preempt_schedule (copy_page_range) 0.120ms (+0.000ms): preempt_schedule (copy_page_range) 0.121ms (+0.000ms): preempt_schedule (copy_page_range) 0.121ms (+0.000ms): preempt_schedule (copy_page_range) 0.121ms (+0.000ms): preempt_schedule (copy_page_range) 0.122ms (+0.000ms): preempt_schedule (copy_page_range) 0.122ms (+0.000ms): preempt_schedule (copy_page_range) 0.122ms (+0.000ms): preempt_schedule (copy_page_range) 0.123ms (+0.000ms): preempt_schedule (copy_page_range) 0.123ms (+0.000ms): preempt_schedule (copy_page_range) 0.123ms (+0.000ms): preempt_schedule (copy_page_range) 0.123ms (+0.000ms): preempt_schedule (copy_page_range) 0.124ms (+0.000ms): preempt_schedule (copy_page_range) 0.124ms (+0.000ms): preempt_schedule (copy_page_range) 0.124ms (+0.000ms): preempt_schedule (copy_page_range) 0.124ms (+0.000ms): preempt_schedule (copy_page_range) 0.125ms (+0.000ms): preempt_schedule (copy_page_range) 0.125ms (+0.000ms): preempt_schedule (copy_page_range) 0.125ms (+0.000ms): preempt_schedule (copy_page_range) 0.125ms (+0.000ms): preempt_schedule (copy_page_range) 0.126ms (+0.000ms): preempt_schedule (copy_page_range) 0.126ms (+0.000ms): preempt_schedule (copy_page_range) 0.126ms (+0.000ms): preempt_schedule (copy_page_range) 0.126ms (+0.000ms): preempt_schedule (copy_page_range) 0.126ms (+0.000ms): preempt_schedule (copy_page_range) 0.127ms (+0.000ms): preempt_schedule (copy_page_range) 0.127ms (+0.000ms): preempt_schedule (copy_page_range) 0.127ms (+0.000ms): preempt_schedule (copy_page_range) 0.127ms (+0.000ms): preempt_schedule (copy_page_range) 0.128ms (+0.000ms): preempt_schedule (copy_page_range) 0.128ms (+0.000ms): preempt_schedule (copy_page_range) 0.128ms (+0.000ms): preempt_schedule (copy_page_range) 0.128ms (+0.000ms): preempt_schedule (copy_page_range) 0.129ms (+0.000ms): preempt_schedule (copy_page_range) 0.129ms (+0.000ms): preempt_schedule (copy_page_range) 0.130ms (+0.000ms): preempt_schedule (copy_page_range) 0.130ms (+0.000ms): preempt_schedule (copy_page_range) 0.130ms (+0.000ms): preempt_schedule (copy_page_range) 0.130ms (+0.000ms): preempt_schedule (copy_page_range) 0.130ms (+0.000ms): preempt_schedule (copy_page_range) 0.131ms (+0.000ms): preempt_schedule (copy_page_range) 0.131ms (+0.000ms): preempt_schedule (copy_page_range) 0.131ms (+0.000ms): preempt_schedule (copy_page_range) 0.132ms (+0.000ms): preempt_schedule (copy_page_range) 0.132ms (+0.000ms): preempt_schedule (copy_page_range) 0.132ms (+0.000ms): preempt_schedule (copy_page_range) 0.133ms (+0.000ms): preempt_schedule (copy_page_range) 0.133ms (+0.000ms): preempt_schedule (copy_page_range) 0.133ms (+0.000ms): preempt_schedule (copy_page_range) 0.133ms (+0.000ms): preempt_schedule (copy_page_range) 0.134ms (+0.000ms): preempt_schedule (copy_page_range) 0.134ms (+0.000ms): preempt_schedule (copy_page_range) 0.134ms (+0.000ms): preempt_schedule (copy_page_range) 0.135ms (+0.000ms): preempt_schedule (copy_page_range) 0.135ms (+0.000ms): preempt_schedule (copy_page_range) 0.135ms (+0.000ms): preempt_schedule (copy_page_range) 0.136ms (+0.000ms): preempt_schedule (copy_page_range) 0.136ms (+0.000ms): preempt_schedule (copy_page_range) 0.136ms (+0.000ms): preempt_schedule (copy_page_range) 0.136ms (+0.000ms): preempt_schedule (copy_page_range) 0.137ms (+0.000ms): preempt_schedule (copy_page_range) 0.137ms (+0.000ms): preempt_schedule (copy_page_range) 0.137ms (+0.000ms): preempt_schedule (copy_page_range) 0.137ms (+0.000ms): preempt_schedule (copy_page_range) 0.137ms (+0.000ms): preempt_schedule (copy_page_range) 0.138ms (+0.000ms): preempt_schedule (copy_page_range) 0.138ms (+0.000ms): preempt_schedule (copy_page_range) 0.139ms (+0.000ms): preempt_schedule (copy_page_range) 0.139ms (+0.000ms): preempt_schedule (copy_page_range) 0.139ms (+0.000ms): preempt_schedule (copy_page_range) 0.139ms (+0.000ms): preempt_schedule (copy_page_range) 0.140ms (+0.000ms): preempt_schedule (copy_page_range) 0.140ms (+0.000ms): preempt_schedule (copy_page_range) 0.140ms (+0.000ms): preempt_schedule (copy_page_range) 0.141ms (+0.000ms): preempt_schedule (copy_page_range) 0.141ms (+0.000ms): preempt_schedule (copy_page_range) 0.141ms (+0.000ms): preempt_schedule (copy_page_range) 0.141ms (+0.000ms): preempt_schedule (copy_page_range) 0.142ms (+0.000ms): preempt_schedule (copy_page_range) 0.142ms (+0.000ms): preempt_schedule (copy_page_range) 0.143ms (+0.000ms): preempt_schedule (copy_page_range) 0.143ms (+0.000ms): preempt_schedule (copy_page_range) 0.143ms (+0.000ms): preempt_schedule (copy_page_range) 0.144ms (+0.000ms): preempt_schedule (copy_page_range) 0.144ms (+0.000ms): preempt_schedule (copy_page_range) 0.144ms (+0.000ms): preempt_schedule (copy_page_range) 0.144ms (+0.000ms): preempt_schedule (copy_page_range) 0.144ms (+0.000ms): preempt_schedule (copy_page_range) 0.145ms (+0.000ms): preempt_schedule (copy_page_range) 0.145ms (+0.000ms): preempt_schedule (copy_page_range) 0.145ms (+0.000ms): preempt_schedule (copy_page_range) 0.146ms (+0.000ms): preempt_schedule (copy_page_range) 0.146ms (+0.000ms): preempt_schedule (copy_page_range) 0.147ms (+0.000ms): preempt_schedule (copy_page_range) 0.147ms (+0.000ms): preempt_schedule (copy_page_range) 0.147ms (+0.000ms): preempt_schedule (copy_page_range) 0.147ms (+0.000ms): preempt_schedule (copy_page_range) 0.148ms (+0.000ms): preempt_schedule (copy_page_range) 0.148ms (+0.000ms): preempt_schedule (copy_page_range) 0.148ms (+0.000ms): preempt_schedule (copy_page_range) 0.148ms (+0.000ms): preempt_schedule (copy_page_range) 0.149ms (+0.000ms): preempt_schedule (copy_page_range) 0.149ms (+0.000ms): preempt_schedule (copy_page_range) 0.149ms (+0.000ms): preempt_schedule (copy_page_range) 0.149ms (+0.000ms): preempt_schedule (copy_page_range) 0.149ms (+0.000ms): preempt_schedule (copy_page_range) 0.150ms (+0.000ms): preempt_schedule (copy_page_range) 0.150ms (+0.000ms): preempt_schedule (copy_page_range) 0.150ms (+0.000ms): preempt_schedule (copy_page_range) 0.151ms (+0.000ms): preempt_schedule (copy_page_range) 0.151ms (+0.000ms): preempt_schedule (copy_page_range) 0.151ms (+0.000ms): preempt_schedule (copy_page_range) 0.152ms (+0.000ms): preempt_schedule (copy_page_range) 0.152ms (+0.000ms): preempt_schedule (copy_page_range) 0.152ms (+0.000ms): preempt_schedule (copy_page_range) 0.152ms (+0.000ms): preempt_schedule (copy_page_range) 0.153ms (+0.000ms): preempt_schedule (copy_page_range) 0.153ms (+0.000ms): preempt_schedule (copy_page_range) 0.153ms (+0.000ms): preempt_schedule (copy_page_range) 0.154ms (+0.000ms): preempt_schedule (copy_page_range) 0.154ms (+0.000ms): preempt_schedule (copy_page_range) 0.154ms (+0.000ms): preempt_schedule (copy_page_range) 0.154ms (+0.000ms): preempt_schedule (copy_page_range) 0.154ms (+0.000ms): preempt_schedule (copy_page_range) 0.155ms (+0.000ms): preempt_schedule (copy_page_range) 0.155ms (+0.000ms): preempt_schedule (copy_page_range) 0.155ms (+0.000ms): preempt_schedule (copy_page_range) 0.155ms (+0.000ms): preempt_schedule (copy_page_range) 0.156ms (+0.000ms): preempt_schedule (copy_page_range) 0.156ms (+0.000ms): preempt_schedule (copy_page_range) 0.156ms (+0.000ms): preempt_schedule (copy_page_range) 0.157ms (+0.000ms): preempt_schedule (copy_page_range) 0.157ms (+0.000ms): preempt_schedule (copy_page_range) 0.157ms (+0.000ms): preempt_schedule (copy_page_range) 0.157ms (+0.000ms): preempt_schedule (copy_page_range) 0.158ms (+0.000ms): preempt_schedule (copy_page_range) 0.158ms (+0.000ms): preempt_schedule (copy_page_range) 0.159ms (+0.000ms): check_preempt_timing (touch_preempt_timing)
preemption latency trace v1.0
-----------------------------
latency: 258 us, entries: 722 (722)
process: sylpheed-claws/796, uid: 1000
nice: 0, policy: 0, rt_priority: 0
=======>
0.000ms (+0.000ms): do_IRQ (common_interrupt)
0.000ms (+0.000ms): mask_and_ack_8259A (do_IRQ)
0.002ms (+0.001ms): generic_redirect_hardirq (do_IRQ) 0.002ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 0.002ms (+0.000ms): timer_interrupt (generic_handle_IRQ_event) 0.002ms (+0.000ms): mark_offset_tsc (timer_interrupt) 0.009ms (+0.007ms): do_timer (timer_interrupt)
0.010ms (+0.000ms): update_process_times (do_timer) 0.010ms (+0.000ms): update_one_process (update_process_times) 0.010ms (+0.000ms): run_local_timers (update_process_times) 0.011ms (+0.000ms): raise_softirq (update_process_times) 0.011ms (+0.000ms): scheduler_tick (update_process_times) 0.011ms (+0.000ms): sched_clock (scheduler_tick)
0.012ms (+0.000ms): task_timeslice (scheduler_tick) 0.013ms (+0.000ms): update_wall_time (do_timer)
0.013ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 0.014ms (+0.000ms): generic_note_interrupt (do_IRQ) 0.014ms (+0.000ms): end_8259A_irq (do_IRQ)
0.015ms (+0.000ms): enable_8259A_irq (do_IRQ)
0.015ms (+0.000ms): do_softirq (do_IRQ)
0.015ms (+0.000ms): __do_softirq (do_softirq)
0.016ms (+0.000ms): wake_up_process (do_softirq)
0.016ms (+0.000ms): try_to_wake_up (wake_up_process) 0.016ms (+0.000ms): task_rq_lock (try_to_wake_up) 0.016ms (+0.000ms): activate_task (try_to_wake_up) 0.016ms (+0.000ms): sched_clock (activate_task)
0.016ms (+0.000ms): recalc_task_prio (activate_task) 0.016ms (+0.000ms): effective_prio (recalc_task_prio) 0.017ms (+0.000ms): enqueue_task (activate_task)
0.017ms (+0.000ms): preempt_schedule (try_to_wake_up) 0.017ms (+0.000ms): preempt_schedule (copy_page_range) 0.018ms (+0.000ms): preempt_schedule (copy_page_range) 0.018ms (+0.000ms): preempt_schedule (copy_page_range) 0.018ms (+0.000ms): preempt_schedule (copy_page_range) 0.018ms (+0.000ms): preempt_schedule (copy_page_range) 0.019ms (+0.000ms): preempt_schedule (copy_page_range) 0.019ms (+0.000ms): preempt_schedule (copy_page_range) 0.019ms (+0.000ms): preempt_schedule (copy_page_range) 0.019ms (+0.000ms): preempt_schedule (copy_page_range) 0.020ms (+0.000ms): preempt_schedule (copy_page_range) 0.020ms (+0.000ms): preempt_schedule (copy_page_range) 0.020ms (+0.000ms): preempt_schedule (copy_page_range) 0.020ms (+0.000ms): preempt_schedule (copy_page_range) 0.020ms (+0.000ms): preempt_schedule (copy_page_range) 0.021ms (+0.000ms): preempt_schedule (copy_page_range) 0.021ms (+0.000ms): preempt_schedule (copy_page_range) 0.021ms (+0.000ms): preempt_schedule (copy_page_range) 0.021ms (+0.000ms): preempt_schedule (copy_page_range) 0.022ms (+0.000ms): preempt_schedule (copy_page_range) 0.022ms (+0.000ms): preempt_schedule (copy_page_range) 0.022ms (+0.000ms): preempt_schedule (copy_page_range) 0.022ms (+0.000ms): preempt_schedule (copy_page_range) 0.023ms (+0.000ms): preempt_schedule (copy_page_range) 0.023ms (+0.000ms): preempt_schedule (copy_page_range) 0.023ms (+0.000ms): preempt_schedule (copy_page_range) 0.023ms (+0.000ms): preempt_schedule (copy_page_range) 0.024ms (+0.000ms): preempt_schedule (copy_page_range) 0.024ms (+0.000ms): preempt_schedule (copy_page_range) 0.024ms (+0.000ms): preempt_schedule (copy_page_range) 0.024ms (+0.000ms): preempt_schedule (copy_page_range) 0.025ms (+0.000ms): preempt_schedule (copy_page_range) 0.025ms (+0.000ms): preempt_schedule (copy_page_range) 0.025ms (+0.000ms): preempt_schedule (copy_page_range) 0.025ms (+0.000ms): preempt_schedule (copy_page_range) 0.026ms (+0.000ms): preempt_schedule (copy_page_range) 0.026ms (+0.000ms): preempt_schedule (copy_page_range) 0.026ms (+0.000ms): preempt_schedule (copy_page_range) 0.026ms (+0.000ms): preempt_schedule (copy_page_range) 0.026ms (+0.000ms): preempt_schedule (copy_page_range) 0.027ms (+0.000ms): preempt_schedule (copy_page_range) 0.027ms (+0.000ms): preempt_schedule (copy_page_range) 0.027ms (+0.000ms): preempt_schedule (copy_page_range) 0.028ms (+0.000ms): preempt_schedule (copy_page_range) 0.028ms (+0.000ms): preempt_schedule (copy_page_range) 0.028ms (+0.000ms): preempt_schedule (copy_page_range) 0.028ms (+0.000ms): preempt_schedule (copy_page_range) 0.029ms (+0.000ms): preempt_schedule (copy_page_range) 0.029ms (+0.000ms): preempt_schedule (copy_page_range) 0.029ms (+0.000ms): preempt_schedule (copy_page_range) 0.029ms (+0.000ms): preempt_schedule (copy_page_range) 0.029ms (+0.000ms): preempt_schedule (copy_page_range) 0.030ms (+0.000ms): preempt_schedule (copy_page_range) 0.030ms (+0.000ms): preempt_schedule (copy_page_range) 0.030ms (+0.000ms): preempt_schedule (copy_page_range) 0.030ms (+0.000ms): preempt_schedule (copy_page_range) 0.031ms (+0.000ms): preempt_schedule (copy_page_range) 0.031ms (+0.000ms): preempt_schedule (copy_page_range) 0.031ms (+0.000ms): preempt_schedule (copy_page_range) 0.032ms (+0.000ms): preempt_schedule (copy_page_range) 0.032ms (+0.000ms): preempt_schedule (copy_page_range) 0.032ms (+0.000ms): preempt_schedule (copy_page_range) 0.032ms (+0.000ms): preempt_schedule (copy_page_range) 0.032ms (+0.000ms): preempt_schedule (copy_page_range) 0.033ms (+0.000ms): preempt_schedule (copy_page_range) 0.033ms (+0.000ms): preempt_schedule (copy_page_range) 0.033ms (+0.000ms): preempt_schedule (copy_page_range) 0.033ms (+0.000ms): preempt_schedule (copy_page_range) 0.034ms (+0.000ms): preempt_schedule (copy_page_range) 0.034ms (+0.000ms): preempt_schedule (copy_page_range) 0.034ms (+0.000ms): preempt_schedule (copy_page_range) 0.034ms (+0.000ms): preempt_schedule (copy_page_range) 0.034ms (+0.000ms): preempt_schedule (copy_page_range) 0.035ms (+0.000ms): preempt_schedule (copy_page_range) 0.035ms (+0.000ms): preempt_schedule (copy_page_range) 0.035ms (+0.000ms): preempt_schedule (copy_page_range) 0.036ms (+0.000ms): preempt_schedule (copy_page_range) 0.036ms (+0.000ms): preempt_schedule (copy_page_range) 0.036ms (+0.000ms): preempt_schedule (copy_page_range) 0.036ms (+0.000ms): preempt_schedule (copy_page_range) 0.036ms (+0.000ms): preempt_schedule (copy_page_range) 0.037ms (+0.000ms): preempt_schedule (copy_page_range) 0.037ms (+0.000ms): preempt_schedule (copy_page_range) 0.037ms (+0.000ms): preempt_schedule (copy_page_range) 0.037ms (+0.000ms): preempt_schedule (copy_page_range) 0.038ms (+0.000ms): preempt_schedule (copy_page_range) 0.038ms (+0.000ms): preempt_schedule (copy_page_range) 0.038ms (+0.000ms): preempt_schedule (copy_page_range) 0.038ms (+0.000ms): preempt_schedule (copy_page_range) 0.039ms (+0.000ms): preempt_schedule (copy_page_range) 0.039ms (+0.000ms): preempt_schedule (copy_page_range) 0.039ms (+0.000ms): preempt_schedule (copy_page_range) 0.040ms (+0.000ms): preempt_schedule (copy_page_range) 0.040ms (+0.000ms): preempt_schedule (copy_page_range) 0.040ms (+0.000ms): preempt_schedule (copy_page_range) 0.040ms (+0.000ms): preempt_schedule (copy_page_range) 0.041ms (+0.000ms): preempt_schedule (copy_page_range) 0.041ms (+0.000ms): preempt_schedule (copy_page_range) 0.041ms (+0.000ms): preempt_schedule (copy_page_range) 0.041ms (+0.000ms): preempt_schedule (copy_page_range) 0.041ms (+0.000ms): preempt_schedule (copy_page_range) 0.042ms (+0.000ms): preempt_schedule (copy_page_range) 0.042ms (+0.000ms): preempt_schedule (copy_page_range) 0.042ms (+0.000ms): preempt_schedule (copy_page_range) 0.042ms (+0.000ms): preempt_schedule (copy_page_range) 0.043ms (+0.000ms): preempt_schedule (copy_page_range) 0.043ms (+0.000ms): preempt_schedule (copy_page_range) 0.043ms (+0.000ms): preempt_schedule (copy_page_range) 0.044ms (+0.000ms): preempt_schedule (copy_page_range) 0.044ms (+0.000ms): preempt_schedule (copy_page_range) 0.044ms (+0.000ms): preempt_schedule (copy_page_range) 0.044ms (+0.000ms): preempt_schedule (copy_page_range) 0.044ms (+0.000ms): preempt_schedule (copy_page_range) 0.045ms (+0.000ms): preempt_schedule (copy_page_range) 0.045ms (+0.000ms): preempt_schedule (copy_page_range) 0.045ms (+0.000ms): preempt_schedule (copy_page_range) 0.045ms (+0.000ms): preempt_schedule (copy_page_range) 0.046ms (+0.000ms): preempt_schedule (copy_page_range) 0.046ms (+0.000ms): preempt_schedule (copy_page_range) 0.046ms (+0.000ms): preempt_schedule (copy_page_range) 0.046ms (+0.000ms): preempt_schedule (copy_page_range) 0.047ms (+0.000ms): preempt_schedule (copy_page_range) 0.047ms (+0.000ms): preempt_schedule (copy_page_range) 0.047ms (+0.000ms): preempt_schedule (copy_page_range) 0.047ms (+0.000ms): preempt_schedule (copy_page_range) 0.048ms (+0.000ms): preempt_schedule (copy_page_range) 0.048ms (+0.000ms): preempt_schedule (copy_page_range) 0.048ms (+0.000ms): preempt_schedule (copy_page_range) 0.048ms (+0.000ms): preempt_schedule (copy_page_range) 0.049ms (+0.000ms): preempt_schedule (copy_page_range) 0.049ms (+0.000ms): preempt_schedule (copy_page_range) 0.049ms (+0.000ms): preempt_schedule (copy_page_range) 0.049ms (+0.000ms): preempt_schedule (copy_page_range) 0.050ms (+0.000ms): preempt_schedule (copy_page_range) 0.050ms (+0.000ms): preempt_schedule (copy_page_range) 0.050ms (+0.000ms): preempt_schedule (copy_page_range) 0.050ms (+0.000ms): preempt_schedule (copy_page_range) 0.051ms (+0.000ms): preempt_schedule (copy_page_range) 0.051ms (+0.000ms): preempt_schedule (copy_page_range) 0.051ms (+0.000ms): preempt_schedule (copy_page_range) 0.051ms (+0.000ms): preempt_schedule (copy_page_range) 0.052ms (+0.000ms): preempt_schedule (copy_page_range) 0.052ms (+0.000ms): preempt_schedule (copy_page_range) 0.052ms (+0.000ms): preempt_schedule (copy_page_range) 0.052ms (+0.000ms): preempt_schedule (copy_page_range) 0.053ms (+0.000ms): preempt_schedule (copy_page_range) 0.053ms (+0.000ms): preempt_schedule (copy_page_range) 0.053ms (+0.000ms): preempt_schedule (copy_page_range) 0.053ms (+0.000ms): preempt_schedule (copy_page_range) 0.054ms (+0.000ms): preempt_schedule (copy_page_range) 0.054ms (+0.000ms): preempt_schedule (copy_page_range) 0.054ms (+0.000ms): preempt_schedule (copy_page_range) 0.054ms (+0.000ms): preempt_schedule (copy_page_range) 0.055ms (+0.000ms): preempt_schedule (copy_page_range) 0.055ms (+0.000ms): preempt_schedule (copy_page_range) 0.055ms (+0.000ms): preempt_schedule (copy_page_range) 0.055ms (+0.000ms): preempt_schedule (copy_page_range) 0.056ms (+0.000ms): preempt_schedule (copy_page_range) 0.056ms (+0.000ms): preempt_schedule (copy_page_range) 0.056ms (+0.000ms): preempt_schedule (copy_page_range) 0.056ms (+0.000ms): preempt_schedule (copy_page_range) 0.056ms (+0.000ms): preempt_schedule (copy_page_range) 0.057ms (+0.000ms): preempt_schedule (copy_page_range) 0.057ms (+0.000ms): preempt_schedule (copy_page_range) 0.057ms (+0.000ms): preempt_schedule (copy_page_range) 0.057ms (+0.000ms): preempt_schedule (copy_page_range) 0.058ms (+0.000ms): preempt_schedule (copy_page_range) 0.058ms (+0.000ms): preempt_schedule (copy_page_range) 0.058ms (+0.000ms): preempt_schedule (copy_page_range) 0.058ms (+0.000ms): preempt_schedule (copy_page_range) 0.059ms (+0.000ms): preempt_schedule (copy_page_range) 0.059ms (+0.000ms): preempt_schedule (copy_page_range) 0.059ms (+0.000ms): preempt_schedule (copy_page_range) 0.059ms (+0.000ms): preempt_schedule (copy_page_range) 0.060ms (+0.000ms): preempt_schedule (copy_page_range) 0.060ms (+0.000ms): preempt_schedule (copy_page_range) 0.060ms (+0.000ms): preempt_schedule (copy_page_range) 0.060ms (+0.000ms): preempt_schedule (copy_page_range) 0.061ms (+0.000ms): preempt_schedule (copy_page_range) 0.061ms (+0.000ms): preempt_schedule (copy_page_range) 0.061ms (+0.000ms): preempt_schedule (copy_page_range) 0.061ms (+0.000ms): preempt_schedule (copy_page_range) 0.062ms (+0.000ms): preempt_schedule (copy_page_range) 0.062ms (+0.000ms): preempt_schedule (copy_page_range) 0.062ms (+0.000ms): preempt_schedule (copy_page_range) 0.062ms (+0.000ms): preempt_schedule (copy_page_range) 0.063ms (+0.000ms): preempt_schedule (copy_page_range) 0.063ms (+0.000ms): preempt_schedule (copy_page_range) 0.063ms (+0.000ms): preempt_schedule (copy_page_range) 0.063ms (+0.000ms): preempt_schedule (copy_page_range) 0.064ms (+0.000ms): preempt_schedule (copy_page_range) 0.064ms (+0.000ms): preempt_schedule (copy_page_range) 0.064ms (+0.000ms): preempt_schedule (copy_page_range) 0.064ms (+0.000ms): preempt_schedule (copy_page_range) 0.065ms (+0.000ms): preempt_schedule (copy_page_range) 0.065ms (+0.000ms): preempt_schedule (copy_page_range) 0.065ms (+0.000ms): preempt_schedule (copy_page_range) 0.065ms (+0.000ms): preempt_schedule (copy_page_range) 0.066ms (+0.000ms): preempt_schedule (copy_page_range) 0.066ms (+0.000ms): preempt_schedule (copy_page_range) 0.066ms (+0.000ms): preempt_schedule (copy_page_range) 0.067ms (+0.000ms): preempt_schedule (copy_page_range) 0.067ms (+0.000ms): preempt_schedule (copy_page_range) 0.067ms (+0.000ms): preempt_schedule (copy_page_range) 0.067ms (+0.000ms): preempt_schedule (copy_page_range) 0.067ms (+0.000ms): preempt_schedule (copy_page_range) 0.068ms (+0.000ms): preempt_schedule (copy_page_range) 0.068ms (+0.000ms): preempt_schedule (copy_page_range) 0.068ms (+0.000ms): preempt_schedule (copy_page_range) 0.068ms (+0.000ms): preempt_schedule (copy_page_range) 0.069ms (+0.000ms): preempt_schedule (copy_page_range) 0.069ms (+0.000ms): preempt_schedule (copy_page_range) 0.069ms (+0.000ms): preempt_schedule (copy_page_range) 0.069ms (+0.000ms): preempt_schedule (copy_page_range) 0.070ms (+0.000ms): preempt_schedule (copy_page_range) 0.070ms (+0.000ms): preempt_schedule (copy_page_range) 0.070ms (+0.000ms): preempt_schedule (copy_page_range) 0.070ms (+0.000ms): preempt_schedule (copy_page_range) 0.071ms (+0.000ms): preempt_schedule (copy_page_range) 0.071ms (+0.000ms): preempt_schedule (copy_page_range) 0.071ms (+0.000ms): preempt_schedule (copy_page_range) 0.071ms (+0.000ms): preempt_schedule (copy_page_range) 0.072ms (+0.000ms): preempt_schedule (copy_page_range) 0.072ms (+0.000ms): preempt_schedule (copy_page_range) 0.072ms (+0.000ms): preempt_schedule (copy_page_range) 0.072ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.074ms (+0.000ms): preempt_schedule (copy_page_range) 0.074ms (+0.000ms): preempt_schedule (copy_page_range) 0.074ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.086ms (+0.000ms): preempt_schedule (copy_page_range) 0.086ms (+0.000ms): preempt_schedule (copy_page_range) 0.086ms (+0.000ms): preempt_schedule (copy_page_range) 0.086ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.092ms (+0.000ms): preempt_schedule (copy_page_range) 0.092ms (+0.000ms): preempt_schedule (copy_page_range) 0.092ms (+0.000ms): preempt_schedule (copy_page_range) 0.092ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.094ms (+0.000ms): preempt_schedule (copy_page_range) 0.094ms (+0.000ms): preempt_schedule (copy_page_range) 0.094ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.105ms (+0.000ms): preempt_schedule (copy_page_range) 0.105ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.109ms (+0.000ms): preempt_schedule (copy_page_range) 0.109ms (+0.000ms): preempt_schedule (copy_page_range) 0.109ms (+0.000ms): preempt_schedule (copy_page_range) 0.109ms (+0.000ms): preempt_schedule (copy_page_range) 0.110ms (+0.000ms): preempt_schedule (copy_page_range) 0.110ms (+0.000ms): preempt_schedule (copy_page_range) 0.110ms (+0.000ms): preempt_schedule (copy_page_range) 0.111ms (+0.000ms): preempt_schedule (copy_page_range) 0.111ms (+0.000ms): preempt_schedule (copy_page_range) 0.111ms (+0.000ms): preempt_schedule (copy_page_range) 0.111ms (+0.000ms): preempt_schedule (copy_page_range) 0.112ms (+0.000ms): preempt_schedule (copy_page_range) 0.112ms (+0.000ms): preempt_schedule (copy_page_range) 0.112ms (+0.000ms): preempt_schedule (copy_page_range) 0.113ms (+0.000ms): preempt_schedule (copy_page_range) 0.113ms (+0.000ms): preempt_schedule (copy_page_range) 0.113ms (+0.000ms): preempt_schedule (copy_page_range) 0.114ms (+0.000ms): preempt_schedule (copy_page_range) 0.114ms (+0.000ms): preempt_schedule (copy_page_range) 0.114ms (+0.000ms): preempt_schedule (copy_page_range) 0.114ms (+0.000ms): preempt_schedule (copy_page_range) 0.115ms (+0.000ms): preempt_schedule (copy_page_range) 0.115ms (+0.000ms): preempt_schedule (copy_page_range) 0.115ms (+0.000ms): preempt_schedule (copy_page_range) 0.116ms (+0.000ms): preempt_schedule (copy_page_range) 0.116ms (+0.000ms): preempt_schedule (copy_page_range) 0.116ms (+0.000ms): preempt_schedule (copy_page_range) 0.116ms (+0.000ms): preempt_schedule (copy_page_range) 0.117ms (+0.000ms): preempt_schedule (copy_page_range) 0.117ms (+0.000ms): preempt_schedule (copy_page_range) 0.117ms (+0.000ms): preempt_schedule (copy_page_range) 0.118ms (+0.000ms): preempt_schedule (copy_page_range) 0.118ms (+0.000ms): preempt_schedule (copy_page_range) 0.118ms (+0.000ms): preempt_schedule (copy_page_range) 0.119ms (+0.000ms): preempt_schedule (copy_page_range) 0.119ms (+0.000ms): preempt_schedule (copy_page_range) 0.119ms (+0.000ms): preempt_schedule (copy_page_range) 0.119ms (+0.000ms): preempt_schedule (copy_page_range) 0.119ms (+0.000ms): preempt_schedule (copy_page_range) 0.119ms (+0.000ms): preempt_schedule (copy_page_range) 0.119ms (+0.000ms): preempt_schedule (copy_page_range) 0.120ms (+0.000ms): preempt_schedule (copy_page_range) 0.120ms (+0.000ms): preempt_schedule (copy_page_range) 0.120ms (+0.000ms): preempt_schedule (copy_page_range) 0.120ms (+0.000ms): preempt_schedule (copy_page_range) 0.121ms (+0.000ms): preempt_schedule (copy_page_range) 0.121ms (+0.000ms): preempt_schedule (copy_page_range) 0.121ms (+0.000ms): preempt_schedule (copy_page_range) 0.122ms (+0.001ms): do_IRQ (common_interrupt)
0.122ms (+0.000ms): mask_and_ack_8259A (do_IRQ)
0.124ms (+0.001ms): preempt_schedule (do_IRQ)
0.124ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 0.124ms (+0.000ms): preempt_schedule (do_IRQ)
0.124ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 0.125ms (+0.000ms): snd_cs46xx_interrupt (generic_handle_IRQ_event) 0.127ms (+0.001ms): snd_pcm_period_elapsed (snd_cs46xx_interrupt) 0.128ms (+0.001ms): snd_cs46xx_playback_direct_pointer
(snd_pcm_period_elapsed)
0.131ms (+0.002ms): __wake_up (snd_pcm_period_elapsed) 0.131ms (+0.000ms): __wake_up_common (__wake_up)
0.131ms (+0.000ms): default_wake_function (__wake_up_common) 0.131ms (+0.000ms): try_to_wake_up (__wake_up_common) 0.131ms (+0.000ms): task_rq_lock (try_to_wake_up) 0.131ms (+0.000ms): activate_task (try_to_wake_up) 0.131ms (+0.000ms): sched_clock (activate_task)
0.131ms (+0.000ms): recalc_task_prio (activate_task) 0.132ms (+0.000ms): effective_prio (recalc_task_prio) 0.132ms (+0.000ms): enqueue_task (activate_task)
0.132ms (+0.000ms): preempt_schedule (try_to_wake_up) 0.132ms (+0.000ms): preempt_schedule (snd_pcm_period_elapsed) 0.132ms (+0.000ms): preempt_schedule (snd_pcm_period_elapsed) 0.133ms (+0.000ms): preempt_schedule (snd_pcm_period_elapsed) 0.133ms (+0.000ms): kill_fasync (snd_pcm_period_elapsed) 0.133ms (+0.000ms): generic_note_interrupt (do_IRQ) 0.134ms (+0.000ms): end_8259A_irq (do_IRQ)
0.134ms (+0.000ms): enable_8259A_irq (do_IRQ)
0.134ms (+0.000ms): preempt_schedule (do_IRQ)
0.134ms (+0.000ms): preempt_schedule (do_IRQ)
0.135ms (+0.000ms): do_softirq (do_IRQ)
0.135ms (+0.000ms): __do_softirq (do_softirq)
0.135ms (+0.000ms): preempt_schedule (copy_page_range) 0.135ms (+0.000ms): preempt_schedule (copy_page_range) 0.135ms (+0.000ms): preempt_schedule (copy_page_range) 0.135ms (+0.000ms): preempt_schedule (copy_page_range) 0.136ms (+0.000ms): preempt_schedule (copy_page_range) 0.136ms (+0.000ms): preempt_schedule (copy_page_range) 0.136ms (+0.000ms): preempt_schedule (copy_page_range) 0.136ms (+0.000ms): preempt_schedule (copy_page_range) 0.136ms (+0.000ms): preempt_schedule (copy_page_range) 0.137ms (+0.000ms): preempt_schedule (copy_page_range) 0.137ms (+0.000ms): preempt_schedule (copy_page_range) 0.137ms (+0.000ms): preempt_schedule (copy_page_range) 0.137ms (+0.000ms): preempt_schedule (copy_page_range) 0.138ms (+0.000ms): preempt_schedule (copy_page_range) 0.138ms (+0.000ms): preempt_schedule (copy_page_range) 0.138ms (+0.000ms): preempt_schedule (copy_page_range) 0.139ms (+0.000ms): preempt_schedule (copy_page_range) 0.139ms (+0.000ms): preempt_schedule (copy_page_range) 0.139ms (+0.000ms): preempt_schedule (copy_page_range) 0.140ms (+0.000ms): preempt_schedule (copy_page_range) 0.140ms (+0.000ms): preempt_schedule (copy_page_range) 0.140ms (+0.000ms): preempt_schedule (copy_page_range) 0.141ms (+0.000ms): do_IRQ (common_interrupt)
0.141ms (+0.000ms): mask_and_ack_8259A (do_IRQ)
0.142ms (+0.001ms): preempt_schedule (do_IRQ)
0.143ms (+0.000ms): generic_redirect_hardirq (do_IRQ) 0.143ms (+0.000ms): preempt_schedule (do_IRQ)
0.143ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 0.143ms (+0.000ms): snd_cs46xx_interrupt (generic_handle_IRQ_event) 0.144ms (+0.001ms): snd_pcm_period_elapsed (snd_cs46xx_interrupt) 0.145ms (+0.001ms): snd_cs46xx_capture_indirect_pointer
(snd_pcm_period_elapsed)
0.146ms (+0.000ms): snd_cs46xx_capture_transfer
(snd_cs46xx_capture_indirect_pointer)
0.148ms (+0.002ms): __wake_up (snd_pcm_period_elapsed) 0.148ms (+0.000ms): __wake_up_common (__wake_up)
0.149ms (+0.000ms): default_wake_function (__wake_up_common) 0.149ms (+0.000ms): try_to_wake_up (__wake_up_common) 0.149ms (+0.000ms): task_rq_lock (try_to_wake_up) 0.149ms (+0.000ms): preempt_schedule (try_to_wake_up) 0.149ms (+0.000ms): preempt_schedule (snd_pcm_period_elapsed) 0.149ms (+0.000ms): preempt_schedule (snd_pcm_period_elapsed) 0.149ms (+0.000ms): preempt_schedule (snd_pcm_period_elapsed) 0.150ms (+0.000ms): kill_fasync (snd_pcm_period_elapsed) 0.150ms (+0.000ms): generic_note_interrupt (do_IRQ) 0.150ms (+0.000ms): end_8259A_irq (do_IRQ)
0.150ms (+0.000ms): enable_8259A_irq (do_IRQ)
0.151ms (+0.000ms): preempt_schedule (do_IRQ)
0.151ms (+0.000ms): preempt_schedule (do_IRQ)
0.151ms (+0.000ms): do_softirq (do_IRQ)
0.151ms (+0.000ms): __do_softirq (do_softirq)
0.151ms (+0.000ms): preempt_schedule (copy_page_range) 0.152ms (+0.000ms): preempt_schedule (copy_page_range) 0.152ms (+0.000ms): preempt_schedule (copy_page_range) 0.152ms (+0.000ms): preempt_schedule (copy_page_range) 0.152ms (+0.000ms): preempt_schedule (copy_page_range) 0.153ms (+0.000ms): preempt_schedule (copy_page_range) 0.153ms (+0.000ms): preempt_schedule (copy_page_range) 0.153ms (+0.000ms): preempt_schedule (copy_page_range) 0.153ms (+0.000ms): preempt_schedule (copy_page_range) 0.153ms (+0.000ms): preempt_schedule (copy_page_range) 0.154ms (+0.000ms): preempt_schedule (copy_page_range) 0.154ms (+0.000ms): preempt_schedule (copy_page_range) 0.154ms (+0.000ms): preempt_schedule (copy_page_range) 0.155ms (+0.000ms): preempt_schedule (copy_page_range) 0.155ms (+0.000ms): preempt_schedule (copy_page_range) 0.155ms (+0.000ms): preempt_schedule (copy_page_range) 0.155ms (+0.000ms): preempt_schedule (copy_page_range) 0.156ms (+0.000ms): preempt_schedule (copy_page_range) 0.156ms (+0.000ms): preempt_schedule (copy_page_range) 0.156ms (+0.000ms): preempt_schedule (copy_page_range) 0.156ms (+0.000ms): preempt_schedule (copy_page_range) 0.156ms (+0.000ms): preempt_schedule (copy_page_range) 0.156ms (+0.000ms): preempt_schedule (copy_page_range) 0.157ms (+0.000ms): preempt_schedule (copy_page_range) 0.157ms (+0.000ms): preempt_schedule (copy_page_range) 0.157ms (+0.000ms): preempt_schedule (copy_page_range) 0.157ms (+0.000ms): preempt_schedule (copy_page_range) 0.158ms (+0.000ms): preempt_schedule (copy_page_range) 0.158ms (+0.000ms): preempt_schedule (copy_page_range) 0.158ms (+0.000ms): preempt_schedule (copy_page_range) 0.158ms (+0.000ms): preempt_schedule (copy_page_range) 0.159ms (+0.000ms): preempt_schedule (copy_page_range) 0.159ms (+0.000ms): preempt_schedule (copy_page_range) 0.159ms (+0.000ms): preempt_schedule (copy_page_range) 0.159ms (+0.000ms): preempt_schedule (copy_page_range) 0.159ms (+0.000ms): preempt_schedule (copy_page_range) 0.160ms (+0.000ms): preempt_schedule (copy_page_range) 0.160ms (+0.000ms): preempt_schedule (copy_page_range) 0.160ms (+0.000ms): preempt_schedule (copy_page_range) 0.160ms (+0.000ms): preempt_schedule (copy_page_range) 0.160ms (+0.000ms): preempt_schedule (copy_page_range) 0.161ms (+0.000ms): preempt_schedule (copy_page_range) 0.161ms (+0.000ms): preempt_schedule (copy_page_range) 0.161ms (+0.000ms): preempt_schedule (copy_page_range) 0.161ms (+0.000ms): preempt_schedule (copy_page_range) 0.162ms (+0.000ms): preempt_schedule (copy_page_range) 0.162ms (+0.000ms): preempt_schedule (copy_page_range) 0.162ms (+0.000ms): preempt_schedule (copy_page_range) 0.162ms (+0.000ms): preempt_schedule (copy_page_range) 0.163ms (+0.000ms): preempt_schedule (copy_page_range) 0.163ms (+0.000ms): preempt_schedule (copy_page_range) 0.163ms (+0.000ms): preempt_schedule (copy_page_range) 0.163ms (+0.000ms): preempt_schedule (copy_page_range) 0.163ms (+0.000ms): preempt_schedule (copy_page_range) 0.164ms (+0.000ms): preempt_schedule (copy_page_range) 0.164ms (+0.000ms): preempt_schedule (copy_page_range) 0.164ms (+0.000ms): preempt_schedule (copy_page_range) 0.164ms (+0.000ms): preempt_schedule (copy_page_range) 0.165ms (+0.000ms): preempt_schedule (copy_page_range) 0.165ms (+0.000ms): preempt_schedule (copy_page_range) 0.165ms (+0.000ms): preempt_schedule (copy_page_range) 0.165ms (+0.000ms): preempt_schedule (copy_page_range) 0.166ms (+0.000ms): preempt_schedule (copy_page_range) 0.166ms (+0.000ms): preempt_schedule (copy_page_range) 0.166ms (+0.000ms): preempt_schedule (copy_page_range) 0.166ms (+0.000ms): preempt_schedule (copy_page_range) 0.167ms (+0.000ms): preempt_schedule (copy_page_range) 0.167ms (+0.000ms): preempt_schedule (copy_page_range) 0.167ms (+0.000ms): preempt_schedule (copy_page_range) 0.167ms (+0.000ms): preempt_schedule (copy_page_range) 0.167ms (+0.000ms): preempt_schedule (copy_page_range) 0.168ms (+0.000ms): preempt_schedule (copy_page_range) 0.168ms (+0.000ms): preempt_schedule (copy_page_range) 0.168ms (+0.000ms): preempt_schedule (copy_page_range) 0.168ms (+0.000ms): preempt_schedule (copy_page_range) 0.169ms (+0.000ms): preempt_schedule (copy_page_range) 0.169ms (+0.000ms): preempt_schedule (copy_page_range) 0.169ms (+0.000ms): preempt_schedule (copy_page_range) 0.169ms (+0.000ms): preempt_schedule (copy_page_range) 0.170ms (+0.000ms): preempt_schedule (copy_page_range) 0.170ms (+0.000ms): preempt_schedule (copy_page_range) 0.170ms (+0.000ms): preempt_schedule (copy_page_range) 0.170ms (+0.000ms): preempt_schedule (copy_page_range) 0.170ms (+0.000ms): preempt_schedule (copy_page_range) 0.170ms (+0.000ms): preempt_schedule (copy_page_range) 0.171ms (+0.000ms): preempt_schedule (copy_page_range) 0.171ms (+0.000ms): preempt_schedule (copy_page_range) 0.171ms (+0.000ms): preempt_schedule (copy_page_range) 0.171ms (+0.000ms): preempt_schedule (copy_page_range) 0.172ms (+0.000ms): preempt_schedule (copy_page_range) 0.172ms (+0.000ms): preempt_schedule (copy_page_range) 0.172ms (+0.000ms): preempt_schedule (copy_page_range) 0.172ms (+0.000ms): preempt_schedule (copy_page_range) 0.173ms (+0.000ms): preempt_schedule (copy_page_range) 0.173ms (+0.000ms): preempt_schedule (copy_page_range) 0.173ms (+0.000ms): preempt_schedule (copy_page_range) 0.173ms (+0.000ms): preempt_schedule (copy_page_range) 0.174ms (+0.000ms): preempt_schedule (copy_page_range) 0.174ms (+0.000ms): preempt_schedule (copy_page_range) 0.174ms (+0.000ms): preempt_schedule (copy_page_range) 0.174ms (+0.000ms): preempt_schedule (copy_page_range) 0.175ms (+0.000ms): preempt_schedule (copy_page_range) 0.175ms (+0.000ms): preempt_schedule (copy_page_range) 0.175ms (+0.000ms): preempt_schedule (copy_page_range) 0.175ms (+0.000ms): preempt_schedule (copy_page_range) 0.175ms (+0.000ms): preempt_schedule (copy_page_range) 0.176ms (+0.000ms): preempt_schedule (copy_page_range) 0.176ms (+0.000ms): preempt_schedule (copy_page_range) 0.176ms (+0.000ms): preempt_schedule (copy_page_range) 0.176ms (+0.000ms): preempt_schedule (copy_page_range) 0.177ms (+0.000ms): preempt_schedule (copy_page_range) 0.177ms (+0.000ms): preempt_schedule (copy_page_range) 0.177ms (+0.000ms): preempt_schedule (copy_page_range) 0.177ms (+0.000ms): preempt_schedule (copy_page_range) 0.177ms (+0.000ms): preempt_schedule (copy_page_range) 0.178ms (+0.000ms): preempt_schedule (copy_page_range) 0.178ms (+0.000ms): preempt_schedule (copy_page_range) 0.178ms (+0.000ms): preempt_schedule (copy_page_range) 0.179ms (+0.000ms): preempt_schedule (copy_page_range) 0.179ms (+0.000ms): preempt_schedule (copy_page_range) 0.179ms (+0.000ms): preempt_schedule (copy_page_range) 0.179ms (+0.000ms): preempt_schedule (copy_page_range) 0.180ms (+0.000ms): preempt_schedule (copy_page_range) 0.180ms (+0.000ms): preempt_schedule (copy_page_range) 0.180ms (+0.000ms): preempt_schedule (copy_page_range) 0.180ms (+0.000ms): preempt_schedule (copy_page_range) 0.181ms (+0.000ms): preempt_schedule (copy_page_range) 0.181ms (+0.000ms): preempt_schedule (copy_page_range) 0.181ms (+0.000ms): preempt_schedule (copy_page_range) 0.181ms (+0.000ms): preempt_schedule (copy_page_range) 0.181ms (+0.000ms): preempt_schedule (copy_page_range) 0.182ms (+0.000ms): preempt_schedule (copy_page_range) 0.182ms (+0.000ms): preempt_schedule (copy_page_range) 0.182ms (+0.000ms): preempt_schedule (copy_page_range) 0.182ms (+0.000ms): preempt_schedule (copy_page_range) 0.182ms (+0.000ms): preempt_schedule (copy_page_range) 0.182ms (+0.000ms): preempt_schedule (copy_page_range) 0.183ms (+0.000ms): preempt_schedule (copy_page_range) 0.183ms (+0.000ms): preempt_schedule (copy_page_range) 0.183ms (+0.000ms): preempt_schedule (copy_page_range) 0.184ms (+0.000ms): preempt_schedule (copy_page_range) 0.184ms (+0.000ms): preempt_schedule (copy_page_range) 0.184ms (+0.000ms): preempt_schedule (copy_page_range) 0.184ms (+0.000ms): preempt_schedule (copy_page_range) 0.185ms (+0.000ms): preempt_schedule (copy_page_range) 0.185ms (+0.000ms): preempt_schedule (copy_page_range) 0.185ms (+0.000ms): preempt_schedule (copy_page_range) 0.185ms (+0.000ms): preempt_schedule (copy_page_range) 0.186ms (+0.000ms): preempt_schedule (copy_page_range) 0.186ms (+0.000ms): preempt_schedule (copy_page_range) 0.186ms (+0.000ms): preempt_schedule (copy_page_range) 0.186ms (+0.000ms): preempt_schedule (copy_page_range) 0.186ms (+0.000ms): preempt_schedule (copy_page_range) 0.187ms (+0.000ms): preempt_schedule (copy_page_range) 0.187ms (+0.000ms): preempt_schedule (copy_page_range) 0.187ms (+0.000ms): preempt_schedule (copy_page_range) 0.187ms (+0.000ms): preempt_schedule (copy_page_range) 0.188ms (+0.000ms): preempt_schedule (copy_page_range) 0.188ms (+0.000ms): preempt_schedule (copy_page_range) 0.188ms (+0.000ms): preempt_schedule (copy_page_range) 0.188ms (+0.000ms): preempt_schedule (copy_page_range) 0.188ms (+0.000ms): preempt_schedule (copy_page_range) 0.189ms (+0.000ms): preempt_schedule (copy_page_range) 0.189ms (+0.000ms): preempt_schedule (copy_page_range) 0.189ms (+0.000ms): preempt_schedule (copy_page_range) 0.189ms (+0.000ms): preempt_schedule (copy_page_range) 0.189ms (+0.000ms): preempt_schedule (copy_page_range) 0.190ms (+0.000ms): preempt_schedule (copy_page_range) 0.190ms (+0.000ms): preempt_schedule (copy_page_range) 0.190ms (+0.000ms): preempt_schedule (copy_page_range) 0.190ms (+0.000ms): preempt_schedule (copy_page_range) 0.191ms (+0.000ms): preempt_schedule (copy_page_range) 0.191ms (+0.000ms): preempt_schedule (copy_page_range) 0.191ms (+0.000ms): preempt_schedule (copy_page_range) 0.191ms (+0.000ms): preempt_schedule (copy_page_range) 0.191ms (+0.000ms): preempt_schedule (copy_page_range) 0.192ms (+0.000ms): preempt_schedule (copy_page_range) 0.192ms (+0.000ms): preempt_schedule (copy_page_range) 0.192ms (+0.000ms): preempt_schedule (copy_page_range) 0.192ms (+0.000ms): preempt_schedule (copy_page_range) 0.193ms (+0.000ms): preempt_schedule (copy_page_range) 0.193ms (+0.000ms): preempt_schedule (copy_page_range) 0.193ms (+0.000ms): preempt_schedule (copy_page_range) 0.193ms (+0.000ms): preempt_schedule (copy_page_range) 0.193ms (+0.000ms): preempt_schedule (copy_page_range) 0.194ms (+0.000ms): preempt_schedule (copy_page_range) 0.194ms (+0.000ms): preempt_schedule (copy_page_range) 0.194ms (+0.000ms): check_preempt_timing (touch_preempt_timing)
preemption latency trace v1.0
-----------------------------
latency: 212 us, entries: 402 (402)
process: sylpheed-claws/796, uid: 1000
nice: 0, policy: 0, rt_priority: 0
=======>
0.000ms (+0.000ms): do_IRQ (common_interrupt)
0.000ms (+0.000ms): mask_and_ack_8259A (do_IRQ)
0.001ms (+0.001ms): generic_redirect_hardirq (do_IRQ) 0.001ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 0.001ms (+0.000ms): snd_cs46xx_interrupt (generic_handle_IRQ_event) 0.003ms (+0.001ms): snd_pcm_period_elapsed (snd_cs46xx_interrupt) 0.004ms (+0.000ms): snd_cs46xx_capture_indirect_pointer
(snd_pcm_period_elapsed)
0.005ms (+0.001ms): snd_cs46xx_capture_transfer
(snd_cs46xx_capture_indirect_pointer)
0.007ms (+0.001ms): __wake_up (snd_pcm_period_elapsed) 0.007ms (+0.000ms): __wake_up_common (__wake_up)
0.007ms (+0.000ms): default_wake_function (__wake_up_common) 0.007ms (+0.000ms): try_to_wake_up (__wake_up_common) 0.007ms (+0.000ms): task_rq_lock (try_to_wake_up) 0.007ms (+0.000ms): activate_task (try_to_wake_up) 0.007ms (+0.000ms): sched_clock (activate_task)
0.007ms (+0.000ms): recalc_task_prio (activate_task) 0.007ms (+0.000ms): effective_prio (recalc_task_prio) 0.008ms (+0.000ms): enqueue_task (activate_task)
0.008ms (+0.000ms): preempt_schedule (try_to_wake_up) 0.008ms (+0.000ms): preempt_schedule (snd_pcm_period_elapsed) 0.008ms (+0.000ms): preempt_schedule (snd_pcm_period_elapsed) 0.008ms (+0.000ms): preempt_schedule (snd_pcm_period_elapsed) 0.008ms (+0.000ms): kill_fasync (snd_pcm_period_elapsed) 0.009ms (+0.000ms): generic_note_interrupt (do_IRQ) 0.009ms (+0.000ms): end_8259A_irq (do_IRQ)
0.009ms (+0.000ms): enable_8259A_irq (do_IRQ)
0.010ms (+0.000ms): preempt_schedule (do_IRQ)
0.010ms (+0.000ms): preempt_schedule (do_IRQ)
0.010ms (+0.000ms): preempt_schedule (copy_page_range) 0.011ms (+0.000ms): preempt_schedule (copy_page_range) 0.011ms (+0.000ms): preempt_schedule (copy_page_range) 0.011ms (+0.000ms): preempt_schedule (copy_page_range) 0.011ms (+0.000ms): preempt_schedule (copy_page_range) 0.012ms (+0.000ms): preempt_schedule (copy_page_range) 0.012ms (+0.000ms): preempt_schedule (copy_page_range) 0.012ms (+0.000ms): preempt_schedule (copy_page_range) 0.012ms (+0.000ms): preempt_schedule (copy_page_range) 0.012ms (+0.000ms): preempt_schedule (copy_page_range) 0.013ms (+0.000ms): preempt_schedule (copy_page_range) 0.013ms (+0.000ms): preempt_schedule (copy_page_range) 0.013ms (+0.000ms): preempt_schedule (copy_page_range) 0.013ms (+0.000ms): preempt_schedule (copy_page_range) 0.014ms (+0.000ms): preempt_schedule (copy_page_range) 0.014ms (+0.000ms): preempt_schedule (copy_page_range) 0.014ms (+0.000ms): preempt_schedule (copy_page_range) 0.014ms (+0.000ms): preempt_schedule (copy_page_range) 0.015ms (+0.000ms): preempt_schedule (copy_page_range) 0.015ms (+0.000ms): preempt_schedule (copy_page_range) 0.015ms (+0.000ms): preempt_schedule (copy_page_range) 0.016ms (+0.000ms): preempt_schedule (copy_page_range) 0.016ms (+0.000ms): preempt_schedule (copy_page_range) 0.016ms (+0.000ms): preempt_schedule (copy_page_range) 0.017ms (+0.000ms): preempt_schedule (copy_page_range) 0.017ms (+0.000ms): preempt_schedule (copy_page_range) 0.017ms (+0.000ms): preempt_schedule (copy_page_range) 0.017ms (+0.000ms): preempt_schedule (copy_page_range) 0.017ms (+0.000ms): preempt_schedule (copy_page_range) 0.018ms (+0.000ms): preempt_schedule (copy_page_range) 0.018ms (+0.000ms): preempt_schedule (copy_page_range) 0.018ms (+0.000ms): preempt_schedule (copy_page_range) 0.018ms (+0.000ms): preempt_schedule (copy_page_range) 0.019ms (+0.000ms): preempt_schedule (copy_page_range) 0.019ms (+0.000ms): preempt_schedule (copy_page_range) 0.019ms (+0.000ms): preempt_schedule (copy_page_range) 0.019ms (+0.000ms): preempt_schedule (copy_page_range) 0.020ms (+0.000ms): preempt_schedule (copy_page_range) 0.020ms (+0.000ms): preempt_schedule (copy_page_range) 0.020ms (+0.000ms): preempt_schedule (copy_page_range) 0.020ms (+0.000ms): preempt_schedule (copy_page_range) 0.021ms (+0.000ms): preempt_schedule (copy_page_range) 0.021ms (+0.000ms): preempt_schedule (copy_page_range) 0.021ms (+0.000ms): preempt_schedule (copy_page_range) 0.021ms (+0.000ms): preempt_schedule (copy_page_range) 0.022ms (+0.000ms): preempt_schedule (copy_page_range) 0.022ms (+0.000ms): preempt_schedule (copy_page_range) 0.022ms (+0.000ms): preempt_schedule (copy_page_range) 0.022ms (+0.000ms): preempt_schedule (copy_page_range) 0.022ms (+0.000ms): preempt_schedule (copy_page_range) 0.022ms (+0.000ms): preempt_schedule (copy_page_range) 0.023ms (+0.000ms): preempt_schedule (copy_page_range) 0.023ms (+0.000ms): preempt_schedule (copy_page_range) 0.024ms (+0.000ms): preempt_schedule (copy_page_range) 0.024ms (+0.000ms): preempt_schedule (copy_page_range) 0.024ms (+0.000ms): preempt_schedule (copy_page_range) 0.024ms (+0.000ms): preempt_schedule (copy_page_range) 0.024ms (+0.000ms): preempt_schedule (copy_page_range) 0.025ms (+0.000ms): preempt_schedule (copy_page_range) 0.025ms (+0.000ms): preempt_schedule (copy_page_range) 0.025ms (+0.000ms): preempt_schedule (copy_page_range) 0.025ms (+0.000ms): preempt_schedule (copy_page_range) 0.025ms (+0.000ms): preempt_schedule (copy_page_range) 0.026ms (+0.000ms): preempt_schedule (copy_page_range) 0.026ms (+0.000ms): preempt_schedule (copy_page_range) 0.026ms (+0.000ms): preempt_schedule (copy_page_range) 0.026ms (+0.000ms): preempt_schedule (copy_page_range) 0.027ms (+0.000ms): preempt_schedule (copy_page_range) 0.027ms (+0.000ms): preempt_schedule (copy_page_range) 0.027ms (+0.000ms): preempt_schedule (copy_page_range) 0.028ms (+0.000ms): preempt_schedule (copy_page_range) 0.028ms (+0.000ms): preempt_schedule (copy_page_range) 0.028ms (+0.000ms): preempt_schedule (copy_page_range) 0.028ms (+0.000ms): preempt_schedule (copy_page_range) 0.028ms (+0.000ms): preempt_schedule (copy_page_range) 0.029ms (+0.000ms): preempt_schedule (copy_page_range) 0.029ms (+0.000ms): preempt_schedule (copy_page_range) 0.029ms (+0.000ms): preempt_schedule (copy_page_range) 0.029ms (+0.000ms): preempt_schedule (copy_page_range) 0.030ms (+0.000ms): preempt_schedule (copy_page_range) 0.030ms (+0.000ms): preempt_schedule (copy_page_range) 0.030ms (+0.000ms): preempt_schedule (copy_page_range) 0.030ms (+0.000ms): preempt_schedule (copy_page_range) 0.031ms (+0.000ms): preempt_schedule (copy_page_range) 0.031ms (+0.000ms): preempt_schedule (copy_page_range) 0.031ms (+0.000ms): preempt_schedule (copy_page_range) 0.031ms (+0.000ms): preempt_schedule (copy_page_range) 0.032ms (+0.000ms): preempt_schedule (copy_page_range) 0.032ms (+0.000ms): preempt_schedule (copy_page_range) 0.032ms (+0.000ms): preempt_schedule (copy_page_range) 0.032ms (+0.000ms): preempt_schedule (copy_page_range) 0.033ms (+0.000ms): preempt_schedule (copy_page_range) 0.033ms (+0.000ms): preempt_schedule (copy_page_range) 0.033ms (+0.000ms): preempt_schedule (copy_page_range) 0.033ms (+0.000ms): preempt_schedule (copy_page_range) 0.033ms (+0.000ms): preempt_schedule (copy_page_range) 0.034ms (+0.000ms): preempt_schedule (copy_page_range) 0.034ms (+0.000ms): preempt_schedule (copy_page_range) 0.034ms (+0.000ms): preempt_schedule (copy_page_range) 0.034ms (+0.000ms): preempt_schedule (copy_page_range) 0.035ms (+0.000ms): preempt_schedule (copy_page_range) 0.035ms (+0.000ms): preempt_schedule (copy_page_range) 0.035ms (+0.000ms): preempt_schedule (copy_page_range) 0.035ms (+0.000ms): preempt_schedule (copy_page_range) 0.036ms (+0.000ms): preempt_schedule (copy_page_range) 0.036ms (+0.000ms): preempt_schedule (copy_page_range) 0.036ms (+0.000ms): preempt_schedule (copy_page_range) 0.036ms (+0.000ms): preempt_schedule (copy_page_range) 0.036ms (+0.000ms): preempt_schedule (copy_page_range) 0.037ms (+0.000ms): preempt_schedule (copy_page_range) 0.037ms (+0.000ms): preempt_schedule (copy_page_range) 0.037ms (+0.000ms): preempt_schedule (copy_page_range) 0.037ms (+0.000ms): preempt_schedule (copy_page_range) 0.038ms (+0.000ms): preempt_schedule (copy_page_range) 0.038ms (+0.000ms): preempt_schedule (copy_page_range) 0.038ms (+0.000ms): preempt_schedule (copy_page_range) 0.039ms (+0.000ms): preempt_schedule (copy_page_range) 0.039ms (+0.000ms): preempt_schedule (copy_page_range) 0.039ms (+0.000ms): preempt_schedule (copy_page_range) 0.039ms (+0.000ms): preempt_schedule (copy_page_range) 0.040ms (+0.000ms): preempt_schedule (copy_page_range) 0.040ms (+0.000ms): preempt_schedule (copy_page_range) 0.040ms (+0.000ms): preempt_schedule (copy_page_range) 0.040ms (+0.000ms): preempt_schedule (copy_page_range) 0.040ms (+0.000ms): preempt_schedule (copy_page_range) 0.041ms (+0.000ms): preempt_schedule (copy_page_range) 0.041ms (+0.000ms): preempt_schedule (copy_page_range) 0.041ms (+0.000ms): preempt_schedule (copy_page_range) 0.042ms (+0.000ms): preempt_schedule (copy_page_range) 0.042ms (+0.000ms): preempt_schedule (copy_page_range) 0.042ms (+0.000ms): preempt_schedule (copy_page_range) 0.043ms (+0.000ms): preempt_schedule (copy_page_range) 0.043ms (+0.000ms): preempt_schedule (copy_page_range) 0.044ms (+0.000ms): preempt_schedule (copy_page_range) 0.044ms (+0.000ms): preempt_schedule (copy_page_range) 0.044ms (+0.000ms): preempt_schedule (copy_page_range) 0.045ms (+0.000ms): preempt_schedule (copy_page_range) 0.045ms (+0.000ms): preempt_schedule (copy_page_range) 0.045ms (+0.000ms): preempt_schedule (copy_page_range) 0.046ms (+0.000ms): preempt_schedule (copy_page_range) 0.046ms (+0.000ms): preempt_schedule (copy_page_range) 0.046ms (+0.000ms): preempt_schedule (copy_page_range) 0.046ms (+0.000ms): preempt_schedule (copy_page_range) 0.047ms (+0.000ms): preempt_schedule (copy_page_range) 0.047ms (+0.000ms): preempt_schedule (copy_page_range) 0.047ms (+0.000ms): preempt_schedule (copy_page_range) 0.047ms (+0.000ms): preempt_schedule (copy_page_range) 0.048ms (+0.000ms): preempt_schedule (copy_page_range) 0.048ms (+0.000ms): preempt_schedule (copy_page_range) 0.049ms (+0.000ms): preempt_schedule (copy_page_range) 0.049ms (+0.000ms): preempt_schedule (copy_page_range) 0.049ms (+0.000ms): preempt_schedule (copy_page_range) 0.049ms (+0.000ms): preempt_schedule (copy_page_range) 0.050ms (+0.000ms): preempt_schedule (copy_page_range) 0.050ms (+0.000ms): preempt_schedule (copy_page_range) 0.050ms (+0.000ms): preempt_schedule (copy_page_range) 0.051ms (+0.000ms): preempt_schedule (copy_page_range) 0.051ms (+0.000ms): preempt_schedule (copy_page_range) 0.051ms (+0.000ms): preempt_schedule (copy_page_range) 0.052ms (+0.000ms): preempt_schedule (copy_page_range) 0.052ms (+0.000ms): preempt_schedule (copy_page_range) 0.052ms (+0.000ms): preempt_schedule (copy_page_range) 0.053ms (+0.000ms): preempt_schedule (copy_page_range) 0.053ms (+0.000ms): preempt_schedule (copy_page_range) 0.054ms (+0.000ms): preempt_schedule (copy_page_range) 0.054ms (+0.000ms): preempt_schedule (copy_page_range) 0.054ms (+0.000ms): preempt_schedule (copy_page_range) 0.054ms (+0.000ms): preempt_schedule (copy_page_range) 0.055ms (+0.000ms): preempt_schedule (copy_page_range) 0.055ms (+0.000ms): preempt_schedule (copy_page_range) 0.055ms (+0.000ms): preempt_schedule (copy_page_range) 0.055ms (+0.000ms): preempt_schedule (copy_page_range) 0.056ms (+0.000ms): preempt_schedule (copy_page_range) 0.056ms (+0.000ms): preempt_schedule (copy_page_range) 0.056ms (+0.000ms): preempt_schedule (copy_page_range) 0.057ms (+0.000ms): preempt_schedule (copy_page_range) 0.057ms (+0.000ms): preempt_schedule (copy_page_range) 0.057ms (+0.000ms): preempt_schedule (copy_page_range) 0.057ms (+0.000ms): preempt_schedule (copy_page_range) 0.058ms (+0.000ms): preempt_schedule (copy_page_range) 0.058ms (+0.000ms): preempt_schedule (copy_page_range) 0.058ms (+0.000ms): preempt_schedule (copy_page_range) 0.059ms (+0.000ms): preempt_schedule (copy_page_range) 0.059ms (+0.000ms): preempt_schedule (copy_page_range) 0.059ms (+0.000ms): preempt_schedule (copy_page_range) 0.060ms (+0.000ms): preempt_schedule (copy_page_range) 0.060ms (+0.000ms): preempt_schedule (copy_page_range) 0.060ms (+0.000ms): preempt_schedule (copy_page_range) 0.061ms (+0.000ms): preempt_schedule (copy_page_range) 0.061ms (+0.000ms): preempt_schedule (copy_page_range) 0.061ms (+0.000ms): preempt_schedule (copy_page_range) 0.061ms (+0.000ms): preempt_schedule (copy_page_range) 0.061ms (+0.000ms): preempt_schedule (copy_page_range) 0.062ms (+0.000ms): preempt_schedule (copy_page_range) 0.062ms (+0.000ms): preempt_schedule (copy_page_range) 0.062ms (+0.000ms): preempt_schedule (copy_page_range) 0.063ms (+0.000ms): preempt_schedule (copy_page_range) 0.063ms (+0.000ms): preempt_schedule (copy_page_range) 0.063ms (+0.000ms): preempt_schedule (copy_page_range) 0.063ms (+0.000ms): preempt_schedule (copy_page_range) 0.064ms (+0.000ms): preempt_schedule (copy_page_range) 0.064ms (+0.000ms): preempt_schedule (copy_page_range) 0.064ms (+0.000ms): preempt_schedule (copy_page_range) 0.064ms (+0.000ms): preempt_schedule (copy_page_range) 0.064ms (+0.000ms): preempt_schedule (copy_page_range) 0.065ms (+0.000ms): preempt_schedule (copy_page_range) 0.065ms (+0.000ms): preempt_schedule (copy_page_range) 0.065ms (+0.000ms): preempt_schedule (copy_page_range) 0.065ms (+0.000ms): preempt_schedule (copy_page_range) 0.066ms (+0.000ms): preempt_schedule (copy_page_range) 0.066ms (+0.000ms): preempt_schedule (copy_page_range) 0.066ms (+0.000ms): preempt_schedule (copy_page_range) 0.067ms (+0.000ms): preempt_schedule (copy_page_range) 0.067ms (+0.000ms): preempt_schedule (copy_page_range) 0.067ms (+0.000ms): preempt_schedule (copy_page_range) 0.067ms (+0.000ms): preempt_schedule (copy_page_range) 0.067ms (+0.000ms): preempt_schedule (copy_page_range) 0.068ms (+0.000ms): preempt_schedule (copy_page_range) 0.068ms (+0.000ms): preempt_schedule (copy_page_range) 0.068ms (+0.000ms): preempt_schedule (copy_page_range) 0.069ms (+0.000ms): preempt_schedule (copy_page_range) 0.069ms (+0.000ms): preempt_schedule (copy_page_range) 0.069ms (+0.000ms): preempt_schedule (copy_page_range) 0.069ms (+0.000ms): preempt_schedule (copy_page_range) 0.070ms (+0.000ms): preempt_schedule (copy_page_range) 0.070ms (+0.000ms): preempt_schedule (copy_page_range) 0.070ms (+0.000ms): preempt_schedule (copy_page_range) 0.070ms (+0.000ms): preempt_schedule (copy_page_range) 0.071ms (+0.000ms): preempt_schedule (copy_page_range) 0.071ms (+0.000ms): preempt_schedule (copy_page_range) 0.071ms (+0.000ms): preempt_schedule (copy_page_range) 0.072ms (+0.000ms): preempt_schedule (copy_page_range) 0.072ms (+0.000ms): preempt_schedule (copy_page_range) 0.072ms (+0.000ms): preempt_schedule (copy_page_range) 0.072ms (+0.000ms): preempt_schedule (copy_page_range) 0.072ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.073ms (+0.000ms): preempt_schedule (copy_page_range) 0.074ms (+0.000ms): preempt_schedule (copy_page_range) 0.074ms (+0.000ms): preempt_schedule (copy_page_range) 0.074ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.075ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.076ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.077ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.078ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.079ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.080ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.081ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.082ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.083ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.084ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.085ms (+0.000ms): preempt_schedule (copy_page_range) 0.086ms (+0.000ms): preempt_schedule (copy_page_range) 0.086ms (+0.000ms): preempt_schedule (copy_page_range) 0.086ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.087ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.088ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.089ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.090ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.091ms (+0.000ms): preempt_schedule (copy_page_range) 0.092ms (+0.000ms): preempt_schedule (copy_page_range) 0.092ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.093ms (+0.000ms): preempt_schedule (copy_page_range) 0.094ms (+0.000ms): preempt_schedule (copy_page_range) 0.094ms (+0.000ms): preempt_schedule (copy_page_range) 0.094ms (+0.000ms): preempt_schedule (copy_page_range) 0.094ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.095ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.096ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.097ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.098ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.099ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.100ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.101ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.102ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.103ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.104ms (+0.000ms): preempt_schedule (copy_page_range) 0.105ms (+0.000ms): preempt_schedule (copy_page_range) 0.105ms (+0.000ms): preempt_schedule (copy_page_range) 0.105ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.106ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.107ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.108ms (+0.000ms): preempt_schedule (copy_page_range) 0.109ms (+0.000ms): check_preempt_timing (touch_preempt_timing)
Signature · 2 lines
Palimm Palimm! http://affenbande.org/~tapas/
Re: [patch] voluntary-preempt-2.6.8.1-P1
Quoted 13 lines · Florian Schmidt
Quoted 10 lines · On Mon, 16 Aug 2004 06:05:15 +0200 Ingo Molnar <[email protected]> wrote:
Quoted 2 lines · Lee Revell
Anyway, the change to
sched.cfixes the mlockall bug, it works perfectly now. Thanks!great! This fix also means that we've got one more lock-break in the ext3 journalling code and one more lock-break in
dcache.c. I've released-P1 with the fix included:
Quoted 9 lines
Hi,
i don't know if this was mentioned before, but i sometimes see traces like this where half the entries are "
preempt_schedule(copy_page_range)". I just wanted to ask if this is normal and expected behaviour.
Yes, Ingo identified an issue with copy_page_range, I don't think it's fixed yet. See the voluntary-preempt-2.6.8.1-P0 thread.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P1
Quoted 7 lines · Lee Revell
Quoted 4 lines
i don't know if this was mentioned before, but i sometimes see traces like this where half the entries are "
preempt_schedule(copy_page_range)". I just wanted to ask if this is normal and expected behaviour.Yes, Ingo identified an issue with
copy_page_range, I don't think it's fixed yet. See the voluntary-preempt-2.6.8.1-P0 thread.
right, it's not fixed yet. It's not a trivial critical section - we are holding two locks and are mapping two atomic kmaps.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P1
Quoted 5 lines · Ingo Molnar
Quoted 2 lines
Yes, Ingo identified an issue with
copy_page_range, I don't think it's fixed yet. See the voluntary-preempt-2.6.8.1-P0 thread.right, it's not fixed yet. It's not a trivial critical section - we are holding two locks and are mapping two atomic kmaps.
fortunately it's easier than i thought - neither the source pmd nor the target pmd can go away so we can simply drop the locks, reschedule, and remap. Does the patch below (ontop of -P3) fix the copy_page_range() latencies you are seeing?
Ingo
mm/memory.c renamed from mm/memory.c.orig +9 -0
@@ -337,6 +337,15 @@ cont_copy_pte_range_noset: } src_pte++; dst_pte++; +if (voluntary_need_resched ()) { +pte_unmap_nested (src_pte); +pte_unmap (dst_pte); +spin_unlock (&src->page_table_lock); +cond_resched_lock (&dst->page_table_lock); +spin_lock (&src->page_table_lock); + dst_pte =pte_offset_map (dst_pmd, address); + src_pte =pte_offset_map_nested (src_pmd, address); + } }while ((unsigned long )src_pte &PTE_TABLE_MASK );pte_unmap_nested (src_pte-1 );pte_unmap (dst_pte-1 );
Re: [patch] voluntary-preempt-2.6.8.1-P1
Quoted 5 lines · Ingo Molnar
Quoted 3 lines · Ingo Molnar
Quoted 2 lines
Yes, Ingo identified an issue with
copy_page_range, I don't think it's fixed yet. See the voluntary-preempt-2.6.8.1-P0 thread.
Quoted 8 lines
Quoted 2 lines
right, it's not fixed yet. It's not a trivial critical section - we are holding two locks and are mapping two atomic kmaps.
fortunately it's easier than i thought - neither the source pmd nor the target pmd can go away so we can simply drop the locks, reschedule, and remap. Does the patch below (ontop of -P3) fix the
copy_page_range()latencies you are seeing?
Yes, this fixes it. Now the most frequent issue is the xfree86 unmap_vmas latency:
http://krustophenia.net/testresults.php?dataset=2.6.8.1-P3
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P1
In reply to Ingo MolnarQuoted 8 lines · Ingo Molnar
Quoted 5 lines · Ingo Molnar
Quoted 2 lines
Yes, Ingo identified an issue with
copy_page_range, I don't think it's fixed yet. See the voluntary-preempt-2.6.8.1-P0 thread.right, it's not fixed yet. It's not a trivial critical section - we are holding two locks and are mapping two atomic kmaps.
Quoted 7 lines
fortunately it's easier than i thought - neither the source pmd nor the target pmd can go away so we can simply drop the locks, reschedule, and remap. Does the patch below (ontop of -P3) fix the
copy_page_range()latencies you are seeing?Ingo
Ooops, sorry about my previous post, I had missed this one. It fixes the problem, thanks.
Thomas
Re: [patch] voluntary-preempt-2.6.8.1-P0
In reply to Ingo MolnarQuoted 11 lines · Ingo Molnar
Quoted 5 lines · Lee Revell
I believe the constant-time behavior that I reported was an artifact of ALSA xrun debugging. Now it seems like the latency produced *does* correspond directly to the amount of memory being mlockall'ed. If ./mlockall-test 1500 triggers an xrun at all it's ~0.2ms. 3000 triggers a ~1ms xrun, and 10000 a ~3 ms xrun.
could you try more extreme values - e.g. does 100 MB cause a 30 msec xrun?
This causes a series of xruns, ranging from maybe 2 to 10ms.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P0
In reply to Lee RevellQuoted 1 line · Lee Revell
nice. (What is the difference between the left-hand and the right-hand graphs - why is the right-hand side one 'wider'?)
Quoted 7 lines
The peaks on this graph should correspond directly to the length of the non-preemptible critical section reported by Ingo's latency tracer. I think the large peak around 580-600usecs is caused by the
extract_entropyissue (which can be hit by regular processes and ksoftirqd), and the large peak around 80-100 by the XFree86unmap_vmasissue, as the times match and these are by far the most common reported inlatency_trace.
just to check this theory, could you make __check_and_rekey() an empty function? This should still produce a working random driver, albeit at much reduced entropy. If these latencies have a relationship to the mlockall() issue then this change should have an effect.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P0
Quoted 7 lines · Ingo Molnar
Quoted 1 line · Lee Revell
nice. (What is the difference between the left-hand and the right-hand graphs - why is the right-hand side one 'wider'?)
The right hand graph is logarithmically scaled on the y axis (set logscale y in gnuplot). Some of the latencies are rare enough to not show up at all on the linear scale, like the peak in the 400s.
Quoted 8 lines
Quoted 7 lines
The peaks on this graph should correspond directly to the length of the non-preemptible critical section reported by Ingo's latency tracer. I think the large peak around 580-600usecs is caused by the
extract_entropyissue (which can be hit by regular processes and ksoftirqd), and the large peak around 80-100 by the XFree86unmap_vmasissue, as the times match and these are by far the most common reported inlatency_trace.
Quoted 5 lines
just to check this theory, could you make
__check_and_rekey()an empty function? This should still produce a working random driver, albeit at much reduced entropy. If these latencies have a relationship to themlockall()issue then this change should have an effect.
Sure, will try this next.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P0
In reply to Ingo MolnarQuoted 4 lines · Ingo Molnar
just to check this theory, could you make
__check_and_rekey()an empty function? This should still produce a working random driver, albeit at much reduced entropy. If these latencies have a relationship to themlockall()issue then this change should have an effect.
hm, could you disable the random driver in the .config rather? It seems that adding to the entropy pool (from hardirq context) alone is quite expensive too.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P0
Quoted 11 lines · Ingo Molnar
Quoted 4 lines · Ingo Molnar
just to check this theory, could you make
__check_and_rekey()an empty function? This should still produce a working random driver, albeit at much reduced entropy. If these latencies have a relationship to themlockall()issue then this change should have an effect.hm, could you disable the random driver in the .config rather? It seems that adding to the entropy pool (from hardirq context) alone is quite expensive too.
Can this be disabled in the .config? I can't find an option for it.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P0
Quoted 11 lines · Lee Revell
Quoted 9 lines
Quoted 4 lines
just to check this theory, could you make
__check_and_rekey()an empty function? This should still produce a working random driver, albeit at much reduced entropy. If these latencies have a relationship to themlockall()issue then this change should have an effect.hm, could you disable the random driver in the .config rather? It seems that adding to the entropy pool (from hardirq context) alone is quite expensive too.
Can this be disabled in the .config? I can't find an option for it.
oh well, indeed it cannot be disabled. Then i'd suggest to return early from extract_entropy(), without doing anything. That is the function that seems to introduce the worst overhead.
Ingo
Re: [patch] voluntary-preempt-2.6.8.1-P0
Quoted 12 lines · Ingo Molnar
Quoted 10 lines · Lee Revell
Quoted 9 lines
Quoted 4 lines
just to check this theory, could you make
__check_and_rekey()an empty function? This should still produce a working random driver, albeit at much reduced entropy. If these latencies have a relationship to themlockall()issue then this change should have an effect.hm, could you disable the random driver in the .config rather? It seems that adding to the entropy pool (from hardirq context) alone is quite expensive too.
Quoted 6 lines
Quoted 1 line
Can this be disabled in the .config? I can't find an option for it.
oh well, indeed it cannot be disabled. Then i'd suggest to return early from
extract_entropy(), without doing anything. That is the function that seems to introduce the worst overhead.
Yes, I really seem to remember this could be disabled in the past... maybe I am thinking of BSD/OS, or maybe this CONFIG_ option was removed.
This was caused by 'Actions -> Run -> rxvt':
preemption latency trace v1.0
-----------------------------
latency: 409 us, entries: 300 (300)
process: gnome-panel/7575, uid: 1000
nice: 0, policy: 0, rt_priority: 0
=======>
0.000ms (+0.000ms): __vma_link_rb (copy_mm)
0.000ms (+0.000ms): rb_insert_color (copy_mm)
0.000ms (+0.000ms): __rb_rotate_left (rb_insert_color) 0.000ms (+0.000ms): copy_page_range (copy_mm)
0.001ms (+0.000ms): pte_alloc_map (copy_page_range) 0.205ms (+0.204ms): do_IRQ (common_interrupt)
0.206ms (+0.000ms): mask_and_ack_8259A (do_IRQ) 0.209ms (+0.003ms): generic_redirect_hardirq (do_IRQ) 0.210ms (+0.000ms): generic_handle_IRQ_event (do_IRQ) 0.210ms (+0.000ms): timer_interrupt (generic_handle_IRQ_event) 0.210ms (+0.000ms): mark_offset_tsc (timer_interrupt) 0.217ms (+0.006ms): do_timer (timer_interrupt)
0.217ms (+0.000ms): update_process_times (do_timer) 0.217ms (+0.000ms): update_one_process (update_process_times) 0.217ms (+0.000ms): run_local_timers (update_process_times) 0.218ms (+0.000ms): raise_softirq (update_process_times) 0.218ms (+0.000ms): scheduler_tick (update_process_times) 0.218ms (+0.000ms): sched_clock (scheduler_tick) 0.219ms (+0.001ms): task_timeslice (scheduler_tick) 0.220ms (+0.000ms): update_wall_time (do_timer) 0.220ms (+0.000ms): update_wall_time_one_tick (update_wall_time) 0.221ms (+0.000ms): generic_note_interrupt (do_IRQ) 0.221ms (+0.000ms): end_8259A_irq (do_IRQ)
0.222ms (+0.000ms): enable_8259A_irq (do_IRQ)
0.223ms (+0.001ms): do_softirq (do_IRQ)
0.223ms (+0.000ms): __do_softirq (do_softirq)
0.224ms (+0.000ms): wake_up_process (do_softirq) 0.224ms (+0.000ms): try_to_wake_up (wake_up_process) 0.224ms (+0.000ms): task_rq_lock (try_to_wake_up) 0.225ms (+0.000ms): activate_task (try_to_wake_up) 0.225ms (+0.000ms): sched_clock (activate_task) 0.225ms (+0.000ms): recalc_task_prio (activate_task) 0.226ms (+0.000ms): effective_prio (recalc_task_prio) 0.226ms (+0.000ms): enqueue_task (activate_task) 0.227ms (+0.000ms): preempt_schedule (try_to_wake_up) 0.228ms (+0.000ms): preempt_schedule (copy_page_range)
[...]
0.399ms (+0.000ms): preempt_schedule (copy_page_range) 0.400ms (+0.000ms): check_preempt_timing (touch_preempt_timing)
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P0
Quoted 9 lines · Lee Revell
This was caused by 'Actions -> Run -> rxvt':
0.001ms (+0.000ms):
pte_alloc_map(copy_page_range) 0.205ms (+0.204ms):do_IRQ(common_interrupt)0.228ms (+0.000ms):
preempt_schedule(copy_page_range)0.399ms (+0.000ms):
preempt_schedule(copy_page_range) 0.400ms (+0.000ms):check_preempt_timing(touch_preempt_timing)
seems we need a lock-break in the innermost loop of copy_page_range(). That loop processes up to 1024 pages currently, before the lock-break in the outer loop happens. Large GUI processes are more likely to have full 4MB regions mapped & populated.
i suspect you could trigger a similarly bad latency by doing a fork() in mlockall-test.cc - the attached mlockall-test2.cc does this. Do you get bad latencies?
Ingo
// here is the code i used to test the mlockall caused xruns #include <sys/mman.h>
#include <iostream>
#include <sstream>
#include <unistd.h>
int main (int argc, char *argv[]) {
if (argc < 2) {
std::cout << "how many kbytes you want allocated and mlockall'ed?" << std::endl;
}
std::stringstream stream(argv[1]);
int kbytes;
stream >> kbytes;
char *mem = new char[kbytes*1024];
std::cout << "filling with 0's" << std::endl;
for (int i = 0; i < kbytes*1024; ++i) {
mem[i] = 0;
}
std::cout << "ok, you want " << kbytes << "kb of memory mlocked. going for it.." << std::endl;
int error = mlockall(MCL_CURRENT);
if (error != 0) { std::cout << "mlock error" << std::endl; }
else { std::cout << "mlock successfull" << std::endl;}
fork();
}Re: [patch] voluntary-preempt-2.6.8.1-P0
Quoted 11 lines · Ingo Molnar
* Lee Revell <[email protected]> wrote:
seems we need a lock-break in the innermost loop of
copy_page_range(). That loop processes up to 1024 pages currently, before the lock-break in the outer loop happens. Large GUI processes are more likely to have full 4MB regions mapped & populated.i suspect you could trigger a similarly bad latency by doing a
fork()in mlockall-test.cc - the attached mlockall-test2.cc does this. Do you get bad latencies?
Yes, mlockall-test2 with 10+MB of memory produces a 200us-3ms xrun about half the time.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P0
In reply to Ingo MolnarQuoted 7 lines · Ingo Molnar
Quoted 5 lines · Lee Revell
Quoted 5 lines
Quoted 4 lines
just to check this theory, could you make
__check_and_rekey()an empty function? This should still produce a working random driver, albeit at much reduced entropy. If these latencies have a relationship to themlockall()issue then this change should have an effect.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
hm, could you disable the random driver in the .config rather? It seems that adding to the entropy pool (from hardirq context) alone is quite expensive too.
Can this be disabled in the .config? I can't find an option for it.
oh well, indeed it cannot be disabled.
Hmm, what happens if CONFIG_HW_RANDOM is set then? I would call it a bug if having a hardware RNG didn't disable the software /dev/random driver. This implies that the software RNG can be easily disabled.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P0
In reply to Ingo MolnarQuoted 12 lines · Ingo Molnar
Quoted 10 lines · Lee Revell
Quoted 9 lines
Quoted 4 lines
just to check this theory, could you make
__check_and_rekey()an empty function? This should still produce a working random driver, albeit at much reduced entropy. If these latencies have a relationship to themlockall()issue then this change should have an effect.hm, could you disable the random driver in the .config rather? It seems that adding to the entropy pool (from hardirq context) alone is quite expensive too.
Quoted 6 lines
Quoted 1 line
Can this be disabled in the .config? I can't find an option for it.
oh well, indeed it cannot be disabled. Then i'd suggest to return early from
extract_entropy(), without doing anything. That is the function that seems to introduce the worst overhead.
OK, I tried the quick hack of just returning 0 before taking the spinlock in extract_entropy, but this broke /dev/random which prevented me from logging in. I guess we will have to properly fake it here.
Alas, it looks like the software RNG is not disabled when CONFIG_HW_RANDOM is set. I would call this a bug - imagine incurring the overhead of software 3D acceleration even with hardware 3D acceleration enabled.
Lee
Re: [patch] voluntary-preempt-2.6.8.1-P0
Quoted 8 lines · Lee Revell
Quoted 3 lines
oh well, indeed it cannot be disabled. Then i'd suggest to return early from
extract_entropy(), without doing anything. That is the function that seems to introduce the worst overhead.OK, I tried the quick hack of just returning 0 before taking the spinlock in
extract_entropy, but this broke/dev/randomwhich prevented me from logging in. I guess we will have to properly fake it here.
you should return nbytes, not zero.
Ingo
Re: [patch] voluntary-preempt-2.6.8-rc4-O7
In reply to Ingo MolnarQuoted 2 lines · Ingo Molnar
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8-rc4-O7
Here is the trace for a long ide i/o completion from the hardirq thread (IRQ 15 in my case). These are getting very long, so I modified my latency-graphing PHP script to display any .txt files I put in that directory. All of the traces for a given kernel can be found like so:
http://krustophenia.net/testresults.php?dataset=2.6.8-rc4-bk3-O7
Lee
Re: [patch] voluntary-preempt-2.6.8-rc4-O7
In reply to Ingo MolnarQuoted 4 lines · Ingo Molnar
* Lee Revell <[email protected]> wrote:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8-rc4-O7
Some more interesting results. I am not seeing much of a difference in performance, but the shape of the distribution changes quite a bit:
http://krustophenia.net/testresults.php?dataset=2.6.8-rc3-O5
http://krustophenia.net/testresults.php?dataset=2.6.8-rc4-bk3-O7
Lee
Re: [patch] voluntary-preempt-2.6.8-rc4-O7
Quoted 8 lines · Lee Revell
Some more interesting results. I am not seeing much of a difference in performance, but the shape of the distribution changes quite a bit:
http://krustophenia.net/testresults.php?dataset=2.6.8-rc3-O5
http://krustophenia.net/testresults.php?dataset=2.6.8-rc4-bk3-O7
nice graphs - i suspect the shape difference is due to the tracing overhead: it adds visible (and measurable) overhead to all kernel execution paths, but since the overwhelming majority of paths have very low latencies it doesnt degrade the max_latency results visibly. This is good empirical data - it means what we are after a few offenders who are more than a magnitude slower than the common paths. It also shows that most of the extra overhead is rather in isolated functions (e.g. the memcpy's), not in some big latency path involving many small functions.
i also suspect that if you magnified the 'sharp' nontracing graphs near zero they'd show a similar shape to the tracing graphs.
Ingo
Re: [patch] voluntary-preempt-2.6.8-rc4-O7
In reply to Ingo MolnarQuoted 3 lines · Ingo Molnar
ok, it seems the lock-break of the outer loop was not enough - the up to 1024 iterations in the inner loop can generate quite high latencies too.
In some of the traces, like this one:
there are calls to voluntary_resched. How is this possible? Does it mean that we called voluntary_resched while holding a spinlock, where we needed to call voluntary_preempt_lock(&foo_lock), and thus failed to reschedule?
Lee
Re: [patch] voluntary-preempt-2.6.8-rc4-O7
Quoted 8 lines · Lee Revell
In some of the traces, like this one:
there are calls to
voluntary_resched. How is this possible? Does it mean that we calledvoluntary_reschedwhile holding a spinlock, where we needed to callvoluntary_preempt_lock(&foo_lock), and thus failed to reschedule?
voluntary_resched() was probably called as part of the might_sleep_if() done in mm/slab.c. If you have CONFIG_DEBUG_SPINLOCK_SLEEP enabled then the kernel should have complained about 'Debug: sleeping function ...'.
but what i think happened here is that reiserfs still uses lock_kernel()/unlock_kernel() quite alot (eg. ext3 or xfs doesnt), which from a preemptability POV is just as much of a critical section as a spinlock, but processes can sleep (the scheduler auto-releases and auto-reacquires the big kernel lock).
Ingo
kernel/sched.c renamed from kernel/sched.c.orig +2 -2
@@ -3210,11 +3210,11 @@ __setup("voluntary-preempt=", voluntary_ int __sched voluntary_resched(void) { -if (kernel_preemption || !voluntary_preemption) -return 0 ;#ifdef CONFIG_DEBUG_SPINLOCK_SLEEP __might_sleep (__FILE__, __LINE__);#endif +if (kernel_preemption || !voluntary_preemption) +return 0 ;/* * The system_state check is somewhat ugly but we might be * called during early boot when we are not yet ready to reschedule.
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
In reply to Ingo MolnarQuoted 6 lines · On Fri, 13 Aug 2004 01:51:16 +0200 Ingo Molnar <[email protected]> wrote:
-O6 also adds another timing option besides
preempt_thresh: ifpreempt_threshis set to 0 then the tracer will automatically track the largest-previous latency. (i.e. the system does a search for the absolute maximum latency.) The/proc/sys/kernel/preempt_max_latencycontrol can be used to reset this value to conduct a new search for a new workload, without having to reboot the system.
Hmm, should a new report only be generated when the newly measured latency is really > than all other ones? I get the feeling that >= seems to be enough. Here a dmesg extract:
(swapper/1): new 6 us maximum-latency critical section.
=> started at: <schedule+0x5c/0x5b0>
=> ended at: <finish_task_switch+0x32/0x90>
[<c01064ae>] dump_stack+0x1e/0x20 [<c0130c64>] check_preempt_timing+0x184/0x1e0 [<c0130db4>] sub_preempt_count+0x44/0x50 [<c0112c22>] finish_task_switch+0x32/0x90 [<c0112c9e>] schedule_tail+0x1e/0x60 [<c0105df6>] ret_from_fork+0x6/0x14 (swapper/1): new 8 us maximum-latency critical section. => started at: <voluntary_resched+0x1e/0x50> => ended at: <voluntary_resched+0x1e/0x50> [<c01064ae>] dump_stack+0x1e/0x20 [<c0130c64>] check_preempt_timing+0x184/0x1e0 [<c0130cf8>] touch_preempt_timing+0x38/0x60 [<c0292a9e>] voluntary_resched+0x1e/0x50 [<c013a1b6>] kmem_cache_alloc+0x56/0x60 [<c01160c0>] copy_process+0x9e0/0xbd0 [<c0116302>] do_fork+0x52/0x1b3 [<c0104135>] kernel_thread+0x85/0x90 [<c012a907>] keventd_create_kthread+0x27/0x50 [<c012a9a7>] kthread_create+0x77/0xd0 [<c031dc9b>] cpu_callback+0x5b/0xc0 [<c031dd22>] spawn_ksoftirqd+0x22/0x50 [<c0100334>] init+0x34/0x170 [<c01040a5>] kernel_thread_helper+0x5/0x10 (swapper/1): new 9 us maximum-latency critical section. => started at: <voluntary_resched+0x1e/0x50> => ended at: <voluntary_resched+0x1e/0x50> [<c01064ae>] dump_stack+0x1e/0x20 [<c0130c64>] check_preempt_timing+0x184/0x1e0 [<c0130cf8>] touch_preempt_timing+0x38/0x60 [<c0292a9e>] voluntary_resched+0x1e/0x50 [<c013a1b6>] kmem_cache_alloc+0x56/0x60 [<c0116009>] copy_process+0x929/0xbd0 [<c0116302>] do_fork+0x52/0x1b3 [<c0104135>] kernel_thread+0x85/0x90 [<c012a907>] keventd_create_kthread+0x27/0x50 [<c012a9a7>] kthread_create+0x77/0xd0 [<c031dc9b>] cpu_callback+0x5b/0xc0 [<c031dd22>] spawn_ksoftirqd+0x22/0x50 [<c0100334>] init+0x34/0x170 [<c01040a5>] kernel_thread_helper+0x5/0x10
Here are two reports with the same maximum-latency (31 us):
(swapper/1): new 31 us maximum-latency critical section.
=> started at: <voluntary_resched+0x1e/0x50>
=> ended at: <voluntary_resched+0x1e/0x50>
[<c01064ae>] dump_stack+0x1e/0x20 [<c0130c64>] check_preempt_timing+0x184/0x1e0 [<c0130cf8>] touch_preempt_timing+0x38/0x60 [<c0292a9e>] voluntary_resched+0x1e/0x50 [<c013a3f0>] __kmalloc+0x80/0x90 [<c0316279>] malloc+0x19/0x20 [<c01024d4>] huft_build+0x2d4/0x590 [<c0102ea1>] inflate_fixed+0xc1/0x1a0 [<c010379a>] inflate+0x4a/0xb0 [<c0103b89>] gunzip+0x2f9/0x530 [<c0316e41>] unpack_to_rootfs+0x191/0x220 [<c0316efb>] populate_rootfs+0x2b/0x130 [<c010033e>] init+0x3e/0x170 [<c01040a5>] kernel_thread_helper+0x5/0x10 (swapper/1): new 31 us maximum-latency critical section. => started at: <voluntary_resched+0x1e/0x50> => ended at: <voluntary_resched+0x1e/0x50> [<c01064ae>] dump_stack+0x1e/0x20 [<c0130c64>] check_preempt_timing+0x184/0x1e0 [<c0130cf8>] touch_preempt_timing+0x38/0x60 [<c0292a9e>] voluntary_resched+0x1e/0x50 [<c013a1b6>] kmem_cache_alloc+0x56/0x60 [<c015c8e9>] getname+0x29/0xc0 [<c015f3fd>] sys_mkdir+0x1d/0xf0 [<c03169af>] do_name+0x10f/0x1b0 [<c0316bb1>] write_buffer+0x21/0x30 [<c0316be4>] flush_buffer+0x24/0x60 [<c0316c64>] flush_window+0x24/0x70 [<c01037db>] inflate+0x8b/0xb0 [<c0103b89>] gunzip+0x2f9/0x530 [<c0316e41>] unpack_to_rootfs+0x191/0x220 [<c0316efb>] populate_rootfs+0x2b/0x130 [<c010033e>] init+0x3e/0x170 [<c01040a5>] kernel_thread_helper+0x5/0x10
NET: Registered protocol family 16
Btw: i do have some regular ca. 300 us latencies.. Here are some traces (these happen with an average frequency of ca. 0.3hz):
Aug 13 12:40:49 mango kernel: (ksoftirqd/0/2): 307 us critical section violates 250 us threshold. Aug 13 12:40:49 mango kernel: => started at: <___do_softirq+0x20/0x90> Aug 13 12:40:49 mango kernel: => ended at: <cond_resched_softirq+0x59/0x70> Aug 13 12:40:49 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 13 12:40:49 mango kernel: [<c0130c64>] check_preempt_timing+0x184/0x1e0 Aug 13 12:40:49 mango kernel: [<c0130cf8>] touch_preempt_timing+0x38/0x60 Aug 13 12:40:49 mango kernel: [<c0113df9>] cond_resched_softirq+0x59/0x70 Aug 13 12:40:49 mango kernel: [<c011faef>] run_timer_softirq+0xdf/0x1d0 Aug 13 12:40:49 mango kernel: [<c011adbc>] ___do_softirq+0x7c/0x90 Aug 13 12:40:49 mango kernel: [<c011ae19>] _do_softirq+0x9/0x10 Aug 13 12:40:49 mango kernel: [<c011b1d4>] ksoftirqd+0x84/0xd0 Aug 13 12:40:49 mango kernel: [<c012a8da>] kthread+0xaa/0xb0 Aug 13 12:40:49 mango kernel: [<c01040a5>] kernel_thread_helper+0x5/0x10 Aug 13 12:40:54 mango kernel: (ksoftirqd/0/2): 307 us critical section violates 250 us threshold. Aug 13 12:40:54 mango kernel: => started at: <___do_softirq+0x20/0x90> Aug 13 12:40:54 mango kernel: => ended at: <cond_resched_softirq+0x59/0x70> Aug 13 12:40:54 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 13 12:40:54 mango kernel: [<c0130c64>] check_preempt_timing+0x184/0x1e0 Aug 13 12:40:54 mango kernel: [<c0130cf8>] touch_preempt_timing+0x38/0x60 Aug 13 12:40:54 mango kernel: [<c0113df9>] cond_resched_softirq+0x59/0x70 Aug 13 12:40:54 mango kernel: [<c011faef>] run_timer_softirq+0xdf/0x1d0 Aug 13 12:40:54 mango kernel: [<c011adbc>] ___do_softirq+0x7c/0x90 Aug 13 12:40:54 mango kernel: [<c011ae19>] _do_softirq+0x9/0x10 Aug 13 12:40:54 mango kernel: [<c011b1d4>] ksoftirqd+0x84/0xd0 Aug 13 12:40:54 mango kernel: [<c012a8da>] kthread+0xaa/0xb0 Aug 13 12:40:54 mango kernel: [<c01040a5>] kernel_thread_helper+0x5/0x10 Aug 13 12:40:59 mango kernel: (ksoftirqd/0/2): 308 us critical section violates 250 us threshold. Aug 13 12:40:59 mango kernel: => started at: <___do_softirq+0x20/0x90> Aug 13 12:40:59 mango kernel: => ended at: <cond_resched_softirq+0x59/0x70> Aug 13 12:40:59 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 13 12:40:59 mango kernel: [<c0130c64>] check_preempt_timing+0x184/0x1e0 Aug 13 12:40:59 mango kernel: [<c0130cf8>] touch_preempt_timing+0x38/0x60 Aug 13 12:40:59 mango kernel: [<c0113df9>] cond_resched_softirq+0x59/0x70 Aug 13 12:40:59 mango kernel: [<c011faef>] run_timer_softirq+0xdf/0x1d0 Aug 13 12:40:59 mango kernel: [<c011adbc>] ___do_softirq+0x7c/0x90 Aug 13 12:40:59 mango kernel: [<c011ae19>] _do_softirq+0x9/0x10 Aug 13 12:40:59 mango kernel: [<c011b1d4>] ksoftirqd+0x84/0xd0 Aug 13 12:40:59 mango kernel: [<c012a8da>] kthread+0xaa/0xb0 Aug 13 12:40:59 mango kernel: [<c01040a5>] kernel_thread_helper+0x5/0x10 Aug 13 12:41:04 mango kernel: (ksoftirqd/0/2): 307 us critical section violates 250 us threshold. Aug 13 12:41:04 mango kernel: => started at: <___do_softirq+0x20/0x90> Aug 13 12:41:04 mango kernel: => ended at: <cond_resched_softirq+0x59/0x70> Aug 13 12:41:04 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 13 12:41:04 mango kernel: [<c0130c64>] check_preempt_timing+0x184/0x1e0 Aug 13 12:41:04 mango kernel: [<c0130cf8>] touch_preempt_timing+0x38/0x60 Aug 13 12:41:04 mango kernel: [<c0113df9>] cond_resched_softirq+0x59/0x70 Aug 13 12:41:04 mango kernel: [<c011faef>] run_timer_softirq+0xdf/0x1d0 Aug 13 12:41:04 mango kernel: [<c011adbc>] ___do_softirq+0x7c/0x90 Aug 13 12:41:04 mango kernel: [<c011ae19>] _do_softirq+0x9/0x10 Aug 13 12:41:04 mango kernel: [<c011b1d4>] ksoftirqd+0x84/0xd0 Aug 13 12:41:04 mango kernel: [<c012a8da>] kthread+0xaa/0xb0 Aug 13 12:41:04 mango kernel: [<c01040a5>] kernel_thread_helper+0x5/0x10 Aug 13 12:41:09 mango kernel: (ksoftirqd/0/2): 307 us critical section violates 250 us threshold. Aug 13 12:41:09 mango kernel: => started at: <___do_softirq+0x20/0x90> Aug 13 12:41:09 mango kernel: => ended at: <cond_resched_softirq+0x59/0x70> Aug 13 12:41:09 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 13 12:41:09 mango kernel: [<c0130c64>] check_preempt_timing+0x184/0x1e0 Aug 13 12:41:09 mango kernel: [<c0130cf8>] touch_preempt_timing+0x38/0x60 Aug 13 12:41:09 mango kernel: [<c0113df9>] cond_resched_softirq+0x59/0x70 Aug 13 12:41:09 mango kernel: [<c011faef>] run_timer_softirq+0xdf/0x1d0 Aug 13 12:41:09 mango kernel: [<c011adbc>] ___do_softirq+0x7c/0x90 Aug 13 12:41:09 mango kernel: [<c011ae19>] _do_softirq+0x9/0x10 Aug 13 12:41:09 mango kernel: [<c011b1d4>] ksoftirqd+0x84/0xd0 Aug 13 12:41:09 mango kernel: [<c012a8da>] kthread+0xaa/0xb0 Aug 13 12:41:09 mango kernel: [<c01040a5>]
kernel_thread_helper+0x5/0x10
Some system info:
I do not habe preempt-tracing enabled, only voluntary preempt and preempt timing..
Linux mango.fruits.de 2.6.8-rc4 #1 Fri Aug 13 03:05:04 CEST 2004 i686 GNU/Linux
-----------
processor : 0vendor_id : AuthenticAMD
cpu family : 6
model : 7
model name : AMD Duron(tm) Processor
stepping : 1
cpu MHz : 1195.497
cache size : 64 KBfdiv_bug : nohlt_bug : nof00f_bug : nocoma_bug : no
fpu : yesfpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca
cmov pat pse36 mmx fxsr sse syscall mp mmxext 3dnowext 3dnow
bogomips : 2359.29
-----------/proc/interrupts:
CPU0
0: 1832325 XT-PIC timer 1: 8892 XT-PIC i8042 2: 0 XT-PIC cascade 5: 0 XT-PIC CS46XX 8: 4 XT-PIC rtc 10: 2066 XT-PIC eth0 12: 39786 XT-PIC i8042 14: 900 XT-PIC ide0 15: 12719 XT-PIC ide1 NMI: 0 ERR: 0
-----------/proc/irq/10/eth0/threaded:1/proc/irq/12/i8042/threaded:1/proc/irq/14/ide0/threaded:1/proc/irq/15/ide1/threaded:1/proc/irq/1/i8042/threaded:1/proc/irq/5/CS46XX/threaded:0/proc/irq/8/rtc/threaded:0
-----------/sys/block/hda/queue/max_sectors_kb:16/sys/block/hdd/queue/max_sectors_kb:16
-----------voluntary_preemption
3kernel_preemption
1
-----------preempt_thresh
250
Signature · 2 lines
Palimm Palimm! http://affenbande.org/~tapas/
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
Quoted 4 lines · Florian Schmidt
Hmm, should a new report only be generated when the newly measured latency is really > than all other ones? I get the feeling that >= seems to be enough. Here a dmesg extract:
Quoted 8 lines
(swapper/1): new 6 us maximum-latency critical section. (swapper/1): new 8 us maximum-latency critical section. (swapper/1): new 9 us maximum-latency critical section.
Here are two reports with the same maximum-latency (31 us):
(swapper/1): new 31 us maximum-latency critical section. (swapper/1): new 31 us maximum-latency critical section.
the latency tracer tracks latencies in cycle units - but they are displayed at microsecond accuracy - hence these 'equal' latencies.
to jump-start all those smaller latencies you can do this:
echo 100 > /proc/sys/kernel/preempt_max_latency echo 0 > /proc/sys/kernel/preempt_thresh
this way the maximum-searching only starts at 100 usecs.
Quoted 6 lines
Btw: i do have some regular ca. 300 us latencies.. Here are some traces (these happen with an average frequency of ca. 0.3hz):
(ksoftirqd/0/2): 307 us critical section violates 250 us threshold.
=> started at: <___do_softirq+0x20/0x90>
=> ended at: <cond_resched_softirq+0x59/0x70>
this is too opaque - could you try -O7, enable tracing and save a /proc/latency_trace instance of such a latency? It looks like some sort of softirq latency - perhaps one particular driver's timer fn causes it
- we'll be able to tell more from the trace.
Ingo
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
Quoted 8 lines · On Fri, 13 Aug 2004 12:54:06 +0200 Ingo Molnar <[email protected]> wrote:
Quoted 3 lines
(ksoftirqd/0/2): 307 us critical section violates 250 us threshold.
=> started at: <___do_softirq+0x20/0x90>
=> ended at: <cond_resched_softirq+0x59/0x70>this is too opaque - could you try -O7, enable tracing and save a
/proc/latency_traceinstance of such a latency? It looks like some sort of softirq latency - perhaps one particular driver's timer fn causes it- we'll be able to tell more from the trace.
Hi, this looks like one of them:
mango:~# cat /proc/latency_trace preemption latency trace v1.0 ----------------------------- latency: 308 us, entries: 12 (12) process: ksoftirqd/0/2, uid: 0 nice: -10, policy: 0, rt_priority: 0 =======> 0.000ms (+0.000ms): run_timer_softirq (___do_softirq) 0.000ms (+0.000ms): sis900_timer (run_timer_softirq) 0.001ms (+0.000ms): mdio_read (sis900_timer) 0.002ms (+0.000ms): mdio_reset (mdio_read) 0.071ms (+0.069ms): mdio_idle (mdio_read) 0.151ms (+0.079ms): mdio_read (sis900_timer) 0.151ms (+0.000ms): mdio_reset (mdio_read) 0.220ms (+0.069ms): mdio_idle (mdio_read) 0.300ms (+0.079ms): __mod_timer (sis900_timer) 0.300ms (+0.000ms): internal_add_timer (__mod_timer) 0.300ms (+0.000ms): cond_resched_softirq (run_timer_softirq) 0.301ms (+0.000ms): check_preempt_timing (touch_preempt_timing)
Flo
Signature · 2 lines
Palimm Palimm! http://affenbande.org/~tapas/
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
Quoted 25 lines · Florian Schmidt
Quoted 8 lines · On Fri, 13 Aug 2004 12:54:06 +0200 Ingo Molnar <[email protected]> wrote:
Quoted 3 lines
(ksoftirqd/0/2): 307 us critical section violates 250 us threshold.
=> started at: <___do_softirq+0x20/0x90>
=> ended at: <cond_resched_softirq+0x59/0x70>this is too opaque - could you try -O7, enable tracing and save a
/proc/latency_traceinstance of such a latency? It looks like some sort of softirq latency - perhaps one particular driver's timer fn causes it- we'll be able to tell more from the trace.Hi, this looks like one of them:
mango:~# cat /proc/latency_trace preemption latency trace v1.0 ----------------------------- latency: 308 us, entries: 12 (12) process: ksoftirqd/0/2, uid: 0 nice: -10, policy: 0, rt_priority: 0 =======> 0.000ms (+0.000ms): run_timer_softirq (___do_softirq) 0.000ms (+0.000ms): sis900_timer (run_timer_softirq) 0.001ms (+0.000ms): mdio_read (sis900_timer) 0.002ms (+0.000ms): mdio_reset (mdio_read)
Quoted 8 lines
0.071ms (+0.069ms): mdio_idle (mdio_read) 0.151ms (+0.079ms): mdio_read (sis900_timer) 0.151ms (+0.000ms): mdio_reset (mdio_read) 0.220ms (+0.069ms): mdio_idle (mdio_read) 0.300ms (+0.079ms): __mod_timer (sis900_timer) 0.300ms (+0.000ms): internal_add_timer (__mod_timer) 0.300ms (+0.000ms): cond_resched_softirq (run_timer_softirq) 0.301ms (+0.000ms): check_preempt_timing (touch_preempt_timing)
indeed this seems to be a driver related timer: driver/net/sis900.c's sis900_timer() function. This timer polls your network card for link status once every second. The mdio_read() function does alot of in/out instructions which are quite slow.
could you try the following and disable mdio_delay():
#define mdio_delay() do { } while (0)normally it should work just fine. Worst-case you'd get a non-working network driver. Does this change reduce the latency?
Ingo
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
Quoted 6 lines · Florian Schmidt
This is something i wanted to ask anyways: In one of your first VP announcements you mentioned you wanted to eliminate all latencies > 1 ms. To me it seems, that that goal is pretty much reached [at least i don't see any longer ones except for at boot and shutdown]. So the question is: What is the lower limit for laterncies that you want to hear reports about?
well ... i'm interested in all latencies that are well above the typical average latencies in the system. E.g. when the average is around 20-30 usecs then reports of 200-300 usecs would be interesting.
there's no hard limit, really. Also, sometimes latencies that are 0.3 msec in the report could be 3 msec if triggered properly. So a seemingly lower than 1 msec latency can very well pinpoint a problem area.
Quoted 7 lines
WRT mlockall: i tried mlockall'ing 500 megs. This produced a new max latency of 299 us. the trace is rather long. This one is with jackd running and the one below this is w/o jackd running:
0.010ms (+0.000ms):
free_page_and_swap_cache(clear_page_tables) 0.010ms (+0.000ms):__page_cache_release(clear_page_tables) 0.010ms (+0.000ms):free_hot_page(clear_page_tables)
hm, the reason for this one is that clear_page_tables() does all the freeing in a single uninterrupted critical section covered by mm->page_table_lock.
This function needs a lock-break i believe. Especially in the process-exit case (exit_mmap()) the lock seems unjustified - the current task is the sole owner of a never-to-be-used-again collection of pagetables.
Ingo
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
In reply to Ingo MolnarQuoted 4 lines · Ingo Molnar
i've uploaded the latest version of the voluntary-preempt patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8-rc4-O6
<snip>
Quoted 3 lines
reports, suggestions welcome,
Ingo
I used O7.
I have tested this for a day now, and I have noticed problems:
1)
kernel syslog gets a record like this:
Quoted 12 lines
(default.hotplug/1470): 121 us critical section violates 100 us threshold.
=> started at: <kmap_high+0x2b/0x2d0>
=> ended at: <kmap_high+0x1a9/0x2d0>[<c0105a23>] dump_stack+0x23/0x30 [<c0140d14>] check_preempt_timing+0x184/0x1e0 [<c0140e84>] sub_preempt_count+0x54/0x5d [<c0152959>] kmap_high+0x1a9/0x2d0 [<c017655a>] copy_strings+0xea/0x230 [<c01766db>] copy_strings_kernel+0x3b/0x50 [<c017840d>] do_execve+0x12d/0x1f0 [<c0103284>] sys_execve+0x44/0x80 [<c0104b95>] sysenter_past_esp+0x52/0x71
and the /proc/latency_trace gets:
Quoted 5 lines
preemption latency trace v1.0
-----------------------------
latency: 121 us, entries: 1032 (1032)
process: default.hotplug/1470, uid: 0
nice: -10, policy: 0,rt_priority: 0
Quoted 8 lines
=======>
0.000ms (+0.000ms):page_address(kmap_high) 0.000ms (+0.000ms):page_slot(page_address) 0.000ms (+0.000ms):flush_all_zero_pkmaps(kmap_high) 0.000ms (+0.000ms):set_page_address(flush_all_zero_pkmaps)
[...]
0.118ms (+0.000ms):page_slot(set_page_address) 0.118ms (+0.000ms):check_preempt_timing(sub_preempt_count)
Could the patch be adjusted to make the syslog and the /proc/latency_trace produce the same output?
2)
I suspect that there is a problem with reiserfs, but when I detect a momentary hang in the system(mouse stops moving), no latency_trace appears.
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
Quoted 18 lines · James Courtier-Dutton
I have tested this for a day now, and I have noticed problems:
1)
kernel syslog gets a record like this:Quoted 1 line
(default.hotplug/1470): 121 us critical section violates 100 us
threshold.
and the
/proc/latency_tracegets:Quoted 10 lines
preemption latency trace v1.0
-----------------------------
latency: 121 us, entries: 1032 (1032)
process: default.hotplug/1470, uid: 0
nice: -10, policy: 0,rt_priority: 0
=======>
0.000ms (+0.000ms):page_address(kmap_high) 0.000ms (+0.000ms):page_slot(page_address) 0.000ms (+0.000ms):flush_all_zero_pkmaps(kmap_high) 0.000ms (+0.000ms):set_page_address(flush_all_zero_pkmaps)
Quoted 2 lines
Could the patch be adjusted to make the syslog and the
/proc/latency_traceproduce the same output?
We cannot include the full trace in the syslog - it's possibly thousands of lines long.
Quoted 3 lines
2)
I suspect that there is a problem with reiserfs, but when I detect a momentary hang in the system(mouse stops moving), nolatency_traceappears.
well, the mouse could stop moving for a number of reasons. It's handled via the X server and if the X server is preempted (for whatever reason) then the mouse pointer isnt updated.
you could try to change the mouse IRQ to be non-threaded. Also, do you have kernel_preemption set to 1? It defaults to 0.
Ingo
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
This is just for info, now that we have a nice latency testing tool, we might as well collect some useful traces that we can later work on.
Here is a trace showing a latency of 39034us. http://www.superbug.demon.co.uk/kernel/
The interesting parts of it are:
0.002ms (+0.000ms): __preempt_spin_lock (schedule) 0.523ms (+0.520ms): do_IRQ (common_interrupt)
...
0.532ms (+0.000ms): end_edge_ioapic_irq (do_IRQ)
0.763ms (+0.230ms): smp_apic_timer_interrupt (apic_timer_interrupt)
...
0.768ms (+0.000ms): preempt_schedule (try_to_wake_up) 1.523ms (+0.754ms): do_IRQ (common_interrupt)
...
1.533ms (+0.000ms): __do_softirq (do_softirq)
1.763ms (+0.229ms): smp_apic_timer_interrupt (apic_timer_interrupt)
...
1.765ms (+0.000ms): __do_softirq (do_softirq)
2.523ms (+0.757ms): do_IRQ (common_interrupt)
...
2.533ms (+0.000ms): do_softirq (do_IRQ)
2.533ms (+0.000ms): __do_softirq (do_softirq)
2.763ms (+0.230ms): smp_apic_timer_interrupt (apic_timer_interrupt)
...
This looks to me to be a bug somewhere. Either in the O7 patch, or in the kernel. Surely, do_IRQ should happen quickly, and not take 39ms.
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
Quoted 8 lines · James Courtier-Dutton
This is just for info, now that we have a nice latency testing tool, we might as well collect some useful traces that we can later work on.
Here is a trace showing a latency of 39034us.
http://www.superbug.demon.co.uk/kernel/This looks to me to be a bug somewhere. Either in the O7 patch, or in the kernel. Surely,
do_IRQshould happen quickly, and not take 39ms.
something's wrong indeed. Is this an x86 SMP system? If it's SMP then please apply the patch below (ontop of -O7), it fixes an SMP false positive bug in the latency timing code.
the process is looping somewhere. Here are the non-IRQ trace entries:
0.001ms (+0.000ms): __switch_to (schedule) 0.002ms (+0.000ms): finish_task_switch (schedule) 0.002ms (+0.000ms): __preempt_spin_lock (schedule)
... [lots of IRQs] ...
38.126ms (+0.362ms): preempt_schedule (schedule) 38.126ms (+0.000ms): sched_clock (schedule) 38.127ms (+0.000ms): find_next_bit (schedule) 38.127ms (+0.000ms): task_timeslice (schedule)
this shows that we are looping in __preempt_spin_lock() - most likely via schedule()'s reqacquire_kernel_lock() code.
i.e. another CPU is holding the big kernel lock and this CPU is looping. _but_ this CPU is fully preemptible so the trace produces this false positive.
Ingo
kernel/sched.c renamed from kernel/sched.c.orig2 +4 -0
@@ -4210,7 +4210,9 @@ void __sched __preempt_spin_lock(spinloc do { preempt_enable(); while (spin_is_locked(lock)) { + preempt_disable(); touch_preempt_timing(); + preempt_enable(); cpu_relax(); } preempt_disable();
@@ -4229,7 +4231,9 @@ void __sched __preempt_write_lock(rwlock do { preempt_enable(); while (rwlock_is_locked(lock)) { + preempt_disable(); touch_preempt_timing(); + preempt_enable(); cpu_relax(); } preempt_disable();
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
Quoted 28 lines · Ingo Molnar
Quoted 8 lines · James Courtier-Dutton
This is just for info, now that we have a nice latency testing tool, we might as well collect some useful traces that we can later work on.
Here is a trace showing a latency of 39034us. http://www.superbug.demon.co.uk/kernel/
This looks to me to be a bug somewhere. Either in the O7 patch, or in the kernel. Surely,
do_IRQshould happen quickly, and not take 39ms.something's wrong indeed. Is this an x86 SMP system? If it's SMP then please apply the patch below (ontop of -O7), it fixes an SMP false positive bug in the latency timing code.
the process is looping somewhere. Here are the non-IRQ trace entries:
0.001ms (+0.000ms):
__switch_to(schedule) 0.002ms (+0.000ms):finish_task_switch(schedule) 0.002ms (+0.000ms):__preempt_spin_lock(schedule)
... [lots of IRQs] ...
38.126ms (+0.362ms):preempt_schedule(schedule) 38.126ms (+0.000ms):sched_clock(schedule) 38.127ms (+0.000ms):find_next_bit(schedule) 38.127ms (+0.000ms):task_timeslice(schedule)
Quoted 8 lines
this shows that we are looping in
__preempt_spin_lock()- most likely via schedule()'sreqacquire_kernel_lock()code.i.e. another CPU is holding the big kernel lock and this CPU is looping. _but_ this CPU is fully preemptible so the trace produces this false positive.
Ingo
I have not seen that particular problem any more, so I think the patch helped.
I have found a new problem though:
# cat latency_trace preemption latency trace v1.0 ----------------------------- latency: 1883455195 us, entries: 1 (1) process: ksoftirqd/1/5, uid: 0 nice: -10, policy: 0, rt_priority: 0 =======> 0.000ms (+0.000ms): cond_resched_softirq (___do_softirq)
That looks bad to me. The user did not notice any latency, but 1883 seconds seems like a high latency to me!
James
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
Quoted 11 lines · James Courtier-Dutton
I have found a new problem though:
# cat latency_trace preemption latency trace v1.0 ----------------------------- latency: 1883455195 us, entries: 1 (1) process: ksoftirqd/1/5, uid: 0 nice: -10, policy: 0, rt_priority: 0 =======> 0.000ms (+0.000ms): cond_resched_softirq (___do_softirq)
Quoted 2 lines
That looks bad to me. The user did not notice any latency, but 1883 seconds seems like a high latency to me!
yeah, it doesnt look healthy. This could be some sort of tracing underflow - what is the 'grep MHz /proc/cpuinfo' value of your box?
Also, could you try the -P4 patch, it has a more robust timestamping mechanism.
Ingo
Re: [patch] Latency Tracer, voluntary-preempt-2.6.8-rc4-O6
In reply to James Courtier-DuttonQuoted 2 lines · James Courtier-Dutton
Could the patch be adjusted to make the syslog and the
/proc/latency_traceproduce the same output?
to further unify the formats i've added the start/end entries to latency_trace in -P4. Saving the stacktrace would be too much trouble though.
Ingo
Re: [patch] voluntary-preempt-2.6.8-rc3-O4
In reply to Ingo MolnarQuoted 5 lines · On Mon, 9 Aug 2004 12:46:49 +0200 Ingo Molnar <[email protected]> wrote:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8-rc3-O4
(it should apply to -rc2 cleanly too.)
-O4 fixes a two more latency sources:
Hi, i also wanted to report that i also still see these mysterious (at least to me) "(swapper/0)" critical section reports (i use O4 with the preempt-timing-on-2.6.8-rc3-O4.patch): These are basically the only xruns/critical section reports i see when using jackd at 2*64 and with some clients connected. I'm not sure if i see them when jackd is not running.. Will test later this evening.
Flo
Aug 10 16:05:50 mango kernel: (swapper/0): 2463us non-preemptible critical section violated 1000 us preempt threshold starting at do_IRQ+0x19/0x170 and ending at do_IRQ+0x126/0x170 Aug 10 16:05:50 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 10 16:05:50 mango kernel: [<c0114735>] sub_preempt_count+0x45/0x60 Aug 10 16:05:50 mango kernel: [<c0107936>] do_IRQ+0x126/0x170 Aug 10 16:05:50 mango kernel: [<c0106098>] common_interrupt+0x18/0x20 Aug 10 16:05:50 mango kernel: [<c0103eb9>] cpu_idle+0x39/0x50 Aug 10 16:05:50 mango kernel: [<c03148c1>] start_kernel+0x1a1/0x1f0 Aug 10 16:05:50 mango kernel: [<c010019f>] 0xc010019f Aug 10 16:05:50 mango kernel: ALSA sound/core/pcm_lib.c:169: XRUN: pcmC0D0p Aug 10 16:05:50 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 10 16:05:50 mango kernel: [<f08c2431>] snd_pcm_period_elapsed+0x311/0x480 [snd_pcm] Aug 10 16:05:50 mango kernel: [<f089ebce>] snd_cs46xx_interrupt+0x1be/0x1f0 [snd_cs46xx] Aug 10 16:05:50 mango kernel: [<c011b55b>] generic_handle_IRQ_event+0x3b/0x70 Aug 10 16:05:50 mango kernel: [<c01078c6>] do_IRQ+0xb6/0x170 Aug 10 16:05:50 mango kernel: [<c0106098>] common_interrupt+0x18/0x20 Aug 10 16:07:11 mango kernel: ALSA sound/core/pcm_lib.c:169: XRUN: pcmC0D0c Aug 10 16:07:11 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 10 16:07:11 mango kernel: [<f08c2431>] snd_pcm_period_elapsed+0x311/0x480 [snd_pcm] Aug 10 16:07:11 mango kernel: [<f089ebce>] snd_cs46xx_interrupt+0x1be/0x1f0 [snd_cs46xx] Aug 10 16:07:11 mango kernel: [<c011b55b>] generic_handle_IRQ_event+0x3b/0x70 Aug 10 16:07:11 mango kernel: [<c01078c6>] do_IRQ+0xb6/0x170 Aug 10 16:07:11 mango kernel: [<c0106098>] common_interrupt+0x18/0x20 Aug 10 16:07:11 mango kernel: [<c0103eb9>] cpu_idle+0x39/0x50 Aug 10 16:07:11 mango kernel: [<c03148c1>] start_kernel+0x1a1/0x1f0 Aug 10 16:07:11 mango kernel: [<c010019f>] 0xc010019f Aug 10 16:07:11 mango kernel: (swapper/0): 2462us non-preemptible critical section violated 1000 us preempt threshold starting at do_IRQ+0x19/0x170 and ending at do_IRQ+0x126/0x170 Aug 10 16:07:11 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 10 16:07:11 mango kernel: [<c0114735>] sub_preempt_count+0x45/0x60 Aug 10 16:07:11 mango kernel: [<c0107936>] do_IRQ+0x126/0x170 Aug 10 16:07:11 mango kernel: [<c0106098>] common_interrupt+0x18/0x20 Aug 10 16:07:11 mango kernel: [<c0103eb9>] cpu_idle+0x39/0x50 Aug 10 16:07:11 mango kernel: [<c03148c1>] start_kernel+0x1a1/0x1f0 Aug 10 16:07:11 mango kernel: [<c010019f>] 0xc010019f Aug 10 16:07:11 mango kernel: ALSA sound/core/pcm_lib.c:169: XRUN: pcmC0D0p Aug 10 16:07:11 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 10 16:07:11 mango kernel: [<f08c2431>] snd_pcm_period_elapsed+0x311/0x480 [snd_pcm] Aug 10 16:07:11 mango kernel: [<f089ebce>] snd_cs46xx_interrupt+0x1be/0x1f0 [snd_cs46xx] Aug 10 16:07:11 mango kernel: [<c011b55b>] generic_handle_IRQ_event+0x3b/0x70 Aug 10 16:07:11 mango kernel: [<c01078c6>] do_IRQ+0xb6/0x170 Aug 10 16:07:11 mango kernel: [<c0106098>] common_interrupt+0x18/0x20 Aug 10 16:09:52 mango kernel: ALSA sound/core/pcm_lib.c:169: XRUN: pcmC0D0p Aug 10 16:09:52 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 10 16:09:52 mango kernel: [<f08c2431>] snd_pcm_period_elapsed+0x311/0x480 [snd_pcm] Aug 10 16:09:52 mango kernel: [<f089ebce>] snd_cs46xx_interrupt+0x1be/0x1f0 [snd_cs46xx] Aug 10 16:09:52 mango kernel: [<c011b55b>] generic_handle_IRQ_event+0x3b/0x70 Aug 10 16:09:52 mango kernel: [<c01078c6>] do_IRQ+0xb6/0x170 Aug 10 16:09:52 mango kernel: [<c0106098>] common_interrupt+0x18/0x20 Aug 10 16:09:52 mango kernel: [<c0103eb9>] cpu_idle+0x39/0x50 Aug 10 16:09:52 mango kernel: [<c03148c1>] start_kernel+0x1a1/0x1f0 Aug 10 16:09:52 mango kernel: [<c010019f>] 0xc010019f Aug 10 16:09:52 mango kernel: (swapper/0): 2459us non-preemptible critical section violated 1000 us preempt threshold starting at do_IRQ+0x19/0x170 and ending at do_IRQ+0x126/0x170 Aug 10 16:09:52 mango kernel: [<c01064ae>] dump_stack+0x1e/0x20 Aug 10 16:09:52 mango kernel: [<c0114735>] sub_preempt_count+0x45/0x60 Aug 10 16:09:52 mango kernel: [<c0107936>] do_IRQ+0x126/0x170 Aug 10 16:09:52 mango kernel: [<c0106098>] common_interrupt+0x18/0x20 Aug 10 16:09:52 mango kernel: [<c0103eb9>] cpu_idle+0x39/0x50 Aug 10 16:09:52 mango kernel: [<c03148c1>] start_kernel+0x1a1/0x1f0 Aug 10 16:09:52 mango kernel: [<c010019f>] 0xc010019f
System info:
Linux mango.fruits.de 2.6.8-rc3 #2 Tue Aug 10 12:46:08 CEST 2004 i686 GNU/Linux
-----------
processor : 0vendor_id : AuthenticAMD
cpu family : 6
model : 7
model name : AMD Duron(tm) Processor
stepping : 1
cpu MHz : 1194.993
cache size : 64 KBfdiv_bug : nohlt_bug : nof00f_bug : nocoma_bug : no
fpu : yesfpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov pat
pse36 mmx fxsr sse syscall mp mmxext 3dnowext 3dnow
bogomips : 2359.29
-----------/proc/interrupts:
CPU0
0: 8341883 XT-PIC timer 1: 11031 XT-PIC i8042 2: 0 XT-PIC cascade 5: 10546999 XT-PIC CS46XX 8: 4 XT-PIC rtc 10: 17097 XT-PIC eth0 12: 292978 XT-PIC i8042 14: 2675 XT-PIC ide0 15: 43327 XT-PIC ide1 NMI: 0 ERR: 0
-----------/proc/irq/10/eth0/threaded:1/proc/irq/12/i8042/threaded:1/proc/irq/14/ide0/threaded:1/proc/irq/15/ide1/threaded:1/proc/irq/1/i8042/threaded:1/proc/irq/5/CS46XX/threaded:0/proc/irq/8/rtc/threaded:0
-----------/sys/block/hda/queue/max_sectors_kb:16/sys/block/hdd/queue/max_sectors_kb:16
-----------voluntary_preemption
3kernel_preemption
1
-----------preempt_thresh
1000
Signature · 2 lines
Palimm Palimm! http://affenbande.org/~tapas/
PATCH 1/1 voluntary-preempt-2.6.8-rc2-J4
In reply to Ingo Molnari've uploaded the -J4 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8-rc2-J4
Changes since -J3:
- make block device max_sectors sysfs tunable. There's a new entry /sys/block/*/queue/max_sectors_kb which stores the current max request size in KB. You can write it to change the size.
Jens: i've attached a standalone patch against 2.6.8-rc2 below. Please apply if it looks good to you. (I've added extra locking to make sure max_sectors and readahead_pages gets updated at once, for the unlikely event of two CPUs updating max_sectors at once.) I've tested this on IDE using the UP kernel.
- include the unlock-tty changes suggested by akpm and tested by Lee Revell. I've also unlocked the tty_read path - only the release/open paths remain under the big kernel lock. Please report any tty related regressions.
Ingo
--
introduce max_sectors_kb tunable.
drivers/block/ll_rw_blk.c renamed from drivers/block/ll_rw_blk.c.orig +44 -0
@@ -3062,13 +3063,50 @@ queue_ra_store(struct request_queue *q,unsigned long ra_kb;ssize_t ret =queue_var_store (&ra_kb, page, count); +spin_lock_irq (q->queue_lock);if (ra_kb > (q->max_sectors >>1 )) ra_kb = (q->max_sectors >>1 ); q->backing_dev_info.ra_pages = ra_kb >> (PAGE_CACHE_SHIFT -10 ); +spin_unlock_irq (q->queue_lock); + +return ret; +} + +static ssize_t queue_max_sectors_show (struct request_queue *q,char *page) +{ +int max_sectors_kb = q->max_sectors >>1 ; + +return queue_var_show (max_sectors_kb, (page)); +} + +static ssize_t +queue_max_sectors_store (struct request_queue *q,const char *page,size_t count) +{ +unsigned long max_sectors_kb; +ssize_t ret =queue_var_store (&max_sectors_kb, page, count); +int ra_kb; + +/* +* Take the queue lock to update the readahead and max_sectors +* values synchronously: +*/ +spin_lock_irq (q->queue_lock); +/* +* Trim readahead window as well, if necessary: +*/ + ra_kb = q->backing_dev_info.ra_pages << (PAGE_CACHE_SHIFT -10 ); +if (ra_kb > max_sectors_kb) + q->backing_dev_info.ra_pages = + max_sectors_kb >> (PAGE_CACHE_SHIFT -10 ); + + q->max_sectors = max_sectors_kb <<1 ; +spin_unlock_irq (q->queue_lock); +return ret; } +static struct queue_sysfs_entry queue_requests_entry = { .attr = {.name ="nr_requests" , .mode =S_IRUGO |S_IWUSR }, .show = queue_requests_show,
@@ -3081,9 +3119,16 @@ static struct queue_sysfs_entry queue_ra .store = queue_ra_store, }; +static struct queue_sysfs_entry queue_max_sectors_entry = { + .attr = {.name ="max_sectors_kb" , .mode =S_IRUGO |S_IWUSR }, + .show = queue_max_sectors_show, + .store = queue_max_sectors_store, +}; +static struct attribute *default_attrs[] = { &queue_requests_entry.attr, &queue_ra_entry.attr, + &queue_max_sectors_entry.attr,NULL , };
Re: [patch] voluntary-preempt-2.6.8-rc2-J4
Quoted 7 lines
Changes since -J3:
- make block device
max_sectorssysfs tunable. There's a new entry/sys/block/*/queue/max_sectors_kbwhich stores the current max request size in KB. You can write it to change the size.
Quoted 5 lines
Jens: i've attached a standalone patch against 2.6.8-rc2 below. Please apply if it looks good to you. (I've added extra locking to make sure
max_sectorsandreadahead_pagesgets updated at once, for the unlikely event of two CPUs updatingmax_sectorsat once.) I've tested this on IDE using the UP kernel.
i've refined the patch (new version attached below): drivers use blk_queue_max_sectors() to set the maximum # of sectors that the driver or hw can handle.
so i've introduced a new queue entry called max_hw_sectors, and the new /sys entry listens to this maximum and only updates max_sectors. This entry is also exported to /sys as a readonly entry. (so that users can see the maximum the driver supports.)
i've added another fix as well: do not allow users to set a max_sectors value lower than the pagecache page size.
these changes should make the new tunable pretty fool-proof.
Ingo
include/linux/blkdev.h renamed from include/linux/blkdev.h.orig +1 -0
@@ -338,6 +338,7 @@ struct request_queueunsigned int nr_congestion_off;unsigned short max_sectors; +unsigned short max_hw_sectors;unsigned short max_phys_segments;unsigned short max_hw_segments;unsigned short hardsect_size;
drivers/block/ll_rw_blk.c renamed from drivers/block/ll_rw_blk.c.orig +62 -1
91 lines not on this page · raw message
no luck with max_sectors_kb (Re: voluntary-preempt-2.6.8-rc2-J4)
Quoted 8 lines
Quoted 7 lines
Changes since -J3:
- make block device
max_sectorssysfs tunable. There's a new entry/sys/block/*/queue/max_sectors_kbwhich stores the current max request size in KB. You can write it to change the size.[...]
Quoted 8 lines
i've refined the patch (new version attached below): drivers use
blk_queue_max_sectors()to set the maximum # of sectors that the driver or hw can handle.so i've introduced a new queue entry called
max_hw_sectors, and the new /sys entry listens to this maximum and only updatesmax_sectors. This entry is also exported to /sys as a readonly entry. (so that users can see the maximum the driver supports.)
Hi there.
I do not seem to have success with tuning the max_sectors_kb value.
After setting it to 32 (the hw max is 128), userland programs fail with I/O errors. Setting it back to 128 gets it back to working, sort of. The errors probably get bufferred somewhere.
During the "bad" setting (32), messages like this one show up in kernel log.
bio too big device hda3 (104 > 64)
I am using J7 voluntary_preemption patch (set at 2:1), cfq io scheduler, via82cxxx IDE driver. I will gladly provide further details.
Rudo.
Re: no luck with max_sectors_kb (Re: voluntary-preempt-2.6.8-rc2-J4)
Quoted 7 lines · Rudo Thomas
After setting it to 32 (the hw max is 128), userland programs fail with I/O errors. Setting it back to 128 gets it back to working, sort of. The errors probably get bufferred somewhere.
During the "bad" setting (32), messages like this one show up in kernel log.
bio too big device hda3 (104 > 64)
does the patch below, ontop of -J7, help?
Ingo
block/ll_rw_blk.c renamed from block/ll_rw_blk.c.orig +2 -2
@@ -2447,11 +2447,11 @@ end_io:break ; } -if (unlikely (bio_sectors (bio) > q->max_sectors)) { +if (unlikely (bio_sectors (bio) > q->max_hw_sectors)) {printk ("bio too big device %s (%u > %u)\n" ,bdevname (bio->bi_bdev, b),bio_sectors (bio), - q->max_sectors); + q->max_hw_sectors); goto end_io; }
Re: no luck with max_sectors_kb (Re: voluntary-preempt-2.6.8-rc2-J4)
Quoted 1 line · Ingo Molnar
does the patch below, ontop of -J7, help?
i've added this patch to -J7, so re-downloading the -J7 patch should give you the patch too.
Ingo
Re: no luck with max_sectors_kb (Re: voluntary-preempt-2.6.8-rc2-J4)
In reply to Ingo MolnarQuoted 9 lines
Quoted 7 lines
After setting it to 32 (the hw max is 128), userland programs fail with I/O errors. Setting it back to 128 gets it back to working, sort of. The errors probably get bufferred somewhere.
During the "bad" setting (32), messages like this one show up in kernel log.
bio too big device hda3 (104 > 64)
does the patch below, ontop of -J7, help?
I tried it, but did not test tuning the value, as the patch has problems of its
own: with device-mapper.
bio too big device dm-0 (256 > 255)
Again, some files cannot be read on the device.
Bye for now.
Rudo.
Re: no luck with max_sectors_kb (Re: voluntary-preempt-2.6.8-rc2-J4)
Quoted 8 lines · Rudo Thomas
Quoted 1 line
does the patch below, ontop of -J7, help?
I tried it, but did not test tuning the value, as the patch has problems of its own: with device-mapper.
bio too big device dm-0 (256 > 255)
Again, some files cannot be read on the device.
ok, dm (and some other layered block drivers) set q->max_sectors directly instead of using blk_queue_max_sectors().
Ingo
Re: no luck with max_sectors_kb (Re: voluntary-preempt-2.6.8-rc2-J4)
Quoted 2 lines · Ingo Molnar
ok, dm (and some other layered block drivers) set
q->max_sectorsdirectly instead of usingblk_queue_max_sectors().
does the patch below fix your DM problems?
Ingo
drivers/md/dm-table.c renamed from drivers/md/dm-table.c.orig +1 -1
@@ -825,7 +825,7 @@ void dm_table_set_restrictions(struct dm* Make sure we obey the optimistic sub devices * restrictions. */ - q->max_sectors = t->limits.max_sectors; +blk_queue_max_sectors (q, t->limits.max_sectors); q->max_phys_segments = t->limits.max_phys_segments; q->max_hw_segments = t->limits.max_hw_segments; q->hardsect_size = t->limits.hardsect_size;
Re: no luck with max_sectors_kb (Re: voluntary-preempt-2.6.8-rc2-J4)
Quoted 4 lines
Quoted 2 lines
ok, dm (and some other layered block drivers) set
q->max_sectorsdirectly instead of usingblk_queue_max_sectors().does the patch below fix your DM problems?
Indeed it does.
Rudo.
Re: [patch] voluntary-preempt-2.6.8-rc2-J4
In reply to Ingo MolnarQuoted 88 lines · Ingo Molnar
i've uploaded the -J4 patch:
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8-rc2-J4Changes since -J3:
- make block device
max_sectorssysfs tunable. There's a new entry/sys/block/*/queue/max_sectors_kbwhich stores the current max request size in KB. You can write it to change the size.Jens: i've attached a standalone patch against 2.6.8-rc2 below. Please apply if it looks good to you. (I've added extra locking to make sure
max_sectorsandreadahead_pagesgets updated at once, for the unlikely event of two CPUs updatingmax_sectorsat once.) I've tested this on IDE using the UP kernel.- include the unlock-tty changes suggested by akpm and tested by Lee Revell. I've also unlocked the
tty_readpath - only the release/open paths remain under the big kernel lock. Please report any tty related regressions.Ingo
--
introducemax_sectors_kbtunable.Signed-off-by Ingo Molnar <[email protected]>drivers/block/ll_rw_blk.c renamed from drivers/block/ll_rw_blk.c.orig +39
-0@@ -3062,13 +3063,50 @@ queue_ra_store(struct request_queue *q,unsigned long ra_kb;ssize_t ret =queue_var_store (&ra_kb, page, count); +spin_lock_irq (q->queue_lock);if (ra_kb > (q->max_sectors >>1 )) ra_kb = (q->max_sectors >>1 ); q->backing_dev_info.ra_pages = ra_kb >> (PAGE_CACHE_SHIFT -10 ); +spin_unlock_irq (q->queue_lock); + +return ret; +} + +static ssize_t queue_max_sectors_show (struct request_queue *q,char *page) +{ +int max_sectors_kb = q->max_sectors >>1 ; + +return queue_var_show (max_sectors_kb, (page)); +} + +static ssize_t +queue_max_sectors_store (struct request_queue *q,const char *page,size_t count) +{ +unsigned long max_sectors_kb; +ssize_t ret =queue_var_store (&max_sectors_kb, page, count); +int ra_kb; + +/* +* Take the queue lock to update the readahead and max_sectors +* values synchronously: +*/ +spin_lock_irq (q->queue_lock); +/* +* Trim readahead window as well, if necessary: +*/ + ra_kb = q->backing_dev_info.ra_pages << (PAGE_CACHE_SHIFT -10 ); +if (ra_kb > max_sectors_kb) + q->backing_dev_info.ra_pages = + max_sectors_kb >> (PAGE_CACHE_SHIFT -10 ); + + q->max_sectors = max_sectors_kb <<1 ; +spin_unlock_irq (q->queue_lock); +return ret; } +static struct queue_sysfs_entry queue_requests_entry = { .attr = {.name ="nr_requests" , .mode =S_IRUGO |S_IWUSR }, .show = queue_requests_show,@@ -3081,9 +3119,16 @@ static struct queue_sysfs_entry queue_ra .store = queue_ra_store, }; +static struct queue_sysfs_entry queue_max_sectors_entry = { + .attr = {.name ="max_sectors_kb" , .mode =S_IRUGO |S_IWUSR },
drivers/block/ll_rw_blk.c static struct queue_sysfs_entry queue_ra · line 3133
Quoted 10 lines
drivers/block/ll_rw_blk.c renamed from drivers/block/ll_rw_blk.c.orig +5
-0+ .show = queue_max_sectors_show, + .store = queue_max_sectors_store, +}; +static struct attribute *default_attrs[] = { &queue_requests_entry.attr, &queue_ra_entry.attr, + &queue_max_sectors_entry.attr,NULL , };
I don't like it. First of all, the implementation really should drain the queue first, then set max value before allowing people to queue more io. The queue lock doesn't help here, readers don't even attempt to serialize access to max_sectors.
Secondly, I don't like the concept of exposing this value. If you want to do something like this, we must split the value into two like proposed (and patched) some months ago into a hardware and user value.
I don't see why we can't just drop ata48 default value to 256kb instead. There's very little command over head on ide, I bet the majority of the change in performance when playing with 256kb vs 1024kb is not the command overhead itself, rather things like read-ahead that could be more intelligent.
Signature · 1 line
Jens Axboe
Re: [patch] voluntary-preempt-2.6.8-rc2-J4
[i've sent a second patch too since the first version.]
Quoted 4 lines · Jens Axboe
I don't like it. First of all, the implementation really should drain the queue first, then set max value before allowing people to queue more io. The queue lock doesn't help here, readers don't even attempt to serialize access to
max_sectors.
why should the queue be drained? There might be a few leftover big requests, but these are not a problem.
Quoted 3 lines
Secondly, I don't like the concept of exposing this value. If you want to do something like this, we must split the value into two like proposed (and patched) some months ago into a hardware and user value.
yes, agreed - that's what the second patch does.
Quoted 5 lines
I don't see why we can't just drop ata48 default value to 256kb instead. There's very little command over head on ide, I bet the majority of the change in performance when playing with 256kb vs 1024kb is not the command overhead itself, rather things like read-ahead that could be more intelligent.
256kb isnt enough from a latency POV either - and if a user wants some extreme setting like 16KB per request why not allow it? Especially since these tunables cause zero runtime overhead.
Ingo
Re: [patch] voluntary-preempt-2.6.8-rc2-J4
Quoted 1 line · Ingo Molnar
[i've sent a second patch too since the first version.]
and here's a third one:
http://redhat.com/~mingo/voluntary-preempt/tune-max-sectors-2.6.8-rc2-A0
this includes the dm-table.c fix and also fixes other stacked drivers and updates blk_queue_stack_limits() to change max_hw_sectors too.
this patch should be pretty complete i believe.
Ingo
Re: [patch] voluntary-preempt-2.6.8-rc2-J4
In reply to Ingo MolnarQuoted 5 lines · Ingo Molnar
[i've sent a second patch too since the first version.]
* Jens Axboe <[email protected]> wrote:
Quoted 7 lines
Quoted 4 lines
I don't like it. First of all, the implementation really should drain the queue first, then set max value before allowing people to queue more io. The queue lock doesn't help here, readers don't even attempt to serialize access to
max_sectors.why should the queue be drained? There might be a few leftover big requests, but these are not a problem.
The patch I commented on did not split sectors value into two, in that case it's a big problem. When split and expose only the one, it's not an issue.
Quoted 5 lines
Quoted 3 lines
Secondly, I don't like the concept of exposing this value. If you want to do something like this, we must split the value into two like proposed (and patched) some months ago into a hardware and user value.
yes, agreed - that's what the second patch does.
Great.
Quoted 9 lines
Quoted 5 lines
I don't see why we can't just drop ata48 default value to 256kb instead. There's very little command over head on ide, I bet the majority of the change in performance when playing with 256kb vs 1024kb is not the command overhead itself, rather things like read-ahead that could be more intelligent.
256kb isnt enough from a latency POV either - and if a user wants some extreme setting like 16KB per request why not allow it? Especially since these tunables cause zero runtime overhead.
Note that I've been travelling and didn't really track the thread closely before that, but it seems to me that most of the latency incurred is an artifact of long code paths hanging off bi_end_io. Maybe it would be better to try and fix some of that and get both good irq completion latencies, while still maintaining decent drive throughput.
That said, this second version looks better from an inclusion point of view. Better fix those manual q->max_sectors settings first.
Signature · 1 line
Jens Axboe
Re: [patch] voluntary-preempt-2.6.8-rc2-J4
Quoted 2 lines · Jens Axboe
That said, this second version looks better from an inclusion point of view. Better fix those manual
q->max_sectorssettings first.
ok - the third patch i sent is supposed to deal with all those cases as well.
Ingo
Re: [patch] voluntary-preempt-2.6.8-rc2-J3
In reply to Ingo MolnarQuoted 2 lines · Ingo Molnar
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.8-rc2-J3
The bigger this thing gets, the more worried I get. Sometime this is going to need to be split up into individual fixes, and they need to be based upon an overall approach which we haven't yet settled on.
In particular your whole approach (with voluntary_need_resched()) doesn't work on SMP.
The approach I'm using is to unconditionally drop locks on every Nth pass around the loop to allow another CPU to grab the lock, do some work, drop the lock, then be preempted. eg:
@@ -773,6 +774,12 @@ int get_user_pages(struct task_struct *t
struct page *map = NULL; int lookup_write = write;
+ if ((++nr_pages & 63) == 0) {
+ spin_unlock(&mm->page_table_lock);
+ cpu_relax();
+ spin_lock(&mm->page_table_lock);
+ }
+
/*
* We don't follow pagetables for VM_IO regions - they
This is a bit nasty, because it assumes that the other CPU will be able to get in there and acquire the lock which is in a different cpu's cache. Long transfer latencies may defeat this.
For voluntary preempt the above will need a cond_resched added to it.
So we need to come up with a suitable approach to covering voluntary and involuntary preemption on both UP and SMP and set up the header file infrastructure to support it all. After that we need to redo all the points-of-high-latency whcih have thus far been discovered to use that infrastructure.
Re: [patch] voluntary-preempt-2.6.8-rc2-J3
Quoted 3 lines · Andrew Morton
The bigger this thing gets, the more worried I get. Sometime this is going to need to be split up into individual fixes, and they need to be based upon an overall approach which we haven't yet settled on.
i will do that splitup. Right now i'm simply mapping how widespread the problem is and what type of fixes we need. The situation isnt all that bad but we might need (an optional) mechanism to make softirqs synchronous. All of this stuff is nicely modular and i'll do a splitup post 2.6.8 (i dont think we want to disturb 2.6.8 with any of this).
Quoted 2 lines
In particular your whole approach (with
voluntary_need_resched()) doesn't work on SMP.
(for the record, voluntary_need_resched() == need_resched() - i'm only keeping a distinction between the two to be able to track progress and regressions between the vanilla and modified kernels.)
need_resched() indeed doesnt do a lock-break for SMP purposes.
Quoted 5 lines
The approach I'm using is to unconditionally drop locks on every Nth pass around the loop to allow another CPU to grab the lock, do some work, drop the lock, then be preempted. eg:
Review int get_user_pages(struct task_struct *t · line 782
Quoted 9 lines
(quoted diff) +6
-0@@ -773,6 +774,12 @@ int get_user_pages(struct task_struct *t struct page *map = NULL; int lookup_write = write; + if ((++nr_pages & 63) == 0) { + spin_unlock(&mm->page_table_lock); + cpu_relax(); + spin_lock(&mm->page_table_lock); + } +
guaranteeing latencies on SMP is hard, because the latency of a CPU might depend on the latency of a task on another CPU - which CPU isnt notified of the rescheduling request.
one alternative technique to yours would be to notify _all_ CPUs that a high-prio RT task is ready to run (via a broadcast need-resched). That way the UP latency-break techniques map to SMP in a 1:1 way.
non-RT tasks dont get this benefit, which is a difference to the UP situation, but i dont think it would be appropriate to use the UP behavior, due to the overhead of broadcasting.
a combination of the two techniques could be used too: a global 'break locks from now on' flag which gets set if a (RT?) task wants to reschedule. Normally this flag would be zero and the cacheline would be clean and shared between all CPUs, causing no overhead. Once a task wants to reschedule it would increase by 1 until that task has been scheduled. This would remove the ugliness factor too, your sample code would look:
Review int get_user_pages(struct task_struct *t
Quoted 6 lines
(quoted diff) +6
-0+ if (need_lock_break()) { + spin_unlock(&mm->page_table_lock); + cpu_relax(); + spin_lock(&mm->page_table_lock); + } +
or a shortcut:
Review
Quoted 1 line
(quoted diff) +1
-0+ cond_lock_break(&mm->page_table_lock);
there are two problems with the unconditional lock-break:
- i'm not sure cpu_relax() guarantees that another CPU can grab this lock. It all depends on whether the lock cacheline can bounce over to the other CPU faster than cpu_relax() finishes and this CPU re-acquires the lock.
- the latencies will still be higher than on UP, in a hard to predict way. Every time the codepath encounters a spinlock it has to take in order to exit for a reschedule, the latency gets re-added. Also, the now scheduled high-prio task will encounter the latency every time it acquires a spinlock - while on UP it has the CPU for itself with no interaction from other CPUs.
Ingo
Re: [patch] voluntary-preempt-2.6.8-rc2-J3
Quoted 9 lines · Ingo Molnar
...
guaranteeing latencies on SMP is hard, because the latency of a CPU might depend on the latency of a task on another CPU - which CPU isnt notified of the rescheduling request.one alternative technique to yours would be to notify _all_ CPUs that a high-prio RT task is ready to run (via a broadcast need-resched). That way the UP latency-break techniques map to SMP in a 1:1 way.
I did exactly that in the 2.4 ll patch. It works, but doesn't seem very nice.
Quoted 10 lines
non-RT tasks dont get this benefit, which is a difference to the UP situation, but i dont think it would be appropriate to use the UP behavior, due to the overhead of broadcasting.
a combination of the two techniques could be used too: a global 'break locks from now on' flag which gets set if a (RT?) task wants to reschedule. Normally this flag would be zero and the cacheline would be clean and shared between all CPUs, causing no overhead. Once a task wants to reschedule it would increase by 1 until that task has been scheduled.
Something like that. The obscure livelock opportunities worry me.
Quoted 12 lines
This would remove the ugliness factor too, your sample code would look:
Review
Quoted 6 lines
(quoted diff) +6
-0+ if (need_lock_break()) { + spin_unlock(&mm->page_table_lock); + cpu_relax(); + spin_lock(&mm->page_table_lock); + } +or a shortcut:
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ cond_lock_break(&mm->page_table_lock);
The usual problem is, of course, long-running loops under locks. These loops usually have some sort of counter (or you add one). The below patch changes cond_resched_lock() so that it takes the counter, and the tuned upper-loop-count limit. It also does the right thing on UP. It needs enhancements for the situations where we already have a counter in the loop.
I dunno if it's sufficient, but I do think that we need to settle on some infrastructure of this sort before proceeding too far into this thing.
Quoted 6 lines
there are two problems with the unconditional lock-break:
- i'm not sure
cpu_relax()guarantees that another CPU can grab this lock. It all depends on whether the lock cacheline can bounce over to the other CPU faster thancpu_relax()finishes and this CPU re-acquires the lock.
precisely.
include/linux/sched.h renamed from include/linux/sched.h~cond_resched_lock-smp-fix +22 -12
@@ -1038,22 +1038,32 @@ static inline void cond_resched(void) }/* -* cond_resched_lock() - if a reschedule is pending, drop the given lock, -* call schedule, and on return reacquire the lock. -* * This works OK both with and without CONFIG_PREEMPT. We do strange low-level * operations here to prevent schedule() from being called twice (once via * spin_unlock(), once by hand). */ -static inline void cond_resched_lock (spinlock_t * lock) -{ -if (need_resched ()) { -_raw_spin_unlock (lock); -preempt_enable_no_resched (); -__cond_resched (); -spin_lock (lock); - } -} + +#ifdef CONFIG_SMP +#define cond_resched_lock (lock, counter, limit) \ + do { \ + if (++(counter) >= limit) { \ + spin_unlock(lock); \ + cpu_relax(); \ + spin_lock(lock); \ + } \ + (counter) = 0; \ + } while (0) +#else +#define cond_resched_lock (lock, counter, limit) \ + do { \ + if (need_resched()) { \ + _raw_spin_unlock(lock); \ + preempt_enable_no_resched(); \ + __cond_resched(); \ + spin_lock(lock); \ + } \ + } while (0) +#endif /* Reevaluate whether the task has signals pending delivery. This is required every time the blocked sigset_t changes.
fs/reiserfs/journal.c renamed from fs/reiserfs/journal.c~cond_resched_lock-smp-fix +3 -2
@@ -796,6 +796,7 @@ static int write_ordered_buffers(spinlocstruct buffer_chunk chunk;struct list_head tmp;INIT_LIST_HEAD (&tmp); +int resched_counter =0 ; chunk.nr =0 ;spin_lock (lock);
@@ -827,7 +828,7 @@ static int write_ordered_buffers(spinloc } loop_next:put_bh (bh); -cond_resched_lock (lock); +cond_resched_lock (lock, resched_counter,64 ); }if (chunk.nr) {spin_unlock (lock);
@@ -848,7 +849,7 @@ loop_next:if (!buffer_uptodate (bh)) ret = -EIO ;put_bh (bh); -cond_resched_lock (lock); +cond_resched_lock (lock, resched_counter,64 ); }spin_unlock (lock);return ret;
mm/shmem.c renamed from mm/shmem.c~cond_resched_lock-smp-fix +2 -1
@@ -424,6 +424,7 @@ static void shmem_truncate(struct inodeswp_entry_t *ptr;int offset;int freed; +int resched_counter =0 ; inode->i_ctime = inode->i_mtime =CURRENT_TIME ; idx = (inode->i_size +PAGE_CACHE_SIZE -1 ) >>PAGE_CACHE_SHIFT ;
@@ -496,7 +497,7 @@ static void shmem_truncate(struct inodeshmem_free_block (inode); } empty = subdir; -cond_resched_lock (&info->lock); +cond_resched_lock (&info->lock, resched_counter,32 ); dir =shmem_dir_map (subdir); } subdir = *dir;
mm/memory.c renamed from mm/memory.c~cond_resched_lock-smp-fix +7 -3
@@ -224,6 +224,7 @@ int copy_page_range(struct mm_struct *dsunsigned long address = vma->vm_start;unsigned long end = vma->vm_end;unsigned long cow; +int resched_counter =0 ;if (is_vm_hugetlb_page (vma))return copy_hugetlb_page_range (dst, src, vma);
@@ -341,7 +342,8 @@ cont_copy_pte_range_noset:pte_unmap_nested (src_pte-1 );pte_unmap (dst_pte-1 );spin_unlock (&src->page_table_lock); -cond_resched_lock (&dst->page_table_lock); +cond_resched_lock (&dst->page_table_lock, + resched_counter,256 ); cont_copy_pmd_range: src_pmd++; dst_pmd++;
@@ -530,6 +532,7 @@ int unmap_vmas(struct mmu_gather **tlbp,int tlb_start_valid =0 ;int ret =0 ;int atomic = details && details->atomic; +int resched_counter =0 ;for ( ; vma && vma->vm_start < end_addr; vma = vma->vm_next) {unsigned long start;
@@ -567,10 +570,11 @@ int unmap_vmas(struct mmu_gather **tlbp, zap_bytes -= block;if ((long )zap_bytes >0 )continue ; -if (!atomic &&need_resched ()) { +if (!atomic) {int fullmm =tlb_is_full_mm (*tlbp);tlb_finish_mmu (*tlbp, tlb_start, start); -cond_resched_lock (&mm->page_table_lock); +cond_resched_lock (&mm->page_table_lock, + resched_counter,1 ); *tlbp =tlb_gather_mmu (mm, fullmm); tlb_start_valid =0 ; }
mm/rmap.c renamed from mm/rmap.c~cond_resched_lock-smp-fix +3 -2
@@ -661,6 +661,7 @@ static inline int try_to_unmap_file(struunsigned long max_nl_cursor =0 ;unsigned long max_nl_size =0 ;unsigned int mapcount; +int resched_counter =0 ;if (!spin_trylock (&mapping->i_mmap_lock))return ret;
@@ -699,7 +700,7 @@ static inline int try_to_unmap_file(stru*/ mapcount = page->mapcount;page_map_unlock (page); -cond_resched_lock (&mapping->i_mmap_lock); +cond_resched_lock (&mapping->i_mmap_lock, resched_counter,1 ); max_nl_size = (max_nl_size +CLUSTER_SIZE -1 ) &CLUSTER_MASK ;if (max_nl_cursor ==0 )
@@ -728,7 +729,7 @@ static inline int try_to_unmap_file(stru (void *) max_nl_cursor; ret =SWAP_AGAIN ; } -cond_resched_lock (&mapping->i_mmap_lock); +cond_resched_lock (&mapping->i_mmap_lock, resched_counter,8 ); max_nl_cursor +=CLUSTER_SIZE ; }while (max_nl_cursor <= max_nl_size);
_
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [patch] voluntary-preempt-2.6.8-rc2-J3
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"AM" == Andrew Morton <[email protected]> writes:
AM> +#ifdef CONFIG_SMP
AM> +#define cond_resched_lock(lock, counter, limit) \
AM> + do { \
AM> + if (++(counter) >= limit) { \
AM> + spin_unlock(lock); \
AM> + cpu_relax(); \
AM> + spin_lock(lock); \
AM> + } \
AM> + (counter) = 0; \
AM> + } while (0)
AM> +#else
I am wondering if you meant to reset the counter to zero inside of the if(){}, probably after reaquiring the lock...
Re: [patch] voluntary-preempt-2.6.8-rc2-J3
In reply to Ingo MolnarQuoted 6 lines · Ingo Molnar
Quoted 3 lines · Andrew Morton
The bigger this thing gets, the more worried I get. Sometime this is going to need to be split up into individual fixes, and they need to be based upon an overall approach which we haven't yet settled on.
Quoted 8 lines
i will do that splitup. Right now i'm simply mapping how widespread the problem is and what type of fixes we need. The situation isnt all that bad but we might need (an optional) mechanism to make softirqs synchronous. All of this stuff is nicely modular and i'll do a splitup post 2.6.8 (i dont think we want to disturb 2.6.8 with any of this).
From a user's perspective, and based on my own testing, I do not see the
patch having to get much bigger, the vast majority of the hot spots have been fixed, and the patch remains quite comprehensible even to someone with a sketchy knowledge of the kernel. There is only one issue I can think of that has not been addressed at all (the PS/2 Caps Lock issue). All that seems to remain are tweaks to individual fixes that are already in the patch.
Lee
Re: [patch] voluntary-preempt-2.6.8-rc2-J3
In reply to Ingo MolnarQuoted 4 lines · Ingo Molnar
one alternative technique to yours would be to notify _all_ CPUs that a high-prio RT task is ready to run (via a broadcast need-resched). That way the UP latency-break techniques map to SMP in a 1:1 way.
Quoted 8 lines
non-RT tasks dont get this benefit, which is a difference to the UP situation, but i dont think it would be appropriate to use the UP behavior, due to the overhead of broadcasting.
a combination of the two techniques could be used too: a global 'break locks from now on' flag which gets set if a (RT?) task wants to reschedule. Normally this flag would be zero and the cacheline would be clean and shared between all CPUs, causing no overhead. Once a task
It might be possible to eliminate the need_resched flag. Here is an article that talk about a event-driven preemption model.
Quoting :
Over the long term, MontaVista is investigating whether preemption locks can be eliminated (or at least greatly reduced in number) by protecting all the short-duration critical regions with spinlocks that also disable interrupts on the local CPU, and the long-duration critical regions with mutex locks. ("Long duration" means much greater than the time taken by two context switches.) This will reduce the overhead of the preemptable kernel, since there will no longer be any need to test for preemption ("polling for preemption") at the end of a preemption-locked region (which could happen tens of thousands of times per second on an average system). Instead, preemption would happen automatically as part of the interrupt servicing that causes a higher-priority process to become runnable ("event-driven preemption"). Typically, this only happens a few times to a few tens of times per second with an average system workload, making the "event-driven preemption" model much more efficient than the "polling for preemption" model. This method also has an added efficiency in that the system will take advantage of the cache disruption caused by the interrupt (which is unavoidable) to continue with the preemption.
Reference:
http://www.linuxdevices.com/cgi-bin/printerfriendly.cgi?id=AT4185744181
Best regards,
Eric St-Laurent
Re: [patch] voluntary-preempt-2.6.8-rc2-J3
Quoted 10 lines · Eric St-Laurent
It might be possible to eliminate the
need_reschedflag. Here is an article that talk about a event-driven preemption model.Quoting :
Over the long term, MontaVista is investigating whether preemption locks can be eliminated (or at least greatly reduced in number) by protecting all the short-duration critical regions with spinlocks that also disable interrupts on the local CPU, and the long-duration critical regions with mutex locks.
This is a pretty old article, from 2000, describing the preemption model they implemented on 2.4. I believe the above paragraph describes the advantages of their model vs. the approach taken by the old 2.4 low latency patches.
My understanding is that the preemptible 2.6 kernel already works this way.
Lee
Quoted 10 lines · William Lee Irwin III
This patch uses the preemption counter increments and decrements to time non-preemptible critical sections.
This is an instrumentation patch intended to help determine the causes of scheduling latency related to long non-preemptible critical sections.
Changes from 2.6.7-based patch:
(1) fixunmap_vmas()check correctly this time
(2) addtouch_preempt_timing()tocond_resched_lock()
(3) depend on preempt until it's worked out wtf goes wrong without it
You preemption-block hold times will improve *enormously* if you move all softirq processing down to the daemon (and possibly raise the daemon to one of the higher SCHED_RR priorities, to compensate for softirq processing no longer happening at interrupt level).
Joe
Quoted 8 lines · William Lee Irwin III
Quoted 8 lines
This patch uses the preemption counter increments and decrements to time non-preemptible critical sections.
This is an instrumentation patch intended to help determine the causes of scheduling latency related to long non-preemptible critical sections. Changes from 2.6.7-based patch:
(1) fixunmap_vmas()check correctly this time
(2) addtouch_preempt_timing()tocond_resched_lock()
(3) depend on preempt until it's worked out wtf goes wrong without it
Quoted 4 lines · Joe Korty
You preemption-block hold times will improve *enormously* if you move all softirq processing down to the daemon (and possibly raise the daemon to one of the higher
SCHED_RRpriorities, to compensate for softirq processing no longer happening at interrupt level).
Plausible. Got a patch?
-- wli
Quoted 14 lines · William Lee Irwin III
Quoted 8 lines · William Lee Irwin III
Quoted 8 lines
This patch uses the preemption counter increments and decrements to time non-preemptible critical sections.
This is an instrumentation patch intended to help determine the causes of scheduling latency related to long non-preemptible critical sections. Changes from 2.6.7-based patch:
(1) fixunmap_vmas()check correctly this time
(2) addtouch_preempt_timing()tocond_resched_lock()
(3) depend on preempt until it's worked out wtf goes wrong without itOn Tue, Jul 13, 2004 at 10:36:00AM -0400, Joe Korty wrote:
Quoted 8 lines
Quoted 4 lines
You preemption-block hold times will improve *enormously* if you move all softirq processing down to the daemon (and possibly raise the daemon to one of the higher
SCHED_RRpriorities, to compensate for softirq processing no longer happening at interrupt level).Plausible. Got a patch?
Make MAX_SOFTIRQ_RESTART 1?
I don't think you should make ksoftirq a realtime task, because that defeats the purpose of having it to prevent livelocking userspace doesn't it?
However, you may want to increase it from nice +19. Probably just to nice 0 would be an idea.
Quoted 17 lines · William Lee Irwin III
Quoted 10 lines · William Lee Irwin III
Quoted 10 lines
This patch uses the preemption counter increments and decrements to time non-preemptible critical sections.
This is an instrumentation patch intended to help determine the causes of scheduling latency related to long non-preemptible critical sections. Changes from 2.6.7-based patch:
(1) fixunmap_vmas()check correctly this time
(2) addtouch_preempt_timing()tocond_resched_lock()
(3) depend on preempt until it's worked out wtf goes wrong without itOn Tue, Jul 13, 2004 at 10:36:00AM -0400, Joe Korty wrote:
Quoted 10 lines
Quoted 6 lines
You preemption-block hold times will improve *enormously* if you move all softirq processing down to the daemon (and possibly raise the daemon to one of the higher
SCHED_RRpriorities, to compensate for softirq processing no longer happening at interrupt level).Plausible. Got a patch?
How about this? It moves softirqs to a thread but keeps the old spill thread.
This is a patch against 2.6.6 written primarily by Scott Wood.
Signature · 1 line
Anyone who quotes me in their sig is an idiot. -- Rusty Russell's sig
include/asm-alpha/hardirq.h +11 -7
@@ -86,14 +86,18 @@#define in_atomic () (preempt_count() != 0)#define IRQ_EXIT_OFFSET HARDIRQ_OFFSET# endif -#define irq_exit () \ -do { \ - preempt_count() -= IRQ_EXIT_OFFSET; \ - if (!in_interrupt() && \ - softirq_pending(smp_processor_id())) \ - do_softirq(); \ - preempt_enable_no_resched(); \ + +#ifndef CONFIG_SOFTIRQ_THREADS +#define irq_exit () \ +do { \ + preempt_count() -= IRQ_EXIT_OFFSET; \ + if (!in_interrupt() && softirq_pending(smp_processor_id())) \ + do_softirq(); \ + preempt_enable_no_resched(); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-arm/hardirq.h +4 -0
@@ -81,6 +81,7 @@ extern asmlinkage void __do_softirq(void); +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= IRQ_EXIT_OFFSET; \
@@ -88,6 +89,9 @@__do_softirq (); \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #define synchronize_irq (irq) barrier()#else
include/asm-arm26/hardirq.h +5 -0
@@ -76,6 +76,8 @@#endif #ifndef CONFIG_SMP + +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= HARDIRQ_OFFSET; \
@@ -83,6 +85,9 @@ __asm__("bl%? __do_softirq" : : :"lr" );/* out of line */ \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #define synchronize_irq (irq) barrier()#else
include/asm-cris/hardirq.h +6 -0
@@ -84,6 +84,9 @@# define in_atomic () (preempt_count() != 0)# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif + + +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= IRQ_EXIT_OFFSET; \
@@ -91,6 +94,9 @@do_softirq (); \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #define synchronize_irq (irq) barrier()
include/asm-h8300/hardirq.h +5 -0
@@ -82,6 +82,8 @@# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif + +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= IRQ_EXIT_OFFSET; \
@@ -89,6 +91,9 @@do_softirq (); \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-i386/hardirq.h +5 -0
@@ -84,6 +84,8 @@# define in_atomic () (preempt_count() != 0)# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif + +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= IRQ_EXIT_OFFSET; \
@@ -91,6 +93,9 @@do_softirq (); \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-m68k/hardirq.h +5 -0
@@ -78,6 +78,8 @@# define in_atomic () (preempt_count() != 0)# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif + +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= IRQ_EXIT_OFFSET; \
@@ -85,6 +87,9 @@do_softirq (); \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #define synchronize_irq (irq) barrier()
include/asm-m68knommu/hardirq.h +4 -0
@@ -86,6 +86,7 @@# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= IRQ_EXIT_OFFSET; \
@@ -93,6 +94,9 @@do_softirq (); \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-mips/hardirq.h +11 -6
@@ -87,13 +87,18 @@# define in_atomic () (preempt_count() != 0)# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif -#define irq_exit () \ -do { \ - preempt_count() -= IRQ_EXIT_OFFSET; \ - if (!in_interrupt() && softirq_pending(smp_processor_id())) \ - do_softirq(); \ - preempt_enable_no_resched(); \ + +#ifndef CONFIG_SOFTIRQ_THREADS +#define irq_exit () \ +do { \ + preempt_count() -= IRQ_EXIT_OFFSET; \ + if (!in_interrupt() && softirq_pending(smp_processor_id())) \ + do_softirq(); \ + preempt_enable_no_resched(); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-parisc/hardirq.h +10 -6
@@ -96,13 +96,17 @@# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif -#define irq_exit () \ -do { \ - preempt_count() -= IRQ_EXIT_OFFSET; \ - if (!in_interrupt() && softirq_pending(smp_processor_id())) \ - do_softirq(); \ - preempt_enable_no_resched(); \ +#ifndef CONFIG_SOFTIRQ_THREADS +#define irq_exit () \ +do { \ + preempt_count() -= IRQ_EXIT_OFFSET; \ + if (!in_interrupt() && softirq_pending(smp_processor_id())) \ + do_softirq(); \ + preempt_enable_no_resched(); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifdef CONFIG_SMP extern void synchronize_irq (unsigned int irq);
include/asm-ppc/hardirq.h +6 -1
@@ -88,13 +88,18 @@# define in_atomic () (preempt_count() != 0)# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif + +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ -do { \ +do { \ preempt_count() -= IRQ_EXIT_OFFSET; \ if (!in_interrupt() && softirq_pending(smp_processor_id())) \ do_softirq(); \ preempt_enable_no_resched(); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-ppc64/hardirq.h +5 -0
@@ -87,6 +87,8 @@# define in_atomic () (preempt_count() != 0)# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif + +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= IRQ_EXIT_OFFSET; \
@@ -94,6 +96,9 @@do_softirq (); \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-sh/hardirq.h +5 -0
@@ -81,6 +81,8 @@# define in_atomic () (preempt_count() != 0)# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif + +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= IRQ_EXIT_OFFSET; \
@@ -88,6 +90,9 @@do_softirq (); \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-sparc/hardirq.h +11 -6
@@ -87,13 +87,18 @@# define in_atomic () (preempt_count() != 0)# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif -#define irq_exit () \ -do { \ - preempt_count() -= IRQ_EXIT_OFFSET; \ - if (!in_interrupt() && softirq_pending(smp_processor_id())) \ - do_softirq(); \ - preempt_enable_no_resched(); \ + +#ifndef CONFIG_SOFTIRQ_THREADS +#define irq_exit () \ +do { \ + preempt_count() -= IRQ_EXIT_OFFSET; \ + if (!in_interrupt() && softirq_pending(smp_processor_id())) \ + do_softirq(); \ + preempt_enable_no_resched(); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-sparc64/hardirq.h +5 -0
@@ -86,6 +86,8 @@# define in_atomic () (preempt_count() != 0)# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif + +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= IRQ_EXIT_OFFSET; \
@@ -93,6 +95,9 @@do_softirq (); \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-v850/hardirq.h +10 -6
@@ -80,13 +80,17 @@# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif -#define irq_exit () \ -do { \ - preempt_count() -= IRQ_EXIT_OFFSET; \ - if (!in_interrupt() && softirq_pending(smp_processor_id())) \ - do_softirq(); \ - preempt_enable_no_resched(); \ +#ifndef CONFIG_SOFTIRQ_THREADS +#define irq_exit () \ +do { \ + preempt_count() -= IRQ_EXIT_OFFSET; \ + if (!in_interrupt() && softirq_pending(smp_processor_id())) \ + do_softirq(); \ + preempt_enable_no_resched(); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/asm-x86_64/hardirq.h +5 -0
@@ -85,6 +85,8 @@# define in_atomic () (preempt_count() != 0)# define IRQ_EXIT_OFFSET HARDIRQ_OFFSET#endif + +#ifndef CONFIG_SOFTIRQ_THREADS #define irq_exit () \ do { \ preempt_count() -= IRQ_EXIT_OFFSET; \
@@ -92,6 +94,9 @@do_softirq (); \preempt_enable_no_resched (); \ } while (0) +#else +#define irq_exit () (preempt_count() -= HARDIRQ_OFFSET) +#endif #ifndef CONFIG_SMP # define synchronize_irq (irq) barrier()
include/linux/interrupt.h +23 -0
@@ -59,6 +59,8 @@#endif /* SoftIRQ primitives. */ +#ifndef CONFIG_SOFTIRQ_THREADS +#define local_bh_disable () \ do { preempt_count() += SOFTIRQ_OFFSET; barrier(); } while (0)#define __local_bh_enable () \
@@ -66,6 +68,27 @@extern void local_bh_enable (void ); +#else + +/* As far as I can tell, local_bh_disable() didn't stop ksoftirqd +from running before. Since all softirqs now run from one of +the ksoftirqds, this shouldn't be necessary. */ + +static inline void local_bh_disable (void ) +{ +} + +static inline void __local_bh_enable (void ) +{ +} + +static inline void local_bh_enable (void ) +{ +} + +#endif + +/* PLEASE, avoid to allocate new softirqs, if you need not _really_ high frequency threaded job scheduling. For almost all the purposes tasklets are more than enough. F.e. all serial device BHs et
include/linux/sched.h +8 -0
@@ -541,6 +541,11 @@#define PF_LESS_THROTTLE 0x00100000/* Throttle me less: I clean memory */ #define PF_SYNCWRITE 0x00200000/* I am doing a sync write */ +/* Thread is an IRQ handler. This is used to determine which softirq +thread to wake. */ + +#define PF_IRQHANDLER 0x10000000 +#ifdef CONFIG_SMP extern int set_cpus_allowed (task_t *p,cpumask_t new_mask);#else
@@ -916,6 +921,9 @@extern void signal_wake_up (struct task_struct *t,int resume_stopped); +asmlinkage long sys_sched_setscheduler (pid_t pid,int policy, +struct sched_param __user *param); +/* * Wrappers for p->thread_info->cpu access. No-op on UP. */
init/Kconfig +12 -0
@@ -265,6 +265,18 @@ If unsure, say N. +config SOFTIRQ_THREADS + bool "Run all softirqs in threads" + depends on PREEMPT + help + This option creates a second softirq thread per CPU, which + runs at high real-time priority, to replace the softirqs + which were previously run immediately. This allows these + softirqs to be prioritized, so as to avoid preempting + very high priority real-time tasks. This also allows + certain spinlocks to be converted into sleeping mutexes, + for futher reduction of scheduling latency. + endmenu # General setup
kernel/softirq.c +99 -4
171 lines not on this page · raw message
kernel/timer.c +5 -1
@@ -910,12 +910,16 @@void do_timer (struct pt_regs *regs) { jiffies_64++; +update_times (); +#ifndef CONFIG_SMP /* SMP process accounting uses the local APIC timer */ +// Do this after update_times(), so that the softirq thread +// is not counted as running all the time. +update_process_times (user_mode (regs));#endif -update_times (); }#if !defined(__alpha__) && !defined(__ia64__)
mm/slab.c +1 -1
@@ -2593,7 +2593,7 @@ {struct list_head *walk; -#if DEBUG +#if DEBUG && !defined(CONFIG_SOFTIRQ_THREADS )BUG_ON (!in_interrupt ());BUG_ON (in_irq ());#endif
net/ipv4/ipconfig.c +7 -2
@@ -1085,8 +1085,13 @@ jiff = jiffies + (d->next ?CONF_INTER_TIMEOUT : timeout);while (time_before (jiffies, jiff) && !ic_got_reply) { -barrier (); -cpu_relax (); +// Wait queues are apparently too hard +// for the ipconfig people, but we +// need to drop the BKL here to allow +// preemption. + +set_current_state (TASK_INTERRUPTIBLE ); +schedule_timeout (1 ); }#ifdef IPCONFIG_DHCP /* DHCP isn't done until we get a DHCPACK. */
Quoted 10 lines · William Lee Irwin III
Quoted 8 lines · William Lee Irwin III
Quoted 8 lines
This patch uses the preemption counter increments and decrements to time non-preemptible critical sections.
This is an instrumentation patch intended to help determine the causes of scheduling latency related to long non-preemptible critical sections. Changes from 2.6.7-based patch:
(1) fixunmap_vmas()check correctly this time
(2) addtouch_preempt_timing()tocond_resched_lock()
(3) depend on preempt until it's worked out wtf goes wrong without it
Quoted 7 lines
Quoted 4 lines · Joe Korty
You preemption-block hold times will improve *enormously* if you move all softirq processing down to the daemon (and possibly raise the daemon to one of the higher
SCHED_RRpriorities, to compensate for softirq processing no longer happening at interrupt level).Plausible. Got a patch?
I can scrape one up. It'll be a day or so.
Joe
PATCH 1/1 fix arbitrarily long preemption lockout times [was: re: preempt-timing-2.6.8-rc1]
In reply to William Lee Irwin IIIQuoted 7 lines · William Lee Irwin III
Quoted 4 lines · Joe Korty
You preemption-block hold times will improve *enormously* if you move all softirq processing down to the daemon (and possibly raise the daemon to one of the higher
SCHED_RRpriorities, to compensate for softirq processing no longer happening at interrupt level).Plausible. Got a patch?
Here it is. Against 2.6.7. Testing: compiles, boots, shutdown ok, ran some disk and networking traffic under it.
IIRC, my first attempt had problems booting some configurations, my guess is that some drivers try to raise softIRQs before the daemons came up. So this version uses the global 'system_running' to switch at runtime between the interrupt/daemon and daemon-only modes of operation.
This also has the advantage of switching back to the safer, interrupt/daemon mode during system shutdown and while crash dumps are in progress (for those who have applied that patch).
IMO, this patch would be better if it introduced a seperate, hi-priority softirq daemon, as La Monte H.P. Yarroll's patch does.
Regards,
Joe
arch/i386/kernel/irq.c +2 -2
@@ -1154,7 +1154,7 @@ extern asmlinkage void __do_softirq(void); -asmlinkage void do_softirq(void) +asmlinkage void do_softirq_arch(void) {unsigned long flags;struct thread_info *curctx;
@@ -1189,5 +1189,5 @@ local_irq_restore(flags); } -EXPORT_SYMBOL (do_softirq); +EXPORT_SYMBOL (do_softirq_arch);#endif
arch/ppc64/kernel/irq.c +2 -2
@@ -1015,7 +1015,7 @@ } } -void do_softirq (void ) +void do_softirq_arch (void ) {unsigned long flags;struct thread_info *curtp, *irqtp;
@@ -1035,7 +1035,7 @@ local_irq_restore(flags); } -EXPORT_SYMBOL (do_softirq); +EXPORT_SYMBOL (do_softirq_arch);#endif /* CONFIG_IRQSTACKS */
include/linux/interrupt.h +1 -0
@@ -93,6 +93,7 @@ }; asmlinkage void do_softirq(void); +asmlinkage void do_softirq_arch(void);extern void open_softirq (int nr,void (*action)(struct softirq_action *),void *data);extern void softirq_init (void );#define __raise_softirq_irqoff (nr) do { local_softirq_pending() |= 1UL << (nr); } while (0)
init/Kconfig +37 -0
@@ -218,6 +218,43 @@ This option enables access to kernel configuration file and build information through /proc/config.gz. +config SOFTIRQ_PREEMPT_BLOCK + bool "SoftIRQs to be affected by preemption blocks" + default y + help + If N then SoftIRQ interrupt handlers are to be blocked by + exactly the same conditions as hard IRQ handlers. If Y then + SoftIRQ handlers are also to be blocked by preemption blocks. + + The reasons why Y is useful are too long to elaborate here. + Suffice it to say that SoftIRQ handlers unlike hardIRQ handlers + have no upper bounds on their execution time and allowing + them to interrupt spinlocked critical regions protected only + by a preemption block means that no such region can have an + upper bound on the lock hold time. + + Realtime systems should say Y here, all others may say N or + Y. If unsure, say N. + +config SOFTIRQ_PRI + int "SoftIRQ daemon priority" + default 0 + help + Select the scheduling policy and priority for the per-cpu + SoftIRQ daemons. + + If >0 then the daemons will have a policy of SCHED_FIFO and + their priorities will each be set to the given value. + + If ==0 then the daemons will run with a SCHED_OTHER policy + unless SOFTIRQ_PREEMPT_BLOCK is set to Y. In that case the + daemons will run under SCHED_FIFO with a priority one less + than the system maximum. + + Most users should say 0 unless there is a specific priority + the softIRQ daemons must run at. If unsure, say 0. + + menuconfig EMBEDDED bool "Configure standard kernel features (for small systems)"
kernel/softirq.c +43 -3
80 lines not on this page · raw message
Hi,
More violations found.
This time during NFS umount/mount. Since (u)mounting is not always
the operation that is only done at boot-time (at least in my case), I thought I report.
umount /nfs/dir
16ms non-preemptible critical section violated 2 ms preempt threshold starting at truncate_inode_pages+0x9d/0x280 and ending at generic_shutdown_super+0xd0/0x190
[<c0153340>] generic_shutdown_super+0xd0/0x190 [<c0117d30>] dec_preempt_count+0x110/0x120 [<c0153340>] generic_shutdown_super+0xd0/0x190 [<c0117c70>] dec_preempt_count+0x50/0x120 [<c0153cf9>] kill_anon_super+0x9/0x40 [<f935884c>] nfs_kill_super+0xc/0x20 [nfs] [<c015317c>] deactivate_super+0x6c/0x90 [<c016809b>] sys_umount+0x3b/0x90 [<c0164a85>] destroy_inode+0x35/0x40 [<c014e645>] __fput+0xb5/0x120 [<c0168107>] sys_oldumount+0x17/0x20 [<c0103ee1>] sysenter_past_esp+0x52/0x71
mount /nfs/dir
nfs warning: mount version older than kernel
15ms non-preemptible critical section violated 2 ms preempt threshold starting at sys_mount+0x78/0x110 and ending at schedule+0x237/0x480
[<c0279677>] schedule+0x237/0x480 [<c0117d30>] dec_preempt_count+0x110/0x120 [<c0279677>] schedule+0x237/0x480 [<c0127c21>] worker_thread+0x1d1/0x240 [<c0127c72>] worker_thread+0x222/0x240 [<c01166f8>] activate_task+0x68/0x80 [<c0219800>] fb_flashcursor+0x0/0x80 [<c0116d60>] default_wake_function+0x0/0x10 [<c0279677>] schedule+0x237/0x480 [<c0116d60>] default_wake_function+0x0/0x10 [<c0127a50>] worker_thread+0x0/0x240 [<c012b2c4>] kthread+0x94/0xa0 [<c012b230>] kthread+0x0/0xa0 [<c010227d>] kernel_thread_helper+0x5/0x18
umount /nfs/dir
19ms non-preemptible critical section violated 2 ms preempt threshold starting at generic_shutdown_super+0x69/0x190 and ending at generic_shutdown_super+0xd0/0x190
[<c0153340>] generic_shutdown_super+0xd0/0x190 [<c0117d30>] dec_preempt_count+0x110/0x120 [<c0153340>] generic_shutdown_super+0xd0/0x190 [<c0117c70>] dec_preempt_count+0x50/0x120 [<c0153cf9>] kill_anon_super+0x9/0x40 [<f935884c>] nfs_kill_super+0xc/0x20 [nfs] [<c015317c>] deactivate_super+0x6c/0x90 [<c016809b>] sys_umount+0x3b/0x90 [<c0164a85>] destroy_inode+0x35/0x40 [<c014e645>] __fput+0xb5/0x120 [<c0168107>] sys_oldumount+0x17/0x20 [<c0103ee1>] sysenter_past_esp+0x52/0x71
Lenar