miserable performance of 2.6.21 under network load
I'm in the process up upgrading a pool of apache servers from
2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load, 2.6.21 has a load average more than 3 times higher, cpu 0 spends well over 90% of its time in interrupts (vs ~30% under 2.6.17). When we hit 3k apache sessions, ksoftirqd eats 100% of cpu0 and our network traffic drops off rapidly. The end result is that 2.6.17 performs twice as well under this load.
Quoted 7 lines · Aaron Porter
I'm in the process up upgrading a pool of apache servers from
2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load, 2.6.21 has a load average more than 3 times higher, cpu 0 spends well over 90% of its time in interrupts (vs ~30% under 2.6.17). When we hit 3k apache sessions, ksoftirqd eats 100% of cpu0 and our network traffic drops off rapidly. The end result is that 2.6.17 performs twice as well under this load.
Can you oprofile it?
-Andi
Quoted 11 lines · Andi Kleen
Quoted 7 lines · Aaron Porter
I'm in the process up upgrading a pool of apache servers from
2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load, 2.6.21 has a load average more than 3 times higher, cpu 0 spends well over 90% of its time in interrupts (vs ~30% under 2.6.17). When we hit 3k apache sessions, ksoftirqd eats 100% of cpu0 and our network traffic drops off rapidly. The end result is that 2.6.17 performs twice as well under this load.Can you oprofile it?
# opreport -l CPU: AMD64 processors, speed 1994.52 MHz (estimated) Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 100000 samples % app name symbol name 914379 48.8404 vmlinux-2.6.21.5 check_poison_obj 341920 18.2632 vmlinux-2.6.21.5 poison_obj 37355 1.9953 nf_conntrack (no symbols) 36777 1.9644 vmlinux-2.6.21.5 __copy_to_user_ll 35095 1.8746 vmlinux-2.6.21.5 skb_copy_bits 24256 1.2956 vmlinux-2.6.21.5 __copy_from_user_ll 23988 1.2813 vmlinux-2.6.21.5 cache_alloc_debugcheck_after 16897 0.9025 vmlinux-2.6.21.5 debug_smp_processor_id 14323 0.7650 vmlinux-2.6.21.5 kfree 13143 0.7020 vmlinux-2.6.21.5 tcp_clean_rtx_queue 11742 0.6272 vmlinux-2.6.21.5 sub_preempt_count 11484 0.6134 vmlinux-2.6.21.5 add_preempt_count 9218 0.4924 libc-2.3.2.so (no symbols) 8878 0.4742 vmlinux-2.6.21.5 tcp_write_xmit 8513 0.4547 vmlinux-2.6.21.5 cache_free_debugcheck 7757 0.4143 vmlinux-2.6.21.5 kmem_cache_free 7622 0.4071 vmlinux-2.6.21.5 obj_size 7352 0.3927 oprofiled (no symbols) 7041 0.3761 vmlinux-2.6.21.5 kfree_skb 6955 0.3715 vmlinux-2.6.21.5 tcp_v4_rcv 6893 0.3682 vmlinux-2.6.21.5 tg3_rx 6731 0.3595 iptable_nat (no symbols) 6673 0.3564 vmlinux-2.6.21.5 ip_queue_xmit 6482 0.3462 vmlinux-2.6.21.5 nf_iterate 6345 0.3389 vmlinux-2.6.21.5 kmem_cache_alloc 6280 0.3354 vmlinux-2.6.21.5 skb_clone 5922 0.3163 vmlinux-2.6.21.5 ip_output 5885 0.3143 vmlinux-2.6.21.5 __alloc_skb 5689 0.3039 vmlinux-2.6.21.5 tg3_start_xmit_dma_bug 5617 0.3000 vmlinux-2.6.21.5 check_irq_off 5038 0.2691 vmlinux-2.6.21.5 _raw_spin_unlock 4891 0.2612 nf_nat (no symbols) 4692 0.2506 vmlinux-2.6.21.5 __tcp_select_window 4665 0.2492 vmlinux-2.6.21.5 obj_offset 4556 0.2434 vmlinux-2.6.21.5 __kfree_skb 4425 0.2364 vmlinux-2.6.21.5 schedule 4278 0.2285 vmlinux-2.6.21.5 tcp_sendmsg 4050 0.2163 vmlinux-2.6.21.5 __kmalloc_track_caller 4035 0.2155 vmlinux-2.6.21.5 preempt_schedule 4014 0.2144 nf_conntrack_ipv4 (no symbols) 3564 0.1904 vmlinux-2.6.21.5 copy_skb_header 3547 0.1895 vmlinux-2.6.21.5 skb_release_data 3436 0.1835 vmlinux-2.6.21.5 _raw_spin_trylock 3403 0.1818 vmlinux-2.6.21.5 tcp_rcv_established 3290 0.1757 vmlinux-2.6.21.5 tcp_set_skb_tso_segs 3004 0.1605 vmlinux-2.6.21.5 check_slabp 2947 0.1574 vmlinux-2.6.21.5 tcp_transmit_skb 2919 0.1559 vmlinux-2.6.21.5 nf_hook_slow 2889 0.1543 vmlinux-2.6.21.5 dev_queue_xmit 2803 0.1497 vmlinux-2.6.21.5 __qdisc_run 2645 0.1413 vmlinux-2.6.21.5 fget_light 2633 0.1406 vmlinux-2.6.21.5 cache_alloc_refill 2587 0.1382 vmlinux-2.6.21.5 dbg_redzone1 2477 0.1323 oprofile (no symbols) 2416 0.1290 vmlinux-2.6.21.5 cache_flusharray 2342 0.1251 vmlinux-2.6.21.5 ip_route_input 2263 0.1209 vmlinux-2.6.21.5 skb_make_writable 2156 0.1152 vmlinux-2.6.21.5 tcp_poll 2141 0.1144 vmlinux-2.6.21.5 local_bh_enable_ip 2119 0.1132 vmlinux-2.6.21.5 tcp_ack 2074 0.1108 vmlinux-2.6.21.5 _decode_session4 2029 0.1084 vmlinux-2.6.21.5 find_get_page 1963 0.1049 vmlinux-2.6.21.5 isolate_lru_pages 1943 0.1038 vmlinux-2.6.21.5 mask_and_ack_8259A 1907 0.1019 vmlinux-2.6.21.5 sock_wfree 1837 0.0981 vmlinux-2.6.21.5 ip_rcv 1796 0.0959 vmlinux-2.6.21.5 kfree_skbmem 1765 0.0943 vmlinux-2.6.21.5 check_spinlock_acquired_node 1749 0.0934 vmlinux-2.6.21.5 __switch_to 1720 0.0919 vmlinux-2.6.21.5 tcp_rtt_estimator 1695 0.0905 vmlinux-2.6.21.5 sk_stream_write_space 1682 0.0898 vmlinux-2.6.21.5 csum_partial 1605 0.0857 vmlinux-2.6.21.5 __wake_up_bit 1511 0.0807 vmlinux-2.6.21.5 find_busiest_group 1494 0.0798 vmlinux-2.6.21.5 pfifo_fast_dequeue 1492 0.0797 vmlinux-2.6.21.5 local_bh_enable 1472 0.0786 vmlinux-2.6.21.5 blk_recount_segments 1449 0.0774 vmlinux-2.6.21.5 __mod_zone_page_state 1430 0.0764 vmlinux-2.6.21.5 copy_from_user 1410 0.0753 vmlinux-2.6.21.5 pfifo_fast_enqueue 1354 0.0723 vmlinux-2.6.21.5 do_readv_writev 1350 0.0721 vmlinux-2.6.21.5 netif_receive_skb 1328 0.0709 vmlinux-2.6.21.5 xfrm_decode_session 1317 0.0703 vmlinux-2.6.21.5 do_gettimeofday 1302 0.0695 vmlinux-2.6.21.5 nf_proto_csum_replace4 1273 0.0680 vmlinux-2.6.21.5 transfer_objects 1265 0.0676 vmlinux-2.6.21.5 find_next_bit 1255 0.0670 vmlinux-2.6.21.5 atomic_notifier_call_chain 1224 0.0654 vmlinux-2.6.21.5 dbg_redzone2 1217 0.0650 vmlinux-2.6.21.5 ip_xfrm_me_harder 1181 0.0631 vmlinux-2.6.21.5 tcp_sacktag_write_queue 1150 0.0614 vmlinux-2.6.21.5 bictcp_cong_avoid 1148 0.0613 vmlinux-2.6.21.5 free_pages_bulk 1136 0.0607 vmlinux-2.6.21.5 dbg_userword 1119 0.0598 vmlinux-2.6.21.5 system_call 1118 0.0597 vmlinux-2.6.21.5 cache_grow 1096 0.0585 vmlinux-2.6.21.5 try_to_wake_up 1094 0.0584 vmlinux-2.6.21.5 __mod_timer 1086 0.0580 vmlinux-2.6.21.5 skb_copy 1082 0.0578 vmlinux-2.6.21.5 _raw_read_unlock 1078 0.0576 vmlinux-2.6.21.5 __rmqueue 1040 0.0556 vmlinux-2.6.21.5 ip_local_deliver 1021 0.0545 vmlinux-2.6.21.5 xfrm_lookup 1017 0.0543 vmlinux-2.6.21.5 free_hot_cold_page 1003 0.0536 vmlinux-2.6.21.5 tcp_mtu_probe 953 0.0509 vmlinux-2.6.21.5 default_idle 943 0.0504 vmlinux-2.6.21.5 kunmap_atomic 942 0.0503 vmlinux-2.6.21.5 do_generic_mapping_read 941 0.0503 vmlinux-2.6.21.5 __first_cpu 930 0.0497 vmlinux-2.6.21.5 shrink_page_list 923 0.0493 vmlinux-2.6.21.5 kfree_debugcheck 913 0.0488 vmlinux-2.6.21.5 kmap_atomic 902 0.0482 vmlinux-2.6.21.5 free_block 896 0.0479 vmlinux-2.6.21.5 restore_nocheck 893 0.0477 vmlinux-2.6.21.5 __bio_add_page 875 0.0467 vmlinux-2.6.21.5 tcp_init_tso_segs 867 0.0463 vmlinux-2.6.21.5 tcp_current_mss 859 0.0459 vmlinux-2.6.21.5 radix_tree_lookup 835 0.0446 vmlinux-2.6.21.5 add_to_page_cache 834 0.0445 httpd apr_poll 832 0.0444 vmlinux-2.6.21.5 eth_type_trans 823 0.0440 vmlinux-2.6.21.5 do_mpage_readpage 816 0.0436 vmlinux-2.6.21.5 slab_put_obj 811 0.0433 vmlinux-2.6.21.5 _raw_write_unlock 785 0.0419 sunrpc (no symbols) 715 0.0382 vmlinux-2.6.21.5 _raw_read_trylock 707 0.0378 vmlinux-2.6.21.5 buffered_rmqueue 706 0.0377 vmlinux-2.6.21.5 __d_lookup 694 0.0371 vmlinux-2.6.21.5 do_poll 680 0.0363 vmlinux-2.6.21.5 radix_tree_insert 678 0.0362 httpd apr_socket_sendv 654 0.0349 vmlinux-2.6.21.5 rmqueue_bulk 612 0.0327 vmlinux-2.6.21.5 recalc_task_prio 590 0.0315 vmlinux-2.6.21.5 local_bh_disable 587 0.0314 httpd blowfishEncrypt 583 0.0311 vmlinux-2.6.21.5 remove_mapping 576 0.0308 httpd writev_it_all 573 0.0306 vmlinux-2.6.21.5 do_sys_poll 572 0.0306 vmlinux-2.6.21.5 kmem_flagcheck 567 0.0303 vmlinux-2.6.21.5 internal_add_timer 566 0.0302 vmlinux-2.6.21.5 irq_entries_start 564 0.0301 vmlinux-2.6.21.5 __link_path_walk 561 0.0300 vmlinux-2.6.21.5 mpage_end_io_read 547 0.0292 vmlinux-2.6.21.5 notifier_call_chain 539 0.0288 vmlinux-2.6.21.5 __activate_task 537 0.0287 vmlinux-2.6.21.5 tg3_set_txd 530 0.0283 vmlinux-2.6.21.5 prep_new_page 522 0.0279 vmlinux-2.6.21.5 radix_tree_delete 521 0.0278 vmlinux-2.6.21.5 _write_lock_bh 515 0.0275 vmlinux-2.6.21.5 __do_page_cache_readahead 515 0.0275 vmlinux-2.6.21.5 _read_lock_irqsave 511 0.0273 vmlinux-2.6.21.5 tcp_fastretrans_alert 503 0.0269 vmlinux-2.6.21.5 schedule_timeout 491 0.0262 vmlinux-2.6.21.5 _raw_write_trylock 491 0.0262 vmlinux-2.6.21.5 get_task_mm 485 0.0259 vmlinux-2.6.21.5 effective_prio 485 0.0259 vmlinux-2.6.21.5 xfrm_policy_get_afinfo 483 0.0258 vmlinux-2.6.21.5 blk_rq_map_sg 481 0.0257 vmlinux-2.6.21.5 enable_8259A_irq 478 0.0255 vmlinux-2.6.21.5 twa_interrupt 472 0.0252 vmlinux-2.6.21.5 __tcp_push_pending_frames 472 0.0252 vmlinux-2.6.21.5 move_tasks 464 0.0248 vmlinux-2.6.21.5 find_vma 454 0.0242 vmlinux-2.6.21.5 number 451 0.0241 httpd core_output_filter 450 0.0240 vmlinux-2.6.21.5 file_read_actor 449 0.0240 vmlinux-2.6.21.5 twa_scsiop_execute_scsi 446 0.0238 vmlinux-2.6.21.5 unlock_page 442 0.0236 vmlinux-2.6.21.5 _read_lock_bh 431 0.0230 vmlinux-2.6.21.5 get_page_from_freelist 430 0.0230 vmlinux-2.6.21.5 sched_clock 413 0.0221 vmlinux-2.6.21.5 mempool_free 412 0.0220 vmlinux-2.6.21.5 free_poll_entry 407 0.0217 httpd apr_wait_for_io_or_timeout 406 0.0217 vmlinux-2.6.21.5 dm_any_congested 402 0.0215 vmlinux-2.6.21.5 tcp_v4_do_rcv 394 0.0210 vmlinux-2.6.21.5 tcp_cwnd_validate 390 0.0208 vmlinux-2.6.21.5 dnotify_parent 386 0.0206 vmlinux-2.6.21.5 do_sync_readv_writev 386 0.0206 vmlinux-2.6.21.5 tg3_tx 377 0.0201 vmlinux-2.6.21.5 tcp_select_window 376 0.0201 vmlinux-2.6.21.5 resched_task 375 0.0200 vmlinux-2.6.21.5 reschedule_interrupt 361 0.0193 vmlinux-2.6.21.5 tcp_check_space 355 0.0190 vmlinux-2.6.21.5 lock_timer_base 352 0.0188 vmlinux-2.6.21.5 sk_stream_wait_memory 348 0.0186 vmlinux-2.6.21.5 page_address 339 0.0181 vmlinux-2.6.21.5 generic_make_request 331 0.0177 vmlinux-2.6.21.5 __pagevec_lru_add 330 0.0176 vmlinux-2.6.21.5 dm_table_any_congested 330 0.0176 vmlinux-2.6.21.5 syscall_exit 330 0.0176 vmlinux-2.6.21.5 task_rq_lock 328 0.0175 vmlinux-2.6.21.5 tcp_v4_send_check 323 0.0173 vmlinux-2.6.21.5 cache_init_objs 316 0.0169 vmlinux-2.6.21.5 dm_request 314 0.0168 vmlinux-2.6.21.5 tg3_recycle_rx 313 0.0167 vmlinux-2.6.21.5 dec_pending 313 0.0167 vmlinux-2.6.21.5 tcp_enter_loss 310 0.0166 vmlinux-2.6.21.5 update_send_head 309 0.0165 vmlinux-2.6.21.5 dm_get_table 307 0.0164 vmlinux-2.6.21.5 cond_resched 305 0.0163 vmlinux-2.6.21.5 mod_zone_page_state 300 0.0160 vmlinux-2.6.21.5 disk_round_stats 298 0.0159 vmlinux-2.6.21.5 tcp_event_data_sent 297 0.0159 vmlinux-2.6.21.5 try_to_del_timer_sync 296 0.0158 vmlinux-2.6.21.5 rt_hash_code 292 0.0156 vmlinux-2.6.21.5 tcp_new_space 290 0.0155 vmlinux-2.6.21.5 dummy_file_permission 286 0.0153 vmlinux-2.6.21.5 slab_get_obj 285 0.0152 vmlinux-2.6.21.5 check_spinlock_acquired 272 0.0145 vmlinux-2.6.21.5 zone_watermark_ok 268 0.0143 vmlinux-2.6.21.5 bio_alloc_bioset 267 0.0143 vmlinux-2.6.21.5 __inc_zone_state 262 0.0140 vmlinux-2.6.21.5 vsnprintf 258 0.0138 vmlinux-2.6.21.5 tg3_write32 256 0.0137 vmlinux-2.6.21.5 mempool_alloc 255 0.0136 vmlinux-2.6.21.5 __netdev_alloc_skb 251 0.0134 vmlinux-2.6.21.5 hrtimer_get_softirq_time 250 0.0134 vmlinux-2.6.21.5 tcp_should_expand_sndbuf 249 0.0133 httpd ap_content_length_filter 244 0.0130 vmlinux-2.6.21.5 do_wait 242 0.0129 vmlinux-2.6.21.5 prepare_to_wait 239 0.0128 vmlinux-2.6.21.5 sys_writev 237 0.0127 vmlinux-2.6.21.5 load_balance_newidle 234 0.0125 vmlinux-2.6.21.5 tcp_ack_update_window 232 0.0124 nfs (no symbols) 229 0.0122 vmlinux-2.6.21.5 release_pages 228 0.0122 vmlinux-2.6.21.5 __dec_zone_state 228 0.0122 vmlinux-2.6.21.5 apic_timer_interrupt 228 0.0122 vmlinux-2.6.21.5 xtSearch 226 0.0121 vmlinux-2.6.21.5 mpage_readpages 222 0.0119 vmlinux-2.6.21.5 mod_timer 221 0.0118 vmlinux-2.6.21.5 put_page 218 0.0116 httpd ap_pass_brigade 216 0.0115 vmlinux-2.6.21.5 dequeue_task 207 0.0111 vmlinux-2.6.21.5 tcp_build_and_update_options 206 0.0110 vmlinux-2.6.21.5 __bio_clone 203 0.0108 httpd buffer_output 200 0.0107 bash (no symbols) 200 0.0107 vmlinux-2.6.21.5 sock_poll 198 0.0106 vmlinux-2.6.21.5 __alloc_pages 194 0.0104 vmlinux-2.6.21.5 sd_init_command 194 0.0104 vmlinux-2.6.21.5 tcp_retransmit_skb 193 0.0103 vmlinux-2.6.21.5 memcpy 192 0.0103 vmlinux-2.6.21.5 __split_bio 191 0.0102 httpd swapu32 190 0.0101 vmlinux-2.6.21.5 __wake_up 189 0.0101 vmlinux-2.6.21.5 as_dispatch_request 189 0.0101 vmlinux-2.6.21.5 idle_balance 186 0.0099 vmlinux-2.6.21.5 as_completed_request 186 0.0099 vmlinux-2.6.21.5 tcp_ack_no_tstamp 184 0.0098 vmlinux-2.6.21.5 find_pid 182 0.0097 httpd apr_bucket_destroy_noop 180 0.0096 vmlinux-2.6.21.5 do_sock_write 178 0.0095 vmlinux-2.6.21.5 vfs_writev 177 0.0095 vmlinux-2.6.21.5 mempool_alloc_slab 177 0.0095 vmlinux-2.6.21.5 poll_freewait 176 0.0094 vmlinux-2.6.21.5 sys_times 175 0.0093 vmlinux-2.6.21.5 __sk_dst_check 175 0.0093 vmlinux-2.6.21.5 activate_task 175 0.0093 vmlinux-2.6.21.5 dev_hard_start_xmit 175 0.0093 vmlinux-2.6.21.5 inet_sendmsg 174 0.0093 httpd get_revent 172 0.0092 vmlinux-2.6.21.5 tcp_ack_packets_out 169 0.0090 vmlinux-2.6.21.5 __pagevec_release_nonlru 168 0.0090 vmlinux-2.6.21.5 nf_ip_checksum 167 0.0089 vmlinux-2.6.21.5 tcp_mark_head_lost 166 0.0089 vmlinux-2.6.21.5 scsi_decide_disposition 165 0.0088 vmlinux-2.6.21.5 vfs_read 163 0.0087 vmlinux-2.6.21.5 twa_post_command_packet 162 0.0087 vmlinux-2.6.21.5 kmem_getpages 161 0.0086 vmlinux-2.6.21.5 enqueue_task 161 0.0086 vmlinux-2.6.21.5 sk_reset_timer 159 0.0085 vmlinux-2.6.21.5 kmem_freepages 158 0.0084 httpd apr_brigade_destroy 157 0.0084 vmlinux-2.6.21.5 sys_poll 154 0.0082 vmlinux-2.6.21.5 do_task_stat 153 0.0082 httpd apr_brigade_cleanup 152 0.0081 vmlinux-2.6.21.5 tcp_xmit_retransmit_queue 149 0.0080 vmlinux-2.6.21.5 linear_map 148 0.0079 vmlinux-2.6.21.5 jfs_get_block 148 0.0079 vmlinux-2.6.21.5 max_io_len 147 0.0079 httpd handleRequest 147 0.0079 vmlinux-2.6.21.5 _atomic_dec_and_lock 147 0.0079 vmlinux-2.6.21.5 tcp_data_queue 146 0.0078 vmlinux-2.6.21.5 __make_request 146 0.0078 vmlinux-2.6.21.5 clone_endio 145 0.0077 httpd perform_idle_server_maintenance 145 0.0077 vmlinux-2.6.21.5 alloc_slabmgmt 145 0.0077 vmlinux-2.6.21.5 tg3_poll 144 0.0077 vmlinux-2.6.21.5 tcp_time_to_recover 144 0.0077 vmlinux-2.6.21.5 twa_map_scsi_sg_data 143 0.0076 vmlinux-2.6.21.5 release_sock 142 0.0076 httpd apr_pool_clear 142 0.0076 vmlinux-2.6.21.5 dm_table_find_target 142 0.0076 vmlinux-2.6.21.5 scsi_request_fn 139 0.0074 vmlinux-2.6.21.5 __wake_up_common 139 0.0074 vmlinux-2.6.21.5 jiffies_read 139 0.0074 vmlinux-2.6.21.5 tcp_try_to_open 138 0.0074 vmlinux-2.6.21.5 down_read 133 0.0071 vmlinux-2.6.21.5 bictcp_acked 131 0.0070 vmlinux-2.6.21.5 twa_scsiop_execute_scsi_complete 130 0.0069 vmlinux-2.6.21.5 ip_append_data 130 0.0069 vmlinux-2.6.21.5 scsi_dispatch_cmd 130 0.0069 vmlinux-2.6.21.5 sys_read 130 0.0069 vmlinux-2.6.21.5 tcp_rcv_state_process 129 0.0069 vmlinux-2.6.21.5 __remove_from_page_cache 129 0.0069 vmlinux-2.6.21.5 run_timer_softirq 128 0.0068 vmlinux-2.6.21.5 generic_file_aio_read 126 0.0067 vmlinux-2.6.21.5 as_update_iohist 123 0.0066 vmlinux-2.6.21.5 rw_copy_check_uvector 122 0.0065 vmlinux-2.6.21.5 page_waitqueue 122 0.0065 vmlinux-2.6.21.5 tcp_snd_test 120 0.0064 vmlinux-2.6.21.5 dm_table_put 119 0.0064 vmlinux-2.6.21.5 poll_initwait 118 0.0063 httpd apr_brigade_write 117 0.0062 vmlinux-2.6.21.5 generic_unplug_device 114 0.0061 vmlinux-2.6.21.5 __end_that_request_first 113 0.0060 httpd apr_vformatter 113 0.0060 vmlinux-2.6.21.5 lock_sock_nested 112 0.0060 httpd __divdi3 112 0.0060 httpd apr_pool_cleanup_kill 112 0.0060 vmlinux-2.6.21.5 softlockup_tick 112 0.0060 vmlinux-2.6.21.5 twa_check_bits 111 0.0059 vmlinux-2.6.21.5 cpu_idle 110 0.0059 vmlinux-2.6.21.5 as_queue_empty 109 0.0058 vmlinux-2.6.21.5 collect_sigign_sigcatch 108 0.0058 vmlinux-2.6.21.5 dm_unplug_all 108 0.0058 vmlinux-2.6.21.5 inotify_inode_queue_event 104 0.0056 vmlinux-2.6.21.5 try_atomic_semop 103 0.0055 vmlinux-2.6.21.5 hrtimer_run_queues 102 0.0054 vmlinux-2.6.21.5 radix_tree_preload 102 0.0054 vmlinux-2.6.21.5 tcp_fast_parse_options 100 0.0053 vmlinux-2.6.21.5 xtLookup 99 0.0053 vmlinux-2.6.21.5 groups_search 98 0.0052 vmlinux-2.6.21.5 dput 97 0.0052 vmlinux-2.6.21.5 __net_timestamp 97 0.0052 vmlinux-2.6.21.5 do_lookup 97 0.0052 vmlinux-2.6.21.5 profile_tick 97 0.0052 vmlinux-2.6.21.5 slab_destroy_objs 95 0.0051 vmlinux-2.6.21.5 generic_permission 94 0.0050 vmlinux-2.6.21.5 end_io_acct 94 0.0050 vmlinux-2.6.21.5 net_rx_action 94 0.0050 vmlinux-2.6.21.5 run_posix_cpu_timers 94 0.0050 vmlinux-2.6.21.5 touch_atime 93 0.0050 vmlinux-2.6.21.5 task_running_tick 92 0.0049 vmlinux-2.6.21.5 cubic_root 91 0.0049 vmlinux-2.6.21.5 remove_wait_queue 91 0.0049 vmlinux-2.6.21.5 tick_handle_periodic 90 0.0048 vmlinux-2.6.21.5 tcp_recvmsg 89 0.0048 ld-2.3.2.so (no symbols) 89 0.0048 vmlinux-2.6.21.5 bio_add_page 89 0.0048 vmlinux-2.6.21.5 elv_next_request 89 0.0048 vmlinux-2.6.21.5 idle_cpu 89 0.0048 vmlinux-2.6.21.5 scheduler_tick 89 0.0048 vmlinux-2.6.21.5 send_IPI_mask_bitmask 89 0.0048 vmlinux-2.6.21.5 stripe_map 89 0.0048 vmlinux-2.6.21.5 up_read 88 0.0047 vmlinux-2.6.21.5 add_timer_randomness 88 0.0047 vmlinux-2.6.21.5 strncpy_from_user 87 0.0046 vmlinux-2.6.21.5 udp_sendmsg 86 0.0046 vmlinux-2.6.21.5 deactivate_task 86 0.0046 vmlinux-2.6.21.5 finish_wait 85 0.0045 httpd get_event 85 0.0045 vmlinux-2.6.21.5 rw_verify_area 85 0.0045 vmlinux-2.6.21.5 tcp_parse_options 84 0.0045 vmlinux-2.6.21.5 __mutex_lock_slowpath 83 0.0044 vmlinux-2.6.21.5 account_system_time 83 0.0044 vmlinux-2.6.21.5 get_unused_fd 83 0.0044 vmlinux-2.6.21.5 tcp_urg 81 0.0043 httpd blowfishSetup 81 0.0043 vmlinux-2.6.21.5 __clone_and_map 81 0.0043 vmlinux-2.6.21.5 sock_aio_write 80 0.0043 vmlinux-2.6.21.5 dm_bio_destructor 80 0.0043 vmlinux-2.6.21.5 do_select 80 0.0043 vmlinux-2.6.21.5 fput 79 0.0042 httpd apr_bucket_free 79 0.0042 vmlinux-2.6.21.5 __do_softirq 79 0.0042 vmlinux-2.6.21.5 inotify_dentry_parent_queue_event 78 0.0042 httpd ap_directory_walk 78 0.0042 vmlinux-2.6.21.5 bio_endio 78 0.0042 vmlinux-2.6.21.5 permission 78 0.0042 vmlinux-2.6.21.5 twa_get_request_id 76 0.0041 vmlinux-2.6.21.5 as_close_req 76 0.0041 vmlinux-2.6.21.5 do_sync_read 75 0.0040 vmlinux-2.6.21.5 can_migrate_task 75 0.0040 vmlinux-2.6.21.5 inet_sk_rebuild_header 75 0.0040 vmlinux-2.6.21.5 sys_close 75 0.0040 vmlinux-2.6.21.5 tcp_cong_avoid 73 0.0039 vmlinux-2.6.21.5 ret_from_intr 72 0.0038 httpd status_handler 72 0.0038 vmlinux-2.6.21.5 __next_cpu 72 0.0038 vmlinux-2.6.21.5 dm_table_unplug_all 72 0.0038 vmlinux-2.6.21.5 put_io_context 71 0.0038 vmlinux-2.6.21.5 __block_prepare_write 70 0.0037 vmlinux-2.6.21.5 as_update_seekdist 70 0.0037 vmlinux-2.6.21.5 mark_page_accessed 70 0.0037 vmlinux-2.6.21.5 scsi_io_completion 69 0.0037 vmlinux-2.6.21.5 clone_bio 69 0.0037 vmlinux-2.6.21.5 scsi_softirq_done 69 0.0037 vmlinux-2.6.21.5 shrink_active_list 68 0.0036 httpd apr_palloc 68 0.0036 vmlinux-2.6.21.5 rcu_pending 68 0.0036 vmlinux-2.6.21.5 tcp_write_wakeup 67 0.0036 libproc.so.3.2.1 (no symbols) 67 0.0036 vmlinux-2.6.21.5 default_wake_function 67 0.0036 vmlinux-2.6.21.5 tcp_write_timer 66 0.0035 vmlinux-2.6.21.5 del_timer 65 0.0035 vmlinux-2.6.21.5 __get_metapage 65 0.0035 vmlinux-2.6.21.5 do_path_lookup 64 0.0034 httpd ap_rwrite 64 0.0034 vmlinux-2.6.21.5 datagram_poll 64 0.0034 vmlinux-2.6.21.5 elv_insert 64 0.0034 vmlinux-2.6.21.5 scsi_done 64 0.0034 vmlinux-2.6.21.5 worker_thread 63 0.0034 vmlinux-2.6.21.5 ext3_do_update_inode 62 0.0033 vmlinux-2.6.21.5 get_request 62 0.0033 vmlinux-2.6.21.5 update_process_times 61 0.0033 vmlinux-2.6.21.5 _write_lock_irqsave 61 0.0033 vmlinux-2.6.21.5 as_get_io_context 61 0.0033 vmlinux-2.6.21.5 blk_complete_request 61 0.0033 vmlinux-2.6.21.5 tick_periodic 60 0.0032 httpd ap_merge_per_dir_configs 60 0.0032 vmlinux-2.6.21.5 __generic_file_aio_write_nolock 60 0.0032 vmlinux-2.6.21.5 do_get_write_access 60 0.0032 vmlinux-2.6.21.5 sd_rw_intr 60 0.0032 vmlinux-2.6.21.5 udp_queue_rcv_skb 59 0.0032 httpd match 59 0.0032 vmlinux-2.6.21.5 as_can_break_anticipation 59 0.0032 vmlinux-2.6.21.5 slab_map_pages 59 0.0032 vmlinux-2.6.21.5 twa_scsi_queue 58 0.0031 vmlinux-2.6.21.5 __rcu_pending 58 0.0031 vmlinux-2.6.21.5 add_wait_queue 58 0.0031 vmlinux-2.6.21.5 queue_work 58 0.0031 vmlinux-2.6.21.5 slab_destroy 58 0.0031 vmlinux-2.6.21.5 tcp_retransmit_timer 57 0.0030 httpd blowfishDecrypt 57 0.0030 vmlinux-2.6.21.5 as_choose_req 57 0.0030 vmlinux-2.6.21.5 kref_get 57 0.0030 vmlinux-2.6.21.5 rcu_check_callbacks 57 0.0030 vmlinux-2.6.21.5 scsi_prep_fn 56 0.0030 httpd run_cleanups 55 0.0029 vmlinux-2.6.21.5 __udp4_lib_lookup 55 0.0029 vmlinux-2.6.21.5 debug_mutex_lock_common 54 0.0029 vmlinux-2.6.21.5 __mark_inode_dirty 54 0.0029 vmlinux-2.6.21.5 __sk_stream_mem_reclaim 54 0.0029 vmlinux-2.6.21.5 rb_erase 53 0.0028 httpd merge_core_dir_configs 53 0.0028 vmlinux-2.6.21.5 blockable_page_cache_readahead 53 0.0028 vmlinux-2.6.21.5 generic_file_buffered_write 53 0.0028 vmlinux-2.6.21.5 ip_push_pending_frames 53 0.0028 vmlinux-2.6.21.5 journal_add_journal_head 52 0.0028 httpd pcre_exec 52 0.0028 vmlinux-2.6.21.5 as_may_queue 52 0.0028 vmlinux-2.6.21.5 dm_table_get_devices 52 0.0028 vmlinux-2.6.21.5 page_referenced 52 0.0028 vmlinux-2.6.21.5 twa_free_request_id 51 0.0027 vmlinux-2.6.21.5 call_rcu 51 0.0027 vmlinux-2.6.21.5 fn_hash_lookup 50 0.0027 httpd apr_table_get 50 0.0027 vmlinux-2.6.21.5 __dentry_open 50 0.0027 vmlinux-2.6.21.5 as_put_io_context 50 0.0027 vmlinux-2.6.21.5 do_wp_page 50 0.0027 vmlinux-2.6.21.5 sk_stream_mem_schedule 50 0.0027 vmlinux-2.6.21.5 update_load 49 0.0026 vmlinux-2.6.21.5 kref_put 49 0.0026 vmlinux-2.6.21.5 scsi_device_unbusy 48 0.0026 vmlinux-2.6.21.5 __ip_route_output_key 48 0.0026 vmlinux-2.6.21.5 note_interrupt 48 0.0026 vmlinux-2.6.21.5 scsi_add_timer 48 0.0026 vmlinux-2.6.21.5 start_this_handle 48 0.0026 vmlinux-2.6.21.5 tcp_cwnd_down 48 0.0026 vmlinux-2.6.21.5 tcp_may_send_now 48 0.0026 vmlinux-2.6.21.5 udp_poll 47 0.0025 vmlinux-2.6.21.5 __inc_zone_page_state 47 0.0025 vmlinux-2.6.21.5 __udp4_lib_rcv 47 0.0025 vmlinux-2.6.21.5 as_add_request 47 0.0025 vmlinux-2.6.21.5 blk_remove_plug 47 0.0025 vmlinux-2.6.21.5 dtSearch 47 0.0025 vmlinux-2.6.21.5 kobject_get 47 0.0025 vmlinux-2.6.21.5 page_cache_readahead 47 0.0025 vmlinux-2.6.21.5 smp_apic_timer_interrupt 47 0.0025 vmlinux-2.6.21.5 submit_bio 46 0.0025 vmlinux-2.6.21.5 __fput 46 0.0025 vmlinux-2.6.21.5 as_remove_queued_request 46 0.0025 vmlinux-2.6.21.5 blk_run_queue 46 0.0025 vmlinux-2.6.21.5 debug_mutex_unlock 46 0.0025 vmlinux-2.6.21.5 init_timer 46 0.0025 vmlinux-2.6.21.5 may_open 46 0.0025 vmlinux-2.6.21.5 rb_next 46 0.0025 vmlinux-2.6.21.5 tasklet_action 45 0.0024 vmlinux-2.6.21.5 clear_bdi_congested 45 0.0024 vmlinux-2.6.21.5 journal_stop 45 0.0024 vmlinux-2.6.21.5 rb_insert_color 45 0.0024 vmlinux-2.6.21.5 run_local_timers 44 0.0024 vmlinux-2.6.21.5 as_fifo_expired 44 0.0024 vmlinux-2.6.21.5 dm_disk 44 0.0024 vmlinux-2.6.21.5 handle_level_irq 44 0.0024 vmlinux-2.6.21.5 tcp_write_timeout 43 0.0023 vmlinux-2.6.21.5 __queue_work 43 0.0023 vmlinux-2.6.21.5 blk_do_ordered 43 0.0023 vmlinux-2.6.21.5 exec_permission_lite 43 0.0023 vmlinux-2.6.21.5 journal_put_journal_head 43 0.0023 vmlinux-2.6.21.5 ordered_bio_endio 43 0.0023 vmlinux-2.6.21.5 rq_init 42 0.0022 httpd ap_update_child_status_from_indexes 42 0.0022 vmlinux-2.6.21.5 bio_fs_destructor 42 0.0022 vmlinux-2.6.21.5 get_empty_filp 42 0.0022 vmlinux-2.6.21.5 page_fault 42 0.0022 vmlinux-2.6.21.5 scsi_finish_command 42 0.0022 vmlinux-2.6.21.5 scsi_init_io 42 0.0022 vmlinux-2.6.21.5 static_prio_timeslice 42 0.0022 vmlinux-2.6.21.5 sys_socketcall 41 0.0022 httpd apr_bucket_transient_create 41 0.0022 httpd apr_sendv 41 0.0022 libcrypto.so.0.9.7 (no symbols) 41 0.0022 vmlinux-2.6.21.5 __pollwait 41 0.0022 vmlinux-2.6.21.5 __rcu_process_callbacks 41 0.0022 vmlinux-2.6.21.5 link_path_walk 41 0.0022 vmlinux-2.6.21.5 run_workqueue 41 0.0022 vmlinux-2.6.21.5 start_io_acct 40 0.0021 vmlinux-2.6.21.5 bio_get_nr_vecs 40 0.0021 vmlinux-2.6.21.5 blk_done_softirq 40 0.0021 vmlinux-2.6.21.5 eligible_child 40 0.0021 vmlinux-2.6.21.5 scsi_get_cmd_from_req 40 0.0021 vmlinux-2.6.21.5 zap_pte_range 39 0.0021 vmlinux-2.6.21.5 activate_page 39 0.0021 vmlinux-2.6.21.5 ext3_get_inode_block 39 0.0021 vmlinux-2.6.21.5 radix_tree_node_alloc 39 0.0021 vmlinux-2.6.21.5 raise_softirq 39 0.0021 vmlinux-2.6.21.5 scsi_end_request 38 0.0020 httpd listener_thread 38 0.0020 vmlinux-2.6.21.5 elv_dispatch_sort 38 0.0020 vmlinux-2.6.21.5 lookup_mnt 38 0.0020 vmlinux-2.6.21.5 update_wall_time 37 0.0020 vmlinux-2.6.21.5 __follow_mount 37 0.0020 vmlinux-2.6.21.5 __scsi_done 37 0.0020 vmlinux-2.6.21.5 as_update_rq 37 0.0020 vmlinux-2.6.21.5 del_timer_sync 37 0.0020 vmlinux-2.6.21.5 end_that_request_last 37 0.0020 vmlinux-2.6.21.5 ext3_permission 37 0.0020 vmlinux-2.6.21.5 journal_dirty_metadata 37 0.0020 vmlinux-2.6.21.5 resume_kernel 36 0.0019 vmlinux-2.6.21.5 do_no_page 36 0.0019 vmlinux-2.6.21.5 handle_IRQ_event 36 0.0019 vmlinux-2.6.21.5 jfs_readpages 36 0.0019 vmlinux-2.6.21.5 lookup_bh_lru 36 0.0019 vmlinux-2.6.21.5 mntput_no_expire 36 0.0019 vmlinux-2.6.21.5 mpage_alloc 36 0.0019 vmlinux-2.6.21.5 mutex_remove_waiter 36 0.0019 vmlinux-2.6.21.5 netif_rx 36 0.0019 vmlinux-2.6.21.5 percpu_counter_mod 36 0.0019 vmlinux-2.6.21.5 release_metapage 36 0.0019 vmlinux-2.6.21.5 scsi_get_command 36 0.0019 vmlinux-2.6.21.5 scsi_setup_fs_cmnd 36 0.0019 vmlinux-2.6.21.5 sigprocmask 35 0.0019 cronolog main 35 0.0019 libdns.so.11.1.3 (no symbols) 35 0.0019 vmlinux-2.6.21.5 core_sys_select 35 0.0019 vmlinux-2.6.21.5 shrink_inactive_list 35 0.0019 vmlinux-2.6.21.5 tcp_update_scoreboard 34 0.0018 httpd apr_bucket_alloc 34 0.0018 vmlinux-2.6.21.5 __free_pages 34 0.0018 vmlinux-2.6.21.5 __mutex_unlock_slowpath 34 0.0018 vmlinux-2.6.21.5 __pagevec_free 34 0.0018 vmlinux-2.6.21.5 in_group_p 34 0.0018 vmlinux-2.6.21.5 pid_revalidate 34 0.0018 vmlinux-2.6.21.5 radix_tree_node_ctor 34 0.0018 vmlinux-2.6.21.5 udp_push_pending_frames 33 0.0018 httpd apr_time_now 33 0.0018 ip_tables (no symbols) 33 0.0018 vmlinux-2.6.21.5 elv_rb_add 33 0.0018 vmlinux-2.6.21.5 scsi_init_cmd_errh 33 0.0018 vmlinux-2.6.21.5 scsi_log_completion 33 0.0018 vmlinux-2.6.21.5 sys_gettimeofday 32 0.0017 vmlinux-2.6.21.5 __generic_unplug_device 32 0.0017 vmlinux-2.6.21.5 as_move_to_dispatch 32 0.0017 vmlinux-2.6.21.5 dnotify_flush 32 0.0017 vmlinux-2.6.21.5 elv_queue_empty 32 0.0017 vmlinux-2.6.21.5 elv_rqhash_add 32 0.0017 vmlinux-2.6.21.5 irq_enter 31 0.0017 httpd ap_filter_flush 31 0.0017 vmlinux-2.6.21.5 as_update_thinktime 31 0.0017 vmlinux-2.6.21.5 ip_route_output_slow 31 0.0017 vmlinux-2.6.21.5 load_balance 31 0.0017 vmlinux-2.6.21.5 open_namei 30 0.0016 httpd add_any_filter_handle 30 0.0016 httpd apr_cpystrn 30 0.0016 httpd apr_table_setn 30 0.0016 vmlinux-2.6.21.5 init_request_from_bio 30 0.0016 vmlinux-2.6.21.5 memmove 30 0.0016 vmlinux-2.6.21.5 read_cache_page 30 0.0016 vmlinux-2.6.21.5 skb_copy_datagram_iovec 29 0.0015 httpd simple_bucket_read 29 0.0015 top (no symbols) 29 0.0015 vmlinux-2.6.21.5 __ext3_get_inode_loc 29 0.0015 vmlinux-2.6.21.5 do_sys_open 29 0.0015 vmlinux-2.6.21.5 find_busiest_queue 29 0.0015 vmlinux-2.6.21.5 pipe_read 29 0.0015 vmlinux-2.6.21.5 run_rebalance_domains 29 0.0015 vmlinux-2.6.21.5 scsi_log_send 28 0.0015 httpd ap_escape_logitem 28 0.0015 httpd apr_socket_opt_set 28 0.0015 httpd proc_mutex_sysv_acquire 28 0.0015 vmlinux-2.6.21.5 as_antic_stop 28 0.0015 vmlinux-2.6.21.5 as_find_next_rq 28 0.0015 vmlinux-2.6.21.5 current_io_context 28 0.0015 vmlinux-2.6.21.5 dm_table_get 28 0.0015 vmlinux-2.6.21.5 drain_array 28 0.0015 vmlinux-2.6.21.5 elv_completed_request 28 0.0015 vmlinux-2.6.21.5 futex_wake 28 0.0015 vmlinux-2.6.21.5 local_apic_timer_interrupt 28 0.0015 vmlinux-2.6.21.5 process_backlog 28 0.0015 vmlinux-2.6.21.5 sys_semtimedop 27 0.0014 httpd .plt 27 0.0014 httpd ap_getword_white 27 0.0014 vmlinux-2.6.21.5 __blk_put_request 27 0.0014 vmlinux-2.6.21.5 __lookup_mnt 27 0.0014 vmlinux-2.6.21.5 bictcp_recalc_ssthresh 27 0.0014 vmlinux-2.6.21.5 blk_queue_bounce 27 0.0014 vmlinux-2.6.21.5 ciCompare 27 0.0014 vmlinux-2.6.21.5 copy_from_dinode 27 0.0014 vmlinux-2.6.21.5 get_futex_key 27 0.0014 vmlinux-2.6.21.5 smp_reschedule_interrupt 26 0.0014 amd nfs_program_2 26 0.0014 httpd allocator_alloc 26 0.0014 httpd ap_parse_htaccess 26 0.0014 vmlinux-2.6.21.5 __handle_mm_fault 26 0.0014 vmlinux-2.6.21.5 __tasklet_schedule 26 0.0014 vmlinux-2.6.21.5 do_tcp_sendpages 26 0.0014 vmlinux-2.6.21.5 ext3_journal_start_sb 26 0.0014 vmlinux-2.6.21.5 find_lock_page 26 0.0014 vmlinux-2.6.21.5 journal_start 26 0.0014 vmlinux-2.6.21.5 pipe_write 26 0.0014 vmlinux-2.6.21.5 rcu_check_quiescent_state 26 0.0014 vmlinux-2.6.21.5 sockfd_lookup_light 26 0.0014 vmlinux-2.6.21.5 sys_open 26 0.0014 vmlinux-2.6.21.5 tcp_enter_cwr 25 0.0013 libncurses.so.5.4 (no symbols) 25 0.0013 vmlinux-2.6.21.5 __copy_user_intel_nocache 25 0.0013 vmlinux-2.6.21.5 __dec_zone_page_state 25 0.0013 vmlinux-2.6.21.5 d_alloc 25 0.0013 vmlinux-2.6.21.5 diRead 25 0.0013 vmlinux-2.6.21.5 get_device 25 0.0013 vmlinux-2.6.21.5 mmput 25 0.0013 vmlinux-2.6.21.5 recalc_sigpending_tsk 24 0.0013 httpd ap_read_request 24 0.0013 httpd apply_rewrite_rule 24 0.0013 httpd core_input_filter 24 0.0013 httpd proxy_detect 24 0.0013 vmlinux-2.6.21.5 autoremove_wake_function 24 0.0013 vmlinux-2.6.21.5 blk_alloc_request 24 0.0013 vmlinux-2.6.21.5 clocksource_adjust 24 0.0013 vmlinux-2.6.21.5 copy_to_user 24 0.0013 vmlinux-2.6.21.5 elv_rqhash_del 24 0.0013 vmlinux-2.6.21.5 fget 24 0.0013 vmlinux-2.6.21.5 find_next_zero_bit 24 0.0013 vmlinux-2.6.21.5 irq_exit 24 0.0013 vmlinux-2.6.21.5 path_release 24 0.0013 vmlinux-2.6.21.5 shrink_slab 23 0.0012 vmlinux-2.6.21.5 __scsi_get_command 23 0.0012 vmlinux-2.6.21.5 add_disk_randomness 23 0.0012 vmlinux-2.6.21.5 as_antic_expired 23 0.0012 vmlinux-2.6.21.5 bictcp_state 23 0.0012 vmlinux-2.6.21.5 debug_mutex_free_waiter 23 0.0012 vmlinux-2.6.21.5 do_IRQ 23 0.0012 vmlinux-2.6.21.5 do_softirq 23 0.0012 vmlinux-2.6.21.5 double_lock_balance 23 0.0012 vmlinux-2.6.21.5 fib_get_table 23 0.0012 vmlinux-2.6.21.5 ip_route_input_slow 23 0.0012 vmlinux-2.6.21.5 jiffies_to_clock_t 23 0.0012 vmlinux-2.6.21.5 udp_recvmsg 22 0.0012 httpd ap_location_walk 22 0.0012 httpd ap_process_request_internal 22 0.0012 libpthread-0.60.so read 22 0.0012 vmlinux-2.6.21.5 __freed_request 22 0.0012 vmlinux-2.6.21.5 as_activate_request 22 0.0012 vmlinux-2.6.21.5 bio_alloc 22 0.0012 vmlinux-2.6.21.5 bio_phys_segments 22 0.0012 vmlinux-2.6.21.5 ext3_prepare_write 22 0.0012 vmlinux-2.6.21.5 freed_request 22 0.0012 vmlinux-2.6.21.5 inet_csk_search_req 22 0.0012 vmlinux-2.6.21.5 inet_lookup_listener_slow 22 0.0012 vmlinux-2.6.21.5 ip_route_output_flow 22 0.0012 vmlinux-2.6.21.5 poll_get_entry 22 0.0012 vmlinux-2.6.21.5 setfl 22 0.0012 vmlinux-2.6.21.5 sys_ipc 22 0.0012 vmlinux-2.6.21.5 tcp_slow_start 21 0.0011 httpd apr_filepath_merge 21 0.0011 httpd apr_pool_create_ex 21 0.0011 httpd hook_uri2file 21 0.0011 vmlinux-2.6.21.5 __release_sock 21 0.0011 vmlinux-2.6.21.5 cache_reap 21 0.0011 vmlinux-2.6.21.5 current_fs_time 21 0.0011 vmlinux-2.6.21.5 do_mmap_pgoff 21 0.0011 vmlinux-2.6.21.5 ip_build_and_send_pkt 21 0.0011 vmlinux-2.6.21.5 mempool_free_slab 21 0.0011 vmlinux-2.6.21.5 nameidata_to_filp 21 0.0011 vmlinux-2.6.21.5 sock_alloc_send_pskb 21 0.0011 vmlinux-2.6.21.5 tcp_cleanup_rbuf 21 0.0011 vmlinux-2.6.21.5 tcp_make_synack 21 0.0011 vmlinux-2.6.21.5 wake_up_bit 20 0.0011 amd run_rpc 20 0.0011 httpd ap_rgetline_core 20 0.0011 libpthread-0.60.so __pthread_disable_asynccancel 20 0.0011 vmlinux-2.6.21.5 __elv_add_request 20 0.0011 vmlinux-2.6.21.5 __scsi_put_command 20 0.0011 vmlinux-2.6.21.5 do_filp_open 20 0.0011 vmlinux-2.6.21.5 filp_close 20 0.0011 vmlinux-2.6.21.5 free_pages 20 0.0011 vmlinux-2.6.21.5 scsi_put_command 20 0.0011 vmlinux-2.6.21.5 scsi_release_buffers 20 0.0011 vmlinux-2.6.21.5 tcp_close 20 0.0011 vmlinux-2.6.21.5 xfrm_policy_put_afinfo 19 0.0010 httpd apr_socket_recv 19 0.0010 libpthread-0.60.so __open_nocancel 19 0.0010 libpthread-0.60.so accept 19 0.0010 libpthread-0.60.so pthread_mutex_unlock 19 0.0010 vmlinux-2.6.21.5 __kmalloc 19 0.0010 vmlinux-2.6.21.5 d_instantiate 19 0.0010 vmlinux-2.6.21.5 do_futex 19 0.0010 vmlinux-2.6.21.5 dtCompare 19 0.0010 vmlinux-2.6.21.5 dummy_inode_permission 19 0.0010 vmlinux-2.6.21.5 fib_semantic_match 19 0.0010 vmlinux-2.6.21.5 jfs_lookup 19 0.0010 vmlinux-2.6.21.5 page_add_file_rmap 19 0.0010 vmlinux-2.6.21.5 rcu_process_callbacks 19 0.0010 vmlinux-2.6.21.5 scsi_run_queue 19 0.0010 vmlinux-2.6.21.5 sock_common_recvmsg 18 9.6e-04 amd do_select 18 9.6e-04 httpd allocator_free 18 9.6e-04 httpd ap_file_walk 18 9.6e-04 httpd ap_recent_rfc822_date 18 9.6e-04 httpd apr_file_write 18 9.6e-04 httpd regexec 18 9.6e-04 httpd worker_thread 18 9.6e-04 vmlinux-2.6.21.5 __ext3_journal_get_write_access 18 9.6e-04 vmlinux-2.6.21.5 __netif_rx_schedule 18 9.6e-04 vmlinux-2.6.21.5 bio_free 18 9.6e-04 vmlinux-2.6.21.5 bio_put 18 9.6e-04 vmlinux-2.6.21.5 bit_waitqueue 18 9.6e-04 vmlinux-2.6.21.5 copy_io_context 18 9.6e-04 vmlinux-2.6.21.5 debug_mutex_add_waiter 18 9.6e-04 vmlinux-2.6.21.5 elv_rb_del 18 9.6e-04 vmlinux-2.6.21.5 get_pid_task 18 9.6e-04 vmlinux-2.6.21.5 get_request_wait 18 9.6e-04 vmlinux-2.6.21.5 ip_generic_getfrag 18 9.6e-04 vmlinux-2.6.21.5 journal_dirty_data 18 9.6e-04 vmlinux-2.6.21.5 raise_softirq_irqoff 18 9.6e-04 vmlinux-2.6.21.5 rcu_do_batch 18 9.6e-04 vmlinux-2.6.21.5 read_pages 18 9.6e-04 vmlinux-2.6.21.5 sk_clone 18 9.6e-04 vmlinux-2.6.21.5 sys_futex 18 9.6e-04 vmlinux-2.6.21.5 sys_select 18 9.6e-04 vmlinux-2.6.21.5 sys_write 18 9.6e-04 vmlinux-2.6.21.5 tcp_ack_saw_tstamp 17 9.1e-04 httpd ap_discard_request_body 17 9.1e-04 httpd ap_get_mime_headers_core 17 9.1e-04 httpd apr_pool_cleanup_register 17 9.1e-04 httpd free_proc_chain 17 9.1e-04 httpd heap_bucket_destroy 17 9.1e-04 vmlinux-2.6.21.5 __path_lookup_intent_open 17 9.1e-04 vmlinux-2.6.21.5 alloc_sock_iocb 17 9.1e-04 vmlinux-2.6.21.5 common_interrupt 17 9.1e-04 vmlinux-2.6.21.5 double_rq_lock 17 9.1e-04 vmlinux-2.6.21.5 ext3_file_write 17 9.1e-04 vmlinux-2.6.21.5 ext3_ordered_commit_write 17 9.1e-04 vmlinux-2.6.21.5 futex_wait 17 9.1e-04 vmlinux-2.6.21.5 iput 17 9.1e-04 vmlinux-2.6.21.5 locks_remove_flock 17 9.1e-04 vmlinux-2.6.21.5 module_put 17 9.1e-04 vmlinux-2.6.21.5 profile_pc 17 9.1e-04 vmlinux-2.6.21.5 rb_prev 17 9.1e-04 vmlinux-2.6.21.5 recalc_sigpending 17 9.1e-04 vmlinux-2.6.21.5 restore_all 17 9.1e-04 vmlinux-2.6.21.5 sys_accept 17 9.1e-04 vmlinux-2.6.21.5 tcp_unhash 16 8.5e-04 httpd ap_byterange_filter 16 8.5e-04 httpd ap_process_http_connection 16 8.5e-04 httpd apr_array_push_noclear 16 8.5e-04 httpd apr_brigade_writev 16 8.5e-04 httpd cached_explode 16 8.5e-04 httpd check_dir_access 16 8.5e-04 httpd conv_10 16 8.5e-04 httpd find_ct 16 8.5e-04 libpthread-0.60.so __i686.get_pc_thunk.bx 16 8.5e-04 vmlinux-2.6.21.5 __inet_lookup_listener 16 8.5e-04 vmlinux-2.6.21.5 __journal_file_buffer 16 8.5e-04 vmlinux-2.6.21.5 _spin_lock_irqsave 16 8.5e-04 vmlinux-2.6.21.5 clocksource_get_next 16 8.5e-04 vmlinux-2.6.21.5 copy_pte_range 16 8.5e-04 vmlinux-2.6.21.5 dentry_iput 16 8.5e-04 vmlinux-2.6.21.5 down_write 16 8.5e-04 vmlinux-2.6.21.5 filemap_nopage 16 8.5e-04 vmlinux-2.6.21.5 find_extend_vma 16 8.5e-04 vmlinux-2.6.21.5 generic_file_aio_write 16 8.5e-04 vmlinux-2.6.21.5 generic_pipe_buf_pin 16 8.5e-04 vmlinux-2.6.21.5 getname 16 8.5e-04 vmlinux-2.6.21.5 jfs_permission 16 8.5e-04 vmlinux-2.6.21.5 journal_get_write_access 16 8.5e-04 vmlinux-2.6.21.5 kthread_should_stop 16 8.5e-04 vmlinux-2.6.21.5 loopback_xmit 16 8.5e-04 vmlinux-2.6.21.5 msecs_to_jiffies 16 8.5e-04 vmlinux-2.6.21.5 proc_fill_cache 16 8.5e-04 vmlinux-2.6.21.5 rt_intern_hash 16 8.5e-04 vmlinux-2.6.21.5 sock_def_wakeup 16 8.5e-04 vmlinux-2.6.21.5 sock_sendmsg 16 8.5e-04 vmlinux-2.6.21.5 tcp_push_one 16 8.5e-04 vmlinux-2.6.21.5 timespec_trunc 16 8.5e-04 vmlinux-2.6.21.5 wake_bit_function 15 8.0e-04 httpd ap_add_input_filter_handle 15 8.0e-04 httpd ap_get_remote_host 15 8.0e-04 httpd ap_getword_conf 15 8.0e-04 httpd ap_lingering_close 15 8.0e-04 httpd ap_unescape_url 15 8.0e-04 httpd apr_base64_decode_binary 15 8.0e-04 httpd apr_file_open 15 8.0e-04 httpd apr_snprintf 15 8.0e-04 httpd apr_table_set 15 8.0e-04 httpd net_time_filter 15 8.0e-04 httpd process_socket 15 8.0e-04 httpd ssl_hook_pre_connection 15 8.0e-04 vmlinux-2.6.21.5 __rb_erase_color 15 8.0e-04 vmlinux-2.6.21.5 __spin_lock_init 15 8.0e-04 vmlinux-2.6.21.5 do_page_fault 15 8.0e-04 vmlinux-2.6.21.5 elv_merge 15 8.0e-04 vmlinux-2.6.21.5 ext3_writepage_trans_blocks 15 8.0e-04 vmlinux-2.6.21.5 hweight32 15 8.0e-04 vmlinux-2.6.21.5 inet_csk_accept 15 8.0e-04 vmlinux-2.6.21.5 inode_init_once 15 8.0e-04 vmlinux-2.6.21.5 jfs_check_acl 15 8.0e-04 vmlinux-2.6.21.5 kobject_put 15 8.0e-04 vmlinux-2.6.21.5 locks_remove_posix 15 8.0e-04 vmlinux-2.6.21.5 make_ahead_window 15 8.0e-04 vmlinux-2.6.21.5 put_pid 15 8.0e-04 vmlinux-2.6.21.5 sock_def_write_space 15 8.0e-04 vmlinux-2.6.21.5 sock_from_file 15 8.0e-04 vmlinux-2.6.21.5 sys_recvmsg 15 8.0e-04 vmlinux-2.6.21.5 tcp_check_req 15 8.0e-04 vmlinux-2.6.21.5 vfs_write 15 8.0e-04 vmlinux-2.6.21.5 vma_adjust 14 7.5e-04 httpd ap_http_header_filter 14 7.5e-04 httpd ap_queue_pop 14 7.5e-04 httpd apr_brigade_create 14 7.5e-04 httpd apr_bucket_flush_create 14 7.5e-04 httpd apr_off_t_toa 14 7.5e-04 httpd apr_table_make 14 7.5e-04 libnetsnmp.so.5.1.2 (no symbols) 14 7.5e-04 vmlinux-2.6.21.5 __copy_from_user_ll_nozero 14 7.5e-04 vmlinux-2.6.21.5 __user_walk_fd 14 7.5e-04 vmlinux-2.6.21.5 __wait_on_bit_lock 14 7.5e-04 vmlinux-2.6.21.5 bio_init 14 7.5e-04 vmlinux-2.6.21.5 clear_inode 14 7.5e-04 vmlinux-2.6.21.5 delayed_work_timer_fn 14 7.5e-04 vmlinux-2.6.21.5 dupfd 14 7.5e-04 vmlinux-2.6.21.5 generic_forget_inode 14 7.5e-04 vmlinux-2.6.21.5 get_wchan 14 7.5e-04 vmlinux-2.6.21.5 jfs_release 14 7.5e-04 vmlinux-2.6.21.5 kmap_high 14 7.5e-04 vmlinux-2.6.21.5 metapage_readpage 14 7.5e-04 vmlinux-2.6.21.5 sock_queue_rcv_skb 14 7.5e-04 vmlinux-2.6.21.5 tcp_v4_conn_request 14 7.5e-04 vmlinux-2.6.21.5 vfs_getattr 14 7.5e-04 vmlinux-2.6.21.5 walk_page_buffers 13 6.9e-04 httpd add_any_filter 13 6.9e-04 httpd ap_run_handler 13 6.9e-04 httpd ap_update_child_status 13 6.9e-04 httpd basic_http_header_check 13 6.9e-04 httpd get_offset 13 6.9e-04 httpd make_array_core 13 6.9e-04 libpthread-0.60.so pthread_cond_wait@@GLIBC_2.3.2 13 6.9e-04 libpthread-0.60.so pthread_mutex_lock 13 6.9e-04 squid (no symbols) 13 6.9e-04 vmlinux-2.6.21.5 __lock_page 13 6.9e-04 vmlinux-2.6.21.5 __netif_schedule 13 6.9e-04 vmlinux-2.6.21.5 _local_bh_enable 13 6.9e-04 vmlinux-2.6.21.5 ea_get 13 6.9e-04 vmlinux-2.6.21.5 elv_iosched_allow_merge 13 6.9e-04 vmlinux-2.6.21.5 elv_rqhash_find 13 6.9e-04 vmlinux-2.6.21.5 expand_files 13 6.9e-04 vmlinux-2.6.21.5 ext3_follow_link 13 6.9e-04 vmlinux-2.6.21.5 file_move 13 6.9e-04 vmlinux-2.6.21.5 inet_sock_destruct 13 6.9e-04 vmlinux-2.6.21.5 init_waitqueue_head 13 6.9e-04 vmlinux-2.6.21.5 journal_cancel_revoke 13 6.9e-04 vmlinux-2.6.21.5 kmap 13 6.9e-04 vmlinux-2.6.21.5 net_tx_action 13 6.9e-04 vmlinux-2.6.21.5 page_remove_rmap 13 6.9e-04 vmlinux-2.6.21.5 pid_task 13 6.9e-04 vmlinux-2.6.21.5 pipe_wait 13 6.9e-04 vmlinux-2.6.21.5 tcp_close_state 13 6.9e-04 vmlinux-2.6.21.5 tcp_send_probe0 13 6.9e-04 vmlinux-2.6.21.5 tcp_time_wait 13 6.9e-04 vmlinux-2.6.21.5 tcp_v4_destroy_sock 13 6.9e-04 vmlinux-2.6.21.5 tcp_v4_hnd_req 13 6.9e-04 vmlinux-2.6.21.5 unmap_vmas 12 6.4e-04 amd fh_to_mp3 12 6.4e-04 httpd ap_http_filter 12 6.4e-04 httpd apply_rewrite_list 12 6.4e-04 httpd apr_pool_destroy 12 6.4e-04 httpd apr_pstrcat 12 6.4e-04 httpd apr_stat 12 6.4e-04 httpd apr_thread_mutex_lock 12 6.4e-04 httpd heap_bucket_read 12 6.4e-04 httpd match_headers 12 6.4e-04 libpthread-0.60.so __errno_location 12 6.4e-04 libpthread-0.60.so __i686.get_pc_thunk.cx 12 6.4e-04 libpthread-0.60.so _pthread_cleanup_push_defer 12 6.4e-04 libpthread-0.60.so fcntl 12 6.4e-04 vmlinux-2.6.21.5 __find_get_block 12 6.4e-04 vmlinux-2.6.21.5 __get_user_4 12 6.4e-04 vmlinux-2.6.21.5 __getblk 12 6.4e-04 vmlinux-2.6.21.5 __journal_temp_unlink_buffer 12 6.4e-04 vmlinux-2.6.21.5 balance_dirty_pages_ratelimited_nr 12 6.4e-04 vmlinux-2.6.21.5 blk_plug_device 12 6.4e-04 vmlinux-2.6.21.5 block_sync_page 12 6.4e-04 vmlinux-2.6.21.5 call_rcu_bh 12 6.4e-04 vmlinux-2.6.21.5 cond_resched_softirq 12 6.4e-04 vmlinux-2.6.21.5 cp_new_stat64 12 6.4e-04 vmlinux-2.6.21.5 d_splice_alias 12 6.4e-04 vmlinux-2.6.21.5 do_sync_write 12 6.4e-04 vmlinux-2.6.21.5 fib_validate_source 12 6.4e-04 vmlinux-2.6.21.5 generic_commit_write 12 6.4e-04 vmlinux-2.6.21.5 generic_fillattr 12 6.4e-04 vmlinux-2.6.21.5 get_new_inode_fast 12 6.4e-04 vmlinux-2.6.21.5 get_nr_files 12 6.4e-04 vmlinux-2.6.21.5 in_sched_functions 12 6.4e-04 vmlinux-2.6.21.5 inet_csk_destroy_sock 12 6.4e-04 vmlinux-2.6.21.5 inet_csk_route_req 12 6.4e-04 vmlinux-2.6.21.5 inet_release 12 6.4e-04 vmlinux-2.6.21.5 iov_fault_in_pages_write 12 6.4e-04 vmlinux-2.6.21.5 kmem_cache_zalloc 12 6.4e-04 vmlinux-2.6.21.5 scsi_alloc_sgtable 12 6.4e-04 vmlinux-2.6.21.5 should_remove_suid 12 6.4e-04 vmlinux-2.6.21.5 skb_dequeue 12 6.4e-04 vmlinux-2.6.21.5 sock_alloc_send_skb 12 6.4e-04 vmlinux-2.6.21.5 sock_def_readable 12 6.4e-04 vmlinux-2.6.21.5 sync_page 12 6.4e-04 vmlinux-2.6.21.5 sys_time 12 6.4e-04 vmlinux-2.6.21.5 tcp_create_openreq_child 12 6.4e-04 vmlinux-2.6.21.5 tcp_rcv_space_adjust 12 6.4e-04 vmlinux-2.6.21.5 tcp_try_undo_dsack 12 6.4e-04 vmlinux-2.6.21.5 vfs_stat_fd 11 5.9e-04 amd nfsproc_getattr_2_svc 11 5.9e-04 httpd ap_headers_insert_output_filter 11 5.9e-04 httpd ap_invoke_handler 11 5.9e-04 httpd ap_make_full_path 11 5.9e-04 httpd ap_pcfg_openfile 11 5.9e-04 httpd ap_process_request 11 5.9e-04 httpd apr_brigade_split_line 11 5.9e-04 httpd apr_bucket_shared_split 11 5.9e-04 httpd apr_socket_accept 11 5.9e-04 httpd config_log_transaction 11 5.9e-04 httpd core_create_conn 11 5.9e-04 httpd multi_log_transaction 11 5.9e-04 httpd prep_walk_cache 11 5.9e-04 httpd table_mergesort 11 5.9e-04 httpd translate_alias_redir 11 5.9e-04 libpthread-0.60.so __pthread_enable_asynccancel 11 5.9e-04 vmlinux-2.6.21.5 __jfs_getxattr 11 5.9e-04 vmlinux-2.6.21.5 __log_space_left 11 5.9e-04 vmlinux-2.6.21.5 __map_bio 11 5.9e-04 vmlinux-2.6.21.5 __put_unused_fd 11 5.9e-04 vmlinux-2.6.21.5 as_can_anticipate 11 5.9e-04 vmlinux-2.6.21.5 ext3_getblk 11 5.9e-04 vmlinux-2.6.21.5 ext3_new_blocks 11 5.9e-04 vmlinux-2.6.21.5 file_kill 11 5.9e-04 vmlinux-2.6.21.5 file_update_time 11 5.9e-04 vmlinux-2.6.21.5 free_hot_page 11 5.9e-04 vmlinux-2.6.21.5 ipc_lock 11 5.9e-04 vmlinux-2.6.21.5 memcpy_toiovec 11 5.9e-04 vmlinux-2.6.21.5 metapage_read_end_io 11 5.9e-04 vmlinux-2.6.21.5 put_device 11 5.9e-04 vmlinux-2.6.21.5 task_dumpable 11 5.9e-04 vmlinux-2.6.21.5 tcp_clear_retrans 11 5.9e-04 vmlinux-2.6.21.5 udp_rcv 11 5.9e-04 vmlinux-2.6.21.5 vfs_permission 10 5.3e-04 commoncap (no symbols) 10 5.3e-04 httpd ap_get_brigade 10 5.3e-04 httpd ap_make_method_list 10 5.3e-04 httpd ap_no2slash 10 5.3e-04 httpd ap_queue_push 10 5.3e-04 httpd ap_run_fixups 10 5.3e-04 httpd ap_run_process_connection 10 5.3e-04 httpd ap_run_quick_handler 10 5.3e-04 httpd ap_save_brigade 10 5.3e-04 httpd apr_pstrdup 10 5.3e-04 httpd apr_pvsprintf 10 5.3e-04 httpd apr_socket_sendfile 10 5.3e-04 httpd apr_socket_timeout_set 10 5.3e-04 httpd apr_table_copy 10 5.3e-04 httpd apr_uri_parse 10 5.3e-04 httpd find_allowdeny 10 5.3e-04 httpd find_entry 10 5.3e-04 httpd fix_hostname 10 5.3e-04 httpd process_item 10 5.3e-04 httpd ssl_hook_Access 10 5.3e-04 httpd terminate_header 10 5.3e-04 httpd translate_userdir 10 5.3e-04 vmlinux-2.6.21.5 __ext3_journal_dirty_metadata 10 5.3e-04 vmlinux-2.6.21.5 __mutex_init 10 5.3e-04 vmlinux-2.6.21.5 arch_get_unmapped_area_topdown 10 5.3e-04 vmlinux-2.6.21.5 balance_pgdat 10 5.3e-04 vmlinux-2.6.21.5 bget_one 10 5.3e-04 vmlinux-2.6.21.5 blk_unplug_timeout 10 5.3e-04 vmlinux-2.6.21.5 change_clocksource 10 5.3e-04 vmlinux-2.6.21.5 do_sock_read 10 5.3e-04 vmlinux-2.6.21.5 fd_install 10 5.3e-04 vmlinux-2.6.21.5 find_get_pages 10 5.3e-04 vmlinux-2.6.21.5 find_vma_prepare 10 5.3e-04 vmlinux-2.6.21.5 io_schedule 10 5.3e-04 vmlinux-2.6.21.5 path_lookup_open 10 5.3e-04 vmlinux-2.6.21.5 pipe_iov_copy_from_user 10 5.3e-04 vmlinux-2.6.21.5 prune_dcache 10 5.3e-04 vmlinux-2.6.21.5 put_cmsg 10 5.3e-04 vmlinux-2.6.21.5 put_filp 10 5.3e-04 vmlinux-2.6.21.5 real_lookup 10 5.3e-04 vmlinux-2.6.21.5 sock_alloc_inode 10 5.3e-04 vmlinux-2.6.21.5 tcp_cwnd_restart 10 5.3e-04 vmlinux-2.6.21.5 tcp_send_fin 10 5.3e-04 vmlinux-2.6.21.5 unmap_page_range 9 4.8e-04 amd clocktime 9 4.8e-04 amd xdr_fattr 9 4.8e-04 httpd ap_default_log_writer 9 4.8e-04 httpd ap_get_request_note 9 4.8e-04 httpd ap_invoke_filter_init 9 4.8e-04 httpd ap_satisfies 9 4.8e-04 httpd apr_base64_decode_len 9 4.8e-04 httpd apr_bucket_shared_destroy 9 4.8e-04 httpd apr_filepath_root 9 4.8e-04 httpd apr_os_sock_get 9 4.8e-04 httpd do_expand 9 4.8e-04 httpd fixup_dir 9 4.8e-04 httpd handle_audio_request 9 4.8e-04 httpd lookup_builtin_method 9 4.8e-04 httpd merge_env_dir_configs 9 4.8e-04 httpd sendfile_it_all 9 4.8e-04 httpd substring_conf 9 4.8e-04 libpthread-0.60.so __read_nocancel 9 4.8e-04 libpthread-0.60.so open 9 4.8e-04 vmlinux-2.6.21.5 __try_to_free_cp_buf 9 4.8e-04 vmlinux-2.6.21.5 alloc_inode 9 4.8e-04 vmlinux-2.6.21.5 as_add_rq_rb 9 4.8e-04 vmlinux-2.6.21.5 blk_backing_dev_unplug 9 4.8e-04 vmlinux-2.6.21.5 current_tick_length 9 4.8e-04 vmlinux-2.6.21.5 debug_mutex_init 9 4.8e-04 vmlinux-2.6.21.5 destroy_inode 9 4.8e-04 vmlinux-2.6.21.5 do_getname 9 4.8e-04 vmlinux-2.6.21.5 file_ra_state_init 9 4.8e-04 vmlinux-2.6.21.5 fn_hash_select_default 9 4.8e-04 vmlinux-2.6.21.5 get_dirty_limits 9 4.8e-04 vmlinux-2.6.21.5 get_vmalloc_info 9 4.8e-04 vmlinux-2.6.21.5 hash_futex 9 4.8e-04 vmlinux-2.6.21.5 inet_shutdown 9 4.8e-04 vmlinux-2.6.21.5 invalidate_inode_buffers 9 4.8e-04 vmlinux-2.6.21.5 jfs_read_inode 9 4.8e-04 vmlinux-2.6.21.5 journal_blocks_per_page 9 4.8e-04 vmlinux-2.6.21.5 mpage_bio_submit 9 4.8e-04 vmlinux-2.6.21.5 new_inode 9 4.8e-04 vmlinux-2.6.21.5 proc_pid_readdir 9 4.8e-04 vmlinux-2.6.21.5 queue_delayed_work 9 4.8e-04 vmlinux-2.6.21.5 radix_tree_tag_set 9 4.8e-04 vmlinux-2.6.21.5 sock_recvmsg 9 4.8e-04 vmlinux-2.6.21.5 split_vma 9 4.8e-04 vmlinux-2.6.21.5 sys_dup 9 4.8e-04 vmlinux-2.6.21.5 tcp_grow_window 9 4.8e-04 vmlinux-2.6.21.5 tcp_sendpage 9 4.8e-04 vmlinux-2.6.21.5 tcp_v4_syn_recv_sock 9 4.8e-04 vmlinux-2.6.21.5 udp4_hwcsum_outgoing 9 4.8e-04 vmlinux-2.6.21.5 unmap_region 9 4.8e-04 xt_tcpudp (no symbols) 8 4.3e-04 amd autofs_handle_fdset 8 4.3e-04 amd get_next_exported_ap 8 4.3e-04 amd xdr_nfs_fh 8 4.3e-04 apache (no symbols) 8 4.3e-04 httpd __moddi3 8 4.3e-04 httpd ap_run_default_port 8 4.3e-04 httpd ap_set_keepalive 8 4.3e-04 httpd apr_array_make 8 4.3e-04 httpd apr_psprintf 8 4.3e-04 httpd blockDecryptCBC 8 4.3e-04 httpd core_create_req 8 4.3e-04 httpd find_default_server 8 4.3e-04 httpd hook_fixup 8 4.3e-04 httpd log_request_time 8 4.3e-04 httpd proc_mutex_sysv_release 8 4.3e-04 httpd proxy_handler 8 4.3e-04 httpd proxy_trans 8 4.3e-04 httpd removeNewlines 8 4.3e-04 httpd ssl_hook_ReadReq 8 4.3e-04 httpd unixd_accept 8 4.3e-04 libnetsnmpagent.so.5.1.2 (no symbols) 8 4.3e-04 libpthread-0.60.so close 8 4.3e-04 syslog-ng (no symbols) 8 4.3e-04 vmlinux-2.6.21.5 bictcp_init 8 4.3e-04 vmlinux-2.6.21.5 blk_unplug_work 8 4.3e-04 vmlinux-2.6.21.5 d_lookup 8 4.3e-04 vmlinux-2.6.21.5 do_tcp_setsockopt 8 4.3e-04 vmlinux-2.6.21.5 ext3_dirty_inode 8 4.3e-04 vmlinux-2.6.21.5 ext3_mark_inode_dirty 8 4.3e-04 vmlinux-2.6.21.5 ext3_release_file 8 4.3e-04 vmlinux-2.6.21.5 ext3_try_to_allocate_with_rsv 8 4.3e-04 vmlinux-2.6.21.5 in_lock_functions 8 4.3e-04 vmlinux-2.6.21.5 init_once 8 4.3e-04 vmlinux-2.6.21.5 ip_mc_drop_socket 8 4.3e-04 vmlinux-2.6.21.5 jfs_open 8 4.3e-04 vmlinux-2.6.21.5 jfs_set_inode_flags 8 4.3e-04 vmlinux-2.6.21.5 journal_commit_transaction 8 4.3e-04 vmlinux-2.6.21.5 neigh_hh_init 8 4.3e-04 vmlinux-2.6.21.5 neigh_lookup 8 4.3e-04 vmlinux-2.6.21.5 new_handle 8 4.3e-04 vmlinux-2.6.21.5 next_tgid 8 4.3e-04 vmlinux-2.6.21.5 no_singlestep 8 4.3e-04 vmlinux-2.6.21.5 rb_first 8 4.3e-04 vmlinux-2.6.21.5 scsi_next_command 8 4.3e-04 vmlinux-2.6.21.5 secure_tcp_sequence_number 8 4.3e-04 vmlinux-2.6.21.5 shrink_dcache_memory 8 4.3e-04 vmlinux-2.6.21.5 skb_recv_datagram 8 4.3e-04 vmlinux-2.6.21.5 smp_send_reschedule 8 4.3e-04 vmlinux-2.6.21.5 sock_fasync 8 4.3e-04 vmlinux-2.6.21.5 sync_sb_inodes 8 4.3e-04 vmlinux-2.6.21.5 sys_brk 8 4.3e-04 vmlinux-2.6.21.5 sys_munmap 8 4.3e-04 vmlinux-2.6.21.5 sys_stat64 8 4.3e-04 vmlinux-2.6.21.5 tcp_add_reno_sack 8 4.3e-04 vmlinux-2.6.21.5 tcp_check_reno_reordering 8 4.3e-04 vmlinux-2.6.21.5 tcp_check_sack_reneging 8 4.3e-04 vmlinux-2.6.21.5 tcp_event_data_recv 8 4.3e-04 vmlinux-2.6.21.5 tcp_probe_timer 8 4.3e-04 vmlinux-2.6.21.5 tcp_try_undo_loss 8 4.3e-04 vmlinux-2.6.21.5 tcp_try_undo_partial 8 4.3e-04 vmlinux-2.6.21.5 tcp_try_undo_recovery 8 4.3e-04 vmlinux-2.6.21.5 unlock_buffer 8 4.3e-04 vmlinux-2.6.21.5 vm_acct_memory 8 4.3e-04 vmlinux-2.6.21.5 wake_up_inode 7 3.7e-04 amd autofs_add_fdset 7 3.7e-04 amd checkup 7 3.7e-04 amd new_ttl 7 3.7e-04 httpd ap_core_translate 7 3.7e-04 httpd ap_index_of_response 7 3.7e-04 httpd ap_run_log_transaction 7 3.7e-04 httpd ap_some_auth_required 7 3.7e-04 httpd ap_update_vhost_from_headers 7 3.7e-04 httpd apr_array_append 7 3.7e-04 httpd apr_table_compress 7 3.7e-04 httpd apr_table_elts 7 3.7e-04 httpd apr_table_mergen 7 3.7e-04 httpd apr_time_ansi_put 7 3.7e-04 httpd apr_unix_mode2perms 7 3.7e-04 httpd check_pipeline_flush 7 3.7e-04 httpd display_info 7 3.7e-04 httpd fill_out_finfo 7 3.7e-04 httpd fixup_redir 7 3.7e-04 httpd include_fixup 7 3.7e-04 httpd read_request_line 7 3.7e-04 httpd try_alias_list 7 3.7e-04 libpthread-0.60.so __lll_mutex_unlock_wake 7 3.7e-04 libpthread-0.60.so write 7 3.7e-04 vmlinux-2.6.21.5 __copy_from_user_ll_nocache_nozero 7 3.7e-04 vmlinux-2.6.21.5 __inet_twsk_hashdance 7 3.7e-04 vmlinux-2.6.21.5 block_prepare_write 7 3.7e-04 vmlinux-2.6.21.5 do_fcntl 7 3.7e-04 vmlinux-2.6.21.5 do_munmap 7 3.7e-04 vmlinux-2.6.21.5 dummy_inode_alloc_security 7 3.7e-04 vmlinux-2.6.21.5 elv_put_request 7 3.7e-04 vmlinux-2.6.21.5 elv_rb_find 7 3.7e-04 vmlinux-2.6.21.5 ext3_reserve_inode_write 7 3.7e-04 vmlinux-2.6.21.5 find_ge_pid 7 3.7e-04 vmlinux-2.6.21.5 find_get_pages_tag 7 3.7e-04 vmlinux-2.6.21.5 find_inode_fast 7 3.7e-04 vmlinux-2.6.21.5 find_undo 7 3.7e-04 vmlinux-2.6.21.5 flush_tlb_page 7 3.7e-04 vmlinux-2.6.21.5 generic_drop_inode 7 3.7e-04 vmlinux-2.6.21.5 get_UCSname 7 3.7e-04 vmlinux-2.6.21.5 half_md4_transform 7 3.7e-04 vmlinux-2.6.21.5 inet_twsk_schedule 7 3.7e-04 vmlinux-2.6.21.5 iov_fault_in_pages_read 7 3.7e-04 vmlinux-2.6.21.5 is_bad_inode 7 3.7e-04 vmlinux-2.6.21.5 jfs_alloc_inode 7 3.7e-04 vmlinux-2.6.21.5 mb_cache_shrink_fn 7 3.7e-04 vmlinux-2.6.21.5 move_addr_to_kernel 7 3.7e-04 vmlinux-2.6.21.5 proc_info_read 7 3.7e-04 vmlinux-2.6.21.5 pull_task 7 3.7e-04 vmlinux-2.6.21.5 shrink_zone 7 3.7e-04 vmlinux-2.6.21.5 skb_queue_tail 7 3.7e-04 vmlinux-2.6.21.5 sock_rfree 7 3.7e-04 vmlinux-2.6.21.5 tcp_cleanup_congestion_control 7 3.7e-04 vmlinux-2.6.21.5 tcp_init_congestion_control 7 3.7e-04 vmlinux-2.6.21.5 tcp_measure_rcv_mss 7 3.7e-04 vmlinux-2.6.21.5 update_queue 7 3.7e-04 vmlinux-2.6.21.5 wake_up_process 6 3.2e-04 httpd analyze_ct 6 3.2e-04 httpd ap_bucket_eoc_create 6 3.2e-04 httpd ap_explode_recent_localtime 6 3.2e-04 httpd ap_find_token 6 3.2e-04 httpd ap_get_server_name 6 3.2e-04 httpd ap_get_server_version 6 3.2e-04 httpd ap_getparents 6 3.2e-04 httpd ap_queue_info_set_idle 6 3.2e-04 httpd ap_queue_info_wait_for_idler 6 3.2e-04 httpd ap_rationalize_mtime 6 3.2e-04 httpd ap_run_create_request 6 3.2e-04 httpd ap_run_type_checker 6 3.2e-04 httpd ap_send_http_trace 6 3.2e-04 httpd apply_rewrite_cond 6 3.2e-04 httpd apr_bucket_alloc_create 6 3.2e-04 httpd apr_bucket_eos_create 6 3.2e-04 httpd apr_poll_revents_get 6 3.2e-04 httpd apr_proc_mutex_unlock 6 3.2e-04 httpd apr_table_vdo 6 3.2e-04 httpd constant_item 6 3.2e-04 httpd default_handler 6 3.2e-04 httpd expires_insert_filter 6 3.2e-04 httpd extractParam 6 3.2e-04 httpd fixup_vary 6 3.2e-04 httpd imap_handler 6 3.2e-04 httpd inet_ntop4 6 3.2e-04 httpd logDRMAccess 6 3.2e-04 httpd proxy_fixup 6 3.2e-04 libpthread-0.60.so __pthread_mutex_unlock_usercnt 6 3.2e-04 libpthread-0.60.so pthread_cond_signal@@GLIBC_2.3.2 6 3.2e-04 named (no symbols) 6 3.2e-04 vmlinux-2.6.21.5 __init_rwsem 6 3.2e-04 vmlinux-2.6.21.5 __vm_enough_memory 6 3.2e-04 vmlinux-2.6.21.5 cascade 6 3.2e-04 vmlinux-2.6.21.5 clear_page_dirty_for_io 6 3.2e-04 vmlinux-2.6.21.5 double_rq_unlock 6 3.2e-04 vmlinux-2.6.21.5 down_read_trylock 6 3.2e-04 vmlinux-2.6.21.5 dummy_inode_getattr 6 3.2e-04 vmlinux-2.6.21.5 elv_set_request 6 3.2e-04 vmlinux-2.6.21.5 ext3_get_blocks_handle 6 3.2e-04 vmlinux-2.6.21.5 ext3_get_branch 6 3.2e-04 vmlinux-2.6.21.5 ext3_get_inode_loc 6 3.2e-04 vmlinux-2.6.21.5 fib_rule_match 6 3.2e-04 vmlinux-2.6.21.5 file_free_rcu 6 3.2e-04 vmlinux-2.6.21.5 filldir64 6 3.2e-04 vmlinux-2.6.21.5 free_pgtables 6 3.2e-04 vmlinux-2.6.21.5 generic_file_open 6 3.2e-04 vmlinux-2.6.21.5 generic_writepages 6 3.2e-04 vmlinux-2.6.21.5 get_unmapped_area 6 3.2e-04 vmlinux-2.6.21.5 inet_accept 6 3.2e-04 vmlinux-2.6.21.5 inet_csk_clone 6 3.2e-04 vmlinux-2.6.21.5 journal_submit_data_buffers 6 3.2e-04 vmlinux-2.6.21.5 kblockd_schedule_work 6 3.2e-04 vmlinux-2.6.21.5 kernel_sendmsg 6 3.2e-04 vmlinux-2.6.21.5 lru_cache_add_active 6 3.2e-04 vmlinux-2.6.21.5 max_select_fd 6 3.2e-04 vmlinux-2.6.21.5 need_resched 6 3.2e-04 vmlinux-2.6.21.5 opost_block 6 3.2e-04 vmlinux-2.6.21.5 out_of_line_wait_on_bit 6 3.2e-04 vmlinux-2.6.21.5 page_add_new_anon_rmap 6 3.2e-04 vmlinux-2.6.21.5 prepare_to_wait_exclusive 6 3.2e-04 vmlinux-2.6.21.5 proc_pid_fill_cache 6 3.2e-04 vmlinux-2.6.21.5 schedule_delayed_work 6 3.2e-04 vmlinux-2.6.21.5 sk_alloc 6 3.2e-04 vmlinux-2.6.21.5 sk_free 6 3.2e-04 vmlinux-2.6.21.5 sk_stop_timer 6 3.2e-04 vmlinux-2.6.21.5 sock_common_setsockopt 6 3.2e-04 vmlinux-2.6.21.5 sock_release 6 3.2e-04 vmlinux-2.6.21.5 sys_mmap2 6 3.2e-04 vmlinux-2.6.21.5 sys_sendmsg 6 3.2e-04 vmlinux-2.6.21.5 sys_setsockopt 6 3.2e-04 vmlinux-2.6.21.5 tcp_fin 6 3.2e-04 vmlinux-2.6.21.5 tcp_init_cwnd 6 3.2e-04 vmlinux-2.6.21.5 tcp_setsockopt 6 3.2e-04 vmlinux-2.6.21.5 tcp_shutdown 6 3.2e-04 vmlinux-2.6.21.5 tcp_update_metrics 6 3.2e-04 vmlinux-2.6.21.5 tcp_v4_send_synack 6 3.2e-04 vmlinux-2.6.21.5 unlink_file_vma 6 3.2e-04 vmlinux-2.6.21.5 unqueue_me 6 3.2e-04 vmlinux-2.6.21.5 verify_iovec 6 3.2e-04 vmlinux-2.6.21.5 vm_normal_page 5 2.7e-04 amd amu_svc_getcaller 5 2.7e-04 httpd action_handler 5 2.7e-04 httpd ap_create_sb_handle 5 2.7e-04 httpd ap_regexec 5 2.7e-04 httpd ap_remove_output_filter 5 2.7e-04 httpd ap_rputc 5 2.7e-04 httpd ap_run_access_checker 5 2.7e-04 httpd ap_run_insert_filter 5 2.7e-04 httpd apr_allocator_free 5 2.7e-04 httpd apr_brigade_length 5 2.7e-04 httpd apr_bucket_eos_make 5 2.7e-04 httpd apr_bucket_heap_make 5 2.7e-04 httpd apr_bucket_simple_split 5 2.7e-04 httpd apr_pool_cleanup_run 5 2.7e-04 httpd apr_shutdown 5 2.7e-04 httpd apr_sockaddr_ip_get 5 2.7e-04 httpd apr_sockaddr_vars_set 5 2.7e-04 httpd apr_thread_cond_wait 5 2.7e-04 httpd apr_thread_mutex_unlock 5 2.7e-04 httpd conv_10_quad 5 2.7e-04 httpd file_bucket_destroy 5 2.7e-04 httpd find_poll_sock 5 2.7e-04 httpd maint_handler 5 2.7e-04 httpd rewritelog 5 2.7e-04 httpd socket_bucket_read 5 2.7e-04 httpd ssl_hook_default_port 5 2.7e-04 httpd table_reindex 5 2.7e-04 libpthread-0.60.so _pthread_cleanup_pop_restore 5 2.7e-04 ntpd (no symbols) 5 2.7e-04 vmlinux-2.6.21.5 __block_commit_write 5 2.7e-04 vmlinux-2.6.21.5 __block_write_full_page 5 2.7e-04 vmlinux-2.6.21.5 __d_rehash 5 2.7e-04 vmlinux-2.6.21.5 _spin_lock_bh 5 2.7e-04 vmlinux-2.6.21.5 block_write_full_page 5 2.7e-04 vmlinux-2.6.21.5 check_userspace 5 2.7e-04 vmlinux-2.6.21.5 create_empty_buffers 5 2.7e-04 vmlinux-2.6.21.5 d_rehash 5 2.7e-04 vmlinux-2.6.21.5 end_bio_bh_io_sync 5 2.7e-04 vmlinux-2.6.21.5 ext3_get_block 5 2.7e-04 vmlinux-2.6.21.5 fib_rules_lookup 5 2.7e-04 vmlinux-2.6.21.5 get_task_comm 5 2.7e-04 vmlinux-2.6.21.5 inet_csk_reqsk_queue_prune 5 2.7e-04 vmlinux-2.6.21.5 inet_select_addr 5 2.7e-04 vmlinux-2.6.21.5 inet_twsk_alloc 5 2.7e-04 vmlinux-2.6.21.5 inotify_d_instantiate 5 2.7e-04 vmlinux-2.6.21.5 ip_cmsg_recv_pktinfo 5 2.7e-04 vmlinux-2.6.21.5 ip_options_echo 5 2.7e-04 vmlinux-2.6.21.5 jfs_strtoUCS 5 2.7e-04 vmlinux-2.6.21.5 journal_get_undo_access 5 2.7e-04 vmlinux-2.6.21.5 journal_grab_journal_head 5 2.7e-04 vmlinux-2.6.21.5 kill_fasync 5 2.7e-04 vmlinux-2.6.21.5 locate_fd 5 2.7e-04 vmlinux-2.6.21.5 lock_task_sighand 5 2.7e-04 vmlinux-2.6.21.5 lru_cache_add 5 2.7e-04 vmlinux-2.6.21.5 metapage_get_blocks 5 2.7e-04 vmlinux-2.6.21.5 move_addr_to_user 5 2.7e-04 vmlinux-2.6.21.5 mpage_readpage 5 2.7e-04 vmlinux-2.6.21.5 neigh_resolve_output 5 2.7e-04 vmlinux-2.6.21.5 proc_pid_statm 5 2.7e-04 vmlinux-2.6.21.5 proc_readdir 5 2.7e-04 vmlinux-2.6.21.5 sk_stream_kill_queues 5 2.7e-04 vmlinux-2.6.21.5 sk_stream_wait_close 5 2.7e-04 vmlinux-2.6.21.5 sys_rt_sigprocmask 5 2.7e-04 vmlinux-2.6.21.5 tcp_mtup_init 5 2.7e-04 vmlinux-2.6.21.5 tcp_send_ack 4 2.1e-04 amd softclock 4 2.1e-04 amd xdr_nfsstat 4 2.1e-04 httpd ap_add_output_filter_handle 4 2.1e-04 httpd ap_add_output_filters_by_type 4 2.1e-04 httpd ap_get_scoreboard_worker 4 2.1e-04 httpd ap_meets_conditions 4 2.1e-04 httpd ap_method_number_of 4 2.1e-04 httpd ap_old_write_filter 4 2.1e-04 httpd ap_run_create_connection 4 2.1e-04 httpd ap_run_map_to_storage 4 2.1e-04 httpd ap_set_content_length 4 2.1e-04 httpd ap_str_tolower 4 2.1e-04 httpd apr_allocator_alloc 4 2.1e-04 httpd apr_array_copy_hdr 4 2.1e-04 httpd apr_inet_ntop 4 2.1e-04 httpd apr_itoa 4 2.1e-04 httpd apr_proc_mutex_lock 4 2.1e-04 httpd apr_rfc822_date 4 2.1e-04 httpd apr_time_exp_gmt 4 2.1e-04 httpd apr_unix_file_cleanup 4 2.1e-04 httpd apr_unix_perms2mode 4 2.1e-04 httpd brigade_cleanup 4 2.1e-04 httpd core_map_to_storage 4 2.1e-04 httpd decrypt 4 2.1e-04 httpd filetype_from_mode 4 2.1e-04 httpd find_closing_bracket 4 2.1e-04 httpd getClientIP 4 2.1e-04 httpd handle_autoindex 4 2.1e-04 httpd hook_mimetype 4 2.1e-04 httpd processV4Query 4 2.1e-04 libpthread-0.60.so __pthread_mutex_cond_lock 4 2.1e-04 libpthread-0.60.so __write_nocancel 4 2.1e-04 mawk (no symbols) 4 2.1e-04 sshd (no symbols) 4 2.1e-04 vmlinux-2.6.21.5 __brelse 4 2.1e-04 vmlinux-2.6.21.5 __ext3_journal_stop 4 2.1e-04 vmlinux-2.6.21.5 __find_get_block_slow 4 2.1e-04 vmlinux-2.6.21.5 __inet_twsk_kill 4 2.1e-04 vmlinux-2.6.21.5 __lookup_hash 4 2.1e-04 vmlinux-2.6.21.5 __lru_add_drain 4 2.1e-04 vmlinux-2.6.21.5 __rwlock_init 4 2.1e-04 vmlinux-2.6.21.5 __tcp_ack_snd_check 4 2.1e-04 vmlinux-2.6.21.5 anon_vma_unlink 4 2.1e-04 vmlinux-2.6.21.5 can_vma_merge_before 4 2.1e-04 vmlinux-2.6.21.5 copy_process 4 2.1e-04 vmlinux-2.6.21.5 detach_vmas_to_be_unmapped 4 2.1e-04 vmlinux-2.6.21.5 diIAGRead 4 2.1e-04 vmlinux-2.6.21.5 dispose_list 4 2.1e-04 vmlinux-2.6.21.5 do_anonymous_page 4 2.1e-04 vmlinux-2.6.21.5 do_sendfile 4 2.1e-04 vmlinux-2.6.21.5 do_timer 4 2.1e-04 vmlinux-2.6.21.5 drive_stat_acct 4 2.1e-04 vmlinux-2.6.21.5 drop_buffers 4 2.1e-04 vmlinux-2.6.21.5 ea_release 4 2.1e-04 vmlinux-2.6.21.5 elv_dequeue_request 4 2.1e-04 vmlinux-2.6.21.5 ext3_bmap 4 2.1e-04 vmlinux-2.6.21.5 ext3_get_group_desc 4 2.1e-04 vmlinux-2.6.21.5 flush_tlb_mm 4 2.1e-04 vmlinux-2.6.21.5 inet_csk_reqsk_queue_hash_add 4 2.1e-04 vmlinux-2.6.21.5 inet_getname 4 2.1e-04 vmlinux-2.6.21.5 ip_cmsg_send 4 2.1e-04 vmlinux-2.6.21.5 ipcperms 4 2.1e-04 vmlinux-2.6.21.5 jfs_readpage 4 2.1e-04 vmlinux-2.6.21.5 journal_clean_one_cp_list 4 2.1e-04 vmlinux-2.6.21.5 last_read_complete 4 2.1e-04 vmlinux-2.6.21.5 load_elf_binary 4 2.1e-04 vmlinux-2.6.21.5 lookup_hash 4 2.1e-04 vmlinux-2.6.21.5 lru_add_drain 4 2.1e-04 vmlinux-2.6.21.5 mutex_unlock 4 2.1e-04 vmlinux-2.6.21.5 n_tty_receive_buf 4 2.1e-04 vmlinux-2.6.21.5 page_put_link 4 2.1e-04 vmlinux-2.6.21.5 pipe_iov_copy_to_user 4 2.1e-04 vmlinux-2.6.21.5 prune_icache 4 2.1e-04 vmlinux-2.6.21.5 put_pages_list 4 2.1e-04 vmlinux-2.6.21.5 radix_tree_tag_clear 4 2.1e-04 vmlinux-2.6.21.5 read_chan 4 2.1e-04 vmlinux-2.6.21.5 remove_vma_list 4 2.1e-04 vmlinux-2.6.21.5 ret_from_exception 4 2.1e-04 vmlinux-2.6.21.5 sk_stream_rfree 4 2.1e-04 vmlinux-2.6.21.5 sock_aio_read 4 2.1e-04 vmlinux-2.6.21.5 sock_destroy_inode 4 2.1e-04 vmlinux-2.6.21.5 sock_wmalloc 4 2.1e-04 vmlinux-2.6.21.5 sys_fcntl64 4 2.1e-04 vmlinux-2.6.21.5 sys_sendfile 4 2.1e-04 vmlinux-2.6.21.5 syscall_call 4 2.1e-04 vmlinux-2.6.21.5 tcp_ack_probe 4 2.1e-04 vmlinux-2.6.21.5 tcp_child_process 4 2.1e-04 vmlinux-2.6.21.5 tcp_fixup_sndbuf 4 2.1e-04 vmlinux-2.6.21.5 tcp_init_buffer_space 4 2.1e-04 vmlinux-2.6.21.5 tcp_mtu_to_mss 4 2.1e-04 vmlinux-2.6.21.5 tcp_select_initial_window 4 2.1e-04 vmlinux-2.6.21.5 tcp_syn_build_options 4 2.1e-04 vmlinux-2.6.21.5 tcp_v4_save_options 4 2.1e-04 vmlinux-2.6.21.5 tg3_alloc_rx_skb 4 2.1e-04 vmlinux-2.6.21.5 unmap_mapping_range 4 2.1e-04 vmlinux-2.6.21.5 vfs_fstat 4 2.1e-04 vmlinux-2.6.21.5 vfs_stat 4 2.1e-04 vmlinux-2.6.21.5 vm_stat_account 3 1.6e-04 amd .plt 3 1.6e-04 amd xdr_attrstat 3 1.6e-04 amd xmalloc 3 1.6e-04 httpd alloc_cleanup 3 1.6e-04 httpd ap_add_output_filter 3 1.6e-04 httpd ap_explode_recent_gmt 3 1.6e-04 httpd ap_getword 3 1.6e-04 httpd ap_log_cerror 3 1.6e-04 httpd ap_make_dirstr_parent 3 1.6e-04 httpd ap_parse_uri 3 1.6e-04 httpd ap_process_connection 3 1.6e-04 httpd ap_run_post_read_request 3 1.6e-04 httpd ap_set_content_type 3 1.6e-04 httpd apr_accept 3 1.6e-04 httpd apr_allocator_mutex_get 3 1.6e-04 httpd apr_allocator_owner_get 3 1.6e-04 httpd apr_bucket_file_create 3 1.6e-04 httpd apr_bucket_heap_create 3 1.6e-04 httpd apr_bucket_shared_make 3 1.6e-04 httpd apr_parse_addr_port 3 1.6e-04 httpd apr_socket_addr_get 3 1.6e-04 httpd apr_table_do 3 1.6e-04 httpd apr_table_unset 3 1.6e-04 httpd apr_thread_cond_signal 3 1.6e-04 httpd apr_time_exp_lt 3 1.6e-04 httpd checkAccess 3 1.6e-04 httpd core_insert_filter 3 1.6e-04 httpd core_opts_merge 3 1.6e-04 httpd core_pre_connection 3 1.6e-04 httpd create_empty_config 3 1.6e-04 httpd decryptBlowfish 3 1.6e-04 httpd etag_ulong_to_hex 3 1.6e-04 httpd explode_time 3 1.6e-04 httpd fix_encoding 3 1.6e-04 httpd fixup_env_module 3 1.6e-04 httpd handle_multi 3 1.6e-04 httpd http_create_request 3 1.6e-04 httpd log_cookie 3 1.6e-04 httpd log_error_core 3 1.6e-04 httpd log_status 3 1.6e-04 httpd lookup_variable 3 1.6e-04 httpd remove_any_filter 3 1.6e-04 httpd set_socket_vars 3 1.6e-04 httpd sononblock 3 1.6e-04 httpd x2c 3 1.6e-04 libpthread-0.60.so __close_nocancel 3 1.6e-04 libresolv-2.3.2.so (no symbols) 3 1.6e-04 snmpd (no symbols) 3 1.6e-04 vmlinux-2.6.21.5 __anon_vma_link 3 1.6e-04 vmlinux-2.6.21.5 __anon_vma_merge 3 1.6e-04 vmlinux-2.6.21.5 __free_pages_ok 3 1.6e-04 vmlinux-2.6.21.5 __inet_put_port 3 1.6e-04 vmlinux-2.6.21.5 __lookup 3 1.6e-04 vmlinux-2.6.21.5 __pagevec_lru_add_active 3 1.6e-04 vmlinux-2.6.21.5 __rb_rotate_right 3 1.6e-04 vmlinux-2.6.21.5 __sync_single_inode 3 1.6e-04 vmlinux-2.6.21.5 __wait_on_bit 3 1.6e-04 vmlinux-2.6.21.5 add_to_page_cache_lru 3 1.6e-04 vmlinux-2.6.21.5 arch_unmap_area_topdown 3 1.6e-04 vmlinux-2.6.21.5 arp_hash 3 1.6e-04 vmlinux-2.6.21.5 arp_process 3 1.6e-04 vmlinux-2.6.21.5 as_antic_waitnext 3 1.6e-04 vmlinux-2.6.21.5 as_merge 3 1.6e-04 vmlinux-2.6.21.5 bh_lru_install 3 1.6e-04 vmlinux-2.6.21.5 block_read_full_page 3 1.6e-04 vmlinux-2.6.21.5 can_vma_merge_after 3 1.6e-04 vmlinux-2.6.21.5 copy_page_range 3 1.6e-04 vmlinux-2.6.21.5 copy_strings 3 1.6e-04 vmlinux-2.6.21.5 csum_partial_copy_generic 3 1.6e-04 vmlinux-2.6.21.5 d_free 3 1.6e-04 vmlinux-2.6.21.5 dummy_ipc_permission 3 1.6e-04 vmlinux-2.6.21.5 end_buffer_async_write 3 1.6e-04 vmlinux-2.6.21.5 ext3_blks_to_allocate 3 1.6e-04 vmlinux-2.6.21.5 ext3_find_entry 3 1.6e-04 vmlinux-2.6.21.5 ext3_journal_dirty_data 3 1.6e-04 vmlinux-2.6.21.5 ext3_mkdir 3 1.6e-04 vmlinux-2.6.21.5 ext3_ordered_writepage 3 1.6e-04 vmlinux-2.6.21.5 ext3_setattr 3 1.6e-04 vmlinux-2.6.21.5 ext3_test_allocatable 3 1.6e-04 vmlinux-2.6.21.5 ext3_try_to_allocate 3 1.6e-04 vmlinux-2.6.21.5 file_send_actor 3 1.6e-04 vmlinux-2.6.21.5 find_vma_prev 3 1.6e-04 vmlinux-2.6.21.5 free_one_page 3 1.6e-04 vmlinux-2.6.21.5 free_pgd_range 3 1.6e-04 vmlinux-2.6.21.5 generic_file_sendfile 3 1.6e-04 vmlinux-2.6.21.5 get_io_context 3 1.6e-04 vmlinux-2.6.21.5 get_write_access 3 1.6e-04 vmlinux-2.6.21.5 hash 3 1.6e-04 vmlinux-2.6.21.5 hrtimer_try_to_cancel 3 1.6e-04 vmlinux-2.6.21.5 inet_bind_bucket_destroy 3 1.6e-04 vmlinux-2.6.21.5 inet_csk_clear_xmit_timers 3 1.6e-04 vmlinux-2.6.21.5 inet_put_port 3 1.6e-04 vmlinux-2.6.21.5 init_once 3 1.6e-04 vmlinux-2.6.21.5 inode_has_buffers 3 1.6e-04 vmlinux-2.6.21.5 ipv4_negative_advice 3 1.6e-04 vmlinux-2.6.21.5 jfs_destroy_inode 3 1.6e-04 vmlinux-2.6.21.5 jfs_get_acl 3 1.6e-04 vmlinux-2.6.21.5 jiffies_64_to_clock_t 3 1.6e-04 vmlinux-2.6.21.5 journal_write_metadata_buffer 3 1.6e-04 vmlinux-2.6.21.5 kunmap 3 1.6e-04 vmlinux-2.6.21.5 load_elf_interp 3 1.6e-04 vmlinux-2.6.21.5 memcpy_fromiovecend 3 1.6e-04 vmlinux-2.6.21.5 nr_blockdev_pages 3 1.6e-04 vmlinux-2.6.21.5 pipe_poll 3 1.6e-04 vmlinux-2.6.21.5 preempt_schedule_irq 3 1.6e-04 vmlinux-2.6.21.5 proc_root_readdir 3 1.6e-04 vmlinux-2.6.21.5 proc_tgid_stat 3 1.6e-04 vmlinux-2.6.21.5 rcu_start_batch 3 1.6e-04 vmlinux-2.6.21.5 refresh_cpu_vm_stats 3 1.6e-04 vmlinux-2.6.21.5 remove_suid 3 1.6e-04 vmlinux-2.6.21.5 requeue_task 3 1.6e-04 vmlinux-2.6.21.5 scsi_delete_timer 3 1.6e-04 vmlinux-2.6.21.5 sock_alloc 3 1.6e-04 vmlinux-2.6.21.5 sprintf 3 1.6e-04 vmlinux-2.6.21.5 sys_fstat64 3 1.6e-04 vmlinux-2.6.21.5 sys_shutdown 3 1.6e-04 vmlinux-2.6.21.5 task_vsize 3 1.6e-04 vmlinux-2.6.21.5 tcp_init_metrics 3 1.6e-04 vmlinux-2.6.21.5 tcp_initialize_rcv_mss 3 1.6e-04 vmlinux-2.6.21.5 tcp_mss_to_mtu 3 1.6e-04 vmlinux-2.6.21.5 tcp_sync_mss 3 1.6e-04 vmlinux-2.6.21.5 tcp_timewait_state_process 3 1.6e-04 vmlinux-2.6.21.5 test_set_page_writeback 3 1.6e-04 vmlinux-2.6.21.5 tg3_write_mem 3 1.6e-04 vmlinux-2.6.21.5 up_write 3 1.6e-04 vmlinux-2.6.21.5 vma_merge 2 1.1e-04 amd path_to_exported_ap 2 1.1e-04 amd xdr_nfstime 2 1.1e-04 grep (no symbols) 2 1.1e-04 httpd alloc_socket 2 1.1e-04 httpd ap_finalize_request_protocol 2 1.1e-04 httpd ap_flush_conn 2 1.1e-04 httpd ap_make_content_type 2 1.1e-04 httpd ap_make_etag 2 1.1e-04 httpd ap_run_header_parser 2 1.1e-04 httpd ap_run_translate_name 2 1.1e-04 httpd ap_set_byterange 2 1.1e-04 httpd ap_set_etag 2 1.1e-04 httpd ap_set_last_modified 2 1.1e-04 httpd ap_time_process_request 2 1.1e-04 httpd ap_update_mtime 2 1.1e-04 httpd ap_update_vhost_given_ip 2 1.1e-04 httpd apr_array_push 2 1.1e-04 httpd apr_bucket_flush_make 2 1.1e-04 httpd apr_bucket_transient_make 2 1.1e-04 httpd apr_hash_get 2 1.1e-04 httpd apr_is_empty_table 2 1.1e-04 httpd apr_pool_allocator_get 2 1.1e-04 httpd apr_pool_tag 2 1.1e-04 httpd apr_pstrmemdup 2 1.1e-04 httpd apr_recv 2 1.1e-04 httpd apr_sendfile 2 1.1e-04 httpd apr_socket_close 2 1.1e-04 httpd apr_time_exp_tz 2 1.1e-04 httpd calcTimePlayed 2 1.1e-04 httpd cgid_handler 2 1.1e-04 httpd clf_log_bytes_sent 2 1.1e-04 httpd do_nothing 2 1.1e-04 httpd end_output_stream 2 1.1e-04 httpd form_header_field 2 1.1e-04 httpd handler_redirect 2 1.1e-04 httpd http_method 2 1.1e-04 httpd http_port 2 1.1e-04 httpd loadBlowfishKey 2 1.1e-04 httpd logMessage 2 1.1e-04 httpd log_remote_user 2 1.1e-04 httpd proxy_map_location 2 1.1e-04 httpd send_all_header_fields 2 1.1e-04 httpd socket_cleanup 2 1.1e-04 httpd ssl_hook_Fixup 2 1.1e-04 httpd toLong 2 1.1e-04 httpd validate_status_line 2 1.1e-04 libpthread-0.60.so __i686.get_pc_thunk.dx 2 1.1e-04 nrpe wait_for_connections 2 1.1e-04 python2.4 PyEval_EvalFrame 2 1.1e-04 vmlinux-2.6.21.5 __bread 2 1.1e-04 vmlinux-2.6.21.5 __get_free_pages 2 1.1e-04 vmlinux-2.6.21.5 __insert_vm_struct 2 1.1e-04 vmlinux-2.6.21.5 __journal_refile_buffer 2 1.1e-04 vmlinux-2.6.21.5 __journal_remove_journal_head 2 1.1e-04 vmlinux-2.6.21.5 __journal_unfile_buffer 2 1.1e-04 vmlinux-2.6.21.5 __page_set_anon_rmap 2 1.1e-04 vmlinux-2.6.21.5 __set_page_dirty_buffers 2 1.1e-04 vmlinux-2.6.21.5 __tcp_grow_window 2 1.1e-04 vmlinux-2.6.21.5 account_user_time 2 1.1e-04 vmlinux-2.6.21.5 alarm_setitimer 2 1.1e-04 vmlinux-2.6.21.5 alloc_buffer_head 2 1.1e-04 vmlinux-2.6.21.5 bio_hw_segments 2 1.1e-04 vmlinux-2.6.21.5 blocking_notifier_call_chain 2 1.1e-04 vmlinux-2.6.21.5 check_irq_on 2 1.1e-04 vmlinux-2.6.21.5 convert_fxsr_to_user 2 1.1e-04 vmlinux-2.6.21.5 cookie_v4_check 2 1.1e-04 vmlinux-2.6.21.5 cpu_quiet 2 1.1e-04 vmlinux-2.6.21.5 dec_zone_page_state 2 1.1e-04 vmlinux-2.6.21.5 do_irq_balance 2 1.1e-04 vmlinux-2.6.21.5 drop_key_refs 2 1.1e-04 vmlinux-2.6.21.5 dst_alloc 2 1.1e-04 vmlinux-2.6.21.5 dummy_file_alloc_security 2 1.1e-04 vmlinux-2.6.21.5 dummy_task_wait 2 1.1e-04 vmlinux-2.6.21.5 dup_fd 2 1.1e-04 vmlinux-2.6.21.5 dup_mm 2 1.1e-04 vmlinux-2.6.21.5 elv_activate_rq 2 1.1e-04 vmlinux-2.6.21.5 ext3_block_to_path 2 1.1e-04 vmlinux-2.6.21.5 ext3_bread 2 1.1e-04 vmlinux-2.6.21.5 ext3_free_blocks_sb 2 1.1e-04 vmlinux-2.6.21.5 ext3_has_free_blocks 2 1.1e-04 vmlinux-2.6.21.5 ext3_new_inode 2 1.1e-04 vmlinux-2.6.21.5 ext3_read_inode 2 1.1e-04 vmlinux-2.6.21.5 ext3_truncate 2 1.1e-04 vmlinux-2.6.21.5 fcntl_setlk 2 1.1e-04 vmlinux-2.6.21.5 fib4_rule_match 2 1.1e-04 vmlinux-2.6.21.5 fib_lookup 2 1.1e-04 vmlinux-2.6.21.5 fib_rules_tclass 2 1.1e-04 vmlinux-2.6.21.5 find_group_orlov 2 1.1e-04 vmlinux-2.6.21.5 find_mergeable_anon_vma 2 1.1e-04 vmlinux-2.6.21.5 find_revoke_record 2 1.1e-04 vmlinux-2.6.21.5 flush_old_files 2 1.1e-04 vmlinux-2.6.21.5 free_uid 2 1.1e-04 vmlinux-2.6.21.5 get_jiffies_64 2 1.1e-04 vmlinux-2.6.21.5 grow_dev_page 2 1.1e-04 vmlinux-2.6.21.5 ifind_fast 2 1.1e-04 vmlinux-2.6.21.5 iget_locked 2 1.1e-04 vmlinux-2.6.21.5 inet_twdr_do_twkill_work 2 1.1e-04 vmlinux-2.6.21.5 inode_add_bytes 2 1.1e-04 vmlinux-2.6.21.5 inode_sub_bytes 2 1.1e-04 vmlinux-2.6.21.5 invalidate_mapping_pages 2 1.1e-04 vmlinux-2.6.21.5 inverted_lock 2 1.1e-04 vmlinux-2.6.21.5 ioprio_best 2 1.1e-04 vmlinux-2.6.21.5 ip_dev_find 2 1.1e-04 vmlinux-2.6.21.5 ip_reply_glue_bits 2 1.1e-04 vmlinux-2.6.21.5 ipc_checkid 2 1.1e-04 vmlinux-2.6.21.5 journal_get_descriptor_buffer 2 1.1e-04 vmlinux-2.6.21.5 journal_try_to_free_buffers 2 1.1e-04 vmlinux-2.6.21.5 ksoftirqd 2 1.1e-04 vmlinux-2.6.21.5 kswapd 2 1.1e-04 vmlinux-2.6.21.5 kunmap_high 2 1.1e-04 vmlinux-2.6.21.5 ll_back_merge_fn 2 1.1e-04 vmlinux-2.6.21.5 lock_hrtimer_base 2 1.1e-04 vmlinux-2.6.21.5 mapping_tagged 2 1.1e-04 vmlinux-2.6.21.5 mark_buffer_dirty 2 1.1e-04 vmlinux-2.6.21.5 may_expand_vm 2 1.1e-04 vmlinux-2.6.21.5 migration_thread 2 1.1e-04 vmlinux-2.6.21.5 mutex_lock 2 1.1e-04 vmlinux-2.6.21.5 neigh_periodic_timer 2 1.1e-04 vmlinux-2.6.21.5 page_slot 2 1.1e-04 vmlinux-2.6.21.5 path_lookup_create 2 1.1e-04 vmlinux-2.6.21.5 prio_tree_expand 2 1.1e-04 vmlinux-2.6.21.5 radix_tree_node_rcu_free 2 1.1e-04 vmlinux-2.6.21.5 recalc_bh_state 2 1.1e-04 vmlinux-2.6.21.5 rt_check_expire 2 1.1e-04 vmlinux-2.6.21.5 rt_may_expire 2 1.1e-04 vmlinux-2.6.21.5 scsi_free_sgtable 2 1.1e-04 vmlinux-2.6.21.5 second_overflow 2 1.1e-04 vmlinux-2.6.21.5 shrink_icache_memory 2 1.1e-04 vmlinux-2.6.21.5 simple_read_from_buffer 2 1.1e-04 vmlinux-2.6.21.5 sock_alloc_fd 2 1.1e-04 vmlinux-2.6.21.5 sock_sendpage 2 1.1e-04 vmlinux-2.6.21.5 sockfs_delete_dentry 2 1.1e-04 vmlinux-2.6.21.5 strnlen_user 2 1.1e-04 vmlinux-2.6.21.5 submit_bh 2 1.1e-04 vmlinux-2.6.21.5 sys_faccessat 2 1.1e-04 vmlinux-2.6.21.5 sys_lstat64 2 1.1e-04 vmlinux-2.6.21.5 sys_sigreturn 2 1.1e-04 vmlinux-2.6.21.5 sys_waitpid 2 1.1e-04 vmlinux-2.6.21.5 task_prio 2 1.1e-04 vmlinux-2.6.21.5 tcp_fixup_rcvbuf 2 1.1e-04 vmlinux-2.6.21.5 tcp_keepalive_timer 2 1.1e-04 vmlinux-2.6.21.5 tcp_remove_reno_sacks 2 1.1e-04 vmlinux-2.6.21.5 tcp_reset 2 1.1e-04 vmlinux-2.6.21.5 tcp_v4_timewait_ack 2 1.1e-04 vmlinux-2.6.21.5 truncate_inode_pages_range 2 1.1e-04 vmlinux-2.6.21.5 try_to_release_page 2 1.1e-04 vmlinux-2.6.21.5 twa_unmap_scsi_data 2 1.1e-04 vmlinux-2.6.21.5 vfs_lstat_fd 2 1.1e-04 vmlinux-2.6.21.5 vfs_readdir 2 1.1e-04 vmlinux-2.6.21.5 wake_futex 2 1.1e-04 vmlinux-2.6.21.5 wakeup_kswapd 2 1.1e-04 vmlinux-2.6.21.5 xfsbufd_wakeup 1 5.3e-05 amd get_exported_ap 1 5.3e-05 amd get_first_exported_ap 1 5.3e-05 cat (no symbols) 1 5.3e-05 httpd ap_allow_options 1 5.3e-05 httpd ap_count_dirs 1 5.3e-05 httpd ap_create_conn_config 1 5.3e-05 httpd ap_create_request_config 1 5.3e-05 httpd ap_get_limit_req_body 1 5.3e-05 httpd ap_get_server_port 1 5.3e-05 httpd ap_increment_counts 1 5.3e-05 httpd ap_run_http_method 1 5.3e-05 httpd ap_run_insert_error_filter 1 5.3e-05 httpd ap_run_pre_connection 1 5.3e-05 httpd apr_brigade_split 1 5.3e-05 httpd apr_bucket_alloc_create_ex 1 5.3e-05 httpd apr_bucket_file_make 1 5.3e-05 httpd apr_bucket_mmap_make 1 5.3e-05 httpd apr_bucket_simple_copy 1 5.3e-05 httpd apr_bucket_socket_create 1 5.3e-05 httpd apr_file_flags_get 1 5.3e-05 httpd apr_is_empty_array 1 5.3e-05 httpd apr_pstrndup 1 5.3e-05 httpd apr_table_addn 1 5.3e-05 httpd apr_table_overlay 1 5.3e-05 httpd apr_vsnprintf 1 5.3e-05 httpd basic_http_header 1 5.3e-05 httpd blowfishFeedback 1 5.3e-05 httpd core_override_type 1 5.3e-05 httpd handle_map_file 1 5.3e-05 httpd log_connection_status 1 5.3e-05 httpd log_remote_host 1 5.3e-05 httpd log_request_line 1 5.3e-05 httpd lookup_header 1 5.3e-05 httpd stuffbuffer 1 5.3e-05 httpd trimString 1 5.3e-05 libnetsnmphelpers.so.5.1.2 (no symbols) 1 5.3e-05 libnetsnmpmibs.so.5.1.2 (no symbols) 1 5.3e-05 libpthread-0.60.so __libc_sigaction 1 5.3e-05 libpthread-0.60.so __waitpid_nocancel 1 5.3e-05 libpthread-0.60.so pthread_rwlock_rdlock 1 5.3e-05 libpthread-0.60.so waitpid 1 5.3e-05 python2.4 Exception__init__ 1 5.3e-05 python2.4 PyDict_GetItem 1 5.3e-05 python2.4 PyEval_EvalCodeEx 1 5.3e-05 python2.4 PyFunction_New 1 5.3e-05 python2.4 PyObject_GetAttr 1 5.3e-05 python2.4 PyObject_Malloc 1 5.3e-05 python2.4 PyObject_Size 1 5.3e-05 python2.4 PyString_FromFormatV 1 5.3e-05 python2.4 load_args 1 5.3e-05 python2.4 posix_listdir 1 5.3e-05 python2.4 sre_charset 1 5.3e-05 python2.4 string_startswith 1 5.3e-05 python2.4 vgetargs1 1 5.3e-05 python2.4 vgetargskeywords 1 5.3e-05 vmlinux-2.6.21.5 __add_entropy_words 1 5.3e-05 vmlinux-2.6.21.5 __clear_page_buffers 1 5.3e-05 vmlinux-2.6.21.5 __d_free 1 5.3e-05 vmlinux-2.6.21.5 __ext3_journal_get_create_access 1 5.3e-05 vmlinux-2.6.21.5 __fib_res_prefsrc 1 5.3e-05 vmlinux-2.6.21.5 __group_send_sig_info 1 5.3e-05 vmlinux-2.6.21.5 __insert_inode_hash 1 5.3e-05 vmlinux-2.6.21.5 __jbd_kmalloc 1 5.3e-05 vmlinux-2.6.21.5 __journal_drop_transaction 1 5.3e-05 vmlinux-2.6.21.5 __journal_insert_checkpoint 1 5.3e-05 vmlinux-2.6.21.5 __journal_remove_checkpoint 1 5.3e-05 vmlinux-2.6.21.5 __lookup_tag 1 5.3e-05 vmlinux-2.6.21.5 __posix_lock_file_conf 1 5.3e-05 vmlinux-2.6.21.5 __put_user_4 1 5.3e-05 vmlinux-2.6.21.5 __random32 1 5.3e-05 vmlinux-2.6.21.5 __rb_rotate_left 1 5.3e-05 vmlinux-2.6.21.5 __set_page_dirty_nobuffers 1 5.3e-05 vmlinux-2.6.21.5 __sigqueue_alloc 1 5.3e-05 vmlinux-2.6.21.5 __sigqueue_free 1 5.3e-05 vmlinux-2.6.21.5 __vma_link_rb 1 5.3e-05 vmlinux-2.6.21.5 __writeback_single_inode 1 5.3e-05 vmlinux-2.6.21.5 _d_rehash 1 5.3e-05 vmlinux-2.6.21.5 _tw32_flush 1 5.3e-05 vmlinux-2.6.21.5 alloc_new_reservation 1 5.3e-05 vmlinux-2.6.21.5 anon_pipe_buf_release 1 5.3e-05 vmlinux-2.6.21.5 anon_vma_prepare 1 5.3e-05 vmlinux-2.6.21.5 arp_bind_neighbour 1 5.3e-05 vmlinux-2.6.21.5 arp_rcv 1 5.3e-05 vmlinux-2.6.21.5 as_antic_timeout 1 5.3e-05 vmlinux-2.6.21.5 as_work_handler 1 5.3e-05 vmlinux-2.6.21.5 bitmap_search_next_usable_block 1 5.3e-05 vmlinux-2.6.21.5 blk_start_queueing 1 5.3e-05 vmlinux-2.6.21.5 can_share_swap_page 1 5.3e-05 vmlinux-2.6.21.5 check_itimerval 1 5.3e-05 vmlinux-2.6.21.5 copy_thread_group_keys 1 5.3e-05 vmlinux-2.6.21.5 de_thread 1 5.3e-05 vmlinux-2.6.21.5 debug_mutex_set_owner 1 5.3e-05 vmlinux-2.6.21.5 do_exit 1 5.3e-05 vmlinux-2.6.21.5 do_mremap 1 5.3e-05 vmlinux-2.6.21.5 do_notify_resume 1 5.3e-05 vmlinux-2.6.21.5 do_setitimer 1 5.3e-05 vmlinux-2.6.21.5 do_signal 1 5.3e-05 vmlinux-2.6.21.5 do_sys_ftruncate 1 5.3e-05 vmlinux-2.6.21.5 do_syslog 1 5.3e-05 vmlinux-2.6.21.5 do_truncate 1 5.3e-05 vmlinux-2.6.21.5 do_writepages 1 5.3e-05 vmlinux-2.6.21.5 dst_run_gc 1 5.3e-05 vmlinux-2.6.21.5 dummy_file_fcntl 1 5.3e-05 vmlinux-2.6.21.5 dummy_file_lock 1 5.3e-05 vmlinux-2.6.21.5 dummy_file_mmap 1 5.3e-05 vmlinux-2.6.21.5 dummy_inode_follow_link 1 5.3e-05 vmlinux-2.6.21.5 dummy_sem_semop 1 5.3e-05 vmlinux-2.6.21.5 dummy_task_to_inode 1 5.3e-05 vmlinux-2.6.21.5 dup_task_struct 1 5.3e-05 vmlinux-2.6.21.5 elv_may_queue 1 5.3e-05 vmlinux-2.6.21.5 elv_merged_request 1 5.3e-05 vmlinux-2.6.21.5 end_buffer_write_sync 1 5.3e-05 vmlinux-2.6.21.5 eth_header 1 5.3e-05 vmlinux-2.6.21.5 exec_mmap 1 5.3e-05 vmlinux-2.6.21.5 exit_mm 1 5.3e-05 vmlinux-2.6.21.5 ext3_alloc_branch 1 5.3e-05 vmlinux-2.6.21.5 ext3_check_dir_entry 1 5.3e-05 vmlinux-2.6.21.5 ext3_clear_inode 1 5.3e-05 vmlinux-2.6.21.5 ext3_releasepage 1 5.3e-05 vmlinux-2.6.21.5 ext3_rsv_window_add 1 5.3e-05 vmlinux-2.6.21.5 ext3_splice_branch 1 5.3e-05 vmlinux-2.6.21.5 ext3_update_dx_flag 1 5.3e-05 vmlinux-2.6.21.5 fasync_helper 1 5.3e-05 vmlinux-2.6.21.5 fib_select_default 1 5.3e-05 vmlinux-2.6.21.5 find_dcookie 1 5.3e-05 vmlinux-2.6.21.5 find_next_usable_block 1 5.3e-05 vmlinux-2.6.21.5 find_or_create_page 1 5.3e-05 vmlinux-2.6.21.5 flock_to_posix_lock 1 5.3e-05 vmlinux-2.6.21.5 forget_original_parent 1 5.3e-05 vmlinux-2.6.21.5 free_pages_and_swap_cache 1 5.3e-05 vmlinux-2.6.21.5 generic_pipe_buf_map 1 5.3e-05 vmlinux-2.6.21.5 get_init_ra_size 1 5.3e-05 vmlinux-2.6.21.5 get_random_int 1 5.3e-05 vmlinux-2.6.21.5 get_signal_to_deliver 1 5.3e-05 vmlinux-2.6.21.5 getnstimeofday 1 5.3e-05 vmlinux-2.6.21.5 handle_signal 1 5.3e-05 vmlinux-2.6.21.5 hrtimer_init 1 5.3e-05 vmlinux-2.6.21.5 hrtimer_start 1 5.3e-05 vmlinux-2.6.21.5 inet_csk_get_port 1 5.3e-05 vmlinux-2.6.21.5 inet_csk_init_xmit_timers 1 5.3e-05 vmlinux-2.6.21.5 inet_csk_reset_keepalive_timer 1 5.3e-05 vmlinux-2.6.21.5 inet_sendpage 1 5.3e-05 vmlinux-2.6.21.5 inet_twdr_hangman 1 5.3e-05 vmlinux-2.6.21.5 init_buffer_head 1 5.3e-05 vmlinux-2.6.21.5 inode_setattr 1 5.3e-05 vmlinux-2.6.21.5 invalidate_complete_page 1 5.3e-05 vmlinux-2.6.21.5 ip_cmsg_recv 1 5.3e-05 vmlinux-2.6.21.5 ipc_unlock 1 5.3e-05 vmlinux-2.6.21.5 ipv4_dst_check 1 5.3e-05 vmlinux-2.6.21.5 itimer_get_remtime 1 5.3e-05 vmlinux-2.6.21.5 jiffies_to_timespec 1 5.3e-05 vmlinux-2.6.21.5 job_control 1 5.3e-05 vmlinux-2.6.21.5 journal_alloc_journal_head 1 5.3e-05 vmlinux-2.6.21.5 journal_remove_journal_head 1 5.3e-05 vmlinux-2.6.21.5 journal_unfile_buffer 1 5.3e-05 vmlinux-2.6.21.5 journal_write_revoke_records 1 5.3e-05 vmlinux-2.6.21.5 kernel_read 1 5.3e-05 vmlinux-2.6.21.5 ktime_get 1 5.3e-05 vmlinux-2.6.21.5 locks_init_lock 1 5.3e-05 vmlinux-2.6.21.5 math_state_restore 1 5.3e-05 vmlinux-2.6.21.5 mutex_trylock 1 5.3e-05 vmlinux-2.6.21.5 n_tty_chars_in_buffer 1 5.3e-05 vmlinux-2.6.21.5 neigh_update 1 5.3e-05 vmlinux-2.6.21.5 next_pidmap 1 5.3e-05 vmlinux-2.6.21.5 nsec_to_clock_t 1 5.3e-05 vmlinux-2.6.21.5 old_mmap 1 5.3e-05 vmlinux-2.6.21.5 page_mkclean_file 1 5.3e-05 vmlinux-2.6.21.5 pmd_ctor 1 5.3e-05 vmlinux-2.6.21.5 posix_lock_file 1 5.3e-05 vmlinux-2.6.21.5 prio_tree_insert 1 5.3e-05 vmlinux-2.6.21.5 proc_alloc_inode 1 5.3e-05 vmlinux-2.6.21.5 proc_flush_task 1 5.3e-05 vmlinux-2.6.21.5 proc_sys_lookup_table_one 1 5.3e-05 vmlinux-2.6.21.5 process_timeout 1 5.3e-05 vmlinux-2.6.21.5 profile_munmap 1 5.3e-05 vmlinux-2.6.21.5 pty_unthrottle 1 5.3e-05 vmlinux-2.6.21.5 pty_write_room 1 5.3e-05 vmlinux-2.6.21.5 put_unused_fd 1 5.3e-05 vmlinux-2.6.21.5 radix_tree_gang_lookup 1 5.3e-05 vmlinux-2.6.21.5 release_task 1 5.3e-05 vmlinux-2.6.21.5 release_vm86_irqs 1 5.3e-05 vmlinux-2.6.21.5 resume_userspace 1 5.3e-05 vmlinux-2.6.21.5 rt_set_nexthop 1 5.3e-05 vmlinux-2.6.21.5 rtc_cmos_read 1 5.3e-05 vmlinux-2.6.21.5 save_i387 1 5.3e-05 vmlinux-2.6.21.5 sched_migrate_task 1 5.3e-05 vmlinux-2.6.21.5 search_reserve_window 1 5.3e-05 vmlinux-2.6.21.5 secure_ip_id 1 5.3e-05 vmlinux-2.6.21.5 select_parent 1 5.3e-05 vmlinux-2.6.21.5 send_group_sig_info 1 5.3e-05 vmlinux-2.6.21.5 seq_read 1 5.3e-05 vmlinux-2.6.21.5 set_page_dirty 1 5.3e-05 vmlinux-2.6.21.5 setup_frame 1 5.3e-05 vmlinux-2.6.21.5 sha_init 1 5.3e-05 vmlinux-2.6.21.5 shrink_dqcache_memory 1 5.3e-05 vmlinux-2.6.21.5 signal_wake_up 1 5.3e-05 vmlinux-2.6.21.5 skb_free_datagram 1 5.3e-05 vmlinux-2.6.21.5 skb_split 1 5.3e-05 vmlinux-2.6.21.5 sock_attach_fd 1 5.3e-05 vmlinux-2.6.21.5 sock_disable_timestamp 1 5.3e-05 vmlinux-2.6.21.5 sync_supers 1 5.3e-05 vmlinux-2.6.21.5 sys_alarm 1 5.3e-05 vmlinux-2.6.21.5 sys_execve 1 5.3e-05 vmlinux-2.6.21.5 sys_exit_group 1 5.3e-05 vmlinux-2.6.21.5 sys_getpid 1 5.3e-05 vmlinux-2.6.21.5 sys_llseek 1 5.3e-05 vmlinux-2.6.21.5 sys_lseek 1 5.3e-05 vmlinux-2.6.21.5 sys_mkdirat 1 5.3e-05 vmlinux-2.6.21.5 sys_pipe 1 5.3e-05 vmlinux-2.6.21.5 sys_wait4 1 5.3e-05 vmlinux-2.6.21.5 tcp_incr_quickack 1 5.3e-05 vmlinux-2.6.21.5 tcp_init_xmit_timers 1 5.3e-05 vmlinux-2.6.21.5 tcp_update_reordering 1 5.3e-05 vmlinux-2.6.21.5 tcp_v4_send_ack 1 5.3e-05 vmlinux-2.6.21.5 tcp_v4_send_reset 1 5.3e-05 vmlinux-2.6.21.5 tcp_xmit_probe_skb 1 5.3e-05 vmlinux-2.6.21.5 test_clear_page_writeback 1 5.3e-05 vmlinux-2.6.21.5 tg3_timer 1 5.3e-05 vmlinux-2.6.21.5 throttle_vm_writeout 1 5.3e-05 vmlinux-2.6.21.5 try_to_free_buffers 1 5.3e-05 vmlinux-2.6.21.5 tty_default_put_char 1 5.3e-05 vmlinux-2.6.21.5 tty_hung_up_p 1 5.3e-05 vmlinux-2.6.21.5 tty_ldisc_ref_wait 1 5.3e-05 vmlinux-2.6.21.5 tty_ldisc_try 1 5.3e-05 vmlinux-2.6.21.5 tty_paranoia_check 1 5.3e-05 vmlinux-2.6.21.5 tty_poll 1 5.3e-05 vmlinux-2.6.21.5 tty_wakeup 1 5.3e-05 vmlinux-2.6.21.5 unlock_new_inode 1 5.3e-05 vmlinux-2.6.21.5 unmap_underlying_metadata 1 5.3e-05 vmlinux-2.6.21.5 unshare_files 1 5.3e-05 vmlinux-2.6.21.5 uptime_read_proc 1 5.3e-05 vmlinux-2.6.21.5 verify_chain 1 5.3e-05 vmlinux-2.6.21.5 vfs_llseek 1 5.3e-05 vmlinux-2.6.21.5 vfs_lstat 1 5.3e-05 vmlinux-2.6.21.5 vma_prio_tree_insert 1 5.3e-05 vmlinux-2.6.21.5 vma_prio_tree_remove 1 5.3e-05 vmlinux-2.6.21.5 vmtruncate 1 5.3e-05 vmlinux-2.6.21.5 wake_up_new_task 1 5.3e-05 vmlinux-2.6.21.5 writeback_inodes
Turn off slab debugging and preempt, and see if that helps.
Jeff
Quoted 6 lines
# opreport -l CPU: AMD64 processors, speed 1994.52 MHz (estimated) Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 100000 samples % app name symbol name 914379 48.8404 vmlinux-2.6.21.5 check_poison_obj 341920 18.2632 vmlinux-2.6.21.5 poison_obj
You have slab debugging enabled. That makes everything slow. It costs you ~66% of your CPU time. Disable it.
-Andi
Quoted 2 lines · Andi Kleen
You have slab debugging enabled. That makes everything slow. It costs you ~66% of your CPU time. Disable it.
Past peak for the day, but this does look like the culprit. Thanks
to all who replied.
Quoted 10 lines · Aaron Porter
Quoted 9 lines · Andi Kleen
Quoted 7 lines · Aaron Porter
I'm in the process up upgrading a pool of apache servers from
2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load, 2.6.21 has a load average more than 3 times higher, cpu 0 spends well over 90% of its time in interrupts (vs ~30% under 2.6.17). When we hit 3k apache sessions, ksoftirqd eats 100% of cpu0 and our network traffic drops off rapidly. The end result is that 2.6.17 performs twice as well under this load.
Quoted 8 lines
Quoted 1 line
Can you oprofile it?
# opreport -l CPU: AMD64 processors, speed 1994.52 MHz (estimated) Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 100000 samples % app name symbol name 914379 48.8404 vmlinux-2.6.21.5 check_poison_obj 341920 18.2632 vmlinux-2.6.21.5 poison_obj
I bet you have CONFIG_DEBUG_SLAB turned off in your 2.6.17 kernel, and turned on in your 2.6.21 kernel.
-- Chris
From: Aaron Porter <[email protected]>
Date: Tue, 24 Jul 2007 11:49:09 -0700
Quoted 4 lines
samples % app name symbol name 914379 48.8404 vmlinux-2.6.21.5 check_poison_obj 341920 18.2632 vmlinux-2.6.21.5 poison_obj 37355 1.9953 nf_conntrack (no symbols)
You have SLAB debugging enabled, turn it off if you want decent performance.
Quoted 7 lines · Aaron Porter
I'm in the process up upgrading a pool of apache servers from
2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load, 2.6.21 has a load average more than 3 times higher, cpu 0 spends well over 90% of its time in interrupts (vs ~30% under 2.6.17). When we hit 3k apache sessions, ksoftirqd eats 100% of cpu0 and our network traffic drops off rapidly. The end result is that 2.6.17 performs twice as well under this load.
Is it always CPU 0, or does it move? Are you running irqbalance? If you're running irqbalance, you can run a script that alternates between 'cat /proc/interrupts' and 'mpstat -P ALL 5 10' and watch the offending interrupt jump around between processors. It's not as informative as oprofile, as Andi suggested, but it's really easy to set up.
-- Chris
Quoted 5 lines · Chris Snook
Is it always CPU 0, or does it move? Are you running irqbalance? If you're running irqbalance, you can run a script that alternates between 'cat
/proc/interrupts' and 'mpstat -P ALL 5 10' and watch the offending interrupt jump around between processors. It's not as informative as oprofile, as Andi suggested, but it's really easy to set up.
Always cpu0
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
0: 166 0 0 0 XT-PIC-XT timer
1: 8 0 0 0 XT-PIC-XT i8042
2: 0 0 0 0 XT-PIC-XT cascade
3: 11 0 0 0 XT-PIC-XT serial
4: 830 0 0 0 XT-PIC-XT serial
5: 370787109 0 0 0 XT-PIC-XT eth0
6: 3 0 0 0 XT-PIC-XT floppy
8: 4 0 0 0 XT-PIC-XT rtc
9: 0 0 0 0 XT-PIC-XT ohci_hcd:usb1, ohci_hcd:usb2
10: 80937699 0 0 0 XT-PIC-XT 3w-9xxx, 3w-9xxx
12: 5 0 0 0 XT-PIC-XT i8042
14: 50931 0 0 0 XT-PIC-XT ide0
NMI: 1110304 275156 276732 233082
LOC: 278410000 278410022 278410013 278410014
ERR: 55003177
MIS: 0
# mpstat -P ALL 5 1
Linux 2.6.21.5 (*.com) 07/24/0711:52:30 CPU %user %nice %system %iowait %irq %soft %idle intr/s 11:52:35 all 0.40 0.00 16.12 13.56 0.25 25.13 44.54 299.80 11:52:35 0 0.00 0.00 0.00 0.00 1.00 99.00 0.00 299.80 11:52:35 1 0.80 0.00 23.65 23.05 0.00 0.40 52.10 0.00 11:52:35 2 0.40 0.00 21.64 14.23 0.00 0.40 63.53 0.00 11:52:35 3 0.60 0.00 19.44 17.03 0.00 0.60 62.73 0.00
Average: CPU %user %nice %system %iowait %irq %soft %idle intr/s Average: all 0.40 0.00 16.12 13.56 0.25 25.13 44.54 299.80 Average: 0 0.00 0.00 0.00 0.00 1.00 99.00 0.00 299.80 Average: 1 0.80 0.00 23.65 23.05 0.00 0.40 52.10 0.00 Average: 2 0.40 0.00 21.64 14.23 0.00 0.40 63.53 0.00 Average: 3 0.60 0.00 19.44 17.03 0.00 0.60 62.73 0.00
#