Skip to contentSkip to conversation

kernel compiling performance challenge

DISCUSSION Nick Piggin 1 participant 2 messages Oct 7, 2005 – Oct 8, 2005

lore
Share & tools
lore
#0 lore raw

I was recently disappointed to find that 2.4 kernels very slightly edge out the latest 2.6 kernels in kernel compiling performance on my dual Xeon, even when 2.6 has HZ set at 100.

So I spent a few days trying some different mm/ optimisations, and managed to reduce total kernel residency (excluding idle time) by 7% on that workload, and now manage to beat 2.4 by a good third of a second.

Here's a diffprofile versus a plain 2.6.14-rc3 kernel:

       123   384.4% __get_zone_counts
        54     0.0% __page_set_anon_rmap
        37    17.0% find_lock_page
        28    31.8% lru_cache_add_active
        26    42.6% path_lookup
        25    22.3% kmem_cache_alloc
        20     6.2% find_vma
        19    17.3% _atomic_dec_and_lock
        18    26.5% __copy_from_user_ll
        17   188.9% shmem_nopage
        17    58.6% unmap_vmas
        15     0.0% __page_state
        14    31.1% copy_pte_range
        14    13.6% __wake_up_bit
        14     0.0% remove_vma
        14    46.7% exit_notify
        13    61.9% sys_close
        13    27.1% anon_vma_prepare
        13     0.0% unlink_file_vma
        13    92.9% do_generic_mapping_read
        12   109.1% free_pgd_range
        12     0.0% vm_stat_account
        12   100.0% sys_mmap2
        10    17.5% get_empty_filp
         .
         .
       -10   -21.7% _spin_unlock_irq
       -10   -90.9% flush_old_exec
       -10   -25.6% vfs_read
       -10    -4.6% __handle_mm_fault
       -12   -17.1% do_shmem_file_read
       -12   -60.0% cond_resched
       -12   -60.0% number
       -12   -46.2% sys_open
       -12  -100.0% __vm_stat_account
       -13   -46.4% inotify_inode_queue_event
       -13   -33.3% dput
       -13    -2.8% release_pages
       -14   -26.9% kmem_cache_free
       -14   -17.7% pte_alloc_map
       -14    -3.2% __link_path_walk
       -16  -100.0% __rmqueue
       -19   -10.9% sysenter_past_esp
       -19   -55.9% vfs_getattr
       -20   -25.0% zone_watermark_ok
       -21   -17.9% may_open
       -21   -15.8% strnlen_user
       -21    -8.8% __pagevec_lru_add_active
       -23  -100.0% remove_vm_struct
       -23    -5.1% zap_pte_range
       -28    -0.6% do_page_fault
       -33   -15.8% do_anonymous_page
       -40    -7.1% __d_lookup
       -44    -5.9% _spin_lock
       -75   -43.1% page_remove_rmap
       -79   -98.8% set_page_dirty
       -81   -23.5% free_hot_cold_page
       -81   -10.0% __copy_to_user_ll
       -94   -98.9% page_add_anon_rmap
      -108   -26.2% do_no_page
      -111   -72.1% prep_new_page
      -143  -100.0% bad_range
      -444   -87.2% __mod_page_state
      -548   -10.1% buffered_rmqueue
     -1634    -7.1% total

Depending on how much interest there is, I might keep a tree around to collect performance improvements. If you have any more[*] I could look at, please send them over. I'll eventually try to get things merged.

* Not just for kbuild, or only mm related, but preferably something that I can easily measure on my little system.

Attached is a rollup against 2.6.14-rc3. I don't currently have any webspace handy, so I can't host a broken-out tarball anywhere yet.

Sorry for the big attachment (actually most of it is Hugh's pagefault scalability prep and my lockless pagecache prep that I'm working on top of).

Nick

Signature · 1 line
SUSE Labs, Novell Inc.
#1 lore raw
Quoted 3 lines · Nick Piggin

Attached is a rollup against 2.6.14-rc3. I don't currently have any webspace handy, so I can't host a broken-out tarball anywhere yet.

Hmm, a wayward hunk to net/core/skbuff.c slipped in there - it is wrong. Just delete the net/core/skbuff.c part of the patch before applying it.

Signature · 3 lines
SUSE Labs, Novell Inc.

Send instant messages to your online friends http://au.messenger.yahoo.com