Skip to contentSkip to conversation

Hotplug CPU support for Oprofile

DISCUSSION Mike Travis 2 participants 2 messages May 5, 2008 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): 1210012443.6374.23.camel@snuffleupagus

#1 lore raw

[adding some possibly interested parties to the cc line.]

Quoted 17 lines · Chris J Arges

Mike,
I was wondering what your rationale behind using per_cpu variables for the cpu buffers in oprofile versus using percpu_alloc, percpu_free? For instance:

        DEFINE_PER_CPU_SHARED_ALIGNED(struct oprofile_cpu_buffer,
        cpu_buffer);

        void free_cpu_buffers(void)
        {
                int i;
        
                for_each_online_cpu(i)
                        vfree(per_cpu(cpu_buffer, i).buffer);
        
        }
Quoted 9 lines

Could it be possible to write it like this:

        struct oprofile_cpu_buffer *cpu_buffer;
        
        void free_cpu_buffers(void)
        {
        	percpu_free(cpu_buffer);
        }

Hi Chris,

There was no particular reason for one or the other. We did have a patch a few months ago that added CPU_ALLOC functionality that was much cleaner than percpu_alloc (and replaced it), but due to circumstances it didn't actually make it into the source base. The percpu approach was the fallback.

Quoted 8 lines

Phillipe, Mike,
I am currently working on a patch for enabling hotplug cpu support for oprofile as currently oprofile crashes whenever cpus are onlined of offlined. So far my patch registers a callback function for hotplug events, and checked if a cpu has been onlined/offlined and then allocs/frees a cpu buffer accordingly.

Quoted 5 lines

I am wondering if you have any further recommendations as to approaching adding hotplug support. I am still fairly new to oprofile/linux kernel but I would like to contribute, please forgive any newbie mistakes. Thanks!

Other than insuring that any routines for bringing up/tearing down processors are in the __cpuinit/data sections and test as many different configs as you can.

Also, increasing NR_CPUS to 4096 is one thing but I've been testing with "possible" cpus == 4096 and have hit some snags. The biggest right now is the enormous amount of memory that the kernel tracing facility requires. So trying out your code with a healthy amount of possible cpus will expose problems like this.

Another thing that I've encountered is that you can add "additional_cpus" to the boot line, and the BIOS can signal that there are cpus present but currently disabled, but there's no way to bring these cpus online after bootup.

One thought would be to register the cpu but make it offline. This has some advantages but it was felt that instead something more like:

echo '32-63' > /sys/devices/system/cpu/online

would startup and register cpus 32-63. This is orthogonal to the showing of which cpu's are online:

# cat /sys/devices/system/cpu/online
0-31

Any suggestions along this area are welcome!

Thanks,
Mike

#2 lore raw

The allocpercpu approach creates a pointer array and results in objects distributed over all of memory. A cpu alloc approach would place dynamically allocated per cpu objects together which is better for memory consumption (no need to align) and performance (same TLB entry, neigboring memory cells, possible shared cachelines).