Hotplug CPU support for Oprofile
A message that is not in the archive (sent off-list or not yet received): 1210012443.6374.23.camel@snuffleupagus
[adding some possibly interested parties to the cc line.]
Quoted 17 lines · Chris J Arges
Mike,
I was wondering what your rationale behind usingper_cpuvariables for the cpu buffers in oprofile versus usingpercpu_alloc,percpu_free? For instance:DEFINE_PER_CPU_SHARED_ALIGNED(struct oprofile_cpu_buffer, cpu_buffer); void free_cpu_buffers(void) { int i; for_each_online_cpu(i) vfree(per_cpu(cpu_buffer, i).buffer); }
Quoted 9 lines
Could it be possible to write it like this:
struct oprofile_cpu_buffer *cpu_buffer; void free_cpu_buffers(void) { percpu_free(cpu_buffer); }
Hi Chris,
There was no particular reason for one or the other. We did have a patch a few months ago that added CPU_ALLOC functionality that was much cleaner than percpu_alloc (and replaced it), but due to circumstances it didn't actually make it into the source base. The percpu approach was the fallback.
Quoted 8 lines
Phillipe, Mike,
I am currently working on a patch for enabling hotplug cpu support for oprofile as currently oprofile crashes whenever cpus are onlined of offlined. So far my patch registers a callback function for hotplug events, and checked if a cpu has been onlined/offlined and then allocs/frees a cpu buffer accordingly.
Quoted 5 lines
I am wondering if you have any further recommendations as to approaching adding hotplug support. I am still fairly new to oprofile/linux kernel but I would like to contribute, please forgive any newbie mistakes. Thanks!
Other than insuring that any routines for bringing up/tearing down processors are in the __cpuinit/data sections and test as many different configs as you can.
Also, increasing NR_CPUS to 4096 is one thing but I've been testing with "possible" cpus == 4096 and have hit some snags. The biggest right now is the enormous amount of memory that the kernel tracing facility requires. So trying out your code with a healthy amount of possible cpus will expose problems like this.
Another thing that I've encountered is that you can add "additional_cpus" to the boot line, and the BIOS can signal that there are cpus present but currently disabled, but there's no way to bring these cpus online after bootup.
One thought would be to register the cpu but make it offline. This has some advantages but it was felt that instead something more like:
echo '32-63' > /sys/devices/system/cpu/online
would startup and register cpus 32-63. This is orthogonal to the showing of which cpu's are online:
# cat /sys/devices/system/cpu/online 0-31
Any suggestions along this area are welcome!
Thanks,
Mike
The allocpercpu approach creates a pointer array and results in objects distributed over all of memory. A cpu alloc approach would place dynamically allocated per cpu objects together which is better for memory consumption (no need to align) and performance (same TLB entry, neigboring memory cells, possible shared cachelines).