Skip to content

RFC clockevents: fix clockevent_devices list corruption after cpu hotplug

Xiaotian Feng lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest4 patches 5 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4clockevents: use list_for_each_entry_safekernel/time/clockevents.c+5 -70 replies
2/4clockevents: convert clockevents_do_notify to intkernel/time/clockevents.c+2 -20 replies
3/4clockevents: add device to clockevent_devices list if notify ADD successkernel/time/clockevents.c+13 -60 replies
4/4clockevents: remove related device from clockevents_released list when cpu is DEADkernel/time/clockevents.c+4 -10 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 0/4] clockevents: fix clockevent_devices list corruption after cpu hotplug

I've met a list_del corruption, which was reported in
http://lkml.org/lkml/2009/11/27/45. But no response, so I try to debug it by myself.

After I added some printks to show all elements in clockevent_devices, I found kernel hangs when I tried to resume from s2ram.

In clockevents_register_device, clockevents_do_notify ADD is always followed by clockevents_notify_released. Although clockevents_do_notify ADD will use tick_check_new_device to add new devices and replace old devices to the clockevents_released list, clockevents_notify_released add them back to clockevent_devices list.

My system is Quad-Core x86_64, with apic and hpet enables, after boot up, the elements in clockevent_devices list is :
clockevent_device->lapic(3)->hpet5(3)->lapic(2)->hpet4(2)->lapic(1)->hpet3(1)-
->lapic(0)->hpet2(0)->hpet(0)
* () means cpu id

But active clock_event_device is hpet2,hpet3,hpet4,hpet5. Then at s2ram stage, cpu 1,2,3 is down, then notify CLOCK_EVT_NOTIFY_CPU_DEAD will calls tick_shutdown, then hpet2,hpet3,hpet4,hpet5 was deleted from clockevent_device list. So after s2ram, elements in clockevent_device list is:
clockevent_device->lapic(3)->lapic(2)->lapic(1)->lapic(0)->hpet2(0)->hpet(0)

Then at resume stage, cpu 1,2,3 is up, it will register lapic again, and then perform list_add lapic on clockevent_device list, e.g. list_add lapic(1) on above list, lapic will move to the clockevent_device->next, but lapic(2)->next is still point to lapic(1), the list is circular and corrupted then.

This patchset aims to fixes above behaviour by:
- on clockevents_register_device, if notify ADD success, move new devices to the clockevent_devices list, otherwise move to clockevents_released list.
- on clockevents_notify_released, same behaviour as above.
- on clockevents_notify CPU_DEAD, remove related devices on dead cpu from clockevents_released list.

It makes sure that only active devices on each cpu is on clockevent_devices list. With this patchset, the list_del corruption disappeared, and suspend/resume, cpu hotplug works fine on my system.