Skip to contentSkip to conversation

cpuset trouble after hibernate

DISCUSSION Nicolas Capit 6 participants 8 messages Sep 8, 2007 – Sep 19, 2007

lore
Share & tools
lore
#0 lore raw

Hello,

This is my situation:
- I mounted the pseudo cpuset filesystem into /dev/cpuset
- I created a cpuset named oar with my 2 cpus

cat /dev/cpuset/oar/cpus
0-1

- Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
- After reboot:

cat /dev/cpuset/oar/cpus
0

Why did I lost a cpu?
Is this a normal behavior???

Thank you for your attention,
Nicolas Capit

Note on my system:
- laptop HP dv2000 with a Turion64x2
- kernel : Linux 2.6.22-1-686 #1 SMP Sun Jul 29 14:37:42 UTC
2007 i686 GNU/Linux

#1 lore raw
Quoted 9 lines · Nicolas Capit

Hello,

This is my situation:
- I mounted the pseudo cpuset filesystem into /dev/cpuset
- I created a cpuset named oar with my 2 cpus

cat /dev/cpuset/oar/cpus
0-1

Quoted 8 lines

- Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
- After reboot:

cat /dev/cpuset/oar/cpus
0

Why did I lost a cpu?
Is this a normal behavior???

Hi Nicolas,

I believe this is related to the fact that hibernation uses the hotplug subsystem to disable all CPUs except the boot CPU.

Thus guarantee_online_cpus() is called on each cpuset and removes all CPUs, except CPU 0, from all cpusets.

I'm not quite sure about if/how this should be fixed in the kernel, though. Looks like a very simple user-land workaround would be enough.

Simon.

#2 lore raw
Quoted 25 lines · On Mon, 10 Sep 2007 11:45:10 +0200 (CEST) Simon Derr <[email protected]>…
Quoted 9 lines · Nicolas Capit

Hello,

This is my situation:
- I mounted the pseudo cpuset filesystem into /dev/cpuset
- I created a cpuset named oar with my 2 cpus

cat /dev/cpuset/oar/cpus
0-1

Quoted 8 lines

- Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
- After reboot:

cat /dev/cpuset/oar/cpus
0

Why did I lost a cpu?
Is this a normal behavior???

Hi Nicolas,

I believe this is related to the fact that hibernation uses the hotplug subsystem to disable all CPUs except the boot CPU.

Quoted 7 lines

Thus guarantee_online_cpus() is called on each cpuset and removes all CPUs, except CPU 0, from all cpusets.

I'm not quite sure about if/how this should be fixed in the kernel, though. Looks like a very simple user-land workaround would be enough.

Simon.

Ok, it is not a big deal for me (you don't hibernate computers in a cluster...) but I wanted to bring you this issue in case of a similar behavior in another case...

Thanks for your response.

Nicolas Capit.

#3 lore raw
In reply to Simon Derr
Quoted 22 lines · On Mon, 10 Sep 2007 11:45:10 +0200 (CEST) Simon Derr <[email protected]>…
Quoted 9 lines · Nicolas Capit

Hello,

This is my situation:
- I mounted the pseudo cpuset filesystem into /dev/cpuset
- I created a cpuset named oar with my 2 cpus

cat /dev/cpuset/oar/cpus
0-1

Quoted 8 lines

- Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
- After reboot:

cat /dev/cpuset/oar/cpus
0

Why did I lost a cpu?
Is this a normal behavior???

Hi Nicolas,

Quoted 9 lines

I believe this is related to the fact that hibernation uses the hotplug subsystem to disable all CPUs except the boot CPU.

Thus guarantee_online_cpus() is called on each cpuset and removes all CPUs, except CPU 0, from all cpusets.

I'm not quite sure about if/how this should be fixed in the kernel, though. Looks like a very simple user-land workaround would be enough.

Yeah. Bug, surely. But I guess it's always been there.

What are the implications of this for cpusets-via-containers?

#6 lore raw
In reply to Nicolas Capit

Hi!

Quoted 7 lines

This is my situation:
- I mounted the pseudo cpuset filesystem into /dev/cpuset
- I created a cpuset named oar with my 2 cpus

cat /dev/cpuset/oar/cpus
0-1

Quoted 8 lines

- Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
- After reboot:

cat /dev/cpuset/oar/cpus
0

Why did I lost a cpu?
Is this a normal behavior???

One of the cpus was unplugged during suspend... perhaps some save/restore is needed during hotplug/unplug?

Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#4 lore raw
In reply to Andrew Morton
Quoted 5 lines · Andrew Morton

Yeah. Bug, surely. But I guess it's always been there.

What are the implications of this for cpusets-via-containers?

I don't think it should be any different from the previous version - I tried to avoid touching those bits of cpusets where possible.

Paul

#5 lore raw
In reply to Andrew Morton
Quoted 29 lines · Andrew Morton
Quoted 27 lines · On Mon, 10 Sep 2007 11:45:10 +0200 (CEST) Simon Derr <[email protected]>…
Quoted 9 lines · Nicolas Capit

Hello,

This is my situation:
- I mounted the pseudo cpuset filesystem into /dev/cpuset
- I created a cpuset named oar with my 2 cpus

cat /dev/cpuset/oar/cpus
0-1

Quoted 8 lines

- Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
- After reboot:

cat /dev/cpuset/oar/cpus
0

Why did I lost a cpu?
Is this a normal behavior???

Hi Nicolas,

I believe this is related to the fact that hibernation uses the hotplug subsystem to disable all CPUs except the boot CPU.

Thus guarantee_online_cpus() is called on each cpuset and removes all CPUs, except CPU 0, from all cpusets.

Quoted 8 lines
Quoted 3 lines

I'm not quite sure about if/how this should be fixed in the kernel, though. Looks like a very simple user-land workaround would be enough.

Yeah. Bug, surely. But I guess it's always been there.

What are the implications of this for cpusets-via-containers?

I suspect the functionality of cpusets is not affected by containers.

I wonder if containers should become suspend/resume aware and pass that option on to controllers. I think it's only the bus drivers and device drivers that do that now.

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#7 lore raw
In reply to Pavel Machek
Quoted 3 lines · Pavel Machek

One of the cpus was unplugged during suspend... perhaps some save/restore is needed during hotplug/unplug?

Or else keep track separately in cpusets of

- cpus that the cpuset can run on
- cpus that the admin has specified for the cpu to run on

hotplug/hotunplug events would only affect the former; userspace would only see/modify the latter. Then when hibernate is over and the CPUs are hotplugged back in, things would be back as before.

Paul