Skip to contentSkip to conversation

Switch APIC to driver model (and make S3 sleep with APIC on)

DISCUSSION Mikael Pettersson 4 participants 12 messages Jan 28, 2003 – Feb 7, 2003

lore
Share & tools
lore
#0 lore raw
Quoted 2 lines · Pavel Machek

This switches apic code to driver model, cleans code up a lot, and makes S3 while apic is used work. Please apply,

Please don't apply this. It breaks stuff:

1. apic_suspend() unconditionally calls disable_apic_nmi_watchdog() apic_resume() unconditionally calls setup_apic_nmi_watchdog() apic_pm_state.perfctr_pmdev removed

- You're calling local-APIC NMI watchdog procedures even if the local-APIC NMI watchdog isn't active. Bad.
- You're hardcoding that the local-APIC NMI watchdog is the only possible sub-client of the local APIC. Not true.
- perfctr_pmdev exists precisely to handle both these cases in a clean way.

2. You unconditionally register apic_driver with its suspend/resume methods through a device_initcall().

This breaks if a UP_APIC or SMP kernel runs on a CPU with no or an unusable local APIC. apic_pm_init2() does a runtime check for successful init before doing a pm_register().

3. You severed the link between the PM API and the local APIC.

This breaks APM suspend when the local APIC is enabled. The machine will hang (or immediately resume). I tested this, and the driver model "stuff" simply doesn't do the right thing yet.

I you just want SOFTWARE_SUSPEND to work, why not simply post the appropriate PM_SUSPEND and PM_RESUME events?
That should work without any changes to apic.c or nmi.c.

/Mikael

#1 lore raw

Hi!

Quoted 11 lines
Quoted 2 lines

This switches apic code to driver model, cleans code up a lot, and makes S3 while apic is used work. Please apply,

Please don't apply this. It breaks stuff:

1. apic_suspend() unconditionally calls disable_apic_nmi_watchdog() apic_resume() unconditionally calls setup_apic_nmi_watchdog() apic_pm_state.perfctr_pmdev removed

- You're calling local-APIC NMI watchdog procedures even if the local-APIC NMI watchdog isn't active. Bad.

Fixed.

Quoted 4 lines

- You're hardcoding that the local-APIC NMI watchdog is the only possible sub-client of the local APIC. Not true.
- perfctr_pmdev exists precisely to handle both these cases in a clean way.

While being as ugly as night, which is even noted in sources:

- /* 'perfctr_pmdev' is here because the current (2.4.1) PM
- callback system doesn't handle hierarchical dependencies */

Nothing prevents more clients from registering as subtrees to APIC. I did not do that for NMI watchdog because it is hardcoded in Makefile, anyway.

Quoted 6 lines

2. You unconditionally register apic_driver with its suspend/resume methods through a device_initcall().

This breaks if a UP_APIC or SMP kernel runs on a CPU with no or an unusable local APIC. apic_pm_init2() does a runtime check for successful init before doing a pm_register().

Fixed.

Quoted 5 lines

3. You severed the link between the PM API and the local APIC.

This breaks APM suspend when the local APIC is enabled. The machine will hang (or immediately resume). I tested this, and the driver model "stuff" simply doesn't do the right thing yet.

I'll fix APM to call device model methods.

Quoted 3 lines

I you just want SOFTWARE_SUSPEND to work, why not simply post the appropriate PM_SUSPEND and PM_RESUME events?
That should work without any changes to apic.c or nmi.c.

Because PM_SUSPEND/PM_RESUME is ugly and can not be made to work (devices are hierarchical, and PM_SUSPEND/PM_RESUME system does not honour that).
Pavel

Signature · 2 lines
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.
#2 lore raw
In reply to Mikael Pettersson

Hi!

Quoted 4 lines
Quoted 2 lines

This switches apic code to driver model, cleans code up a lot, and makes S3 while apic is used work. Please apply,

Please don't apply this. It breaks stuff:

This one should be okay. [oprofile not tested because I don't know how to use it... If someone could test it that would be cool.]
Pavel

arch/i386/kernel/apic.c +61 -99
@@ -10,6 +10,7 @@
  *					for testing these extensively.
  *	Maciej W. Rozycki	:	Various updates and fixes.
  *	Mikael Pettersson	:	Power Management for UP-APIC.
+ *	Pavel Machek		:	Converted to driver model.
  */
 
 #include <linux/config.h>
@@ -23,6 +24,10 @@
 #include <linux/interrupt.h>
 #include <linux/mc146818rtc.h>
 #include <linux/kernel_stat.h>
+#include <linux/delay.h>
+#include <linux/slab.h>
+#include <linux/device.h>
+#include <linux/pm.h>
 
 #include <asm/atomic.h>
 #include <asm/smp.h>
@@ -54,6 +59,8 @@
 int prof_old_multiplier[NR_CPUS] = { 1, };
 int prof_counter[NR_CPUS] = { 1, };
 
+unsigned long apic_phys;
+
 int get_maxlvt(void)
 {
 	unsigned int v, ver, maxlvt;
@@ -292,6 +299,27 @@
 	apic_write_around(APIC_LVT1, value);
 }
 
+static struct {
+	/* 'active' is true if the local APIC was enabled by us and
+	   not the BIOS; this signifies that we are also responsible
+	   for disabling it before entering apm/acpi suspend */
+	int active;
+	/* r/w apic fields */
+	unsigned int apic_id;
+	unsigned int apic_taskpri;
+	unsigned int apic_ldr;
+	unsigned int apic_dfr;
+	unsigned int apic_spiv;
+	unsigned int apic_lvtt;
+	unsigned int apic_lvtpc;
+	unsigned int apic_lvt0;
+	unsigned int apic_lvt1;
+	unsigned int apic_lvterr;
+	unsigned int apic_tmict;
+	unsigned int apic_tdcr;
+	unsigned int apic_thmr;
+} apic_pm_state;
+
 void __init setup_local_APIC (void)
 {
 	unsigned long value, ver, maxlvt;
@@ -435,44 +463,19 @@
 
 	if (nmi_watchdog == NMI_LOCAL_APIC)
 		setup_apic_nmi_watchdog();
+
+	apic_pm_state.active = 1;
 }
 
 #ifdef CONFIG_PM
-
-#include <linux/slab.h>
-#include <linux/pm.h>
-
-static struct {
-	/* 'active' is true if the local APIC was enabled by us and
-	   not the BIOS; this signifies that we are also responsible
-	   for disabling it before entering apm/acpi suspend */
-	int active;
-	/* 'perfctr_pmdev' is here because the current (2.4.1) PM
-	   callback system doesn't handle hierarchical dependencies */
-	struct pm_dev *perfctr_pmdev;
-	/* r/w apic fields */
-	unsigned int apic_id;
-	unsigned int apic_taskpri;
-	unsigned int apic_ldr;
-	unsigned int apic_dfr;
-	unsigned int apic_spiv;
-	unsigned int apic_lvtt;
-	unsigned int apic_lvtpc;
-	unsigned int apic_lvt0;
-	unsigned int apic_lvt1;
-	unsigned int apic_lvterr;
-	unsigned int apic_tmict;
-	unsigned int apic_tdcr;
-	unsigned int apic_thmr;
-} apic_pm_state;
-
-static void apic_pm_suspend(void *data)
+static int apic_suspend(struct device *dev, u32 state, u32 level)
 {
 	unsigned int l, h;
 	unsigned long flags;
 
-	if (apic_pm_state.perfctr_pmdev)
-		pm_send(apic_pm_state.perfctr_pmdev, PM_SUSPEND, data);
+	if (level != SUSPEND_DISABLE)
+		return 0;
+
 	apic_pm_state.apic_id = apic_read(APIC_ID);
 	apic_pm_state.apic_taskpri = apic_read(APIC_TASKPRI);
 	apic_pm_state.apic_ldr = apic_read(APIC_LDR);
@@ -493,13 +496,19 @@
 	l &= ~MSR_IA32_APICBASE_ENABLE;
 	wrmsr(MSR_IA32_APICBASE, l, h);
 	local_irq_restore(flags);
+	return 0;
 }
 
-static void apic_pm_resume(void *data)
+static int apic_resume(struct device *dev, u32 level)
 {
 	unsigned int l, h;
 	unsigned long flags;
 
+	if (level != RESUME_POWER_ON)
+		return 0;
+
+	set_fixmap_nocache(FIX_APIC_BASE, apic_phys);		/* FIXME: this is needed for S3 resume, but why? */
+
 	local_irq_save(flags);
 	rdmsr(MSR_IA32_APICBASE, l, h);
 	l &= ~MSR_IA32_APICBASE_BASE;
@@ -524,74 +533,34 @@
 	apic_write(APIC_ESR, 0);
 	apic_read(APIC_ESR);
 	local_irq_restore(flags);
-	if (apic_pm_state.perfctr_pmdev)
-		pm_send(apic_pm_state.perfctr_pmdev, PM_RESUME, data);
-}
-
-static int apic_pm_callback(struct pm_dev *dev, pm_request_t rqst, void *data)
-{
-	switch (rqst) {
-	case PM_SUSPEND:
-		apic_pm_suspend(data);
-		break;
-	case PM_RESUME:
-		apic_pm_resume(data);
-		break;
-	}
 	return 0;
 }
 
-/* perfctr driver should call this instead of pm_register() */
-struct pm_dev *apic_pm_register(pm_dev_t type,
-				unsigned long id,
-				pm_callback callback)
-{
-	struct pm_dev *dev;
-
-	if (!apic_pm_state.active)
-		return pm_register(type, id, callback);
-	if (apic_pm_state.perfctr_pmdev)
-		return NULL;	/* we're busy */
-	dev = kmalloc(sizeof(struct pm_dev), GFP_KERNEL);
-	if (dev) {
-		memset(dev, 0, sizeof(*dev));
-		dev->type = type;
-		dev->id = id;
-		dev->callback = callback;
-		apic_pm_state.perfctr_pmdev = dev;
-	}
-	return dev;
-}
-
-/* perfctr driver should call this instead of pm_unregister() */
-void apic_pm_unregister(struct pm_dev *dev)
-{
-	if (!apic_pm_state.active) {
-		pm_unregister(dev);
-	} else if (dev == apic_pm_state.perfctr_pmdev) {
-		apic_pm_state.perfctr_pmdev = NULL;
-		kfree(dev);
-	}
-}
+static struct device_driver apic_driver = {
+	.name		= "apic",
+	.bus		= &system_bus_type,
+	.resume		= apic_resume,
+	.suspend	= apic_suspend,
+};
+
+struct sys_device device_apic = {
+	.name		= "apic",
+	.id		= 0,
+	.dev		= {
+		.name	= "APIC",
+		.driver	= &apic_driver,
+	},
+};
 
-static void __init apic_pm_init1(void)
-{
-	/* can't pm_register() at this early stage in the boot process
-	   (causes an immediate reboot), so just set the flag */
-	apic_pm_state.active = 1;
-}
-
-static void __init apic_pm_init2(void)
+static int __init init_apic_devicefs(void)
 {
+	driver_register(&apic_driver);
 	if (apic_pm_state.active)
-		pm_register(PM_SYS_DEV, 0, apic_pm_callback);
+		return sys_device_register(&device_apic);
+	return 0;
 }
 
-#else	/* CONFIG_PM */
-
-static inline void apic_pm_init1(void) { }
-static inline void apic_pm_init2(void) { }
-
+device_initcall(init_apic_devicefs);
 #endif	/* CONFIG_PM */
 
 /*
@@ -658,9 +627,6 @@
 		nmi_watchdog = NMI_LOCAL_APIC;
 
 	printk("Found and enabled local APIC!\n");
-
-	apic_pm_init1();
-
 	return 0;
 
 no_apic:
@@ -670,8 +636,6 @@
 
 void __init init_apic_mappings(void)
 {
-	unsigned long apic_phys;
-
 	/*
 	 * If no local APIC can be found then set up a fake all
 	 * zeroes page to simulate the local APIC and another
@@ -1141,8 +1105,6 @@
 	phys_cpu_present_map = 1;
 	apic_write_around(APIC_ID, boot_cpu_physical_apicid);
 
-	apic_pm_init2();
-
 	setup_local_APIC();
 
 	if (nmi_watchdog == NMI_LOCAL_APIC)
arch/i386/kernel/apm.c +9 -0
@@ -1263,6 +1263,11 @@
 		}
 		printk(KERN_CRIT "apm: suspend was vetoed, but suspending anyway.\n");
 	}
+
+	device_suspend(3, SUSPEND_NOTIFY);
+	device_suspend(3, SUSPEND_SAVE_STATE);
+	device_suspend(3, SUSPEND_DISABLE);
+
 	/* serialize with the timer interrupt */
 	write_lock_irq(&xtime_lock);
 
@@ -1283,6 +1288,8 @@
 	if (err != APM_SUCCESS)
 		apm_error("suspend", err);
 	err = (err == APM_SUCCESS) ? 0 : -EIO;
+	device_resume(RESUME_RESTORE_STATE);
+	device_resume(RESUME_ENABLE);
 	pm_send_all(PM_RESUME, (void *)0);
 	queue_event(APM_NORMAL_RESUME, NULL);
  out:
@@ -1396,6 +1403,8 @@
 				write_lock_irq(&xtime_lock);
 				set_time();
 				write_unlock_irq(&xtime_lock);
+				device_resume(RESUME_RESTORE_STATE);
+				device_resume(RESUME_ENABLE);
 				pm_send_all(PM_RESUME, (void *)0);
 				queue_event(event, NULL);
 			}
arch/i386/kernel/i386_ksyms.c +0 -4
@@ -161,10 +161,6 @@
 EXPORT_SYMBOL(flush_tlb_page);
 #endif
 
-#if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_PM)
-EXPORT_SYMBOL_GPL(set_nmi_pm_callback);
-EXPORT_SYMBOL_GPL(unset_nmi_pm_callback);
-#endif
 #ifdef CONFIG_X86_IO_APIC
 EXPORT_SYMBOL(IO_APIC_get_PCI_irq_vector);
 #endif
arch/i386/kernel/nmi.c +41 -33

125 lines not on this page · raw message

arch/i386/oprofile/nmi_int.c +48 -28

112 lines not on this page · raw message

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#3 lore raw
Quoted 2 lines · Pavel Machek

This one should be okay. [oprofile not tested because I don't know how to use it...

It's not hard you know[1].

arch/i386/kernel/nmi.c line 182

Quoted 10 lines
arch/i386/kernel/nmi.c +5 -4
-struct pm_dev * set_nmi_pm_callback(pm_callback callback)
+static int nmi_resume(struct device *dev, u32 level)
 {
-	apic_pm_unregister(nmi_pmdev);
-	return apic_pm_register(PM_SYS_DEV, 0, callback);
-}
+	if (level != RESUME_POWER_ON)
+		return 0;
+	setup_apic_nmi_watchdog();
+	return 0;

I don't pretend to understand the PM layer at all, but it looks like that both nmi.c's and oprofile's resume functions will get called. This won't work: if oprofile has control of the perfctr's/nmi stuff, you can't let the NMI watchdog's resume() be called, as it may conflict with what oprofile is trying to resume.

Quoted 1 line

+ return device_register(&device_nmi);

There's some evil spaces in your patch.

regards
john

[1] opcontrol --setup --vmlinux=/boot/vmlinux --ctr0-count=20000

	--ctr0-event=CPU_CLK_UNHALTED && opcontrol --start
#4 lore raw

Hi!

Quoted 4 lines
Quoted 2 lines

This one should be okay. [oprofile not tested because I don't know how to use it...

It's not hard you know[1].

should apt-get install oprofile then opcontrol --setup --vmlinux=/boot/vmlinux --ctr0-count=20000 --ctr0-event=CPU_CLK_UNHALTED && opcontrol --start

do the trick? If that works does it mean oprofile is okay?

Quoted 8 lines
Quoted 8 lines
arch/i386/kernel/nmi.c +3 -4
-struct pm_dev * set_nmi_pm_callback(pm_callback callback)
+static int nmi_resume(struct device *dev, u32 level)
 {
-	apic_pm_unregister(nmi_pmdev);
-	return apic_pm_register(PM_SYS_DEV, 0, callback);
-}
+	if (level != RESUME_POWER_ON)
+		return 0;
Quoted 8 lines

arch/i386/kernel/nmi.c line 182

Quoted 2 lines
arch/i386/kernel/nmi.c +2 -0
+	setup_apic_nmi_watchdog();
+	return 0;

I don't pretend to understand the PM layer at all, but it looks like that both nmi.c's and oprofile's resume functions will get called. This won't work: if oprofile has control of the perfctr's/nmi stuff, you can't let the NMI watchdog's resume() be called, as it may conflict with what oprofile is trying to resume.

oprofile() should already have checks to prevent that, and I added one

[ if (nmi_watchdog == NMI_LOCAL_APIC)
]

to nmi.c. I hope that's okay.
Pavel

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#5 lore raw
Quoted 1 line · Pavel Machek

should apt-get install

not packaged yet I'm afraid. Get
http://movementarian.org/oprofile-0.5.tar.gz, ./configure \
--with-kernel-support && make && make install

Quoted 1 line

--ctr0-event=CPU_CLK_UNHALTED && opcontrol --start

If it's an Intel CPU (not pentium 4) this should be OK For Athlon, use INST_RETIRED

Quoted 1 line

do the trick? If that works does it mean oprofile is okay?

Run a few kernel compiles and stress it a bit. Run "op_time" to check you get output...

Quoted 7 lines
Quoted 5 lines

I don't pretend to understand the PM layer at all, but it looks like that both nmi.c's and oprofile's resume functions will get called. This won't work: if oprofile has control of the perfctr's/nmi stuff, you can't let the NMI watchdog's resume() be called, as it may conflict with what oprofile is trying to resume.

oprofile() should already have checks to prevent that, and I added one

I haven't seen where you added these checks. They did not exist before:
oprofile took responsibility for the NMI/perfctr handling off the NMI watchdog entirely, then handed it back when oprofile finished.

Basically you have to ensure that only the right pm callback is called, not both.

Quoted 1 line

[ if (nmi_watchdog == NMI_LOCAL_APIC)

I don't see an added line like this ?

regards
john

#6 lore raw

Hi!

Quoted 10 lines
Quoted 1 line

should apt-get install

not packaged yet I'm afraid. Get
http://movementarian.org/oprofile-0.5.tar.gz, ./configure \
--with-kernel-support && make && make install

Quoted 1 line

--ctr0-event=CPU_CLK_UNHALTED && opcontrol --start

If it's an Intel CPU (not pentium 4) this should be OK For Athlon, use INST_RETIRED

Unfortunately my systems seems to miss bfd.h, required by libutil++.

Quoted 11 lines
Quoted 7 lines
Quoted 5 lines

I don't pretend to understand the PM layer at all, but it looks like that both nmi.c's and oprofile's resume functions will get called. This won't work: if oprofile has control of the perfctr's/nmi stuff, you can't let the NMI watchdog's resume() be called, as it may conflict with what oprofile is trying to resume.

oprofile() should already have checks to prevent that, and I added one

I haven't seen where you added these checks. They did not exist before:
oprofile took responsibility for the NMI/perfctr handling off the NMI watchdog entirely, then handed it back when oprofile finished.

Ouch, I did not see that code. It seems not to be in nmi_int.c; where is that?

Quoted 6 lines

Basically you have to ensure that only the right pm callback is called, not both.

Quoted 1 line

[ if (nmi_watchdog == NMI_LOCAL_APIC)

I don't see an added line like this ?

I added it to my local tree only (sorry). (Updated patch is attached).

Pavel

arch/i386/kernel/apic.c +61 -99
@@ -10,6 +10,7 @@
  *					for testing these extensively.
  *	Maciej W. Rozycki	:	Various updates and fixes.
  *	Mikael Pettersson	:	Power Management for UP-APIC.
+ *	Pavel Machek		:	Converted to driver model.
  */
 
 #include <linux/config.h>
@@ -23,6 +24,10 @@
 #include <linux/interrupt.h>
 #include <linux/mc146818rtc.h>
 #include <linux/kernel_stat.h>
+#include <linux/delay.h>
+#include <linux/slab.h>
+#include <linux/device.h>
+#include <linux/pm.h>
 
 #include <asm/atomic.h>
 #include <asm/smp.h>
@@ -54,6 +59,8 @@
 int prof_old_multiplier[NR_CPUS] = { 1, };
 int prof_counter[NR_CPUS] = { 1, };
 
+unsigned long apic_phys;
+
 int get_maxlvt(void)
 {
 	unsigned int v, ver, maxlvt;
@@ -292,6 +299,27 @@
 	apic_write_around(APIC_LVT1, value);
 }
 
+static struct {
+	/* 'active' is true if the local APIC was enabled by us and
+	   not the BIOS; this signifies that we are also responsible
+	   for disabling it before entering apm/acpi suspend */
+	int active;
+	/* r/w apic fields */
+	unsigned int apic_id;
+	unsigned int apic_taskpri;
+	unsigned int apic_ldr;
+	unsigned int apic_dfr;
+	unsigned int apic_spiv;
+	unsigned int apic_lvtt;
+	unsigned int apic_lvtpc;
+	unsigned int apic_lvt0;
+	unsigned int apic_lvt1;
+	unsigned int apic_lvterr;
+	unsigned int apic_tmict;
+	unsigned int apic_tdcr;
+	unsigned int apic_thmr;
+} apic_pm_state;
+
 void __init setup_local_APIC (void)
 {
 	unsigned long value, ver, maxlvt;
@@ -435,44 +463,19 @@
 
 	if (nmi_watchdog == NMI_LOCAL_APIC)
 		setup_apic_nmi_watchdog();
+
+	apic_pm_state.active = 1;
 }
 
 #ifdef CONFIG_PM
-
-#include <linux/slab.h>
-#include <linux/pm.h>
-
-static struct {
-	/* 'active' is true if the local APIC was enabled by us and
-	   not the BIOS; this signifies that we are also responsible
-	   for disabling it before entering apm/acpi suspend */
-	int active;
-	/* 'perfctr_pmdev' is here because the current (2.4.1) PM
-	   callback system doesn't handle hierarchical dependencies */
-	struct pm_dev *perfctr_pmdev;
-	/* r/w apic fields */
-	unsigned int apic_id;
-	unsigned int apic_taskpri;
-	unsigned int apic_ldr;
-	unsigned int apic_dfr;
-	unsigned int apic_spiv;
-	unsigned int apic_lvtt;
-	unsigned int apic_lvtpc;
-	unsigned int apic_lvt0;
-	unsigned int apic_lvt1;
-	unsigned int apic_lvterr;
-	unsigned int apic_tmict;
-	unsigned int apic_tdcr;
-	unsigned int apic_thmr;
-} apic_pm_state;
-
-static void apic_pm_suspend(void *data)
+static int apic_suspend(struct device *dev, u32 state, u32 level)
 {
 	unsigned int l, h;
 	unsigned long flags;
 
-	if (apic_pm_state.perfctr_pmdev)
-		pm_send(apic_pm_state.perfctr_pmdev, PM_SUSPEND, data);
+	if (level != SUSPEND_DISABLE)
+		return 0;
+
 	apic_pm_state.apic_id = apic_read(APIC_ID);
 	apic_pm_state.apic_taskpri = apic_read(APIC_TASKPRI);
 	apic_pm_state.apic_ldr = apic_read(APIC_LDR);
@@ -493,13 +496,19 @@
 	l &= ~MSR_IA32_APICBASE_ENABLE;
 	wrmsr(MSR_IA32_APICBASE, l, h);
 	local_irq_restore(flags);
+	return 0;
 }
 
-static void apic_pm_resume(void *data)
+static int apic_resume(struct device *dev, u32 level)
 {
 	unsigned int l, h;
 	unsigned long flags;
 
+	if (level != RESUME_POWER_ON)
+		return 0;
+
+	set_fixmap_nocache(FIX_APIC_BASE, apic_phys);		/* FIXME: this is needed for S3 resume, but why? */
+
 	local_irq_save(flags);
 	rdmsr(MSR_IA32_APICBASE, l, h);
 	l &= ~MSR_IA32_APICBASE_BASE;
@@ -524,74 +533,34 @@
 	apic_write(APIC_ESR, 0);
 	apic_read(APIC_ESR);
 	local_irq_restore(flags);
-	if (apic_pm_state.perfctr_pmdev)
-		pm_send(apic_pm_state.perfctr_pmdev, PM_RESUME, data);
-}
-
-static int apic_pm_callback(struct pm_dev *dev, pm_request_t rqst, void *data)
-{
-	switch (rqst) {
-	case PM_SUSPEND:
-		apic_pm_suspend(data);
-		break;
-	case PM_RESUME:
-		apic_pm_resume(data);
-		break;
-	}
 	return 0;
 }
 
-/* perfctr driver should call this instead of pm_register() */
-struct pm_dev *apic_pm_register(pm_dev_t type,
-				unsigned long id,
-				pm_callback callback)
-{
-	struct pm_dev *dev;
-
-	if (!apic_pm_state.active)
-		return pm_register(type, id, callback);
-	if (apic_pm_state.perfctr_pmdev)
-		return NULL;	/* we're busy */
-	dev = kmalloc(sizeof(struct pm_dev), GFP_KERNEL);
-	if (dev) {
-		memset(dev, 0, sizeof(*dev));
-		dev->type = type;
-		dev->id = id;
-		dev->callback = callback;
-		apic_pm_state.perfctr_pmdev = dev;
-	}
-	return dev;
-}
-
-/* perfctr driver should call this instead of pm_unregister() */
-void apic_pm_unregister(struct pm_dev *dev)
-{
-	if (!apic_pm_state.active) {
-		pm_unregister(dev);
-	} else if (dev == apic_pm_state.perfctr_pmdev) {
-		apic_pm_state.perfctr_pmdev = NULL;
-		kfree(dev);
-	}
-}
+static struct device_driver apic_driver = {
+	.name		= "apic",
+	.bus		= &system_bus_type,
+	.resume		= apic_resume,
+	.suspend	= apic_suspend,
+};
+
+struct sys_device device_apic = {
+	.name		= "apic",
+	.id		= 0,
+	.dev		= {
+		.name	= "APIC",
+		.driver	= &apic_driver,
+	},
+};
 
-static void __init apic_pm_init1(void)
-{
-	/* can't pm_register() at this early stage in the boot process
-	   (causes an immediate reboot), so just set the flag */
-	apic_pm_state.active = 1;
-}
-
-static void __init apic_pm_init2(void)
+static int __init init_apic_devicefs(void)
 {
+	driver_register(&apic_driver);
 	if (apic_pm_state.active)
-		pm_register(PM_SYS_DEV, 0, apic_pm_callback);
+		return sys_device_register(&device_apic);
+	return 0;
 }
 
-#else	/* CONFIG_PM */
-
-static inline void apic_pm_init1(void) { }
-static inline void apic_pm_init2(void) { }
-
+device_initcall(init_apic_devicefs);
 #endif	/* CONFIG_PM */
 
 /*
@@ -658,9 +627,6 @@
 		nmi_watchdog = NMI_LOCAL_APIC;
 
 	printk("Found and enabled local APIC!\n");
-
-	apic_pm_init1();
-
 	return 0;
 
 no_apic:
@@ -670,8 +636,6 @@
 
 void __init init_apic_mappings(void)
 {
-	unsigned long apic_phys;
-
 	/*
 	 * If no local APIC can be found then set up a fake all
 	 * zeroes page to simulate the local APIC and another
@@ -1141,8 +1105,6 @@
 	phys_cpu_present_map = 1;
 	apic_write_around(APIC_ID, boot_cpu_physical_apicid);
 
-	apic_pm_init2();
-
 	setup_local_APIC();
 
 	if (nmi_watchdog == NMI_LOCAL_APIC)
arch/i386/kernel/apm.c +9 -0
@@ -1263,6 +1263,11 @@
 		}
 		printk(KERN_CRIT "apm: suspend was vetoed, but suspending anyway.\n");
 	}
+
+	device_suspend(3, SUSPEND_NOTIFY);
+	device_suspend(3, SUSPEND_SAVE_STATE);
+	device_suspend(3, SUSPEND_DISABLE);
+
 	/* serialize with the timer interrupt */
 	write_lock_irq(&xtime_lock);
 
@@ -1283,6 +1288,8 @@
 	if (err != APM_SUCCESS)
 		apm_error("suspend", err);
 	err = (err == APM_SUCCESS) ? 0 : -EIO;
+	device_resume(RESUME_RESTORE_STATE);
+	device_resume(RESUME_ENABLE);
 	pm_send_all(PM_RESUME, (void *)0);
 	queue_event(APM_NORMAL_RESUME, NULL);
  out:
@@ -1396,6 +1403,8 @@
 				write_lock_irq(&xtime_lock);
 				set_time();
 				write_unlock_irq(&xtime_lock);
+				device_resume(RESUME_RESTORE_STATE);
+				device_resume(RESUME_ENABLE);
 				pm_send_all(PM_RESUME, (void *)0);
 				queue_event(event, NULL);
 			}
arch/i386/kernel/i386_ksyms.c +0 -4
@@ -161,10 +161,6 @@
 EXPORT_SYMBOL(flush_tlb_page);
 #endif
 
-#if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_PM)
-EXPORT_SYMBOL_GPL(set_nmi_pm_callback);
-EXPORT_SYMBOL_GPL(unset_nmi_pm_callback);
-#endif
 #ifdef CONFIG_X86_IO_APIC
 EXPORT_SYMBOL(IO_APIC_get_PCI_irq_vector);
 #endif
arch/i386/kernel/nmi.c +43 -33

127 lines not on this page · raw message

arch/i386/oprofile/nmi_int.c +48 -28

112 lines not on this page · raw message

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#7 lore raw
Quoted 1 line · Pavel Machek

Unfortunately my systems seems to miss bfd.h, required by libutil++.

The debian package is apparently called binutils-dev

Quoted 6 lines
Quoted 3 lines

I haven't seen where you added these checks. They did not exist before:
oprofile took responsibility for the NMI/perfctr handling off the NMI watchdog entirely, then handed it back when oprofile finished.

Ouch, I did not see that code. It seems not to be in nmi_int.c; where is that?

oprofile saves the perfctr config when it starts in nmi_int.c, then calls the set_nmi_pm_callback. Then the suspend/resume calls oprofile's, and we assume we have unique access to controlling the generated NMIs (ie it becomes oprofile's job to ensure no NMIs are generated by the perfctrs whilst suspended).

It's pretty ugly, so a correct cleanup would be appreciated.

I suppose this is quite an unusual case for the PM model ?

regards
john

#8 lore raw

Hi!

Quoted 3 lines
Quoted 1 line

Unfortunately my systems seems to miss bfd.h, required by libutil++.

The debian package is apparently called binutils-dev

Thanx. oprofile now seems to work for me.

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

I haven't seen where you added these checks. They did not exist before:
oprofile took responsibility for the NMI/perfctr handling off the NMI watchdog entirely, then handed it back when oprofile finished.

Quoted 8 lines
Quoted 2 lines

Ouch, I did not see that code. It seems not to be in nmi_int.c; where is that?

oprofile saves the perfctr config when it starts in nmi_int.c, then calls the set_nmi_pm_callback. Then the suspend/resume calls oprofile's, and we assume we have unique access to controlling the generated NMIs (ie it becomes oprofile's job to ensure no NMIs are generated by the perfctrs whilst suspended).

I can see set_nmi_pm_callback() in nmi_setup(), but can not see where nmi.c's pm_callback is restored in nmi_shutdown(). Perhaps that's a bug?

Quoted 3 lines

It's pretty ugly, so a correct cleanup would be appreciated.

I suppose this is quite an unusual case for the PM model ?

Yes, it is pretty unusual.
Pavel

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#9 lore raw
Quoted 2 lines · Pavel Machek

I can see set_nmi_pm_callback() in nmi_setup(), but can not see where nmi.c's pm_callback is restored in nmi_shutdown().

It's the very first line :

    149         unset_nmi_pm_callback(oprofile_pmdev);

The unset will also restore the nmi_pmdev back.

regards
john

#10 lore raw
In reply to Pavel Machek

arch/i386/kernel/apic.c line 510

Quoted 1 line · Pavel Machek
arch/i386/kernel/apic.c +1 -0
+	set_fixmap_nocache(FIX_APIC_BASE, apic_phys);		/* FIXME: this is needed for S3 resume, but why? */

Intel recommends a strong uncacheable mapping otherwise it may exhibit undeterministic behaviour. Perhaps that could be it?

Zwane

#11 lore raw

Hi!

Quoted 4 lines

arch/i386/kernel/apic.c line 510

Quoted 1 line
arch/i386/kernel/apic.c +1 -0
+	set_fixmap_nocache(FIX_APIC_BASE, apic_phys);		/* FIXME: this is needed for S3 resume, but why? */

Intel recommends a strong uncacheable mapping otherwise it may exhibit undeterministic behaviour. Perhaps that could be it?

But... set_fixmp_nocache seems to only flip bits in page tables... that should be nop after S3 because memory is preserved.

Pavel

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?