Skip to contentSkip to conversation

Switch APIC (+nmi, +oprofile) to driver model

DISCUSSION Mikael Pettersson 5 participants 17 messages Feb 9, 2003 – Feb 16, 2003

lore
Share & tools
lore
#0 lore raw
Quoted 2 lines · Pavel Machek

Here's next attempt at moving APIC (+nmi, +oprofile) to the driver model. If it looks good I'l submit it to Linus.

I'm sorry to be a killjoy, but this still doesn't look right.

arch/i386/kernel/apic.c

Quoted 12 lines
arch/i386/kernel/apic.c +1 -1
-static void apic_pm_resume(void *data)
+static int apic_resume(struct device *dev, u32 level)
{
	unsigned int l, h;
	unsigned long flags;
arch/i386/kernel/apic.c +4 -0
+	if (level != RESUME_POWER_ON)
+		return 0;
+
+	set_fixmap_nocache(FIX_APIC_BASE, apic_phys);		/* FIXME: this is needed for S3 resume, but why? */

This is horrible! The only reason this might be needed is if the page tables weren't restored properly at resume, and that indicates a bug somewhere else.

Also, apic_phys is (or should be) APIC_DEFAULT_PHYS_BASE, so you shouldn't need to make apic_phys global.

Quoted 6 lines
arch/i386/kernel/apic.c +6 -0
+static struct device_driver apic_driver = {
+	.name		= "apic",
+	.bus		= &system_bus_type,
+	.resume		= apic_resume,
+	.suspend	= apic_suspend,
+};

...

Quoted 7 lines

+static int __init init_apic_devicefs(void)
{

arch/i386/kernel/apic.c +1 -0
+	driver_register(&apic_driver);

if (apic_pm_state.active)

arch/i386/kernel/apic.c +2 -1
-		pm_register(PM_SYS_DEV, 0, apic_pm_callback);
+		return sys_device_register(&device_apic);
+	return 0;

...

Quoted 1 line

+device_initcall(init_apic_devicefs);

init_apic_devicefs() registers apic_driver even if active is false. This probably breaks any machine where cpu_has_apic is false, since apic_suspend/resume will access non-existent APIC registers & MSRs.

I don't like the idea of registering apic_driver when !cpu_has_apic, but it might be needed for the local-APIC NMI watchdog. A fix could be to guard apic_suspend/resume with "if(!cpu_has_apic)return;".

arch/i386/kernel/nmi.c line 126

Quoted 13 lines
arch/i386/kernel/nmi.c +0 -4
@@ -118,10 +121,6 @@
 	 * missing bits. Right now Intel P6/P4 and AMD K7 only.
 	 */
 	if ((nmi == NMI_LOCAL_APIC) &&
-			(boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) &&
-			(boot_cpu_data.x86 == 6 || boot_cpu_data.x86 == 15))
-		nmi_watchdog = nmi;
-	if ((nmi == NMI_LOCAL_APIC) &&
 			(boot_cpu_data.x86_vendor == X86_VENDOR_AMD) &&
 	  		(boot_cpu_data.x86 == 6 || boot_cpu_data.x86 == 15))
 		nmi_watchdog = nmi;

You just killed NMI_LOCAL_APIC support on Intel.

arch/i386/kernel/nmi.c

Quoted 10 lines
arch/i386/kernel/nmi.c +8 -1
+static int __init init_nmi_devicefs(void)
+{
+	driver_register(&nmi_driver);
+
+	device_nmi.parent = &device_apic.dev;
+        return device_register(&device_nmi);
+}
 
-#define __pminit	__init
+device_initcall(init_nmi_devicefs);

Doesn't this require that init_apic_devicefs() has been done? Can you guarantee that?

Quoted 3 lines

+#define __pminit
#endif /* CONFIG_PM */

__pminit is no longer defined when !CONFIG_PM, which will cause compile errors. Possibly the remaining uses of __pminit should be removed.

Quoted 2 lines
arch/i386/oprofile/nmi_int.c +0 -0

...

Quoted 4 lines
arch/i386/oprofile/nmi_int.c +4 -0
+	if (nmi_watchdog == NMI_LOCAL_APIC) {
+		disable_apic_nmi_watchdog();
+		nmi_watchdog = NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE;
+	}

...

Quoted 4 lines
arch/i386/oprofile/nmi_int.c +4 -0
+	if (nmi_watchdog == NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE) {
+		nmi_watchdog = NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE;
+		setup_apic_nmi_watchdog();
+	}

...

include/asm-i386/apic.h line 98

Quoted 7 lines
include/asm-i386/apic.h +1 -0
@@ -95,6 +95,7 @@
 #define NMI_IO_APIC	1
 #define NMI_LOCAL_APIC	2
 #define NMI_INVALID	3
+#define NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE	4

This is ugly like h*ll. Surely oprofile could just do:

static unsigned int prev_nmi_watchdog;
..
prev_nmi_watchdog = nmi_watchdog;

if (prev_nmi_watchdog == NMI_LOCAL_APIC) {
	disable_apic_nmi_watchdog();
	nmi_watchdog = NMI_NONE;
}
...
if (prev_nmi_watchdog == NMI_LOCAL_APIC) {
	nmi_watchdog = NMI_LOCAL_APIC;
	setup_apic_nmi_watchdog();
}

without having to add crap to the global namespace?

/Mikael

#1 lore raw
Quoted 2 lines · Mikael Pettersson

I don't like the idea of registering apic_driver when !cpu_has_apic, but it might be needed for the local-APIC NMI watchdog.

Really ?

Quoted 7 lines
Quoted 1 line

+#define NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE 4

This is ugly like h*ll. Surely oprofile could just do:

static unsigned int prev_nmi_watchdog;
..
prev_nmi_watchdog = nmi_watchdog;

as long as it's exported to modules. I'd probably prefer to just have :

	disable_nmi_watchdog();
	...
	enable_nmi_watchdog();

and have those do the right thing depending on a (nmi.c local) nmi_watchdog.

regards,
john

#2 lore raw
In reply to Mikael Pettersson

Hi!

Quoted 22 lines
Quoted 2 lines

Here's next attempt at moving APIC (+nmi, +oprofile) to the driver model. If it looks good I'l submit it to Linus.

I'm sorry to be a killjoy, but this still doesn't look right.

arch/i386/kernel/apic.c

Quoted 12 lines
arch/i386/kernel/apic.c +1 -1
-static void apic_pm_resume(void *data)
+static int apic_resume(struct device *dev, u32 level)
{
	unsigned int l, h;
	unsigned long flags;
arch/i386/kernel/apic.c +4 -0
+	if (level != RESUME_POWER_ON)
+		return 0;
+
+	set_fixmap_nocache(FIX_APIC_BASE, apic_phys);		/* FIXME: this is needed for S3 resume, but why? */

This is horrible! The only reason this might be needed is if the page tables weren't restored properly at resume, and that indicates a bug somewhere else.

Quoted 2 lines

Also, apic_phys is (or should be) APIC_DEFAULT_PHYS_BASE, so you shouldn't need to make apic_phys global.

Really?

        /*
         * If no local APIC can be found then set up a fake all
         * zeroes page to simulate the local APIC and another
         * one for the IO-APIC.
         */
        if (!smp_found_config && detect_init_APIC()) {
                apic_phys = (unsigned long) alloc_bootmem_pages(PAGE_SIZE);
                apic_phys = __pa(apic_phys);
        } else
                apic_phys = mp_lapic_addr;

        set_fixmap_nocache(FIX_APIC_BASE, apic_phys);

So it seems to me it really is variable.

Quoted 7 lines
Quoted 7 lines
arch/i386/kernel/nmi.c +0 -1
@@ -118,10 +121,6 @@
 	 * missing bits. Right now Intel P6/P4 and AMD K7 only.
 	 */
 	if ((nmi == NMI_LOCAL_APIC) &&
-			(boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) &&
Quoted 8 lines

arch/i386/kernel/nmi.c line 126

Quoted 6 lines
arch/i386/kernel/nmi.c +0 -3
-			(boot_cpu_data.x86 == 6 || boot_cpu_data.x86 == 15))
-		nmi_watchdog = nmi;
-	if ((nmi == NMI_LOCAL_APIC) &&
 			(boot_cpu_data.x86_vendor == X86_VENDOR_AMD) &&
 	  		(boot_cpu_data.x86 == 6 || boot_cpu_data.x86 == 15))
 		nmi_watchdog = nmi;

You just killed NMI_LOCAL_APIC support on Intel.

Oops, sorry, I seen two identical blocks of code... and they were not identical. Sorry.

Quoted 13 lines
Quoted 2 lines
arch/i386/oprofile/nmi_int.c +0 -0

...

Quoted 4 lines
arch/i386/oprofile/nmi_int.c +4 -0
+	if (nmi_watchdog == NMI_LOCAL_APIC) {
+		disable_apic_nmi_watchdog();
+		nmi_watchdog = NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE;
+	}

...

Quoted 4 lines
arch/i386/oprofile/nmi_int.c +4 -0
+	if (nmi_watchdog == NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE) {
+		nmi_watchdog = NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE;
+		setup_apic_nmi_watchdog();
+	}

...

Quoted 9 lines

include/asm-i386/apic.h line 98

Quoted 7 lines
include/asm-i386/apic.h +1 -0
@@ -95,6 +95,7 @@
 #define NMI_IO_APIC	1
 #define NMI_LOCAL_APIC	2
 #define NMI_INVALID	3
+#define NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE	4

This is ugly like h*ll. Surely oprofile could just do:

Yes, whole oprofile/nmi interaction is ugly like hell. This way it is at least explicit, so people *know* its ugly.
Pavel

Signature · 2 lines
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.
#3 lore raw
Quoted 2 lines · Pavel Machek

Yes, whole oprofile/nmi interaction is ugly like hell. This way it is at least explicit, so people *know* its ugly.

That's no reason not do something like Mikael or I suggested.

john

#4 lore raw

Hi!

Quoted 4 lines
Quoted 2 lines

Yes, whole oprofile/nmi interaction is ugly like hell. This way it is at least explicit, so people *know* its ugly.

That's no reason not do something like Mikael or I suggested.

Well, I believe that having

extern unsigned int nmi_watchdog;

#define NMI_NONE        0
#define NMI_IO_APIC     1
#define NMI_LOCAL_APIC  2
#define NMI_INVALID     3

#define NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE 4

makes what happens there pretty explicit. Whole thing can be controlled by single variable...

Here's new version of diff...
Pavel

arch/i386/kernel/apic.c +65 -99
@@ -10,6 +10,7 @@
  *					for testing these extensively.
  *	Maciej W. Rozycki	:	Various updates and fixes.
  *	Mikael Pettersson	:	Power Management for UP-APIC.
+ *	Pavel Machek		:	Converted to driver model.
  */
 
 #include <linux/config.h>
@@ -23,6 +24,10 @@
 #include <linux/interrupt.h>
 #include <linux/mc146818rtc.h>
 #include <linux/kernel_stat.h>
+#include <linux/delay.h>
+#include <linux/slab.h>
+#include <linux/device.h>
+#include <linux/pm.h>
 
 #include <asm/atomic.h>
 #include <asm/smp.h>
@@ -54,6 +59,8 @@
 int prof_old_multiplier[NR_CPUS] = { 1, };
 int prof_counter[NR_CPUS] = { 1, };
 
+unsigned long apic_phys;
+
 int get_maxlvt(void)
 {
 	unsigned int v, ver, maxlvt;
@@ -292,6 +299,27 @@
 	apic_write_around(APIC_LVT1, value);
 }
 
+static struct {
+	/* 'active' is true if the local APIC was enabled by us and
+	   not the BIOS; this signifies that we are also responsible
+	   for disabling it before entering apm/acpi suspend */
+	int active;
+	/* r/w apic fields */
+	unsigned int apic_id;
+	unsigned int apic_taskpri;
+	unsigned int apic_ldr;
+	unsigned int apic_dfr;
+	unsigned int apic_spiv;
+	unsigned int apic_lvtt;
+	unsigned int apic_lvtpc;
+	unsigned int apic_lvt0;
+	unsigned int apic_lvt1;
+	unsigned int apic_lvterr;
+	unsigned int apic_tmict;
+	unsigned int apic_tdcr;
+	unsigned int apic_thmr;
+} apic_pm_state;
+
 void __init setup_local_APIC (void)
 {
 	unsigned long value, ver, maxlvt;
@@ -435,44 +463,21 @@
 
 	if (nmi_watchdog == NMI_LOCAL_APIC)
 		setup_apic_nmi_watchdog();
+
+	apic_pm_state.active = 1;
 }
 
 #ifdef CONFIG_PM
-
-#include <linux/slab.h>
-#include <linux/pm.h>
-
-static struct {
-	/* 'active' is true if the local APIC was enabled by us and
-	   not the BIOS; this signifies that we are also responsible
-	   for disabling it before entering apm/acpi suspend */
-	int active;
-	/* 'perfctr_pmdev' is here because the current (2.4.1) PM
-	   callback system doesn't handle hierarchical dependencies */
-	struct pm_dev *perfctr_pmdev;
-	/* r/w apic fields */
-	unsigned int apic_id;
-	unsigned int apic_taskpri;
-	unsigned int apic_ldr;
-	unsigned int apic_dfr;
-	unsigned int apic_spiv;
-	unsigned int apic_lvtt;
-	unsigned int apic_lvtpc;
-	unsigned int apic_lvt0;
-	unsigned int apic_lvt1;
-	unsigned int apic_lvterr;
-	unsigned int apic_tmict;
-	unsigned int apic_tdcr;
-	unsigned int apic_thmr;
-} apic_pm_state;
-
-static void apic_pm_suspend(void *data)
+static int apic_suspend(struct device *dev, u32 state, u32 level)
 {
 	unsigned int l, h;
 	unsigned long flags;
 
-	if (apic_pm_state.perfctr_pmdev)
-		pm_send(apic_pm_state.perfctr_pmdev, PM_SUSPEND, data);
+	if (level != SUSPEND_DISABLE)
+		return 0;
+	if (!apic_pm_state.active)
+		return 0;
+
 	apic_pm_state.apic_id = apic_read(APIC_ID);
 	apic_pm_state.apic_taskpri = apic_read(APIC_TASKPRI);
 	apic_pm_state.apic_ldr = apic_read(APIC_LDR);
@@ -493,13 +498,21 @@
 	l &= ~MSR_IA32_APICBASE_ENABLE;
 	wrmsr(MSR_IA32_APICBASE, l, h);
 	local_irq_restore(flags);
+	return 0;
 }
 
-static void apic_pm_resume(void *data)
+static int apic_resume(struct device *dev, u32 level)
 {
 	unsigned int l, h;
 	unsigned long flags;
 
+	if (level != RESUME_POWER_ON)
+		return 0;
+	if (!apic_pm_state.active)
+		return 0;
+
+	set_fixmap_nocache(FIX_APIC_BASE, apic_phys);		/* FIXME: this is needed for S3 resume, but why? */
+
 	local_irq_save(flags);
 	rdmsr(MSR_IA32_APICBASE, l, h);
 	l &= ~MSR_IA32_APICBASE_BASE;
@@ -524,74 +537,34 @@
 	apic_write(APIC_ESR, 0);
 	apic_read(APIC_ESR);
 	local_irq_restore(flags);
-	if (apic_pm_state.perfctr_pmdev)
-		pm_send(apic_pm_state.perfctr_pmdev, PM_RESUME, data);
-}
-
-static int apic_pm_callback(struct pm_dev *dev, pm_request_t rqst, void *data)
-{
-	switch (rqst) {
-	case PM_SUSPEND:
-		apic_pm_suspend(data);
-		break;
-	case PM_RESUME:
-		apic_pm_resume(data);
-		break;
-	}
 	return 0;
 }
 
-/* perfctr driver should call this instead of pm_register() */
-struct pm_dev *apic_pm_register(pm_dev_t type,
-				unsigned long id,
-				pm_callback callback)
-{
-	struct pm_dev *dev;
-
-	if (!apic_pm_state.active)
-		return pm_register(type, id, callback);
-	if (apic_pm_state.perfctr_pmdev)
-		return NULL;	/* we're busy */
-	dev = kmalloc(sizeof(struct pm_dev), GFP_KERNEL);
-	if (dev) {
-		memset(dev, 0, sizeof(*dev));
-		dev->type = type;
-		dev->id = id;
-		dev->callback = callback;
-		apic_pm_state.perfctr_pmdev = dev;
-	}
-	return dev;
-}
+static struct device_driver apic_driver = {
+	.name		= "apic",
+	.bus		= &system_bus_type,
+	.resume		= apic_resume,
+	.suspend	= apic_suspend,
+};
+
+struct sys_device device_apic = {
+	.name		= "apic",
+	.id		= 0,
+	.dev		= {
+		.name	= "APIC",
+		.driver	= &apic_driver,
+	},
+};
 
-/* perfctr driver should call this instead of pm_unregister() */
-void apic_pm_unregister(struct pm_dev *dev)
-{
-	if (!apic_pm_state.active) {
-		pm_unregister(dev);
-	} else if (dev == apic_pm_state.perfctr_pmdev) {
-		apic_pm_state.perfctr_pmdev = NULL;
-		kfree(dev);
-	}
-}
-
-static void __init apic_pm_init1(void)
-{
-	/* can't pm_register() at this early stage in the boot process
-	   (causes an immediate reboot), so just set the flag */
-	apic_pm_state.active = 1;
-}
-
-static void __init apic_pm_init2(void)
+static int __init init_apic_devicefs(void)
 {
+	driver_register(&apic_driver);
 	if (apic_pm_state.active)
-		pm_register(PM_SYS_DEV, 0, apic_pm_callback);
+		return sys_device_register(&device_apic);
+	return 0;
 }
 
-#else	/* CONFIG_PM */
-
-static inline void apic_pm_init1(void) { }
-static inline void apic_pm_init2(void) { }
-
+device_initcall(init_apic_devicefs);
 #endif	/* CONFIG_PM */
 
 /*
@@ -658,9 +631,6 @@
 		nmi_watchdog = NMI_LOCAL_APIC;
 
 	printk("Found and enabled local APIC!\n");
-
-	apic_pm_init1();
-
 	return 0;
 
 no_apic:
@@ -670,8 +640,6 @@
 
 void __init init_apic_mappings(void)
 {
-	unsigned long apic_phys;
-
 	/*
 	 * If no local APIC can be found then set up a fake all
 	 * zeroes page to simulate the local APIC and another
@@ -1141,8 +1109,6 @@
 	phys_cpu_present_map = 1;
 	apic_write_around(APIC_ID, boot_cpu_physical_apicid);
 
-	apic_pm_init2();
-
 	setup_local_APIC();
 
 	if (nmi_watchdog == NMI_LOCAL_APIC)
arch/i386/kernel/apm.c +9 -0
@@ -1263,6 +1263,11 @@
 		}
 		printk(KERN_CRIT "apm: suspend was vetoed, but suspending anyway.\n");
 	}
+
+	device_suspend(3, SUSPEND_NOTIFY);
+	device_suspend(3, SUSPEND_SAVE_STATE);
+	device_suspend(3, SUSPEND_DISABLE);
+
 	/* serialize with the timer interrupt */
 	write_lock_irq(&xtime_lock);
 
@@ -1283,6 +1288,8 @@
 	if (err != APM_SUCCESS)
 		apm_error("suspend", err);
 	err = (err == APM_SUCCESS) ? 0 : -EIO;
+	device_resume(RESUME_RESTORE_STATE);
+	device_resume(RESUME_ENABLE);
 	pm_send_all(PM_RESUME, (void *)0);
 	queue_event(APM_NORMAL_RESUME, NULL);
  out:
@@ -1396,6 +1403,8 @@
 				write_lock_irq(&xtime_lock);
 				set_time();
 				write_unlock_irq(&xtime_lock);
+				device_resume(RESUME_RESTORE_STATE);
+				device_resume(RESUME_ENABLE);
 				pm_send_all(PM_RESUME, (void *)0);
 				queue_event(event, NULL);
 			}
arch/i386/kernel/i386_ksyms.c +0 -4
@@ -161,10 +161,6 @@
 EXPORT_SYMBOL(flush_tlb_page);
 #endif
 
-#if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_PM)
-EXPORT_SYMBOL_GPL(set_nmi_pm_callback);
-EXPORT_SYMBOL_GPL(unset_nmi_pm_callback);
-#endif
 #ifdef CONFIG_X86_IO_APIC
 EXPORT_SYMBOL(IO_APIC_get_PCI_irq_vector);
 #endif
arch/i386/kernel/nmi.c +50 -40

177 lines not on this page · raw message

arch/i386/oprofile/nmi_int.c +56 -25

119 lines not on this page · raw message

include/asm-i386/apic.h +1 -0
@@ -95,6 +95,7 @@
 #define NMI_IO_APIC	1
 #define NMI_LOCAL_APIC	2
 #define NMI_INVALID	3
+#define NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE	4
 
 #endif /* CONFIG_X86_LOCAL_APIC */
 
Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#5 lore raw
Quoted 9 lines · Pavel Machek
Quoted 4 lines
Quoted 2 lines

Yes, whole oprofile/nmi interaction is ugly like hell. This way it is at least explicit, so people *know* its ugly.

That's no reason not do something like Mikael or I suggested.

makes what happens there pretty explicit. Whole thing can be controlled by single variable...

Here's new version of diff...

Are you going to continue ignoring me when I point out the bugs in your patch ? You're still not exporting nmi_watchdog,setup/disable_watchdog to modules.

Of course, exporting three things when you only need two is ugly, but since you refuse to do it like that ...

john

#6 lore raw

Hi!

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines

Yes, whole oprofile/nmi interaction is ugly like hell. This way it is at least explicit, so people *know* its ugly.

That's no reason not do something like Mikael or I suggested.

Quoted 8 lines
Quoted 4 lines

makes what happens there pretty explicit. Whole thing can be controlled by single variable...

Here's new version of diff...

Are you going to continue ignoring me when I point out the bugs in your patch ? You're still not exporting nmi_watchdog,setup/disable_watchdog to modules.

I try not to ignore comments, but when there's too much of them... :-( [I hope it is okay to EXPORT_SYMBOL() from normal .c file?] Updated patch:
Pavel

arch/i386/kernel/apic.c +63 -98
@@ -10,6 +10,7 @@
  *					for testing these extensively.
  *	Maciej W. Rozycki	:	Various updates and fixes.
  *	Mikael Pettersson	:	Power Management for UP-APIC.
+ *	Pavel Machek		:	Converted to driver model.
  */
 
 #include <linux/config.h>
@@ -23,6 +24,10 @@
 #include <linux/interrupt.h>
 #include <linux/mc146818rtc.h>
 #include <linux/kernel_stat.h>
+#include <linux/delay.h>
+#include <linux/slab.h>
+#include <linux/device.h>
+#include <linux/pm.h>
 
 #include <asm/atomic.h>
 #include <asm/smp.h>
@@ -292,6 +297,27 @@
 	apic_write_around(APIC_LVT1, value);
 }
 
+static struct {
+	/* 'active' is true if the local APIC was enabled by us and
+	   not the BIOS; this signifies that we are also responsible
+	   for disabling it before entering apm/acpi suspend */
+	int active;
+	/* r/w apic fields */
+	unsigned int apic_id;
+	unsigned int apic_taskpri;
+	unsigned int apic_ldr;
+	unsigned int apic_dfr;
+	unsigned int apic_spiv;
+	unsigned int apic_lvtt;
+	unsigned int apic_lvtpc;
+	unsigned int apic_lvt0;
+	unsigned int apic_lvt1;
+	unsigned int apic_lvterr;
+	unsigned int apic_tmict;
+	unsigned int apic_tdcr;
+	unsigned int apic_thmr;
+} apic_pm_state;
+
 void __init setup_local_APIC (void)
 {
 	unsigned long value, ver, maxlvt;
@@ -435,44 +461,21 @@
 
 	if (nmi_watchdog == NMI_LOCAL_APIC)
 		setup_apic_nmi_watchdog();
+
+	apic_pm_state.active = 1;
 }
 
 #ifdef CONFIG_PM
-
-#include <linux/slab.h>
-#include <linux/pm.h>
-
-static struct {
-	/* 'active' is true if the local APIC was enabled by us and
-	   not the BIOS; this signifies that we are also responsible
-	   for disabling it before entering apm/acpi suspend */
-	int active;
-	/* 'perfctr_pmdev' is here because the current (2.4.1) PM
-	   callback system doesn't handle hierarchical dependencies */
-	struct pm_dev *perfctr_pmdev;
-	/* r/w apic fields */
-	unsigned int apic_id;
-	unsigned int apic_taskpri;
-	unsigned int apic_ldr;
-	unsigned int apic_dfr;
-	unsigned int apic_spiv;
-	unsigned int apic_lvtt;
-	unsigned int apic_lvtpc;
-	unsigned int apic_lvt0;
-	unsigned int apic_lvt1;
-	unsigned int apic_lvterr;
-	unsigned int apic_tmict;
-	unsigned int apic_tdcr;
-	unsigned int apic_thmr;
-} apic_pm_state;
-
-static void apic_pm_suspend(void *data)
+static int apic_suspend(struct device *dev, u32 state, u32 level)
 {
 	unsigned int l, h;
 	unsigned long flags;
 
-	if (apic_pm_state.perfctr_pmdev)
-		pm_send(apic_pm_state.perfctr_pmdev, PM_SUSPEND, data);
+	if (level != SUSPEND_DISABLE)
+		return 0;
+	if (!apic_pm_state.active)
+		return 0;
+
 	apic_pm_state.apic_id = apic_read(APIC_ID);
 	apic_pm_state.apic_taskpri = apic_read(APIC_TASKPRI);
 	apic_pm_state.apic_ldr = apic_read(APIC_LDR);
@@ -493,13 +496,21 @@
 	l &= ~MSR_IA32_APICBASE_ENABLE;
 	wrmsr(MSR_IA32_APICBASE, l, h);
 	local_irq_restore(flags);
+	return 0;
 }
 
-static void apic_pm_resume(void *data)
+static int apic_resume(struct device *dev, u32 level)
 {
 	unsigned int l, h;
 	unsigned long flags;
 
+	if (level != RESUME_POWER_ON)
+		return 0;
+	if (!apic_pm_state.active)
+		return 0;
+
+	set_fixmap_nocache(FIX_APIC_BASE, APIC_DEFAULT_PHYS_BASE);	/* FIXME: this is needed for S3 resume, but why? */
+
 	local_irq_save(flags);
 	rdmsr(MSR_IA32_APICBASE, l, h);
 	l &= ~MSR_IA32_APICBASE_BASE;
@@ -524,74 +535,34 @@
 	apic_write(APIC_ESR, 0);
 	apic_read(APIC_ESR);
 	local_irq_restore(flags);
-	if (apic_pm_state.perfctr_pmdev)
-		pm_send(apic_pm_state.perfctr_pmdev, PM_RESUME, data);
-}
-
-static int apic_pm_callback(struct pm_dev *dev, pm_request_t rqst, void *data)
-{
-	switch (rqst) {
-	case PM_SUSPEND:
-		apic_pm_suspend(data);
-		break;
-	case PM_RESUME:
-		apic_pm_resume(data);
-		break;
-	}
 	return 0;
 }
 
-/* perfctr driver should call this instead of pm_register() */
-struct pm_dev *apic_pm_register(pm_dev_t type,
-				unsigned long id,
-				pm_callback callback)
-{
-	struct pm_dev *dev;
-
-	if (!apic_pm_state.active)
-		return pm_register(type, id, callback);
-	if (apic_pm_state.perfctr_pmdev)
-		return NULL;	/* we're busy */
-	dev = kmalloc(sizeof(struct pm_dev), GFP_KERNEL);
-	if (dev) {
-		memset(dev, 0, sizeof(*dev));
-		dev->type = type;
-		dev->id = id;
-		dev->callback = callback;
-		apic_pm_state.perfctr_pmdev = dev;
-	}
-	return dev;
-}
+static struct device_driver apic_driver = {
+	.name		= "apic",
+	.bus		= &system_bus_type,
+	.resume		= apic_resume,
+	.suspend	= apic_suspend,
+};
+
+struct sys_device device_apic = {
+	.name		= "apic",
+	.id		= 0,
+	.dev		= {
+		.name	= "APIC",
+		.driver	= &apic_driver,
+	},
+};
 
-/* perfctr driver should call this instead of pm_unregister() */
-void apic_pm_unregister(struct pm_dev *dev)
-{
-	if (!apic_pm_state.active) {
-		pm_unregister(dev);
-	} else if (dev == apic_pm_state.perfctr_pmdev) {
-		apic_pm_state.perfctr_pmdev = NULL;
-		kfree(dev);
-	}
-}
-
-static void __init apic_pm_init1(void)
-{
-	/* can't pm_register() at this early stage in the boot process
-	   (causes an immediate reboot), so just set the flag */
-	apic_pm_state.active = 1;
-}
-
-static void __init apic_pm_init2(void)
+static int __init init_apic_devicefs(void)
 {
+	driver_register(&apic_driver);
 	if (apic_pm_state.active)
-		pm_register(PM_SYS_DEV, 0, apic_pm_callback);
+		return sys_device_register(&device_apic);
+	return 0;
 }
 
-#else	/* CONFIG_PM */
-
-static inline void apic_pm_init1(void) { }
-static inline void apic_pm_init2(void) { }
-
+device_initcall(init_apic_devicefs);
 #endif	/* CONFIG_PM */
 
 /*
@@ -658,9 +629,6 @@
 		nmi_watchdog = NMI_LOCAL_APIC;
 
 	printk("Found and enabled local APIC!\n");
-
-	apic_pm_init1();
-
 	return 0;
 
 no_apic:
@@ -671,7 +639,6 @@
 void __init init_apic_mappings(void)
 {
 	unsigned long apic_phys;
-
 	/*
 	 * If no local APIC can be found then set up a fake all
 	 * zeroes page to simulate the local APIC and another
@@ -1141,8 +1108,6 @@
 	phys_cpu_present_map = 1;
 	apic_write_around(APIC_ID, boot_cpu_physical_apicid);
 
-	apic_pm_init2();
-
 	setup_local_APIC();
 
 	if (nmi_watchdog == NMI_LOCAL_APIC)
arch/i386/kernel/apm.c +9 -0
@@ -1263,6 +1263,11 @@
 		}
 		printk(KERN_CRIT "apm: suspend was vetoed, but suspending anyway.\n");
 	}
+
+	device_suspend(3, SUSPEND_NOTIFY);
+	device_suspend(3, SUSPEND_SAVE_STATE);
+	device_suspend(3, SUSPEND_DISABLE);
+
 	/* serialize with the timer interrupt */
 	write_lock_irq(&xtime_lock);
 
@@ -1283,6 +1288,8 @@
 	if (err != APM_SUCCESS)
 		apm_error("suspend", err);
 	err = (err == APM_SUCCESS) ? 0 : -EIO;
+	device_resume(RESUME_RESTORE_STATE);
+	device_resume(RESUME_ENABLE);
 	pm_send_all(PM_RESUME, (void *)0);
 	queue_event(APM_NORMAL_RESUME, NULL);
  out:
@@ -1396,6 +1403,8 @@
 				write_lock_irq(&xtime_lock);
 				set_time();
 				write_unlock_irq(&xtime_lock);
+				device_resume(RESUME_RESTORE_STATE);
+				device_resume(RESUME_ENABLE);
 				pm_send_all(PM_RESUME, (void *)0);
 				queue_event(event, NULL);
 			}
arch/i386/kernel/i386_ksyms.c +0 -4
@@ -161,10 +161,6 @@
 EXPORT_SYMBOL(flush_tlb_page);
 #endif
 
-#if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_PM)
-EXPORT_SYMBOL_GPL(set_nmi_pm_callback);
-EXPORT_SYMBOL_GPL(unset_nmi_pm_callback);
-#endif
 #ifdef CONFIG_X86_IO_APIC
 EXPORT_SYMBOL(IO_APIC_get_PCI_irq_vector);
 #endif
arch/i386/kernel/nmi.c +55 -40

185 lines not on this page · raw message

arch/i386/oprofile/nmi_int.c +56 -25

119 lines not on this page · raw message

include/asm-i386/apic.h +1 -0
@@ -95,6 +95,7 @@
 #define NMI_IO_APIC	1
 #define NMI_LOCAL_APIC	2
 #define NMI_INVALID	3
+#define NMI_LOCAL_APIC_SUSPENDED_BY_OPROFILE	4
 
 #endif /* CONFIG_X86_LOCAL_APIC */
 
Signature · 2 lines
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
#7 lore raw

Here is my modified version of Pavel's latest patch to convert apm/apic/nmi to the driver model. It's a minimalistic patch, intendended ONLY to convert the old-fashioned PM support code to the driver model. It seems to work for me, except that initiating a suspend (via apm --suspend) triggers a BUG_ON somewhere in ide-disk.c, which prevents the suspend and causes a hang at shutdown.

John: I didn't change oprofile's nmi_int.c from Pavel's version so it's broken in this patch. I did add enable_local_apic_nmi_watchdog() and disable_local_apic_nmi_watchdog() and EXPORT_SYMBOL'd them, so you should be able to use that interface.

/Mikael

arch/i386/kernel/apic.c renamed from arch/i386/kernel/apic.c.~1~ +41 -66
@@ -10,6 +10,8 @@
  *					for testing these extensively.
  *	Maciej W. Rozycki	:	Various updates and fixes.
  *	Mikael Pettersson	:	Power Management for UP-APIC.
+ *	Pavel Machek and
+ *	Mikael Pettersson	:	PM converted to driver model.
  */
 
 #include <linux/config.h>
@@ -439,17 +441,13 @@
 
 #ifdef CONFIG_PM
 
-#include <linux/slab.h>
-#include <linux/pm.h>
+#include <linux/device.h>
 
 static struct {
 	/* 'active' is true if the local APIC was enabled by us and
 	   not the BIOS; this signifies that we are also responsible
 	   for disabling it before entering apm/acpi suspend */
 	int active;
-	/* 'perfctr_pmdev' is here because the current (2.4.1) PM
-	   callback system doesn't handle hierarchical dependencies */
-	struct pm_dev *perfctr_pmdev;
 	/* r/w apic fields */
 	unsigned int apic_id;
 	unsigned int apic_taskpri;
@@ -466,13 +464,14 @@
 	unsigned int apic_thmr;
 } apic_pm_state;
 
-static void apic_pm_suspend(void *data)
+static int local_apic_suspend(struct device *dev, u32 state, u32 level)
 {
 	unsigned int l, h;
 	unsigned long flags;
 
-	if (apic_pm_state.perfctr_pmdev)
-		pm_send(apic_pm_state.perfctr_pmdev, PM_SUSPEND, data);
+	if (level != SUSPEND_DISABLE)
+		return 0;
+
 	apic_pm_state.apic_id = apic_read(APIC_ID);
 	apic_pm_state.apic_taskpri = apic_read(APIC_TASKPRI);
 	apic_pm_state.apic_ldr = apic_read(APIC_LDR);
@@ -493,13 +492,17 @@
 	l &= ~MSR_IA32_APICBASE_ENABLE;
 	wrmsr(MSR_IA32_APICBASE, l, h);
 	local_irq_restore(flags);
+	return 0;
 }
 
-static void apic_pm_resume(void *data)
+static int local_apic_resume(struct device *dev, u32 level)
 {
 	unsigned int l, h;
 	unsigned long flags;
 
+	if (level != RESUME_POWER_ON)
+		return 0;
+
 	local_irq_save(flags);
 	rdmsr(MSR_IA32_APICBASE, l, h);
 	l &= ~MSR_IA32_APICBASE_BASE;
@@ -524,73 +527,47 @@
 	apic_write(APIC_ESR, 0);
 	apic_read(APIC_ESR);
 	local_irq_restore(flags);
-	if (apic_pm_state.perfctr_pmdev)
-		pm_send(apic_pm_state.perfctr_pmdev, PM_RESUME, data);
-}
-
-static int apic_pm_callback(struct pm_dev *dev, pm_request_t rqst, void *data)
-{
-	switch (rqst) {
-	case PM_SUSPEND:
-		apic_pm_suspend(data);
-		break;
-	case PM_RESUME:
-		apic_pm_resume(data);
-		break;
-	}
 	return 0;
 }
 
-/* perfctr driver should call this instead of pm_register() */
-struct pm_dev *apic_pm_register(pm_dev_t type,
-				unsigned long id,
-				pm_callback callback)
-{
-	struct pm_dev *dev;
-
-	if (!apic_pm_state.active)
-		return pm_register(type, id, callback);
-	if (apic_pm_state.perfctr_pmdev)
-		return NULL;	/* we're busy */
-	dev = kmalloc(sizeof(struct pm_dev), GFP_KERNEL);
-	if (dev) {
-		memset(dev, 0, sizeof(*dev));
-		dev->type = type;
-		dev->id = id;
-		dev->callback = callback;
-		apic_pm_state.perfctr_pmdev = dev;
-	}
-	return dev;
-}
-
-/* perfctr driver should call this instead of pm_unregister() */
-void apic_pm_unregister(struct pm_dev *dev)
-{
-	if (!apic_pm_state.active) {
-		pm_unregister(dev);
-	} else if (dev == apic_pm_state.perfctr_pmdev) {
-		apic_pm_state.perfctr_pmdev = NULL;
-		kfree(dev);
-	}
-}
+static struct device_driver local_apic_driver = {
+	.name		= "local_apic",
+	.bus		= &system_bus_type,
+	.resume		= local_apic_resume,
+	.suspend	= local_apic_suspend,
+};
+
+/* not static, needed by child devices */
+struct sys_device device_local_apic = {
+	.name		= "local_apic",
+	.id		= 0,
+	.dev		= {
+		.name	= "local_apic",
+		.driver	= &local_apic_driver,
+	},
+};
 
-static void __init apic_pm_init1(void)
+static void __init apic_pm_activate(void)
 {
-	/* can't pm_register() at this early stage in the boot process
-	   (causes an immediate reboot), so just set the flag */
 	apic_pm_state.active = 1;
 }
 
-static void __init apic_pm_init2(void)
+static int __init init_local_apic_devicefs(void)
 {
-	if (apic_pm_state.active)
-		pm_register(PM_SYS_DEV, 0, apic_pm_callback);
+	if (!cpu_has_apic)
+		return 0;
+	if (!apic_pm_state.active) {
+		local_apic_driver.resume = NULL;
+		local_apic_driver.suspend = NULL;
+	}
+	driver_register(&local_apic_driver);
+	return sys_device_register(&device_local_apic);
 }
+device_initcall(init_local_apic_devicefs);
 
 #else	/* CONFIG_PM */
 
-static inline void apic_pm_init1(void) { }
-static inline void apic_pm_init2(void) { }
+static inline void apic_pm_activate(void) { }
 
 #endif	/* CONFIG_PM */
 
@@ -659,7 +636,7 @@
 
 	printk("Found and enabled local APIC!\n");
 
-	apic_pm_init1();
+	apic_pm_activate();
 
 	return 0;
 
@@ -1141,8 +1118,6 @@
 	phys_cpu_present_map = 1;
 	apic_write_around(APIC_ID, boot_cpu_physical_apicid);
 
-	apic_pm_init2();
-
 	setup_local_APIC();
 
 	if (nmi_watchdog == NMI_LOCAL_APIC)
arch/i386/kernel/apm.c renamed from arch/i386/kernel/apm.c.~1~ +10 -0
@@ -218,6 +218,7 @@
 #include <linux/time.h>
 #include <linux/sched.h>
 #include <linux/pm.h>
+#include <linux/device.h>
 #include <linux/kernel.h>
 #include <linux/smp.h>
 #include <linux/smp_lock.h>
@@ -1263,6 +1264,11 @@
 		}
 		printk(KERN_CRIT "apm: suspend was vetoed, but suspending anyway.\n");
 	}
+
+	device_suspend(3, SUSPEND_NOTIFY);
+	device_suspend(3, SUSPEND_SAVE_STATE);
+	device_suspend(3, SUSPEND_DISABLE);
+
 	/* serialize with the timer interrupt */
 	write_seqlock_irq(&xtime_lock);
 
@@ -1283,6 +1289,8 @@
 	if (err != APM_SUCCESS)
 		apm_error("suspend", err);
 	err = (err == APM_SUCCESS) ? 0 : -EIO;
+	device_resume(RESUME_RESTORE_STATE);
+	device_resume(RESUME_ENABLE);
 	pm_send_all(PM_RESUME, (void *)0);
 	queue_event(APM_NORMAL_RESUME, NULL);
  out:
@@ -1396,6 +1404,8 @@
 				write_seqlock_irq(&xtime_lock);
 				set_time();
 				write_sequnlock_irq(&xtime_lock);
+				device_resume(RESUME_RESTORE_STATE);
+				device_resume(RESUME_ENABLE);
 				pm_send_all(PM_RESUME, (void *)0);
 				queue_event(event, NULL);
 			}
arch/i386/kernel/i386_ksyms.c renamed from arch/i386/kernel/i386_ksyms.c.~1~ +0 -4
@@ -162,10 +162,6 @@
 EXPORT_SYMBOL(flush_tlb_page);
 #endif
 
-#if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_PM)
-EXPORT_SYMBOL_GPL(set_nmi_pm_callback);
-EXPORT_SYMBOL_GPL(unset_nmi_pm_callback);
-#endif
 #ifdef CONFIG_X86_IO_APIC
 EXPORT_SYMBOL(IO_APIC_get_PCI_irq_vector);
 #endif
arch/i386/kernel/nmi.c renamed from arch/i386/kernel/nmi.c.~1~ +63 -38

180 lines not on this page · raw message

arch/i386/oprofile/nmi_int.c renamed from arch/i386/oprofile/nmi_int.c.~1~ +56 -25

119 lines not on this page · raw message

include/asm-i386/apic.h renamed from include/asm-i386/apic.h.~1~ +2 -2
@@ -84,8 +84,8 @@
 extern void disable_APIC_timer(void);
 extern void enable_APIC_timer(void);
 
-extern struct pm_dev *apic_pm_register(pm_dev_t, unsigned long, pm_callback);
-extern void apic_pm_unregister(struct pm_dev*);
+extern void disable_local_apic_nmi_watchdog(void);
+extern void enable_local_apic_nmi_watchdog(void);
 
 extern int check_nmi_watchdog (void);
 extern void enable_NMI_through_LVT0 (void * dummy);
#8 lore raw

Hi!

Quoted 2 lines

Here is my modified version of Pavel's latest patch to convert apm/apic/nmi to the driver model. It's a minimalistic patch,

It looks good.

Quoted 5 lines

intendended ONLY to convert the old-fashioned PM support code to the driver model. It seems to work for me, except that initiating a suspend (via apm --suspend) triggers a BUG_ON somewhere in ide-disk.c, which prevents the suspend and causes a hang at shutdown.

If you want to workaround that hang,

arch/i386/kernel/apm.c line 1269

Quoted 7 lines
arch/i386/kernel/apm.c renamed from arch/i386/kernel/apm.c.~1~ +3 -0
@@ -1263,6 +1264,11 @@
 		}
 		printk(KERN_CRIT "apm: suspend was vetoed, but suspending anyway.\n");
 	}
+
+	device_suspend(3, SUSPEND_NOTIFY);
+	device_suspend(3, SUSPEND_SAVE_STATE);

Kill these two lines ^

arch/i386/kernel/apm.c line 1292

Quoted 10 lines
arch/i386/kernel/apm.c renamed from arch/i386/kernel/apm.c.~1~ +3 -0
+	device_suspend(3, SUSPEND_DISABLE);
+
 	/* serialize with the timer interrupt */
 	write_seqlock_irq(&xtime_lock);
 
@@ -1283,6 +1289,8 @@
 	if (err != APM_SUCCESS)
 		apm_error("suspend", err);
 	err = (err == APM_SUCCESS) ? 0 : -EIO;
+	device_resume(RESUME_RESTORE_STATE);

This line ^

arch/i386/kernel/apm.c line 1407

Quoted 9 lines
arch/i386/kernel/apm.c renamed from arch/i386/kernel/apm.c.~1~ +2 -0
+	device_resume(RESUME_ENABLE);
 	pm_send_all(PM_RESUME, (void *)0);
 	queue_event(APM_NORMAL_RESUME, NULL);
  out:
@@ -1396,6 +1404,8 @@
 				write_seqlock_irq(&xtime_lock);
 				set_time();
 				write_sequnlock_irq(&xtime_lock);
+				device_resume(RESUME_RESTORE_STATE);

And this line ^.

Its not right thing to do, but it should make it work for you.

Pavel

Signature · 2 lines
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.
#9 lore raw
In reply to Mikael Pettersson

arch/i386/kernel/apic.c line 557

Quoted 5 lines · Mikael Pettersson
arch/i386/kernel/apic.c renamed from arch/i386/kernel/apic.c.~1~ +2 -2
+static int __init init_local_apic_devicefs(void)
 {
-	if (apic_pm_state.active)
-		pm_register(PM_SYS_DEV, 0, apic_pm_callback);
+	if (!cpu_has_apic)

This looks broken -- what if an external local APIC is present?

Quoted 8 lines
arch/i386/kernel/apic.c renamed from arch/i386/kernel/apic.c.~1~ +7 -0
+		return 0;
+	if (!apic_pm_state.active) {
+		local_apic_driver.resume = NULL;
+		local_apic_driver.suspend = NULL;
+	}
+	driver_register(&local_apic_driver);
+	return sys_device_register(&device_local_apic);
 }
Signature · 3 lines
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +
#10 lore raw
Quoted 9 lines · Maciej W. Rozycki

arch/i386/kernel/apic.c line 557

Quoted 5 lines · Mikael Pettersson
arch/i386/kernel/apic.c renamed from arch/i386/kernel/apic.c.~1~ +2 -2
+static int __init init_local_apic_devicefs(void)
 {
-	if (apic_pm_state.active)
-		pm_register(PM_SYS_DEV, 0, apic_pm_callback);
+	if (!cpu_has_apic)

This looks broken -- what if an external local APIC is present?

My goal was to not change any behaviour from our current code, and from what I can tell, our current code does not support PM suspend and resume for old external-local-APIC machines. (They're mostly 486 MPs, right?)

The suspend/resume procedures only work on P6/K7 and up. There's a bug there in that we may try to run the suspend on a UP P5 with enabled local APIC, which won't work. So far, no one seems to have noticed :->

(Admittedly, the P5 errata sheets say "BIOS should disable the P5 local APIC on UP" so I don't think this case is very likely.)

/Mikael

#11 lore raw
Quoted 3 lines · Mikael Pettersson

My goal was to not change any behaviour from our current code, and from what I can tell, our current code does not support PM suspend and resume for old external-local-APIC machines. (They're mostly 486 MPs, right?)

Both i486 and Pentium (typically for quad support, etc.).

Quoted 3 lines

The suspend/resume procedures only work on P6/K7 and up. There's a bug there in that we may try to run the suspend on a UP P5 with enabled local APIC, which won't work. So far, no one seems to have noticed :->

OK, but then the question is: are the following calls:

+ driver_register(&local_apic_driver);
+ return sys_device_register(&device_local_apic);

for suspend/resume exclusively?

Signature · 3 lines
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +
#12 lore raw
Quoted 6 lines · Maciej W. Rozycki

OK, but then the question is: are the following calls:

arch/i386/kernel/apic.c renamed from arch/i386/kernel/apic.c.~1~ +2 -0
+	driver_register(&local_apic_driver);
+	return sys_device_register(&device_local_apic);

for suspend/resume exclusively?

Yes.

We could register the device also in other cases (!PM, SMP) but the methods would then be nullified and we'd have a device node with a name but no operations. I could do that, I just think it's pointless.

Getting suspend to work on SMP is a separate project. This patch is just about upgrading to the new infrastructure.

/Mikael

#13 lore raw
Quoted 8 lines · Mikael Pettersson
Quoted 6 lines

OK, but then the question is: are the following calls:

arch/i386/kernel/apic.c renamed from arch/i386/kernel/apic.c.~1~ +2 -0
+	driver_register(&local_apic_driver);
+	return sys_device_register(&device_local_apic);

for suspend/resume exclusively?

Yes.

OK, then.

Quoted 4 lines

We could register the device also in other cases (!PM, SMP) but the methods would then be nullified and we'd have a device node with a name but no operations. I could do that, I just think it's pointless.

Agreed if the interface is not going to be extended further, i.e. the
intent is to cover PM-capable devices only.

I'd prefer the discrete APIC support not to get broken accidentally as
such systems are rare thus testing is limited.

Signature · 3 lines
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +
#14 lore raw
In reply to Mikael Pettersson

Hi!

Quoted 21 lines

Here is my modified version of Pavel's latest patch to convert apm/apic/nmi to the driver model. It's a minimalistic patch, intendended ONLY to convert the old-fashioned PM support code to the driver model. It seems to work for me, except that initiating a suspend (via apm --suspend) triggers a BUG_ON somewhere in ide-disk.c, which prevents the suspend and causes a hang at shutdown.

arch/i386/kernel/apm.c renamed from arch/i386/kernel/apm.c.~1~ +1 -0
@@ -218,6 +218,7 @@
 #include <linux/time.h>
 #include <linux/sched.h>
 #include <linux/pm.h>
+#include <linux/device.h>
 #include <linux/kernel.h>
 #include <linux/smp.h>
 #include <linux/smp_lock.h>

arch/i386/kernel/apm.c line 1269

Quoted 6 lines
arch/i386/kernel/apm.c renamed from arch/i386/kernel/apm.c.~1~ +3 -0
@@ -1263,6 +1264,11 @@
 		}
 		printk(KERN_CRIT "apm: suspend was vetoed, but suspending anyway.\n");
 	}
+
+	device_suspend(3, SUSPEND_NOTIFY);
+	device_suspend(3, SUSPEND_SAVE_STATE);

Comment these two lines... and all RESTORE_STATEs. System needs to be stopped in order for SAVE_STATE to work, and it is not in apm case.

arch/i386/kernel/apm.c

Quoted 6 lines
arch/i386/kernel/apm.c renamed from arch/i386/kernel/apm.c.~1~ +6 -0
+static struct device_driver local_apic_nmi_driver = {
+	.name		= "local_apic_nmi",
+	.bus		= &system_bus_type,
+	.resume		= nmi_resume,
+	.suspend	= nmi_suspend,
+};

Do you think it is neccessary to call it "*local_*apic_nmi_driver"? It seems way too long.

arch/i386/kernel/apm.c

Quoted 11 lines
arch/i386/kernel/apm.c renamed from arch/i386/kernel/apm.c.~1~ +11 -0
+extern struct sys_device device_local_apic;
+
+static struct sys_device device_local_apic_nmi = {
+	.name		= "local_apic_nmi",
+	.id		= 0,
+	.dev		= {
+		.name	= "local_apic_nmi",
+		.driver	= &local_apic_nmi_driver,
+		.parent = &device_local_apic.dev,
+	},
+};

Why did you convert device_apic_nmi to *sys_*device?

arch/i386/kernel/apm.c line 429

Quoted 8 lines
arch/i386/kernel/apm.c renamed from arch/i386/kernel/apm.c.~1~ +4 -0
@@ -402,3 +423,7 @@
 		wrmsr(nmi_perfctr_msr, -(cpu_khz/nmi_hz*1000), -1);
 	}
 }
+
+EXPORT_SYMBOL(nmi_watchdog);
+EXPORT_SYMBOL(disable_local_apic_nmi_watchdog);
+EXPORT_SYMBOL(enable_local_apic_nmi_watchdog);

This is good, if we have disable_, we should have enable_, not setup_; but I killed _local_ part as it is way too long, then.

Pavel

Signature · 2 lines
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
#15 lore raw
In reply to Mikael Pettersson

Hi!

Quoted 5 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+	set_fixmap_nocache(FIX_APIC_BASE, apic_phys);		/* FIXME: this is needed for S3 resume, but why? */

This is horrible! The only reason this might be needed is if the page tables weren't restored properly at resume, and that indicates a bug somewhere else.

Pagetables should be restored okay, I do not know what is going on.

Quoted 10 lines
Quoted 7 lines

+static int __init init_apic_devicefs(void)
{

(quoted diff) +1 -0
+	driver_register(&apic_driver);

if (apic_pm_state.active)

(quoted diff) +2 -1
-		pm_register(PM_SYS_DEV, 0, apic_pm_callback);
+		return sys_device_register(&device_apic);
+	return 0;

...

Quoted 1 line

+device_initcall(init_apic_devicefs);

Quoted 7 lines

init_apic_devicefs() registers apic_driver even if active is false. This probably breaks any machine where cpu_has_apic is false, since apic_suspend/resume will access non-existent APIC registers & MSRs.

I don't like the idea of registering apic_driver when !cpu_has_apic, but it might be needed for the local-APIC NMI watchdog. A fix could be to guard apic_suspend/resume with "if(!cpu_has_apic)return;".

cpu_has_apic test seems wrong: AFAICS, there are CPUs that have apic but linux does not know how to use it.

Quoted 3 lines
Quoted 1 line
		nmi_watchdog = nmi;

You just killed NMI_LOCAL_APIC support on Intel.

Fixed.

Quoted 4 lines
Quoted 1 line

+device_initcall(init_nmi_devicefs);

Doesn't this require that init_apic_devicefs() has been done? Can you guarantee that?

Moved to late_initcall().

Quoted 7 lines
Quoted 3 lines

+#define __pminit
#endif /* CONFIG_PM */

__pminit is no longer defined when !CONFIG_PM, which will cause compile errors. Possibly the remaining uses of __pminit should be removed.

Fixed.
Pavel

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#16 lore raw
Quoted 2 lines · Pavel Machek

cpu_has_apic test seems wrong: AFAICS, there are CPUs that have apic but linux does not know how to use it.

Please give a reference. I don't know of any x86-type CPU that fits your description above.

/Mikael