Skip to contentSkip to conversation

agpgart-amd64 not initialized in 2.6.33-rc5 if iommu=allowed in kernel command line

BUG Marin Mitov 2 participants 9 messages Jan 23, 2010 – Feb 4, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

If I start the kernel without command line parameters (4GB RAM) iommu-swiotlb is used due to quirk in VIA K8T800Pro Host Bridge. In that case all is OK.

If I put in the kernel command line: iommu=allowed (+other parameters) I have iommu-gart used but no AGP. dmesg output:

[drm:mga_do_agp_dma_bootstrap] *ERROR* Unable to acquire AGP: -19

For debugging I patched the kernel (mostly prink(), shown at the end) producing the output in the dmesg:

no command line
MDM: agp_amd64_init entered
MDM: agp_off: false
MDM: agp_bridges_found: 0
MDM: gart_iommu_aperture: 0
MDM: agp_amd64_probe entered

command line: iommu=allowed:
MDM: gart_iommu_init
MDM: before: no_agp = 0
MDM: agp_amd64_init entered
MDM: agp_off: false
MDM: agp_bridges_found: 0
MDM: after: no_agp = 1
MDM: agp_amd64_init() < 0: 1
MDM: agp_copy_info() < 0: -1
MDM: agp_amd64_init entered
MDM: agp_off: false
MDM: agp_bridges_found: 0

One see in the last case ( iommu=allowed) agp_amd64_init() is executed twice, but agp_amd64_probe() is not executed at all (agp_bridges_found is incremented only in agp_amd64_probe())

Is this expected behavior?

Thanks,

Marin Mitov

The testing patch:

drivers/char/agp/amd64-agp.c renamed from drivers/char/agp/amd64-agp.c.orig +5 -4
@@ -498,7 +498,7 @@
 	struct agp_bridge_data *bridge;
 	u8 cap_ptr;
 	int err;
-
+	printk(KERN_DEBUG "MDM: agp_amd64_probe entered\n");
 	cap_ptr = pci_find_capability(pdev, PCI_CAP_ID_AGP);
 	if (!cap_ptr)
 		return -ENODEV;
@@ -725,13 +725,14 @@
 int __init agp_amd64_init(void)
 {
 	int err = 0;
-
+	printk(KERN_DEBUG "MDM: agp_amd64_init entered\n");
 	if (agp_off)
 		return -EINVAL;
-
+	printk(KERN_DEBUG "MDM: agp_off: false\n");
+	printk(KERN_DEBUG "MDM: agp_bridges_found: %d\n", agp_bridges_found);
 	if (gart_iommu_aperture)
 		return agp_bridges_found ? 0 : -ENODEV;
-
+	printk(KERN_DEBUG "MDM: gart_iommu_aperture: %d\n", gart_iommu_aperture);
 	err = pci_register_driver(&agp_amd64_pci_driver);
 	if (err < 0)
 		return err;
arch/x86/kernel/pci-gart_64.c renamed from arch/x86/kernel/pci-gart_64.c.orig +8 -3
@@ -734,20 +734,25 @@
 	unsigned long start_pfn, end_pfn;
 	unsigned long scratch;
 	long i;
+	int tmp1 = -1, tmp2 = -1;
 
 	if (cache_k8_northbridges() < 0 || num_k8_northbridges == 0)
 		return 0;
 
+	printk(KERN_DEBUG "MDM: gart_iommu_init\n");
 #ifndef CONFIG_AGP_AMD64
 	no_agp = 1;
 #else
 	/* Makefile puts PCI initialization via subsys_initcall first. */
 	/* Add other K8 AGP bridge drivers here */
+	printk(KERN_DEBUG "MDM: before: no_agp = %d\n", no_agp);
 	no_agp = no_agp ||
-		(agp_amd64_init() < 0) ||
-		(agp_copy_info(agp_bridge, &info) < 0);
+		(tmp1 = (agp_amd64_init() < 0)) ||
+		(tmp2 = (agp_copy_info(agp_bridge, &info) < 0));
 #endif
-
+	printk(KERN_DEBUG "MDM: after: no_agp = %d\n", no_agp);
+	printk(KERN_DEBUG "MDM: agp_amd64_init() < 0: %d\n", tmp1);
+	printk(KERN_DEBUG "MDM: agp_copy_info() < 0: %d\n", tmp2); 
 	if (no_iommu ||
 	    (!force_iommu && max_pfn <= MAX_DMA32_PFN) ||
 	    !gart_iommu_aperture ||
#1 lore raw
Quoted 34 lines · On Sat, 23 Jan 2010 23:14:55 +0200 Marin Mitov <[email protected]> wrote:

Hi,

If I start the kernel without command line parameters (4GB RAM) iommu-swiotlb is used due to quirk in VIA K8T800Pro Host Bridge. In that case all is OK.

If I put in the kernel command line: iommu=allowed (+other parameters) I have iommu-gart used but no AGP. dmesg output:

[drm:mga_do_agp_dma_bootstrap] *ERROR* Unable to acquire AGP: -19

For debugging I patched the kernel (mostly prink(), shown at the end) producing the output in the dmesg:

no command line
MDM: agp_amd64_init entered
MDM: agp_off: false
MDM: agp_bridges_found: 0
MDM: gart_iommu_aperture: 0
MDM: agp_amd64_probe entered

command line: iommu=allowed:
MDM: gart_iommu_init
MDM: before: no_agp = 0
MDM: agp_amd64_init entered
MDM: agp_off: false
MDM: agp_bridges_found: 0
MDM: after: no_agp = 1
MDM: agp_amd64_init() < 0: 1
MDM: agp_copy_info() < 0: -1
MDM: agp_amd64_init entered
MDM: agp_off: false
MDM: agp_bridges_found: 0

Quoted 5 lines

One see in the last case ( iommu=allowed) agp_amd64_init() is executed twice, but agp_amd64_probe() is not executed at all (agp_bridges_found is incremented only in agp_amd64_probe())

Is this expected behavior?

Duh, it's my stupid mistake. Sorry about that.

This works for you?

Can you test this patch with set CONFIG_AGP_AMD64 to both y and m (also loading/unloading the module twice)?

Thanks,

=
From: FUJITA Tomonori <[email protected]>
Subject: [PATCH] x86/agp: fix agp_amd64_init regression

This fixes the regression introduced by the commit 42590a75019a50012f25a962246498dead428433.

The above commit changes agp_amd64_init() not to do anything if gart_iommu_aperture is not zero.

If GART iommu calls agp_amd64_init(), we need to skip agp_amd64_init() when it's called later via module_init.

The problem is that gart_iommu_init() calls agp_amd64_init() with not zero gart_iommu_aperture so agp_amd64_init() is never initialized.

When gart_iommu_init() calls agp_amd64_init(), agp should be always initialized.

---
 drivers/char/agp/amd64-agp.c |   13 +++++++++----
 1 files changed, 9 insertions(+), 4 deletions(-)
drivers/char/agp/amd64-agp.c +9 -4
index 1afb896..34cf04e 100644
@@ -729,9 +729,6 @@ int __init agp_amd64_init(void)
 	if (agp_off)
 		return -EINVAL;
 
-	if (gart_iommu_aperture)
-		return agp_bridges_found ? 0 : -ENODEV;
-
 	err = pci_register_driver(&agp_amd64_pci_driver);
 	if (err < 0)
 		return err;
@@ -768,6 +765,14 @@ int __init agp_amd64_init(void)
 	return err;
 }
 
+static int __init agp_amd64_mod_init(void)
+{
+	if (gart_iommu_aperture)
+		return agp_bridges_found ? 0 : -ENODEV;
+
+	return agp_amd64_init();
+}
+
 static void __exit agp_amd64_cleanup(void)
 {
 	if (gart_iommu_aperture)
@@ -777,7 +782,7 @@ static void __exit agp_amd64_cleanup(void)
 	pci_unregister_driver(&agp_amd64_pci_driver);
 }
 
-module_init(agp_amd64_init);
+module_init(agp_amd64_mod_init);
 module_exit(agp_amd64_cleanup);
 
 MODULE_AUTHOR("Dave Jones <[email protected]>, Andi Kleen");
Signature · 1 line
1.5.6.5
#2 lore raw
Quoted 41 lines · FUJITA Tomonori
Quoted 38 lines · On Sat, 23 Jan 2010 23:14:55 +0200 Marin Mitov <[email protected]> wrote:

Hi,

If I start the kernel without command line parameters (4GB RAM) iommu-swiotlb is used due to quirk in VIA K8T800Pro Host Bridge. In that case all is OK.

If I put in the kernel command line: iommu=allowed (+other parameters) I have iommu-gart used but no AGP. dmesg output:

[drm:mga_do_agp_dma_bootstrap] *ERROR* Unable to acquire AGP: -19

For debugging I patched the kernel (mostly prink(), shown at the end) producing the output in the dmesg:

no command line
MDM: agp_amd64_init entered
MDM: agp_off: false
MDM: agp_bridges_found: 0
MDM: gart_iommu_aperture: 0
MDM: agp_amd64_probe entered

command line: iommu=allowed:
MDM: gart_iommu_init
MDM: before: no_agp = 0
MDM: agp_amd64_init entered
MDM: agp_off: false
MDM: agp_bridges_found: 0
MDM: after: no_agp = 1
MDM: agp_amd64_init() < 0: 1
MDM: agp_copy_info() < 0: -1
MDM: agp_amd64_init entered
MDM: agp_off: false
MDM: agp_bridges_found: 0

One see in the last case ( iommu=allowed) agp_amd64_init() is executed twice, but agp_amd64_probe() is not executed at all (agp_bridges_found is incremented only in agp_amd64_probe())

Quoted 8 lines
Quoted 1 line

Is this expected behavior?

Duh, it's my stupid mistake. Sorry about that.

This works for you?

Can you test this patch with set CONFIG_AGP_AMD64 to both y and m (also loading/unloading the module twice)?

If CONFIG_AGP_AMD64=y it works (I have agp) both with no kernel command line parameters as well as with iommu=allowed in kernel command line.

If CONFIG_AGP_AMD64=m it works (I have agp) with no kernel command line parameters. If I boot to no graphics (runlevel 3 in slackware), lsmod output is:

amd64_agp 7463 1
agpgart 27765 1 amd64_agp

amd64_agp is in use (not known by who), cannot be unloaded and I cannot test load/unload.

If CONFIG_AGP_AMD64=m and iommu=allowed is in kernel command line I have no agp.
dmesg:
[drm:mga_do_agp_dma_bootstrap] *ERROR* Unable to acquire AGP: -19 If I boot to no graphics and try:
modprobe amd64_agp
FATAL: Error inserting amd64_agp (/lib/modules/2.6.33-rc5/kernel/drivers/char/agp/amd64-agp.ko): No such device

For info: iommu=allowed changes gart_iommu_aperture_allowed from 0 to 1.

I am ready for further testing.

Marin Mitov

Quoted 65 lines

Thanks,

=
From: FUJITA Tomonori <[email protected]>
Subject: [PATCH] x86/agp: fix agp_amd64_init regression

This fixes the regression introduced by the commit 42590a75019a50012f25a962246498dead428433.

The above commit changes agp_amd64_init() not to do anything if gart_iommu_aperture is not zero.

If GART iommu calls agp_amd64_init(), we need to skip agp_amd64_init() when it's called later via module_init.

The problem is that gart_iommu_init() calls agp_amd64_init() with not zero gart_iommu_aperture so agp_amd64_init() is never initialized.

When gart_iommu_init() calls agp_amd64_init(), agp should be always initialized.

---
 drivers/char/agp/amd64-agp.c |   13 +++++++++----
 1 files changed, 9 insertions(+), 4 deletions(-)
drivers/char/agp/amd64-agp.c +9 -4
index 1afb896..34cf04e 100644
@@ -729,9 +729,6 @@ int __init agp_amd64_init(void)
 	if (agp_off)
 		return -EINVAL;
 
-	if (gart_iommu_aperture)
-		return agp_bridges_found ? 0 : -ENODEV;
-
 	err = pci_register_driver(&agp_amd64_pci_driver);
 	if (err < 0)
 		return err;
@@ -768,6 +765,14 @@ int __init agp_amd64_init(void)
 	return err;
 }
 
+static int __init agp_amd64_mod_init(void)
+{
+	if (gart_iommu_aperture)
+		return agp_bridges_found ? 0 : -ENODEV;
+
+	return agp_amd64_init();
+}
+
 static void __exit agp_amd64_cleanup(void)
 {
 	if (gart_iommu_aperture)
@@ -777,7 +782,7 @@ static void __exit agp_amd64_cleanup(void)
 	pci_unregister_driver(&agp_amd64_pci_driver);
 }
 
-module_init(agp_amd64_init);
+module_init(agp_amd64_mod_init);
 module_exit(agp_amd64_cleanup);
 
 MODULE_AUTHOR("Dave Jones <[email protected]>, Andi Kleen");
#3 lore raw

Really sorry for taking so long. I had been away from my workplace.

Quoted 5 lines · On Mon, 25 Jan 2010 09:37:53 +0200 Marin Mitov <[email protected]> wrote:
Quoted 2 lines

Can you test this patch with set CONFIG_AGP_AMD64 to both y and m (also loading/unloading the module twice)?

If CONFIG_AGP_AMD64=y it works (I have agp) both with no kernel command line parameters as well as with iommu=allowed in kernel command line.

Good.

Quoted 7 lines

If CONFIG_AGP_AMD64=m it works (I have agp) with no kernel command line parameters. If I boot to no graphics (runlevel 3 in slackware), lsmod output is:

amd64_agp 7463 1
agpgart 27765 1 amd64_agp

Quoted 7 lines

amd64_agp is in use (not known by who), cannot be unloaded and I cannot test load/unload.

If CONFIG_AGP_AMD64=m and iommu=allowed is in kernel command line I have no agp.
dmesg:
[drm:mga_do_agp_dma_bootstrap] *ERROR* Unable to acquire AGP: -19

This works with 2.6.32, right? The following patch works?

Sorry for inconvenience.

Quoted 5 lines

If I boot to no graphics and try:
modprobe amd64_agp
FATAL: Error inserting amd64_agp (/lib/modules/2.6.33-rc5/kernel/drivers/char/agp/amd64-agp.ko): No such device

drivers/char/agp/amd64-agp.c +1 -1
index 34cf04e..8d67923 100644
@@ -768,7 +768,7 @@ int __init agp_amd64_init(void)
 static int __init agp_amd64_mod_init(void)
 {
 	if (gart_iommu_aperture)
-		return agp_bridges_found ? 0 : -ENODEV;
+		return 0;
 
 	return agp_amd64_init();
 }
#4 lore raw
Quoted 10 lines · On Tue, 2 Feb 2010 20:23:40 +0900 FUJITA Tomonori…
Quoted 10 lines

If CONFIG_AGP_AMD64=m it works (I have agp) with no kernel command line parameters. If I boot to no graphics (runlevel 3 in slackware), lsmod output is:

amd64_agp 7463 1
agpgart 27765 1 amd64_agp

amd64_agp is in use (not known by who), cannot be unloaded and I cannot test load/unload.

Quoted 6 lines
Quoted 4 lines

If CONFIG_AGP_AMD64=m and iommu=allowed is in kernel command line I have no agp.
dmesg:
[drm:mga_do_agp_dma_bootstrap] *ERROR* Unable to acquire AGP: -19

This works with 2.6.32, right? The following patch works?

Duh, sorry, please this instead:

drivers/char/agp/amd64-agp.c +4 -1
index 34cf04e..fd50ead 100644
@@ -767,16 +767,19 @@ int __init agp_amd64_init(void)
 
 static int __init agp_amd64_mod_init(void)
 {
+#ifndef MODULE
 	if (gart_iommu_aperture)
 		return agp_bridges_found ? 0 : -ENODEV;
-
+#endif
 	return agp_amd64_init();
 }
 
 static void __exit agp_amd64_cleanup(void)
 {
+#ifndef MODULE
 	if (gart_iommu_aperture)
 		return;
+#endif
 	if (aperture_resource)
 		release_resource(aperture_resource);
 	pci_unregister_driver(&agp_amd64_pci_driver);
#5 lore raw
Quoted 13 lines · FUJITA Tomonori
Quoted 10 lines · On Tue, 2 Feb 2010 20:23:40 +0900 FUJITA Tomonori…
Quoted 10 lines

If CONFIG_AGP_AMD64=m it works (I have agp) with no kernel command line parameters. If I boot to no graphics (runlevel 3 in slackware), lsmod output is:

amd64_agp 7463 1
agpgart 27765 1 amd64_agp

amd64_agp is in use (not known by who), cannot be unloaded and I cannot test load/unload.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

If CONFIG_AGP_AMD64=m and iommu=allowed is in kernel command line I have no agp.
dmesg:
[drm:mga_do_agp_dma_bootstrap] *ERROR* Unable to acquire AGP: -19

This works with 2.6.32, right? The following patch works?

Duh, sorry, please this instead:

Sorry to be late.

First, the patch do not apply cleanly on 2.6.33-rc6 (I'm running now). I have to apply the patch in:

http://lkml.org/lkml/2010/1/25/4

first, then to apply:

http://lkml.org/lkml/2010/2/2/241

over it.

In this configuration (both patches applied) it works for me with/without kernel command line (iommu=allowed + ....) when either amd64-agp is built in the kernel (CONFIG_AGP_AMD64=y), or as a module (CONFIG_AGP_AMD64=m).

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

lsmod output:

amd64_agp 7463 1
agpgart 27765 1 amd64_agp

I have still amd64_agp used, so I cannot test load/unload amd64_agp:

Thanks,

Marin Mitov

Quoted 27 lines
drivers/char/agp/amd64-agp.c +4 -1
index 34cf04e..fd50ead 100644
@@ -767,16 +767,19 @@ int __init agp_amd64_init(void)
 
 static int __init agp_amd64_mod_init(void)
 {
+#ifndef MODULE
 	if (gart_iommu_aperture)
 		return agp_bridges_found ? 0 : -ENODEV;
-
+#endif
 	return agp_amd64_init();
 }
 
 static void __exit agp_amd64_cleanup(void)
 {
+#ifndef MODULE
 	if (gart_iommu_aperture)
 		return;
+#endif
 	if (aperture_resource)
 		release_resource(aperture_resource);
 	pci_unregister_driver(&agp_amd64_pci_driver);
#6 lore raw
Quoted 3 lines · On Wed, 3 Feb 2010 17:19:48 +0200 Marin Mitov <[email protected]> wrote:

In this configuration (both patches applied) it works for me with/without kernel command line (iommu=allowed + ....) when either amd64-agp is built in the kernel (CONFIG_AGP_AMD64=y), or as a module (CONFIG_AGP_AMD64=m).

Great, thanks a lot!

I've just resent the patch in the proper format.

Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

lsmod output:

amd64_agp 7463 1
agpgart 27765 1 amd64_agp

I have still amd64_agp used, so I cannot test load/unload amd64_agp:

I work on it if someone reports a problem about it.

#7 lore raw
In reply to FUJITA Tomonori
Quoted 1 line · FUJITA Tomonori

Duh, sorry, please this instead:

Applies cleanly to 2.6.33-rc6-git3. Works for me.
Details - see my previous post..

Thanks.

Marin Mitov

Quoted 27 lines
drivers/char/agp/amd64-agp.c +4 -1
index 34cf04e..fd50ead 100644
@@ -767,16 +767,19 @@ int __init agp_amd64_init(void)
 
 static int __init agp_amd64_mod_init(void)
 {
+#ifndef MODULE
 	if (gart_iommu_aperture)
 		return agp_bridges_found ? 0 : -ENODEV;
-
+#endif
 	return agp_amd64_init();
 }
 
 static void __exit agp_amd64_cleanup(void)
 {
+#ifndef MODULE
 	if (gart_iommu_aperture)
 		return;
+#endif
 	if (aperture_resource)
 		release_resource(aperture_resource);
 	pci_unregister_driver(&agp_amd64_pci_driver);
#8 lore raw

[tip:x86/urgent] x86/agp: Fix agp_amd64_init regression

In reply to FUJITA Tomonori
Commit-ID:  61684ceaad4f65d1a9832c722f7bd5e7fc714de9
Gitweb:     http://git.kernel.org/tip/61684ceaad4f65d1a9832c722f7bd5e7fc714de9
Author:     FUJITA Tomonori <[email protected]>
AuthorDate: Mon, 25 Jan 2010 14:10:47 +0900
Committer:  Ingo Molnar <[email protected]>
CommitDate: Sun, 31 Jan 2010 07:52:26 +0100

x86/agp: Fix agp_amd64_init regression

This fixes the regression introduced by commit 42590a75019a50012f25a962246498dead428433 ("x86/agp: Fix agp_amd64_init and agp_amd64_cleanup").

The above commit changes agp_amd64_init() not to do anything if gart_iommu_aperture is not zero.

If GART iommu calls agp_amd64_init(), we need to skip agp_amd64_init() when it's called later via module_init.

The problem is that gart_iommu_init() calls agp_amd64_init() with not zero gart_iommu_aperture so agp_amd64_init() is never initialized.

When gart_iommu_init() calls agp_amd64_init(), agp should be always initialized.

LKML-Reference: <[email protected]>

---
 drivers/char/agp/amd64-agp.c |   13 +++++++++----
 1 files changed, 9 insertions(+), 4 deletions(-)
drivers/char/agp/amd64-agp.c +9 -4
index 1afb896..34cf04e 100644
@@ -729,9 +729,6 @@ int __init agp_amd64_init(void)
 	if (agp_off)
 		return -EINVAL;
 
-	if (gart_iommu_aperture)
-		return agp_bridges_found ? 0 : -ENODEV;
-
 	err = pci_register_driver(&agp_amd64_pci_driver);
 	if (err < 0)
 		return err;
@@ -768,6 +765,14 @@ int __init agp_amd64_init(void)
 	return err;
 }
 
+static int __init agp_amd64_mod_init(void)
+{
+	if (gart_iommu_aperture)
+		return agp_bridges_found ? 0 : -ENODEV;
+
+	return agp_amd64_init();
+}
+
 static void __exit agp_amd64_cleanup(void)
 {
 	if (gart_iommu_aperture)
@@ -777,7 +782,7 @@ static void __exit agp_amd64_cleanup(void)
 	pci_unregister_driver(&agp_amd64_pci_driver);
 }
 
-module_init(agp_amd64_init);
+module_init(agp_amd64_mod_init);
 module_exit(agp_amd64_cleanup);
 
 MODULE_AUTHOR("Dave Jones <[email protected]>, Andi Kleen");