Skip to contentSkip to conversation

2.6.14, udev: unknown symbols for ehci_hcd

DISCUSSION Harald Dunkel 8 participants 22 messages Nov 5, 2005 – Nov 11, 2005 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

Hi folks,

I can't say since when this problem is in, but currently I get error messages about unknown symbols at boot time (after mounting the root disk, as it seems):

:
:
scsi0 : sata_sil
ata2: no device found (phy stat 00000000)
scsi1 : sata_sil
Vendor: ATA Model: SAMSUNG SP1614C Rev: SW10
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sda: 312581808 512-byte hdwr sectors (160042 MB)
SCSI device sda: drive cache: write back
SCSI device sda: 312581808 512-byte hdwr sectors (160042 MB)
SCSI device sda: drive cache: write back
sda: sda1 sda2 sda3 sda4
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
ReiserFS: sda1: found reiserfs format "3.6" with standard journal
ReiserFS: sda1: using ordered data mode
ReiserFS: sda1: journal params: device sda1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
ReiserFS: sda1: checking transaction log (sda1)
ReiserFS: sda1: Using r5 hash to sort names
NET: Registered protocol family 1
parport: PnPBIOS parport detected.
parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
usbcore: registered new driver usbfs
usbcore: registered new driver hub
ehci_hcd: Unknown symbol usb_hcd_pci_suspend
ehci_hcd: Unknown symbol usb_free_urb
ehci_hcd: Unknown symbol usb_hub_tt_clear_buffer
ehci_hcd: Unknown symbol usb_hcd_pci_probe
ehci_hcd: Unknown symbol usb_disabled
ehci_hcd: Unknown symbol usb_unlock_device
ehci_hcd: Unknown symbol usb_put_dev
ehci_hcd: Unknown symbol usb_get_dev
:

If I modprobe ehci_hcd later, then there is no error message. It is loaded as expected.

uname -a:
Linux pluto 2.6.14 #1 PREEMPT Sat Nov 5 08:47:20 CET 2005 x86_64 GNU/Linux udev is version 0.071-1.

???

Regards

Harri

#1 lore raw
Quoted 5 lines · Harald Dunkel

Hi folks,

I can't say since when this problem is in, but currently I get error messages about unknown symbols at boot time (after mounting the root disk, as it seems):

Are you using Debian?

thanks,

greg k-h

#14 lore raw
In reply to Harald Dunkel
Quoted 35 lines · Harald Dunkel

I can't say since when this problem is in, but currently
I get error messages about unknown symbols at boot time
(after mounting the root disk, as it seems):
:
scsi0 : sata_sil
ata2: no device found (phy stat 00000000)
scsi1 : sata_sil
Vendor: ATA Model: SAMSUNG SP1614C Rev: SW10
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sda: 312581808 512-byte hdwr sectors (160042 MB)
SCSI device sda: drive cache: write back
SCSI device sda: 312581808 512-byte hdwr sectors (160042 MB)
SCSI device sda: drive cache: write back
sda: sda1 sda2 sda3 sda4
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
ReiserFS: sda1: found reiserfs format "3.6" with standard journal
ReiserFS: sda1: using ordered data mode
ReiserFS: sda1: journal params: device sda1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
ReiserFS: sda1: checking transaction log (sda1)
ReiserFS: sda1: Using r5 hash to sort names
NET: Registered protocol family 1
parport: PnPBIOS parport detected.
parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
usbcore: registered new driver usbfs
usbcore: registered new driver hub
ehci_hcd: Unknown symbol usb_hcd_pci_suspend
ehci_hcd: Unknown symbol usb_free_urb
ehci_hcd: Unknown symbol usb_hub_tt_clear_buffer
ehci_hcd: Unknown symbol usb_hcd_pci_probe
ehci_hcd: Unknown symbol usb_disabled
ehci_hcd: Unknown symbol usb_unlock_device
ehci_hcd: Unknown symbol usb_put_dev
ehci_hcd: Unknown symbol usb_get_dev
:

Quoted 6 lines

If I modprobe ehci_hcd later, then there is no error message. It is loaded as expected.

uname -a:
Linux pluto 2.6.14 #1 PREEMPT Sat Nov 5 08:47:20 CET 2005 x86_64 GNU/Linux udev is version 0.071-1.

We (Debian and SUSE udev) are currently doing "coldplug" with udev, sysfs and the kernel exported $MODALIAS value instead of the old hotplug rc-scripts. Sysfs is scanned for all devices and hotplug events are synthesized, or with kernel 2.6.14+ are just triggered by a "uevent" file in sysfs. These events look like the same hotplug event as it happened (but was lost in initramfs) at device creation time. This approach does a heavy parallel module loading, which may nobody have ever tried before and seen this "bug" ...

I've got these messages several times on the experimental SUSE too, but can't reproduce it so far, even without Rusty's patch to modprobe they have disappeared. See here for the details:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333052

Thanks,
Kay

#15 lore raw
Quoted 4 lines · Kay Sievers

I've got these messages several times on the experimental SUSE too, but can't reproduce it so far, even without Rusty's patch to modprobe they have disappeared. See here for the details:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333052

Just to let you know, I've also met this problem (on another distro), and did not know about this bugreport until now.
So I've done another workaround: modprobe already parses /proc/modules to check whether the modules needed are already loaded, and this file also shows us the state of the modules, being "Loading", "Live" or "Unloading".

With my patch, modprobe waits until the needed modules come out of the "Loading" or "Unloading" state.

Signature · 49 lines
pozsy


--- module-init-tools-3.2-pre4.orig/modprobe.c	2005-05-08 09:38:52.000000000 +0200
+++ module-init-tools-3.2-pre4/modprobe.c	2005-10-24 13:19:39.000000000 +0200
@@ -363,6 +363,7 @@
 	FILE *proc_modules;
 	char *line;
 
+start:
 	/* Might not be mounted yet.  Don't fail. */
 	proc_modules = fopen("/proc/modules", "r");
 	if (!proc_modules)
@@ -373,12 +374,31 @@
 
 		if (entry && strcmp(entry, modname) == 0) {
 			/* If it exists, usecount is the third entry. */
-			if (usecount) {
-				entry = strtok(NULL, " \n");
-				if (entry
-				    && (entry = strtok(NULL, " \n")) != NULL)
+			if (!(entry = strtok(NULL, " \n")))
+				goto out;
+
+			if (!(entry = strtok(NULL, " \n"))) /* usecount */
+				goto out;
+			else
+				if (usecount)
 					*usecount = atoi(entry);
+
+			if (!(entry = strtok(NULL, " \n"))) /* "-" */
+				goto out;
+
+			if (!(entry = strtok(NULL, " \n"))) /* status */
+				goto out;
+			else {
+				if (!strcmp(entry, "Loading") || !strcmp(entry, "Unloading")) {
+					usleep(100000);
+					free(line);
+					fclose(proc_modules);
+					goto start;
+				}
+				goto out;
 			}
+
+		out:
 			free(line);
 			fclose(proc_modules);
 			return 1;
#16 lore raw
Quoted 14 lines · Pozsar Balazs
Quoted 4 lines · Kay Sievers

I've got these messages several times on the experimental SUSE too, but can't reproduce it so far, even without Rusty's patch to modprobe they have disappeared. See here for the details:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333052

Just to let you know, I've also met this problem (on another distro), and did not know about this bugreport until now.
So I've done another workaround: modprobe already parses /proc/modules to check whether the modules needed are already loaded, and this file also shows us the state of the modules, being "Loading", "Live" or "Unloading".

Quoted 2 lines

With my patch, modprobe waits until the needed modules come out of the "Loading" or "Unloading" state.

Yes, this was going to be my solution. However, we only need to resort to this is locking fails (read-only root filesystem).

Rusty.

Signature · 1 line
A bad analogy is like a leaky screwdriver -- Richard Braakman
#2 lore raw
In reply to Greg KH
Quoted 11 lines · Greg KH
Quoted 5 lines · Harald Dunkel

Hi folks,

I can't say since when this problem is in, but currently I get error messages about unknown symbols at boot time (after mounting the root disk, as it seems):

Are you using Debian?

Of course :=)

Regards

Harri

#17 lore raw

Re: Bug#333052: 2.6.14, udev: unknown symbols for ehci_hcd

In reply to Pozsar Balazs
Quoted 7 lines · Pozsar Balazs

On Sat, Nov 05, 2005 at 06:31:04PM +0100, Kay Sievers wrote:

With my patch, modprobe waits until the needed modules come out of the "Loading" or "Unloading" state.

For testing I have added it to Debian's
module-init-tools 3.2-pre9. Works for me.

Regards

Harri

#18 lore raw

Re: Bug#333052: 2.6.14, udev: unknown symbols for ehci_hcd

Quoted 4 lines · Harald Dunkel

For testing I have added it to Debian's
module-init-tools 3.2-pre9. Works for me.

No, it doesn't. After the 3rd reboot the
problem was back.

Regards

Harri

#19 lore raw

Re: Bug#333052: 2.6.14, udev: unknown symbols for ehci_hcd

Quoted 8 lines · Harald Dunkel
Quoted 4 lines · Harald Dunkel

For testing I have added it to Debian's
module-init-tools 3.2-pre9. Works for me.

No, it doesn't. After the 3rd reboot the
problem was back.

Well, that's really wierd, It Should Work(tm) :)
Did you apply both patches (Rusty's + mine), or only the latter?

Could you send me debug output please? The first time I met the problem, I used a modprobe wrapper which dumped /proc/modules and modprobe stdout/stderr to a temp file.

I would like to also mention, that my patch leaves a very little time window open, but that's only a problem if module unloading is also
happening: after parsing /proc/modules, but before actually loading the module, it is possible that an rmmod unloads (starts to unload) a dependant module. But this does not affect booting.

Signature · 1 line
pozsy
#3 lore raw
In reply to Harald Dunkel
Quoted 11 lines · El Sábado, 5 de Noviembre de 2005 23:59, escribió:
Quoted 8 lines · Greg KH
Quoted 5 lines · Harald Dunkel

Hi folks,

I can't say since when this problem is in, but currently I get error messages about unknown symbols at boot time (after mounting the root disk, as it seems):

Are you using Debian?

Of course :=)

I was having that problem using busybox insmod, changing to latest kernel tools was fixed for me, I don't know why, but in my case was a ramdisk to load sata drivers before mounting the root disk. Anyway module-init-tools is not too bigger even building them with glibc instead uClibc as static binaries.

:)

Signature · 3 lines
Gustavo Guillermo Pérez
Compunauta uLinux
www.compunauta.com
#20 lore raw

Re: Bug#333052: 2.6.14, udev: unknown symbols for ehci_hcd

In reply to Pozsar Balazs
Quoted 4 lines · Pozsar Balazs

Well, that's really wierd, It Should Work(tm) :)
Did you apply both patches (Rusty's + mine), or only the latter?

I hadn't seen Rusty's patch on Debian's bts, until you mentioned it. I have applied both patches now, and rebooted twice: By now it worked. But that's what I thought before.

Quoted 4 lines

Could you send me debug output please? The first time I met the problem, I used a modprobe wrapper which dumped /proc/modules and modprobe stdout/stderr to a temp file.

If the problem comes back then I will do.

Quoted 7 lines

I would like to also mention, that my patch leaves a very little time window open, but that's only a problem if module unloading is also
happening: after parsing /proc/modules, but before actually loading the module, it is possible that an rmmod unloads (starts to unload) a dependant module. But this does not affect booting.

Are there several modprobe's running in parallel? Or does modprobe return SUCCESS while the kernel is still busy "making the module usable somehow"?

Regards

Harri

#21 lore raw

Re: Bug#333052: 2.6.14, udev: unknown symbols for ehci_hcd

Quoted 3 lines · Harald Dunkel

I hadn't seen Rusty's patch on Debian's bts, until you mentioned it. I have applied both patches now, and rebooted twice: By now it worked. But that's what I thought before.

It cannot be relevant, because when the bug is triggered / is still read-only.

Quoted 1 line

Are there several modprobe's running in parallel? Or does modprobe

Yes.

Signature · 2 lines
ciao,
Marco
#5 lore raw
In reply to Harald Dunkel
Quoted 5 lines · Harald Dunkel
Quoted 4 lines · Greg KH
Quoted 2 lines · Harald Dunkel

Hi folks,

Quoted 8 lines
Quoted 7 lines
Quoted 3 lines

I can't say since when this problem is in, but currently I get error messages about unknown symbols at boot time (after mounting the root disk, as it seems):

Are you using Debian?

Of course :=)

This seems to be a Debian issue for some odd reason, I suggest filing a bug against the udev package (or just tagging onto the existing bug for this problem, I've seen it in there already...)

Good luck,

greg k-h

#22 lore raw

Re: Bug#333052: 2.6.14, udev: unknown symbols for ehci_hcd

In reply to Marco d'Itri
Quoted 5 lines · Marco d'Itri
Quoted 2 lines

Are there several modprobe's running in parallel? Or does modprobe

Yes.

Is this supposed to be synchronized in user space, or in the kernel?

Regards

Harri

#6 lore raw
In reply to Greg KH
Quoted 3 lines · Greg KH

This seems to be a Debian issue for some odd reason, I suggest filing a bug against the udev package (or just tagging onto the existing bug for this problem, I've seen it in there already...)

The reason this is usually seen only on Debian systems is that I am the first one who shipped an udev package which runs many parallel modprobe commands, but this is a genuine kernel/modprobe bug.

Signature · 2 lines
ciao,
Marco
#7 lore raw
Quoted 8 lines · Marco d'Itri
Quoted 3 lines · Greg KH

This seems to be a Debian issue for some odd reason, I suggest filing a bug against the udev package (or just tagging onto the existing bug for this problem, I've seen it in there already...)

The reason this is usually seen only on Debian systems is that I am the first one who shipped an udev package which runs many parallel modprobe commands, but this is a genuine kernel/modprobe bug.

I'm pretty sure OpenSuSE 10.0 does the same thing, and I don't think anyone has reported the same kind of bugs there. Makes me wonder what is really happening here...

thanks,

greg k-h

#8 lore raw
Quoted 6 lines · Greg KH
Quoted 5 lines · Marco d'Itri
Quoted 3 lines · Greg KH

This seems to be a Debian issue for some odd reason, I suggest filing a bug against the udev package (or just tagging onto the existing bug for this problem, I've seen it in there already...)

Quoted 7 lines
Quoted 3 lines

The reason this is usually seen only on Debian systems is that I am the first one who shipped an udev package which runs many parallel modprobe commands, but this is a genuine kernel/modprobe bug.

I'm pretty sure OpenSuSE 10.0 does the same thing, and I don't think anyone has reported the same kind of bugs there. Makes me wonder what is really happening here...

If module A depends on module B, and "modprobe A" and "modprobe B" are run parallel, there is time window when module B is already listed in /proc/modules, but not completely loaded/initialized, it is in the state "Loading". At this point "modprobe A" checks /proc/modules if module B is already loaded, but it does not take into account that it is in the state "Loading" and not yet "Live". So it tries to load module A, but it fails, because there are missing symbols because module A did not register them yet.

Signature · 1 line
pozsy
#9 lore raw
Quoted 11 lines · Pozsar Balazs
Quoted 10 lines · Greg KH
Quoted 8 lines · Marco d'Itri
Quoted 3 lines · Greg KH

This seems to be a Debian issue for some odd reason, I suggest filing a bug against the udev package (or just tagging onto the existing bug for this problem, I've seen it in there already...)

The reason this is usually seen only on Debian systems is that I am the first one who shipped an udev package which runs many parallel modprobe commands, but this is a genuine kernel/modprobe bug.

Quoted 6 lines
Quoted 3 lines

I'm pretty sure OpenSuSE 10.0 does the same thing, and I don't think anyone has reported the same kind of bugs there. Makes me wonder what is really happening here...

If module A depends on module B, and "modprobe A" and "modprobe B" are run parallel

Why would they be run in parallel? modprobe doesn't do this, why would you?

Quoted 7 lines

, there is time window when module B is already listed in /proc/modules, but not completely loaded/initialized, it is in the state "Loading". At this point "modprobe A" checks /proc/modules if module B is already loaded, but it does not take into account that it is in the state "Loading" and not yet "Live". So it tries to load module A, but it fails, because there are missing symbols because module A did not register them yet.

Sounds like a locking issue within the module core. I thought we could only load one at a time, otherwise we have other races within the kernel.

thanks,

greg k-h

#10 lore raw
Quoted 16 lines · Greg KH
Quoted 15 lines · Pozsar Balazs
Quoted 13 lines · Greg KH
Quoted 8 lines · Marco d'Itri
Quoted 3 lines · Greg KH

This seems to be a Debian issue for some odd reason, I suggest filing a bug against the udev package (or just tagging onto the existing bug for this problem, I've seen it in there already...)

The reason this is usually seen only on Debian systems is that I am the first one who shipped an udev package which runs many parallel modprobe commands, but this is a genuine kernel/modprobe bug.

I'm pretty sure OpenSuSE 10.0 does the same thing, and I don't think anyone has reported the same kind of bugs there. Makes me wonder what is really happening here...

Quoted 5 lines
Quoted 2 lines

If module A depends on module B, and "modprobe A" and "modprobe B" are run parallel

Why would they be run in parallel? modprobe doesn't do this, why would you?

On my machine it happened while loading pcmcia modules. It was two modprobes running in parallel (invoked by udev), not 1 modprobe loading modules in parallel.

Quoted 11 lines
Quoted 7 lines

, there is time window when module B is already listed in /proc/modules, but not completely loaded/initialized, it is in the state "Loading". At this point "modprobe A" checks /proc/modules if module B is already loaded, but it does not take into account that it is in the state "Loading" and not yet "Live". So it tries to load module A, but it fails, because there are missing symbols because module A did not register them yet.

Sounds like a locking issue within the module core. I thought we could only load one at a time, otherwise we have other races within the kernel.

That's what I thought too, but this is not the case it seems.

Signature · 1 line
pozsy
#11 lore raw
Quoted 20 lines · Pozsar Balazs
Quoted 19 lines · Greg KH
Quoted 17 lines · Pozsar Balazs
Quoted 6 lines · Greg KH
Quoted 5 lines · Marco d'Itri
Quoted 3 lines · Greg KH

This seems to be a Debian issue for some odd reason, I suggest filing a bug against the udev package (or just tagging onto the existing bug for this problem, I've seen it in there already...)

Quoted 7 lines
Quoted 3 lines

The reason this is usually seen only on Debian systems is that I am the first one who shipped an udev package which runs many parallel modprobe commands, but this is a genuine kernel/modprobe bug.

I'm pretty sure OpenSuSE 10.0 does the same thing, and I don't think anyone has reported the same kind of bugs there. Makes me wonder what is really happening here...

If module A depends on module B, and "modprobe A" and "modprobe B" are run parallel

Quoted 6 lines
Quoted 2 lines

Why would they be run in parallel? modprobe doesn't do this, why would you?

On my machine it happened while loading pcmcia modules. It was two modprobes running in parallel (invoked by udev), not 1 modprobe loading modules in parallel.

Ah, ok, that explains how it is happening, thanks.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

, there is time window when module B is already listed in /proc/modules, but not completely loaded/initialized, it is in the state "Loading". At this point "modprobe A" checks /proc/modules if module B is already loaded, but it does not take into account that it is in the state "Loading" and not yet "Live". So it tries to load module A, but it fails, because there are missing symbols because module A did not register them yet.

Quoted 5 lines
Quoted 3 lines

Sounds like a locking issue within the module core. I thought we could only load one at a time, otherwise we have other races within the kernel.

That's what I thought too, but this is not the case it seems.

Um, why not? Rusty?

thanks,

greg k-h

#13 lore raw
In reply to a message not in the archive
Quoted 4 lines · Rusty Russell

The current simple fix for that (thanks Pozsar!) is to poll while a module we rely on is still loading as indicated in /proc/modules. This fix will be needed to cover existing kernels, even if we were to get fancy in new kernels.

I have *not* been able to verify this, but at least two people still reported problems after using this patch.

Signature · 2 lines
ciao,
Marco
#4 lore raw
In reply to Gustavo Guillermo Pérez
Quoted 13 lines · Gustavo Guillermo Pérez
Quoted 11 lines · El Sábado, 5 de Noviembre de 2005 23:59, escribió:
Quoted 8 lines · Greg KH
Quoted 5 lines · Harald Dunkel

Hi folks,

I can't say since when this problem is in, but currently I get error messages about unknown symbols at boot time (after mounting the root disk, as it seems):

Are you using Debian?

Of course :=)

Quoted 7 lines

I was having that problem using busybox insmod, changing to latest kernel tools was fixed for me, I don't know why, but in my case was a ramdisk to load sata drivers before mounting the root disk. Anyway module-init-tools is not too bigger even building them with glibc instead uClibc as static binaries.

:)

Which version of busybox? (If something's wrong with it, I'm interested in fixing it. Insmod underwent a lot of changes to support 2.6, and it took a while to stabilize again...)

Rob