Skip to contentSkip to conversation

Flames over -- Re: Which is simpler?

DISCUSSION David Brownell 4 participants 16 messages Feb 13, 2006 – Feb 21, 2006

lore
Share & tools
lore
#0 lore raw

I think "simpler" isn't the issue; "swsusp" is a kind of checkpoint even in its ACPI versions, while standby/str modes are substantially different beasts.

True standby/suspend modes are classic "power management" things:
part of the system stay in low power modes (not necessarily "off") to reduce power consumption, while other parts are in active use. The more deeply they're suspended, the less the system looks like "idle" and the more it looks like "suspend-to-ram".

Quoted 3 lines · Phillip Susi

Thus when suspended to ram,
typically your usb hard drive and almost allways your ide/sata/scsi drive will be completely powered off.

What ide drive? Oh, you're talking about PC-ish systems, not embedded ones that don't _have_ rotating media to power off.

Your experience is very different from mine; I've observed that most PC hardware keeps USB powered in suspend-to-ram states, so a keyboard or mouse action may wake the system up, just as it can with many PS2 style keyboards and mice. Same thing for Ethernet, using various types of wakeup event including "magic packet". See /proc/acpi/wakeup, and the related parts of the ACPI specs. (And USB specs, and lots more ... this info is widespread.)

At the PCI hardware level, that basically means staying in what's called a D3hot state: controller maintains power, and might be a wakeup event source. If you read the EHCI spec you'll notice careful specification of how to maintain USB connections by providing only minimal VBUS current using the Vaux supply ... not all boards go that extra distance for power savings, maintaining USB state (and allowing wakeup) using a lower power D3cold state (with ACPI S3) not D3hot.

(And there's gobs of "legacy" PCI PM hardware still being manufactured, using out-of-band signaling rather than the standardized PCI PM stuff. For that, see the relevant USB controller specs ... e.g. OHCI, Intel UHCI, VIA UHCI.)

For non-PCI hardware, that's actually a system option. Most systems have many low power modes, including ones where it'd make sense to support wakeup from USB and ones where it would not; external transceivers seem to give the most flexibility.

You should also remember a useful point from the ACPI spec:
if you're taking the system apart with a screwdriver, then you've gone out-of-spec. So swapping IDE drives is strongly in the "user error" category ... unlike swapping USB drives, which is equally strongly in the "normal operation" category. (So it would be an error if Linux didn't properly detect when users unplug all usb devices after putting their laptops into STR and before carrying them someplace else...)

Quoted 4 lines

Then your motherboard keeps the bus in a lower power state such that it is capable of causing a wake event when state changes. I'm also fairly sure that when such a wake event happens, there is no indication to the kernel about why it was woken up.

Read about the #PME signal status in the PCI PM capabilities.

And the USB remote wakeup reporting done by USB hubs; you can even look at the drivers/usb/core/hub.c code and see how usb wakeup events (of various types) are handled.

You don't seem to know what you're talking about here.

Quoted 5 lines

As I said before, once woken up,
the kernel must probe all hardware and if there is a new device, it will find it. The hardware only detected that something changed and thus, generated a wake even, it did not notice exactly what that change was and inform the kernel.

You were wrong then too... both about probing "all hardware" and about not being able to distinguish wakeup event sources.

- Dave

#1 lore raw
Quoted 3 lines · David Brownell

What ide drive? Oh, you're talking about PC-ish systems, not embedded ones that don't _have_ rotating media to power off.

Yes, that's exactly what the discussion is about; disk drives with mounted filesystems and what happens to them when you suspend/resume.

Quoted 8 lines

Your experience is very different from mine; I've observed that most PC hardware keeps USB powered in suspend-to-ram states, so a keyboard or mouse action may wake the system up, just as it can with many PS2 style keyboards and mice. Same thing for Ethernet, using various types of wakeup event including "magic packet". See /proc/acpi/wakeup, and the related parts of the ACPI specs. (And USB specs, and lots more ... this info is widespread.)

As I have said before, some systems can keep the USB bus in a low power mode where it can wake the system, but AFAIK, waking the system is all they can do in this state; they can not tell the kernel that device x has been connected and device y has been disconnected, the kernel must figure that out by probing the hardware and comparing the list with what it knew to be connected prior to suspending. Even if some hardware can provide that information, kernel can not rely on it.

<snip>

Quoted 10 lines

You should also remember a useful point from the ACPI spec:
if you're taking the system apart with a screwdriver, then you've gone out-of-spec. So swapping IDE drives is strongly in the "user error" category ... unlike swapping USB drives, which is equally strongly in the "normal operation" category. (So it would be an error if Linux didn't properly detect when users unplug all usb devices after putting their laptops into STR and before carrying them someplace else...)

Yes, it would be an error if the kernel did not notice that you actually did unplug a usb disk while it was suspended. It is just as much an error however, if the kernel erroneously decides that you disconnected the drive, and thus breaks the mount on it, when you in fact, did no such thing.

Quoted 10 lines

Read about the #PME signal status in the PCI PM capabilities.

And the USB remote wakeup reporting done by USB hubs; you can even look at the drivers/usb/core/hub.c code and see how usb wakeup events (of various types) are handled.

You don't seem to know what you're talking about here.

Which is why I qualified my statements with "AFAIK". Maybe you could enlighten me. Does the #PME signal carry enough information to inform the kernel that the reason the system is being woken up is because you unplugged the mouse from the usb hub? I was under the impression that this was not the case, instead the kernel at best knows it is being woken up because the USB host controller generated a wake event. Because of this and the fact that SCSI busses that I have seen do no such thing, I assumed the kernel would handle USB in the same way, specifically, that it would reenumerate and provided the same hardware is still there that it expects to be there, then it would continue operation as normal rather than deciding the drive was unplugged.

Quoted 3 lines

You were wrong then too... both about probing "all hardware" and about not being able to distinguish wakeup event sources.

Are you quite certain about that? This is not the case for SCSI disks, but for USB, maybe it can provide sufficient information to the kernel about state changes without having to do a full rescan. If that is the case, and the hardware is erroneously reporting that all devices were disconnected and reconnected after an ACPI suspend to disk, then such hardware is broken and the kernel should be patched to work around it.

#2 lore raw
Quoted 6 lines · Phillip Susi
Quoted 2 lines · David Brownell

What ide drive? Oh, you're talking about PC-ish systems, not embedded ones that don't _have_ rotating media to power off.

Yes, that's exactly what the discussion is about; disk drives with mounted filesystems and what happens to them when you suspend/resume.

Odd, the original stuff I noticed talked about Linux PM in general, and specifically why STR deserves a heck of a lot more attention than it's been getting so far ... not just systems with rotating media. (Too much attention on rotating media means that really low power systems have been getting a bit shortchanged in Linux.)

Be that as it may:

Quoted 8 lines
Quoted 7 lines

Your experience is very different from mine; I've observed that most PC hardware keeps USB powered in suspend-to-ram states, so a keyboard or mouse action may wake the system up, just as it can with many PS2 style keyboards and mice. Same thing for Ethernet, using various types of wakeup event including "magic packet". See /proc/acpi/wakeup, and the related parts of the ACPI specs. (And USB specs, and lots more ... this info is widespread.)

Quoted 4 lines

As I have said before, some systems can keep the USB bus in a low power mode where it can wake the system, but AFAIK, waking the system is all they can do in this state; they can not tell the kernel that device x has been connected and device y has been disconnected, ...

No, not "AFAIK" ... since when I told you explicitly that was untrue, you then ignored that statement. And didn't look at the specs that I pointed you towards, which provide the details. (USB 2.0 spec re hubs; and of course the Linux-USB hub driver ... www.usb.org)

The events that a hub receives say pretty exactly what happened. You should know that already, since USB behaves that way even when the system is _not_ suspended ...

The full mechanism for USB is more like wakeup signaling on USB triggering hub wakeup (possibly cascading through a few layers of external hub), at some point triggering root hub wakeup, which maps to a PME# signal. That relies on no more than VBUS being powered at a fraction of a milliAmpere, and the equivalent of a pair of voltage comparators triggering wakeup when USB signaling changes from J to K states for something like 10 msec.

Quoted 6 lines
Quoted 6 lines

Read about the #PME signal status in the PCI PM capabilities.

And the USB remote wakeup reporting done by USB hubs; you can even look at the drivers/usb/core/hub.c code and see how usb wakeup events (of various types) are handled.

Quoted 6 lines
Quoted 1 line

You don't seem to know what you're talking about here.

Which is why I qualified my statements with "AFAIK". Maybe you could enlighten me. Does the #PME signal carry enough information to inform the kernel that the reason the system is being woken up is because you unplugged the mouse from the usb hub?

Did you read about the PME# signal in the PCI PM spec? www.pci-sig.com Maybe you could try that.

Also the ACPI spec ... the early chapters give a decent overview of the different components of that model. (ISTR two chapters try that, with the second being more to-the-point despite some duplicated graphics.)

- Dave

#3 lore raw
Quoted 4 lines · David Brownell

No, not "AFAIK" ... since when I told you explicitly that was untrue, you then ignored that statement. And didn't look at the specs that I pointed you towards, which provide the details. (USB 2.0 spec re hubs; and of course the Linux-USB hub driver ... www.usb.org)

I ignored nothing. I fully accepted your explanation as true and pointed out that it changes nothing; data loss in this perfectly valid use case just because the kernel can not be absolutely certain the user did not do something stupid while suspended is unacceptable.

Quoted 5 lines

The events that a hub receives say pretty exactly what happened. You should know that already, since USB behaves that way even when the system is _not_ suspended ...

How it behaves while running and how it behaves while suspended are usually two very different things.

Quoted 9 lines

The full mechanism for USB is more like wakeup signaling on USB triggering hub wakeup (possibly cascading through a few layers of external hub), at some point triggering root hub wakeup, which maps to a PME# signal. That relies on no more than VBUS being powered at a fraction of a milliAmpere, and the equivalent of a pair of voltage comparators triggering wakeup when USB signaling changes from J to K states for something like 10 msec.

Quoted 3 lines

Did you read about the PME# signal in the PCI PM spec? www.pci-sig.com Maybe you could try that.

No, I took your word for it without protest as it doesn't change my main argument: data loss in the face of normal usage is not acceptable. Claiming that it has to be this way because the alternative _might_ result in data loss in the worst (mis)use case is an untenable position.

Quoted 8 lines

Also the ACPI spec ... the early chapters give a decent overview of the different components of that model. (ISTR two chapters try that, with the second being more to-the-point despite some duplicated graphics.)

- Dave

#4 lore raw
Quoted 8 lines · Phillip Susi
Quoted 4 lines · David Brownell

No, not "AFAIK" ... since when I told you explicitly that was untrue, you then ignored that statement. And didn't look at the specs that I pointed you towards, which provide the details. (USB 2.0 spec re hubs; and of course the Linux-USB hub driver ... www.usb.org)

I ignored nothing. I fully accepted your explanation as true and pointed out that it changes nothing;

Sorry, I still can't see a way to read your response to me in that way. When I said "X", you said "AFAIK, X is false". More than once in the same post ... e.g. you say "all hardware must be re-probed", I said "all is wrong" and provided a common counterexample with USB, then you still said "all/AFAIK". And then tried to switch to another topic (see below). I don't have time to waste on such non-dialogue.

Quoted 3 lines

data loss in this perfectly valid
use case just because the kernel can not be absolutely certain the user did not do something stupid while suspended is unacceptable.

Odd, data loss wasn't even mentioned in any of the comments of yours to which I was responding. I was providing corrections to what you were writing about suspend-to-RAM cases.

- Dave

#5 lore raw
In reply to Phillip Susi
Quoted 11 lines

Are you quite certain about that? This is not the case for SCSI disks, but for USB, maybe it can provide sufficient information to the kernel about state changes without having to do a full rescan. If that is the case, and the hardware is erroneously reporting that all devices were disconnected and reconnected after an ACPI suspend to disk, then such hardware is broken and the kernel should be patched to work around it.

No patch was attached...

Signature · 1 line
64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms         
#6 lore raw
Quoted 7 lines · Pavel Machek
Quoted 7 lines

Are you quite certain about that? This is not the case for SCSI disks, but for USB, maybe it can provide sufficient information to the kernel about state changes without having to do a full rescan. If that is the case, and the hardware is erroneously reporting that all

Hardware is CORRECTLY reporting electrical disconnects, but Philip is wanting Linux to ignore those reports.

Quoted 6 lines
Quoted 4 lines

devices were disconnected and reconnected after an ACPI suspend to disk, then such hardware is broken and the kernel should be patched to work around it.

No patch was attached...

No patch possible. Reading the other messages in that thread, Philip is advocating Linux ignore the USB spec. (Which is what _he_ appears to have been doing...)

What he has to do is more than submit a patch. He first needs to lobby the USB-IF to change the USB spec, and get every peripheral vendor to stop shipping USB devices and instead switch over to "Philip-USB". Then get all the billions of USB peripherals to go into the recycle bin and be replaced with products conforming to his new variant. It all seems highly unlikely. ;)

But yes, you're right ... if he's serious about changing all that stuff, he also needs stop being a member of the "never submitted a USB patch" club. Ideally, starting with small things.

#7 lore raw
Quoted 5 lines · David Brownell

Hardware is CORRECTLY reporting electrical disconnects, but Philip is wanting Linux to ignore those reports.

No, not ignore, just realize that an electrical disconnection does not necessarily mean that the volume can no longer be accessed.

Quoted 5 lines

No patch possible. Reading the other messages in that thread, Philip is advocating Linux ignore the USB spec. (Which is what _he_ appears to have been doing...)

Non sequitur. The USB spec does not say the kernel must force unmount the drive.

Quoted 9 lines

What he has to do is more than submit a patch. He first needs to lobby the USB-IF to change the USB spec, and get every peripheral vendor to stop shipping USB devices and instead switch over to "Philip-USB". Then get all the billions of USB peripherals to go into the recycle bin and be replaced with products conforming to his new variant. It all seems highly unlikely. ;)

Quoted 5 lines

But yes, you're right ... if he's serious about changing all that stuff, he also needs stop being a member of the "never submitted a USB patch" club. Ideally, starting with small things.

You're moving off into left field.

#8 lore raw
Quoted 7 lines · Phillip Susi
Quoted 3 lines · David Brownell

Hardware is CORRECTLY reporting electrical disconnects, but Philip is wanting Linux to ignore those reports.

No, not ignore, just realize that an electrical disconnection does not necessarily mean that the volume can no longer be accessed.

Exactly: ignore those disconnects in "some" cases. Suspend-to-RAM will typically never report disconnects without a real unplug. You want to add special casing for hibernate/swsusp. (A point in favor of someone's claim that hibernate/swsusp is structurally harder.)

Now with /dev/input/mice, the driver stack above USB is able to mask such disconnects. It's not like mice maintain state that matters. The "ignore" is in stack layers way above USB, which can know a very important thing about mice: they are stateless.

But with storage media, there is no such mechanism ... and there's significant state involved. Adding a "reconnect" mechanism, and getting it wrong for storage, likely means corrupted file systems. And where even if it _is_ the same physical disk, there's no good reason to expect it hasn't been modified on some other usb host. (Toss hardware in bag, reuse as needed...)

No thanks, I prefer data integrity. And for that matter, re $SUBJECT, the much simpler approach of working _with_ the hardware architecture, not against it.

Quoted 6 lines
Quoted 4 lines

But yes, you're right ... if he's serious about changing all that stuff, he also needs stop being a member of the "never submitted a USB patch" club. Ideally, starting with small things.

You're moving off into left field.

Not hardly. Unless all you're doing here is flaming? One point of $SUBJECT was that flames were _over_ ...

#9 lore raw
Quoted 6 lines · David Brownell

Exactly: ignore those disconnects in "some" cases. Suspend-to-RAM will typically never report disconnects without a real unplug. You want to add special casing for hibernate/swsusp. (A point in favor of someone's claim that hibernate/swsusp is structurally harder.)

Typical != always. It may be more common for suspend to maintain usb power, but both suspend and hibernate may or may not maintain usb power, so the kernel needs to be prepared to deal with that in both cases.

Quoted 6 lines

Now with /dev/input/mice, the driver stack above USB is able to mask such disconnects. It's not like mice maintain state that matters. The "ignore" is in stack layers way above USB, which can know a very important thing about mice: they are stateless.

Quoted 7 lines

But with storage media, there is no such mechanism ... and there's significant state involved. Adding a "reconnect" mechanism, and getting it wrong for storage, likely means corrupted file systems. And where even if it _is_ the same physical disk, there's no good reason to expect it hasn't been modified on some other usb host. (Toss hardware in bag, reuse as needed...)

And this is exactly how non USB hardware has behaved for eons, and it hasn't been a problem. If you want to add some verification to make reasonably sure that the media has not been modified, that is great, but you can't just automatically dump the filesystem and kill running processes and loose data just because something bad _may_ have happened.

Quoted 4 lines

No thanks, I prefer data integrity. And for that matter, re $SUBJECT, the much simpler approach of working _with_ the hardware architecture, not against it.

Again, the hardware is perfectly free to power off the usb bus during suspend to ram. Most systems choose not to because they allow wake from usb, but not all do.

Quoted 10 lines
Quoted 6 lines
Quoted 5 lines

But yes, you're right ... if he's serious about changing all that stuff, he also needs stop being a member of the "never submitted a USB patch" club. Ideally, starting with small things.

You're moving off into left field.

Not hardly. Unless all you're doing here is flaming? One point of $SUBJECT was that flames were _over_ ...

Left field is where flames are, which is what the comment was that I was replying to -- a flame.

#10 lore raw
Quoted 3 lines · Phillip Susi

Again, the hardware is perfectly free to power off the usb bus during suspend to ram. Most systems choose not to because they allow wake from usb, but not all do.

USB has this additional problem that devices lose their addresses when the power is removed (it's very agressively hotplug). So you can have the devices moving around under your feet between poweroff and poweron just because the devices happened to have enumerated in a different order at boot time.

OG.

#11 lore raw
Quoted 6 lines · Olivier Galibert

USB has this additional problem that devices lose their addresses when the power is removed (it's very agressively hotplug). So you can have the devices moving around under your feet between poweroff and poweron just because the devices happened to have enumerated in a different order at boot time.

Isn't that address abstracted out by the usb layer? i.e. there is no relationship between the device number and the usb bus address, so there's no reason the usb layer can't update it's internal data structures to point the old device to the new bus address. Also if the USB host controller wants to, can't it assign any address it likes to each device on the bus? It doesn't HAVE to assign them in sequence as the devices are enumerated does it?

I'm not all that familiar with USB, but I'd imagine it is somewhat like I2C/SMBUS: each device has a descriptor block that contains information about itself. This is going to be unique for any given device. The host controller begins by querying the broadcast address, and all unconfigured devices respond. At some point a bit in their descriptor blocks will differ, and there will be a collision, at which point, the device trying to transmit a high bit will yield and let the others continue. Eventually only one device will be left having sent its entire descriptor block to the host, and the host can then assign a unique address to that device. The host repeats this until all devices have been assigned an address.

Because of this, given the same hardware on the bus, the same enumeration will happen every time, and the host can assign whatever address it wants to each device should it choose to do so rather than just assign them in ascending order. If the host wanted to, it could power down the bus, and when it powers back up, it could assign the same addresses that they had before to the devices as they are enumerated, and new devices would get unused addresses.

#12 lore raw
Quoted 3 lines · Phillip Susi

I'm not all that familiar with USB, but I'd imagine it is somewhat like I2C/SMBUS: each device has a descriptor block that contains information about itself. This is going to be unique for any given device.

That one of the problems, there are no reliable serial numbers or things of the kind. It all becomes heuristics (model name, capacity, partitioning, actual contents...) which kinda suck to have in kernel space.

Quoted 2 lines

The host controller begins by querying the broadcast address, and all unconfigured devices respond.

If I understand USB correctly, it's all point-to-point, there is no broadcast going on. For enumeration purposes hubs are not transparent, and ports are separated. But I wouldn't rely on the ports numbers on hubs to stay constant w.r.t the physical ports. I don't think it's required.

Quoted 4 lines

Because of this, given the same hardware on the bus, the same enumeration will happen every time, and the host can assign whatever address it wants to each device should it choose to do so rather than just assign them in ascending order.

Since it's an hotplug bus, enumeration before suspend happened as the devices were plugged in. So the order a reboot enumeration will see them is unknown in practice.

It may be fixable by storing some kind of physical address in the tree, but losing a filesystem because you replugged the usb drive in the wrong physical port between suspend and resume would be annoying. And I don't know how stable the "physical" positions are in the first place.

OG.

#13 lore raw
Quoted 7 lines · Olivier Galibert

If I understand USB correctly, it's all point-to-point, there is no broadcast going on. For enumeration purposes hubs are not transparent, and ports are separated. But I wouldn't rely on the ports numbers on hubs to stay constant w.r.t the physical ports. I don't think it's required.

Hrm... that last part is crucial I think. If the kernel can tell there is a device still physically connected to the same port after the suspend, then it could reconnect that device to the existing node, and maybe notify the filesystem that the disk may have changed, so it should attempt to verify it, the way windows does.

As for the heuristics, don't most filesystems keep a mount count in the superblock? What better way to decide if things are changed than having the filesystem check that the superblock looks to be the same and specifically that it's mount count has not increased. I don't think the tests need to be that complex to get rather reasonably accurate results, and trying to put them in user space is asking for a lot of race conditions that would be solved by having the filesystem handle it.

Quoted 4 lines

Since it's an hotplug bus, enumeration before suspend happened as the devices were plugged in. So the order a reboot enumeration will see them is unknown in practice.

Ahh, right... I didn't think of that; the bus may have changed since the initial enumeration, so a re-enumeration will be different. I was just thinking of the case where it didn't change while suspended.

Quoted 5 lines

It may be fixable by storing some kind of physical address in the tree, but losing a filesystem because you replugged the usb drive in the wrong physical port between suspend and resume would be annoying. And I don't know how stable the "physical" positions are in the first place.

#14 lore raw
In reply to Phillip Susi
Quoted 3 lines · Phillip Susi

And this is exactly how non USB hardware has behaved for eons, and it hasn't been a problem.

How many billions of years exactly? :)

Of course it sometimes _has_ been a problem. Repeating your claim doesn't make it true. And the user model of USB was certainly so those problems could be _prevented_ rather than continued forever into new generations of hardware.

The fact that MS-DOS did something does not make it a good idea.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

But yes, you're right ... if he's serious about changing all that stuff, he also needs stop being a member of the "never submitted a USB patch" club. Ideally, starting with small things.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

You're moving off into left field.

Not hardly. Unless all you're doing here is flaming? One point of $SUBJECT was that flames were _over_ ...

Left field is where flames are, which is what the comment was that I was replying to -- a flame.

This is LKML. Pointing out when patches are overdue can't realistically be taken as a flame; it's a standard way of moving beyond discussion to action. (Or helping someone self-educate about issues they won't see until they, too, look more deeply ...)

However, responding to a "request for patch" in that way certainly does come across as a flame.

#15 lore raw
Quoted 5 lines · David Brownell
Quoted 1 line · Phillip Susi

And this is exactly how non USB hardware has behaved for eons, and it hasn't been a problem.

How many billions of years exactly? :)

<G>

Quoted 5 lines

Of course it sometimes _has_ been a problem. Repeating your claim doesn't make it true. And the user model of USB was certainly so those problems could be _prevented_ rather than continued forever into new generations of hardware.

But it hasn't been prevented, just changed into a less destructive, but more prevelant problem. If you want to try to solve the problem then it should be solved in such a way that it does not cause other problems ( breaking mounts when you suspend ) and the solution should be generalized to all disks rather than just USB.

Quoted 9 lines

The fact that MS-DOS did something does not make it a good idea.

This is LKML. Pointing out when patches are overdue can't realistically be taken as a flame; it's a standard way of moving beyond discussion to action. (Or helping someone self-educate about issues they won't see until they, too, look more deeply ...)

I think you got the thread confused. The flame was:

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

changing all that stuff, he also needs stop being a member of the "never submitted a USB patch" club.

However, responding to a "request for patch" in that way certainly does come across as a flame.