ATA over Ethernet __init calling __exit
Subsystems (2)
static void __exitaoe_exit(void)
{
...
}
static int __initaoe_init(void)
{
...
aoe_exit();
...
}
Enough said, please shoot the author of that in the foot. You can't call functions marked __exit (which may be discarded) from functions which aren't. If you want to do this, please loose the __exit on aoe_exit().
In addition, please shoot the author in the other foot for:
config ATA_OVER_ETH
tristate "ATA over Ethernet support"
depends on NET
default m <==== this line.
That's not nice for embedded guys who do a "make xxx_defconfig" and unsuspectingly end up with ATA over Ethernet built in for their platform.
Signature · 4 lines
Russell King
Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/
maintainer of: 2.6 PCMCIA - http://pcmcia.arm.linux.org.uk/
2.6 Serial coreQuoted 10 lines · Russell King
In addition, please shoot the author in the other foot for:
config
ATA_OVER_ETH
tristate "ATA over Ethernet support"
depends on NET
default m <==== this line.That's not nice for embedded guys who do a "make
xxx_defconfig" and unsuspectingly end up with ATA over Ethernet built in for their platform.
That annoyed me, too. There's no reason for standard kernel driver modules to assume they should be selected, especially true for something as special case as ata over ethernet.
Signature · 1 line
Jens Axboe
Quoted 6 lines · Russell King
static void
__exitaoe_exit(void)
{
...
}
Quoted 7 lines
static int
__initaoe_init(void)
{
...aoe_exit();...
}
Thanks for catching that. I cleaned up the error handling, too.
Quoted 10 lines
In addition, please shoot the author in the other foot for:
config
ATA_OVER_ETH
tristate "ATA over Ethernet support"
depends on NET
default m <==== this line.That's not nice for embedded guys who do a "make
xxx_defconfig" and unsuspectingly end up with ATA over Ethernet built in for their platform.
OK, thanks. The patch below gets rid of that Kconfig line and fixes the bug where __init code calls __exit code.
It depends on the recently-submitted patch that elimimates sleeping with interrupts off, so it applies cleanly to Greg KH's usb tree.
Don't call __exit functions from __init functions. Default to not configuring AoE support in kernel.
drivers/block/Kconfig +0 -1
@@ -449,7 +449,6 @@ source "drivers/block/Kconfig.iosched" config ATA_OVER_ETH tristate "ATA over Ethernet support" depends on NET - default m help This driver provides Support for ATA over Ethernet block devices like the Coraid EtherDrive (R) Storage Blade.
drivers/block/aoe/aoechr.c +1 -1
@@ -266,7 +266,7 @@ aoechr_init(void)return 0 ; } -void __exit +void aoechr_exit (void ) {int i;
drivers/block/aoe/aoedev.c +1 -1
@@ -159,7 +159,7 @@ aoedev_freedev(struct aoedev *d)kfree (d); } -void __exit +void aoedev_exit (void ) {struct aoedev *d;
drivers/block/aoe/aoemain.c +32 -14
64 lines not on this page · raw message
drivers/block/aoe/aoenet.c +1 -1
@@ -167,7 +167,7 @@ aoenet_init(void)return 0 ; } -void __exit +void aoenet_exit (void ) {dev_remove_pack (&aoe_pt);
Signature · 2 lines
-- Ed L Cashin <[email protected]>
Quoted 8 lines · Jens Axboe
Quoted 7 lines · Russell King
In addition, please shoot the author in the other foot for:
config
ATA_OVER_ETH
tristate "ATA over Ethernet support"
depends on NET
default m <==== this line.
Quoted 7 lines
Quoted 3 lines
That's not nice for embedded guys who do a "make
xxx_defconfig" and unsuspectingly end up with ATA over Ethernet built in for their platform.That annoyed me, too. There's no reason for standard kernel driver modules to assume they should be selected, especially true for something as special case as ata over ethernet.
In general I think it was a bad idea to merge this driver at all. The protocol is obviously broken by design - they use a 16 bit sequence number space which has been proven for many years (in ip fragmentation) to be far too small for modern network performance.
Also the memory allocation without preallocation in the block write out path looks quite broken too and will most likely will lead to deadlocks under high load.
(I wrote a detailed review when it was posted but apparently it disappeared or I never got any answer at least)
IMHO this thing should have never been merged in this form. Can it still be backed out?
-Andi
Andi Kleen <[email protected]> writes:
...
Quoted 4 lines
In general I think it was a bad idea to merge this driver at all. The protocol is obviously broken by design - they use a 16 bit sequence number space which has been proven for many years (in ip fragmentation) to be far too small for modern network performance.
While that experience may apply well to IP, this is a non-IP protocol for a single LAN. For any given AoE device, there are only a few outstanding packets at any given time.
For existing AoE devices that number of outstanding packets is only three! So, with only three packets on the wire at any time for a given device, 16 bits is overkill. In fact, the AoE protocol allows the AoE device to specify how many outstanding packets it supports. That number is only 16 bits wide.
If it ever did become desirable, we could use a couple more bits for the sequence number by borrowing from the low bits of jiffies that we use to estimate the RTT, but it doesn't seem likely to ever be desirable.
Quoted 6 lines
Also the memory allocation without preallocation in the block write out path looks quite broken too and will most likely will lead to deadlocks under high load.
(I wrote a detailed review when it was posted but apparently it disappeared or I never got any answer at least)
I think you're talking about your suggestion that the skb allocation could lead to a deadlock. If I'm correct, this issue is similar to the one that led us to create a mempool for the buf structs we use.
Quoted 2 lines
IMHO this thing should have never been merged in this form. Can it still be backed out?
The sequence number is a non-issue, and for the deadlock danger you originally suggested that we add a warning against using aoe storage for swap. We could add such a warning to the driver's documentation and start working on a patch that avoids deadlocking on the skb allocation. There's no need to panic and force users go get a third party driver.
Signature · 1 line
Ed L Cashin <[email protected]>
Quoted 12 lines · Ed L Cashin
Andi Kleen <[email protected]> writes:
...
Quoted 4 lines
In general I think it was a bad idea to merge this driver at all. The protocol is obviously broken by design - they use a 16 bit sequence number space which has been proven for many years (in ip fragmentation) to be far too small for modern network performance.
While that experience may apply well to IP, this is a non-IP protocol for a single LAN. For any given AoE device, there are only a few outstanding packets at any given time.
Quoted 7 lines
For existing AoE devices that number of outstanding packets is only three! So, with only three packets on the wire at any time for a given device, 16 bits is overkill. In fact, the AoE protocol allows the AoE device to specify how many outstanding packets it supports. That number is only 16 bits wide.
If it ever did become desirable, we could use a couple more bits for
It likely will if someone ever adds significant write cache to such devices.
Quoted 3 lines
the sequence number by borrowing from the low bits of jiffies that we use to estimate the RTT, but it doesn't seem likely to ever be desirable.
Can this be done now?
Quoted 11 lines
Quoted 7 lines
Also the memory allocation without preallocation in the block write out path looks quite broken too and will most likely will lead to deadlocks under high load.
(I wrote a detailed review when it was posted but apparently it disappeared or I never got any answer at least)
I think you're talking about your suggestion that the skb allocation could lead to a deadlock. If I'm correct, this issue is similar to the one that led us to create a mempool for the buf structs we use.
For the skbuffs? I don't think it's possible to preallocate them because the network stack (intentionally) misses hooks to give them back to you.
BTW iirc your submit patch did too much allocations anyways because in modern Linux skbs you can just stick a pointer to the page into the skb when the device announces NETIF_F_SG.
Quoted 7 lines
Quoted 3 lines
IMHO this thing should have never been merged in this form. Can it still be backed out?
The sequence number is a non-issue, and for the deadlock danger you originally suggested that we add a warning against using aoe storage for swap. We could add such a warning to the driver's documentation
It's not only swap, the same problem occurs when any file on such a device is writeable mmaped.
-Andi
Quoted 21 lines · Andi Kleen
Quoted 12 lines · Ed L Cashin
Andi Kleen <[email protected]> writes:
...
Quoted 4 lines
In general I think it was a bad idea to merge this driver at all. The protocol is obviously broken by design - they use a 16 bit sequence number space which has been proven for many years (in ip fragmentation) to be far too small for modern network performance.
While that experience may apply well to IP, this is a non-IP protocol for a single LAN. For any given AoE device, there are only a few outstanding packets at any given time.
Quoted 7 lines
For existing AoE devices that number of outstanding packets is only three! So, with only three packets on the wire at any time for a given device, 16 bits is overkill. In fact, the AoE protocol allows the AoE device to specify how many outstanding packets it supports. That number is only 16 bits wide.
If it ever did become desirable, we could use a couple more bits for
Quoted 8 lines
It likely will if someone ever adds significant write cache to such devices.
Quoted 3 lines
the sequence number by borrowing from the low bits of jiffies that we use to estimate the RTT, but it doesn't seem likely to ever be desirable.
Can this be done now?
It seems rash to make the change now, because there is no need for it.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Also the memory allocation without preallocation in the block write out path looks quite broken too and will most likely will lead to deadlocks under high load.
(I wrote a detailed review when it was posted but apparently it disappeared or I never got any answer at least)
Quoted 7 lines
Quoted 3 lines
I think you're talking about your suggestion that the skb allocation could lead to a deadlock. If I'm correct, this issue is similar to the one that led us to create a mempool for the buf structs we use.
For the skbuffs? I don't think it's possible to preallocate them because the network stack (intentionally) misses hooks to give them back to you.
For the skbuffs, we could use a mempool with GFP_NOIO allocation and then skb_get them before the network layer ever sees them. We already keep track of the packets that we've sent out, so we'll just keep a pointer to the skbuffs.
Quoted 3 lines
BTW iirc your submit patch did too much allocations anyways because in modern Linux skbs you can just stick a pointer to the page into the skb when the device announces
NETIF_F_SG.
I thought there was some shared information at the end of the data, but that's interesting, thanks. I'll look into it.
Signature · 1 line
Ed L Cashin <[email protected]>
Quoted 1 line · Ed L Cashin
It seems rash to make the change now, because there is no need for it.
It would be better to future proof the driver at least a bit.
Quoted 5 lines
For the skbuffs, we could use a mempool with
GFP_NOIOallocation and thenskb_getthem before the network layer ever sees them. We already keep track of the packets that we've sent out, so we'll just keep a pointer to the skbuffs.
That won't work because you don't know when the driver is done with it.
There is a callback, but it's already used by socket buffer management (if you don't use a socket it may be usable)
-Andi
Quoted 7 lines · Andi Kleen
Quoted 1 line · Ed L Cashin
It seems rash to make the change now, because there is no need for it.
It would be better to future proof the driver at least a bit.
Quoted 1 line
Quoted 8 lines
Quoted 4 lines
For the skbuffs, we could use a mempool with
GFP_NOIOallocation and thenskb_getthem before the network layer ever sees them. We already keep track of the packets that we've sent out, so we'll just keep a pointer to the skbuffs.That won't work because you don't know when the driver is done with it.
There is a callback, but it's already used by socket buffer management
The destructor member of struct sk_buff, right?
Quoted 1 line
(if you don't use a socket it may be usable)
You mean a struct sock? No, we don't use that because we're not IP. Thanks for the lead.
Signature · 1 line
Ed L Cashin <[email protected]>
Quoted 4 lines · Andi Kleen
Quoted 1 line
If it ever did become desirable, we could use a couple more bits for
It likely will if someone ever adds significant write cache to such devices.
ATA doesn't support tagged queueing. Therefore write cache is irrelevant.
Alan