Skip to contentSkip to conversation

No more module removal -- Unconference track

Ksummit-discuss Theodore Ts'o 11 participants 20 messages Aug 19, 2014 – Aug 27, 2014

lore
Share & tools
lore
#0 lore raw

This has been scheduled at 2pm at the request of Rusty. (Reminder: if you're going to propose a topic, please send e-mail to start a thread on ksummit-discuss).

- Ted

#1 lore raw
Quoted 4 lines · Theodore Ts'o

This has been scheduled at 2pm at the request of Rusty. (Reminder: if you're going to propose a topic, please send e-mail to start a thread on ksummit-discuss).

Do we have a context ? I am using insert/remove module a lot during testing, and would hate to see it go. It also permits module updates without having to reboot the kernel. There must be lots of other reasons to support module removal. So I would really dislike if it was no longer available, and I don't really see the point.

Count me in.

Guenter

#2 lore raw
Quoted 10 lines · Guenter Roeck
Quoted 4 lines · Theodore Ts'o

This has been scheduled at 2pm at the request of Rusty. (Reminder: if you're going to propose a topic, please send e-mail to start a thread on ksummit-discuss).

Do we have a context ? I am using insert/remove module a lot during testing, and would hate to see it go. It also permits module updates without having to reboot the kernel. There must be lots of other reasons to support module removal. So I would really dislike if it was no longer available, and I don't really see the point.

Rusty has been trying to nuke module removal for years.

Unfortunately, it's been incredibly useful for many reasons.

In addition to the reasons you've suggested, it's the only way that I can reset a malfunctioning sound driver without rebooting, and I'd hate to have to regress to windows style "reboot to fix the problem".

It was also noted during the kernel summit core day that because so many people depend on module removal working, the bind and unbind functions are much more reliable, and so kexec should perhaps consider migrating to using bind/unbind.

As a result, what tends to happen is that officially, module unload is not supported, and if the driver is actively in use, it may oops the kernel. However, in practice, this feature is heavily used, which is perhaps why Rusty wants to make another try at removing this feature. :-)

- Ted

#3 lore raw

Why can't we just taint the kernel on module removal?

regards,
dan carpenter

#4 lore raw
Quoted 2 lines · Dan Carpenter

Why can't we just taint the kernel on module removal?

Why not just fix its bugs ?

If you want to taint the kernel on module removal because it is known that many drivers have bugs in their removal code, you should taint the kernel if any code is used which is known to have a bug. Or, in other words, just taint the kernel, period.

Guenter

#5 lore raw
Quoted 9 lines · Guenter Roeck
Quoted 2 lines · Dan Carpenter

Why can't we just taint the kernel on module removal?

Why not just fix its bugs ?

If you want to taint the kernel on module removal because it is known that many drivers have bugs in their removal code, you should taint the kernel if any code is used which is known to have a bug. Or, in other words, just taint the kernel, period.

If you rmmod a module it probably means that your code was buggy before you started the rmmod. We already taint the kernel when the kernel oopses. It's the same thing.

regards,
dan carpenter

#6 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Dan" == Dan Carpenter <[email protected]> writes:

Dan> If you rmmod a module it probably means that your code was buggy Dan> before you started the rmmod. We already taint the kernel when the Dan> kernel oopses. It's the same thing.

Buggy does not imply that the rest of the kernel is left in a mangled state. Also, I load and unload device driver modules many, many times per day. A significant win on large systems that take half an hour to boot.

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
#7 lore raw
In reply to Dan Carpenter
Quoted 6 lines · Dan Carpenter
Quoted 5 lines · Guenter Roeck
Quoted 2 lines · Dan Carpenter

Why can't we just taint the kernel on module removal?

Why not just fix its bugs ?

Quoted 7 lines
Quoted 4 lines

If you want to taint the kernel on module removal because it is known that many drivers have bugs in their removal code, you should taint the kernel if any code is used which is known to have a bug. Or, in other words, just taint the kernel, period.

If you rmmod a module it probably means that your code was buggy before you started the rmmod.

Not necessarily.

I often rmmod the kvm module (and then load a new one) to test performance enhancements.

For tracing, tools like systemtap insert kernel modules to trace kernel code, and will remove them again when the tracing script exits.

#8 lore raw
In reply to Dan Carpenter
Quoted 6 lines · Dan Carpenter
Quoted 5 lines · Guenter Roeck
Quoted 2 lines · Dan Carpenter

Why can't we just taint the kernel on module removal?

Why not just fix its bugs ?

Quoted 9 lines
Quoted 4 lines

If you want to taint the kernel on module removal because it is known that many drivers have bugs in their removal code, you should taint the kernel if any code is used which is known to have a bug. Or, in other words, just taint the kernel, period.

If you rmmod a module it probably means that your code was buggy before you started the rmmod. We already taint the kernel when the kernel oopses. It's the same thing.

I am also unloading modules for testing purposes. And if/when the respective hardware has been removed. It is quite useful with hardware supporting OIR. Sure, it is also used to replace it with a version which does have bug fixes, but that doesn't mean those bugs, if hit, would actually crash the kernel. It may be a stabilization patch, for example, or one with a performacne improvement. But in any case, even if a module is replaced to fix a bug, that doesn't mean it _hit_ that bug. That is quite different to an OOPS, where it is implied that a bug was hit.

Again, if your logic is that module removal should cause the kernel to be tained because it _could_ be that the removal occurred because of a bug, you might as well taint the kernel all the time. If you want module removal functionality removed because it is buggy, you might as well remove the entire kernel because it is buggy.

So, sorry, I don't really understand your logic.

Guenter

#9 lore raw
In reply to Guenter Roeck
Quoted 3 lines · Guenter Roeck

If you want to taint the kernel on module removal because it is known that many drivers have bugs in their removal code,

Most drivers don't have "removal code" per se. They have "unbind" code. Which you can still exercise without actually unloading the driver.

Signature · 1 line
dwmw2
#10 lore raw
Quoted 7 lines · David Woodhouse
Quoted 3 lines · Guenter Roeck

If you want to taint the kernel on module removal because it is known that many drivers have bugs in their removal code,

Most drivers don't have "removal code" per se. They have "unbind" code. Which you can still exercise without actually unloading the driver.

Many platform, i2c, spi drivers have moved to macro generated module removal code that does nothing but unregister the struct device_driver.

g.

#11 lore raw
Quoted 6 lines · Grant Likely
Quoted 5 lines · David Woodhouse
Quoted 3 lines · Guenter Roeck

If you want to taint the kernel on module removal because it is known that many drivers have bugs in their removal code,

Quoted 6 lines
Quoted 2 lines

Most drivers don't have "removal code" per se. They have "unbind" code. Which you can still exercise without actually unloading the driver.

Many platform, i2c, spi drivers have moved to macro generated module removal code that does nothing but unregister the struct device_driver.

That's exactly what I mean. There is no interesting code for unloading the module. There is only the code for unbinding the device from the driver (as in hot-unplug). Which doesn't go away if you disable module unloading. It just gets harder to test.

Signature · 1 line
dwmw2
#12 lore raw
In reply to Theodore Ts'o

At Tue, 19 Aug 2014 11:23:50 -0400,

Quoted 13 lines · Theodore Ts'o
Quoted 10 lines · Guenter Roeck
Quoted 4 lines · Theodore Ts'o

This has been scheduled at 2pm at the request of Rusty. (Reminder: if you're going to propose a topic, please send e-mail to start a thread on ksummit-discuss).

Do we have a context ? I am using insert/remove module a lot during testing, and would hate to see it go. It also permits module updates without having to reboot the kernel. There must be lots of other reasons to support module removal. So I would really dislike if it was no longer available, and I don't really see the point.

Quoted 7 lines

Rusty has been trying to nuke module removal for years.

Unfortunately, it's been incredibly useful for many reasons.

In addition to the reasons you've suggested, it's the only way that I can reset a malfunctioning sound driver without rebooting, and I'd hate to have to regress to windows style "reboot to fix the problem".

Oh, please report such a problem.

Or wait, maybe we shouldn't debug it for keeping one more vote for the election "save module removal" :)

Takashi

#13 lore raw
In reply to Guenter Roeck
Quoted 10 lines · Guenter Roeck
Quoted 4 lines · Theodore Ts'o

This has been scheduled at 2pm at the request of Rusty. (Reminder: if you're going to propose a topic, please send e-mail to start a thread on ksummit-discuss).

Do we have a context ? I am using insert/remove module a lot during testing, and would hate to see it go. It also permits module updates without having to reboot the kernel. There must be lots of other reasons to support module removal. So I would really dislike if it was no longer available, and I don't really see the point.

I have to explain why, since I asked Rusty to improving removing.

What I found is that the module unloading involving 2 stop_machine()s for each module removing. It must not be needed. However, since the module's ref-counter is over-optimized for BIG SMP machine, we can't remove it without replacing it. But it means some performance regression can happen on such big-scale SMP machines (not the laptop nor normal smp machine).
So I asked him to introduce something like lock-up option which locks up the given module, and the kernel skips ref-counting on that module.

Anyway, I sent the series right now :)

https://lkml.org/lkml/2014/8/25/142

Please check it if it is good for your use-cases.

Thank you,

Signature · 4 lines
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]
#14 lore raw
Quoted 6 lines · Masami Hiramatsu

What I found is that the module unloading involving 2 stop_machine()s for each module removing. It must not be needed. However, since the module's ref-counter is over-optimized for BIG SMP machine, we can't remove it without replacing it. But it means some performance regression can happen on such big-scale SMP machines (not the laptop nor normal smp machine).

Is that really a big problem in practice?

I.e. are there valid usecase scenarios where module load / unload should be considered a hotpath where every ms of performance would matter?

Thanks,

Signature · 2 lines
Jiri Kosina
SUSE Labs
#15 lore raw
Quoted 9 lines · Jiri Kosina
Quoted 6 lines · Masami Hiramatsu

What I found is that the module unloading involving 2 stop_machine()s for each module removing. It must not be needed. However, since the module's ref-counter is over-optimized for BIG SMP machine, we can't remove it without replacing it. But it means some performance regression can happen on such big-scale SMP machines (not the laptop nor normal smp machine).

Quoted 4 lines

Is that really a big problem in practice?

I.e. are there valid usecase scenarios where module load / unload should be considered a hotpath where every ms of performance would matter?

There could be. However, in usual usecases, I guess people will not want to unload it. Systemtap or something "additional" module users will need to unload modules. (kpatch could be one of them, but I also think no one want to remove applied patches anyway.)

I just tried to show that the kmodule unload is the one who uses stop_machine heavily but it is not necessary :)

Thank you,

Signature · 4 lines
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]
#16 lore raw
Quoted 7 lines · Masami Hiramatsu

There could be. However, in usual usecases, I guess people will not want to unload it. Systemtap or something "additional" module users will need to unload modules. (kpatch could be one of them, but I also think no one want to remove applied patches anyway.)

I just tried to show that the kmodule unload is the one who uses stop_machine heavily but it is not necessary :)

Right, but if the argument that kmodule unload is slow on huge NUMA machines is that Rusty submits a patch which gets rid of the support for module unload entirely, then that would be very sad. (And **really** screw over systemtap...)

- Ted

#17 lore raw
In reply to Masami Hiramatsu
Quoted 2 lines · Masami Hiramatsu

What I found is that the module unloading involving 2 stop_machine()s for each module removing. It must not be needed.

IIRC, at least one of the stop_machine() calls was to prevent problems between module removal oopsing and the symbol lookup trying to look up whilst dumping the oops.

David

#18 lore raw
Quoted 6 lines · David Howells
Quoted 2 lines

What I found is that the module unloading involving 2 stop_machine()s for each module removing. It must not be needed.

IIRC, at least one of the stop_machine() calls was to prevent problems between module removal oopsing and the symbol lookup trying to look up whilst dumping the oops.

This would be solved if symbol lookup would be RCU-ized though.

Signature · 2 lines
Jiri Kosina
SUSE Labs
#19 lore raw
Quoted 11 lines · Jiri Kosina
Quoted 6 lines · David Howells
Quoted 2 lines

What I found is that the module unloading involving 2 stop_machine()s for each module removing. It must not be needed.

IIRC, at least one of the stop_machine() calls was to prevent problems between module removal oopsing and the symbol lookup trying to look up whilst dumping the oops.

This would be solved if symbol lookup would be RCU-ized though.

Yes, I did that :)

https://lkml.org/lkml/2014/8/25/143
https://lkml.org/lkml/2014/8/25/144
https://lkml.org/lkml/2014/8/25/145

Thank you,

Signature · 4 lines
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]