log-buf-len dynamic
Subsystems (9)
Hi,
I'm rejecting on the log-buf-len feature in 2.4.23pre5, the code in mainline is worthless for any distributor, shipping another rpm package just for the bufsize would be way overkill.
Please backout the below (extracted from bkcvs) and apply this one instead:
http://www.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.22aa1/00_log-buf-len-1
BTW, not sure why but bkcvs on kernel.org is still stuck at 23pre4, I hope it's not a malfunction and just a normal delay. For now I've to use a mixture between bkcvs and patches.
---------------------
PatchSet 3787
Date: 2003/08/26 19:28:13
Author: willy
Branch: HEAD
Tag: (none)
Log:
[PATCH] make log buffer length selectable
Quoted 3 lines · Marcelo Tosatti
final:
- 2.4.22-rc4 was released as 2.4.22 with no changes.
Hi Marcelo,
as you requested, here is the log_buf_len patch for inclusion in 23-pre.
Cheers,
Willy
BKrev: 3f4ba6bdLGpAZvSSSz7LVia3xLNP6g
Members:
ChangeSet:1.3787->1.3788 Documentation/Configure.help:1.220->1.221 arch/i386/config.in:1.47->1.48 kernel/printk.c:1.12->1.13
Index: linux-2.4/Documentation/Configure.help
Documentation/Configure.help renamed from Documentation/Configure.help:1.220 +13 -0
@@ -25102,6 +25102,19 @@ output to the second serial port on these devices. Saying N will cause the debug messages to appear on the first serial port. +Kernel log buffer length shift +CONFIG_LOG_BUF_SHIFT + The kernel log buffer has a fixed size of : + 64 kB (2^16) on MULTIQUAD and IA64, + 128 kB (2^17) on S390 + 32 kB (2^15) on SMP systems + 16 kB (2^14) on UP systems + + You have the ability to change this size with this paramter which + fixes the bit shift of to get the buffer length (which must be a + power of 2). Eg: a value of 16 sets the buffer to 64 kB (2^16). + The default value of 0 uses standard values above. + Disable pgtable cache CONFIG_NO_PGT_CACHE Normally the kernel maintains a `quicklist' of preallocated
Index: linux-2.4/arch/i386/config.in
arch/i386/config.in renamed from arch/i386/config.in:1.47 +2 -0
@@ -477,6 +477,8 @@ bool ' Compile the kernel with frame pointers' CONFIG_FRAME_POINTER fi +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/kernel/printk.c
kernel/printk.c renamed from kernel/printk.c:1.12 +4 -0
@@ -29,6 +29,7 @@#include <asm/uaccess.h> +#if !defined(CONFIG_LOG_BUF_SHIFT ) || (CONFIG_LOG_BUF_SHIFT -0 ==0 )#if defined(CONFIG_MULTIQUAD ) || defined(CONFIG_IA64 )#define LOG_BUF_LEN (65536)#elif defined(CONFIG_ARCH_S390 )
@@ -37,6 +38,9 @@#define LOG_BUF_LEN (32768)#else #define LOG_BUF_LEN (16384)/* This must be a power of two */ +#endif +#else /* CONFIG_LOG_BUF_SHIFT */ +#define LOG_BUF_LEN (1 << CONFIG_LOG_BUF_SHIFT)#endif #define LOG_BUF_MASK (LOG_BUF_LEN-1)
---------------------
PatchSet 3862
Date: 2003/08/30 15:32:03
Author: willy
Branch: HEAD
Tag: (none)
Log:
[PATCH] Fix log buffer length issues
Quoted 1 line · Marcelo Tosatti
I'm waiting for your selectable log buffer lengh fixes :)
OK, Marcelo, here comes my proposed fix. Tom, does it look better for you ? BTW, Tom, I found the exact reason for the line which irritated you :
#if !defined(CONFIG_LOG_BUF_SHIFT) || (CONFIG_LOG_BUF_SHIFT - 0 == 0)
it was because because I was worried about CONFIG_LOG_BUG_SHIFT being defined to the empty string (a simple #define CONFIG_LOG_BUG_SHIFT is enough), as I remember it happened to me when working on it. But there's no reason here for this to happen, otherwise we would have far worse problems with values extracted from .config, so I fixed it.
I added the config line to all archs. I used copy paste, so either I broke them all, either none. I'll start a compilation on my alpha while I'm on the way home, to check. I you have questions, please do ask.
Cheers,
Willy
BKrev: 3f50b563j2T0yJ7uwm8xVJWuE9qDnQ
Members:
ChangeSet:1.3862->1.3863 arch/alpha/config.in:1.28->1.29 arch/arm/config.in:1.24->1.25 arch/cris/config.in:1.18->1.19 arch/ia64/config.in:1.21->1.22 arch/m68k/config.in:1.21->1.22 arch/mips/config-shared.in:1.7->1.8 arch/parisc/config.in:1.10->1.11 arch/ppc/config.in:1.37->1.38 arch/ppc64/config.in:1.11->1.12 arch/s390/config.in:1.13->1.14 arch/s390x/config.in:1.13->1.14 arch/sh/config.in:1.14->1.15 arch/sh64/config.in:1.7->1.8 arch/sparc/config.in:1.19->1.20 arch/sparc64/config.in:1.30->1.31 arch/x86_64/config.in:1.10->1.11 kernel/printk.c:1.13->1.14
Index: linux-2.4/arch/alpha/config.in
arch/alpha/config.in renamed from arch/alpha/config.in:1.28 +2 -0
@@ -454,6 +454,8 @@ define_tristate CONFIG_MATHEMU y fi +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/arch/arm/config.in
arch/arm/config.in renamed from arch/arm/config.in:1.24 +3 -0
@@ -730,6 +730,9 @@ dep_bool ' Kernel low-level debugging functions' CONFIG_DEBUG_LL $CONFIG_DEBUG_KERNEL dep_bool ' Kernel low-level debugging messages via footbridge serial port' CONFIG_DEBUG_DC21285_PORT $CONFIG_DEBUG_LL $CONFIG_FOOTBRIDGE dep_bool ' Kernel low-level debugging messages via UART2' CONFIG_DEBUG_CLPS711X_UART2 $CONFIG_DEBUG_LL $CONFIG_ARCH_CLPS711X + +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/arch/cris/config.in
arch/cris/config.in renamed from arch/cris/config.in:1.18 +2 -0
@@ -272,6 +272,8 @@ int ' Profile shift count' CONFIG_PROFILE_SHIFT 2 fi +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + source crypto/Config.in source lib/Config.in endmenu
Index: linux-2.4/arch/ia64/config.in
arch/ia64/config.in renamed from arch/ia64/config.in:1.21 +2 -0
@@ -294,4 +294,6 @@ bool ' Turn on irq debug checks (slow!)' CONFIG_IA64_DEBUG_IRQ fi +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu
Index: linux-2.4/arch/m68k/config.in
arch/m68k/config.in renamed from arch/m68k/config.in:1.21 +2 -0
@@ -556,6 +556,8 @@ bool ' Verbose BUG() reporting' CONFIG_DEBUG_BUGVERBOSE fi +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/arch/mips/config-shared.in
arch/mips/config-shared.in renamed from arch/mips/config-shared.in:1.7 +3 -0
@@ -1042,6 +1042,9 @@ else int 'Maximum number of CPUs (2-32)' CONFIG_NR_CPUS 32 fi + +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/arch/parisc/config.in
arch/parisc/config.in renamed from arch/parisc/config.in:1.10 +3 -0
@@ -198,6 +198,9 @@ #bool 'Debug kmalloc/kfree' CONFIG_DEBUG_MALLOC bool 'Magic SysRq key' CONFIG_MAGIC_SYSRQ + +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/arch/ppc/config.in
arch/ppc/config.in renamed from arch/ppc/config.in:1.37 +3 -0
@@ -563,4 +563,7 @@ if [ "$CONFIG_GEN550" = "y" ]; then bool 'Support for early boot texts over serial port' CONFIG_SERIAL_TEXT_DEBUG fi + +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu
Index: linux-2.4/arch/ppc64/config.in
arch/ppc64/config.in renamed from arch/ppc64/config.in:1.11 +3 -0
@@ -263,4 +263,7 @@ dep_bool ' LKCD RLE compression' CONFIG_DUMP_COMPRESS_RLE $CONFIG_DUMP dep_bool ' LKCD GZIP compression' CONFIG_DUMP_COMPRESS_GZIP $CONFIG_DUMP fi + +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu
Index: linux-2.4/arch/s390/config.in
arch/s390/config.in renamed from arch/s390/config.in:1.13 +3 -0
@@ -80,6 +80,9 @@ # bool 'Remote GDB kernel debugging' CONFIG_REMOTE_DEBUG #fi bool 'Magic SysRq key' CONFIG_MAGIC_SYSRQ + +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/arch/s390x/config.in
arch/s390x/config.in renamed from arch/s390x/config.in:1.13 +3 -0
@@ -84,6 +84,9 @@ # bool 'Remote GDB kernel debugging' CONFIG_REMOTE_DEBUG #fi bool 'Magic SysRq key' CONFIG_MAGIC_SYSRQ + +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/arch/sh/config.in
arch/sh/config.in renamed from arch/sh/config.in:1.14 +3 -0
@@ -465,6 +465,9 @@ 8 CONFIG_KGDB_DEFBITS_8" 8 endmenu fi + +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/arch/sh64/config.in
arch/sh64/config.in renamed from arch/sh64/config.in:1.7 +2 -0
@@ -301,6 +301,8 @@ bool "Debug: audit page tables on return from syscall/exception/interrupt" CONFIG_SH64_PAGE_TABLE_AUDIT dep_bool "Debug: report TLB fill/purge activity through /proc/tlb" CONFIG_SH64_PROC_TLB $CONFIG_PROC_FS +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source lib/Config.in
Index: linux-2.4/arch/sparc/config.in
arch/sparc/config.in renamed from arch/sparc/config.in:1.19 +2 -0
@@ -277,6 +277,8 @@ bool ' Spinlock debugging' CONFIG_DEBUG_SPINLOCK fi +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/arch/sparc64/config.in
arch/sparc64/config.in renamed from arch/sparc64/config.in:1.30 +2 -0
@@ -311,6 +311,8 @@ define_bool CONFIG_MCOUNT y fi +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source crypto/Config.in
Index: linux-2.4/arch/x86_64/config.in
arch/x86_64/config.in renamed from arch/x86_64/config.in:1.10 +3 -0
@@ -245,6 +245,9 @@ bool ' IOMMU leak tracing' CONFIG_IOMMU_LEAK bool ' Probalistic stack overflow check' CONFIG_DEBUG_STACKOVERFLOW fi + +int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0 + endmenu source lib/Config.in
Index: linux-2.4/kernel/printk.c
kernel/printk.c renamed from kernel/printk.c:1.13 +1 -1
@@ -29,7 +29,7 @@#include <asm/uaccess.h> -#if !defined(CONFIG_LOG_BUF_SHIFT ) || (CONFIG_LOG_BUF_SHIFT -0 ==0 ) +#if !defined(CONFIG_LOG_BUF_SHIFT ) || (CONFIG_LOG_BUF_SHIFT ==0 )#if defined(CONFIG_MULTIQUAD ) || defined(CONFIG_IA64 )#define LOG_BUF_LEN (65536)#elif defined(CONFIG_ARCH_S390 )
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
Quoted 2 lines · Andrea Arcangeli
Date: 2003/08/26 19:28:13
Author: willy
Must be a different willy.
Quoted 3 lines
I added the config line to all archs. I used copy paste, so either I broke them all, either none. I'll start a compilation on my alpha while I'm on the way home, to check. I you have questions, please do ask.
... because I don't have an Alpha ;-)
Signature · 3 lines
"It's not Hollywood. War is real, war is primarily not about defeat or victory, it is about death. I've seen thousands and thousands of dead bodies. Do you think I want to have an academic debate on this subject?" -- Robert Fisk
Quoted 11 lines · Matthew Wilcox
Quoted 2 lines · Andrea Arcangeli
Date: 2003/08/26 19:28:13
Author: willyMust be a different willy.
Quoted 3 lines
I added the config line to all archs. I used copy paste, so either I broke them all, either none. I'll start a compilation on my alpha while I'm on the way home, to check. I you have questions, please do ask.
... because I don't have an Alpha ;-)
;)
if Marcelo would be using open source code to exports the patchsets in his tree, we could fix it to add the email address along the name in the checkin logs metadata, to avoid this sort of mistakes.
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
Hi,
(it was from me)
Quoted 10 lines · Andrea Arcangeli
Hi,
I'm rejecting on the log-buf-len feature in 2.4.23pre5, the code in mainline is worthless for any distributor, shipping another rpm package just for the bufsize would be way overkill.
Please backout the below (extracted from bkcvs) and apply this one instead:
http://www.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.22aa1/00_log-buf-len-1
Well, Andrea, I've looked at your patch. I really like the dynamic size reconfiguration, but:
- now it becomes mandatory to add a command line option, just for this. It's annoying when you want to build install images for many systems. I've always considered that command line parameters should be limited to the strict minimum to have a system boot reliably (root=, console=, very few IDE/SCSI/ACPI tuning when absolutely needed) and that's all.
- what does the initial __log_buf[] become after log_buf_len_setup() ? can these 64 kB be freed or are they definitely lost ?
I think that perhaps we should merge the two things, but reconfigure them differently :
- be able to specify de DEFAULT buffer size at compile time.
- have it reconfigurable at run time with a sysctl (this could be something next to 'prink', or even a write to kmsg). This way, if you detect that your system is still loosing messages under load, you have a chance to catch them all.
- initialize the buffer with allocated memory from the beginning so that we can free it when changing the buffer size.
I can spend a few hours working with you on this if you're interested. But be assured that I know enough people who would complain about being forced to add a new boot option to their lilo.conf.
Cheers,
Willy
Quoted 37 lines · Willy Tarreau
Hi,
(it was from me)
Quoted 10 lines · Andrea Arcangeli
Hi,
I'm rejecting on the log-buf-len feature in 2.4.23pre5, the code in mainline is worthless for any distributor, shipping another rpm package just for the bufsize would be way overkill.
Please backout the below (extracted from bkcvs) and apply this one instead:
http://www.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.22aa1/00_log-buf-len-1Well, Andrea, I've looked at your patch. I really like the dynamic size reconfiguration, but:
- now it becomes mandatory to add a command line option, just for this. It's annoying when you want to build install images for many systems. I've always considered that command line parameters should be limited to the strict minimum to have a system boot reliably (root=, console=, very few IDE/SCSI/ACPI tuning when absolutely needed) and that's all.
- what does the initial __log_buf[] become afterlog_buf_len_setup()? can these 64 kB be freed or are they definitely lost ?I think that perhaps we should merge the two things, but reconfigure them differently :
- be able to specify de DEFAULT buffer size at compile time.
- have it reconfigurable at run time with a sysctl (this could be something next to 'prink', or even a write to kmsg). This way, if you detect that your system is still loosing messages under load, you have a chance to catch them all.
- initialize the buffer with allocated memory from the beginning so that we can free it when changing the buffer size.
Quoted 3 lines
I can spend a few hours working with you on this if you're interested. But be assured that I know enough people who would complain about being forced to add a new boot option to their lilo.conf.
The point here is that the default must work for 99% of the userbase. Either that or the default is totally broken.
So the rest of 1% should be ok to add the command line, and they should be very happy that when they overflow again even with 128k because their cpu is too slow to keep up in klogd or whatever, they can change it to 256k without replacing kernels, some embedded platforms especially should like it. And only this 1% would be the one recompiling the kernel by themself anyways, and if they can recompile the kernel they can as well edit the defaults in kernel/printk.c without pain.
I don't buy much the lazyness argument in changing lilo.conf, the people who need this feature is a marginal part so they must be ok with the parmeter. And don't tell me that you don't pass root= to the kernel at boot. Do you want to fix that too? I do pass plenty of argumetns all the time, starting from profile=0 (and often acpi=off).
however I won't complain if you put the compile time configurator on top of my patch (that's easy) but personaly I think it's not needed, and having it dyanmic is an order of magnitude more important than having it static (again: if you can afford to recompile the kernel than you could edit kernel/printk.c in the first place without much slowdown).
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
Hi Andrea,
Quoted 2 lines
The point here is that the default must work for 99% of the userbase. Either that or the default is totally broken.
You're only thinking about what distro vendors should do. If all kernels should suit their needs, then there would not be any config anymore, and everyone would have the same kernel with the same wagon of command line arguments.
Lots of people actually do recompile their kernel to change a few options from the vendor config, and that's the expected behaviour. And be it 1%, I consider that 1% of Linux users are *lots* of people.
Quoted 3 lines
And only this 1% would be the one recompiling the kernel by themself anyways, and if they can recompile the kernel they can as well edit the defaults in
kernel/printk.cwithout pain.
I don't agree. That's what I did for a long time, and finally got bored of it. Then I wrote a patch and proposed it. Randy Dunlap had one included in 2.6, and Marcelo thought this one would be useful to lots of people, *exactly* because it avoids what you describe here, ie. changins constants by hand.
Quoted 5 lines
I don't buy much the lazyness argument in changing lilo.conf, the people who need this feature is a marginal part so they must be ok with the parmeter. And don't tell me that you don't pass root= to the kernel at boot. Do you want to fix that too? I do pass plenty of argumetns all the time, starting from profile=0 (and often acpi=off).
sorry, I don't agree with you on this. My lilo.conf contains "root=/dev/root" and "boot=/dev/mbr" on nearly all my systems, just because it's easier ton maintain, especially those which are patched remotely. You are speaking about a developer's system, and I too have lots of crap on my desktop machines, and I too am regularly annoyed because lilo tells me that 19 boot images is the max. But on production systems, you need to avoid tricks, and even more when other people manage the system after you install it.
Quoted 4 lines
however I won't complain if you put the compile time configurator on top of my patch (that's easy) but personaly I think it's not needed, and having it dyanmic is an order of magnitude more important than having it static
I said I agree on the dynamic advantage. BTW, you didn't reply me about what the statically allocated 64 kB became with your patch. Will this memory be wasted forever or is there a way to free it ? If I want to pre-allocate it dynamically with alloc_space() as you did, what can I use to free it ? is kfree() the right thing ?
Cheers,
Willy
Quoted 8 lines · Willy Tarreau
Hi Andrea,
Quoted 2 lines
The point here is that the default must work for 99% of the userbase. Either that or the default is totally broken.
You're only thinking about what distro vendors should do. If all kernels should suit their needs, then there would not be any config anymore, and everyone would have the same kernel with the same wagon of command line arguments.
this is actually not true. Many options are way too costly to recompile every time, or to keep enabled at runtime, it can't be the case for this one. My own self compiled kernel has a very stripped down .config, it's not like a distro kernel with everything included.
Quoted 2 lines
I said I agree on the dynamic advantage. BTW, you didn't reply me about what the statically allocated 64 kB became with your patch. Will this memory be
the 64k are wasted only when you use the option, if you need more than 64k it's unlikely you care about 64k lost. But I'm ok to fix it, it just looks a low priority to fix. Also since you will never use this option anyways since you don't like adding parameters to the kernel, then you don't care about fixing it either, you simply want the additional config option on top of this.
My whole argument is that being able to do it dynamic is an order of magnitude more important than being able to do it statically, no matter the command line argument and no matter ther 64k lost. You must agree with that. the amount of userbase is just not comparable.
Quoted 3 lines
wasted forever or is there a way to free it ? If I want to pre-allocate it dynamically with
alloc_space()as you did, what can I use to free it ? iskfree()the right thing ?
we should simply allocate it from the bootmem pool, it's trivial to change it so the 64k aren't lost.
Not sure what I should do, personally I consider the additional .config option as configuration pollution, but since you care that much I can also change my patch not to reject yours, and to allow the two things to coexist, but I don't think it's really needed. I believe people should be forced to use the kernel params, and it's much more manageable to know all the kernels behave the same if no param is passed (otherwise you have to check the System.map to see how big the buffer is, to know if this was the right or wrong kernel during your remote maintainance, either that or you should check /proc/config.gz if you have it). It's just a mess if each kernel with the same revision number behaves differently, everything becomes less and less predictable. The number of System.map checks increases and increases before I can identify what kernel is that. Is there anybody else that has opinions on the matter?
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
Andrea,
since you ask...
I consider myself a not very typical Linux user in that I am a programmer too and I (sometimes) write programs for Linux as well. And yes, I do compile my own kernels.
I think it is pretty senseless to have a configuation option for the default size of that buffer. Especially if I can, in one of the first rc scripts, do something like 'echo 128 > /proc/sys/kernel/printkbuffer'. The only hard requirement for that default size is that all output up till that rc script fits into a buffer of that size.
And yes, I do care about that static 64K buffer. I still have an old pentium that only has 16M. Every byte counts!
On the other hand there are the really large systems with many CPUs and such. Those might have problems with the small default. But, well, those really could #ifdef a different default based on some configuration options...
-- Jan Evert
Quoted 68 lines · Andrea Arcangeli
Quoted 9 lines · Willy Tarreau
Hi Andrea,
Quoted 2 lines
The point here is that the default must work for 99% of the userbase. Either that or the default is totally broken.
You're only thinking about what distro vendors should do. If all kernels should suit their needs, then there would not be any config anymore, and everyone would have the same kernel with the same wagon of command line arguments.
this is actually not true. Many options are way too costly to recompile every time, or to keep enabled at runtime, it can't be the case for this one. My own self compiled kernel has a very stripped down .config, it's not like a distro kernel with everything included.
Quoted 2 lines
I said I agree on the dynamic advantage. BTW, you didn't reply me about what the statically allocated 64 kB became with your patch. Will this memory be
the 64k are wasted only when you use the option, if you need more than 64k it's unlikely you care about 64k lost. But I'm ok to fix it, it just looks a low priority to fix. Also since you will never use this option anyways since you don't like adding parameters to the kernel, then you don't care about fixing it either, you simply want the additional config option on top of this.
My whole argument is that being able to do it dynamic is an order of magnitude more important than being able to do it statically, no matter the command line argument and no matter ther 64k lost. You must agree with that. the amount of userbase is just not comparable.
Quoted 3 lines
wasted forever or is there a way to free it ? If I want to pre-allocate it dynamically with
alloc_space()as you did, what can I use to free it ? iskfree()the right thing ?we should simply allocate it from the bootmem pool, it's trivial to change it so the 64k aren't lost.
Not sure what I should do, personally I consider the additional .config option as configuration pollution, but since you care that much I can also change my patch not to reject yours, and to allow the two things to coexist, but I don't think it's really needed. I believe people should be forced to use the kernel params, and it's much more manageable to know all the kernels behave the same if no param is passed (otherwise you have to check the System.map to see how big the buffer is, to know if this was the right or wrong kernel during your remote maintainance, either that or you should check
/proc/config.gzif you have it). It's just a mess if each kernel with the same revision number behaves differently, everything becomes less and less predictable. The number of System.map checks increases and increases before I can identify what kernel is that. Is there anybody else that has opinions on the matter?Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 10 lines · Jan Evert van Grootheest
Andrea,
since you ask...
I consider myself a not very typical Linux user in that I am a programmer too and I (sometimes) write programs for Linux as well. And yes, I do compile my own kernels.I think it is pretty senseless to have a configuation option for the default size of that buffer. Especially if I can, in one of the first rc scripts, do something like 'echo 128 >
/proc/sys/kernel/printkbuffer'.
having a sysctl can be an additional option (though it's tricky to implement due the needed callbacks), but the problem I guess is that most people needs a larger buf for the boot logs, so having only the sysctl would be too late...
Quoted 5 lines
The only hard requirement for that default size is that all output up till that rc script fits into a buffer of that size.
And yes, I do care about that static 64K buffer. I still have an old pentium that only has 16M. Every byte counts!
Note however that on the old 16M pentium it's unlikely that your boot logs will overflow the current 64k, so for your pentium I doubt there would be any need to use the feature. At least from my point of view the feature is meant mainly for the high end, like 16/32-way HT with 64G of ram etc.. where 64k hardly matters.
Quoted 4 lines
On the other hand there are the really large systems with many CPUs and such. Those might have problems with the small default. But, well, those really could #ifdef a different default based on some configuration options...
note the 64k are only wasted when you use the feature, there's nothing wasted if you don't use it.
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
Quoted 2 lines · Andrea Arcangeli
note the 64k are only wasted when you use the feature, there's nothing wasted if you don't use it.
There's 48k wasted compared to the default 16k, because Jan will not have the choice of not allocating them with your current patch. And I'm not sure he'll be happy to follow your advice, to tap into the source file to get them back :-)
Willy
Quoted 6 lines · Willy Tarreau
Quoted 2 lines · Andrea Arcangeli
note the 64k are only wasted when you use the feature, there's nothing wasted if you don't use it.
There's 48k wasted compared to the default 16k, because Jan will not have the
48k wasted where? What arch, I thought you meant the original log_buf memory is not released, I will fix that, then the waste will be zero and there will be no advantage in having it at compile time anymore, only disavantages.
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
Quoted 9 lines · Andrea Arcangeli
Quoted 3 lines · Jan Evert van Grootheest
I think it is pretty senseless to have a configuation option for the default size of that buffer. Especially if I can, in one of the first rc scripts, do something like 'echo 128 >
/proc/sys/kernel/printkbuffer'.having a sysctl can be an additional option (though it's tricky to implement due the needed callbacks), but the problem I guess is that most people needs a larger buf for the boot logs, so having only the sysctl would be too late...
IMO there are two issues:
1. On some systems it is possible to overflow the log buffer during boot and before init runs
2. On some systems, there is enough going on that klogd cannot read the log quick enough and so the log is missing lines.
IME (1) is the irritating one. I can (and have) edited the source, but it is irritating. Setting up the config option is ok, but for default kernels (e.g. a distro one) where you can't easily recompile, it's insufficient.
Proposal:
1. The default buffer size is huge: 128K or more, which should enable the initial mesages to be kept.
2. The default buffer size is modifiable on the command line, but does not have to be set there (mainly for those with v. limited RAM).
3. The log buffer can be resized smaller and bigger: using sysctl seems nice, but might a dmesg option be ok?
4. klogd and dmesg, at all times, can tell how big the buffer is and where the start and end are. That is, there is no need for the user to tell dmesg how big the buffer is.
5. Lets just do something and move on: it's not important enough to waste weeks talking about :-)
HTH,
Ruth
Signature · 2 lines
Ruth Ivimey-Cook Software engineer and technical writer.
Quoted 30 lines · Ruth Ivimey-Cook
Quoted 8 lines · Andrea Arcangeli
Quoted 1 line · Jan Evert van Grootheest
I think it is pretty senseless to have a configuation option for the default size of that buffer. Especially if I can, in one of the first rc scripts, do something like 'echo 128 >
/proc/sys/kernel/printkbuffer'.having a sysctl can be an additional option (though it's tricky to implement due the needed callbacks), but the problem I guess is that most people needs a larger buf for the boot logs, so having only the sysctl would be too late...
IMO there are two issues:
1. On some systems it is possible to overflow the log buffer during boot and before init runs
2. On some systems, there is enough going on that klogd cannot read the log quick enough and so the log is missing lines.
IME (1) is the irritating one. I can (and have) edited the source, but it is irritating. Setting up the config option is ok, but for default kernels (e.g. a distro one) where you can't easily recompile, it's insufficient.
Proposal:
1. The default buffer size is huge: 128K or more, which should enable the initial mesages to be kept.
2. The default buffer size is modifiable on the command line, but does not have to be set there (mainly for those with v. limited RAM).
3. The log buffer can be resized smaller and bigger: using sysctl seems nice, but might a dmesg option be ok?
Quoted 9 lines
4. klogd and dmesg, at all times, can tell how big the buffer is and where the start and end are. That is, there is no need for the user to tell dmesg how big the buffer is.
5. Lets just do something and move on: it's not important enough to waste weeks talking about :-)
HTH,
Ruth
Hi,
It may be a little over the top for 2.4, but I posted a patch for a dynamically resizable printk a while back:
http://marc.theaimsgroup.com/?l=linux-kernel&m=105828314919697&w=2
I depends on relayfs, which I also posted at the same time.
Basically, the idea is that the static printk buffer is used only at init time for boot messages and is then discarded (so can be made very large), and after that point printk uses the relayfs channel, which is automatically grown and shrunk as necessary. Config options control the buffer sizes (LOG_BUF_SHIFT for initial static, MIN_LOG_BUF_SHIFT for the minimum dynamic) and the relay_open() call controls the maximum dynamic size and other channel characteristics.
Changes I'm currently making to relayfs should simplify the patch significantly and fix some other problems, but I thought I'd point it out as an alternative approach.
Tom
Quoted 6 lines · Tom Zanussi
It may be a little over the top for 2.4, but I posted a patch for a dynamically resizable printk a while back:
http://marc.theaimsgroup.com/?l=linux-kernel&m=105828314919697&w=2
I depends on relayfs, which I also posted at the same time.
I don't think we can merge this in 2.4 but it looks good for 2.6. However note you can't close the overflow issue, you must still have an high limit, but you can make lots more reliable that way, it's similar to netlink anyways, that uses dynamic skb to avoid wasting ram. But it's too invasive for 2.4, just to make a comparison look what my patch looks like (it will get a bit more complicated because of the request not to waste the 64k, you know we throw some hundred megs in the per-cpu freelists so nobody could notice the 64k lost in the 16-way..):
kernel/printk.c +40 -5
@@ -30,13 +30,13 @@#include <asm/uaccess.h> #if defined(CONFIG_MULTIQUAD ) || defined(CONFIG_IA64 ) -#define LOG_BUF_LEN (65536) +#define __LOG_BUF_LEN (65536)#elif defined(CONFIG_ARCH_S390 ) -#define LOG_BUF_LEN (131072) +#define __LOG_BUF_LEN (131072)#elif defined(CONFIG_SMP ) -#define LOG_BUF_LEN (32768) +#define __LOG_BUF_LEN (32768)#else -#define LOG_BUF_LEN (16384)/* This must be a power of two */ +#define __LOG_BUF_LEN (16384)/* This must be a power of two */ #endif #define LOG_BUF_MASK (LOG_BUF_LEN-1)
@@ -78,7 +78,9 @@ struct console *console_drivers;*/ static spinlock_t logbuf_lock =SPIN_LOCK_UNLOCKED ; -static char log_buf[LOG_BUF_LEN ]; +static char __log_buf[__LOG_BUF_LEN]; +static char * log_buf = __log_buf; +static int LOG_BUF_LEN = __LOG_BUF_LEN;#define LOG_BUF (idx) (log_buf[(idx) & LOG_BUF_MASK])/*
@@ -151,6 +153,39 @@ static int __init console_setup(char *st__setup ("console=" , console_setup); +static int __init log_buf_len_setup (char *str) +{ +unsigned long size =simple_strtoul (str, &str,0 ); + +if (size & (size-1 )) +printk ("log_buf_len: invalid size - needs a power of two\n" ); + +spin_lock_irq (&logbuf_lock); +if (size >LOG_BUF_LEN ) { +unsigned long start, end, dest_idx, offset; + +LOG_BUF_LEN = size; + log_buf =alloc_exact (LOG_BUF_LEN ); + + offset = start =min (con_start, log_start); + end = log_end; + dest_idx =0 ; +while (start != end) { + log_buf[dest_idx] = __log_buf[start & (__LOG_BUF_LEN -1 )]; + start++; + dest_idx++; + } + log_start -= offset; + con_start -= offset; + log_end -= offset; + } +spin_unlock_irq (&logbuf_lock); + +return 1 ; +} + +__setup ("log_buf_len=" , log_buf_len_setup); +/* * Commands to do_syslog: *
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
Quoted 2 lines · Andrea Arcangeli
I don't think we can merge this in 2.4 but it looks good for 2.6.
I think so: doing it via a __setup parameter is much smarter than requiring a rebuild.
kernel/printk.c static int __init console_setup(char *st · line 161
Quoted 6 lines
kernel/printk.c +6
-0+static int __init log_buf_len_setup (char *str) +{ +unsigned long size =simple_strtoul (str, &str,0 ); + +if (size & (size-1 )) +printk ("log_buf_len: invalid size - needs a power of two\n" );
You need to either panic or return here.
Quoted 6 lines · Andrew Morton
Quoted 2 lines · Andrea Arcangeli
I don't think we can merge this in 2.4 but it looks good for 2.6.
I think so: doing it via a
__setupparameter is much smarter than requiring a rebuild.
agreed. That's why I also preferred my simple hack. After I fix the 64k memory loss, I will see no disavantages in the kernel boot time parameter compared to the compile time option.
Quoted 9 lines
Review
Quoted 7 lines
(quoted diff) +6
-0+static int __init log_buf_len_setup(char *str) +{ + unsigned long size = simple_strtoul(str, &str, 0); + + if (size & (size-1)) + printk("log_buf_len: invalid size - needs a power of two\n");You need to either panic or return here.
woops sorry, that was the old version ;) it had other bugs too, the new one is this one (Marcelo please don't merge the old version, I thought it was just uptodate in 22aa1 but it was not, the new version wasn't in the ftp site yet infact)
Also I have to fix the request for the 64k memory loss reported by Willy first.
thanks for the review!
kernel/printk.c +46 -4
@@ -27,15 +27,16 @@#include <linux/interrupt.h> /* For in_interrupt() */ #include <linux/config.h> #include <linux/delay.h> +#include <linux/bootmem.h> #include <asm/uaccess.h> #if defined(CONFIG_MULTIQUAD ) || defined(CONFIG_IA64 ) -#define LOG_BUF_LEN (65536) +#define __LOG_BUF_LEN (65536)#elif defined(CONFIG_ARCH_S390 ) || defined(CONFIG_PPC64 ) || defined(CONFIG_PPC ) -#define LOG_BUF_LEN (131072) +#define __LOG_BUF_LEN (131072)#else -#define LOG_BUF_LEN (32768)/* This must be a power of two */ +#define __LOG_BUF_LEN (32768)/* This must be a power of two */ #endif #define LOG_BUF_MASK (LOG_BUF_LEN-1)
@@ -78,7 +79,9 @@ struct console *console_drivers;*/ static spinlock_t logbuf_lock =SPIN_LOCK_UNLOCKED ; -static char log_buf[LOG_BUF_LEN ]; +static char __log_buf[__LOG_BUF_LEN]; +static char * log_buf = __log_buf; +static int LOG_BUF_LEN = __LOG_BUF_LEN;#define LOG_BUF (idx) (log_buf[(idx) & LOG_BUF_MASK])/*
@@ -165,6 +168,45 @@ static int __init console_setup(char *st__setup ("console=" , console_setup); +static int __init log_buf_len_setup (char *str) +{ +unsigned long size =memparse (str, &str); + +if (size >LOG_BUF_LEN ) { +unsigned long start, dest_idx, offset; +char * new_log_buf; + + new_log_buf =alloc_bootmem (size); +if (!new_log_buf) { +printk ("log_buf_len: allocation failed\n" ); + goto out; + } + +spin_lock_irq (&logbuf_lock); +LOG_BUF_LEN = size; + log_buf = new_log_buf; + + offset = start =min (con_start, log_start); + dest_idx =0 ; +while (start != log_end) { + log_buf[dest_idx] = __log_buf[start & (__LOG_BUF_LEN -1 )]; + start++; + dest_idx++; + } + log_start -= offset; + con_start -= offset; + log_end -= offset; +spin_unlock_irq (&logbuf_lock); + +printk ("log_buf_len: %d\n" ,LOG_BUF_LEN ); + } + out: + +return 1 ; +} + +__setup ("log_buf_len=" , log_buf_len_setup); +/* * Commands to do_syslog: *
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
Quoted 2 lines · Andrea Arcangeli
+__setup("log_buf_len=",
log_buf_len_setup);
Specifying this in units of bytes is a pain.
How about we make it "log_buf_kbytes", and multiply it by 1024?
Quoted 7 lines · Andrew Morton
Quoted 2 lines · Andrea Arcangeli
+__setup("log_buf_len=",
log_buf_len_setup);Specifying this in units of bytes is a pain.
How about we make it "
log_buf_kbytes", and multiply it by 1024?
note: that's a memparse, that's not unit of bytes, you can wite 256k or 1M or 2m just now and it'll just work
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
[replying to both Jan and Andrea at once]
-- Jan:
Quoted 5 lines
I think it is pretty senseless to have a configuation option for the default size of that buffer. Especially if I can, in one of the first rc scripts, do something like 'echo 128 >
/proc/sys/kernel/printkbuffer'. The only hard requirement for that default size is that all output up till that rc script fits into a buffer of that size.
I totally agree with this possibility, and it's one I proposed too. I only proposed to keep the default size in order to be able to catch all boot messages, but I think that if we make this buffer hot-resizable (and a sysctl is good for this), then we could systematically pre-allocate 64k at boot which should be enough for boot messages (16k is too short), and could be reduced if needed by a write to the sysctl. This way, the config option can go away.
Quoted 2 lines
And yes, I do care about that static 64K buffer. I still have an old pentium that only has 16M. Every byte counts!
My previous firewall add 8M, and I second your statement : every byte counts !
Quoted 11 lines
Quoted 10 lines · Andrea Arcangeli
Quoted 4 lines
You're only thinking about what distro vendors should do. If all kernels should
suit their needs, then there would not be any config anymore, and everyone would have the same kernel with the same wagon of command line arguments.this is actually not true. Many options are way too costly to recompile every time, or to keep enabled at runtime, it can't be the case for this one. My own self compiled kernel has a very stripped down .config, it's not like a distro kernel with everything included.
I think you're mistaken about what I meant. I didn't mean that I want to recompile each kernel with a different option, I meant that for a large set of identical systems, it's more convenient to have a single, stripped down .config with the most default values, and propagate this same kernel everywhere, than having to duplicate special kernel parameters in every lilo.conf.
Quoted 2 lines
Quoted 2 lines
the 64k are wasted only when you use the option, if you need more than 64k it's unlikely you care about 64k lost.
Absolutely, my concern was most about the case where I need less than 64k and I have too few memory to agree to waste them.
Quoted 5 lines
Quoted 5 lines
But I'm ok to fix it, it just
looks a low priority to fix. Also since you will never use this option anyways since you don't like adding parameters to the kernel, then you don't care about fixing it either, you simply want the additional config option on top of this.
:-)
Andrea, I'm not *that* selfish ! If I did this only for me, I wouldn't even post it here and risk complaints from people who ask me to add other archs too. I can live with this as a patch in my kernel, next to your VM and others. It's just that I felt a need for it on production machines and thought there were potentially other people in my case. In the same way, I could say that you're wrong in believing that everyone wants to add a boot option, but I may be wrong WRT this. That's not the problem. If I asked how to fix it, it's because I'm interested in doing it myself, but I'm clueless about how to do it (don't know how to allocate boot mem, don't know how to free it, don't know if I can reallocate new mem further and free it again by the same means).
Quoted 4 lines
Quoted 4 lines
My whole argument is that being able to do it dynamic is an order of magnitude more important than being able to do it statically, no matter the command line argument and no matter ther 64k lost. You must agree with that. the amount of userbase is just not comparable.
I said I did agree with that, but it's not the same feature. And concerning the amount of userbase, I agree it's not comparable... I don't imagine desktop users adding "logbuflen=1048576" to their boot command line, nor can I imagine them recompiling their kernel changing this option.
Quoted 5 lines
Quoted 5 lines
Not sure what I should do, personally I consider the additional .config option as configuration pollution, but since you care that much I can also change my patch not to reject yours, and to allow the two things to coexist, but I don't think it's really needed. I believe people should be forced to use the kernel params,
^^^^^^^^^^^^^^^^^^^
why do you want to force people to do something the way you think is best ? I personnaly think that both solutions are a convenient way of achieving the same goal for different people. Please let them choose.
Quoted 2 lines
Quoted 2 lines
just a mess if each kernel with the same revision number behaves differently, everything becomes less and less predictable.
nothing new here. There are already many options which will make your kernel behave differently : CPU/ACPI/APM/SCSI/IDE DMA by default/Shutdown Watchdog on close, to name a few... That's why many of us add a version suffix to production kernels identifying a known .config.
Quoted 2 lines
Quoted 2 lines
The number of System.map checks increases and increases before I can identify what kernel is that.
Then don't rely on the default value for your kernels, but only on the boot option, and the problem is gone. Let us all have the choice. I insist that your patch is appealing, but it is not the only solution and does not cover all needs, nor does mine.
Cheers,
Willy
Quoted 1 line · Willy Tarreau
why do you want to force people to do something the way you think is best ?
because it gives you no real disavantage to pass the parameter in grub. You've to set root=/dev/root anyways, this way you would simply need to add the parameter for log buf as well. I've always to add lots of other stuff anyways, including vga= profile= etc..
Quoted 2 lines
I personnaly think that both solutions are a convenient way of achieving the same goal for different people. Please let them choose.
they can already choose with the parameter at boot, it's not that I don't let them choose.
You simply want *two* different APIs, where one is worhtless and obsoleted by the more powerful one.
That's why I think it's useless to have *two* APIs where one overlaps on the other *completely*.
My API is good for everyone, yours is not, I don't see why we've to keep both when nobody depends on yours yet.
Quoted 2 lines
your patch is appealing, but it is not the only solution and does not cover all needs, nor does mine.
I can easily fix the 64k ram waste, after that I can't see why my API can't cover you needs. The init code even gets released because it's __init.
I just don't buy the fact that you don't like to pass params to the kernel because you already do, you have to or it won't boot on a system different than the one that you compiled the kernel.
If you don't like to pass buf_log_len= then how can you be ok to pass root=/dev/root on all your thousand of lilo.conf in production? How can root=/dev/root be remotely acceptable if buf_log_len= can't be as well added to your thousand of distributed lilo.confs for your all images out there? This makes no sense at all to me.
Yeah, I'm forcing you to waste a dozen bytes of disk space in /etc/lilo.conf, you can definitely claim that, but I don't that as an argument either (delete the spaces/tabs and you may save more). And maybe thanks to this you won't run into the number of kernel images limitation anymore.
I'm deinitely against worthless APIs in the kernel, when more powerful one exists, and they give you zero disavantage (I will fix the 64k waste don't worry, I definitely agree that such bit must be fixed to make everyone happy).
As for decreasing the buf size, my option will allow you to decrease it too after I fix it! While the shift only let you increase the buf size, not decrease. So you should prefer the dynamic API too.
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
[Cc: list stripped down]
Quoted 2 lines · Andrea Arcangeli
You simply want *two* different APIs, where one is worhtless and obsoleted by the more powerful one.
I never said I wanted *two* of them. I proposed first a fairly non-intrusive one which doesn't add any line of code, just defines, and managed to reunify architectures which used different values by default.
*You* proposed a second one. I have no objections against it, as I said, since I certainly understand that some people would prefer to use it. But *you* decided that mine was totally unusable and obsoleted by yours, reason for it to be removed. And this is *that* I objected to. Since the can both coexist and fit different people's needs without adding a byte of code (at least for mine), I don't see the goal in eliminating it, deciding unilaterally that nobody will use it.
I repeat, at the end, I don't mind. I have the patch and can live with one more patch, as I did before. It's just the fact that you decide for everyone that always adding a command line option is more convenient than a once-for-all fixed config option.
Quoted 1 line
My API is good for everyone, yours is not
I'm impressed that you know so many people. I know that mine at least satisfies a few collegues, customers, and I. So I deduced that it might be useful to others too. Even Marcelo thought the same a time ago.
Quoted 1 line
I don't see why we've to keep both when nobody depends on yours yet.
Nobody depends on mine, since it doesn't change defaults nor semantics. Perhaps some people depend on yours *not* being in. You changed LOG_BUF_LEN from a define to a static, and THAT could break other external patches. BTW, I've just checked 2.4.22aa1, I noticed that you removed kmsgdump, is it because it doesn't apply, compile or work anymore since your API change ?
Honnestly, I simply took a conservative approach. My proposal not being what will save the world, I'm 100% with you on that. Mine being useless to everyone, I'm only 20% with you on that. Yours being "good for everyone", I'm only 20% with you on that too, and perhaps less since it might break some compatibility. Yours *is* interesting to some aspects, but is not an equivalent.
Quoted 3 lines
I just don't buy the fact that you don't like to pass params to the kernel because you already do, you have to or it won't boot on a system different than the one that you compiled the kernel.
Believe it or not, there are people who don't like to pass params to the kernel. I just told a friend while I was typing this mail, and he replied me "he must be joking !". So at least, we're two on earth.
Quoted 5 lines
Yeah, I'm forcing you to waste a dozen bytes of disk space in
/etc/lilo.conf, you can definitely claim that, but I don't that as an argument either (delete the spaces/tabs and you may save more). And maybe thanks to this you won't run into the number of kernel images limitation anymore.
OK Andrea, you're in a bad mood today. You're trying to prove me crazy. I won't fight. You're the kernel hacker. Not me. I have never been proud of putting a patch in the kernel and never will be. I'm more ashamed when I send an obvious "fix" which breaks someone's setup. Your attitude today seems to be the opposite. Your patch is useful, I won't contest that. It won't annoy anyone, I'll let others report it to you themselves, I've done my part of it. But they will waste their time since you don't want to listen and will try to sell your patch anyway because "it's good for everyone" as you stated it.
Quoted 2 lines
I'm deinitely against worthless APIs in the kernel, when more powerful one exists, and they give you zero disavantage
There are people out there who are against worthless lines in config files, and against APIs which break other patches.
Quoted 3 lines
As for decreasing the buf size, my option will allow you to decrease it too after I fix it! While the shift only let you increase the buf size, not decrease. So you should prefer the dynamic API too.
I don't see why you say that ! Set LOG_BUF_SHIFT to 10 and you'll have 1kB !
Have a good night anyway,
Willy
Quoted 5 lines · Willy Tarreau
Quoted 1 line
My API is good for everyone, yours is not
I'm impressed that you know so many people. I know that mine at least satisfies a few collegues, customers, and I. So I deduced that it might be useful to others too. Even Marcelo thought the same a time ago.
I know your patch was just very good for many people, but not for everyone. I really didn't want to say your patch didn't make any good, I acknowledge it was just very good.
Here I need an argument from you that explains me why should I work further on the current code to make the config option still usable. I see no reason to spend time in that effort because the config option will provide no benefit compared to the kernel parameter.
If you can explain me _why_ you don't want to pass the kernel parameter than you will convince me, if it's just that "you don't want it" then I can't buy that. Give me a valid argument and I will have a reason to retain the config option.
I definitely agree my patch (btw, I posted the last one that had a few bugs too) needs fixing to release the 64k of ram, and to allow a smaller bufsize too (the latter will happen automatically while addressing the former)
The fact you don't want to touch the lilo.conf doesn't sound to me. Especially with lilo (not grub) you've to run lilo anyways every time you replace the kernel, so a simple script adding the parameter in every lilo.conf sounds very easy to provide (you can add it in all kernels, the old ones will ignore it).
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
Quoted 3 lines · Andrea Arcangeli
I know your patch was just very good for many people, but not for everyone. I really didn't want to say your patch didn't make any good, I acknowledge it was just very good.
It's not about it to be good or not, but be useful. There's a difference. When two things are good and do nearly the same, you get the best of them and that's all. When two things are useful and nearly identical, there's a huge potential that one is useful to some people, and the other one to other people. I'm certain I would use your patch on my desktop PC (if it doesn't break kmsgdump as it seems, though), because it will allow me to easily extend my log buffer when I get repetitive oopses. But most probably not on remote systems.
Quoted 4 lines
I definitely agree my patch (btw, I posted the last one that had a few bugs too) needs fixing to release the 64k of ram, and to allow a smaller bufsize too (the latter will happen automatically while addressing the former)
That doesn't matter, I'm fairly certain you will fix these issues, they are details since the patch is at its early stage.
Quoted 5 lines
The fact you don't want to touch the lilo.conf doesn't sound to me. Especially with lilo (not grub) you've to run lilo anyways every time you replace the kernel, so a simple script adding the parameter in every lilo.conf sounds very easy to provide (you can add it in all kernels, the old ones will ignore it).
Yes it's the real reason, and I think that others will second me on this one. You're seeing the benefit of *changing* the value on a kernel compile once. I see the benefit of *deploying* a same kernel to a lot of remote hosts while touching the fewest possible files. Sending a pre-configured kernel and running lilo is already a lot.
I've had private discussions off-list with a few other people who don't agree on the *obligation* to enter a new option on the command line because they will not be able to dynamically generate their boot files anymore and will have to add a config option to them just for this. Believe me, when you remotely and blindly upgrade systems with commands such as :
for host in to_update/*; do update $host && mv $host up_to_date/ ; done
*not* having to touch a boot file is really recomforting. It's not funny to remotely run 'ed' scripts on lilo.conf when you know that there's nobody to replace the box if you fail because one of the file had once been hand-edited and is slightly different. May be you'll say "well, simply prepare all your lilo.conf, check them all then send them". If you think this, it's because you obviously never felt your heart accelerate and beat very loud during these operations and cannot understand why some people prefer to keep these delicate upgrades as trivial as possible. When I said to you that I use root=/dev/root and boot=/dev/mbr on all these hosts, it's exactly not to have to touch these sensible files.
One of us told me that he will have to rebuild his BASE package just to integrate this change, and deploy it to 1200 hosts around the world. When I see how I worry for a few tens, I think he'd better revert your patch and stay on the previous one. Think that some of these people are still running 2.2 because they don't trust 2.4 enough to risk a remote upgrade.
Andrea, I won't blame you for not understanding that some people are afraid of doing certain operations remotely. Perhaps you'll think that I'm totally stupid. That's not a problem for me. I just wanted to let you know that this sort of changes will be a regression for a few of us (mainly the few who've been using the previous patch).
As I told you the first time, if I had the choice for a dynamic buffer resizement, I'd be all for a sysctl so that I wouldn't need to touch any boot file, and that would allow to resize a live system without needing a reboot. Then, simply pre-initialize a 64kB buffer so that no boot message gets lost, and the init scripts adjust the size to what is needed.
I repeat my proposal. I agree to spend a few of my time to work with you in this direction, because I know it is a useful feature. But please don't make it more difficult for people who already have a hard time.
Thanks,
Willy
Quoted 2 lines · Willy Tarreau
because you obviously never felt your heart accelerate and beat very loud during these operations and cannot understand why some people prefer to keep
well, I kind of remeber that ;)
So ok, that sounds a good enough argument to keep the old patch too, despite you may be very few people using this.
But let's do it only in 2.4, 2.6 should have it only dynamic.
clearly if you can try to go 2.6 (possibly using a wrong .config) then you can risk editing the lilo/grub file as well. And with 2.6, with the printk rewrite that shrinks and grows the buffer, you should have no need of the parameter any more.
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk
Quoted 8 lines · Andrea Arcangeli
Quoted 2 lines · Willy Tarreau
because you obviously never felt your heart accelerate and beat very loud during these operations and cannot understand why some people prefer to keep
well, I kind of remeber that ;)
So ok, that sounds a good enough argument to keep the old patch too, despite you may be very few people using this.
Thanks for your understanding ;-)
Quoted 6 lines
But let's do it only in 2.4, 2.6 should have it only dynamic.
clearly if you can try to go 2.6 (possibly using a wrong .config) then you can risk editing the lilo/grub file as well. And with 2.6, with the printk rewrite that shrinks and grows the buffer, you should have no need of the parameter any more.
I have not tried it remotely yet, but I may adapt to a new behaviour or write a patch :-)
Cheers,
Willy
Quoted 1 line · Andrea Arcangeli
+#if !defined(CONFIG_LOG_BUF_SHIFT) || (
CONFIG_LOG_BUF_SHIFT- 0 == 0)
It shouldn't have an empty string value anyway, so this is just:
#if !CONFIG_LOG_BUF_SHIFT
That handles both !defined and !0.
Signature · 2 lines
Daniel Jacobowitz MontaVista Software Debian GNU/Linux Developer
Quoted 5 lines · Andrea Arcangeli
Hi,
I'm rejecting on the log-buf-len feature in 2.4.23pre5, the code in mainline is worthless for any distributor, shipping another rpm package just for the bufsize would be way overkill.
Andrea,
As Willy stated previously this is useful for people who want to change the log buf size without having to change the code manually, and I think this is a useful and non intrusive change.
Do you see any problem with this?
Quoted 8 lines · On Thu, Sep 25, 2003 at 02:40:15PM +0100…
Quoted 5 lines · Andrea Arcangeli
Hi,
I'm rejecting on the log-buf-len feature in 2.4.23pre5, the code in mainline is worthless for any distributor, shipping another rpm package just for the bufsize would be way overkill.
Quoted 7 lines
Andrea,
As Willy stated previously this is useful for people who want to change the log buf size without having to change the code manually, and I think this is a useful and non intrusive change.
Do you see any problem with this?
as wrote in a later email, technically the dynamic solution obsoletes the static one, but only as far as the kernel is concerned.
It happens that to enable the dynamic solution you've to change 1 line to lilo.conf/grub.lst. since 2.4 is already being used in production distributed scenarios, there is at least a psycology value in providing a static solution that doesn't involve lilo.conf changes, so people can deploy the feature without having to edit conf files at all (with grub they don't need to run lilo either).
So I will rewrite my patch in a way that doesn't reject the static setting for next -aa (together with some other tons of pending things, but the vm merge is already completed on my tree, so I'm in sync again with pre5 on that side and that was top priority to fixup to be sure nothing was missing, thanks for merging the watermarks already btw ;).
Andrea - If you prefer relying on open source software, check these links:
rsync.kernel.org::pub/scm/linux/kernel/bkcvs/linux-2.[45]/ http://www.cobite.com/cvsps/ svn://svn.kernel.org/linux-2.[46]/trunk