Skip to contentSkip to conversation

crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does not

DISCUSSION Petr Vandrovec 17 participants 40 messages Jan 31, 2002 – Feb 7, 2002

Subsystems (4)
lore
Share & tools
lore
#0 lore raw
Quoted 7 lines · Jeff Garzik
Quoted 5 lines · Petr Vandrovec

I've got strange idea and tried to build diskless machine around
2.5.3... Besides problem with segfaulting crc32 (it is initialized after net/ipv4/ipconfig.c due to lib/lib.a being a library... I had to hardcode lib/crc32.o before --start-group in main Makefile, but it is another story)

Quoted 5 lines

Would you be willing to cook up a patch for this problem?

I ran into this too. It was solved by setting CONFIG_CRC32=n and letting the Makefile rules pull it in... but lib/lib.a needs to be lib/lib.o really.

Unfortunately during conversion I found that there is lib/bust_spinlocks.c, which is always included in lib.a, is always compiled, even if architecture provides its own bust_spinlocks function.

As no other module in lib/ uses module_init() initalization, it looks to me like that we should move crc32.c from lib/ to kernel/, instead of turning lib.a into lib.o.

But of course if there is consensus that I should convert lib/lib.a into lib/lib.o, I can either create Config.in symbol CONFIG_NEED_GENERIC_BUST_SPINLOCK, or add HAVE_ARCH_BUST_SPINLOCK #define into some of i386, ia64, mips64, s390 and s390x architecture dependent headers.
Thanks,
Petr Vandrovec [email protected]

#1 lore raw
Quoted 10 lines · Petr Vandrovec
Quoted 9 lines · Jeff Garzik
Quoted 5 lines · Petr Vandrovec

I've got strange idea and tried to build diskless machine around
2.5.3... Besides problem with segfaulting crc32 (it is initialized after net/ipv4/ipconfig.c due to lib/lib.a being a library... I had to hardcode lib/crc32.o before --start-group in main Makefile, but it is another story)

Would you be willing to cook up a patch for this problem?

Quoted 7 lines
Quoted 3 lines

I ran into this too. It was solved by setting CONFIG_CRC32=n and letting the Makefile rules pull it in... but lib/lib.a needs to be lib/lib.o really.

Unfortunately during conversion I found that there is lib/bust_spinlocks.c, which is always included in lib.a, is always compiled, even if architecture provides its own bust_spinlocks function.

Yep

Quoted 3 lines

As no other module in lib/ uses module_init() initalization, it looks to me like that we should move crc32.c from lib/ to kernel/, instead of turning lib.a into lib.o.

Having lib.a is really a special case, and we can easily fix that by fixing bust_spinlocks, not by moving what is truly a library routine to somewhere other than lib/

Quoted 5 lines

But of course if there is consensus that I should convert lib/lib.a into lib/lib.o, I can either create Config.in symbol CONFIG_NEED_GENERIC_BUST_SPINLOCK, or add HAVE_ARCH_BUST_SPINLOCK #define into some of i386, ia64, mips64, s390 and s390x architecture dependent headers.

Implementing HAVE_ARCH_BUST_SPINLOCK would follow kernel convention quite nicely...

Jeff

#2 lore raw
In reply to Petr Vandrovec
Quoted 6 lines · Jeff Garzik
Quoted 3 lines

Unfortunately during conversion I found that there is lib/bust_spinlocks.c, which is always included in lib.a, is always compiled, even if architecture provides its own bust_spinlocks function.

Yep

Quoted 8 lines
Quoted 5 lines

But of course if there is consensus that I should convert lib/lib.a into lib/lib.o, I can either create Config.in symbol CONFIG_NEED_GENERIC_BUST_SPINLOCK, or add HAVE_ARCH_BUST_SPINLOCK #define into some of i386, ia64, mips64, s390 and s390x architecture dependent headers.

Implementing HAVE_ARCH_BUST_SPINLOCK would follow kernel convention quite nicely...

Hi Linus,
included patch (for 2.5.3) fixes problems with late initialization
of lib/crc32.o - as it was part of .a library file, it was picked by link process AFTER at least one .o file required it - for example when ipv4 autoconfiguration file needed it. So crc32's initalization function was invoked after ipconfig's one - and it crashed inside of ipconfig due to this problem.

Jeff Garzik advised me that I should convert lib.a to lib.o instead
of moving crc32 somewhere else, so I did it. I decided to put HAVE_ARCH_BUST_SPINLOCKS macro to asm/system.h headers - if you disagree with this location, tell me.

Couple of architectures adds $(TOPDIR)/lib/lib.o into its LIBS
instead of reusing already set $(LIBS) variable. I was not sure what PPC people meant with this, so I left it as is.

I'd like to ask architecture maintainers (especially ia64, PPC, Sparc
and Sparc64 maintainers) for verification that my changes did not break anything... I tested only i386, but changes were obvious...

Best regards,
Petr Vandrovec [email protected]

Makefile +1 -1
diff -urdN linux/Makefile linux/Makefile
@@ -121,7 +121,7 @@
 CORE_FILES	=kernel/kernel.o mm/mm.o fs/fs.o ipc/ipc.o
 NETWORKS	=net/network.o
 
-LIBS		=$(TOPDIR)/lib/lib.a
+LIBS		=$(TOPDIR)/lib/lib.o
 SUBDIRS		=kernel lib drivers mm fs net ipc
 
 DRIVERS-n :=
arch/ia64/boot/Makefile +1 -1
diff -urdN linux/arch/ia64/boot/Makefile linux/arch/ia64/boot/Makefile
@@ -23,7 +23,7 @@
 all:	$(targets-y)
 
 bootloader: $(OBJECTS)
-	$(LD) $(LINKFLAGS) $(OBJECTS) $(TOPDIR)/lib/lib.a $(TOPDIR)/arch/$(ARCH)/lib/lib.a \
+	$(LD) $(LINKFLAGS) $(OBJECTS) $(TOPDIR)/lib/lib.o $(TOPDIR)/arch/$(ARCH)/lib/lib.a \
 	-o bootloader
 
 clean:
arch/ppc/boot/chrp/Makefile +1 -1
diff -urdN linux/arch/ppc/boot/chrp/Makefile linux/arch/ppc/boot/chrp/Makefile
@@ -23,7 +23,7 @@
 
 OBJS = ../common/crt0.o start.o main.o misc.o ../common/string.o image.o \
 	../common/ofcommon.o
-LIBS = $(TOPDIR)/lib/lib.a ../lib/zlib.a
+LIBS = $(TOPDIR)/lib/lib.o ../lib/zlib.a
 ADDNOTE = ../utils/addnote
 PIGGYBACK = ../utils/piggyback
 
arch/ppc/boot/pmac/Makefile +1 -1
diff -urdN linux/arch/ppc/boot/pmac/Makefile linux/arch/ppc/boot/pmac/Makefile
@@ -15,7 +15,7 @@
 COMMONOBJS = start.o misc.o ../common/string.o image.o ../common/ofcommon.o
 COFFOBJS = ../common/coffcrt0.o $(COMMONOBJS) coffmain.o
 CHRPOBJS = ../common/crt0.o $(COMMONOBJS) chrpmain.o
-LIBS = $(TOPDIR)/lib/lib.a ../lib/zlib.a
+LIBS = $(TOPDIR)/lib/lib.o ../lib/zlib.a
 
 MKNOTE := ../utils/mknote
 SIZE := ../utils/size
arch/sparc/Makefile +1 -1
diff -urdN linux/arch/sparc/Makefile linux/arch/sparc/Makefile
@@ -41,7 +41,7 @@
 CORE_FILES := arch/sparc/kernel/kernel.o arch/sparc/mm/mm.o $(CORE_FILES) \
 	arch/sparc/math-emu/math-emu.o
 
-LIBS := $(TOPDIR)/lib/lib.a $(LIBS) $(TOPDIR)/arch/sparc/prom/promlib.a \
+LIBS := $(LIBS) $(TOPDIR)/arch/sparc/prom/promlib.a \
 	$(TOPDIR)/arch/sparc/lib/lib.a
 
 # This one has to come last
arch/sparc64/Makefile +1 -1
diff -urdN linux/arch/sparc64/Makefile linux/arch/sparc64/Makefile
@@ -69,7 +69,7 @@
 
 CORE_FILES += arch/sparc64/math-emu/math-emu.o
 
-LIBS := $(TOPDIR)/lib/lib.a $(LIBS) $(TOPDIR)/arch/sparc64/prom/promlib.a \
+LIBS := $(LIBS) $(TOPDIR)/arch/sparc64/prom/promlib.a \
 	$(TOPDIR)/arch/sparc64/lib/lib.a
 
 vmlinux.aout: vmlinux
include/asm-i386/system.h +2 -0
diff -urdN linux/include/asm-i386/system.h linux/include/asm-i386/system.h
@@ -123,6 +123,8 @@
 	__asm__("movl %0,%%cr4": :"r" (x));
 #define stts() write_cr0(8 | read_cr0())
 
+#define HAVE_ARCH_BUST_SPINLOCKS
+
 #endif	/* __KERNEL__ */
 
 #define wbinvd() \
include/asm-ia64/system.h +2 -0
diff -urdN linux/include/asm-ia64/system.h linux/include/asm-ia64/system.h
@@ -412,6 +412,8 @@
 } while (0)
 #endif
 
+#define HAVE_ARCH_BUST_SPINLOCKS
+
 #endif /* __KERNEL__ */
 
 #endif /* __ASSEMBLY__ */
include/asm-mips64/system.h +2 -0
diff -urdN linux/include/asm-mips64/system.h linux/include/asm-mips64/system.h
@@ -256,4 +256,6 @@
 
 extern void set_except_vector(int n, void *addr);
 
+#define HAVE_ARCH_BUST_SPINLOCKS
+
 #endif /* _ASM_SYSTEM_H */
include/asm-s390/system.h +3 -0
diff -urdN linux/include/asm-s390/system.h linux/include/asm-s390/system.h
@@ -250,6 +250,9 @@
 extern void save_fp_regs(s390_fp_regs *fpregs);
 extern int restore_fp_regs1(s390_fp_regs *fpregs);
 extern void restore_fp_regs(s390_fp_regs *fpregs);
+
+#define HAVE_ARCH_BUST_SPINLOCKS
+
 #endif
 
 #endif
include/asm-s390x/system.h +3 -0
diff -urdN linux/include/asm-s390x/system.h linux/include/asm-s390x/system.h
@@ -260,6 +260,9 @@
 extern void save_fp_regs(s390_fp_regs *fpregs);
 extern int restore_fp_regs1(s390_fp_regs *fpregs);
 extern void restore_fp_regs(s390_fp_regs *fpregs);
+
+#define HAVE_ARCH_BUST_SPINLOCKS
+
 #endif
 
 #endif
lib/Makefile +1 -1
diff -urdN linux/lib/Makefile linux/lib/Makefile
@@ -6,7 +6,7 @@
 # unless it's something special (ie not a .c file).
 #
 
-L_TARGET := lib.a
+O_TARGET := lib.o
 
 export-objs := cmdline.o dec_and_lock.o rwsem-spinlock.o rwsem.o crc32.o
 
lib/bust_spinlocks.c +3 -1
diff -urdN linux/lib/bust_spinlocks.c linux/lib/bust_spinlocks.c
@@ -14,6 +14,8 @@
 #include <linux/wait.h>
 #include <linux/vt_kern.h>
 
+#ifndef HAVE_ARCH_BUST_SPINLOCKS
+
 extern spinlock_t timerlist_lock;
 
 void bust_spinlocks(int yes)
@@ -38,4 +40,4 @@
 	}
 }
 
-
+#endif /* HAVE_ARCH_BUST_SPINLOCKS */
#3 lore raw

From: Petr Vandrovec <[email protected]>
Date: Thu, 31 Jan 2002 23:53:06 +0100

I'd like to ask architecture maintainers (especially ia64, PPC, Sparc
and Sparc64 maintainers) for verification that my changes did not break anything... I tested only i386, but changes were obvious...

I'll test this out up after Linus releases a pre-patch with this included.

As a side note, this thing is so tiny (less than 4K on sparc64!) so why don't we just include it unconditionally instead of having all of this "turn it on for these drivers" stuff?

#4 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 3 lines

As a side note, this thing is so tiny (less than 4K on sparc64!) so why don't we just include it unconditionally instead of having all of this "turn it on for these drivers" stuff?

Because 100 4K drivers suddenly becomes 0.5Mb. There are those of us trying to stuff Linux into embedded devices who if anything want more configuration options not people taking stuff out.

What I'd much rather see if this is an issue is:

bool 'Do you want to customise for a very small system'

which auto enables all the random small stuff if you say no, and goes much deeper into options if you say yes.

Alan

#5 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

From: Alan Cox <[email protected]>
Date: Thu, 31 Jan 2002 23:24:10 +0000 (GMT)

Quoted 3 lines

As a side note, this thing is so tiny (less than 4K on sparc64!) so why don't we just include it unconditionally instead of having all of this "turn it on for these drivers" stuff?

Because 100 4K drivers suddenly becomes 0.5Mb.

However this isn't a driver, the crc library stuff is more akin to "strlen()". Are you suggesting to provide a CONFIG_STRINGOPS=n too? I wish you luck building that kernel :-)

#6 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 5 lines

Because 100 4K drivers suddenly becomes 0.5Mb.

However this isn't a driver, the crc library stuff is more akin to "strlen()". Are you suggesting to provide a CONFIG_STRINGOPS=n too? I wish you luck building that kernel :-)

For a large number of systems you don't need the CRC library. There are no systems where you don't need memcpy, so your comparison is stupid to say the least.

"Gee making this link is hard" is not a good reason to simplify the config file, its a good reason to simplify the make file system

#7 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller

"David S. Miller" <[email protected]> said:

[...]

Quoted 3 lines

However this isn't a driver, the crc library stuff is more akin to "strlen()". Are you suggesting to provide a CONFIG_STRINGOPS=n too? I wish you luck building that kernel :-)

libc.a was invented precisely to handle such stuff automatically when linking... if it doesn't work, it should be fixed. I.e., making .a --> .o is a step in the wrong direction IMVHO.

Signature · 1 line
Horst von Brand			     http://counter.li.org # 22616
#8 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 8 lines · Horst von Brand
Quoted 3 lines · David S. Miller

However this isn't a driver, the crc library stuff is more akin to "strlen()". Are you suggesting to provide a CONFIG_STRINGOPS=n too? I wish you luck building that kernel :-)

libc.a was invented precisely to handle such stuff automatically when linking... if it doesn't work, it should be fixed. I.e., making .a --> .o is a step in the wrong direction IMVHO.

There are two contradictory requirements. crc32.o must only be linked if anything in the kernel needs it, linker puts crc32.o after the code that uses it. crc32.o must be linked before anything that uses it so the crc32 __init code can initialize first.

#9 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

From: Keith Owens <[email protected]>
Date: Fri, 01 Feb 2002 21:28:52 +1100

Quoted 3 lines · Horst von Brand

libc.a was invented precisely to handle such stuff automatically when linking... if it doesn't work, it should be fixed. I.e., making .a --> .o is a step in the wrong direction IMVHO.

There are two contradictory requirements. crc32.o must only be linked if anything in the kernel needs it, linker puts crc32.o after the code that uses it. crc32.o must be linked before anything that uses it so the crc32 __init code can initialize first.

Horst isn't even talking about the initcall issues, he's talking about how linking with libc works in that it only brings in the routines you actually reference.

Will you get over the initcall thing already, you must be dreaming about it at this point. I mean really, just GET OVER IT. :-)

#10 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 9 lines · David S. Miller

From: Keith Owens <[email protected]>
Date: Fri, 01 Feb 2002 21:28:52 +1100

Quoted 3 lines · Horst von Brand

libc.a was invented precisely to handle such stuff automatically when linking... if it doesn't work, it should be fixed. I.e., making .a --> .o is a step in the wrong direction IMVHO.

Quoted 8 lines

There are two contradictory requirements. crc32.o must only be linked if anything in the kernel needs it, linker puts crc32.o after the code that uses it. crc32.o must be linked before anything that uses it so the crc32 __init code can initialize first.

Horst isn't even talking about the initcall issues, he's talking about how linking with libc works in that it only brings in the routines you actually reference.

Go back and read the entire thread Dave. It started with

http://www.lib.uaa.alaska.edu/linux-kernel/archive/2002-Week-04/1235.html

"Besides problem with segfaulting crc32 (it is initialized after net/ipv4/ipconfig.c due to lib/lib.a being a library... I had to hardcode lib/crc32.o before --start-group in main Makefile, but it is another story)"

http://www.lib.uaa.alaska.edu/linux-kernel/archive/2002-Week-04/1384.html

"included patch (for 2.5.3) fixes problems with late initialization of lib/crc32.o - as it was part of .a library file, it was picked by link process AFTER at least one .o file required it - for example when ipv4 autoconfiguration file needed it. So crc32's initalization function was invoked after ipconfig's one - and it crashed inside of ipconfig due to this problem."

Horst was not talking about initcall but his suggestion of using lib.a for crc32.o had already been proven not to work, because of the (surprise) initcall problem. I was merely pointing out to Horst why lib.a was not working and why other options had to be considered.

Quoted 2 lines

Will you get over the initcall thing already, you must be dreaming about it at this point. I mean really, just GET OVER IT. :-)

Nightmare is more like it.

#11 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller
Quoted 6 lines · Keith Owens
Quoted 2 lines · David S. Miller

Will you get over the initcall thing already, you must be dreaming about it at this point. I mean really, just GET OVER IT. :-)

Nightmare is more like it.

Exaggeration is more like it :)

Jeff

#12 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to Alan Cox
Quoted 8 lines · Alan Cox
Quoted 3 lines

As a side note, this thing is so tiny (less than 4K on sparc64!) so why don't we just include it unconditionally instead of having all of this "turn it on for these drivers" stuff?

Because 100 4K drivers suddenly becomes 0.5Mb. There are those of us trying to stuff Linux into embedded devices who if anything want more configuration options not people taking stuff out.

Quoted 6 lines

What I'd much rather see if this is an issue is:

bool 'Do you want to customise for a very small system'

which auto enables all the random small stuff if you say no, and goes much deeper into options if you say yes.

I mostly agree. Except when I have looked at trying to get the kernel (compiled size down) the biggest bloat was in the core. Things like having both a page and a block cache.

Getting code reuse in the core higher would cut down on kernel size a lot. But that isn't quick fix territory.

Eric

#13 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to Alan Cox
Quoted 15 lines · Eric W. Biederman
Quoted 8 lines
Quoted 3 lines

As a side note, this thing is so tiny (less than 4K on sparc64!) so why don't we just include it unconditionally instead of having all of this "turn it on for these drivers" stuff?

Because 100 4K drivers suddenly becomes 0.5Mb. There are those of us trying to stuff Linux into embedded devices who if anything want more configuration options not people taking stuff out.

Quoted 6 lines

What I'd much rather see if this is an issue is:

bool 'Do you want to customise for a very small system'

which auto enables all the random small stuff if you say no, and goes much deeper into options if you say yes.

Quoted 6 lines

I mostly agree. Except when I have looked at trying to get the kernel (compiled size down) the biggest bloat was in the core. Things like having both a page and a block cache.

Getting code reuse in the core higher would cut down on kernel size a lot. But that isn't quick fix territory.

Is it really worth the effort? During the past year the average size of embedded systems that people want to use for seems to have increased dramatically. In case of the MIPS port the core activity is about to move away from the 32-bit to 64-bit kernel.

Ralf

#14 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 4 lines

Is it really worth the effort? During the past year the average size of embedded systems that people want to use for seems to have increased dramatically. In case of the MIPS port the core activity is about to move away from the 32-bit to 64-bit kernel.

Embedded system and fancy handheld toys are not really the same thing. One subset of the market gets cheaper the other gets flashier and more high powered.

Mips is now dead in the handheld market if windows ce drops it.

#15 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to Alan Cox

On Sun, Feb 03, 2002 at 08:01:35AM +0100, Ralf Baechle wrote:

Is it really worth the effort? During the past year the average size of embedded systems that people want to use for seems to have increased dramatically. In case of the MIPS port the core activity is about to move away from the 32-bit to 64-bit kernel.

For some hand-held devices (eg. iPAQ), we want as much memory free as possible as the only filesystem available is ramfs... moving to 64-bit would be unthinkable! :)

--cw

#16 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 2 lines · [email protected] said:

For some hand-held devices (eg. iPAQ), we want as much memory free as
possible as the only filesystem available is ramfs...

You took the flash out of your iPAQ? How does it boot?

--
dwmw2

#17 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to Chris Wedgwood

On Sun, Feb 03, 2002 at 12:16:00PM +0000, David Woodhouse wrote:

[email protected] said:

You took the flash out of your iPAQ? How does it boot?

Obviously it still has flash... but much of the filesystem is ramfs and thus ram is rather precious.

For now anyhow, I think I'll upgrade it to 128M shortly.

--cw

#18 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 2 lines · [email protected] said:

Obviously it still has flash... but much of the filesystem is ramfs
and thus ram is rather precious.

Ah, I see. What are you doing with it? Mine hardly touches the ramfs.

sh-2.03# df
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/mtdblock/2 32256 26908 5348 84% /
tmpfs 15428 260 15168 2% /mnt/ramfs

--
dwmw2

#19 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller
Quoted 6 lines · Alan Cox

What I'd much rather see if this is an issue is:

bool 'Do you want to customise for a very small system'

which auto enables all the random small stuff if you say no, and goes much deeper into options if you say yes.

CONFIG_SMALL or similar would indeed be nice...

Jeff

#20 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller

Em Thu, Jan 31, 2002 at 11:24:10PM +0000, Alan Cox escreveu:

Quoted 8 lines
Quoted 3 lines

As a side note, this thing is so tiny (less than 4K on sparc64!) so why don't we just include it unconditionally instead of having all of this "turn it on for these drivers" stuff?

Because 100 4K drivers suddenly becomes 0.5Mb. There are those of us trying to stuff Linux into embedded devices who if anything want more configuration options not people taking stuff out.

Quoted 6 lines

What I'd much rather see if this is an issue is:

bool 'Do you want to customise for a very small system'

which auto enables all the random small stuff if you say no, and goes much deeper into options if you say yes.

heh, after I've read that you managed to boot 2.4 + rmap in a machine with just 4 MB after tweaking some table sizes I thought about devoting some time to identify those tables and making them options in make *config, with even a nice CONFIG_TINY, like you said 8)

I'll eventually do this, and I'd appreciate if people send me suggestions of tables/data structures that can be trimmed/reduced. Yeah, I'll take a look at the .config files used in the embedded distros.

- Arnaldo

#21 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 9 lines · [email protected] said:

heh, after I've read that you managed to boot 2.4 + rmap in a machine
with just 4 MB after tweaking some table sizes I thought about devoting some time to identify those tables and making them options in make *config, with even a nice CONFIG_TINY, like you said 8)

I'll eventually do this, and I'd appreciate if people send me suggestions of tables/data structures that can be trimmed/reduced. Yeah, I'll take a look at the .config files used in the embedded distros.

CONFIG_BLK_DEV=n should allow you to trim a lot of cruft that most of my embedded boxen never use.

--
dwmw2

#22 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller

On Thu, Jan 31, 2002 at 11:24:10PM +0000, Alan Cox wrote:

Because 100 4K drivers suddenly becomes 0.5Mb. There are those of us trying to stuff Linux into embedded devices who if anything want more configuration options not people taking stuff out.

Well, I'm more or less in agreement here, especially when working with small embedded devices which have a few (say 16 or 32) MB of RAM for EVERYTHING, kernel, userspace AND filesystems.

However, I wonder if we can't have the linker remove unnecessary and unreferences objects, functions and variables?

What I'd much rather see if this is an issue is:

bool 'Do you want to customise for a very small system'

_IF_ the linker can remove things, it would simplify this too --- we could if a few important places produce code slightly differently to favour speed over size and not reference various things. Also, the above option would turn-off inlining as that seems to makie quite a difference at times (BTW, I'm not sure about this, but it seems gcc and C99 don't agree with static/extern inline semantics?)

--cw

#23 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller
Quoted 7 lines · Chris Wedgwood

On Thu, Jan 31, 2002 at 11:24:10PM +0000, Alan Cox wrote:

Because 100 4K drivers suddenly becomes 0.5Mb. There are those of us trying to stuff Linux into embedded devices who if anything want more configuration options not people taking stuff out.

Quoted 6 lines

Well, I'm more or less in agreement here, especially when working with small embedded devices which have a few (say 16 or 32) MB of RAM for EVERYTHING, kernel, userspace AND filesystems.

However, I wonder if we can't have the linker remove unnecessary and unreferences objects, functions and variables?

We can. Graham Stoney had all this going against 2.2. See

http://www.google.com/search?q=stoney+ffunction-sections&hl=en&start=10&sa=N
http://www.cs.helsinki.fi/linux/linux-kernel/Year-2000/2000-29/0415.html

Quoted 10 lines

What I'd much rather see if this is an issue is:

bool 'Do you want to customise for a very small system'

_IF_ the linker can remove things, it would simplify this too --- we could if a few important places produce code slightly differently to favour speed over size and not reference various things. Also, the above option would turn-off inlining as that seems to makie quite a difference at times (BTW, I'm not sure about this, but it seems gcc and C99 don't agree with static/extern inline semantics?)

The kernel doesn't link when you compile with -fno-inline because of all the extern inline' qualifiers. These need to be converted to static
inline'. Jim Houston has a script which does this. See

	http://www.uwsg.iu.edu/hypermail/linux/kernel/0201.3/0888.html

It would be rather good if we could get that script run across the tree - no-inline has its uses at times.

-

#24 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller

On Fri, Feb 01, 2002 at 07:01:56PM -0800, Andrew Morton wrote:

We can. Graham Stoney had all this going against 2.2. See

http://www.google.com/search?q=stoney+ffunction-sections&hl=en&start=10&sa=N
http://www.cs.helsinki.fi/linux/linux-kernel/Year-2000/2000-29/0415.html

This puts every function it's own section. That seems not only cumbersome to me, but also a little complex. That said, it may be a good way if run every now and then to detect when cruft starts to accumulate for any given .config and allow people to tweak things for smaller kernels.

Is there no way to write something like:

--snip-- foo.c --snip--
void
blem()
{
}

void
bar()

{
    blem();
    return 0;
}

int foo()

{
    return 1;
}

int main(...)

{
    return foo();
}

--snip-- foo.c --snip--

compile and link it, have the linker know main or some part of crt0 is special, build a graph of the final ELF object, see that bar and blem are not connected to 'main' and discard them?

I'm pretty sure (but not 100% certain) that oher compilers can do this, maybe someone can test on other platforms?

A really smart linker (if given enough compiler help) could build a directional graph and still remove this code even if blem called foo.

Perhaps I'm making something that's extremely complex sound simple, but it doesn't seem to me that this should be that complex... maybe someone more farmiliar with binutils and/or gcc can comment and tell me why I'm being a fool :)

The kernel doesn't link when you compile with -fno-inline because of all the extern inline' qualifiers. These need to be converted to static inline'. Jim Houston has a script which does this.

Semantically, in gcc land, someone explain the difference between:

inline

extern inline

static inline

please?

My tests seem to indicte:

inline creates an non-inline functions. Simple tests failed to have this function inlined at all

static inline and extern inline functions can and will be inlined depending on optimization level

extern inline doesn't product a non-inlined function (even if it is referenced) and hence barfs if you don't compile with optimisations for my simple test

Now, I wonder

why 'inline' for me, never inlines?

is there a way to force inlining of a function?

are non-inlined functions ever called when optimizations are enabled?

Thanks,

--cw

#25 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

cc list trimmed.
Quoted 9 lines · Chris Wedgwood

compile and link it, have the linker know main or some part of crt0 is special, build a graph of the final ELF object, see that bar and blem are not connected to 'main' and discard them?

I'm pretty sure (but not 100% certain) that oher compilers can do this, maybe someone can test on other platforms?

A really smart linker (if given enough compiler help) could build a directional graph and still remove this code even if blem called foo.

AFAICT it has to be done in two stages. Identify extern symbols that are not referenced and convert them to static, then compile and let gcc warn about unused statics and remove them.

The two stage approach is required because ld does not have enough information to identify all function calls. ld views an object as opaque blobs of text and data with relocation entries that tie them together. When a function is called from the same source file, gcc/gas tend to use PC relative branches for the inter-source call. PC relative local branches do not usually generate relocation entries. Without a relocation entry, ld cannot detect the inter-source call.

There is also a problem with exported symbols. To ld, EXPORT_SYMBOL looks like a reference to the symbol, but the export entry is irrelevant, what really matters is if any module refers to those symbols. ld cannot distinguish between used and unused exported symbols. To make matters worse, if you define foo as static and EXPORT_SYMBOL(foo) in the same source then gcc thinks that the symbol is used, because of the EXPORT_SYMBOL() reference. Again what really matters is if any module uses the exported symbol.

The script below runs the .o files for the kernel and identifies extern symbols with no extern references, i.e. the first part of the problem. It knows about exported symbols and modules and handles them correctly. I have not run the script for years, the list of conglomerates and the list starting with mod_use_count_ will need updating.

Beware that you get a lot of false positives because of the kernel config mechanism. A symbol may be listed as unused but changing the config may cause it to be used. This is especially true for exported symbols.

<davem>What, no initc**l reference?</davem>

#!/usr/bin/perl -w
#
# namespace.pl. Mon Jan 27 22:25:47 EST 1997
#
# Perform a name space analysis on the linux kernel.
#
# Copyright Keith Owens <[email protected]>. GPL.
#
# Invoke by changing directory to the top of the kernel source
# tree then namespace.pl, no parameters.
#
# Tuned for 2.1.x kernels with the new module handling, it will
# work with 2.0 kernels as well. Last change 2.1.81.
#
# The source must be compiled/assembled first, the object files
# are the primary input to this script. Incomplete or missing
# objects will result in a flawed analysis.
#
# Even with complete objects, treat the result of the analysis
# with caution. Some external references are only used by
# certain architectures, others with certain combinations of
# configuration parameters. Ideally the source should include
# something like
#
# #ifndef CONFIG_...
# static
# #endif
# symbol_definition;
#
# so the symbols are defined as static unless a particular
# CONFIG_... requires it to be external.
#

require 5; # at least perl 5
use strict;
use File::Find;

my $nm = "/usr/bin/nm -p"; # in case somebody moves nm

if ($#ARGV != -1) {
	print STDERR "usage: $0 takes no parameters\n";
	die("giving up\n");
}

my %nmdata = (); # nm data for each object
my %def = (); # all definitions for each name
my %ksymtab = (); # names that appear in __ksymtab_
my %ref = (); # $ref{$name} exists if there is a true external reference to $name
my %export = (); # $export{$name} exists if there is an EXPORT_... of $name

&find(\&linux_objects, '.'); # find the objects and do_nm on them
list_multiply_defined();
resolve_external_references();
list_extra_externals();

exit(0);

sub linux_objects

{
	# Select objects, ignoring objects which are only created by
	# merging other objects.  Also ignore all of modules, scripts
	# and compressed.
	my $basename = $_;
	$_ = $File::Find::name;
	s:^\./::;
	if (/.*\.o$/ && ! (
	    m:/fs.o$: || m:/isofs.o$: || m:/nfs.o$:
	    || m:/xiafs.o$: || m:/umsdos.o$: || m:/hpfs.o$:
	    || m:/smbfs.o$: || m:/ncpfs.o$: || m:/ufs.o$:
	    || m:/affs.o$: || m:/romfs.o$: || m:/kernel.o$:
	    || m:/mm.o$: || m:/ipc.o$: || m:/ext.o$:
	    || m:/msdos.o$: || m:/proc/proc.o$: || m:/minix.o$:
	    || m:/ext2.o$: || m:/sysv.o$: || m:/fat.o$:
	    || m:/vfat.o$: || m:/unix.o$: || m:/802.o$:
	    || m:/appletalk.o$: || m:/ax25.o$: || m:/core.o$:
	    || m:/ethernet.o$: || m:/ipv4.o$: || m:/ipx.o$:
	    || m:/netrom.o$: || m:/ipv6.o$: || m:/x25.o$:
	    || m:/rose.o$: || m:/bridge.o$: || m:/lapb.o$:
	    || m:/sock_n_syms.o$: || m:/teles.o$: || m:/pcbit.o$:
	    || m:/isdn.o$: || m:/ftape.o$: || m:/scsi_mod.o$:
	    || m:/sd_mod.o$: || m:/sr_mod.o$: || m:/lowlevel.o$:
	    || m:/sound.o$: || m:/piggy.o$: || m:/bootsect.o$:
	    || m:/boot/setup.o$: || m:^modules/: || m:^scripts/:
	    || m:/compressed/:
	    || m:/autofs.o$: || m:/lockd/lockd.o$: || m:/nfsd.o$:
	    || m:/sunrpc.o$: || m:/scsi_n_syms.o$:
	    || m:/boot/bbootsect.o$: || m:/boot/bsetup.o$:
	    || m:/misc/parport.o$: || m:/nls/nls.o$:
	    || m:/debug/debug.o$: || m:/netlink/netlink.o$:
	    || m:/sched/sched.o$: || m:/sound/sb.o$: 
	    || m:/sound/soundcore.o$: || m:/pci/pci_syms.o$: 
	    || m:/devpts/devpts.o$: || m:/video/fbdev.o$: 
	  )
	) {
		do_nm($basename, $_);
	}
	$_ = $basename;		# File::Find expects $_ untouched (undocumented)
}

sub do_nm

{
	my ($basename, $fullname) = @_;
	my ($source, $type, $name);
	if (! -e $basename) {
		printf STDERR "$basename does not exist\n";
		return;
	}
	if ($fullname !~ /\.o$/) {
		printf STDERR "$fullname is not an object file\n";
		return;
	}
	$source = $basename;
	$source =~ s/\.o$//;
	if (! -e "$source.c" && ! -e "$source.S") {
		printf STDERR "No source file found for $fullname\n";
		return;
	}
	if (! open(NMDATA, "$nm $basename|")) {
		printf STDERR "$nm $fullname failed $!\n";
		return;
	}
	my @nmdata;
	while (<NMDATA>) {
		chop;
		($type, $name) = (split(/ +/, $_, 3))[1..2];
		# Expected types
		# B weak external reference to data that has been resolved
		# C global variable, uninitialised
		# D global variable, initialised
		# R global array, initialised
		# T global label/procedure
		# U external reference
		# W weak external reference to text that has been resolved
		# a assembler equate
		# b static variable, uninitialised
		# d static variable, initialised
		# r static array, initialised
		# t static label/procedures
		# ? undefined type, used a lot by modules
		if ($type !~ /^[BCDRTUWabdrt?]$/) {
			printf STDERR "nm output for $fullname contains unknown type '$_'\n";
		}
		elsif ($name =~ /\./) {
			# name with '.' is local static
		}
		else {
			$name =~ s/_R[a-f0-9]{8}$//;	# module versions adds this
			if ($type =~ /[BCDRTW]/ &&
				$name ne 'init_module' &&
				$name ne 'cleanup_module' &&
				$name ne 'Using_Versions' &&
				$name !~ /^Version_[0-9]+$/ &&
				$name ne 'kernel_version') {
				if (!exists($def{$name})) {
					$def{$name} = [];
				}
				push(@{$def{$name}}, $fullname);
			}
			push(@nmdata, "$type $name");
			if ($name =~ /^__ksymtab_/) {
				$name = substr($name, 10);
				if (!exists($ksymtab{$name})) {
					$ksymtab{$name} = [];
				}
				push(@{$ksymtab{$name}}, $fullname);
			}
		}
	}
	close(NMDATA);
	if ($#nmdata < 0) {
		printf "No nm data for $fullname\n";
		return;
	}
	$nmdata{$fullname} = \@nmdata;
}

sub list_multiply_defined

{
	my ($name, $module);
	foreach $name (keys(%def)) {
		if ($#{$def{$name}} > 0) {
			printf "$name is multiply defined in :-\n";
			foreach $module (@{$def{$name}}) {
				printf "\t$module\n";
			}
		}
	}
}

sub resolve_external_references

{
	my ($object, $type, $name, $i, $j, $kstrtab, $ksymtab, $export);
	printf "\n";
	foreach $object (keys(%nmdata)) {
		my $nmdata = $nmdata{$object};
		for ($i = 0; $i <= $#{$nmdata}; ++$i) {
			($type, $name) = split(' ', $nmdata->[$i], 2);
			if ($type eq "U") {
				if (exists($def{$name}) || exists($ksymtab{$name})) {
					# add the owning object to the nmdata
					$nmdata->[$i] = "$type $name $object";
					# only count as a reference if it is not EXPORT_...
					$kstrtab = "? __kstrtab_$name";
					$ksymtab = "? __ksymtab_$name";
					$export = 0;
					for ($j = 0; $j <= $#{$nmdata}; ++$j) {
						if ($nmdata->[$j] eq $kstrtab ||
						    $nmdata->[$j] eq $ksymtab) {
							$export = 1;
							last;
						}
					}
					if ($export) {
						$export{$name} = "";
					}
					else {
						$ref{$name} = ""
					}
				}
				elsif ($name ne "mod_use_count_" &&
				       $name ne "__this_module" &&
				       $name ne "_etext" &&
				       $name ne "_edata" &&
				       $name ne "_end" &&
				       $name ne "__start___ksymtab" &&
				       $name ne "__start___ex_table" &&
				       $name ne "__stop___ksymtab" &&
				       $name ne "__stop___ex_table" &&
				       $name ne "__stop___ex_table" &&
				       $name ne "__bss_start" &&
				       $name ne "_text" &&
				       $name ne "__init_begin" &&
				       $name ne "__init_end") {
					printf "Cannot resolve reference to $name from $object\n";
				}
			}
		}
	}
}

sub list_extra_externals

{
	my %noref = ();
	my ($name, @module, $module, $export);
	foreach $name (keys(%def)) {
		if (! exists($ref{$name})) {
			@module = @{$def{$name}};
			foreach $module (@module) {
				if (! exists($noref{$module})) {
					$noref{$module} = [];
				}
				push(@{$noref{$module}}, $name);
			}
		}
	}
	if (%noref) {
		printf "\nExternally defined symbols with no external references\n";
		foreach $module (sort(keys(%noref))) {
			printf "  $module\n";
			foreach (sort(@{$noref{$module}})) {
				if (exists($export{$_})) {
					$export = " (export only)";
				}
				else {
					$export = "";
				}
				printf "    $_$export\n";
			}
		}
	}
}
#26 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 2 lines · [email protected] said:

There is also a problem with exported symbols. To ld, EXPORT_SYMBOL
looks like a reference to the symbol,

Er, surely that's not a problem at all? This is desired behaviour?

Quoted 2 lines

but the export entry is irrelevant, what really matters is if any module
refers to those symbols.

Absolutely not. If we mark a symbol EXPORT_SYMBOL, we want it exported. No questions asked.

The export entry _is_ relevant; furthermore it's the _only_ thing that's relevant, and there is no way of knowing if there's a module that isn't in the tree, or maybe even a module that _is_ in the tree but not compiled today, that needs the symbol in question.

I sincerely hope it's just too early in the morning here and I'm misunderstanding you :)

--
dwmw2

#27 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 11 lines · David Woodhouse
Quoted 2 lines · [email protected] said:

There is also a problem with exported symbols. To ld, EXPORT_SYMBOL
looks like a reference to the symbol,

Er, surely that's not a problem at all? This is desired behaviour?

Quoted 2 lines

but the export entry is irrelevant, what really matters is if any module
refers to those symbols.

Absolutely not. If we mark a symbol EXPORT_SYMBOL, we want it exported. No questions asked.

Does anything still use this symbol or is it dead? Quite valid question.

Quoted 4 lines

The export entry _is_ relevant; furthermore it's the _only_ thing that's relevant, and there is no way of knowing if there's a module that isn't in the tree, or maybe even a module that _is_ in the tree but not compiled today, that needs the symbol in question.

I said that you get false positives because the code that uses the symbol might not be compiled in your kernel. The script flags symbols that are extern or exported and are not used in the current kernel. Somebody then has to go through the report and decide if those symbols are required under other config settings or by third party modules, assuming that they care about third party modules.

The script is NOT an automatic list of what can be cleaned up. I doubt that such a list can be generated in the face of the kernel config system and third party modules. It is only a starting point for symbols that need reviewing.

#28 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 2 lines · [email protected] said:

Does anything still use this symbol or is it dead? Quite valid
question.

Indeed.

Quoted 4 lines

The script is NOT an automatic list of what can be cleaned up. I
doubt that such a list can be generated in the face of the kernel config system and third party modules. It is only a starting point for symbols that need reviewing.

OK, cool. As it came out of the discussion about -ffunction-sections and automatically garbage-collecting unused functions during the build, I thought you had other, less sane intentions. Sorry for the misunderstanding.

--
dwmw2

#29 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller
Quoted 30 lines · Chris Wedgwood

Is there no way to write something like:

--snip-- foo.c --snip--
void
blem()
{
}

void
bar()

{
    blem();
    return 0;
}

int foo()

{
    return 1;
}

int main(...)

{
    return foo();
}

--snip-- foo.c --snip--

compile and link it, have the linker know main or some part of crt0 is special, build a graph of the final ELF object, see that bar and blem are not connected to 'main' and discard them?

Quoted 5 lines

I'm pretty sure (but not 100% certain) that oher compilers can do this, maybe someone can test on other platforms?

A really smart linker (if given enough compiler help) could build a directional graph and still remove this code even if blem called foo.

One piece of the necessary compiler help would be -ffunction-sections. If they are in the same section in the same object file, they simply can not be removed safely. Relocation information for calls to local functions is not reliably available at link time.

Signature · 2 lines
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer
#30 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller
Quoted 2 lines · Chris Wedgwood

A really smart linker (if given enough compiler help) could build a directional graph and still remove this code even if blem called foo.

Agreed and this has been my objection to the function-sections patch. There is no need for it if you make the toolchain smarter.

For example I would love to see a !CONFIG_MODULES build rip out all the code that was not actively referenced, such as EXPORT_SYMBOL functions which are never called.

Jeff

#31 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller
Quoted 3 lines · Daniel Jacobowitz

One piece of the necessary compiler help would be -ffunction-sections. If they are in the same section in the same object file, they simply can not be removed safely.

Such as the patch that was mentioned earlier in this thread :)

Quoted 2 lines

Relocation information for calls to local
functions is not reliably available at link time.

With a smarter toolchain it could be.

One will need a smarter toolchain in order to re-order functions anyway, which is an area where benchmarks on other compilers are showing benefits. (ie. moving "cold" functions to the end of the module, given profiling feedback)

Jeff

#32 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller
Quoted 8 lines · Arnaldo Carvalho de Melo

Em Thu, Jan 31, 2002 at 11:24:10PM +0000, Alan Cox escreveu:

Quoted 6 lines

What I'd much rather see if this is an issue is:

bool 'Do you want to customise for a very small system'

which auto enables all the random small stuff if you say no, and goes much deeper into options if you say yes.

Quoted 8 lines

heh, after I've read that you managed to boot 2.4 + rmap in a machine with just 4 MB after tweaking some table sizes I thought about devoting some time to identify those tables and making them options in make *config, with even a nice CONFIG_TINY, like you said 8)

I'll eventually do this, and I'd appreciate if people send me suggestions of tables/data structures that can be trimmed/reduced. Yeah, I'll take a look at the .config files used in the embedded distros.

One of inhabitants of my "Don't delete!" mail folder:

Re: Limited RAM - how to save it?
From: Lars Brinkhoff <[email protected]>
To: Jan-Benedict Glaw <[email protected]>
Cc: [email protected]

Quoted 3 lines · Jan-Benedict Glaw

I'm working on a 4MB linux system (for a customer) which has quite limited resources [...] If you've got further ideas on getting the kernel a bit smaller, would be nice to get a mail dropped...

I started a port of Linux 2.3.99 to a MIPS device which usually has 2M flash and 4M RAM.

To reduce the size of the text and data sections, I sorted the output of the "size" command and investigated the files with the largest sections.

These are the memory-saving changes I arrived at. They are not really tested, so some of them may break some functionality. Also, I don't know whether they apply to the current kernels.

fs/dcache.c

Changed HASH_BITS from 14 to 8. This reduces the size of the cache from 128K to 2K.

fs/inode.c

Changed HASH_BITS from 14 to 8. This reduces the size of the cache from 128K to 2K.

include/linux/blk.h

Changed NR_REQUEST from 256 to 16. This reduces the number of requests that can be queued. The size of the queue is reduced from 16K to 1K.

include/linux/major.h

Changed MAX_BLKDEV and MAX_CHRDEV from 256 to 10 and 20, respectively. This reduces the number of block and character devices and saves about 40K.

kernel/printk.c

Changed LOG_BUF_LEN from 16384 bytes to 2048 bytes.

include/linux/tty.h

Changed NR_PTYS and NR_LDISCS from 256 and 16, respectively, to 16 and 4, respectively. Saved about 12K.

Warning: this change may break the pty driver, in which case further modifications will have to be done to drivers/char/pty.c.

kernel/panic.c

Changed a buffer from 1024 bytes to 200 bytes.

include/linux/sched.h

Changed PIDHASH_SZ from 1024 to 16, which saves 1008 bytes.

include/linux/mmzone.h

NR_GPFINDEX from 0x100 to 0x10. Saves 4800 bytes, but I'm not sure it doesn't break anything.

net/Makefile, net/socket.c, net/nosocket.c

Replacing socket.c with nosocket.c, a file containing dummy replacement functions for those in socket.c, saves about 24K.

Warning: this disables the socket API entirely, but it is currently not used in the product.

mm/Makefile, mm/swapfile.c, mm/swap_state.c, mm/noswapfile.c, mm/noswap_state.c

Replacing swapfile.c with noswapfile.c, and swap_state.c with noswap_state.c saves about 12K. The no*.c files contains empty replacement functions.

Warning: this disables swapping of anonymous memory, which isn't used in the product. But note that demand paging of executables still works.

mm/Makefile, mm/mmap.c

The functions in mmap.c could probably also be replaced by empty functions. Estimated saving: 9K (not included in the grand total below).

*, CONFIG_MESSAGES

Applying the CONFIG_MESSAGES patch and disabling all kernel messages saves about 80K.

The CONFIG_MESSAGES patch was written by Graham Stoney <[email protected]>.

With all of the above, and only this enabled in .config:

        CONFIG_EXPERIMENTAL=y
        CONFIG_CPU_R3000=y
        CONFIG_CPU_LITTLE_ENDIAN=y
        CONFIG_ELF_KERNEL=y
        CONFIG_BINFMT_ELF=y
        CONFIG_MODULES=y
        CONFIG_MODVERSIONS=y
        CONFIG_KMOD=y
        CONFIG_CROSSCOMPILE=y

, the kernel is down to about 550K.

Here is the output of "size vmlinux". I think this is without the CONFIG_MESSAGES patch (it was long since I worked with this).

text    data    bss     dec     hex     filename
572128  41964   15860   629952  99cc0   vmlinux
Signature · 7 lines
Lars Brinkhoff          http://lars.nocrew.org/     Linux, GCC, PDP-10
Brinkhoff Consulting    http://www.brinkhoff.se/    programming
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#33 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller
Quoted 5 lines

include/linux/blk.h

Changed NR_REQUEST from 256 to 16. This reduces the number of requests that can be queued. The size of the queue is reduced from 16K to 1K.

You can do even more than this -- I dunno what I/O interface these embedded system generally uses (the mtd stuff?). Provided you provide a direct make_request_fn entry into these instead of using the queue, you can basically cut all of ll_rw_blk.c and elevator.c. ll_rw_block, submit_bh, and generic_make_request would be all that is left.

That should reclaim quite a lot of space. If there's any interest in this (has it already been done?), I can help out getting it done.

Signature · 1 line
Jens Axboe
#34 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

Quoted 8 lines · In article <[email protected]> you wrote:
Quoted 5 lines

include/linux/blk.h

Changed NR_REQUEST from 256 to 16. This reduces the number of requests that can be queued. The size of the queue is reduced from 16K to 1K.

You can do even more than this -- I dunno what I/O interface these embedded system generally uses (the mtd stuff?)

the MTD stuff wants CONFIG_BLOCKLAYER ;) flash + jffs2 doesn't need any blocklayer at all ;)

#35 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to Jens Axboe
Quoted 7 lines · On Sat, Feb 02 2002, [email protected] wrote:
Quoted 5 lines · In article <[email protected]> you wrote:
Quoted 5 lines

include/linux/blk.h

Changed NR_REQUEST from 256 to 16. This reduces the number of requests that can be queued. The size of the queue is reduced from 16K to 1K.

Quoted 5 lines
Quoted 2 lines

You can do even more than this -- I dunno what I/O interface these embedded system generally uses (the mtd stuff?)

the MTD stuff wants CONFIG_BLOCKLAYER ;) flash + jffs2 doesn't need any blocklayer at all ;)

Right, that was exactly my point...

Signature · 1 line
Jens Axboe
#36 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to Jens Axboe
Quoted 4 lines · [email protected] said:

You can do even more than this -- I dunno what I/O interface these
embedded system generally uses (the mtd stuff?). Provided you provide a direct make_request_fn entry into these instead of using the queue, you can basically cut all of ll_rw_blk.c and elevator.c.

We don't even do that - look at jffs2_read_super(). The only reason it even _looks_ like we're using a block device is because it was the easiest way to arrange mounting. We only allow you to mount the 'mtdblock' device, which isn't actually used - all we do is use the minor number to look up the real underlying MTD device. The dependency on _any_ of the block code isn't real - I can fix it as soon as there's an incentive to do so (like CONFIG_BLK_DEV).

Quoted 1 line

ll_rw_block, submit_bh, and generic_make_request would be all that is left.

Those can go too. Likewise page->buffers.

Quoted 2 lines

That should reclaim quite a lot of space. If there's any interest in
this (has it already been done?), I can help out getting it done.

I had a go once. Long enough ago that it's probably not worth trying to find it again. I'd suggest a boolean like CONFIG_BUFFER_CACHE which does the really intrusive stuff like removing page->buffers, and CONFIG_BLK_DEV which can be modular (if CONFIG_BUFFER_CACHE is on), and which controls compilation of everything in drivers/block/ and fs/block_dev.c

--
dwmw2

#37 lore raw

Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3 does

In reply to David S. Miller
Quoted 6 lines · Jeff Garzik
Quoted 3 lines · Daniel Jacobowitz

One piece of the necessary compiler help would be -ffunction-sections. If they are in the same section in the same object file, they simply can not be removed safely.

Such as the patch that was mentioned earlier in this thread :)

Yes, the patch that he was objecting to :)

Quoted 9 lines
Quoted 2 lines

Relocation information for calls to local
functions is not reliably available at link time.

With a smarter toolchain it could be.

One will need a smarter toolchain in order to re-order functions anyway, which is an area where benchmarks on other compilers are showing benefits. (ie. moving "cold" functions to the end of the module, given profiling feedback)

No, the linker simply can not assume that this information is present. That's a limitation of ELF. The right way to do this sort of optimization is to compile every function into its own section and then handle the (very small) performance cost of doing so by a relaxation pass in the linker. You can't safely remove any part of an input section, no matter what relocation information you think you have.

Signature · 2 lines
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer
#38 lore raw
In reply to Petr Vandrovec
Quoted 3 lines · David S. Miller

As a side note, this thing is so tiny (less than 4K on sparc64!) so why don't we just include it unconditionally instead of having all of this "turn it on for these drivers" stuff?

Does that 4K include the BE and LE crc tables?

<shrug> I don't mind much either way, except that I am general resistant to "turn this on unconditionally" for bloat reasons. [ie. its a reflex :)]

Jeff

#39 lore raw

remember that CML2 claims to be able to detect that it needs to be on and turn it on when needed

David Lang

Quoted 29 lines · Jeff Garzik

Date: Thu, 31 Jan 2002 18:08:42 -0500
From: Jeff Garzik <[email protected]>
To: David S. Miller <[email protected]>
Cc: [email protected], [email protected], [email protected],
[email protected], [email protected], [email protected]
Subject: Re: [PATCH] Re: crc32 and lib.a (was Re: [PATCH] nbd in 2.5.3
does not

Quoted 3 lines · David S. Miller

As a side note, this thing is so tiny (less than 4K on sparc64!) so why don't we just include it unconditionally instead of having all of this "turn it on for these drivers" stuff?

Does that 4K include the BE and LE crc tables?

<shrug> I don't mind much either way, except that I am general resistant to "turn this on unconditionally" for bloat reasons. [ie. its a reflex :)]

Jeff

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/