CONFIG_TINY

Hi,

Inspired by the recent lowmem threads on l-k and in loose conjunction with acme, I am trying to do a CONFIG_TINY patchset which would reduce the kernel image size and memory footprint. Below is my list of ideas, collected from the aforementioned threads and from acme's input. Note that some of these are already being persued by other people (notably [by me] Andrew Morton).

Not all, if any, of this will ever be fit to go into the kernel proper. This is fine, since that is not the primary purpose for me right now. Starting off, the idea is to explore whether certain things are doable or not and what their actual payoffs are. From there cleanliness will be considered. I expect to learn as I go, tying neatly into the 'cleanliness later' approach :)

Nevertheless, some of these are also painful enough that I won't do them if directly told that they never will makes it into the kernel (the printk and __initstr changes comes to mind here) (acme, I think you wrote that you had gotten encouraging noises from linus on the __initstr one?)

The purpose of this mail is to get feedback in form of further areas where kernel size could be reduced, to learn if others are looking at some of these points and to see if I get some of the aforementioned 'never going to make it' noises.

Anyway, the list, terse form (there is no other):

Compile time:
o acme's __initstr (mark strings from __init functions as __initdata).
o reduce usage of prinkt in kernel by #defining iprintk for INFO messages etc and let the desired (minimum) logging level be decided at compile time.
o configure out #! exec stuff (Mentioned by Alan Cox in the 'end of embedded linux' thread).
o configure out procfs in ide (arjan's CONFIG_SMALL from redhat kernels) (I might have missed the point here, I only inspected briefly). Make procfs configurable for the IDE subsystem?
o reduce usage of inlines in the kernel. (Andrew Morton has an old patch I need to play a bit more with).
o SWAP and BLOCK_DEV as modules

Runtime:
o vfsmounts hash reduced to one page
o TCP(networking?) hashes reduced
o mempools reduced (Andrew Morton seems to be attacking this)

I have made a few crude, preliminary stabs at patches for some of the above against 2.5.43, applies against 44 as well. The noswap and noinline ones have been boot tested, the printk one compile tested. Comments welcome:

Config is 'allnoconfig', base kernel size is

text data bss dec hex filename
478401 50720 254816 783937 bf641 vmlinux

No swap: I stubbed out all swap calls in mm/ by jackhammer. New kernel size:

text data bss dec hex filename
469133 50433 252192 771758 bc6ae vmlinux

(Patch at www.jaquet.dk/kernel/config_tiny/2.5.43-noswap)

printk: So far I have defined {i,d,n,w}prink corresponding to KERN_ {INFO,DEBUG,NOTICE,WARNING} and have converted files to use this. Higher levels are left untouched.

Comments:
1) It may seem inconsistent to have iprintk("Fine day today") followed by printk(KERN_CRIT "Aiee. Drive on fire!!"). But converting _all_ printks seems quite intrusive. Or I am just waffling here?
2) The WARNING level is included in this cleanup since it is the default log level and lots of places dont specify a log level. If higher log levels are not converted to this printk format, it might make sense to make these places use KERN_WARNING instead of letting these be configured away at compile time?
3) The current names are short. However, the dprintk is troublesome for obvious reasons. Currently, I am using raa_printk :) but that would change. printk_info etc have been suggested as an alternative.

File touched so far is kernel/*, init/*, fs/*, ipc/*, mm/* and kernel.h (where I stuck the #defines).
New kernel size:

text data bss dec hex filename
478109 50337 254560 783006 bf29e vmlinux (woo! :)

(Patch at www.jaquet.dk/kernel/config_tiny/2.5.43-printk)

Reduce inlines: This is an old patch I got from Andrew Morton, eliminating some inlines in fs/, mm/, kernel/ and lib/. So far I have only brought it forward to 2.5.43, I plan to do a more aggressive version soon.

text data bss dec hex filename
474129 50720 254816 779665 be591 vmlinux

(Patch at www.jaquet.dk/kernel/config_tiny/2.5.43-noinlines)

Your comments will be appreciated.

Regards,
Rasmus

modules won't work I guess :) but allowing to disable them is a good idea. Not that swap depends on the block device code, though.

I have implemented CONFIG_SWAP for the uClinux patches, but there are still some small bits missing to allow CONFIG_MMU && !CONFIG_SWAP

Should be made smaller unconditionally, IMHO.

The module approach was just a happy thought jotted down from an IRC log. I guess there is a number of reasons that that won't work.

I did not meant to imply that SWAP and block was related, just that both (perhaph) could be made modules.

Thanks for your comments,
Rasmus

kernel.h already has #defines for pr_debug and pr_info. perhaps all 8 current KERN_xxx's could be pr_#define'd and the printks converted during 2.7+

I think this should be done together with Larry Kessler's kernel logging proposal.

Possibly. Certainly fine by me.

I only followed this discussion superficially. Would you mind providing a link for the thread or giving me a short roundup of the conclusion on that?

Thanks,
Rasmus

Instead of doing it this way, why not use a preprocessor scheme like this one which doesn't require you to patch anything (my apologies for it being somewhat obtuse):

#define DBG_LVL 1
#define UNTAGGED_DBG_LVL 0

#define KERN_MSG_LVL 5
#define KERN_MSG_STRING "<5>"

#define printk(a, arg...) do { \
        { \
		if(DBG_LVL < UNTAGGED_DBG_LVL) \
			switch(UNTAGGED_DBG_LVL) { \
				default: \
					_printk("" a, ##arg); \
			} \
		} \
	} while(0)

#define KERN_WARNING 				); \
				case UNTAGGED_DBG_LVL: \
			} \
			if(DBG_LVL < KERN_MSG_LVL) { \
				_printk(KERN_MSG_STRING

(Yes, I enjoyed writing these macros)

The strings themselves will still be included, but you should be able to run a string pruning program on the output objects since there is no longer any reference to them at all.

Does this make more sense? Maintaining a patch that changes all the printks in the world is going to hurt!

-J

The previous patch added some complex, ugly macros to replace printk() in the kernel in a way which allows printk() calls to be selectively disabled at compile time. These macros cannot work in all situations, because some parts of the kernel depend on eg. KERN_ERR actually being a simple string.

This patch modifies certain kernel sources to fix this. It either removes the dependency, or changes these places to use a new KERN_ERR_STR etc. macro.

This is not at all a complete patch - it was just the minimal change required to get the core kernel to build with my own particular configuration. I'm only including it since the previous patch doesn't build at all without it.

-J

@@ -379,8 +379,8 @@
 		goto err1;
 	if (mod_user_size < (unsigned long)&((struct module *)0L)->persist_start
 	    || mod_user_size > sizeof(struct module) + 16*sizeof(void*)) {
-		printk(KERN_ERR "init_module: Invalid module header size.\n"
-		       KERN_ERR "A new version of the modutils is likely "
+		printk(KERN_ERR     "init_module: Invalid module header size.\n"
+		       KERN_ERR_STR "A new version of the modutils is likely "
 				"needed.\n");
 		error = -EINVAL;
 		goto err1;

@@ -658,7 +658,7 @@
 	unsigned long flags, void (*ctor)(void*, kmem_cache_t *, unsigned long),
 	void (*dtor)(void*, kmem_cache_t *, unsigned long))
 {
-	const char *func_nm = KERN_ERR "kmem_create: ";
+	const char *func_nm = "kmem_create: ";
 	size_t left_over, align, slab_size;
 	kmem_cache_t *cachep = NULL;
 

@@ -676,13 +676,13 @@
 #if DEBUG
 	if ((flags & SLAB_DEBUG_INITIAL) && !ctor) {
 		/* No constructor, but inital state check requested */
-		printk("%sNo con, but init state check requested - %s\n", func_nm, name);
+		printk(KERN_ERR "%sNo con, but init state check requested - %s\n", func_nm, name);
 		flags &= ~SLAB_DEBUG_INITIAL;
 	}
 
 	if ((flags & SLAB_POISON) && ctor) {
 		/* request for poisoning, but we can't do that with a constructor */
-		printk("%sPoisoning requested, but con given - %s\n", func_nm, name);
+		printk(KERN_ERR "%sPoisoning requested, but con given - %s\n", func_nm, name);
 		flags &= ~SLAB_POISON;
 	}
 #if FORCED_DEBUG

@@ -717,7 +717,7 @@
 	if (size & (BYTES_PER_WORD-1)) {
 		size += (BYTES_PER_WORD-1);
 		size &= ~(BYTES_PER_WORD-1);
-		printk("%sForcing size word alignment - %s\n", func_nm, name);
+		printk(KERN_ERR "%sForcing size word alignment - %s\n", func_nm, name);
 	}
 	
 #if DEBUG

@@ -788,7 +788,7 @@
 	} while (1);
 
 	if (!cachep->num) {
-		printk("kmem_cache_create: couldn't create cache %s.\n", name);
+		printk(KERN_ERR "kmem_cache_create: couldn't create cache %s.\n", name);
 		kmem_cache_free(&cache_cache, cachep);
 		cachep = NULL;
 		goto opps;

@@ -56,7 +56,7 @@
 #include <linux/blk.h>
 
 #define DEBUG 0
-#define dprintk(x...) ((void)(DEBUG && printk(x)))
+#define dprintk(x...) do { if(DEBUG) printk(x); } while(0)
 
 
 #ifndef MODULE

@@ -2977,7 +2977,7 @@
 	struct list_head *tmp, *rtmp;
 
 
-	dprintk(KERN_INFO "md: recovery thread got woken up ...\n");
+	dprintk(KERN_INFO_STR "md: recovery thread got woken up ...\n");
 
 	ITERATE_MDDEV(mddev,tmp) if (mddev_lock(mddev)==0) {
 		if (!mddev->raid_disks || !mddev->pers || mddev->ro)

@@ -3054,7 +3054,7 @@
 	unlock:
 		mddev_unlock(mddev);
 	}
-	dprintk(KERN_INFO "md: recovery thread finished ...\n");
+	dprintk(KERN_INFO_STR "md: recovery thread finished ...\n");
 
 }
 

@@ -212,9 +212,9 @@
 			      const char *routine)
 {
 #ifdef TTY_PARANOIA_CHECK
-	static const char badmagic[] = KERN_WARNING
+	static const char badmagic[] = KERN_WARNING_STR
 		"Warning: bad magic number for tty struct (%s) in %s\n";
-	static const char badtty[] = KERN_WARNING
+	static const char badtty[] = KERN_WARNING_STR
 		"Warning: null TTY for (%s) in %s\n";
 
 	if (!tty) {

@@ -3653,7 +3653,7 @@
 		if (name) {
 			const char * prepend = ",";
 			if (first) {
-				prepend = KERN_INFO "Floppy drive(s):";
+				prepend = KERN_INFO_STR "Floppy drive(s):";
 				first = 0;
 			}
 			printk("%s fd%d is %s", prepend, drive, name);

@@ -338,7 +338,7 @@
 	} else {
 		rpc_clear_running(task);
 		if (task->tk_callback) {
-			dprintk(KERN_ERR "RPC: %4d overwrites an active callback\n", task->tk_pid);
+			dprintk(KERN_ERR_STR "RPC: %4d overwrites an active callback\n", task->tk_pid);
 			BUG();
 		}
 		task->tk_callback = action;

@@ -909,7 +909,7 @@
 	task->tk_client->cl_stats->rpcgarbage++;
 	if (task->tk_garb_retry) {
 		task->tk_garb_retry--;
-		dprintk(KERN_WARNING "RPC: garbage, retrying %4d\n", task->tk_pid);
+		dprintk(KERN_WARNING_STR "RPC: garbage, retrying %4d\n", task->tk_pid);
 		task->tk_action = call_encode;
 		return NULL;
 	}

@@ -715,7 +715,7 @@
 	 * tell us anything. For now just warn about unpriv connections.
 	 */
 	if (ntohs(sin.sin_port) >= 1024) {
-		dprintk(KERN_WARNING
+		dprintk(KERN_WARNING_STR
 			"%s: connect from unprivileged port: %u.%u.%u.%u:%d\n",
 			serv->sv_name, 
 			NIPQUAD(sin.sin_addr.s_addr), ntohs(sin.sin_port));

@@ -1304,7 +1304,7 @@
 		kfree(svsk);
 	} else {
 		spin_unlock_bh(&serv->sv_lock);
-		dprintk(KERN_NOTICE "svc: server socket destroy delayed\n");
+		dprintk(KERN_NOTICE_STR "svc: server socket destroy delayed\n");
 		/* svsk->sk_server = NULL; */
 	}
 }

@@ -584,15 +584,16 @@
 		fl.fl4_src = 0;
 
 	if ((err=ip_route_output_key(&rt, &fl)) != 0) {
+#ifdef CONFIG_IP_ROUTE_FWMARK
+#define FWMARK (*pskb)->nfmark
+#else
+#define FWMARK 0UL
+#endif
 		printk("route_me_harder: ip_route_output_key(dst=%u.%u.%u.%u, src=%u.%u.%u.%u, oif=%d, tos=0x%x, fwmark=0x%lx) error %d\n",
 			NIPQUAD(iph->daddr), NIPQUAD(iph->saddr),
 			(*pskb)->sk ? (*pskb)->sk->bound_dev_if : 0,
 			RT_TOS(iph->tos)|RTO_CONN,
-#ifdef CONFIG_IP_ROUTE_FWMARK
-			(*pskb)->nfmark,
-#else
-			0UL,
-#endif
+                        FWMARK,
 			err);
 		goto out;
 	}

@@ -1882,7 +1882,8 @@
 static inline void unix_sysctl_unregister(void) {}
 #endif
 
-static char banner[] __initdata = KERN_INFO "NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.\n";
+static char banner[] __initdata = KERN_INFO_STR 
+	"NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.\n";
 
 static int __init af_unix_init(void)
 {

@@ -35,7 +35,7 @@
 static void tcp_delack_timer(unsigned long);
 static void tcp_keepalive_timer (unsigned long data);
 
-const char timer_bug_msg[] = KERN_DEBUG "tcpbug: unknown timer value\n";
+const char timer_bug_msg[] = KERN_DEBUG_STR "tcpbug: unknown timer value\n";
 
 /*
  * Using different timers for retransmit, delayed acks and probes

@@ -323,7 +323,7 @@
 	movl %eax,%ds
 	movl %eax,%es
 	pushl $int_msg
-	call printk
+	call _printk
 	popl %eax
 	popl %ds
 	popl %es

@@ -756,7 +756,7 @@
 			NO_MATCH, NO_MATCH, NO_MATCH
 			} },
 
-	{ print_if_true, KERN_WARNING "IBM T23 - BIOS 1.03b+ and controller firmware 1.02+ may be needed for Linux APM.", {
+	{ print_if_true, KERN_WARNING_STR "IBM T23 - BIOS 1.03b+ and controller firmware 1.02+ may be needed for Linux APM.", {
 			MATCH(DMI_SYS_VENDOR, "IBM"),
 			MATCH(DMI_BIOS_VERSION, "1AET38WW (1.01b)"),
 			NO_MATCH, NO_MATCH

Here is an untested patch example against 2.5.44 to add some ugly printk macros to the kernel, to reduce kernel image size for tiny systems.

I verified that this builds a kernel image on my system (if I fix some 2.5.44 build failures, obviously), with the fixup patch I'm sending next to modify some of the sources for the new printk style.

This patch is hardly complete, and is just an example of using macros to selectively cut out printk string/code bloat from the kernel. By changing the TAG_PRINTK_LVL and UNTAG_PRINTK_LVL numbers, selective levels of printks can be compiled out of the kernel.

With my config and gcc-3.1 (which has dead-string pruning), I got these build-size numbers with different TAG_PRINTK values:

Define printk like normal (#define printk(a,arg...) _printk(a, ##arg)) System is 1093 kB

All levels on (7 TAG/7 UNTAG)
System is 1090 kB (3k less, not sure why)

Only tagged warning or above (4/7)
System is 1024 kB

Only tagged alert or above (1/7)
System is 1002 kB

Only tagged emergency (0/7)
System is 1002 kB

All printk off (-1/7)
System is 997 kB

Kill printk completely (#define printk(a...) 0)
System is 997 kB

There are several obvious problems with this implementation:

* The KERN_ERR etc. macros can't work in all situations, since they are no longer simple strings. The second patch fixes some of these situations.
* Changing the printk symbol to be _printk is probably a bad idea. It would be best to keep the symbol name the same in a real implementation.
* It's somewhat hard to read.

Comments?

-J

@@ -13,6 +13,7 @@
 #include <linux/types.h>
 #include <linux/compiler.h>
 #include <asm/byteorder.h>
+#include <linux/printk.h>
 
 /* Optimization barrier */
 /* The "volatile" is due to gcc bugs */

@@ -29,15 +30,6 @@
 
 #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
 
-#define	KERN_EMERG	"<0>"	/* system is unusable			*/
-#define	KERN_ALERT	"<1>"	/* action must be taken immediately	*/
-#define	KERN_CRIT	"<2>"	/* critical conditions			*/
-#define	KERN_ERR	"<3>"	/* error conditions			*/
-#define	KERN_WARNING	"<4>"	/* warning conditions			*/
-#define	KERN_NOTICE	"<5>"	/* normal but significant condition	*/
-#define	KERN_INFO	"<6>"	/* informational			*/
-#define	KERN_DEBUG	"<7>"	/* debug-level messages			*/
-
 struct completion;
 
 #ifdef CONFIG_DEBUG_KERNEL

@@ -78,9 +70,6 @@
 
 extern int session_of_pgrp(int pgrp);
 
-asmlinkage int printk(const char * fmt, ...)
-	__attribute__ ((format (printf, 1, 2)));
-
 extern int console_loglevel;
 
 static inline void console_silent(void)

@@ -296,7 +296,7 @@
 		if (copy_from_user(lbuf, buf, len))
 			break;
 		lbuf[len] = '\0';
-		error = printk("%s", lbuf);
+		error = _printk("%s", lbuf);
 		break;
 	default:
 		error = -EINVAL;

@@ -419,7 +419,7 @@
  * then changes console_loglevel may break. This is because console_loglevel
  * is inspected when the actual printing occurs.
  */
-asmlinkage int printk(const char *fmt, ...)
+asmlinkage int _printk(const char *fmt, ...)
 {
 	va_list args;
 	unsigned long flags;

@@ -488,7 +488,7 @@
 out:
 	return printed_len;
 }
-EXPORT_SYMBOL(printk);
+EXPORT_SYMBOL(_printk);
 
 /**
  * acquire_console_sem - lock the console system for exclusive use.

[...]

Memo to me: Dont use the kernel compiled to check #define correctness for enabling printk output to measure the kernel size when prinkt is disabled. New, better kernel size:

text data bss dec hex filename
475629 50913 252512 779054 be32e vmlinux

Regards,
Rasmus

Well, actually the main point is to get the CONFIG_TINY symbol into mainline, even if it doesn't do much at first. That in itself will be enough to inspire various artists to contribute their own creative minimalizing hacks to the subsystems in which they have specific expertise.

I'll bet you beer that some of the minimized subsystems eventually supplant their more bloated brethren, as superior in every way.

Daniel