Stop printk printing non-printable chars

Hi,

I have had problems recently with the output from dmesg. Somewhere in the depths of ACPI (drivers/acpi/tables.c:104) the header->asl_compiler_id contained non-printable characters, and it made xterm stop displaying any more output. dmesg|less had to be used as less filters out the duff chars.

The main problem seems to be in ACPI, but I don't see any reason for printk to even consider printing _any_ non-printable characters at all. It makes all characters out of the range 32..126 (except for newline) print as a '?'.

Patch is for 2.6.7.

Matthew

@@ -14,6 +14,8 @@
  *     [email protected]
  * Rewrote bits to get rid of console_lock
  *	01Mar01 Andrew Morton <[email protected]>
+ * Stop emit_log_char from emitting non-ASCII chars.
+ *  Matthew Newton, 18 June 2004 <[email protected]>
  */
 
 #include <linux/kernel.h>

@@ -538,7 +540,11 @@
 			}
 			log_level_unknown = 0;
 		}
-		emit_log_char(*p);
+		if (p[0] != '\n' && (p[0] < 32 || p[0] > 126)) {
+			emit_log_char('?');
+		} else {
+			emit_log_char(*p);
+		}
 		if (*p == '\n')
 			log_level_unknown = 1;
 	}

How about emitting them as \xxx, so that you see what they are. And using a case-statement to make it easy and clear when to do exceptions (I think we should accept \t too, no?).

Linus

Would there be any reason not to allow all the standard C escape sequences
- true, they are hardly used atm (I see a few \f uses with grep, but not much else), but it's not unthinkable they could be useful somewhere in some cases (I'm thinking \f could be useful for console on line-printer for example, \a could be useful for critical errors on boxes without a monitor - or maybe that's all too far fetched)...

\a alert (bell)
\b backspace
\f formfeed
\n newline
\r carriage return
\t horizontal tab
\v vertical tab
\\ backslash
\? question mark
\' single quote
\" double quote

(the last few are in the 32..126 range, just listing for completeness)... none of them should cause trouble on output, so little reason to exclude them if someone find a use for them at some point - or am I not making sense?

--
Jesper Juhl <[email protected]>

I must admit, I don't think I've even seen a tab before (not that you'd actually _see_ a tab). Oh, grep tells me that powernow uses it. By the time that gets through syslog it's changed into "^I", so it would probably be better to not actually use tabs, either (or fix syslog).

New patch below outputs as \xxx if it's not a "nice" character. "Nice" is now 32..126, \n and \t.

@@ -14,6 +14,8 @@
  *     [email protected]
  * Rewrote bits to get rid of console_lock
  *	01Mar01 Andrew Morton <[email protected]>
+ * Stop emit_log_char from emitting non-ASCII chars.
+ *  Matthew Newton, 18 June 2004 <[email protected]>
  */
 
 #include <linux/kernel.h>

@@ -472,6 +474,17 @@
 }
 
 /*
+ * Emit character in numeric (octal) form
+ */
+static void emit_log_char_octal(char c)
+{
+	emit_log_char('\\');
+	emit_log_char(((c >> 6) & 3) + '0');
+	emit_log_char(((c >> 3) & 7) + '0');
+	emit_log_char((c & 7) + '0');
+}
+
+/*
  * Zap console related locks when oopsing. Only zap at most once
  * every 10 seconds, to leave time for slow consoles to print a
  * full oops.

@@ -538,7 +551,17 @@
 			}
 			log_level_unknown = 0;
 		}
-		emit_log_char(*p);
+		switch (*p) {
+			case '\n':
+			case '\t':
+				emit_log_char(*p);
+				break;
+			default:
+				if (*p > 31 && *p < 127)
+					emit_log_char(*p);
+				else
+					emit_log_char_octal(*p);
+		}
 		if (*p == '\n')
 			log_level_unknown = 1;
 	}

Matthew

Hi,

Don't most of us handle hex easier than octal ? I'd prefer to read \xE9 than \351, but that's only personal taste.

Umm, they're not sensible. (You're mixing up word meanings.) Log levels are not a big problem, because the log level is always printed first. Usually.

IMHO, the stuff should make sense to a human reader, and it should be possible to figure out quickly and unambiguously what the random bit pattern that inadvertently got printed actually *is*.

That means: Escape anything <\x31 except \n. Escape \\. Escape anything with the 8th bit set. Use hex escapes \x## -- let's face it, they're as long as octal and easier to read these days.

Matthias Urlichs

I've been meaning to fix that for a while, and kept forgetting about it. I just fixed it in my local cpufreq tree, and will push it along with the next lot of updates.

Thanks,

Dave

No, you're not. ;-)

Reason: They're not intended to be meaningful. If the kernel prints them, the reason isn't that somebody actually used an \a or \v in there, so doing that isn't helpful. (Quick, what's the ASCII for \v?)

Matthias Urlichs

Ok, I had a feeling that might be so. But I did not intend them to be printed as '"meaningful" C escapes', I meant "why filter out \v or \f, someone might find a clever use for them and they do no real harm otherwhise"...

What I meant was not for the kernel to attempt to print something like \a , but it could be useful for it's original purpose of making a sound.. If it's simply filtering out what goes to the screen (log, serial line, whatever), but not preventing other uses, then my comments made no sense... and 0x0B is \v I believe...

--
Jesper Juhl <[email protected]>

On the console, from the kernel? No such use exists today.

IMHO: Filter them out. If (big IF, methinks) somebody thinks of something that actually makes sense, they can add an exception.

Matthias Urlichs

On Fri, 2004-06-18 21:53:55 +0100, [email protected] <[email protected]> wrote in message <[email protected]>:

Right.

It's dandy if you pump out some data via serial link.

I don't see why that's needed. I'd say let's better fix ACPI to put those strings as a hexdump or something like that.

   Jan-Benedict Glaw       [email protected]    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

There's actually some other cases where doing the escaping would be a good idea.

Try grabbing an old a.out binary, renaming it to something like \n<0>Oops: ,running it and see what happens...

Combined with an old-enough terminal emulator there could actually be a bit more trouble than a messed-up screen...

Pekka Pietikainen

Is printk ever used to send anything out via a serial link? I assumed it was only kernel log messages (that should really be fairly sane). Log messages sent to serial printer, etc, don't want dodgy chars in them that may mess up the printer, do they?

Looking at the ACPI code (and not understanding it too well) it looks like this data is retrieved from the BIOS, but is only printed here for info and not actually used anywhere. In this case, I'd think there isn't a lot of point checking for data correctness in the ACPI code. As someone else pointed out, though, other things can cause the kernel log to print nasty chars that are unwanted, so there should really be a check here anyway.

The string has just been through the equivalent of sprintf, so I guess this is hardly going to slow it down much more. Used p[0] to look tidier, matching another "if" statement 5 lines up. In new patch used *p, as it doesn't really matter.

Thanks,

Matthew

On Sat, 2004-06-19 01:03:30 +0100, [email protected] <[email protected]> wrote in message <[email protected]>:

printk() is a printf() like interface to the log buffer. From there, all console drivers are fed, and in development (not for office or private use), you often attach a seruak cibsike to capture Oops messages and the like.

MfG, JBG

   Jan-Benedict Glaw       [email protected]    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

Please don't do that -- it makes printing UTF-8 impossible. While I'd not argue that now is the time to start outputting UTF-8 all over the place, I wouldn't accept that it's a good time to _prevent_ it either, as your patch would do.

If you want to post-process printk output, don't do it in the kernel.

I'd suggest that in this instance you should be fixing the ACPI code instead, so it doesn't print the characters to which you object.

dwmw2

Please forgive me if I'm wrong on this, but I seem to remember reading something a while ago indicating that the kernel is and always will be internally English (i.e. debugging messages and the like) as there is no need to bloat it with many different languages (that can be done in userspace). As printk is really just a log system, I personally don't see any way that it should ever print anything other than ASCII.

(Yes, of course some parts of the kernel, like filesystems, do need to be able to handle UTF-8 etc, but not all.)

Matthew

english != no-UTF8.

Names of people and things still can be UTF8 ....

It's very naïve of you to think that English means nothing but ASCII. Non-ASCII characters play a very important rôle even in English communication.

dwmw2

A LKML meta-patch:

@On Sat, 2004-06-19 at 16:49 +0100, [email protected] wrote:
Please forgive me if I'm wrong on this, but I seem to remember reading something a while ago indicating that the kernel is and always will be
-internally English (i.e. debugging messages and the like) as there is no +internally that subset of English representable in 7-bit ASCII +(i.e. debugging messages and the like) as there is no
need to bloat it with many different languages (that can be done in userspace). As printk is really just a log system, I personally don't see any way that it should ever print anything other than ASCII.

Thank you for illustrating the point that maximizing portability and compatibility favors simplicity. --
Jeff Woods <[email protected]>

[...]

Messages including user-level stuff (file names, ...) could very well be UTF-8.

Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513