Documentation of kernel messages
Subsystems (2)
Greetings,
The operation of a Linux system sometimes requires to decode the meaning of a specific kernel message, e.g. an error message of a driver. Especially on our mainframe zSeries platform system administrators want to have descriptions for Linux kernel messages. They are used to that, because all other operating systems on that platform like z/OS, z/VM or z/VSE have message catalogs with detailed descriptions about the semantics of the messages.
In general we think, that also for Linux it is a good thing to have documentation for the most important kernel/driver messages. Even kernel hackers not always are aware of the meaning of kernel messages for components, which they don't know in detail. Most of the messages are self explaining but sometimes you get something like "Clocksource tsc unstable (delta = 7304132729 ns)" and you wonder if your system is going to explode.
Unfortunately currently there is no general infrastructure in the Linux kernel for the documentation of messages. I worked on a proposal, how that could be implemented in an easy way using the already existing kernel-doc infrastructure and using printk. The proposal is as follows
1. We use message identifiers in order to map messages to message descriptions. A message identifier consists out of a component name and within that component unique message number.
2. Messages and message descriptions are maintained together in the kernel sources in order to keep them up to date. Messages catalog are generated automatically for exactly one kernel level.
3. A special tool checks, if messages are not documented or if there are message descriptions without corresponding messages.
4. We use the already existing kernel-doc tool to generate an online message catalog or e.g. a pdf for offline documentation.
Current prototype implementation:
=================================
The structure of a kernel message is: <component>.<msg number>: <msg>
* component: Name of the kernel or driver component e.g. "pci", "ide", etc.
* msg number: Within the component unique number of a kernel message.
* msg: printk message
New macros KMSG_ERR(), KMSG_WARN(), etc. are defined, which have to be used in printk. These macros have as parameter the message number and are using a per c-file defined macro KMSG_COMPONENT.
Example: Define message 2 in component "kmsgtest":
#define KMSG_COMPONENT "kmsgtest"
void f(void)
{
printk(KMSG_ERR(1) "device %x not online\n", devno);
}The output of that kernel message would be:
"kmsgtest.1: device 4711 not online"
The messages have to be documented within the C source file in the following way:
/**
* message
* @0: device number of device.
*
* Description:
* An operation has been performed on the msgtest device, but the
* device has not been set online. Therefore the operation failed *
* User Response:
* Operator should set device online.
* Issue "chccwdev -e <device number>".
*
*/
KMSG_DOC(kmsgtest, 2, "Device %x not online");
I created a patch for the kernel-doc tool so it can be used to generate a catalog of all kernel messages:
Quoted 2 lines
Quoted 2 lines
kernel-doc -man
kmsgtest.c> kmsgtest.2
man ./kmsgtest.2
# Kernel API(9) Linux Messages Kernel API(9)
#
# MESSAGE:
# kmsgtest.2: "device %x not online"
#
# Parameter:
# 1 Device number of device.
#
# Description:
# An operation has been performed on the msgtest device, but
# the device has not been set online. Therefore the operation failed.
#
# User Response:
# Operator should set device online.
# Issue "chccwdev -e <device number>".
#
# May 2007 kmsgtest.2 Kernel API(9)
A nice thing would be to include the online kernel message catalog in the kernel rpm. One possibility for that would be to have one man page per message. If an operator finds the message kmsgtest.2 in var/log/messages and wants to know what the message means, he simply issues "man kmsgtest.2" and gets the description.
To ensure that all messages are documented and there are no message descriptions without corresponding messages, a checker tool is provided. To enable message checking, in the toplevel Makefile the following has to be added:
CHECK = scripts/kmsg_check.pl check
To enable message checking during kernel build, the "C" option has to be used:
Quoted 1 line
Quoted 1 line
make modules C=1
CHK include/linux/version.h CHK include/linux/utsrelease.h CHECK drivers/kmsgtest/kmsgtest.c
drivers/kmsgtest/kmsgtest.c: Missing description for: kmsgtest.1 drivers/kmsgtest/kmsgtest.c: Description without message for: kmsgtest.3
Please note, that the patch for the kernel-doc and kmsg-doc tools is just a prototype and is neither complete nor perfect.
Michael
---
Makefile | 5 drivers/Makefile | 1 drivers/kmsgtest/Makefile | 5 drivers/kmsgtest/kmsgtest.c | 59 +++++++++++ include/linux/kmsg.h | 32 ++++++ scripts/kernel-doc | 116 +++++++++++++++++++++- scripts/kmsg-doc | 231 ++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 445 insertions(+), 4 deletions(-)
Makefile +2 -3
@@ -293,9 +293,8 @@ DEPMOD = /sbin/depmod KALLSYMS = scripts/kallsyms PERL = perl -CHECK = sparse - -CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ -Wbitwise $(CF) +CHECK = scripts/kmsg-doc check +CHECKFLAGS = MODFLAGS = -DMODULE CFLAGS_MODULE = $(MODFLAGS) AFLAGS_MODULE = $(MODFLAGS)
drivers/Makefile +1 -0
@@ -8,6 +8,7 @@ obj-$(CONFIG_PCI) += pci/ obj-$(CONFIG_PARISC) += parisc/ obj-$(CONFIG_RAPIDIO) += rapidio/ +obj-m += kmsgtest/ obj-y += video/ obj-$(CONFIG_ACPI) += acpi/ # PnP must come after ACPI since it will eventually need to check if acpi
drivers/kmsgtest/Makefile +5 -0
@@ -0,0 +1,5 @@ +# +# Makefile for kernel message test module +# + +obj-m += kmsgtest.o
drivers/kmsgtest/kmsgtest.c +59 -0
59 lines not on this page · raw message
include/linux/kmsg.h +32 -0
@@ -0,0 +1,32 @@ +#ifndef _LINUX_KMSG_H +#define _LINUX_KMSG_H + +#ifdef __KMSG_CHECKER + +#define KMSG_EMERG (num) __KMSG_CHECK(EMERG, num) +#define KMSG_ALERT (num) __KMSG_CHECK(ALERT, num) +#define KMSG_CRIT (num) __KMSG_CHECK(CRIT, num) +#define KMSG_ERR (num) __KMSG_CHECK(ERR, num) +#define KMSG_WARNING (num) __KMSG_CHECK(WARNING, num) +#define KMSG_NOTICE (num) __KMSG_CHECK(NOTICE, num) +#define KMSG_INFO (num) __KMSG_CHECK(INFO, num) +#define KMSG_DEBUG (num) __KMSG_CHECK(DEBUG, num) + +#define KMSG_DOC (comp, num, str) __KMSG_DOC(comp, num, str) + +#else + +#define KMSG_EMERG (num) KERN_EMERG KMSG_COMPONENT "." #num ": " +#define KMSG_ALERT (num) KERN_ALERT KMSG_COMPONENT "." #num ": " +#define KMSG_CRIT (num) KERN_CRIT KMSG_COMPONENT "." #num ": " +#define KMSG_ERR (num) KERN_ERR KMSG_COMPONENT "." #num ": " +#define KMSG_WARNING (num) KERN_WARNING KMSG_COMPONENT "." #num ": " +#define KMSG_NOTICE (num) KERN_NOTICE KMSG_COMPONENT "." #num ": " +#define KMSG_INFO (num) KERN_INFO KMSG_COMPONENT "." #num ": " +#define KMSG_DEBUG (num) KERN_DEBUG KMSG_COMPONENT "." #num ": " + +#define KMSG_DOC (comp, num, str) + +#endif /* __KMSG_CHECKER */ + +#endif /* _LINUX_KMSG_H */
scripts/kernel-doc +115 -1
140 lines not on this page · raw message
scripts/kmsg-doc +231 -0
231 lines not on this page · raw message
Quoted 7 lines · holzheu
The operation of a Linux system sometimes requires to decode the meaning of a specific kernel message, e.g. an error message of a driver. Especially on our mainframe zSeries platform system administrators want to have descriptions for Linux kernel messages. They are used to that, because all other operating systems on that platform like z/OS, z/VM or z/VSE have message catalogs with detailed descriptions about the semantics of the messages.
I'm not sure we want to make Linux more like z/* in this regard. :)
The problem with your proposal is that every time a new message in the kernel is created or modified, you need somebody to go update that documentation. It's going to get out-of-sync very fast if this isn't done, and you either need to convince and teach each and every kernel contributor to follow your lead, or have a team of highly trained code monkeys to watch git-commits and resubmit documentation for every diff that touches a printk.
I think it's a great idea to go and update some of these obtuse kernel messages with more understandable wording. It might even be nice to update the Documentation/ about what kinds of messages are good or bad to have. But, I just don't think this is viable to convince everybody to do this. Do you have a list of printks that are causing your customers trouble?
Does every printk need one of these? What do we do with the existing printks? Who will convert them over? How do we enforce the rules if new printks must have documentation written for them? Who writes the documentation if the printk author does not?
If we have a catalog of kernel messages, it obviously needs to be versioned, but does that mean that we need to keep it in the kernel tree?
-- Dave
Quoted 11 lines · Dave Hansen
Quoted 7 lines · holzheu
The operation of a Linux system sometimes requires to decode the meaning of a specific kernel message, e.g. an error message of a driver. Especially on our mainframe zSeries platform system administrators want to have descriptions for Linux kernel messages. They are used to that, because all other operating systems on that platform like z/OS, z/VM or z/VSE have message catalogs with detailed descriptions about the semantics of the messages.
I'm not sure we want to make Linux more like z/* in this regard. :)
Quoted 7 lines
The problem with your proposal is that every time a new message in the kernel is created or modified, you need somebody to go update that documentation. It's going to get out-of-sync very fast if this isn't done, and you either need to convince and teach each and every kernel contributor to follow your lead, or have a team of highly trained code monkeys to watch git-commits and resubmit documentation for every diff that touches a printk.
This is no more and no less the same situation that we have with the kernel-doc documented functions/data in the kernel.
I like the concept that the description is kept close to the actual usage, the tool support and that in general looks like ordinary kernel-doc documentation.
And if people do not dare to update the kernel-doc documentation of a function then maybe they should not send patches in first place..
If we then really want to have the important printk's documented is another story.
Sam
Hi Dave,
On Wed, 2007-06-13 at 09:37 -0700, Dave Hansen wrote:
[snip]
Quoted 5 lines
I'm not sure we want to make Linux more like z/* in this regard. :)
The problem with your proposal is that every time a new message in the kernel is created or modified, you need somebody to go update that documentation.
You only have to document printks, which are using the KMSG macros. Since there are tons of self-explaining printks, most of them do not have to use them.
If you change the meaning of a KMSG printk and you don't update the documentation, it is the same thing like changing a kernel API function and forgetting to update the kernel-doc comment.
Quoted 5 lines
It's going to get out-of-sync very fast if this isn't
done, and you either need to convince and teach each and every kernel contributor to follow your lead, or have a team of highly trained code monkeys to watch git-commits and resubmit documentation for every diff that touches a printk.
It is a matter of discipline. If a device driver maintainer decides to document some messages using the KMSG macros, he has to take care that the documentation is up-to-date. Since the printks and the descriptions are maintained together in the kernel code, I think, that's doable.
And the checker tool can help to keep the descriptions up-to-date, since it produces warnings, if you use C=1 during the kernel build.
Quoted 5 lines
I think it's a great idea to go and update some of these obtuse kernel messages with more understandable wording. It might even be nice to update the
Documentation/about what kinds of messages are good or bad to have. But, I just don't think this is viable to convince everybody to do this.
Again, not everybody has to use KMSG printks.
Quoted 2 lines
Do you have a list of printks that are causing your customers trouble?
There are quite a lot of messages in our s390 device drivers which should be documented. Especially for character device drivers (e.g. tape), where the user has direct interaction with the device.
One Example from the tape device driver: "Another host has reserved the tape device"
We should document, how to find out the host, which reserved the tape and what has to be done on the other host in order to release the tape etc.
Quoted 2 lines
Does every printk need one of these? What do we do with the existing printks?
Again: Most printks don't need documentation. We can leave them as they are.
Quoted 3 lines
Who will convert them over? How do we enforce the rules if new printks must have documentation written for them? Who writes the documentation if the printk author does not?
The maintainer of the component (e.g. device driver) is responsible for the KMSG printks. If he accepts a patch, which modifies KMSG printks, he has to take care, that the patch also updates the KMSG_DOC part.
Quoted 3 lines
If we have a catalog of kernel messages, it obviously needs to be versioned, but does that mean that we need to keep it in the kernel tree?
For each kernel a separate message catalog can be automatically created. This message catalog fits exactly to one kernel and should be installed together with the kernel.
Michael
Quoted 3 lines · holzheu
They are used to that, because all other operating systems on that platform like z/OS, z/VM or z/VSE have message catalogs with detailed descriptions about the semantics of the messages.
25 years ago, I did OS/MVT and OS/VS1 for a living, so I know *all* about the infamous "What does IEF507E mean again?"...
Quoted 7 lines
In general we think, that also for Linux it is a good thing to have documentation for the most important kernel/driver messages. Even kernel hackers not always are aware of the meaning of kernel messages for components, which they don't know in detail. Most of the messages are self explaining but sometimes you get something like "Clocksource tsc unstable (delta = 7304132729 ns)" and you wonder if your system is going to explode.
This is probably best addressed by cleaning up the actual messages so they're a bit more informative.
Quoted 3 lines
New macros
KMSG_ERR(),KMSG_WARN(), etc. are defined, which have to be used in printk. These macros have as parameter the message number and are using a per c-file defined macroKMSG_COMPONENT.
Gaak. *NO*.
The *only* reason that the MVS and VM message catalogs worked at all is because each component had a message repository that went across *all* the source files - the instant you saw IEFnnns, you knew that IEF covered the job scheduler, nnn was a *unique* number, and s was a Severe/Warning/Info flag. IGG was always data management, and so on. This breaks horribly if you have 2 C files that define subtly different KMSG_COMPONENT values (or even worse, 2 or more duplicates).
[/usr/src/linux-2.6.22-rc4-mm2] find . -name '*.c' | wc -l 9959
[/usr/src/linux-2.6.22-rc4-mm2] find . -name '*.h' | wc -l 9933
[/usr/src/linux-2.6.22-rc4-mm2] find . -type d | wc -l 1736
You plan to maintain message uniqueness how?
[/usr/src/linux-2.6.22-rc4-mm2]1 find . -name '*.c' | sed -r 's?.*/([^/]*)?\1?' | sort | uniq -c | sort -nr | head
105 setup.c
90 irq.c
66 time.c
58 init.c
50 inode.c
39 io.c
38 pci.c
37 file.c
32 signal.c
32 ptrace.cLooks like you're going to have to embed a lot of the path in that KMSG_COMPONENT to make it unique - and you want to keep that message under 80 or so chars total.
Quoted 7 lines
/**
* message
* @0: device number of device.
*
* Description:
* An operation has been performed on the msgtest device, but the
* device has not been set online. Therefore the operation failed
If you don't understand 'Device /dev/foo offline', this description doesn't help any. And that's true for *most* of the kernel messages already - if you don't understand the message already, a paragraph explanation isn't going to help much. Consider the average OOPS message, which contains stuff like 'EIP=0x..'. Telling the user that EIP means Execution Instruction Pointer isn't likely to help - if they knew what the pointer *did*, they'd probably already know EIP.
Quoted 4 lines
*
* User Response:
* Operator should set device online.
* Issue "chccwdev -e <device number>".
And this is where the weakness of this scheme *really* hits. I've actually run into cases where an operator followed the listed "Operator Response" for a "device offline", and issued a 'VARY 0C0,ONLINE'. And then we got a flood of I/O errors because the previous shift downed the device because it was having issues. The response the operator *should* have done is "assign a different tape drive, like, oh maybe the operational ones at 0C1 through 0C4"...
And it's the same here - if you get a message that /dev/sdb1 has no media present, there's a good chance that you typo'ed, and meant /dev/sda1 or /dev/sdc1 So following the directions for 'sdb1 offline' and putting in a blank DVD because sdb is the DVD burner won't fix things if what you were trying to do is mkfs something on another disk... ;)
And while we're at it, I'll point out that any attempt to "fix" the kernel messages on this scale had *better* solve all the I18N problems while we're there....
Quoted 7 lines · On Wednesday 13 June 2007 12:50:55 [email protected] wrote:
Quoted 7 lines
In general we think, that also for Linux it is a good thing to have documentation for the most important kernel/driver messages. Even kernel hackers not always are aware of the meaning of kernel messages for components, which they don't know in detail. Most of the messages are self explaining but sometimes you get something like "Clocksource tsc unstable (delta = 7304132729 ns)" and you wonder if your system is going to explode.
Isn't this what the severity levels are for? Not just filtering messages out but telling you whether or not it's important?
Quoted 2 lines
This is probably best addressed by cleaning up the actual messages so they're a bit more informative.
Seconded.
The point of a diagnostic message is to convey information. Deep in the bowels of chipsets there are diagnostic messages that get spit out as hexcodes that some program needs to interpret, but that's why we have tools like http://kernel.org/pub/linux/kernel/people/davej/tools/parsemce.c and such.
If the diagnostic messages need a talmud, I see this is as more of a problem with said diagnostic messages, rather than excuse to add another layer.
Quoted 5 lines
Quoted 3 lines
New macros
KMSG_ERR(),KMSG_WARN(), etc. are defined, which have to be used in printk. These macros have as parameter the message number and are using a per c-file defined macroKMSG_COMPONENT.Gaak. *NO*.
If you're going to tweak printk, please make the darn thing take an integer first argument rather than a string first argument, so we can use clever macros to remove them at compile time rather than having them compiled in but not displaying.
This is one of those things buried down on my todo list. I think it can be converted incrementally
Going back to the original patch here, there are exactly two examples of what this brave new infrastructure would be used for:
Quoted 10 lines
drivers/kmsgtest/kmsgtest.c +10
-0+/** +* message +* @0: Device number of device +* @1: Status of device +* +* Description: +* Information message about the status of our virtual msgtest device. The +* following values for the status parameter are available. +* +* 0 - Device is offline
drivers/kmsgtest/kmsgtest.c line 45
Quoted 10 lines
drivers/kmsgtest/kmsgtest.c +10
-0+* +* 1 - Device is online +* +* 2 - Device is broken +* +* User Response: +* If device is broken, replace it or fix it. +*/ + +KMSG_DOC (kmsgtest,1 ,"device %x has status %i" );
This is just a thought: why not have the actual kprintf say "Device number %x is %s" where %s is "online", "offline", or "broken"?
I.E. have the kernel output a user readable message in the first place.
Review
Quoted 13 lines
+/**
+ * message
+ * @0: Device number of device.
+ *
+ * Description:
+ * An operation has been performed on the msgtest device, but the device has + * not been set online. Therefore the operation failed.
+ *
+ * User Response:
+ * Operator should set device online. Issue "chccwdev -e <device number>".
+ */(quoted diff) +2
-0+ +KMSG_DOC(kmsgtest, 2, "device %x not online");
kprintf(LEVEL "device %x not online, try 'chccwdev -e %x'", devnum, devnum)
Rob
Hi Valdis,
Quoted 7 lines · On Wed, 2007-06-13 at 12:50 -0400, [email protected] wrote:
Quoted 3 lines · holzheu
They are used to that, because all other operating systems on that platform like z/OS, z/VM or z/VSE have message catalogs with detailed descriptions about the semantics of the messages.
25 years ago, I did OS/MVT and OS/VS1 for a living, so I know *all* about the infamous "What does IEF507E mean again?"...
:-)
Quoted 10 lines
Quoted 7 lines
In general we think, that also for Linux it is a good thing to have documentation for the most important kernel/driver messages. Even kernel hackers not always are aware of the meaning of kernel messages for components, which they don't know in detail. Most of the messages are self explaining but sometimes you get something like "Clocksource tsc unstable (delta = 7304132729 ns)" and you wonder if your system is going to explode.
This is probably best addressed by cleaning up the actual messages so they're a bit more informative.
Of course that would be good, too.
But I think, that we sometimes have the dilemma, that we want to keep the printks short, but also want to provide as much information as possible.
If the information is to big for the printk itself, because you would need 10 lines to explain what happened, wouldn't it be good to have a place where to put that information?
Quoted 34 lines
Quoted 3 lines
New macros
KMSG_ERR(),KMSG_WARN(), etc. are defined, which have to be used in printk. These macros have as parameter the message number and are using a per c-file defined macroKMSG_COMPONENT.Gaak. *NO*.
The *only* reason that the MVS and VM message catalogs worked at all is because each component had a message repository that went across *all* the source files - the instant you saw IEFnnns, you knew that IEF covered the job scheduler, nnn was a *unique* number, and s was a Severe/Warning/Info flag. IGG was always data management, and so on. This breaks horribly if you have 2 C files that define subtly different
KMSG_COMPONENTvalues (or even worse, 2 or more duplicates).[
/usr/src/linux-2.6.22-rc4-mm2] find . -name '*.c' | wc -l 9959
[/usr/src/linux-2.6.22-rc4-mm2] find . -name '*.h' | wc -l 9933
[/usr/src/linux-2.6.22-rc4-mm2] find . -type d | wc -l 1736You plan to maintain message uniqueness how?
[/usr/src/linux-2.6.22-rc4-mm2]1 find . -name '*.c' | sed -r 's?.*/([^/]*)?\1?' | sort | uniq -c | sort -nr | head105 setup.c 90 irq.c 66 time.c 58 init.c 50 inode.c 39 io.c 38 pci.c 37 file.c 32 signal.c 32 ptrace.c
Quoted 3 lines
Looks like you're going to have to embed a lot of the path in that
KMSG_COMPONENTto make it unique - and you want to keep that message under 80 or so chars total.
For each kernel component, like a device driver, we could have one KMSG_COMPONENT (e.g. "acpi", "pci", etc). Within that component the message ids have to be unique. A tool could check, if messages are unique within the kernel sources.
We could use something like a Documentation/kmsg-components file with a list of all component names using KMSG printks.
Quoted 8 lines
Quoted 7 lines
/**
* message
* @0: device number of device.
*
* Description:
* An operation has been performed on the msgtest device, but the
* device has not been set online. Therefore the operation failed
Quoted 7 lines
If you don't understand 'Device
/dev/foooffline', this description doesn't help any. And that's true for *most* of the kernel messages already - if you don't understand the message already, a paragraph explanation isn't going to help much. Consider the average OOPS message, which contains stuff like 'EIP=0x..'. Telling the user that EIP means Execution Instruction Pointer isn't likely to help - if they knew what the pointer *did*, they'd probably already know EIP.
I agree with you, that most of the kernel messages do not need further documentation. But I am convinced, that there are plenty of printks, where additional documentation would be helpful.
Quoted 11 lines
Quoted 4 lines
*
* User Response:
* Operator should set device online.
* Issue "chccwdev -e <device number>".And this is where the weakness of this scheme *really* hits. I've actually run into cases where an operator followed the listed "Operator Response" for a "device offline", and issued a 'VARY 0C0,ONLINE'. And then we got a flood of I/O errors because the previous shift downed the device because it was having issues. The response the operator *should* have done is "assign a different tape drive, like, oh maybe the operational ones at 0C1 through 0C4"...
I can understand your frustration here. But that's a general problem with documentation. You never can foresee everything.
But should this mean, that we shouldn't document anything?
Michael
Hi,
Quoted 3 lines · holzheu
If the information is to big for the printk itself, because you would need 10 lines to explain what happened, wouldn't it be good to have a place where to put that information?
I think if a message really has to be 10 lines long to be clear then it should just be 10 lines long. The keyword is "really".
Example: oops.
Signature · 1 line
Krzysztof Halasa
Quoted 7 lines · On Jun 13 2007 12:50, [email protected] wrote:
Quoted 7 lines
In general we think, that also for Linux it is a good thing to have documentation for the most important kernel/driver messages. Even kernel hackers not always are aware of the meaning of kernel messages for components, which they don't know in detail. Most of the messages are self explaining but sometimes you get something like "Clocksource tsc unstable (delta = 7304132729 ns)" and you wonder if your system is going to explode.
I suppose no. If it is going to explode, it will tell you :-)
/* fs/super.c */
printk("VFS: Busy inodes after unmount of %s. "
"Self-destruct in 5 seconds. Have a nice day...\n", sb->s_id);
Quoted 2 lines
This is probably best addressed by cleaning up the actual messages so they're a bit more informative.
Jan
Makefile line 297
Quoted 5 lines · holzheu
Makefile +2
-3-CHECK = sparse - -CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ -Wbitwise $(CF) +CHECK = scripts/kmsg-doc check +CHECKFLAGS =
Ick. Don't touch those checking kernel with sparse.
Quoted 8 lines · Alexey Dobriyan
Makefile line 297
Quoted 5 lines · holzheu
Makefile +2
-3-CHECK = sparse - -CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ -Wbitwise $(CF) +CHECK = scripts/kmsg-doc check +CHECKFLAGS =Ick. Don't touch those checking kernel with sparse.
Of course we don't want to have that in the vanilla kernel. I just wanted to show, how the checker tool has to be used.
Michael
Quoted 14 lines · holzheu
Current prototype implementation:
=================================The structure of a kernel message is: <component>.<msg number>: <msg>
* component: Name of the kernel or driver component e.g. "pci", "ide", etc.
* msg number: Within the component unique number of a kernel message.
* msg: printk messageNew macros
KMSG_ERR(),KMSG_WARN(), etc. are defined, which have to be used in printk. These macros have as parameter the message number and are using a per c-file defined macroKMSG_COMPONENT.
Quoted 8 lines
Example: Define message 2 in component "kmsgtest":
#define
KMSG_COMPONENT"kmsgtest"void f(void)
{ printk(KMSG_ERR(1) "device %x not online\n", devno); }
Ick, why are you ignoring what we have already with dev_printk() and friends? We are just finally getting developers to use that, I think it will be almost impossible to get people to change to something else, especially one that isn't even as "correct" as what dev_printk() offers you today, will be quite hard.
So, why not use what we already have and work off of it?
thanks,
greg k-h
Hi Greg,
Quoted 7 lines
Ick, why are you ignoring what we have already with
dev_printk()and friends? We are just finally getting developers to use that, I think it will be almost impossible to get people to change to something else, especially one that isn't even as "correct" as whatdev_printk()offers you today, will be quite hard.So, why not use what we already have and work off of it?
dev_printk() and friends are great, since they already define something like KMSG_COMPONENT: The driver name.
But they unfortunately do not solve our problem. We need an identifier for a documented message in order to find the right description for a message.
We could add new dev_printk macros for KMSG printks:
#define dev_printk_kmsg(kmsg, dev, format, arg...) \
printk(kmsg "%s: " format , (dev)->bus_id , ## arg)#define dev_err_kmsg(dev, nr, format, arg...) \
dev_printk_kmsg(KMSG_ERR(nr) , dev , format , ## arg)
E.g. hub.c:
===========
#define KMSG_COMPONENT "hub"
dev_err_kmsg(&udev->dev, 1,
"can't read configurations, error %d\n", err);
Which would result in:
hub.1 5-0:1.0: can't read configuration error 4711
Michael
Quoted 8 lines · holzheu
Hi Greg,
Quoted 6 lines
Ick, why are you ignoring what we have already with
dev_printk()and friends? We are just finally getting developers to use that, I think it will be almost impossible to get people to change to something else, especially one that isn't even as "correct" as whatdev_printk()offers you today, will be quite hard.
Quoted 4 lines
Quoted 1 line
So, why not use what we already have and work off of it?
dev_printk()and friends are great, since they already define something likeKMSG_COMPONENT: The driver name.
They provide way more than that, they also provide the explicit device that is being discussed, as well as other things depending on the device.
So if you are going to do this, please use the already-in-place macros to hook into, don't try to get the driver authors to pick up something new and different, as it's going to be _very_ difficult, trust me...
thanks,
greg k-h
Quoted 8 lines · Greg KH
Quoted 2 lines
dev_printk()and friends are great, since they already definesomething
Quoted 1 line
like
KMSG_COMPONENT: The driver name.They provide way more than that, they also provide the explicit device that is being discussed, as well as other things depending on the device.
Instead of keeping a list of kernel message ids somewhere, can you base this off the dev_printk() format argument?
You could keep a list of those somewhere, extracted from the kernel source, and correlate documentation with the _format_ instead of an arbitrarily assigned message number.
-- Dave
Quoted 13 lines · Greg KH
Quoted 11 lines · holzheu
Hi Greg,
Quoted 7 lines
Ick, why are you ignoring what we have already with
dev_printk()and friends? We are just finally getting developers to use that, I think it will be almost impossible to get people to change to something else, especially one that isn't even as "correct" as whatdev_printk()offers you today, will be quite hard.So, why not use what we already have and work off of it?
dev_printk()and friends are great, since they already define something likeKMSG_COMPONENT: The driver name.
Quoted 7 lines
They provide way more than that, they also provide the explicit device that is being discussed, as well as other things depending on the device.
So if you are going to do this, please use the already-in-place macros to hook into, don't try to get the driver authors to pick up something new and different, as it's going to be _very_ difficult, trust me...
absolutely... I'm currently trying to convince everyone to get a generic netdevice-centric printk macro (dev_printk does not print the interface name, which the netdevice can't store locally since it can change) in and even that is a pain :)
Auke
Quoted 9 lines · Greg KH
Quoted 3 lines
Quoted 1 line
So, why not use what we already have and work off of it?
dev_printk()and friends are great, since they already definesomething
Quoted 1 line
like
KMSG_COMPONENT: The driver name.They provide way more than that, they also provide the explicit device that is being discussed, as well as other things depending on the device.
dev_printk() and friends provide additional information for a printk that is related to a device. Not every printk is about a device, so I think Michaels proposal is orthorgonal to dev_printk. We definitly should have a dev_printk variant that uses the kmsg documentation tag AND we should have a normal printk variant as well that uses the kmsg tagging.
Quoted 3 lines
So if you are going to do this, please use the already-in-place macros to hook into, don't try to get the driver authors to pick up something new and different, as it's going to be _very_ difficult, trust me...
There is no doubt that it will be difficult to get a larger part of the developers use the kmsg scheme (e.g. see the reaction of Dave M.). We do not have the illusion that we can replace every single message in the kernel, nor do we think that it would make sense to do so. What we would do for s390 is to check each message in drivers/s390 and think hard if the message should be 1) removed, 2) replaced with a {dev_}printk_kmsg or 3) left as it is. Fortunatly for us there are not too many drivers for s390 ..
Signature · 4 lines
blue skies, Martin. "Reality continues to ruin my life." - Calvin.
Quoted 3 lines · holzheu
But they unfortunately do not solve our problem. We need an identifier for a documented message in order to find the right description for a message.
I believe it better to simply add __FILE__ & __LINE__ to the macro rather than some other externally specified unique identifier that adds developer overhead and easily gets stale.
If you go overboard, you could do something like:
#define dev_dbg(dev, format, arg...) \
dev_printk(KERN_DEBUG, dev, DBG_FMT format DBG_ARG, ## arg)
#define dev_err(dev, format, arg...) \
dev_printk(KERN_ERR, dev, PR_FMT format PR_ARG, ## arg)
etc...
and use config variables to control what additional info is printk'ed.
Go overboard without a life preserver and add __TIMESTAMP__ too.
/*
*/
#if defined DBG_FMT
#undef DBG_FMT
#endif
#if defined DBG_ARG
#undef DBG_ARG
#endif
#if defined DBG_FILE && defined DBG_FUNCTION && defined DBG_LINE
#define DBG_FMT "(%s:%s:%u) "
#define DBG_ARG , __FILE__ , __FUNCTION__ , __LINE__
#elif defined DBG_FILE && defined DBG_FUNCTION && !defined DBG_LINE #define DBG_FMT "(%s:%s) "
#define DBG_ARG , __FILE__ , __FUNCTION__
#elif defined DBG_FILE && !defined DBG_FUNCTION && defined DBG_LINE #define DBG_FMT "(%s:%u) "
#define DBG_ARG , __FILE__ , __LINE__
#elif defined DBG_FILE && !defined DBG_FUNCTION && !defined DBG_LINE #define DBG_FMT "(%s) "
#define DBG_ARG , __FILE__
#elif !defined DBG_FILE && defined DBG_FUNCTION && defined DBG_LINE #define DBG_FMT "(%s:%u) "
#define DBG_ARG , __FUNCTION__ , __LINE__
#elif !defined DBG_FILE && defined DBG_FUNCTION && !defined DBG_LINE #define DBG_FMT "(%s) "
#define DBG_ARG , __FUNCTION__
#elif !defined DBG_FILE && !defined DBG_FUNCTION && defined DBG_LINE #define DBG_FMT "(%u) "
#define DBG_ARG , __LINE__
#else
#define DBG_FMT
#define DBG_ARG
#endif
/*
*/
#if defined PR_FMT
#undef PR_FMT
#endif
#if defined PR_ARG
#undef PR_ARG
#endif
#if defined PR_FILE && defined PR_FUNCTION && defined PR_LINE
#define PR_FMT "(%s:%s:%u) "
#define PR_ARG , __FILE__ , __FUNCTION__ , __LINE__ #elif defined PR_FILE && defined PR_FUNCTION && !defined PR_LINE #define PR_FMT "(%s:%s) "
#define PR_ARG , __FILE__ , __FUNCTION__
#elif defined PR_FILE && !defined PR_FUNCTION && defined PR_LINE #define PR_FMT "(%s:%u) "
#define PR_ARG , __FILE__ , __LINE__
#elif defined PR_FILE && !defined PR_FUNCTION && !defined PR_LINE #define PR_FMT "(%s) "
#define PR_ARG , __FILE__
#elif !defined PR_FILE && defined PR_FUNCTION && defined PR_LINE #define PR_FMT "(%s:%u) "
#define PR_ARG , __FUNCTION__ , __LINE__
#elif !defined PR_FILE && defined PR_FUNCTION && !defined PR_LINE #define PR_FMT "(%s) "
#define PR_ARG , __FUNCTION__
#elif !defined PR_FILE && !defined PR_FUNCTION && defined PR_LINE #define PR_FMT "(%u) "
#define PR_ARG , __LINE__
#else
#define PR_FMT
#define PR_ARG
#endif
Quoted 3 lines · Joe Perches
I believe it better to simply add
__FILE__&__LINE__to the macro rather than some other externally specified unique identifier that adds developer overhead and easily gets stale.
There's been plenty of times I've wished for that. Now if we just found a way to do something sane for drivers/net/wireless/mac80211/iwlwifi/base.c ;)
Of course, I'm probably atypical - for me, kernel messages come in two forms, the kind that are immediately obvious at first reading, and the kind that you end up having to look at the code and wonder WTF? it was doing inside that never-should-happen-on-this-hardware while loop in the first place... :)
Quoted 9 lines · [email protected] wrote:
Quoted 3 lines · Joe Perches
I believe it better to simply add
__FILE__&__LINE__to the macro rather than some other externally specified unique identifier that adds developer overhead and easily gets stale.There's been plenty of times I've wished for that. Now if we just found a way to do something sane for
drivers/net/wireless/mac80211/iwlwifi/base.c;)
Why don't you just redefine the KERN_* macros which is used by almost every printk in the system to contain said __FILE__ and __LINE__, then?
-hpa
Quoted 6 lines · Joe Perches
Quoted 3 lines · holzheu
But they unfortunately do not solve our problem. We need an identifier for a documented message in order to find the right description for a message.
I believe it better to simply add
__FILE__&__LINE__to the
Erk, no. That bloats the kernel to the max. Especially since kernel files are compiled with a rather longish path.
Jan
Quoted 130 lines · On Wed, 13 Jun 2007 17:06:57 +0200 holzheu <[email protected]> wrote:
Greetings,
The operation of a Linux system sometimes requires to decode the meaning of a specific kernel message, e.g. an error message of a driver. Especially on our mainframe zSeries platform system administrators want to have descriptions for Linux kernel messages. They are used to that, because all other operating systems on that platform like z/OS, z/VM or z/VSE have message catalogs with detailed descriptions about the semantics of the messages.
In general we think, that also for Linux it is a good thing to have documentation for the most important kernel/driver messages. Even kernel hackers not always are aware of the meaning of kernel messages for components, which they don't know in detail. Most of the messages are self explaining but sometimes you get something like "Clocksource tsc unstable (delta = 7304132729 ns)" and you wonder if your system is going to explode.
Unfortunately currently there is no general infrastructure in the Linux kernel for the documentation of messages. I worked on a proposal, how that could be implemented in an easy way using the already existing kernel-doc infrastructure and using printk. The proposal is as follows
1. We use message identifiers in order to map messages to message descriptions. A message identifier consists out of a component name and within that component unique message number.
2. Messages and message descriptions are maintained together in the kernel sources in order to keep them up to date. Messages catalog are generated automatically for exactly one kernel level.
3. A special tool checks, if messages are not documented or if there are message descriptions without corresponding messages.
4. We use the already existing kernel-doc tool to generate an online message catalog or e.g. a pdf for offline documentation.
Current prototype implementation:
=================================The structure of a kernel message is: <component>.<msg number>: <msg>
* component: Name of the kernel or driver component e.g. "pci", "ide", etc.
* msg number: Within the component unique number of a kernel message.
* msg: printk messageNew macros
KMSG_ERR(),KMSG_WARN(), etc. are defined, which have to be used in printk. These macros have as parameter the message number and are using a per c-file defined macroKMSG_COMPONENT.Example: Define message 2 in component "kmsgtest":
#define
KMSG_COMPONENT"kmsgtest"void f(void)
{ printk(KMSG_ERR(1) "device %x not online\n", devno); }The output of that kernel message would be:
"kmsgtest.1: device 4711 not online"The messages have to be documented within the C source file in the following way:
/**
* message
* @0: device number of device.
*
* Description:
* An operation has been performed on the msgtest device, but the
* device has not been set online. Therefore the operation failed *
* User Response:
* Operator should set device online.
* Issue "chccwdev -e <device number>".
*
*/
KMSG_DOC(kmsgtest, 2, "Device %x not online");I created a patch for the kernel-doc tool so it can be used to generate a catalog of all kernel messages:
Quoted 2 lines
Quoted 2 lines
kernel-doc -man
kmsgtest.c> kmsgtest.2
man ./kmsgtest.2# Kernel API(9) Linux Messages Kernel API(9)
#
# MESSAGE:
# kmsgtest.2: "device %x not online"
#
# Parameter:
# 1 Device number of device.
#
# Description:
# An operation has been performed on the msgtest device, but
# the device has not been set online. Therefore the operation failed.
#
# User Response:
# Operator should set device online.
# Issue "chccwdev -e <device number>".
#
# May 2007 kmsgtest.2 Kernel API(9)A nice thing would be to include the online kernel message catalog in the kernel rpm. One possibility for that would be to have one man page per message. If an operator finds the message kmsgtest.2 in var/log/messages and wants to know what the message means, he simply issues "man kmsgtest.2" and gets the description.
To ensure that all messages are documented and there are no message descriptions without corresponding messages, a checker tool is provided. To enable message checking, in the toplevel
Makefilethe following has to be added:CHECK =
scripts/kmsg_check.plcheckTo enable message checking during kernel build, the "C" option has to be used:
Quoted 1 line
Quoted 1 line
make modules C=1
CHK include/linux/version.h CHK include/linux/utsrelease.h CHECK drivers/kmsgtest/kmsgtest.c
drivers/kmsgtest/kmsgtest.c: Missing description for: kmsgtest.1drivers/kmsgtest/kmsgtest.c: Description without message for: kmsgtest.3
Quoted 8 lines
Please note, that the patch for the kernel-doc and kmsg-doc tools is just a prototype and is neither complete nor perfect.
Michael
Acked-by Heiko Carstens <[email protected]>Signed-off-by Michael Holzheu <[email protected]>
Your proposal is similar to one I made to some Japanese developers earlier this year. I was more modest, proposing that we
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);
- An externally managed database will provide translations, diagnostics, remedial actions, etc, keyed off the msgid string
- Format and management of the msgid hasn't been clearly identified yet as far as I know. But all we really need is that each ID be unique, kernel-wide. We develop a simple tool which can check the whole tree for duplicated msgids.
- From a practical day-to-day POV what this means is that a person from the kernel-messages team will prepare a patch for your driver which adds the msgids and you the driver author should take care not to break the tagging.
This is deliberately designed to be minimum-impact upon general developers. We want a system in place where driver/fs/etc developers can concentrate on what they do, and kernel-message developers can as independently as possibe take care of what _they_ do.
- A project has started up and there is a mailing list (cc'ed here) and meetings are happening at the LF collaboration summit this week.
Please see https://lists.linux-foundation.org/mailman/listinfo/lf_kernel_messages
and don't miss the next conference call ;)
(argh. The lf_kernel_messages archives are subscriber-only and it could be that only members can post to it. Oh well. Please subscribe, and review the archives)
Quoted 527 lines
Makefile | 5 drivers/Makefile | 1 drivers/kmsgtest/Makefile | 5 drivers/kmsgtest/kmsgtest.c | 59 +++++++++++ include/linux/kmsg.h | 32 ++++++ scripts/kernel-doc | 116 +++++++++++++++++++++- scripts/kmsg-doc | 231 ++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 445 insertions(+), 4 deletions(-)diff -Naur linux-2.6.21/Makefile linux-2.6.21-kmsg/Makefile
Makefile +2
-3@@ -293,9 +293,8 @@ DEPMOD = /sbin/depmod KALLSYMS = scripts/kallsyms PERL = perl -CHECK = sparse - -CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ -Wbitwise $(CF) +CHECK = scripts/kmsg-doc check +CHECKFLAGS = MODFLAGS = -DMODULE CFLAGS_MODULE = $(MODFLAGS) AFLAGS_MODULE = $(MODFLAGS)diff -Naur linux-2.6.21/drivers/Makefile linux-2.6.21-kmsg/drivers/Makefile
drivers/Makefile +1
-0@@ -8,6 +8,7 @@ obj-$(CONFIG_PCI) += pci/ obj-$(CONFIG_PARISC) += parisc/ obj-$(CONFIG_RAPIDIO) += rapidio/ +obj-m += kmsgtest/ obj-y += video/ obj-$(CONFIG_ACPI) += acpi/ # PnP must come after ACPI since it will eventually need to check if acpidiff -Naur linux-2.6.21/drivers/kmsgtest/Makefile linux-2.6.21-kmsg/drivers/kmsgtest/Makefile
drivers/kmsgtest/Makefile +5
-0@@ -0,0 +1,5 @@ +# +# Makefile for kernel message test module +# + +obj-m += kmsgtest.odiff -Naur
linux-2.6.21/drivers/kmsgtest/kmsgtest.clinux-2.6.21-kmsg/drivers/kmsgtest/kmsgtest.cdrivers/kmsgtest/kmsgtest.c +59
-0@@ -0,0 +1,59 @@ +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/kmsg.h> + +static int devno =0x4711 ; +static int status =1 ; + +#define KMSG_COMPONENT "kmsgtest" + +static int __init kmsgtest_init (void ) +{ + printk(KMSG_INFO(1) "device %x has status %i\n", devno, status); + printk(KMSG_ERR(2) "device %x not online\n", devno); + +return 0 ; +} + +static void __exit kmsgtest_exit(void) +{ +printk ("kmsgtest module exit\n" ); +} + +module_init (kmsgtest_init); +module_exit (kmsgtest_exit); + +/** +* message +* @0: Device number of device +* @1: Status of device +* +* Description: +* Information message about the status of our virtual msgtest device. The +* following values for the status parameter are available. +* +* 0 - Device is offline +* +* 1 - Device is online +* +* 2 - Device is broken +* +* User Response: +* If device is broken, replace it or fix it. +*/ + +KMSG_DOC (kmsgtest,1 ,"device %x has status %i" ); + +/** +* message +* @0: Device number of device. +* +* Description: +* An operation has been performed on the msgtest device, but the device has +* not been set online. Therefore the operation failed. +* +* User Response: +* Operator should set device online. Issue "chccwdev -e <device number>". +*/ + +KMSG_DOC (kmsgtest,2 ,"device %x not online" );diff -Naur
linux-2.6.21/include/linux/kmsg.hlinux-2.6.21-kmsg/include/linux/kmsg.hinclude/linux/kmsg.h +32
-0@@ -0,0 +1,32 @@ +#ifndef _LINUX_KMSG_H +#define _LINUX_KMSG_H + +#ifdef __KMSG_CHECKER + +#define KMSG_EMERG (num) __KMSG_CHECK(EMERG, num) +#define KMSG_ALERT (num) __KMSG_CHECK(ALERT, num) +#define KMSG_CRIT (num) __KMSG_CHECK(CRIT, num) +#define KMSG_ERR (num) __KMSG_CHECK(ERR, num) +#define KMSG_WARNING (num) __KMSG_CHECK(WARNING, num) +#define KMSG_NOTICE (num) __KMSG_CHECK(NOTICE, num) +#define KMSG_INFO (num) __KMSG_CHECK(INFO, num) +#define KMSG_DEBUG (num) __KMSG_CHECK(DEBUG, num) + +#define KMSG_DOC (comp, num, str) __KMSG_DOC(comp, num, str) + +#else + +#define KMSG_EMERG (num) KERN_EMERG KMSG_COMPONENT "." #num ": " +#define KMSG_ALERT (num) KERN_ALERT KMSG_COMPONENT "." #num ": " +#define KMSG_CRIT (num) KERN_CRIT KMSG_COMPONENT "." #num ": " +#define KMSG_ERR (num) KERN_ERR KMSG_COMPONENT "." #num ": " +#define KMSG_WARNING (num) KERN_WARNING KMSG_COMPONENT "." #num ": " +#define KMSG_NOTICE (num) KERN_NOTICE KMSG_COMPONENT "." #num ": " +#define KMSG_INFO (num) KERN_INFO KMSG_COMPONENT "." #num ": " +#define KMSG_DEBUG (num) KERN_DEBUG KMSG_COMPONENT "." #num ": " + +#define KMSG_DOC (comp, num, str) + +#endif /* __KMSG_CHECKER */ + +#endif /* _LINUX_KMSG_H */ diff -Naur linux-2.6.21/scripts/kernel-doc linux-2.6.21-kmsg/scripts/kernel-doc
scripts/kernel-doc +115
-1@@ -256,7 +256,7 @@ my $in_doc_sect; #declaration types: can be -# 'function', 'struct', 'union', 'enum', 'typedef' +# 'function', 'struct', 'union', 'enum', 'typedef', 'message' my $decl_type; my $doc_special = "\@\%\$\&";@@ -1163,6 +1163,55 @@ output_section_text(@_); } +## +# output message in text +sub output_message_text(%) { + my %args = %{$_[0]}; + my ($parameter); + + print $args{'id'}.": \"".$args{'message'}."\"\n\n"; + + print "Parameters:\n\n"; + foreach $parameter (@{$args{'parameterlist'}}) { + ($parameter =~ /^#/) && next; + my $parameter_name = $parameter; + $parameter_name =~ s/\[.*//; + ($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next; + print "$parameter\n\t"; + print $args{'parameterdescs'}{$parameter_name}."\n"; + } + output_section_text(@_); +} + +## +# output message in man +sub output_message_man(%) { + my %args = %{$_[0]}; + my ($parameter, $section); + + print ".TH \"$args{'module'}\" 9 \"".$args{'id'}."\" \"$man_date\" \"Linux Messages\" LINUX\n"; + + print ".SH MESSAGE\n"; + print $args{'id'}.": "."\"".$args{'message'}."\"\n"; + + print ".SH Parameters\n"; + foreach $parameter (@{$args{'parameterlist'}}) { + ($parameter =~ /^#/) && next; + + my $parameter_name = $parameter; + $parameter_name =~ s/\[.*//; + + ($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next; + print ".IP \"".$parameter."\" 12\n"; + output_highlight($args{'parameterdescs'}{$parameter_name}); + } + foreach $section (@{$args{'sectionlist'}}) { + print ".SH \"$section\"\n"; + output_highlight($args{'sections'}{$section}); + print "\n"; + } +} + #output sections in text sub output_section_text(%) { my %args = %{$_[0]};@@ -1607,6 +1656,69 @@ }); } +sub create_parameterlist_msg($$) { + my $args = shift; + my $file = shift; + my $splitter = "%"; + my $type; + my $param = 0; + my $first = 1; + + # temporarily replace commas + while ($args =~ /(\([^\),]+),/) { + $args =~ s/(\([^\),]+),/$1#/g; + } + + foreach my $arg (split($splitter, $args)) { + if ($first) { + $first = 0; + next; + } + $type = substr($arg,0,1); + + # XXX introduce better type checking + + push_parameter($param, $type, $file); + $param += 1; + } +} + +## +# takes a message prototype and the name of the current file being +# processed and spits out all the details stored in the global +# arrays/hashes. +sub dump_message($$) { + my $x = shift; + my $file = shift; + + if ($x =~/(KMSG_DOC)\(\s*(\w+)\s*\,\s*(\w+)\s*\,\s*\"(.*)\"\)/) { + $declaration_name = "$2.$3"; + my $members = $4; + # strip comments: + $members =~ s/\/\*.*?\*\///gos; + + create_parameterlist_msg($members, $file); + + output_declaration($declaration_name, + 'message', + {'message' => $members, + 'id' => $declaration_name, + 'module' => $modulename, + 'parameterlist' => \@parameterlist, + 'parameterdescs' => \%parameterdescs, + 'parametertypes' => \%parametertypes, + 'sectionlist' => \@sectionlist, + 'sections' => \%sections, + 'purpose' => $declaration_purpose, + 'type' => $decl_type + }); + } + else { + print STDERR "Error(${file}:$.): Cannot parse message!\n"; + ++$errors; + } +} + sub process_file($); # Read the file that maps relative names to absolute names for@@ -1782,6 +1894,8 @@ $decl_type = 'enum'; } elsif ($identifier =~ m/^typedef/) { $decl_type = 'typedef'; + } elsif ($identifier =~ m/^message/) { + $decl_type = 'message'; } else { $decl_type = 'function'; }diff -Naur linux-2.6.21/scripts/kmsg-doc linux-2.6.21-kmsg/scripts/kmsg-doc
scripts/kmsg-doc +231
-0@@ -0,0 +1,231 @@ +#!/usr/bin/perl +# +# Tool to check kernel messages +# +# Can be used in toplevel Linux Makefile in the following way: +# +# CHECK = scripts/kmsg-doc check +# CHECKFLAGS = +# +# Note: This is just a prototype and neither perfect nor complete! +# +# Copyright (C) IBM Corp. 2007 +# Author(s): Michael Holzheu <[email protected]> +# + +sub create_message($$$$$) +{ + my ($sev, $component, $number, $text, $params) = @_; + + $text =~ s/\\n//; # remove trailing newline character + $message_id = "$component.$number"; + $messages{$message_count}->{'ID'} = $message_id; + $messages{$message_count}->{'COMP'} = $component; + $messages{$message_count}->{'NR'} = $number; + $messages{$message_count}->{'MSG'} = $text; + $messages{$message_count}->{'SEV'} = $sev; + + @parms = split(/[\s]*,[\s]*/,$params); + $parm_count = 0; + foreach $parm (@parms) { + if (!($parm eq "")) { + $messages{$message_count}->{'PARM_NAME'}->{$parm_count} = $parm; + $parm_count += 1; + } + } + $messages{$message_count}->{'PARM_COUNT'} = $parm_count; + $message_count += 1; +} + +sub get_msgs($) +{ + my ($filename)=@_; + + $message_count = 0; + open(FD, $filename); + my @lines=<FD>; + foreach $line (@lines) { + if ($line =~ /\s*printk\([\s]*__KMSG_CHECK\((.*)\,[\s]*(.*)\)[\s]*"(.*)"[\s]*(.*)[\s]*\);/) { + create_message($1, $component, $2, $3, $4); + } + } +} + +sub get_descriptions($) +{ + my ($filename)=@_; + my $desc_start; + + $description_count = 0; + $desc_start = 0; + open(FD, $filename); + my @lines=<FD>; + foreach $line (@lines) { + if ($line =~ /#define [\s]*KMSG_COMPONENT [\s]*"(.*)"/) { + $component = $1; + } + if ($line =~ /\s*\/\*\*$/) { + $msg_start = 1; + $parm_count = 0; + next; + } + if (($msg_start == 1) && ($line =~ / \* message/)) { + $desc_start = 1; + next; + } + if ($line =~ / \*\//) { + $desc_start = 0; + next; + } + if ($desc_start == 1) { + $descriptions{$description_count}->{'DESC'} .= "$line"; + next; + } + if ($line =~ + /\s*KMSG_DOC\(\s*(.*)\s*\,\s*(.*)\s*\,\s*\"(.*)\"\s*\);/) { + my $param_count = 0; + my $first = 1; + my $type; + + $descriptions{$description_count}->{'ID'} = "$1\.$2"; + $descriptions{$description_count}->{'COMP'} = "$1"; + $descriptions{$description_count}->{'NR'} = "$2"; + $descriptions{$description_count}->{'MSG'} = "$3"; + foreach my $arg (split("%", $3)) { + if ($first) { + $first = 0; + next; + } + $type = substr($arg, 0, 1); + $descriptions{$description_count}->{'PARM_TYPE'}->{$param_count} = $type; + $param_count += 1; + } + $descriptions{$description_count}->{'PARM_COUNT'} = $param_count; + $description_count += 1; + } + } +} + +sub print_messages() +{ + for ($i = 0; $i < $message_count; $i++) { + print "MESSAGE: $messages{$i}->{'ID'}\n"; + } +} + +sub print_descriptions($) +{ + my ($message_id)=@_; + + for ($i = 0; $i < $description_count; $i++) { + if (($descriptions{$i}->{'ID'} eq $message_id) || $message_id eq "all") { + print "==============================================================================\n"; + print "\[$descriptions{$i}->{'COMP'}\.$descriptions{$i}->{'NR'}\] $descriptions{$i}->{'MSG'}\n"; + print "\n"; + print "Parameters:\n"; + for ($j = 0; $j < $descriptions{$i}->{'PARM_COUNT'}; $j++) { + print " $descriptions{$i}->{'PARM_TYPE'}->{$j}: $descriptions{$i}->{'PARM_DESC'}->{$j}\n"; + } + print "\n"; + print "Description:\n"; + print "$descriptions{$i}->{'DESC'}\n"; + print "==============================================================================\n"; + } + } +} + +sub check_messages($) +{ + my ($filename)=@_; + + for ($i = 0; $i < $message_count; $i++) { + $found = 0; + for ($j = 0; $j < $description_count; $j++) { + if ($messages{$i}->{'ID'} eq $descriptions{$j}->{'ID'}) { + $found = 1; + last; + } + } + if (!$found) { + print STDERR "$filename: Missing description for: $messages{$i}->{'ID'}\n"; + } + } + for ($i = 0; $i < $description_count; $i++) { + $found = 0; + for ($j = 0; $j < $message_count; $j++) { + if ($messages{$j}->{'ID'} eq $descriptions{$i}->{'ID'}) { + $found = 1; + last; + } + } + if (!$found) { + print STDERR "$filename: Description without message for: $descriptions{$i}->{'ID'}\n"; + } + } +} + +sub print_templates() +{ + + for ($i = 0; $i < $message_count; $i++) { + $found = 0; + for ($j = 0; $j < $description_count; $j++) { + if ($messages{$i}->{'ID'} eq $descriptions{$j}->{'ID'}) { + $found = 1; + } + } + if (!$found) { + print "/**\n"; + print " * message\n"; + for ($k = 0; $k < $messages{$i}->{'PARM_COUNT'}; $k++) { + print " * \@$k: $messages{$i}->{'PARM_NAME'}->{$k}\n"; + } + print " *\n"; + print " * Description:\n"; + print " *\n"; + print " * User Response:\n"; + print " */\n"; + print "\n"; + print "KMSG_DOC($messages{$i}->{'COMP'}, $messages{$i}->{'NR'}, \"$messages{$i}->{'MSG'}\");\n" + } + } +} + +sub usage() +{ + print "USAGE: kmsg_tool print | check <file>\n"; + exit 1; +} + +$option = shift; + +if ($option eq "check") { + $gcc_options = "-E -D __KMSG_CHECKER "; + do { + $filename = $tmp; + $tmp = shift; + $tmp =~ s/\(/\\\(/; + $tmp =~ s/\)/\\\)/; + $gcc_options .= " $tmp"; + } while (!($tmp eq "")); + + $gcc_options =~ s/-Wbitwise//; # XXX hack to remove -Wbitwise CHECKFLAG + $gcc_options =~ s/-D__STDC__//; # XXX hack to remove -D__STDC__ + $tmp_file = "$filename.msg"; + system("gcc $gcc_options > $tmp_file"); + get_descriptions($filename); + get_msgs($tmp_file); + check_messages($filename); + print_templates(); + system("rm $tmp_file"); +} elsif ($option eq "print") { + $filename = shift; + do { + print STDERR "Processing: $filename\n"; + get_descriptions($filename); + print_descriptions("all"); + $filename = shift; + } while (!($filename eq "")); +} else { + usage(); +}
Quoted 2 lines · Andrew Morton
Your proposal is similar to one I made to some Japanese developers earlier this year.
Interesting. Our requirement comes from Japan as well.
Quoted 5 lines
I was more modest, proposing that we
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);
Some kind of id is always required to be able to identify the message. The task is to make the tagging as simple as possible.
Quoted 2 lines
- An externally managed database will provide translations, diagnostics, remedial actions, etc, keyed off the msgid string
There I have my doubts if this can work. If you keep an external database with the additional information about the messages the kernel code and the database will get out of sync. Thats why we advocate the kernel-doc style approach: the message catalogue will always be in sync because it is delivered with the kernel.
Quoted 4 lines
- Format and management of the msgid hasn't been clearly identified yet as far as I know. But all we really need is that each ID be unique, kernel-wide. We develop a simple tool which can check the whole tree for duplicated msgids.
This is a paint point with Michaels approach as well. The KMSG_COMPONENT defines are rather ugly. A clever way how to generate kernel unique IDs would be nice. Intervention required..
Quoted 9 lines
- From a practical day-to-day POV what this means is that a person from the kernel-messages team will prepare a patch for your driver which adds the msgids and you the driver author should take care not to break the tagging.
This is deliberately designed to be minimum-impact upon general developers. We want a system in place where driver/fs/etc developers can concentrate on what they do, and kernel-message developers can as independently as possibe take care of what _they_ do.
With the KMSG approach all that needs to be done is to replace the KERN_xxx with KMSG_xxx(<number) and to add a comment at the end of the file.
Quoted 8 lines
- A project has started up and there is a mailing list (cc'ed here) and meetings are happening at the LF collaboration summit this week.
Please see https://lists.linux-foundation.org/mailman/listinfo/lf_kernel_messages
and don't miss the next conference call ;)(argh. The
lf_kernel_messagesarchives are subscriber-only and it could be that only members can post to it. Oh well. Please subscribe, and review the archives)
Very interesting. We did not know about this project. Thanks for the hint.
Signature · 4 lines
blue skies, Martin. "Reality continues to ruin my life." - Calvin.
<snip>
Quoted 6 lines
Your proposal is similar to one I made to some Japanese developers earlier this year. I was more modest, proposing that we
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);
Maybe a stupid idea but why do we want to assign these numbers by hand?
I can imagine it could introduce collisions when merging tons of patches with new messages... Wouldn't it be better to compute say, 8-byte hash from the message and use it as it's identifier? We could do this automagically at compile time. I know it also has it's problems - you fix a spelling and the message gets a different id and you have to update translation/documentation catalogue but maybe that could be solved too...
Honza
Quoted 5 lines · Jan Kara
<snip>
Quoted 3 lines
Your proposal is similar to one I made to some Japanese developers earlier this year. I was more modest, proposing that we
Quoted 8 lines
Quoted 3 lines
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);Maybe a stupid idea but why do we want to assign these numbers by hand?
I can imagine it could introduce collisions when merging tons of patches with new messages... Wouldn't it be better to compute say, 8-byte hash from the message and use it as it's identifier? We could do this automagically at compile time.
Of course automatically generated message numbers would be great and something like:
hub.4a5bcd77: Detected some problem.
looks acceptable for me.
We could generate the hash using the format string of the printk. Since we specify the format string also in KMSG_DOC, the hash for the KMSG_DOC and the printk should match and we have the required link between printk and description.
So technically that's probably doable.
Problems are:
* hashes are not unique
* We need an additional preprocessor step
* The might be people, who find 8 character hash values ugly in printks
The big advantage is, that we do not need to maintain message numbers.
Quoted 4 lines
I know it also has it's problems - you
fix a spelling and the message gets a different id and you have to update translation/documentation catalogue but maybe that could be solved too...
Since in our approach the message catalog is created automatically for exactly one kernel and the message catalog belongs therefore to exactly one kernel, I think the problem of changing error numbers is not too big.
Michael
Quoted 15 lines · holzheu
Quoted 5 lines · Jan Kara
<snip>
Quoted 3 lines
Your proposal is similar to one I made to some Japanese developers earlier this year. I was more modest, proposing that we
Quoted 8 lines
Quoted 3 lines
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);Maybe a stupid idea but why do we want to assign these numbers by hand?
I can imagine it could introduce collisions when merging tons of patches with new messages... Wouldn't it be better to compute say, 8-byte hash from the message and use it as it's identifier? We could do this automagically at compile time.
Quoted 4 lines
Of course automatically generated message numbers would be great and something like:
hub.4a5bcd77: Detected some problem.
Sorry, I first read: 8 characters not 8 bytes.
Indeed, "hub.d41d8cd98f00b204: Detected some problem." ... does not look like very beautiful.
But maybe also 4 bytes would be enough, since the hash only has to be unique within one component e.g. "hub".
Michael
Quoted 21 lines · holzheu
Quoted 15 lines · holzheu
Quoted 5 lines · Jan Kara
<snip>
Quoted 3 lines
Your proposal is similar to one I made to some Japanese developers earlier this year. I was more modest, proposing that we
Quoted 8 lines
Quoted 3 lines
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);Maybe a stupid idea but why do we want to assign these numbers by hand?
I can imagine it could introduce collisions when merging tons of patches with new messages... Wouldn't it be better to compute say, 8-byte hash from the message and use it as it's identifier? We could do this automagically at compile time.Quoted 4 lines
Of course automatically generated message numbers would be great and something like:
hub.4a5bcd77: Detected some problem.
Quoted 7 lines
Sorry, I first read: 8 characters not 8 bytes.
Indeed, "hub.d41d8cd98f00b204: Detected some problem." ... does not look like very beautiful.
But maybe also 4 bytes would be enough, since the hash only has to be unique within one component e.g. "hub".
It depends how large components you expect. For example for 10000
messages there is already 1% probability of collision so it means sooner or later we are going to hit it... For 1000 messages the probability is roughly 0.1% which is still not so small that I'd be comfortable with it. On the other hand we could use something like gperf to generate perfect hashing function and then we don't have to worry about colisions.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SuSE CR Labs
On Thu, 2007-06-14 at 14:26 +0200, Jan Kara wrote:
<snip>
Quoted 6 lines
Quoted 2 lines
But maybe also 4 bytes would be enough, since the hash only has to be unique within one component e.g. "hub".
It depends how large components you expect. For example for 10000
messages there is already 1% probability of collision so it means sooner or later we are going to hit it... For 1000 messages the probability is roughly 0.1% which is still not so small that I'd be comfortable with it.
I wouldn't expect, that a "normal" component like a device driver will have more than 100 documented messages.
Collisions are ugly, but will not hurt much. The operator will see for a message two possible descriptions. Normally it should be possible to figure out manually which description is the right one.
Michael
Quoted 42 lines · holzheu
Quoted 5 lines · Jan Kara
<snip>
Quoted 3 lines
Your proposal is similar to one I made to some Japanese developers earlier this year. I was more modest, proposing that we
Quoted 8 lines
Quoted 3 lines
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);Maybe a stupid idea but why do we want to assign these numbers by hand?
I can imagine it could introduce collisions when merging tons of patches with new messages... Wouldn't it be better to compute say, 8-byte hash from the message and use it as it's identifier? We could do this automagically at compile time.Of course automatically generated message numbers would be great and something like:
hub.4a5bcd77: Detected some problem.
looks acceptable for me.
We could generate the hash using the format string of the printk. Since we specify the format string also in
KMSG_DOC, the hash for theKMSG_DOCand the printk should match and we have the required link between printk and description.So technically that's probably doable.
Problems are:
* hashes are not unique
* We need an additional preprocessor step
* The might be people, who find 8 character hash values ugly in printksThe big advantage is, that we do not need to maintain message numbers.
Quoted 4 lines
I know it also has it's problems - you
fix a spelling and the message gets a different id and you have to update translation/documentation catalogue but maybe that could be solved too...
Quoted 4 lines
Since in our approach the message catalog is created automatically for exactly one kernel and the message catalog belongs therefore to exactly one kernel, I think the problem of changing error numbers is not too big.
We just had a meeting with the Japanese and several other participants from the vendor and community side and came up with a potential proposal that is similar to many things discussed here. It has the benefit that it seems implementable and low/no overhead on most kernel developers.
The basic proposal is to use a tool, run by the kernel Makefile to extract kernel messages from either the kernel source or the .i files (both have advantages, although I prefer the source to the .i file since it 1) gets all messages and 2) is probably a little quicker with less impact to the standard kernel make.
These messages would be stored in a file in the source tree, e.g. usr/src/linux/Translations/English. As each message is added to that file, we calculate, say, an MD5 sum of the printk (dev_printk, sdev_printk, etc.) string, and the text file ultimately contains:
MD5 Checksum of text; the printk text itself, the File name, the line number.
The checksum is run over just the printk. We definitely would not include the line number since the line number is too volatile. Including the file name in the hash *might* help disambiguate the hash a bit better in the case of duplicates, but there was some debate that duplicates might be better handled in other ways.
Andrew mentioned a mechanism for adding a subsystem tag or other tag which helps disambiguate the message, either in the message file or in the end user documentation (e.g. the Message Pedia/mPedia that the Japanese have already created with ~350 messages, and a total of ~700 targetted by the end of the year).
That tag could be appended to the beginning of the printk, to the end of the printk, or even in a formatted comment at the end of the printk that the tool could extract.
Then, the translations could be managed by anyone outside of the normal/ core kernel community, by simply creating a translation file, e.g. usr/src/linux/Translations/Japanese, which contained the MD5 sum, the translated message, the file name and line number (the last two redundent perhaps but informational, and automatically generated if possible).
The files in the Translations directory could be uesd as the unique keys for an external database (such as the Message Pedia, vendors or distributions help pages, etc.) to help look up and explain root cause of a problem. The key property here is that the MD5 sum becomes the key to all database entries to look up that key.
Further, yet another kernel config option could allow distros to output the calculated MD5 sum to be printed, much like we do with timestamps today.
End result is that these in-kernel message catalogs for translation are updated automatically (mostly no kernel developer changes needed) and the translations can be maintained by anyone who is interested.
On the topic of MD5 collisions, using a disambiguating tag would be a simple addition for the few cases where that happens, the tool could be educated to use that tag in the calculation of the MD5 sum, and we have a 98% solution which impacts <1% of the kernel developers.
Folks present for this discussion included Ted T'so, James Bottomley, several of the key Japenese folks interested in using this for debugging, and reps from several vendors who would find this sort of info useful for their folks supporting Linux in the field.
Comments?
gerrit
Quoted 105 lines · Gerrit Huizenga
Quoted 36 lines · holzheu
Quoted 5 lines · Jan Kara
<snip>
Quoted 3 lines
Your proposal is similar to one I made to some Japanese developers earlier this year. I was more modest, proposing that we
Quoted 8 lines
Quoted 3 lines
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);Maybe a stupid idea but why do we want to assign these numbers by hand?
I can imagine it could introduce collisions when merging tons of patches with new messages... Wouldn't it be better to compute say, 8-byte hash from the message and use it as it's identifier? We could do this automagically at compile time.Of course automatically generated message numbers would be great and something like:
hub.4a5bcd77: Detected some problem.
looks acceptable for me.
We could generate the hash using the format string of the printk. Since we specify the format string also in
KMSG_DOC, the hash for theKMSG_DOCand the printk should match and we have the required link between printk and description.So technically that's probably doable.
Problems are:
* hashes are not unique
* We need an additional preprocessor step
* The might be people, who find 8 character hash values ugly in printksThe big advantage is, that we do not need to maintain message numbers.
Quoted 8 lines
Quoted 4 lines
I know it also has it's problems - you
fix a spelling and the message gets a different id and you have to update translation/documentation catalogue but maybe that could be solved too...Since in our approach the message catalog is created automatically for exactly one kernel and the message catalog belongs therefore to exactly one kernel, I think the problem of changing error numbers is not too big.
We just had a meeting with the Japanese and several other participants from the vendor and community side and came up with a potential proposal that is similar to many things discussed here. It has the benefit that it seems implementable and low/no overhead on most kernel developers.
The basic proposal is to use a tool, run by the kernel
Makefileto extract kernel messages from either the kernel source or the .i files (both have advantages, although I prefer the source to the .i file since it 1) gets all messages and 2) is probably a little quicker with less impact to the standard kernel make.These messages would be stored in a file in the source tree, e.g.
usr/src/linux/Translations/English. As each message is added to that file, we calculate, say, an MD5 sum of the printk (dev_printk,sdev_printk, etc.) string, and the text file ultimately contains:MD5 Checksum of text; the printk text itself, the File name, the line number.
The checksum is run over just the printk. We definitely would not include the line number since the line number is too volatile. Including the file name in the hash *might* help disambiguate the hash a bit better in the case of duplicates, but there was some debate that duplicates might be better handled in other ways.
Andrew mentioned a mechanism for adding a subsystem tag or other tag which helps disambiguate the message, either in the message file or in the end user documentation (e.g. the Message Pedia/mPedia that the Japanese have already created with ~350 messages, and a total of ~700 targetted by the end of the year).
That tag could be appended to the beginning of the printk, to the end of the printk, or even in a formatted comment at the end of the printk that the tool could extract.
Then, the translations could be managed by anyone outside of the normal/ core kernel community, by simply creating a translation file, e.g.
usr/src/linux/Translations/Japanese, which contained the MD5 sum, the translated message, the file name and line number (the last two redundent perhaps but informational, and automatically generated if possible).The files in the Translations directory could be uesd as the unique keys for an external database (such as the Message Pedia, vendors or distributions help pages, etc.) to help look up and explain root cause of a problem. The key property here is that the MD5 sum becomes the key to all database entries to look up that key.
Further, yet another kernel config option could allow distros to output the calculated MD5 sum to be printed, much like we do with timestamps today.
End result is that these in-kernel message catalogs for translation are updated automatically (mostly no kernel developer changes needed) and the translations can be maintained by anyone who is interested.
On the topic of MD5 collisions, using a disambiguating tag would be a simple addition for the few cases where that happens, the tool could be educated to use that tag in the calculation of the MD5 sum, and we have a 98% solution which impacts <1% of the kernel developers.
Quoted 6 lines
Folks present for this discussion included Ted T'so, James Bottomley, several of the key Japenese folks interested in using this for debugging, and reps from several vendors who would find this sort of info useful for their folks supporting Linux in the field.
Comments?
For those of us who have not been in on these meetings, I think that some serious justifications are needed. The last paragraph began to go in that direction, but it needs to be more detailed and convincing. And "for debugging" doesn't cut it IMO.
Signature · 2 lines
~Randy *** Remember to use Documentation/SubmitChecklist when testing your code ***
Quoted 100 lines · Randy Dunlap
Quoted 90 lines · Gerrit Huizenga
Quoted 36 lines · holzheu
Quoted 5 lines · Jan Kara
<snip>
Quoted 3 lines
Your proposal is similar to one I made to some Japanese developers earlier this year. I was more modest, proposing that we
Quoted 8 lines
Quoted 3 lines
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);Maybe a stupid idea but why do we want to assign these numbers by hand?
I can imagine it could introduce collisions when merging tons of patches with new messages... Wouldn't it be better to compute say, 8-byte hash from the message and use it as it's identifier? We could do this automagically at compile time.Of course automatically generated message numbers would be great and something like:
hub.4a5bcd77: Detected some problem.
looks acceptable for me.
We could generate the hash using the format string of the printk. Since we specify the format string also in
KMSG_DOC, the hash for theKMSG_DOCand the printk should match and we have the required link between printk and description.So technically that's probably doable.
Problems are:
* hashes are not unique
* We need an additional preprocessor step
* The might be people, who find 8 character hash values ugly in printksThe big advantage is, that we do not need to maintain message numbers.
Quoted 8 lines
Quoted 4 lines
I know it also has it's problems - you
fix a spelling and the message gets a different id and you have to update translation/documentation catalogue but maybe that could be solved too...Since in our approach the message catalog is created automatically for exactly one kernel and the message catalog belongs therefore to exactly one kernel, I think the problem of changing error numbers is not too big.
We just had a meeting with the Japanese and several other participants from the vendor and community side and came up with a potential proposal that is similar to many things discussed here. It has the benefit that it seems implementable and low/no overhead on most kernel developers. The basic proposal is to use a tool, run by the kernel
Makefileto extract kernel messages from either the kernel source or the .i files (both have advantages, although I prefer the source to the .i file since it 1) gets all messages and 2) is probably a little quicker with less impact to the standard kernel make.
These messages would be stored in a file in the source tree, e.g.usr/src/linux/Translations/English. As each message is added to that file, we calculate, say, an MD5 sum of the printk (dev_printk,sdev_printk, etc.) string, and the text file ultimately contains:
MD5 Checksum of text; the printk text itself, the File name, the line number.
The checksum is run over just the printk. We definitely would not include the line number since the line number is too volatile. Including the file name in the hash *might* help disambiguate the hash a bit better in the case of duplicates, but there was some debate that duplicates might be better handled in other ways.
Andrew mentioned a mechanism for adding a subsystem tag or other tag which helps disambiguate the message, either in the message file or in the end user documentation (e.g. the Message Pedia/mPedia that the Japanese have already created with ~350 messages, and a total of ~700 targetted by the end of the year).
That tag could be appended to the beginning of the printk, to the end of the printk, or even in a formatted comment at the end of the printk that the tool could extract.
Then, the translations could be managed by anyone outside of the normal/ core kernel community, by simply creating a translation file, e.g.usr/src/linux/Translations/Japanese, which contained the MD5 sum, the translated message, the file name and line number (the last two redundent perhaps but informational, and automatically generated if possible). The files in the Translations directory could be uesd as the unique keys for an external database (such as the Message Pedia, vendors or distributions help pages, etc.) to help look up and explain root cause of a problem. The key property here is that the MD5 sum becomes the key to all database entries to look up that key.
Further, yet another kernel config option could allow distros to output the calculated MD5 sum to be printed, much like we do with timestamps today.
End result is that these in-kernel message catalogs for translation are updated automatically (mostly no kernel developer changes needed) and the translations can be maintained by anyone who is interested. On the topic of MD5 collisions, using a disambiguating tag would be aQuoted 8 lines
simple addition for the few cases where that happens, the tool could be educated to use that tag in the calculation of the MD5 sum, and we have a 98% solution which impacts <1% of the kernel developers. Folks present for this discussion included Ted T'so, James Bottomley, several of the key Japenese folks interested in using this for debugging, and reps from several vendors who would find this sort of info useful for their folks supporting Linux in the field.
Comments?
Quoted 4 lines
For those of us who have not been in on these meetings, I think that some serious justifications are needed. The last paragraph began to go in that direction, but it needs to be more detailed and convincing. And "for debugging" doesn't cut it IMO.
Full "justification" has never really been clear outside of the need to try to duplicate what other Unixes have had for their support departments in the past. And yes, I've seen and sat through the presentation by the Japanese group many different times, and talked to the individual people who want this.
And you know, that's fine. If anyone wants to go off and do such a project like this, in a way that is almost entirely self-contained with no affect on the development process or any kernel developer, I have no objection to it at all.
And I think the proposal that Andrew made meets that requirement, so let's wait and see what actual code comes out of this and judge it on that merit at that time.
thanks,
greg k-h
Quoted 8 lines · Randy Dunlap
For those of us who have not been in on these meetings, I think that some serious justifications are needed. The last paragraph began to go in that direction, but it needs to be more detailed and convincing. And "for debugging" doesn't cut it IMO.
Signature · 1 line
~Randy
Fair point, Randy.
This came up this time because people in Japan supporting Linux customers are not always familiar with Linux kernel internals and thus debugging problems based on a terse, English language message is just painful. Also, any given message doesn't really provide any input about what *really* failed, what should be done about it, how to correct this issue, or what parts to pull out of your machine and replace with properly working parts.
So, the Message Pedia is basically a Japanese language wiki today where people doing support can annotate the messages with real life experience on what to *do* about the error.
Part of the problem is that the error messages when extracted don't stand alone and don't easily allow tracing back to the real source. When the problem is handed from the second line support folks deeper into the support or development organization, the message has to be found in the source tree in some non-ambiguous fashion.
The other problem that comes up in other, non-English locales is that debugging your Linux box when you are a non-native English speaker is just plain impossible. Providing a means for getting localized kernel messages allows more end users to at least reasonably debug their own machine. Most every user level project today allows localization directly but the kernel has never had any reasonable mechanism for localisation.
Now, the proposal here doesn't make the kernel "as good as" user level application, but it would at least provide a searchable key to help people get localized guidance on problems.
I'm sure there are other benefits that most can see based on just simple debuggability. But the Japanese goal is really to construct a database of debugging info & context based on their experiences.
And, I'm sure that others can speak for themselves. I'm mostly playing scribe on this one - we abandoned our error and event subsystem work years ago, even though this was one of the biggest weaknesses we saw in customer support 3-5 years ago, simply because our changes were still viewed as too invasive for mainline kernel. These proposed changes help a lot, while remaining almost completely invisible to most developers.
gerrit
Hi Gerrit,
The common thing of your and our approach is, that we need an ID to identify a message either by:
* Using hashes + component name (maybe)
* Or using hand-made message numbers + component name. Numbers have to be unique within the kernel component
I think the main difference of your approach is, that documentation will be maintained outside the kernel. This has advantages, but also disadvantages:
(+) Developers have less work to do :-)
(+) Kernel sources and patches have less lines of code, since message
descriptions are not contained in the kernel sources.
(+) You can support multiple languages
(-) Hard to keep printks and descriptions up-to-date. If using hashes,
each typo fix in a printk breaks the link to your documentation.
(-) Since the developer knows the meaning of a printk best, he probably
would be the right person to write the description.
(-) For every Distribution or vanilla kernel you probably need separate
external message databases.
I like the fact, that every printk gets an ID in your approach and you do not need additional printk macros.
Maybe both approaches are not mutually exclusive. If developers would like to document their messages within the kernel sources, they still could use kernel-doc and our KMSG_DOC macro do do that.
I admit, that I am not sure about the the best solution here.
Michael
Quoted 4 lines · holzheu
Hi Gerrit,
The common thing of your and our approach is, that we need an ID to identify a message either by:
Maybe I am missing something big, but why is an ID needed? The message IS the ID right? That's the only thing that is robust against code moving about....
(And yes I am aware that very occasionally there might be a duplicate message that REALLY is different, but I suspect that that is so incredibly rare that it's just simpler to reword one of the two in such case)
Quoted 10 lines · Arjan van de Ven
Quoted 4 lines · holzheu
Hi Gerrit,
The common thing of your and our approach is, that we need an ID to identify a message either by:
Maybe I am missing something big, but why is an ID needed? The message IS the ID right? That's the only thing that is robust against code moving about....
I think the problem is messages contain device numbers, and all similar
kind of data. So what you'd really like is to use >format string< as ID. But that is not easily accessible for the user, so you'd have to print it with each message. And that's ... ugly IMHO.
Quoted 4 lines
(And yes I am aware that very occasionally there might be a duplicate message that REALLY is different, but I suspect that that is so incredibly rare that it's just simpler to reword one of the two in such case)
Honza
Signature · 2 lines
Jan Kara <[email protected]> SuSE CR Labs
Quoted 10 lines · Arjan van de Ven
Quoted 4 lines · holzheu
Hi Gerrit,
The common thing of your and our approach is, that we need an ID to identify a message either by:
Maybe I am missing something big, but why is an ID needed? The message IS the ID right? That's the only thing that is robust against code moving about....
Yes. As already discussed with Pavel, it is one option to use the format string of the message as message ID. The disadvantage compared to message IDs like hashes is, that format strings might be even less unique than hashes and it's probably less convenient for searching by operators.
An operator has to issue either:
Quoted 1 line
Quoted 1 line
info lp.4711
or
Quoted 1 line
Quoted 1 line
info "lp0: on fire"
Michael
Quoted 8 lines · holzheu
Quoted 7 lines · Arjan van de Ven
Quoted 4 lines · holzheu
Hi Gerrit,
The common thing of your and our approach is, that we need an ID to identify a message either by:
Quoted 8 lines
Quoted 3 lines
Maybe I am missing something big, but why is an ID needed? The message IS the ID right? That's the only thing that is robust against code moving about....
Yes. As already discussed with Pavel, it is one option to use the format string of the message as message ID. The disadvantage compared to message IDs like hashes is, that format strings might be even less unique than hashes
if the hash comes from the string in the first place I have a hard time believing that.
Quoted 2 lines
and it's probably less convenient for searching by operators.
I'm not convinced of that...
Quoted 6 lines
An operator has to issue either:
Quoted 1 line
Quoted 1 line
info lp.4711
or
Quoted 1 line
Quoted 1 line
info "lp0: on fire"
well.... surely the messages are caught by some userspace program, right? (like syslog).. that can do the lookup and make it all conveniently lookup-able and cross-referencable etc etc....
Signature · 2 lines
if you want to mail me at work (you don't), use arjan (at) linux.intel.com Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
Quoted 13 lines · Arjan van de Ven
Quoted 12 lines · holzheu
Quoted 10 lines · Arjan van de Ven
Quoted 4 lines · holzheu
Hi Gerrit,
The common thing of your and our approach is, that we need an ID to identify a message either by:
Maybe I am missing something big, but why is an ID needed? The message IS the ID right? That's the only thing that is robust against code moving about....
Quoted 8 lines
Quoted 4 lines
Yes. As already discussed with Pavel, it is one option to use the format string of the message as message ID. The disadvantage compared to message IDs like hashes is, that format strings might be even less unique than hashes
if the hash comes from the string in the first place I have a hard time believing that.
Just think of all messages containing %s, which can expand to every possible string.
Michael
Quoted 19 lines · holzheu
Quoted 18 lines · Arjan van de Ven
Quoted 16 lines · holzheu
Quoted 10 lines · Arjan van de Ven
Quoted 4 lines · holzheu
Hi Gerrit,
The common thing of your and our approach is, that we need an ID to identify a message either by:
Maybe I am missing something big, but why is an ID needed? The message IS the ID right? That's the only thing that is robust against code moving about....
Yes. As already discussed with Pavel, it is one option to use the format string of the message as message ID. The disadvantage compared to message IDs like hashes is, that format strings might be even less unique than hashes
Quoted 6 lines
Quoted 3 lines
if the hash comes from the string in the first place I have a hard time believing that.
Just think of all messages containing %s, which can expand to every possible string.
of course you hash on the string before you do the format stuff....
Signature · 2 lines
if you want to mail me at work (you don't), use arjan (at) linux.intel.com Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
On Mon, 2007-06-18 at 18:36 -0700, Arjan van de Ven wrote:
<snip>
Quoted 3 lines
well.... surely the messages are caught by some userspace program, right? (like syslog).. that can do the lookup and make it all conveniently lookup-able and cross-referencable etc etc....
Ok, I agree. Maybe that's really a good idea!
So what about the following proposal:
* Use printk format strings as message IDs
* Use kernel-doc (or similar mechanism) for message documentation
* Use Subsystem tag/component ID for each printk. This might be optional, but would make sense to get messages more unique.
* Implement a new tool (kmsg) to allow operators
to access kernel message descriptions in a convenient way.
The kmsg tool will read the dmsg buffer together with the list of documented printks, which has been created in the kernel build using the format strings defined in the KMSG_DOC macros. It matches the known format strings to the dmesg messages and informs the operator, for which message he can get a description.
Something like:
Quoted 1 line
Quoted 1 line
kmsg
Linux version 2.6.9-42.0.3.ELsmp...
....
[<ID>] lp0 on fire
Quoted 1 line
Quoted 1 line
kmsg <ID>
Desciption: your printer failed basic tests, you should check cabling
The <ID> is generated by the kmsg tool and could be e.g. the hashes for the matching format strings.
Michael
Hi Michael,
I initiated this discussion in The Linux Foundation Summit@Google Campus last week from the viewpoint of requirements by Japanese vendors/users.
As for a file to store message descriptions, I perfectly agree with your analysis and we also noticed some (+) and (-) you pointed out in the meeting last week. Our conclusion was to have a file outside of the kernel source codes to lessen developers burdens and to encourage better documentations of descriptions (as Andrew mentioned in his initial message).
Takashi Kunai,
The Linux Foundation Japan
Quoted 43 lines · holzheu
Hi Gerrit,
The common thing of your and our approach is, that we need an ID to identify a message either by:
* Using hashes + component name (maybe)
* Or using hand-made message numbers + component name. Numbers have to be unique within the kernel componentI think the main difference of your approach is, that documentation will be maintained outside the kernel. This has advantages, but also disadvantages:
(+) Developers have less work to do :-)
(+) Kernel sources and patches have less lines of code, since message
descriptions are not contained in the kernel sources.
(+) You can support multiple languages(-) Hard to keep printks and descriptions up-to-date. If using hashes,
each typo fix in a printk breaks the link to your documentation.
(-) Since the developer knows the meaning of a printk best, he probably
would be the right person to write the description.
(-) For every Distribution or vanilla kernel you probably need separate
external message databases.I like the fact, that every printk gets an ID in your approach and you do not need additional printk macros.
Maybe both approaches are not mutually exclusive. If developers would like to document their messages within the kernel sources, they still could use kernel-doc and our
KMSG_DOCmacro do do that.I admit, that I am not sure about the the best solution here.
Michael
_______________________________________________
Lf_kernel_messagesmailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/lf_kernel_messages
Signature · 5 lines
Kunai, Takashi The Linux Foundation Japan (New since '07/2/5) Shibuya Mark City West 22nd Floor 1-12-1 Dogenzaka,Shibuya-ku,Tokyo,Japan zip150-0043 tel+81-3-4360-5493
Quoted 10 lines · Gerrit Huizenga
Andrew mentioned a mechanism for adding a subsystem tag or other tag which helps disambiguate the message, either in the message file or in the end user documentation (e.g. the Message Pedia/mPedia that the Japanese have already created with ~350 messages, and a total of ~700 targetted by the end of the year).
That tag could be appended to the beginning of the printk, to the end of the printk, or even in a formatted comment at the end of the printk that the tool could extract.
The best way to maintain such a subsystem tag most be supported by kbuild. So say we have the subsystem "scsi" then we can set this subsystem tag in drivers/scsi/Makefile and let drivers/scsi/arm/Makefile inherit this vaule doing nothing.
Addign the subsystem tag to the printk could than be done using the C preprocessor and we are all fine.
If we go by the kernel-doc way to document kernel messages (the few that actually needs additional explanation) then we need to teach kernel-doc to collect the same subsystem tag but that should be doable.
Sam
Quoted 17 lines · On Mon, 18 Jun 2007 15:11:57 +0200 Sam Ravnborg wrote:
Quoted 10 lines · Gerrit Huizenga
Andrew mentioned a mechanism for adding a subsystem tag or other tag which helps disambiguate the message, either in the message file or in the end user documentation (e.g. the Message Pedia/mPedia that the Japanese have already created with ~350 messages, and a total of ~700 targetted by the end of the year).
That tag could be appended to the beginning of the printk, to the end of the printk, or even in a formatted comment at the end of the printk that the tool could extract.
The best way to maintain such a subsystem tag most be supported by kbuild. So say we have the subsystem "scsi" then we can set this subsystem tag in
drivers/scsi/Makefileand letdrivers/scsi/arm/Makefileinherit this vaule doing nothing.
Quoted 6 lines
Addign the subsystem tag to the printk could than be done using the C preprocessor and we are all fine.
If we go by the kernel-doc way to document kernel messages (the few that actually needs additional explanation) then we need to teach kernel-doc to collect the same subsystem tag but that should be doable.
Actually I would prefer that such a tool be a clone of scripts/kernel-doc and use a (slightly?) modified tag instead of the kernel-doc "/**" tag if this route is used at all.
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
Quoted 5 lines · Gerrit Huizenga
Further, yet another kernel config option could allow distros to output the calculated MD5 sum to be printed, much like we do with timestamps today.
Comments?
Would the compiled-in text then also become replaceable? Or is the MD5 sum output expected to be in addition to the regular English message?
If message replacement at compile-time is supported, this could allow for creating "short" versions of the messages, which could have a beneficial impact on kernel size.
Right now, it is possible to completely disable printks and re-claim about 100K of memory. However, in some embedded configurations, even if you are space-constrained it's desirable to retain some of the printks, for in-field debugging. Thus not very many embedded developers disable printks completely, even though the option has been supported for a while. (That, and many aren't caught up to the kernel version where it was introduced (2.6.10) :-)
But compressed messages (shortened text through abbreviations, or just outputting the ID alone, etc.) could save SOME of the space, in trade for less readability. Heck, just removing all vowels would probably save 10k, and not hurt readability that much.
Finally, for testing, it's handy to also
have automatic translation generators.
At a former company I worked for, they had translators that would output:
* all messages shortened by 20%
* all messages lengthened by 20%
* every message converted to pig-latin
These were mostly used for testing if the strings broke screen real-estate constraints (which don't apply to kernel messages). But the automatic translators would sometimes catch messages with weird attributes.
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America
=============================
Quoted 10 lines · Tim Bird
Quoted 5 lines · Gerrit Huizenga
Further, yet another kernel config option could allow distros to output the calculated MD5 sum to be printed, much like we do with timestamps today.
Comments?
Would the compiled-in text then also become replaceable? Or is the MD5 sum output expected to be in addition to the regular English message?
The MD5 sum would be calculated at print time, no proposal in here to replace the in-kernel text. And, I'm not sure that replacing it with an MD5 sum would every be significantly shorter, ergo I don't think this helps your problem.
The methods of post-processing to "shrink" the kernel text here seem too ugly to ponder. And I just pondered a few that made my head hurt (sort the MD5 sums, re-insert the number of the MD5 sum as an integer instead of the original text, and, beyond being gross, what do you do with the formatting info about the args?).
Quoted 4 lines
If message replacement at compile-time is supported, this could allow for creating "short" versions of the messages, which could have a beneficial impact on kernel size.
Quoted 8 lines
Right now, it is possible to completely disable printks and re-claim about 100K of memory. However, in some embedded configurations, even if you are space-constrained it's desirable to retain some of the printks, for in-field debugging. Thus not very many embedded developers disable printks completely, even though the option has been supported for a while. (That, and many aren't caught up to the kernel version where it was introduced (2.6.10) :-)
Which ones matter? Odds are you could use the KERNEL_LOGLEVEL or such to mark those, then compile out the rest.
Quoted 5 lines
But compressed messages (shortened text through abbreviations, or just outputting the ID alone, etc.) could save SOME of the space, in trade for less readability. Heck, just removing all vowels would probably save 10k, and not hurt readability that much.
Ick. Are you serious? How about running them through a valley girl filter and then converting to high school text messaging?
Quoted 7 lines
Finally, for testing, it's handy to also
have automatic translation generators.
At a former company I worked for, they had translators that would output:
* all messages shortened by 20%
* all messages lengthened by 20%
* every message converted to pig-latin
Double yucko.
Quoted 4 lines
These were mostly used for testing if the strings broke screen real-estate constraints (which don't apply to kernel messages). But the automatic translators would sometimes catch messages with weird attributes.
I don't think people are worried about the correctness of the messages and message formats - much of that can actually be detected by simple tools. The goal here was to at least allow people that support operating systems and for whom English (and English collation sequences) is not a primary language do some initial system diagosis.
I think compiling out the messages of something below some LOGLEVEL is a lot more practical.
gerrit
Hi!
Quoted 10 lines
Quoted 6 lines
Your proposal is similar to one I made to some Japanese developers earlier this year. I was more modest, proposing that we
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);Maybe a stupid idea but why do we want to assign these numbers by hand?
I can imagine it could introduce collisions when merging tons of patches with new messages... Wouldn't it be better to compute say, 8-byte hash from the message and use it as it's identifier? We could do this
Why hashes? Just key it off full message, and treat it like normal translation problem.
'lp%d: on fire' -> 'your printer failed basic tests, you should check cabling'.
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Hi Pavel,
Quoted 13 lines · Pavel Machek
Hi!
Quoted 10 lines
Quoted 6 lines
Your proposal is similar to one I made to some Japanese developers earlier this year. I was more modest, proposing that we
- add an enhanced printk
xxprintk(msgid, KERN_ERR "some text %d\n", some_number);Maybe a stupid idea but why do we want to assign these numbers by hand?
I can imagine it could introduce collisions when merging tons of patches with new messages... Wouldn't it be better to compute say, 8-byte hash from the message and use it as it's identifier? We could do this
Quoted 5 lines
Why hashes? Just key it off full message, and treat it like normal translation problem.
'lp%d: on fire' -> 'your printer failed basic tests, you should check cabling'.
Hmmm... What about extracting all format strings from the KMSG_DOC macros and storing them in a file. E.g.:
printk("lp%d: on fire"); /**
* message
*
* Description:
* Your printer failed basic tests, you should check cabling
...
KMSG_DOC(lp, "lp%d: on fire");
Some build mechanism creates a file with all format strings of the KMSG_DOC macros:
E.g.: cat kmsgs.txt
# MSG: "lp%d: on fire"
# COMPONENT: lp
# DESC: Your printer failed basic tests, you should check cabling
Then we provide a tool e.g. "kmsg", which gets as input the actual kernel message and searches the message database for matching format strings:
Quoted 2 lines
Quoted 2 lines
kmsg "lp0: on fire"
Your printer ...
Of course, we can get duplicate hits, too.
The advantage of that approach is, that we don't need additional message identifiers like hashes.
Michael
Hi!
Quoted 8 lines
Quoted 2 lines
'lp%d: on fire' -> 'your printer failed basic tests, you should check cabling'.
Hmmm... What about extracting all format strings from the
KMSG_DOCmacros and storing them in a file. E.g.:printk("lp%d: on fire");
Quoted 8 lines
/**
* message
*
* Description:
* Your printer failed basic tests, you should check cabling
...KMSG_DOC(lp, "lp%d: on fire");
Yes, that was the idea. But we have already tools and standard format for this (gettext), we should just use that.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Documentation of kernel messages (Summary)
In reply to Andrew MortonHi all,
Any idea, how to proceed with this topic? Do you think that any of the suggested solutions for documentation / translation of kernel messages will have a chance to be included in the kernel?
I tried to summarize the outcome of this discussion...
There are two main issues:
* Translate messages
* Document messages
For both, we have to give messages identifiers in order to match them to the corresponding documentation / translation. Three possible solutions have been suggested:
1. Use message numbers maintained by driver owners
2. Automatically create hashes for printks
3. Use printk format string as message identifier
Regarding the question where to put documentation and translations we have two options:
* In the kernel tree
* Somewhere outside the kernel tree
Another issue of the discussion was the usage of the component name for printks.
Useful comments:
================
Andrew Morton:
Suggested an alternative approach using a new printk function with message ID parameter. Documentation / Translation should be done outside the kernel.
Gerrit Huizenga:
Suggested an alternative approach using hashes as message IDs.
Greg Kroah-Hartman:
He pointed out, that we need a solution which integrates dev_xxx() macros.
Randy Dunlop:
He doesn't want to have message documentation included in the kernel-doc tool. We should create new tool for kernel messages.
Pavel Machek:
Suggested to use gettext for message documentation / translation.
Useful links:
=============
There already exists a mailing list for the kernel messages topic:
https://lists.linux-foundation.org/mailman/listinfo/lf_kernel_messages
There was an LWN article summarizing some aspects of the discussion:
http://lwn.net/Articles/238948/
Pros and Cons of the different Message ID methods:
==================================================
Message numbers maintained by driver owners:
+ Unique IDs
+ Can be kept stable between different kernel versions
+ Efficient way of matching printk to Documentation or translation
- Difficult to maintain
printk hashes:
+ Easy to maintain
+ Efficient way of matching printk to Documentation or translation
- Additional preprocessor step needed
- No unique IDs
- Ugly in messages, since hashes can have many characters
- Volatile, since typo fixes change hash values
Format strings:
+ No change needed for printks
+ Matching of printk to corresponding documentation or translation is not as efficient as using hashes
- No unique IDs
Pros and Cons of external documentation (vs. internal documentation):
====================================================================
+ Developers have less work to do
+ Kernel sources and patches have less lines of code, since message descriptions are not contained in the kernel sources.
+ You can support multiple languages
- Hard to keep printks and descriptions up-to-date. If using hashes, each typo fix in a printk breaks the link to your documentation.
- Since the developer knows the meaning of a printk best, he probably would be the right person to write the description.
- For every Distribution or vanilla kernel you probably need separate external message databases.
Pros and Cons of using component names in printks:
==================================================
+ Makes it easier to identify source of message
+ Makes message more unique (especially, when using Hashes or format strings as message IDs)
- Changes of printks necessary
Implementation issues (depending on what solution we implement):
===============================================================
* Preprocessor tool for hashes
* Check tool for messages/message description
* Tool to extract kernel documentation from kernel tree
* Sysadmin tool to query message descriptions
* New kernel macros for Messages to support Component names / message numbers
Re: Documentation of kernel messages (Summary)
Quoted 5 lines · On Monday 25 June 2007 09:48:41 Michael Holzheu wrote:
Hi all,
Any idea, how to proceed with this topic? Do you think that any of the suggested solutions for documentation / translation of kernel messages will have a chance to be included in the kernel?
Personally? No to the second question, which renders the first "do it yourself outside of the tree".
Just a guess, and I don't speak for anyone else here, but I think most of us are waiting to see how long it takes you to lose interest.
Quoted 6 lines
I tried to summarize the outcome of this discussion...
There are two main issues:
* Translate messages
* Document messages
I'm somewhat sympathetic to translation (although I tend to think of it as a distro's job, and still wonder why IBM is pushing this directly rather than trying to bounce it off Red Hat or somebody). But you could put a sed filter around dmesg to give you swahili output entirely in userspace. Have you shown the simple approach to be insufficient yet?
Is "document messages" the same problem as "translate messages", or are these two different problems conflated together?
I find "document messages" to be a horrible idea conceptually, because I think the messages should _be_ documentation. If the message is unclear it should be clarified. "There's too much garbage on the floor, we should laminate it so we can't smell it anymore." Er, no...
Quoted 5 lines
Three possible solutions have been suggested:
1. Use message numbers maintained by driver owners
2. Automatically create hashes for printks
3. Use printk format string as message identifier
#2 has the advantage that you can do it without buy-in from the community, and without touching the code. (Of course it doesn't necessarily help you if the printk string is "%s:%s".)
But I'm still convinced you could tackle over 90% of the problem from userspace, without touching the kernel.
Quoted 7 lines
Pros and Cons of the different Message ID methods:
==================================================
Message numbers maintained by driver owners:
+ Unique IDs
+ Can be kept stable between different kernel versions
+ Efficient way of matching printk to Documentation or translation
- Difficult to maintain
Will at best only ever be done for some drivers, not all. But that's probably inescapable no matter what your approach is.
Quoted 2 lines
printk hashes:
- Additional preprocessor step needed
Only for you guys. Those of us building our own kernels and NOT translating them or producing "the big book of explanations of kernel messages for IBM customers who don't understand them" shouldn't have to run this preprocessor step.
Quoted 11 lines
Pros and Cons of external documentation (vs. internal documentation):
====================================================================
+ Developers have less work to do
+ Kernel sources and patches have less lines of code, since message descriptions are not contained in the kernel sources.
+ You can support multiple languages- Hard to keep printks and descriptions up-to-date. If using hashes, each typo fix in a printk breaks the link to your documentation.
- Since the developer knows the meaning of a printk best, he probably would be the right person to write the description.
Imposing additional requirements on developers isn't going to work unless you reject patches from developers who don't follow your procedures and submit the appropriately filled-out forms with each patch for processing by the central bureaucracy.
Signed-off-by works because A) Linus enforces it, B) The overhead's very low,
C) attribution is already highly good thing within our community values, D) it annoys SCO.
The javadoc function comments in the source don't have complete coverage and probably never will (although they're getting better). The premise of doing something similar for messages is that a translating dmesg in userspace can't necessarily get complete coverage, so you need a solution modeled on something that doesn't have complete covereage years after its introduction...
Quoted 2 lines
- For every Distribution or vanilla kernel you probably need separate external message databases.
"If you're resigned to doing that anyway you don't actually need him to lie there all the time, do you?" - Ford Prefect
If you need an external message database what's wrong with doing it entirely in userspace via a sed-based translating dmesg?
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 4 lines
I find "document messages" to be a horrible idea conceptually, because I think the messages should _be_ documentation. If the message is unclear it should be clarified. "There's too much garbage on the floor, we should laminate it so we can't smell it anymore." Er, no...
"Document message" is for the cases where the message itself does not convey enough info.
No need to add more text when the initial message is good enough.
Sam
Re: Documentation of kernel messages (Summary)
In reply to Rob LandleyQuoted 9 lines · Rob Landley
Quoted 5 lines · On Monday 25 June 2007 09:48:41 Michael Holzheu wrote:
Hi all,
Any idea, how to proceed with this topic? Do you think that any of the suggested solutions for documentation / translation of kernel messages will have a chance to be included in the kernel?
Personally? No to the second question, which renders the first "do it yourself outside of the tree".
If that is the opinion of the majority here, fine. If there is no hard rule on how to define printk macros, one option for us would be to define some new s390 specific printk macros for our device drivers. Similar to hundreds of other driver specific printk macros in the kernel.
Quoted 2 lines
Just a guess, and I don't speak for anyone else here, but I think most of us are waiting to see how long it takes you to lose interest.
:-) Work is not always fun. Sometimes it is just a duty.
Michael
Re: Documentation of kernel messages (Summary)
Hi Micheal,
This is a late response to your summary below.
First of all, there is a lot of interest in this issue among Japanese vendor community. Recently, I have built a Japanese mailing list to discuss this issue. I'm posting this comment resulting from such discussion.
(1) Your kernel development proposal will be greatly supported by Japanese vendor community. At the same time, it needs support from the kernel communities, as well.
(2) As for message ID's, you proposed 3 ways; (a)adding message numbers, (b)printk hashes and (c)Format strings. (a) seems difficult to get supports from kernel developers and (c) lacks uniqueness of each message. Though (b) also lacks uniqueness, adding component id and/or file name (files name could be hashed, as well), it could achieve some practical level of uniqueness. To avoid ugly message format, it could be possible to introduce a system parameter to suppress this hash ids. Thus, our preference is (b).
(3) As for a documentation file, outside of kernel source tree would be fine.
(4) When this kernel message scheme become feasible, it will be needed to talk about the standardized contents of each description (meanings/actions/etc).
Takashi Kunai
Signature · 5 lines
Kunai, Takashi The Linux Foundation Japan (New since '07/2/5) Shibuya Mark City West 22nd Floor 1-12-1 Dogenzaka,Shibuya-ku,Tokyo,Japan zip150-0043 tel+81-3-4360-5493
Quoted 26 lines · Michael Holzheu
Quoted 8 lines · Rob Landley
Quoted 5 lines · On Monday 25 June 2007 09:48:41 Michael Holzheu wrote:
Hi all,
Any idea, how to proceed with this topic? Do you think that any of the suggested solutions for documentation / translation of kernel messages will have a chance to be included in the kernel?
Personally? No to the second question, which renders the first "do it yourself outside of the tree".
If that is the opinion of the majority here, fine. If there is no hard rule on how to define printk macros, one option for us would be to define some new s390 specific printk macros for our device drivers. Similar to hundreds of other driver specific printk macros in the kernel.
Quoted 2 lines
Just a guess, and I don't speak for anyone else here, but I think most of us are waiting to see how long it takes you to lose interest.
:-) Work is not always fun. Sometimes it is just a duty.
Michael
Re: Documentation of kernel messages (Summary)
Quoted 3 lines · Takashi
(1) Your kernel development proposal will be greatly supported by Japanese vendor community. At the same time, it needs support from the kernel communities, as well.
There is a very strong reason for the kernel community to NOT support
this: it makes it much harder to deal with bug reports.
Like it or not, English is a lingua franca[1] of the technology industry. When I deal with the rest of the kernel community, I use English, instead of my native language (Swedish).
For things that are targetted toward end users, that's a different matter, of course, but kernel messages are inherently directed toward developers and technically sophisticated users.
-hpa
Re: Documentation of kernel messages (Summary)
In regard to translating kernel messages:
Quoted 7 lines · On Monday 09 July 2007 01:36:31 H. Peter Anvin wrote:
Quoted 3 lines · Takashi
(1) Your kernel development proposal will be greatly supported by Japanese vendor community. At the same time, it needs support from the kernel communities, as well.
There is a very strong reason for the kernel community to NOT support
this: it makes it much harder to deal with bug reports.
Agreed.
There's a proposal in play for translations of documentation, and to recruit "language maintainers" (which is an idea I see that Matt Mackall came up with before I did).
A language maintainer is somebody who can accept a patch in language X, translate its description to post on the appropriate english-only list with a
cc: to the appropriate maintainer(s), and translate questions and reviews back and forth for the original author until the patch can be merged. (This means that documentation translations don't result in a stream of unmergeable patches.)
As for the documentation translations themselves, I note that Eric Raymond dissuaded me from actually hosting translations of kernel documentation on
http://kernel.org/doc due to his experience with translations of his own writings. If he hosts the translations on his website, they never get updated again. But if he makes the contributor host them on their own website, then they sometimes get updated.
For my part, I can't _tell_ when a given translation is out of date because I can't read it, and I certainly can't update it. So I agree with Eric and I'm linking to sites hosting kernel documentation in other languages rather than hosting them myself. I've got a good link to a Japanese site, need to ping the contributors of the chinese documentation and see if they have a site for it...
I await hearing back from Satoru Ueda who might have a lead on a potential Japanese maintainer, probably after some event in Japan (a "Jamboree") in a week or two...
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 12 lines · Rob Landley
In regard to translating kernel messages:
Quoted 7 lines · On Monday 09 July 2007 01:36:31 H. Peter Anvin wrote:
Quoted 3 lines · Takashi
(1) Your kernel development proposal will be greatly supported by Japanese vendor community. At the same time, it needs support from the kernel communities, as well.
There is a very strong reason for the kernel community to NOT support
this: it makes it much harder to deal with bug reports.Agreed.
[...]
Quoted 8 lines
As for the documentation translations themselves, I note that Eric Raymond dissuaded me from actually hosting translations of kernel documentation on
http://kernel.org/doc due to his experience with translations of his own writings. If he hosts the translations on his website, they never get updated again. But if he makes the contributor host them on their own website, then they sometimes get updated.
Quoted 6 lines
For my part, I can't _tell_ when a given translation is out of date because I can't read it, and I certainly can't update it. So I agree with Eric and I'm linking to sites hosting kernel documentation in other languages rather than hosting them myself. I've got a good link to a Japanese site, need to ping the contributors of the chinese documentation and see if they have a site for it...
Yeah, but it seems like having a translations directory in the kernel avoids that problem - anyone can update, it is a single source, no digging for sites that aren't tied to the kernel, available in the distros directly, etc.
I'm not sure that the web site hosting argument is a good one. Arch maintainers and Language Maintainers have some similarities. Also, being able to clearly mark which version of a document was last translated would help a bit with the fact that most of us aren't proficient in all of the world's languages. But, knowing who the translation maintainer is, and when the translation was last updated allows both the original doc maintainer and the translation document maintainer to see when a document likely needs to be updated. And that is probably good enough.
gerrit
Re: Documentation of kernel messages (Summary)
Quoted 30 lines · On Monday 09 July 2007 13:18:57 Gerrit Huizenga wrote:
Quoted 12 lines · Rob Landley
In regard to translating kernel messages:
Quoted 7 lines · On Monday 09 July 2007 01:36:31 H. Peter Anvin wrote:
Quoted 3 lines · Takashi
(1) Your kernel development proposal will be greatly supported by Japanese vendor community. At the same time, it needs support from the kernel communities, as well.
There is a very strong reason for the kernel community to NOT support
this: it makes it much harder to deal with bug reports.Agreed.
[...]
Quoted 7 lines
As for the documentation translations themselves, I note that Eric Raymond dissuaded me from actually hosting translations of kernel documentation on http://kernel.org/doc due to his experience with translations of his own writings. If he hosts the translations on his website, they never get updated again. But if he makes the contributor host them on their own website, then they sometimes get updated.
Quoted 6 lines
For my part, I can't _tell_ when a given translation is out of date because I can't read it, and I certainly can't update it. So I agree with Eric and I'm linking to sites hosting kernel documentation in other languages rather than hosting them myself. I've got a good link to a Japanese site, need to ping the contributors of the chinese documentation and see if they have a site for it...
Quoted 4 lines
Yeah, but it seems like having a translations directory in the kernel avoids that problem - anyone can update, it is a single source, no digging for sites that aren't tied to the kernel, available in the distros directly, etc.
No. It doesn't help.
99% of the kernel directory is C. That means any random passerby can review code. Everyone who has the kernel tarball should be able to review code that's in there, plus when you compile it breaks. So merging _code_ into the kernel helps keep it up to date.
Merging documentation into the kernel doesn't help keep it up to date, because documentation being out of date doesn't break the build. It may get the documentation more review, but the existing state of Documentation/* argues against that. It's a struggle to keep the english versions on the same continent as "up to date" or "complete", and most of the _good_ documentation is out in OLS papers and such (which I'm off indexing as we speak).
Now add in the non-english nature of the new Documentation/stale subdirectories you're proposing. Almost nobody in the existing kernel community can even _read_ this documentation, let alone update it. People who update Documentation/* _can't_ update the translations, and that will _never_change_.
Merging into the kernel is a great way to keep CODE up to date. Don't think it's magic pixie dust for documentation. It never has been before.
Quoted 2 lines
I'm not sure that the web site hosting argument is a good one. Arch maintainers and Language Maintainers have some similarities.
Ah, but It's not a language maintainer's job to update documentation. It's their job to ACCEPT PATCHES. Translate patches, translate questions back and forth. This has NOTHING to do with documentation unless they're converting documentation accompanying the patch one way; into english.
Quoted 4 lines
Also, being
able to clearly mark which version of a document was last translated would help a bit with the fact that most of us aren't proficient in all of the world's languages.
Keeping translations up to date is not something I can do, and thus not my problem.
Quoted 4 lines
But, knowing who the translation maintainer is,
and when the translation was last updated allows both the original doc maintainer and the translation document maintainer to see when a document likely needs to be updated. And that is probably good enough.
Thank you. You've just scared away all potential language maintainers by making them think we want them to translate all the world's existing documentation. Do you know how long it takes just to read through the existing Documentation directory? (Have you done it?)
When I'm talking about language maintainers I am NOT looking for them to translate existing documentation or keep Documentation/* up to date for some other langauge. That's a more than full-time job. I've been looking around for weeks for someone to just accept Japanese patches, and everybody I've talked to says it's too much time commitment without any extra work piled on top of it in by armchair commentators who won't be _doing_ that work.
Quoted 1 line
gerrit
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Hi Rob,
On Tue, 2007-07-10 at 12:12 -0400, Rob Landley wrote:
[snip]
Quoted 12 lines
Quoted 4 lines
Yeah, but it seems like having a translations directory in the kernel avoids that problem - anyone can update, it is a single source, no digging for sites that aren't tied to the kernel, available in the distros directly, etc.
No. It doesn't help.
99% of the kernel directory is C. That means any random passerby can review code. Everyone who has the kernel tarball should be able to review code that's in there, plus when you compile it breaks. So merging _code_ into the kernel helps keep it up to date.
Quoted 6 lines
Merging documentation into the kernel doesn't help keep it up to date, because documentation being out of date doesn't break the build. It may get the documentation more review, but the existing state of
Documentation/*argues against that. It's a struggle to keep the english versions on the same continent as "up to date" or "complete", and most of the _good_ documentation is out in OLS papers and such (which I'm off indexing as we speak).
With the checker tool, which we suggested in the initial proposal, it is possible to verify
* that every marked message has a description
* that there are no descriptions without corresponding messages
* that format strings of message and description match
So when compiling the kernel using C=1, you will at least see warnings, when a message has changed or a message disappeared:
Quoted 1 line
Quoted 1 line
make modules C=1
CHK include/linux/version.h CHK include/linux/utsrelease.h CHECK drivers/kmsgtest/kmsgtest.c
drivers/kmsgtest/kmsgtest.c: Missing description for: kmsgtest.1 drivers/kmsgtest/kmsgtest.c: Description without message for: kmsgtest.3
Michael
Re: Documentation of kernel messages (Summary)
In reply to Rob LandleyQuoted 71 lines · Rob Landley
Quoted 6 lines · On Monday 09 July 2007 13:18:57 Gerrit Huizenga wrote:
Quoted 5 lines · Rob Landley
In regard to translating kernel messages:
Quoted 2 lines · On Monday 09 July 2007 01:36:31 H. Peter Anvin wrote:
Quoted 1 line · Takashi
(1) Your kernel development proposal will be greatly supported
by
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Japanese vendor community. At the same time, it needs support
from
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
the kernel communities, as well.
There is a very strong reason for the kernel community to NOT
support
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
this: it makes it much harder to deal with bug reports.
Agreed.
[...]
Quoted 1 line
As for the documentation translations themselves, I note that
Eric
Quoted 3 lines
Quoted 3 lines
Raymond dissuaded me from actually hosting translations of kernel documentation on http://kernel.org/doc due to his experience with translations of his own writings. If he hosts the translations on
his
Quoted 1 line
Quoted 1 line
website, they never get updated again. But if he makes the
contributor
Quoted 3 lines
Quoted 3 lines
host them on their own website, then they sometimes get updated.
For my part, I can't _tell_ when a given translation is out of
date
Quoted 1 line
Quoted 1 line
because I can't read it, and I certainly can't update it. So I
agree
Quoted 1 line
Quoted 1 line
with Eric and I'm linking to sites hosting kernel documentation in
other
Quoted 1 line
Quoted 1 line
languages rather than hosting them myself. I've got a good link
to a
Quoted 1 line
Quoted 1 line
Japanese site, need to ping the contributors of the chinese
documentation
Quoted 3 lines
Quoted 1 line
and see if they have a site for it...
Yeah, but it seems like having a translations directory in the
kernel
Quoted 1 line
avoids that problem - anyone can update, it is a single source, no
digging
Quoted 2 lines
for sites that aren't tied to the kernel, available in the distros directly, etc.
No. It doesn't help.
99% of the kernel directory is C. That means any random passerby can review
code. Everyone who has the kernel tarball should be able to review code
that's in there, plus when you compile it breaks. So merging _code_ into the
kernel helps keep it up to date.Merging documentation into the kernel doesn't help keep it up to date, because
documentation being out of date doesn't break the build. It may get the
documentation more review, but the existing state ofDocumentation/*argues
against that. It's a struggle to keep the english versions on the same
continent as "up to date" or "complete", and most of the _good_ documentation
is out in OLS papers and such (which I'm off indexing as we speak).
Quoted 8 lines
Now add in the non-english nature of the new
Documentation/stalesubdirectories you're proposing. Almost nobody in the existing kernel
community can even _read_ this documentation, let alone update it. People
who updateDocumentation/*_can't_ update the translations, and that will
_never_change_.
There are quite a lot kernel developers for each of the popular language, AFAIK. For non-popular languages, there shouldn't be translation available in the first place. It was really a surprise when I post my Chinese translation on the LKML so many Chinese speakers have commented on the translation and encouraged the translation work. They are not visible as they usually don't talk too much. :) So I set up the Chinese kernel development maillist([email protected]), there will be more and more experienced kernel developer who can read and update the Chinese documents after they read the translated documentation and become kernel hacker.
Quoted 4 lines
Merging into the kernel is a great way to keep CODE up to date. Don't think
it's magic pixie dust for documentation. It never has been before.
IMHO, having contribution merged into the kernel has the MAGIC to attract people to work for recognition. When more and more people volunteer to work on it, the documentation will be up to date magically.
Quoted 5 lines
Quoted 2 lines
I'm not sure that the web site hosting argument is a good one.
Arch
Quoted 1 line
maintainers and Language Maintainers have some similarities.
Quoted 7 lines
Ah, but It's not a language maintainer's job to update documentation. It's
their job to ACCEPT PATCHES. Translate patches, translate questions back and
forth. This has NOTHING to do with documentation unless they're converting
documentation accompanying the patch one way; into english.
Language maintainers can integrate updates to the documentation just like integrating any updates to the code. Working on the documentation is ,IMHO, a perfect task for novice kernel hacker to get familiar with the process.
Quoted 26 lines
Quoted 3 lines
Also, being
able to clearly mark which version of a document was lasttranslated
Quoted 1 line
would help a bit with the fact that most of us aren't proficient in
all
Quoted 1 line
of the world's languages.
Keeping translations up to date is not something I can do, and thus not my
problem.Quoted 2 lines
But, knowing who the translation maintainer is,
and when the translation was last updated allows both the originaldoc
Quoted 1 line
maintainer and the translation document maintainer to see when a
document
Quoted 1 line
likely needs to be updated. And that is probably good enough.
Thank you. You've just scared away all potential language maintainers by
making them think we want them to translate all the world's existing documentation. Do you know how long it takes just to read through the
existing Documentation directory? (Have you done it?)
Quoted 11 lines
When I'm talking about language maintainers I am NOT looking for them to
translate existing documentation or keepDocumentation/*up to date for some
other langauge. That's a more than full-time job. I've been looking around
for weeks for someone to just accept Japanese patches, and everybody I've
talked to says it's too much time commitment without any extra work piled on
top of it in by armchair commentators who won't be _doing_ that work.
It won't be too hard if the work is shared by a community. Like the one I'm trying to establish.
- Leo
Advertisement time:
Chinese kernel development community http://zh-kernel.org :)
Re: Documentation of kernel messages (Summary)
Quoted 3 lines · Li Yang
There are quite a lot kernel developers for each of the popular language, AFAIK. For non-popular languages, there shouldn't be translation available in the first place.
I don't distinguish between "popular" and "non-popular" languages. If somebody's willing to do the translation work, it's popular. If nobody's willing to do the work, then even a language 1/3 of the planet's population speaks isn't "popular" for kernel development.
I wouldn't discourage a translator into Klingon if they were willing to keep their translation up to date and/or it actually resulted in patches.
Quoted 8 lines
It was really a surprise when
I post my Chinese translation on the LKML so many Chinese speakers have commented on the translation and encouraged the translation work. They are not visible as they usually don't talk too much. :) So I set up the Chinese kernel development maillist([email protected]), there will be more and more experienced kernel developer who can read and update the Chinese documents after they read the translated documentation and become kernel hacker.
The chinese mailing list is highly cool, and my first instinct was to note it on kernel.org/doc, but it would be better if the chinese website I already link to notes it instead. (That way I don't have to worry about how to spam-guard your email address. :)
This also highlights the need for language maintainers to help the patches go upstream to the english list. (If we can avoid armchair commentators saddling them with the task of translating the entire Documentation directory and keeping such a translation up to date, we might actually get one too. Fielding patches and questions sounds like plenty to me...)
Could you ask on said list if anyone is likely to volunteer for the position? (JUST translating patches and questions, as part of pushing code upstream.)
Quoted 7 lines
Quoted 3 lines
Merging into the kernel is a great way to keep CODE up to date. Don't think
it's magic pixie dust for documentation. It never has been before.IMHO, having contribution merged into the kernel has the MAGIC to attract people to work for recognition. When more and more people volunteer to work on it, the documentation will be up to date magically.
Obvious counter-arguments include the floppy driver, the floppy tape driver, the tty layer, and most of the existing english Documentation directory...
I'll happily stay out of the way of people who actually want to merge translations of documentation into the kernel. I reserve the right to say "I told you so" in about five years.
Quoted 8 lines
Quoted 7 lines
Ah, but It's not a language maintainer's job to update documentation. It's
their job to ACCEPT PATCHES. Translate patches, translate questions back and
forth. This has NOTHING to do with documentation unless they're converting
documentation accompanying the patch one way; into english.
Quoted 6 lines
Language maintainers can integrate updates to the documentation just like integrating any updates to the code. Working on the documentation is ,IMHO, a perfect task for novice kernel hacker to get familiar with the process.
From a language maintainer's perspective documentation patches wouldn't be any
different than any other patches. Translate the description and questions going back and forth. The patch itself doesn't get translated when it's C and applies to scripts/kconfig/, why would it when it's UTF-8 and applies to Documentation/?
Of course this brings up the question "what kind of function and variable names do chinese people pick?" (I honestly don't know, but I note that attempts to use names that don't fit in 7-bit ascii would probably be frowned upon in a big way...)
Quoted 2 lines
It won't be too hard if the work is shared by a community. Like the one I'm trying to establish.
A list works fine as a point of contact. I note that in general, maintainers are individuals (who delegate like mad, of course), because otherwise agenda items languish with everyone thinking it's someone else's responsibility.
http://www.trumanlibrary.org/buckstop.htm
Quoted 4 lines
- Leo
Advertisement time:
Chinese kernel development community http://zh-kernel.org :)
Hmmm... Now I'm wondering if I should link directly to the docs pages of the chinese and japanese sites, or to the top level community pages?
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 10 lines · Rob Landley
Quoted 3 lines
IMHO, having contribution merged into the kernel has the MAGIC to attract people to work for recognition. When more and more people volunteer to work on it, the documentation will be up to date magically.
Obvious counter-arguments include the floppy driver, the floppy tape driver, the tty layer, and most of the existing english Documentation directory...
I'll happily stay out of the way of people who actually want to merge translations of documentation into the kernel. I reserve the right to say "I told you so" in about five years.
One thing that would be a VERY good idea is to make sure that each translated document is tagged with when it was last translated, and against which kernel version (using either a GIT commit id or a 2.6.x.y revision, I don't care), and who is responsible for keeping that particular document translated. That way, it becomes a lot easier to recognize when something is stale, and if has gotten stale, we can just delete it.
If someone keeping some files translated into Klingon up to date for 3 months, and then disappears, we can just delete it, and it's no big deal. You can even say "I told you so" when you submit the patch to delete the stale translation. :-)
Quoted 3 lines
A list works fine as a point of contact. I note that in general, maintainers are individuals (who delegate like mad, of course), because otherwise agenda items languish with everyone thinking it's someone else's responsibility.
I agree; if we're going to have a language translated in Documentation/<lang>, there ought to be one or two overall language maintainer, plus a maintainer for each file that is being translated, who is responsible for updating the translated file when the base English Documentation file is updated.
- Ted
Re: Documentation of kernel messages (Summary)
Quoted 3 lines
One thing that would be a VERY good idea is to make sure that each translated document is tagged with when it was last translated, and against which kernel version (using either a GIT commit id or a
The English versions need a last updated too, that way we would know when they are past their best before date (as most of them are)
Re: Documentation of kernel messages (Summary)
Quoted 6 lines · Alan Cox
Quoted 3 lines
One thing that would be a VERY good idea is to make sure that each translated document is tagged with when it was last translated, and against which kernel version (using either a GIT commit id or a
The English versions need a last updated too, that way we would know when they are past their best before date (as most of them are)
Good idea. So a "last updated" date field will be required for both the English and translated documents. Date will be more straight forward than using commit id, IMHO.
- Leo
Re: Documentation of kernel messages (Summary)
Quoted 11 lines · Li Yang
Quoted 6 lines · Alan Cox
Quoted 3 lines
One thing that would be a VERY good idea is to make sure that each translated document is tagged with when it was last translated, and against which kernel version (using either a GIT commit id or a
The English versions need a last updated too, that way we would know when they are past their best before date (as most of them are)
Good idea. So a "last updated" date field will be required for both the English and translated documents. Date will be more straight forward than using commit id, IMHO.
Well, date and kernel version id, then. Otherwise they may be confusion if the translator is using 2.6.16.35 as opposed to 2.6.22-git3, etc. I would hope that the translator would always be using only the latest development branch, but I wouldn't want to always bet on that. One advantage of using git commit id is that there's no question about what the source document version was for the translation. It also makes it easier for the translator since they can just diff between the last git commit id, and HEAD, and then update their translation accordingly.
- Ted
Re: Documentation of kernel messages (Summary)
In reply to Alan CoxQuoted 2 lines · Alan Cox
The English versions need a last updated too, that way we would know when they are past their best before date (as most of them are)
I've got the impression that whenever I saw a "last updated:" note in a documentation, this note was out of date.
Signature · 3 lines
Stefan Richter -=====-=-=== -=== -==-= http://arcgraph.de/sr/
Re: Documentation of kernel messages (Summary)
Quoted 6 lines · Stefan Richter
Quoted 2 lines · Alan Cox
The English versions need a last updated too, that way we would know when they are past their best before date (as most of them are)
I've got the impression that whenever I saw a "last updated:" note in a documentation, this note was out of date.
RCS and SCCS let you stick stuff like $DATE and $VERSION into the source, so it would be automagically replaced at checkin/checkout time. Not sure how well that would work across a mesh of 'git pull' and 'git push' across trees. :)
Re: Documentation of kernel messages (Summary)
In reply to Alan CoxQuoted 6 lines · Alan Cox
Quoted 3 lines
One thing that would be a VERY good idea is to make sure that each translated document is tagged with when it was last translated, and against which kernel version (using either a GIT commit id or a
The English versions need a last updated too, that way we would know when they are past their best before date (as most of them are)
You can get this out of git annotate. (Or in my case, hg annotate from
http://kernel.org/hg/linux-2.6 .)
Cleaning up Documentation is on my todo list, but for this month I'm trying to integrate the existing Documentation files with things like the OLS papers, linux journal articles, lwn.net kernel articles, man-pages, and so on into one big index. (Indexing it requires reading large chunks of it. My brain hurts.)
If there's interest, I can push some patches to clean up Documentation by moving files into subdirectories, but Documentation's not well-suited to link out to the web. (You need html for that, and it's text.)
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 9 lines · On Fri, 13 Jul 2007 14:05:49 -0400 Rob Landley wrote:
Cleaning up Documentation is on my todo list, but for this month I'm trying to integrate the existing Documentation files with things like the OLS papers, linux journal articles, lwn.net kernel articles, man-pages, and so on into one big index. (Indexing it requires reading large chunks of it. My brain hurts.)
If there's interest, I can push some patches to clean up Documentation by moving files into subdirectories, but Documentation's not well-suited to link out to the web. (You need html for that, and it's text.)
I think that you should start moving Documentation/ files around and adding subdirectories -- if you are pretty sure of where they should go (i.e., they won't likely be moved again later on).
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
Re: Documentation of kernel messages (Summary)
Quoted 7 lines · Randy Dunlap
Quoted 3 lines
If there's interest, I can push some patches to clean up Documentation by moving files into subdirectories, but Documentation's not well-suited to link out to the web. (You need html for that, and it's text.)
I think that you should start moving
Documentation/files around and adding subdirectories -- if you are pretty sure of where they should go (i.e., they won't likely be moved again later on).
You mean like these?
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2925.html
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2924.html
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 5 lines · On Sat, 14 Jul 2007 21:56:15 -0400 Rob Landley wrote:
Quoted 4 lines · Randy Dunlap
Quoted 3 lines
If there's interest, I can push some patches to clean up Documentation by moving files into subdirectories, but Documentation's not well-suited to link out to the web. (You need html for that, and it's text.)
Quoted 7 lines
Quoted 3 lines
I think that you should start moving
Documentation/files around and adding subdirectories -- if you are pretty sure of where they should go (i.e., they won't likely be moved again later on).You mean like these?
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2925.html
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2924.html
Yes. Have any of these been merged?
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
Re: Documentation of kernel messages (Summary)
Quoted 11 lines · Randy Dunlap
Quoted 10 lines · On Sat, 14 Jul 2007 21:56:15 -0400 Rob Landley wrote:
Quoted 8 lines · Randy Dunlap
Quoted 4 lines
If there's interest, I can push some patches to clean up Documentation by moving files into subdirectories, but Documentation's not well-suited to link out to the web. (You need html for that, and it's text.)
I think that you should start moving
Documentation/files around and adding subdirectories -- if you are pretty sure of where they should go (i.e., they won't likely be moved again later on).
Quoted 5 lines
Quoted 3 lines
You mean like these?
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2925.html
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2924.htmlYes. Have any of these been merged?
Not that I know of. They seemed to meet with resounding indifference, so I went on to other things...
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 12 lines · On Mon, 16 Jul 2007 15:53:06 -0400 Rob Landley wrote:
Quoted 11 lines · Randy Dunlap
Quoted 10 lines · On Sat, 14 Jul 2007 21:56:15 -0400 Rob Landley wrote:
Quoted 8 lines · Randy Dunlap
Quoted 4 lines
If there's interest, I can push some patches to clean up Documentation by moving files into subdirectories, but Documentation's not well-suited to link out to the web. (You need html for that, and it's text.)
I think that you should start moving
Documentation/files around and adding subdirectories -- if you are pretty sure of where they should go (i.e., they won't likely be moved again later on).
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
You mean like these?
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2925.html
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2924.htmlYes. Have any of these been merged?
Not that I know of. They seemed to meet with resounding indifference, so I went on to other things...
You need to be persistent. Please re-submit this.
and since it's just file & dir moves and done via git, it should go to Linus. However, he may not want it right now since the primary merge window for 2.6.23 has closed (although he has been merging some other doc updates recently).
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
Re: Documentation of kernel messages (Summary)
Quoted 17 lines · Randy Dunlap
Quoted 16 lines · On Mon, 16 Jul 2007 15:53:06 -0400 Rob Landley wrote:
Quoted 15 lines · Randy Dunlap
Quoted 13 lines · On Sat, 14 Jul 2007 21:56:15 -0400 Rob Landley wrote:
Quoted 8 lines · Randy Dunlap
Quoted 4 lines
If there's interest, I can push some patches to clean up Documentation by moving files into subdirectories, but Documentation's not well-suited to link out to the web. (You need html for that, and it's text.)
I think that you should start moving
Documentation/files around and adding subdirectories -- if you are pretty sure of where they should go (i.e., they won't likely be moved again later on).You mean like these?
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2925.html
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2924.html
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Yes. Have any of these been merged?
Not that I know of. They seemed to meet with resounding indifference, so I went on to other things...
You need to be persistent. Please re-submit this.
Greg KH thinks it's a good idea to add language directories to the top level of Documentation, and there are slightly more than two of those:
http://www.translatorscafe.com/cafe/Help.asp?HelpID=59
Since you think it's worth doing, I'll resubmit the work I've already done here, but I no longer think trying to shovel Documentation into some semblance of order against the will of people like Greg is a good use of time.
These days I'm trying to create an html index that links into Documentation in a coherent order (with categories and everything), and using automated tools to detect files that aren't linked to, or links that point to a file that isn't there anymore. This is obviously still a work in progress, but I think it's a better approach.
Quoted 4 lines
and since it's just file & dir moves and done via git, it should go to Linus. However, he may not want it right now since the primary merge window for 2.6.23 has closed (although he has been merging some other doc updates recently).
If moving text files from one location to another in the Documentation directory breaks the build or causes bugs in the kernel, we have bigger problems. :)
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 25 lines · Rob Landley
Quoted 17 lines · Randy Dunlap
Quoted 16 lines · On Mon, 16 Jul 2007 15:53:06 -0400 Rob Landley wrote:
Quoted 15 lines · Randy Dunlap
Quoted 13 lines · On Sat, 14 Jul 2007 21:56:15 -0400 Rob Landley wrote:
Quoted 8 lines · Randy Dunlap
Quoted 4 lines
If there's interest, I can push some patches to clean up Documentation by moving files into subdirectories, but Documentation's not well-suited to link out to the web. (You need html for that, and it's text.)
I think that you should start moving
Documentation/files around and adding subdirectories -- if you are pretty sure of where they should go (i.e., they won't likely be moved again later on).You mean like these?
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2925.html
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2924.htmlQuoted 6 lines
Quoted 4 lines
Quoted 1 line
Yes. Have any of these been merged?
Not that I know of. They seemed to meet with resounding indifference, so I went on to other things...
You need to be persistent. Please re-submit this.
Quoted 8 lines
Greg KH thinks it's a good idea to add language directories to the top level of Documentation, and there are slightly more than two of those:
http://www.translatorscafe.com/cafe/Help.asp?HelpID=59Since you think it's worth doing, I'll resubmit the work I've already done here, but I no longer think trying to shovel Documentation into some semblance of order against the will of people like Greg is a good use of time.
My "will" is only that some of these documents be translated _and_ put in the main kernel source tree. I really have no opinion on where they need to go within that directory. If you have a better place for them, please let me know.
thanks,
greg k-h
Re: Documentation of kernel messages (Summary)
Quoted 36 lines · Greg KH
Quoted 26 lines · Rob Landley
Quoted 18 lines · Randy Dunlap
Quoted 17 lines · On Mon, 16 Jul 2007 15:53:06 -0400 Rob Landley wrote:
Quoted 16 lines · Randy Dunlap
Quoted 14 lines · On Sat, 14 Jul 2007 21:56:15 -0400 Rob Landley wrote:
Quoted 9 lines · Randy Dunlap
Quoted 4 lines
If there's interest, I can push some patches to clean up Documentation by moving files into subdirectories, but Documentation's not well-suited to link out to the web. (You need html for that, and it's text.)
I think that you should start moving
Documentation/files around and adding subdirectories -- if you are pretty sure of where they should go (i.e., they won't likely be moved again later on).You mean like these?
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2925.html
http://www.uwsg.iu.edu/hypermail/linux/kernel/0705.3/2924.htmlQuoted 6 lines
Quoted 4 lines
Quoted 1 line
Yes. Have any of these been merged?
Not that I know of. They seemed to meet with resounding indifference, so I went on to other things...
You need to be persistent. Please re-submit this.
Quoted 8 lines
Greg KH thinks it's a good idea to add language directories to the top level of Documentation, and there are slightly more than two of those:
http://www.translatorscafe.com/cafe/Help.asp?HelpID=59Since you think it's worth doing, I'll resubmit the work I've already done here, but I no longer think trying to shovel Documentation into some semblance of order against the will of people like Greg is a good use of time.
Quoted 4 lines
My "will" is only that some of these documents be translated _and_ put in the main kernel source tree. I really have no opinion on where they need to go within that directory. If you have a better place for them, please let me know.
A common subdirectory. I'd probably put it in "Documentation/translated" with a README at the top level of that explaining what it's for.
I can push a patch to move them, but reorganizing Documentation doesn't turn it into an index that can link out resources outside itself. If reorganizing it to be less of a compost heap is a separate goal in and of itself, I can do that...
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
In reply to Rob LandleyQuoted 6 lines · Rob Landley
These days I'm trying to create an html index that links into Documentation in a coherent order (with categories and everything), and using automated tools to detect files that aren't linked to, or links that point to a file that isn't there anymore. This is obviously still a work in progress, but I think it's a better approach.
Better than cleaning up what we have in the kernel source tree? Why not work on that first, then the "automated" type stuff would be much easier to do later, right?
thanks,
greg k-h
Re: Documentation of kernel messages (Summary)
Quoted 8 lines · Greg KH
Quoted 5 lines · Rob Landley
These days I'm trying to create an html index that links into Documentation in a coherent order (with categories and everything), and using automated tools to detect files that aren't linked to, or links that point to a file that isn't there anymore. This is obviously still a work in progress, but I think it's a better approach.
Better than cleaning up what we have in the kernel source tree?
Yes, I just said that.
Quoted 2 lines
Why not work on that first, then the "automated" type stuff would be much easier to do later, right?
How does an automated 404 checker that identifies files nothing's linking to get easier if the source files are in a different order? They could be alphabetical in one big directory and that would work the same.
Moving Documentation around is pointless churn that does nothing to prevent you from adding language directories to the top level on a whim, as if there were only two instead of (as I pointed out last message), several dozen. (While Randy Dunlap's poking me to resubmit my patch to group the architecture documentation into an architecture subdirectory, you're adding language directories to the top level instead of in their own subdirectory.)
Documentation doesn't even cross-link to the output of make htmldocs (which has its own structure imposed on it due to being extracted from the kernel sources). The kernel tarball has _two_ documentation sources that don't significantly cross-reference each other, and Rusty just submitted "make Preparation!" for lguest that's totally unrelated to either of them (and starts from a README file buried in the source code). None of this links to the menuconfig help entries, and the only reference in Documentation/ to "make help" is in Documentation/kbuild/makefiles.txt which explains that its purpose is to list the available architecture targets (something it does not, in my experience, actually do).
The idea that the kernel Documentation directory is the master repository of kernel documentation is an unworkable fantasy. The Documentation directory cannot index for all the kernel documentation resources out on the web, because it's in text not html. Documentation was created on the assumption that it would contain all interesting resources, as text files, but that doesn't match reality. Documentation is merely one resource among many, and to link _out_ you need HTML.
Some of the resources out there are organized chronologically, such as the Linux Journal archives http://www.linuxjournal.com/xstatic/magazine/archives,
the Ottawa Linux Symposium papers (http://kernel.org/doc/ols), or the kernel-traffic archives
http://www.kernel-traffic.org/kernel-traffic/archives.html. Some have their own indexes, such as the Linux Weekly News kernel articles
http://lwn.net/Kernel/Index/ and the Linux Device Drivers book
http://lwn.net/Kernel/LDD3/. Some are random bits picked out of developer blogs, found with google, reasonably coherent articles on wikipedia. Some are huge self-contained lumps on specific topics such as Mel Gorman's mm documentation or lots of the embedded stuff.
So no matter what reorganization I do to Documentation, its structure (or lack thereof) is incidental to coherently indexing most of the kernel documentation that's out there. (Right now the best index is "google" but that's only useful if you know what questions to ask. But getting up-to-date versions of Documentation and the output of make htmldocs on the web lets google find it. (Last year, back when I was still working on BusyBox, I did a google trawl for ext2 documentation to replace the horrible mke2fs busybox used to have, and the first three pages of hits did _NOT_ include a copy of Documentation/filesystems/ext2.txt. Ironically, if you google for "ext2 documentation" today the sixth hit is the unfinished ext2 documentation I'd just started to write before I found Documentation/filesystems/ext2.txt.)
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 2 lines · Rob Landley
Documentation is merely one resource among many, and to link _out_ you need HTML.
Do you suggest HTML files in linux/Documentation? I think we can follow URIs and other references in plaintext files just fine.
(People who want clickable links in plaintext files can use plaintext viewers which implement this, or utilities or browser plugins which follow URLs in the clipboard.)
Signature · 3 lines
Stefan Richter -=====-=-=== =--- --=-- http://arcgraph.de/sr/
Re: Documentation of kernel messages (Summary)
Quoted 5 lines · Stefan Richter
Quoted 2 lines · Rob Landley
Documentation is merely one resource among many, and to link _out_ you need HTML.
Do you suggest HTML files in linux/Documentation?
I've thought about it, but right now Documentation is text. Converting all the existing text files to html is a fairly intrusive change, and having multiple file formats in there seems even more untidy. (What next, pdf?).
I'm aware there are some docbook files in there, plus an ancient graphic file, plus the bloody stains of various chicken sacrificies, but the point would be how best to clean it up...
Quoted 2 lines
I think we can follow
URIs and other references in plaintext files just fine.
Go for it. Have fun. I'll be over here.
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
In reply to Rob LandleyQuoted 2 lines · Rob Landley
I wouldn't discourage a translator into Klingon if they were willing to keep their translation up to date and/or it actually resulted in patches.
The guys at the Klingon Language Institute are going to have a fit - what's the Klingon word for 'freezer' as used in our suspend code? :)
Re: Documentation of kernel messages (Summary)
Quoted 6 lines · On Wednesday 11 July 2007 5:53:54 pm [email protected] wrote:
Quoted 2 lines · Rob Landley
I wouldn't discourage a translator into Klingon if they were willing to keep their translation up to date and/or it actually resulted in patches.
The guys at the Klingon Language Institute are going to have a fit - what's the Klingon word for 'freezer' as used in our suspend code? :)
Considering that Klingon didn't have "to be" before Star Trek 6 needed hamlet, I suspect they'll manage somehow.
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
RE: Documentation of kernel messages (Summary)
In reply to Rob LandleyQuoted 7 lines · Rob Landley
Quoted 3 lines · Li Yang
There are quite a lot kernel developers for each of the popular language, AFAIK. For non-popular languages, there shouldn't be translation available in the first place.
I don't distinguish between "popular" and "non-popular" languages. If somebody's willing to do the translation work, it's popular. If
nobody's
Quoted 1 line
willing to do the work, then even a language 1/3 of the planet's
population
Quoted 3 lines
speaks isn't "popular" for kernel development.
I wouldn't discourage a translator into Klingon if they were willing
to keep
Quoted 4 lines
their translation up to date and/or it actually resulted in patches.
Quoted 2 lines
It was really a surprise when
I post my Chinese translation on the LKML so many Chinese speakers
have
Quoted 1 line
Quoted 1 line
commented on the translation and encouraged the translation work.
They
Quoted 1 line
Quoted 1 line
are not visible as they usually don't talk too much. :) So I set up
the
Quoted 1 line
Quoted 1 line
Chinese kernel development maillist([email protected]),
there
Quoted 5 lines
Quoted 3 lines
will be more and more experienced kernel developer who can read and update the Chinese documents after they read the translated documentation and become kernel hacker.
The chinese mailing list is highly cool, and my first instinct was to
note it
Quoted 1 line
on kernel.org/doc, but it would be better if the chinese website I
already
Quoted 1 line
link to notes it instead. (That way I don't have to worry about how
to
Quoted 1 line
spam-guard your email address. :)
The benefit of a localized maillist is that patches can be reviewed and commented in native language for common problems (like style, API).
Quoted 2 lines
This also highlights the need for language maintainers to help the
patches go
Quoted 2 lines
upstream to the english list. (If we can avoid armchair commentators saddling them with the task of translating the entire Documentation
directory
Quoted 1 line
and keeping such a translation up to date, we might actually get one
too.
Quoted 1 line
Fielding patches and questions sounds like plenty to me...)
I do think the documentation translation is very necessary even when there is a language maintainer, especially for the policy documents as HOWTO, codestyle , and etc. The contributors should go through these policies and check their code for compliance before going to the language maintainer for help, or there will be too much for the language maintainer to translate. The language maintainer doesn't need to translate all the documents himself, but he can help to coordinate the translation effort and help to make it update to date.
Quoted 2 lines
Could you ask on said list if anyone is likely to volunteer for the
position?
Quoted 1 line
(JUST translating patches and questions, as part of pushing code
upstream.)
If we do need a contact person, I can do it. However I don't think there will be much translation work to do here. As I stated before, most Chinese programmers are more or less capable of read/write technical English. The difficult part is to let them know the benefit of merging code in kernel and teach them how to do it. That's why the policy documents in native language will be very useful.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Merging into the kernel is a great way to keep CODE up to date.
Don't
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
think
it's magic pixie dust for documentation. It never has been
before.
Quoted 4 lines
Quoted 4 lines
IMHO, having contribution merged into the kernel has the MAGIC to attract people to work for recognition. When more and more people volunteer to work on it, the documentation will be up to date
magically.
Quoted 2 lines
Obvious counter-arguments include the floppy driver, the floppy tape
driver,
Quoted 1 line
the tty layer, and most of the existing english Documentation
directory...
Quoted 3 lines
I'll happily stay out of the way of people who actually want to merge translations of documentation into the kernel. I reserve the right to
say "I
Quoted 3 lines
told you so" in about five years.
Quoted 1 line
Quoted 1 line
Ah, but It's not a language maintainer's job to update
documentation.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
It's
their job to ACCEPT PATCHES. Translate patches, translate
questions
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
back and
forth. This has NOTHING to do with documentation unless they're converting
documentation accompanying the patch one way; into english.Language maintainers can integrate updates to the documentation just like integrating any updates to the code. Working on the
documentation
Quoted 1 line
Quoted 1 line
is ,IMHO, a perfect task for novice kernel hacker to get familiar
with
Quoted 3 lines
Quoted 1 line
the process.
From a language maintainer's perspective documentation patches
wouldn't be any
Quoted 1 line
different than any other patches. Translate the description and
questions
Quoted 1 line
going back and forth. The patch itself doesn't get translated when
it's C
Quoted 1 line
and applies to
scripts/kconfig/, why would it when it's UTF-8 and
applies to
Quoted 3 lines
Documentation/?Of course this brings up the question "what kind of function and
variable
Quoted 1 line
names do chinese people pick?" (I honestly don't know, but I note
that
Quoted 1 line
attempts to use names that don't fit in 7-bit ascii would probably be
frowned
Quoted 3 lines
upon in a big way...)
Quoted 1 line
It won't be too hard if the work is shared by a community. Like the
one
Quoted 3 lines
Quoted 1 line
I'm trying to establish.
A list works fine as a point of contact. I note that in general,
maintainers
Quoted 1 line
are individuals (who delegate like mad, of course), because otherwise
agenda
Quoted 1 line
items languish with everyone thinking it's someone else's
responsibility.
Got it. I can do it as long as it doesn't consume too much time. :)
Quoted 9 lines
http://www.trumanlibrary.org/buckstop.htm
Quoted 4 lines
- Leo
Advertisement time:
Chinese kernel development community http://zh-kernel.org :)Hmmm... Now I'm wondering if I should link directly to the docs pages
of the
Quoted 1 line
chinese and japanese sites, or to the top level community pages?
The documents and maillist are also linked in the front page. It's better to link the top level page.
- Leo
PATCH 1/1 Chinese Language Maintainer
Proposed patch adding Li Yang to MAINTAINERS, and Documentation describing what a language maintainer is.
--
Quoted 10 lines · On Thursday 12 July 2007 9:53:54 am Li Yang-r58472 wrote:
Quoted 3 lines
Quoted 1 line
I'm trying to establish.
A list works fine as a point of contact. I note that in general,
maintainers
Quoted 1 line
are individuals (who delegate like mad, of course), because otherwise
agenda
Quoted 1 line
items languish with everyone thinking it's someone else's
responsibility.
Got it. I can do it as long as it doesn't consume too much time. :)
Woot! See attached patch and sign-off-by if you want the job. :)
The bit about "the maintainer forwards, not the list" is a suggestion to prevent duplicates. It doesn't mean you can't delegate, just that if it's open season on the list translating stuff _and_ forwarding it, then linux-kernel will get dupes. (Plus other maintainers are more likely to pay attention to somebody they've heard from before.)
Greg KH had entire an OLS presentation about how the developer -> maintainer -> subsystem -> anderew+linus forwarding is more an ideal than a reality, but at least it gives us a frame of reference to diverge from:
http://lwn.net/Articles/240402/
MAINTAINERS +7 -0
@@ -2146,6 +2146,13 @@ W: http://auxdisplay.googlepages.com/ W: http://auxdisplay.googlepages.com/ S: Maintained +LANGUAGE (CHINESE) +P: Li Yang +M: [email protected] +L: [email protected] +W: http://zh-kernel.org +S: Maintained + LAPB module L: [email protected] S: Orphan
Documentation/language-maintainers.txt new file +20 -0
@@ -0,0 +1,20 @@ +The langauges in which the Linux kernel is developed are C and English. +(Note that Linus Torvalds' native language is Swedish, yet he chose +to hold technical discussions in English.) + +The job of a language maintainer is to provide a point of contact for +non-english speaking developers who wish to merge their patches into the +Linux kernel. Each language needs a specific language maintainer, who +accepts non-english patch submissions on behalf of the Linux kernel. + +A language maintainer accepts patches to the Linux kernel, written in C, from +authors who do not also speak English. The language maintainer translates the +description of each patch into English, forwards the patches to linux-kernel +and to the appropriate maintainers for inclusion in the Linux kernel, and +translates questions and replies about such patches as part of the +patch review process. + +Some language maintainers provide a mailing list as a point of contact, to +distribute the translation work, but the maintainer is still the person who +ultimately forwards the results (to prevent duplicates), and the one to contact +if patches and questions don't get translated and forwarded in a timely fashion.
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: [PATCH] Chinese Language Maintainer
Documentation/language-maintainers.txt line 1
Quoted 4 lines · Rob Landley
Documentation/language-maintainers.txt new file +1
-0@@ -0,0 +1,20 @@ +The langauges in which the Linux kernel is developed are C and English.
^^^^^^^^^
languagesDocumentation/language-maintainers.txt line 8
Quoted 7 lines
Documentation/language-maintainers.txt new file +7
-0+(Note that Linus Torvalds' native language is Swedish, yet he chose +to hold technical discussions in English.) + +The job of a language maintainer is to provide a point of contact for +non-english speaking developers who wish to merge their patches into the +Linux kernel. Each language needs a specific language maintainer, who +accepts non-english patch submissions on behalf of the Linux kernel.
^^^^^^^^^^^ non-English
Documentation/language-maintainers.txt line 11
Quoted 3 lines
Documentation/language-maintainers.txt new file +3
-0+ +A language maintainer accepts patches to the Linux kernel, written in C, from +authors who do not also speak English. The language maintainer translates the
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Re: [PATCH] Chinese Language Maintainer
In reply to Rob LandleyHi!
Documentation/language-maintainers.txt line 2
Quoted 3 lines
Documentation/language-maintainers.txt new file +2
-0@@ -0,0 +1,20 @@ +The langauges in which the Linux kernel is developed are C and English. +(Note that Linus Torvalds' native language is Swedish, yet he chose
choose?
Documentation/language-maintainers.txt line 15
Quoted 6 lines
Documentation/language-maintainers.txt new file +6
-0+A language maintainer accepts patches to the Linux kernel, written in C, from +authors who do not also speak English. The language maintainer translates the +description of each patch into English, forwards the patches to linux-kernel +and to the appropriate maintainers for inclusion in the Linux kernel, and +translates questions and replies about such patches as part of the +patch review process.
Poor person... I'm not sure this can work for non-trivial patches.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Re: [PATCH] Chinese Language Maintainer
Quoted 7 lines · Pavel Machek
Documentation/language-maintainers.txt line 2
Quoted 3 lines
Documentation/language-maintainers.txt new file +2
-0@@ -0,0 +1,20 @@ +The langauges in which the Linux kernel is developed are C and English. +(Note that Linus Torvalds' native language is Swedish, yet he chosechoose?
From `dict choose':
Choose \Choose\, v. t. [imp. {Chose}; p. p. {Chosen}, {Chose}
(Obs.); p. pr. & vb. n. {Choosing}.] [OE. chesen, cheosen, AS. ce['o]san; akin to OS. kiosan, D. kiezen, G. kiesen, Icel. kj[=o]sa, Goth. kiusan, L. gustare to taste, Gr. ?, Skr. jush to enjoy. [root]46. Cf. {Choice}, 2d {Gust}.]
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Re: [PATCH] Chinese Language Maintainer
In reply to Pavel MachekQuoted 7 lines · Pavel Machek
Hi!
Documentation/language-maintainers.txt line 2
Quoted 3 lines
Documentation/language-maintainers.txt new file +2
-0@@ -0,0 +1,20 @@ +The langauges in which the Linux kernel is developed are C and English. +(Note that Linus Torvalds' native language is Swedish, yet he chosechoose?
I was trying for past tense.
Quoted 8 lines
Quoted 6 lines
(quoted diff) +1
-0+A language maintainer accepts patches to the Linux kernel, written in C,from +authors who do not also speak English. The language maintainer translates the +description of each patch into English, forwards the patches to linux-kernel +and to the appropriate maintainers for inclusion in the Linux kernel, and +translates questions and replies about such patches as part of the +patch review process.
Poor person... I'm not sure this can work for non-trivial patches.
We'll see. Personally, I'm more worried about how people who don't speak english are supposed to choose function and variable names. But translating _just_ the patch is not the end of it, and I wanted to make that clear.
I suppose a translator can always push back and ask for a _concise_ review, for the sake of translation. But let's not invent problems before they actually occur...
Quoted 1 line
Pavel
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: [PATCH] Chinese Language Maintainer
In reply to Rob LandleyQuoted 4 lines · Rob Landley
Proposed patch adding Li Yang to
MAINTAINERS, and Documentation describing what a language maintainer is.Signed-off-by Rob Landley <[email protected]>
Quoted 55 lines
--
Quoted 10 lines · On Thursday 12 July 2007 9:53:54 am Li Yang-r58472 wrote:
Quoted 3 lines
Quoted 1 line
I'm trying to establish.
A list works fine as a point of contact. I note that in general,
maintainers
Quoted 1 line
are individuals (who delegate like mad, of course), because otherwise
agenda
Quoted 1 line
items languish with everyone thinking it's someone else's
responsibility.
Got it. I can do it as long as it doesn't consume too much time. :)
Woot! See attached patch and sign-off-by if you want the job. :)
The bit about "the maintainer forwards, not the list" is a suggestion to prevent duplicates. It doesn't mean you can't delegate, just that if it's open season on the list translating stuff _and_ forwarding it, then linux-kernel will get dupes. (Plus other maintainers are more likely to pay attention to somebody they've heard from before.)
Greg KH had entire an OLS presentation about how the developer -> maintainer -> subsystem -> anderew+linus forwarding is more an ideal than a reality, but at least it gives us a frame of reference to diverge from:
http://lwn.net/Articles/240402/diff -r
edfd2d6f670dMAINTAINERSMAINTAINERS +7
-0@@ -2146,6 +2146,13 @@ W: http://auxdisplay.googlepages.com/ W: http://auxdisplay.googlepages.com/ S: Maintained +LANGUAGE (CHINESE) +P: Li Yang +M: [email protected] +L: [email protected] +W: http://zh-kernel.org +S: Maintained + LAPB module L: [email protected] S: OrphanDocumentation/language-maintainers.txt new file +5
-0@@ -0,0 +1,20 @@ +The langauges in which the Linux kernel is developed are C and English. +(Note that Linus Torvalds' native language is Swedish, yet he chose +to hold technical discussions in English.) + +The job of a language maintainer is to provide a point of contact for
Documentation/language-maintainers.txt line 15
Quoted 10 lines
Documentation/language-maintainers.txt new file +10
-0+non-english speaking developers who wish to merge their patches into the +Linux kernel. Each language needs a specific language maintainer, who +accepts non-english patch submissions on behalf of the Linux kernel. + +A language maintainer accepts patches to the Linux kernel, written in C, from +authors who do not also speak English. The language maintainer translates the +description of each patch into English, forwards the patches to linux-kernel +and to the appropriate maintainers for inclusion in the Linux kernel, and +translates questions and replies about such patches as part of the +patch review process.
In addiction to this responsibility, I would like to add two more which, in my opinion, are more important. And these are what I'm trying to do. :)
First, promoting contribution to Linux kernel in local language. Second, coordinate the translation effort of key kernel documents.
Quoted 6 lines
Documentation/language-maintainers.txt new file +5
-0+ +Some language maintainers provide a mailing list as a point of contact, to +distribute the translation work, but the maintainer is still the person who +ultimately forwards the results (to prevent duplicates), and the one to contact +if patches and questions don't get translated and forwarded in a timely fashion.
Re: [PATCH] Chinese Language Maintainer
Quoted 7 lines · Li Yang
Quoted 4 lines · Rob Landley
Proposed patch adding Li Yang to
MAINTAINERS, and Documentation describing what a language maintainer is.Signed-off-by Rob Landley <[email protected]>Signed-off-by Li Yang <[email protected]>
Yay!
Quoted 7 lines
Quoted 6 lines
+A language maintainer accepts patches to the Linux kernel, written in C, from +authors who do not also speak English. The language maintainer translates the +description of each patch into English, forwards the patches to linux-kernel +and to the appropriate maintainers for inclusion in the Linux kernel, and +translates questions and replies about such patches as part of the +patch review process.
Quoted 5 lines
In addiction to this responsibility, I would like to add two more which, in my opinion, are more important. And these are what I'm trying to do. :)
First, promoting contribution to Linux kernel in local language. Second, coordinate the translation effort of key kernel documents.
Cool. It's good to do that, but not the problem I'm worried about solving.
I was trying to describe the minimum requirements for being a language maintainer, I.E. what non-english users need in order to be able to merge their patches. Because without someone to contribute patches to (I.E a language maintainer), documentation in non-english languages promotes the creation of patches that can't be merged. That's the problem I'm trying to solve.
To me, finding language maintainers is the flip side of the coin of translating documentation.
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: [PATCH] Chinese Language Maintainer
Quoted 24 lines · Rob Landley
Quoted 8 lines · Li Yang
Quoted 6 lines · Rob Landley
+A language maintainer accepts patches to the Linux kernel, written in C, from +authors who do not also speak English. The language maintainer translates the +description of each patch into English, forwards the patches to linux-kernel +and to the appropriate maintainers for inclusion in the Linux kernel, and +translates questions and replies about such patches as part of the +patch review process.
Quoted 5 lines
In addiction to this responsibility, I would like to add two more which, in my opinion, are more important. And these are what I'm trying to do. :)
First, promoting contribution to Linux kernel in local language. Second, coordinate the translation effort of key kernel documents.Cool. It's good to do that, but not the problem I'm worried about solving.
I was trying to describe the minimum requirements for being a language maintainer, I.E. what non-english users need in order to be able to merge their patches. Because without someone to contribute patches to (I.E a language maintainer), documentation in non-english languages promotes the creation of patches that can't be merged. That's the problem I'm trying to solve.
Quoted 2 lines
To me, finding language maintainers is the flip side of the coin of translating documentation.
I think you worried too much about this problem. :) Let me explain the situation here in China more clearly. Actually, English is mandatory in most schools and universities. Only very few people learn other language as a second language. Therefore software developers who are almost educated should have the basic English skill. However, that doesn't mean that they can read English or communicate with native English speaker very easily. Consider your second language learn in school for analogy. Read in English will be much slower and more likely to cause misunderstanding. This will reduce the likelihood greatly of English documentation being read. If we are promoting contribution to the Linux community, we should maximum the possibility that these key documents being read. Translation will serve this purpose very well.
So the possibility is very little that a translator is needed between the Linux maintainer and a Chinese developer. Although sometimes help is needed when there is misunderstanding.
After a brief talk with the Japanese translator, I think the case is similar for Japanese too.
Therefore, in my opinion, language maintainer should be more a helper and promoter rather than a gatekeeper. I will give a proposed process later about how this helper mechanism works.
- Leo
Re: [PATCH] Chinese Language Maintainer
Quoted 9 lines · Li Yang
I think you worried too much about this problem. :) Let me explain the situation here in China more clearly. Actually, English is mandatory in most schools and universities. Only very few people learn other language as a second language. Therefore software developers who are almost educated should have the basic English skill. However, that doesn't mean that they can read English or communicate with native English speaker very easily. Consider your second language learn in school for analogy.
Actually, I disagree. English *is* the second language learned in school for most European developers (except, obviously, the ones from the British isles), and we don't have that problem.
Quoted 6 lines
Read in English will be much slower and more likely to cause misunderstanding. This will
reduce the likelihood greatly of English documentation being read. If we are promoting contribution to the Linux community, we should maximum the possibility that these key documents being read. Translation will serve this purpose very well.
What we have found in Europe, is that that it has limited value, and that the closer to the core you are, the less value it is, because at that stage you should be communicating more with other developers. Putting yourself behind a wall of translation is unfortunately a detriment in that way.
-hpa
Re: [PATCH] Chinese Language Maintainer
O> Actually, I disagree. English *is* the second language learned in
Quoted 2 lines
school for most European developers (except, obviously, the ones from the British isles), and we don't have that problem.
Not all those from the British Isles. A first language an English as school language is not that uncommon for segments of the population, and in Wales schooling may also be such that English is learned as a first foreign language.
That aside, please remember that Europe as a whole is a small place on the bigger world stage. The total volume of potential developers in the huge and rapidly modernising nations like India and China is vast, and there are large highly skilled technical nations that don't teach English to everyone technical by default.
Key documents in other languages is a big help, especially those about values, culture and standards because they are things that are not easy to understand if your use of English is technically oriented.
Alan
Re: [PATCH] Chinese Language Maintainer
Quoted 15 lines · Alan Cox
O> Actually, I disagree. English *is* the second language learned in
Quoted 2 lines
school for most European developers (except, obviously, the ones from the British isles), and we don't have that problem.
Not all those from the British Isles. A first language an English as school language is not that uncommon for segments of the population, and in Wales schooling may also be such that English is learned as a first foreign language.
That aside, please remember that Europe as a whole is a small place on the bigger world stage. The total volume of potential developers in the huge and rapidly modernising nations like India and China is vast, and there are large highly skilled technical nations that don't teach English to everyone technical by default.
Quoted 3 lines
Key documents in other languages is a big help, especially those about values, culture and standards because they are things that are not easy to understand if your use of English is technically oriented.
No doubt. My point was merely that the closer to the core of development, the less translated documents help as the emphasis on interaction *has* to increase.
In that sense, a translation of "Linux Kernel Internals" and the other books written on the Linux kernel is more useful. That doesn't mean, of course, that there isn't documentation distributed with the kernel that is intended for users and therefore more useful in translation. However, I do feel that trying to keep up-to-date translations of design documentation is at least to some degree a fool's errand, which ends up having people rely on incomplete and outdated documentation, and cut them off from the overall community.
-hpa
Re: [PATCH] Chinese Language Maintainer
Quoted 8 lines · H. Peter Anvin
Quoted 3 lines · Alan Cox
Key documents in other languages is a big help, especially those about values, culture and standards because they are things that are not easy to understand if your use of English is technically oriented.
In that sense, a translation of "Linux Kernel Internals" and the other books written on the Linux kernel is more useful.
Several groups of people are translating the "important bits" (a different selection for each culture) of the kernelnewbies documentation to different languages already:
http://kernelnewbies.org/RegionalNewbies
I can easily set up additional wikis for anybody who wants to set up such a kernel documentation site in their language.
Signature · 3 lines
Politics is the struggle between those who want to make their country the best in the world, and those who believe it already is. Each group calls the other unpatriotic.
Re: [PATCH] Chinese Language Maintainer
In reply to Alan CoxQuoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Alan Cox <[email protected]> writes:
That aside, please remember that Europe as a whole is a small place on the bigger world stage. The total volume of potential developers in the huge and rapidly modernising nations like India and China is vast, and there are large highly skilled technical nations that don't teach English to everyone technical by default.
Speaking for India only, while the medium of instruction in primary and high school is available in native languages, English is almost universally preferred. Moreover, technical education is completely in English. I don't see much value in translating developer documentation into Indian languages. User documentation is another story.
Ganesan
Signature · 1 line
Ganesan Rajagopal
Re: [PATCH] Chinese Language Maintainer
Quoted 3 lines
preferred. Moreover, technical education is completely in English. I don't see much value in translating developer documentation into Indian languages. User documentation is another story.
I am not convinced that users do not become developers, nor that we should have language barriers that make this unneccessarily hard.
Alan
Re: [PATCH] Chinese Language Maintainer
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Alan Cox <[email protected]> writes:
preferred. Moreover, technical education is completely in English. I don't see much value in translating developer documentation into Indian languages. User documentation is another story.
I am not convinced that users do not become developers, nor that we should have language barriers that make this unneccessarily hard.
Again speaking only from the Indian context, we're somewhat unique when compared to China and Japan ;-). It's extremely unlikely that a non-english speaking user will become a developer. As I mentioned in my previous mail technical education is almost completely in English.
Most Indian IT work places communicate almost exclusively in English, including verbal communication because of employees from all over India with different native languages. English is not a language barrier for developers or even technical users from India.
Ganesan
Signature · 1 line
Ganesan Rajagopal
Re: [PATCH] Chinese Language Maintainer
In reply to H. Peter AnvinQuoted 11 lines · H. Peter Anvin
Quoted 9 lines · Li Yang
I think you worried too much about this problem. :) Let me explain the situation here in China more clearly. Actually, English is mandatory in most schools and universities. Only very few people learn other language as a second language. Therefore software developers who are almost educated should have the basic English skill. However, that doesn't mean that they can read English or communicate with native English speaker very easily. Consider your second language learn in school for analogy.
Quoted 3 lines
Actually, I disagree. English *is* the second language learned in school for most European developers (except, obviously, the ones from the British isles), and we don't have that problem.
Then this could be the main reason for the disagreement we have on this topic. Many people here don't grasp English very well even after the English classes.
- Leo
Re: [PATCH] Chinese Language Maintainer
In reply to H. Peter AnvinQuoted 11 lines · H. Peter Anvin
Quoted 8 lines · Li Yang
I think you worried too much about this problem. :) Let me explain the situation here in China more clearly. Actually, English is mandatory in most schools and universities. Only very few people learn other language as a second language. Therefore software developers who are almost educated should have the basic English skill. However, that doesn't mean that they can read English or communicate with native English speaker very easily. Consider your second language learn in school for analogy
Quoted 3 lines
Actually, I disagree. English *is* the second language learned in school for most European developers (except, obviously, the ones from the British isles), and we don't have that problem.
There is somewhat of a difference though: English and both our native languages are very much related. English, Dutch (my native language) and Swedish (yours) are all Germanic languages -- English and Dutch both West Germanic and Swedish only slightly farther removed as North Germanic. Word for word translations not _too_ infrequently actually make okay or at least understandable Dutch...
Other than Germanic, the Slavic (Polish, Czech, Russian, ...) and Latin (Italian, Spanish, French, ...) language families are two other direct Indo-European descendants that are fairly well represented in the kernel community but I believe it's a largely unsurprising observation that members of both these families have a somewhat harder time adopting English. And Chinese is not even Indo-European...
However -- in Europe I notice culture might be even more important than school _or_ language family is and as such I believe the above argument isn't all that important anyway. English is the second language we learn in school here in the Netherlands but it's much more popular than say German, a language even closer to Dutch, due to us having heard English basically from the time we're old enough to hear music and watch TV. Most _Dutch_ bands sing in English and the ones that don't are for a part targetting the elderly and/or mentally handicapped...
German is close enough to Dutch (and English itself) to rule out most differences related to native language, but in Germany a significantly smaller percentage of people speaks English well enough to be comfortable with it than in the Netherlands simply due to them producing more of their own culture. It's a larger language zone to market to and they dub the remaining English-language content on TV. Over here in the Netherlands we sub-title.
France is another good example. While a bit farther removed from English family-wise, English has had lots of influences from French as well and in any case, learning English shouldn't be harder for a Frenchman than learning French is for a Dutchman (we are, or were when I was there, taught French in school as well) which is to say not very hard. Yet, mastery of English is extremely poor in France. Not as a coincedence, most _all_ of the French culture is in the French language including dubbed originally English songs for example.
Both the German and, slower, French examples get less true with every new generation, but both still hold...
Popular culture in the sense of music, tv and these days very much games is something you start to experience at a very young age, years younger than the Chinese will be taught English in school and age is extremely important in mastering a language -- the human brain is by far best at it at the time you start mastering your native language (in fact, this is what defines "native") and every single year after that makes it harder.
That is -- let's just solve our Chinese translation problem by overthrowing the Chinese government and forcing the sub-titled Harry Potter film-series down the throat of the population... |-/
Seriously, I only wanted to say that both language family and (import of) culture are very important and as such, concentrating on schooling alone might not be all that sensible. I did not want to say that I feel that all these translations make a great deal of sense. Some of us have an easier time learning English than others do, the Chinese probably don't have an easy time at all, but a single common language is still the thing to aim for. The subset and semblance of English spoken on this list seems like something that should serve well as that common language, especially given the _help_ English language education gives in it. Also, I hear China is in fact fairly rapidly opening up to Western (American) popular culture which might be an argument if you have a generation or two to spare...
Apologies for ranting...
Cheers,
Rene.
Re: [PATCH] Chinese Language Maintainer
In reply to H. Peter AnvinWhat I would like to see is the Americans learning English as a FIRST language.
Keith (England)
----- Original Message -----
From: "H. Peter Anvin" <[email protected]>
To: "Li Yang" <[email protected]>
Cc: "Rob Landley" <[email protected]>; "Gerrit Huizenga" <[email protected]>; "Kunai, Takashi" <[email protected]>; <[email protected]>; "Andrew Morton" <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>; <[email protected]>
Sent: Sunday, July 15, 2007 7:12 PM
Subject: Re: [PATCH] Chinese Language Maintainer
Quoted 33 lines
Quoted 9 lines · Li Yang
I think you worried too much about this problem. :) Let me explain the situation here in China more clearly. Actually, English is mandatory in most schools and universities. Only very few people learn other language as a second language. Therefore software developers who are almost educated should have the basic English skill. However, that doesn't mean that they can read English or communicate with native English speaker very easily. Consider your second language learn in school for analogy.
Actually, I disagree. English *is* the second language learned in school for most European developers (except, obviously, the ones from the British isles), and we don't have that problem.
Quoted 6 lines
Read in English will be much slower and more likely to cause misunderstanding. This will
reduce the likelihood greatly of English documentation being read. If we are promoting contribution to the Linux community, we should maximum the possibility that these key documents being read. Translation will serve this purpose very well.What we have found in Europe, is that that it has limited value, and that the closer to the core you are, the less value it is, because at that stage you should be communicating more with other developers. Putting yourself behind a wall of translation is unfortunately a detriment in that way.
(quoted diff) +0
-2-hpa -To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Chinese Language Maintainer
In reply to Li YangQuoted 42 lines · leo
Quoted 24 lines · Rob Landley
Quoted 8 lines · Li Yang
Quoted 6 lines · Rob Landley
+A language maintainer accepts patches to the Linux kernel, written in C, from +authors who do not also speak English. The language maintainer translates the +description of each patch into English, forwards the patches to linux-kernel +and to the appropriate maintainers for inclusion in the Linux kernel, and +translates questions and replies about such patches as part of the +patch review process.
Quoted 5 lines
In addiction to this responsibility, I would like to add two more which, in my opinion, are more important. And these are what I'm trying to do. :)
First, promoting contribution to Linux kernel in local language. Second, coordinate the translation effort of key kernel documents.Cool. It's good to do that, but not the problem I'm worried about solving.
I was trying to describe the minimum requirements for being a language maintainer, I.E. what non-english users need in order to be able to merge their patches. Because without someone to contribute patches to (I.E a language maintainer), documentation in non-english languages promotes the creation of patches that can't be merged. That's the problem I'm trying to solve.
Quoted 2 lines
To me, finding language maintainers is the flip side of the coin of translating documentation.
I think you worried too much about this problem. :) Let me explain the situation here in China more clearly. Actually, English is mandatory in most schools and universities. Only very few people learn other language as a second language. Therefore software developers who are almost educated should have the basic English skill. However, that doesn't mean that they can read English or communicate with native English speaker very easily. Consider your second language learn in school for analogy. Read in English will be much slower and more likely to cause misunderstanding. This will reduce the likelihood greatly of English documentation being read. If we are promoting contribution to the Linux community, we should maximum the possibility that these key documents being read. Translation will serve this purpose very well.
Quoted 6 lines
So the possibility is very little that a translator is needed between the Linux maintainer and a Chinese developer. Although sometimes help is needed when there is misunderstanding.
After a brief talk with the Japanese translator, I think the case is similar for Japanese too.
Yes, In Japan, situation is mostly the same.
We are trying to increase number of Linux community developer with Linux Foundation Japan or CELF people in Japan.
In our discussion, the problem is not only Language.
In case of some developer, once he step forward (he try to send patch or comment on LKML), he got some comment and he can work with community even if it's slow (because of he was non-native). So, I thought if some key document are available in Japanese like HOWTO, that will help such early stage of developers.
Quoted 3 lines
Therefore, in my opinion, language maintainer should be more a helper and promoter rather than a gatekeeper. I will give a proposed process later about how this helper mechanism works.
I will be able to help this as a stand point of Japanese situation.
Quoted 8 lines
- Leo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Chinese Language Maintainer
Quoted 6 lines · Tsugikazu Shibata
Yes, In Japan, situation is mostly the same.
We are trying to increase number of Linux community developer with Linux Foundation Japan or CELF people in Japan.
In our discussion, the problem is not only Language.
Quoted 6 lines
In case of some developer, once he step forward (he try to send patch or comment on LKML), he got some comment and he can work with community even if it's slow (because of he was non-native). So, I thought if some key document are available in Japanese like HOWTO, that will help such early stage of developers.
I think you're absolutely right about that. The stuff that will help most of have translated is (kernel)newbie type documentation, the stuff one wants when being introduced to the community.
-hpa
Re: [PATCH] Chinese Language Maintainer
In reply to Tsugikazu ShibataQuoted 58 lines · Tsugikazu Shibata
Quoted 39 lines · leo
Quoted 22 lines · Rob Landley
Quoted 7 lines · Li Yang
Quoted 6 lines · Rob Landley
+A language maintainer accepts patches to the Linux kernel, written in C, from +authors who do not also speak English. The language maintainer translates the +description of each patch into English, forwards the patches to linux-kernel +and to the appropriate maintainers for inclusion in the Linux kernel, and +translates questions and replies about such patches as part of the +patch review process.
Quoted 5 lines
In addiction to this responsibility, I would like to add two more which, in my opinion, are more important. And these are what I'm trying to do. :)
First, promoting contribution to Linux kernel in local language. Second, coordinate the translation effort of key kernel documents.Cool. It's good to do that, but not the problem I'm worried about solving.
I was trying to describe the minimum requirements for being a language maintainer, I.E. what non-english users need in order to be able to merge their patches. Because without someone to contribute patches to (I.E a language maintainer), documentation in non-english languages promotes the creation of patches that can't be merged. That's the problem I'm trying to solve.
Quoted 2 lines
To me, finding language maintainers is the flip side of the coin of translating documentation.
I think you worried too much about this problem. :) Let me explain the situation here in China more clearly. Actually, English is mandatory in most schools and universities. Only very few people learn other language as a second language. Therefore software developers who are almost educated should have the basic English skill. However, that doesn't mean that they can read English or communicate with native English speaker very easily. Consider your second language learn in school for analogy. Read in English will be much slower and more likely to cause misunderstanding. This will reduce the likelihood greatly of English documentation being read. If we are promoting contribution to the Linux community, we should maximum the possibility that these key documents being read. Translation will serve this purpose very well.
Quoted 6 lines
So the possibility is very little that a translator is needed between the Linux maintainer and a Chinese developer. Although sometimes help is needed when there is misunderstanding.
After a brief talk with the Japanese translator, I think the case is similar for Japanese too.
Yes, In Japan, situation is mostly the same.
We are trying to increase number of Linux community developer with Linux Foundation Japan or CELF people in Japan.
In our discussion, the problem is not only Language.In case of some developer, once he step forward (he try to send patch or comment on LKML), he got some comment and he can work with community even if it's slow (because of he was non-native). So, I thought if some key document are available in Japanese like HOWTO, that will help such early stage of developers.
Quoted 5 lines
Quoted 3 lines
Therefore, in my opinion, language maintainer should be more a helper and promoter rather than a gatekeeper. I will give a proposed process later about how this helper mechanism works.
I will be able to help this as a stand point of Japanese situation.
Here is the helper process I propose to help more people to participate.
Suggestions and comments are welcomed.
1) Developer who can't speak English or has a problem going through the submission process sends patches to the Language maintainer and cc the local mail list provided.
2) Language maintainer and experienced Linux developer on the local mail list can review the patch and give comments in native language.
3) When the language maintainer thinks the patch is good in general, he will pass the patch onto corresponding subsystem maintainer and mail list for further review, cc the author and local list.
4) The author communicates directly with the community.
5) If the author has a problem understanding the comments or expressing himself in English, he can ask for help in native language on the local list. The language maintainer and the developers on the list are responsible to help out.
6) When the developer gets familiar with the submission process and can go through it with no problem, he can then submit patch directly without the helper process and probably join the mail list to help other new developer.
- Leo
Re: [PATCH] Chinese Language Maintainer
Quoted 12 lines · Li Yang
Here is the helper process I propose to help more people to participate.
Suggestions and comments are welcomed.1) Developer who can't speak English or has a problem going through the submission process sends patches to the Language maintainer and cc the local mail list provided.
2) Language maintainer and experienced Linux developer on the local mail list can review the patch and give comments in native language.
3) When the language maintainer thinks the patch is good in general, he will pass the patch onto corresponding subsystem maintainer and mail list for further review, cc the author and local list.
4) The author communicates directly with the community.
Quoted 8 lines
5) If the author has a problem understanding the comments or expressing himself in English, he can ask for help in native language on the local list. The language maintainer and the developers on the list are responsible to help out.
6) When the developer gets familiar with the submission process and can go through it with no problem, he can then submit patch directly without the helper process and probably join the mail list to help other new developer.
You know your area better than I do. If you think that will work, I certainly have no objections...
Quoted 1 line
- Leo
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
In reply to Li Yang-r58472Quoted 51 lines · LeoLi
Quoted 7 lines · Rob Landley
Quoted 3 lines · Li Yang
There are quite a lot kernel developers for each of the popular language, AFAIK. For non-popular languages, there shouldn't be translation available in the first place.
I don't distinguish between "popular" and "non-popular" languages. If somebody's willing to do the translation work, it's popular. If
nobody's
Quoted 1 line
willing to do the work, then even a language 1/3 of the planet's
population
Quoted 3 lines
speaks isn't "popular" for kernel development.
I wouldn't discourage a translator into Klingon if they were willing
to keep
Quoted 4 lines
their translation up to date and/or it actually resulted in patches.
Quoted 2 lines
It was really a surprise when
I post my Chinese translation on the LKML so many Chinese speakershave
Quoted 1 line
Quoted 1 line
commented on the translation and encouraged the translation work.
They
Quoted 1 line
Quoted 1 line
are not visible as they usually don't talk too much. :) So I set up
the
Quoted 1 line
Quoted 1 line
Chinese kernel development maillist([email protected]),
there
Quoted 5 lines
Quoted 3 lines
will be more and more experienced kernel developer who can read and update the Chinese documents after they read the translated documentation and become kernel hacker.
The chinese mailing list is highly cool, and my first instinct was to
note it
Quoted 1 line
on kernel.org/doc, but it would be better if the chinese website I
already
Quoted 1 line
link to notes it instead. (That way I don't have to worry about how
to
Quoted 1 line
spam-guard your email address. :)
The benefit of a localized maillist is that patches can be reviewed and commented in native language for common problems (like style, API).
Quoted 2 lines
This also highlights the need for language maintainers to help the
patches go
Quoted 2 lines
upstream to the english list. (If we can avoid armchair commentators saddling them with the task of translating the entire Documentation
directory
Quoted 1 line
and keeping such a translation up to date, we might actually get one
too.
Quoted 1 line
Fielding patches and questions sounds like plenty to me...)
Quoted 8 lines
I do think the documentation translation is very necessary even when there is a language maintainer, especially for the policy documents as HOWTO, codestyle , and etc. The contributors should go through these policies and check their code for compliance before going to the language maintainer for help, or there will be too much for the language maintainer to translate. The language maintainer doesn't need to translate all the documents himself, but he can help to coordinate the translation effort and help to make it update to date.
I would like to comment from Japanese situation.
I do also think the documentation translation is necessary even if there are language maintainer are not there. HOWTO like very fundamental documents will great help for early stage of developers. The contributors/early stage of developer will be able to follow coding style and so on if the documents are easy to read.
So, maintain this kind of document in local language are necessary even if there are no language maintainer.
I believe the language maintainer should mostly take care about the patch.
Quoted 11 lines
Quoted 1 line
Could you ask on said list if anyone is likely to volunteer for the
position?
Quoted 1 line
(JUST translating patches and questions, as part of pushing code
upstream.)
If we do need a contact person, I can do it. However I don't think there will be much translation work to do here. As I stated before, most Chinese programmers are more or less capable of read/write technical English. The difficult part is to let them know the benefit of merging code in kernel and teach them how to do it. That's why the policy documents in native language will be very useful.
I also think that most Japanese developers can read/write technical English at some level.
Quoted 85 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Merging into the kernel is a great way to keep CODE up to date.
Don't
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
think
it's magic pixie dust for documentation. It never has beenbefore.
Quoted 4 lines
Quoted 4 lines
IMHO, having contribution merged into the kernel has the MAGIC to attract people to work for recognition. When more and more people volunteer to work on it, the documentation will be up to date
magically.
Quoted 2 lines
Obvious counter-arguments include the floppy driver, the floppy tape
driver,
Quoted 1 line
the tty layer, and most of the existing english Documentation
directory...
Quoted 3 lines
I'll happily stay out of the way of people who actually want to merge translations of documentation into the kernel. I reserve the right to
say "I
Quoted 3 lines
told you so" in about five years.
Quoted 1 line
Quoted 1 line
Ah, but It's not a language maintainer's job to update
documentation.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
It's
their job to ACCEPT PATCHES. Translate patches, translatequestions
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
back and
forth. This has NOTHING to do with documentation unless they're converting
documentation accompanying the patch one way; into english.Language maintainers can integrate updates to the documentation just like integrating any updates to the code. Working on the
documentation
Quoted 1 line
Quoted 1 line
is ,IMHO, a perfect task for novice kernel hacker to get familiar
with
Quoted 3 lines
Quoted 1 line
the process.
From a language maintainer's perspective documentation patches
wouldn't be any
Quoted 1 line
different than any other patches. Translate the description and
questions
Quoted 1 line
going back and forth. The patch itself doesn't get translated when
it's C
Quoted 1 line
and applies to
scripts/kconfig/, why would it when it's UTF-8 andapplies to
Quoted 3 lines
Documentation/?Of course this brings up the question "what kind of function and
variable
Quoted 1 line
names do chinese people pick?" (I honestly don't know, but I note
that
Quoted 1 line
attempts to use names that don't fit in 7-bit ascii would probably be
frowned
Quoted 3 lines
upon in a big way...)
Quoted 1 line
It won't be too hard if the work is shared by a community. Like the
one
Quoted 3 lines
Quoted 1 line
I'm trying to establish.
A list works fine as a point of contact. I note that in general,
maintainers
Quoted 1 line
are individuals (who delegate like mad, of course), because otherwise
agenda
Quoted 1 line
items languish with everyone thinking it's someone else's
responsibility.
Got it. I can do it as long as it doesn't consume too much time. :)
Quoted 9 lines
http://www.trumanlibrary.org/buckstop.htm
Quoted 4 lines
- Leo
Advertisement time:
Chinese kernel development community http://zh-kernel.org :)Hmmm... Now I'm wondering if I should link directly to the docs pages
of the
Quoted 1 line
chinese and japanese sites, or to the top level community pages?
The documents and maillist are also linked in the front page. It's better to link the top level page.
- Leo
Re: Documentation of kernel messages (Summary)
In reply to Li Yang-r58472Quoted 10 lines · On Thursday 12 July 2007 9:53:54 am Li Yang-r58472 wrote:
Quoted 1 line
Fielding patches and questions sounds like plenty to me...)
I do think the documentation translation is very necessary even when there is a language maintainer, especially for the policy documents as HOWTO, codestyle , and etc. The contributors should go through these policies and check their code for compliance before going to the language maintainer for help, or there will be too much for the language maintainer to translate. The language maintainer doesn't need to translate all the documents himself, but he can help to coordinate the translation effort and help to make it update to date.
It would help if all the policy documents got grouped into a single Documentation/development directory so we could separate "policy documents in each language would be nice" from "that document about the amiga zorro bus really needs to be kept up-to-date in Navajo and that should be in the kernel tarball please".
Lemme see, which ones are we talking about? The candidates are:
applying-patches.txt
BUG-HUNTING
Changes
CodingStyle
debugging-modules.txt
feature-removal-schedule.txt
HOWTO
kernel-docs.txt
language-maintainers.txt
ManagementStyle
oops-tracing.txt
SecurityBugs
sparse.txt
stable_api_nonsense.txt
stable_kernel_rules.txt
SubmitChecklist
SubmittingDrivers
SubmittingPatches
volatile-considered-harmful.txt
That's everything I noticed in the top level directory that's a good candidate to be grouped into a "development" subdirectory. Did I miss anything?
I note that Changes is a bit stale in places (16 bit assembly?), feature-removal-schedule.txt changes often but is good to know, kernel-docs.txt might be useless to translate considering it's mostly links to english documentation, language-maintainers.txt is assuming my patch from earlier today gets accepted...
I can submit a patch grouping all that stuff together into a subdirectory if it would help...
Quoted 6 lines
If we do need a contact person, I can do it. However I don't think there will be much translation work to do here. As I stated before, most Chinese programmers are more or less capable of read/write technical English. The difficult part is to let them know the benefit of merging code in kernel and teach them how to do it. That's why the policy documents in native language will be very useful.
Does the above look like a good list? There are more that need to be written, but that's what I saw in Documentation...
Rob
P.S. Dear kmail/postfix developers: having a transient DNS lookup failure on one address in a long cc: list is _NOT_ a reason to have the message stay in the kmail outbox. It should go into the postfix send queue and be retried from there. Right now, if I tell it to resend, how do I know who it has and hasn't been successfully distributed to on the first attempt? I've gotten dinged for trimming the cc: list before, but now I'm about to send out duplicates. Wheee...
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 18 lines · rob
Quoted 10 lines · On Thursday 12 July 2007 9:53:54 am Li Yang-r58472 wrote:
Quoted 1 line
Fielding patches and questions sounds like plenty to me...)
I do think the documentation translation is very necessary even when there is a language maintainer, especially for the policy documents as HOWTO, codestyle , and etc. The contributors should go through these policies and check their code for compliance before going to the language maintainer for help, or there will be too much for the language maintainer to translate. The language maintainer doesn't need to translate all the documents himself, but he can help to coordinate the translation effort and help to make it update to date.
It would help if all the policy documents got grouped into a single
Documentation/developmentdirectory so we could separate "policy documents in each language would be nice" from "that document about the amiga zorro bus really needs to be kept up-to-date in Navajo and that should be in the kernel tarball please".
Quoted 6 lines
Lemme see, which ones are we talking about? The candidates are:
applying-patches.txt
BUG-HUNTING
Changes
CodingStyle
debugging-modules.txt
This file seems mostly technical. may not policy related document.
Quoted 6 lines
feature-removal-schedule.txt
HOWTO
kernel-docs.txt
language-maintainers.txt
ManagementStyle
oops-tracing.txt
Quoted 8 lines
SecurityBugs
sparse.txt
stable_api_nonsense.txt
stable_kernel_rules.txt
SubmitChecklist
SubmittingDrivers
SubmittingPatches
volatile-considered-harmful.txt
How about adding;
kernel-doc-nano-HOWTO.txt
These three slightly include some policy in the documents but purpose are mostly technical. (So, it's just comment)
devices.txt
kernel-parameters.txt
unicode.txt
Quoted 31 lines
That's everything I noticed in the top level directory that's a good candidate to be grouped into a "development" subdirectory. Did I miss anything?
I note that Changes is a bit stale in places (16 bit assembly?), feature-removal-schedule.txt changes often but is good to know, kernel-docs.txt might be useless to translate considering it's mostly links to english documentation, language-maintainers.txt is assuming my patch from earlier today gets accepted...
I can submit a patch grouping all that stuff together into a subdirectory if it would help...
Quoted 6 lines
If we do need a contact person, I can do it. However I don't think there will be much translation work to do here. As I stated before, most Chinese programmers are more or less capable of read/write technical English. The difficult part is to let them know the benefit of merging code in kernel and teach them how to do it. That's why the policy documents in native language will be very useful.
Does the above look like a good list? There are more that need to be written, but that's what I saw in Documentation...
Rob
P.S. Dear kmail/postfix developers: having a transient DNS lookup failure on one address in a long cc: list is _NOT_ a reason to have the message stay in the kmail outbox. It should go into the postfix send queue and be retried from there. Right now, if I tell it to resend, how do I know who it has and hasn't been successfully distributed to on the first attempt? I've gotten dinged for trimming the cc: list before, but now I'm about to send out duplicates. Wheee...
Re: Documentation of kernel messages (Summary)
Quoted 19 lines · Tsugikazu Shibata
Quoted 18 lines · rob
Quoted 10 lines · On Thursday 12 July 2007 9:53:54 am Li Yang-r58472 wrote:
Quoted 1 line
Fielding patches and questions sounds like plenty to me...)
I do think the documentation translation is very necessary even when there is a language maintainer, especially for the policy documents as HOWTO, codestyle , and etc. The contributors should go through these policies and check their code for compliance before going to the language maintainer for help, or there will be too much for the language maintainer to translate. The language maintainer doesn't need to translate all the documents himself, but he can help to coordinate the translation effort and help to make it update to date.
It would help if all the policy documents got grouped into a single
Documentation/developmentdirectory so we could separate "policy documents in each language would be nice" from "that document about the amiga zorro bus really needs to be kept up-to-date in Navajo and that should be in the kernel tarball please".
Quoted 8 lines
Quoted 6 lines
Lemme see, which ones are we talking about? The candidates are:
applying-patches.txt
BUG-HUNTING
Changes
CodingStyle
debugging-modules.txtThis file seems mostly technical. may not policy related document.
Yeah, I guess so. My mental filter was actually more like "how to do linux development", and it seemed both short and easy to translate, and also generally relevant to the majority of developers no matter what subsystem they're working on.
But I agree, it's not policy.
Quoted 8 lines
Quoted 8 lines
feature-removal-schedule.txt
HOWTO
kernel-docs.txt
language-maintainers.txt
ManagementStyle
oops-tracing.txt
SecurityBugs
sparse.txt
That one's not policy either, but it is general non-subsystem-specific development. Probably "applying-patches.txt" above goes in the same bucket.
Whether or not to include that bucket in the new directory depends on whether the directory is labeled "development" or "policy" or something else...
Quoted 9 lines
Quoted 6 lines
stable_api_nonsense.txt
stable_kernel_rules.txt
SubmitChecklist
SubmittingDrivers
SubmittingPatches
volatile-considered-harmful.txtHow about adding;
kernel-doc-nano-HOWTO.txt
The problem is, the generated htmdocs are in english. This file is about how to generate (and author) English documentation that won't be translated. What's the point of translating the instructions if the result won't be translated?
On the other hand, if the authors of patches _do_ put javadoc comments into the source code, the language maintainer should be aware of them and should have some easy way to translate them for the list...
Quoted 5 lines
These three slightly include some policy in the documents but purpose are mostly technical. (So, it's just comment)
devices.txt
kernel-parameters.txt
unicode.txt
Extracting the policy out into separate documents I could see, but I agree those are primarily technical.
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 45 lines · rob
Quoted 19 lines · Tsugikazu Shibata
Quoted 18 lines · rob
Quoted 10 lines · On Thursday 12 July 2007 9:53:54 am Li Yang-r58472 wrote:
Quoted 1 line
Fielding patches and questions sounds like plenty to me...)
I do think the documentation translation is very necessary even when there is a language maintainer, especially for the policy documents as HOWTO, codestyle , and etc. The contributors should go through these policies and check their code for compliance before going to the language maintainer for help, or there will be too much for the language maintainer to translate. The language maintainer doesn't need to translate all the documents himself, but he can help to coordinate the translation effort and help to make it update to date.
It would help if all the policy documents got grouped into a single
Documentation/developmentdirectory so we could separate "policy documents in each language would be nice" from "that document about the amiga zorro bus really needs to be kept up-to-date in Navajo and that should be in the kernel tarball please".Quoted 8 lines
Quoted 6 lines
Lemme see, which ones are we talking about? The candidates are:
applying-patches.txt
BUG-HUNTING
Changes
CodingStyle
debugging-modules.txtThis file seems mostly technical. may not policy related document.
Yeah, I guess so. My mental filter was actually more like "how to do linux development", and it seemed both short and easy to translate, and also generally relevant to the majority of developers no matter what subsystem they're working on.
But I agree, it's not policy.
Quoted 8 lines
Quoted 8 lines
feature-removal-schedule.txt
HOWTO
kernel-docs.txt
language-maintainers.txt
ManagementStyle
oops-tracing.txt
SecurityBugs
sparse.txt
Quoted 5 lines
That one's not policy either, but it is general non-subsystem-specific development. Probably "applying-patches.txt" above goes in the same bucket.
Whether or not to include that bucket in the new directory depends on whether the directory is labeled "development" or "policy" or something else...
Agree.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
stable_api_nonsense.txt
stable_kernel_rules.txt
SubmitChecklist
SubmittingDrivers
SubmittingPatches
volatile-considered-harmful.txt
Quoted 7 lines
Quoted 2 lines
How about adding;
kernel-doc-nano-HOWTO.txtThe problem is, the generated htmdocs are in english. This file is about how to generate (and author) English documentation that won't be translated. What's the point of translating the instructions if the result won't be translated?
People (who even in non-native) would better to know and read it such htmldocs because there are good and important documents even in English. I thought that translation of this file may help.
Quoted 3 lines
On the other hand, if the authors of patches _do_ put javadoc comments into the source code, the language maintainer should be aware of them and should have some easy way to translate them for the list...
maybe.
Quoted 9 lines
Quoted 6 lines
These three slightly include some policy in the documents but purpose are mostly technical. (So, it's just comment)
devices.txt
kernel-parameters.txt
unicode.txtExtracting the policy out into separate documents I could see, but I agree those are primarily technical.
Further comment on this is that we may need to have another 00-INDEX file, for this new directory.
Quoted 6 lines
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 11 lines · Tsugikazu Shibata
Quoted 7 lines
Quoted 2 lines
How about adding;
kernel-doc-nano-HOWTO.txtThe problem is, the generated htmdocs are in english. This file is about how to generate (and author) English documentation that won't be translated. What's the point of translating the instructions if the result won't be translated?
People (who even in non-native) would better to know and read it such htmldocs because there are good and important documents even in English. I thought that translation of this file may help.
The english version of htmldocs is generated from the source code. If
translating remotely current versions of htmldocs into other languages was feasible, then the english version wouldn't need to be in the source code in the first place.
Translating a document _about_ htmldocs boils down to "Dear non-english
speaker: here's something you can't read, nor can you update it either except though your language maintainer."
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 13 lines · rob
Quoted 11 lines · Tsugikazu Shibata
Quoted 7 lines
Quoted 2 lines
How about adding;
kernel-doc-nano-HOWTO.txtThe problem is, the generated htmdocs are in english. This file is about how to generate (and author) English documentation that won't be translated. What's the point of translating the instructions if the result won't be translated?
People (who even in non-native) would better to know and read it such htmldocs because there are good and important documents even in English. I thought that translation of this file may help.
Quoted 8 lines
The english version of htmldocs is generated from the source code. If
translating remotely current versions of htmldocs into other languages was feasible, then the english version wouldn't need to be in the source code in the first place.Translating a document _about_ htmldocs boils down to "Dear non-english
speaker: here's something you can't read, nor can you update it either except though your language maintainer."
No, kernel-doc-nano-HOWTO includes explanation of tools to extract the document, format of extractable document in source files and so on. I thought this file would help non-native people, how to extract or add in-kernel extractable documents.
I understand that these tools will generate English documents but that's OK. Such large volume of documents are not easy to translate into other languages.
Quoted 6 lines
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 19 lines · Tsugikazu Shibata
Quoted 18 lines · rob
Quoted 11 lines · Tsugikazu Shibata
Quoted 7 lines
Quoted 2 lines
How about adding;
kernel-doc-nano-HOWTO.txtThe problem is, the generated htmdocs are in english. This file is about how to generate (and author) English documentation that won't be translated. What's the point of translating the instructions if the result won't be translated?
People (who even in non-native) would better to know and read it such htmldocs because there are good and important documents even in English. I thought that translation of this file may help.
The english version of htmldocs is generated from the source code. If
translating remotely current versions of htmldocs into other languages was feasible, then the english version wouldn't need to be in the source code in the first place.
Quoted 8 lines
Quoted 3 lines
Translating a document _about_ htmldocs boils down to "Dear non-english
speaker: here's something you can't read, nor can you update it either except though your language maintainer."No, kernel-doc-nano-HOWTO includes explanation of tools to extract the document, format of extractable document in source files and so on. I thought this file would help non-native people, how to extract or add in-kernel extractable documents.
Which will be in english, so why can't the instructions on how to do it be in english too?
Quoted 2 lines
I understand that these tools will generate English documents but that's OK.
Then the instructions how to do it in english should be ok too. If they can't read the instructions, they won't be able to read the result.
Quoted 2 lines
Such large volume of documents are not easy to translate into other languages.
"Here's how to get a result that will be useless to you."
Why? What's the point? I'm not seeing it.
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 29 lines · rob
Quoted 19 lines · Tsugikazu Shibata
Quoted 18 lines · rob
Quoted 11 lines · Tsugikazu Shibata
Quoted 7 lines
Quoted 2 lines
How about adding;
kernel-doc-nano-HOWTO.txtThe problem is, the generated htmdocs are in english. This file is about how to generate (and author) English documentation that won't be translated. What's the point of translating the instructions if the result won't be translated?
People (who even in non-native) would better to know and read it such htmldocs because there are good and important documents even in English. I thought that translation of this file may help.
The english version of htmldocs is generated from the source code. If
translating remotely current versions of htmldocs into other languages was feasible, then the english version wouldn't need to be in the source code in the first place.Quoted 8 lines
Quoted 3 lines
Translating a document _about_ htmldocs boils down to "Dear non-english
speaker: here's something you can't read, nor can you update it either except though your language maintainer."No, kernel-doc-nano-HOWTO includes explanation of tools to extract the document, format of extractable document in source files and so on. I thought this file would help non-native people, how to extract or add in-kernel extractable documents.
Quoted 8 lines
Which will be in english, so why can't the instructions on how to do it be in english too?
Quoted 2 lines
I understand that these tools will generate English documents but that's OK.
Then the instructions how to do it in english should be ok too. If they can't read the instructions, they won't be able to read the result.
It seems my explanation was not enough.
We (like me, usually not using English) can read English but it takes more time than you (and any natives). So, reading large document takes huge amount of time. In case of htmldocs, it is actually large and developer may give up before start.
I thought that if some part (like how to do it) are translated into his/her familiar language, he/she may not give it up.
Yes, it may not be.
Another viewpoint is that if he/she need to modify some interface and it may need to change some part of extractable document, this file includes, "How to add extractable documentation to your source files". It will be some help as well.
Yes, He/she should write English and it may take long time but hopefully the translation result would push his/her back.
Quoted 12 lines
Quoted 2 lines
Such large volume of documents are not easy to translate into other languages.
"Here's how to get a result that will be useless to you."
Why? What's the point? I'm not seeing it.
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
In reply to Rob LandleyQuoted 58 lines · Rob Landley
Quoted 10 lines · On Thursday 12 July 2007 9:53:54 am Li Yang-r58472 wrote:
Quoted 1 line
Fielding patches and questions sounds like plenty to me...)
I do think the documentation translation is very necessary even when there is a language maintainer, especially for the policy documents as HOWTO, codestyle , and etc. The contributors should go through these policies and check their code for compliance before going to the language maintainer for help, or there will be too much for the language maintainer to translate. The language maintainer doesn't need to translate all the documents himself, but he can help to coordinate the translation effort and help to make it update to date.
It would help if all the policy documents got grouped into a single
Documentation/developmentdirectory so we could separate "policy documents in each language would be nice" from "that document about the amiga zorro bus really needs to be kept up-to-date in Navajo and that should be in the kernel tarball please".Lemme see, which ones are we talking about? The candidates are:
applying-patches.txt
BUG-HUNTING
Changes
CodingStyle
debugging-modules.txt
feature-removal-schedule.txt
HOWTO
kernel-docs.txt
language-maintainers.txt
ManagementStyle
oops-tracing.txt
SecurityBugs
sparse.txt
stable_api_nonsense.txt
stable_kernel_rules.txt
SubmitChecklist
SubmittingDrivers
SubmittingPatches
volatile-considered-harmful.txtThat's everything I noticed in the top level directory that's a good candidate to be grouped into a "development" subdirectory. Did I miss anything?
I note that Changes is a bit stale in places (16 bit assembly?), feature-removal-schedule.txt changes often but is good to know, kernel-docs.txt might be useless to translate considering it's mostly links to english documentation, language-maintainers.txt is assuming my patch from earlier today gets accepted...
I can submit a patch grouping all that stuff together into a subdirectory if it would help...
Quoted 6 lines
If we do need a contact person, I can do it. However I don't think there will be much translation work to do here. As I stated before, most Chinese programmers are more or less capable of read/write technical English. The difficult part is to let them know the benefit of merging code in kernel and teach them how to do it. That's why the policy documents in native language will be very useful.
Quoted 2 lines
Does the above look like a good list? There are more that need to be written, but that's what I saw in Documentation...
Yes, I do think this is a good list for translation.
Greg,
Do you have anything to add?
- Leo
Re: Documentation of kernel messages (Summary)
Quoted 4 lines · Li Yang
Yes, I do think this is a good list for translation.
Greg,
Do you have anything to add?
There are a few documents not in the kernel tree that you may want to translate too, if your goal is to increase the number of kernel developers:
http://kernelnewbies.org/KernelGlossary
http://kernelnewbies.org/UpstreamMerge
Signature · 3 lines
Politics is the struggle between those who want to make their country the best in the world, and those who believe it already is. Each group calls the other unpatriotic.
Re: Documentation of kernel messages (Summary)
In reply to Rob LandleyQuoted 10 lines · Rob Landley
...
[regarding translated documentation]For my part, I can't _tell_ when a given translation is out of date because I can't read it, and I certainly can't update it. So I agree with Eric and I'm linking to sites hosting kernel documentation in other languages rather than hosting them myself. I've got a good link to a Japanese site, need to ping the contributors of the chinese documentation and see if they have a site for it...
...
Include the last git commit on this file in the translated document?
This would also make it easier for a translator to update a document because a diff on the original document since the last translation would be trivial.
Quoted 1 line
Rob
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: Documentation of kernel messages (Summary)
Quoted 12 lines · On Monday 09 July 2007 14:33:09 Adrian Bunk wrote:
Quoted 10 lines · Rob Landley
...
[regarding translated documentation]For my part, I can't _tell_ when a given translation is out of date because I can't read it, and I certainly can't update it. So I agree with Eric and I'm linking to sites hosting kernel documentation in other languages rather than hosting them myself. I've got a good link to a Japanese site, need to ping the contributors of the chinese documentation and see if they have a site for it...
...
Quoted 5 lines
Include the last git commit on this file in the translated document?
This would also make it easier for a translator to update a document because a diff on the original document since the last translation would be trivial.
Ok, back up.
Let's look at _english_ documentation.
If you go to http://kernel.org/doc/ols you should find, nicely split up, all the OLS papers from 2002-2007. By my count, there are 337 of them, totaling 61.8 megabytes when they're split up like that. Being PDFs, they don't compress all that well.
Which subset of these should be copied into the Documentation directory? How do you update them? This is the documentation from just ONE SOURCE.
And PDF isn't close to the worst of it: Linuxconf.au has lots of video, as does google video. (I don't even have _english_ transcripts for those videos, which would be nice.) Do you want to merge all of this into the kernel tarball as well?
How does this help keep anything up to date?
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
Quoted 9 lines · Rob Landley
...
Let's look at _english_ documentation.If you go to http://kernel.org/doc/ols you should find, nicely split up, all the OLS papers from 2002-2007. By my count, there are 337 of them, totaling 61.8 megabytes when they're split up like that. Being PDFs, they don't compress all that well.
Which subset of these should be copied into the Documentation directory?
You don't have to copy everything.
And you anyway have to handle cases like e.g. most books where you are not permitted to copy them, but might want to point people to them.
Quoted 2 lines
How
do you update them? This is the documentation from just ONE SOURCE.
You aren't legally permitted to update any OLS paper unless you've gotten explicit permission by the author. And when he gives legal permission, he can as well give you the LaTeX source. I'd expect legal and technical update possibilities to usually be related this way.
And you shouldn't include non-free documentation into the kernel sources.
Quoted 4 lines
And PDF isn't close to the worst of it: Linuxconf.au has lots of video, as does google video. (I don't even have _english_ transcripts for those videos, which would be nice.) Do you want to merge all of this into the kernel tarball as well?
Not at all.
Quoted 1 line
How does this help keep anything up to date?
You can only update things you are legally and technically able to update. For the rest, you can only ship pointers like kernel-docs.txt.
Quoted 1 line
Rob
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: Documentation of kernel messages (Summary)
Quoted 9 lines · On Tuesday 10 July 2007 14:54:15 Adrian Bunk wrote:
Quoted 8 lines · Rob Landley
...
Let's look at _english_ documentation.If you go to http://kernel.org/doc/ols you should find, nicely split up, all the OLS papers from 2002-2007. By my count, there are 337 of them, totaling 61.8 megabytes when they're split up like that. Being PDFs, they don't compress all that well.
Quoted 6 lines
Quoted 1 line
Which subset of these should be copied into the Documentation directory?
You don't have to copy everything.
And you anyway have to handle cases like e.g. most books where you are not permitted to copy them, but might want to point people to them.
This is exactly what I'm saying. I don't want to copy the translated documentation, I want to point people to it. Out on the web. Where it's maintained by the people who translated it, which history says they're noticeably less likely to do with a copy.
Quoted 5 lines
Quoted 2 lines
How
do you update them? This is the documentation from just ONE SOURCE.You aren't legally permitted to update any OLS paper unless you've gotten explicit permission by the author.
I wasn't planning to. I'm pointing out that it's hard.
Quoted 1 line
And you shouldn't include non-free documentation into the kernel sources.
A) http://www.linuxsymposium.org/2007/cfp.php under "Publication Rights":
Further, as stated in the official templates, and on the Credits page from the Proceedings of this year and prior years: "Authors retain copyright to all submitted papers, but have granted unlimited redistribution rights to all as a condition of submission."
So it's free as in speech but not beer.
B) I'm using these as examples of things to NOT merge into the kernel sources. Thank you for making my point. There's a LOT of this stuff out there.
Quoted 4 lines
Quoted 1 line
How does this help keep anything up to date?
You can only update things you are legally and technically able to update. For the rest, you can only ship pointers like kernel-docs.txt.
As an english speaker, I am not technically able to update non-english documentation. Since the only common language required of linux kernel developers has ever been English, only the english documentation makes sense to merge into the kernel tarball. Merging anything else guarantees that the vast majority of the contributors will never be able to review or update it, so merging it into the kernel tarball does not result in extra review or maintenance.
Extrapolating the review benefits of C code that breaks the build for everybody if you do it wrong to chinese documentation you can't even _read_... Apples and oranges, it's a false comparison. I don't see how merging translations of documentation is ever likely to be a win.
Quoted 4 lines
Quoted 1 line
Rob
cu
Adrian
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
In reply to Rob LandleyQuoted 2 lines · Rob Landley
If you go to http://kernel.org/doc/ols you should find, nicely split up, all the OLS papers from 2002-2007.
Oooh! That's nice! I didn't notice the "nicely split up" part earlier. Any chance we can get the original docbook inputs that OLS uses for paper submissions? Have you asked Andrew or Craig about this?
Also, it would be nice to correlate the talk names with the individual PDFs. Do you want me to solicit inside CELF for a volunteer for this? We'd probably produce a wiki page of links, based on your list.txt file and the proceedings index.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America
=============================
Re: Documentation of kernel messages (Summary)
Quoted 5 lines · Tim Bird
Oooh! That's nice! I didn't notice the "nicely split up" part earlier. Any chance we can get the original docbook inputs that OLS uses for paper submissions? Have you asked Andrew or Craig about this?
OLS uses LaTeX, not DocBook, for submissions AFAIK.
-hpa
Re: Documentation of kernel messages (Summary)
Quoted 6 lines · H. Peter Anvin
Quoted 3 lines · Tim Bird
Oooh! That's nice! I didn't notice the "nicely split up" part earlier. Any chance we can get the original docbook inputs that OLS uses for paper submissions? Have you asked Andrew or Craig about this?
OLS uses LaTeX, not DocBook, for submissions AFAIK.
Yeah, but their retention of the original latex has been... Iffy. And their indexing was a bit rushed. http://www.linuxsymposium.org/proceedings.php
goes up to 2005. If you go to
http://www.linuxsymposium.org/2006/proceedings.php you'll see a mirror of the other page with 2006 added (why not update the top level one? No idea.) Then 2007 is off on a Red Hat site (mis-numbered) and not even hosted on linuxsymposium.org.
If you look at http://kernel.org/doc/ols/2006/slides/ you'll notice I mirrored all the presentation slides (not the same as the papers). Two of them had already gone down before I got there, but I managed to get a fresh copy of Steven Hill's uClibc NPTL slides when I bumped into him at OLS. (My mirror has it, the original site it links to is 404.) I still haven't tracked down Kristen Accardi's slides, and that's one that was theoretically mirrored on the OLS website itself, yet the link is 404...
I'm not complaining, by the way: busy people get the data out and then move on, it's up to people like me to go back and clean it up if you want any kind of data retention. (And THEN you have to periodically figure out what's still relevant and what's only of historical interest. Wheee...)
Right now, I'd be happy to just get the PDFs of 2001 and earlier. There's a apparently one year they only have on paper, and one of the others is on a CD that Andrew thinks he knows where it is, but hadn't had time to dig up when last I poked him...
If latex shows up, I'm all for it. But for now, I'm happy with the PDFs. (Hey Tim, you want something for a volunteer to do? Index the 2006 papers, then match up the 2006 slides with the corresponding papers. I can go through and add in summaries afterwards... And note that the index.html should have _relative_ links, not absolute ones, because I want anyone to be able to tarball up the whole site and locally mirror it. I need to do a big script that fetches the external things like the ols papers and breaks them up so you have reproducible local copies without blindly copying mine. It's on my todo list...)
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
Re: Documentation of kernel messages (Summary)
In reply to Tim BirdQuoted 7 lines · Tim Bird
Quoted 2 lines · Rob Landley
If you go to http://kernel.org/doc/ols you should find, nicely split up, all the OLS papers from 2002-2007.
Oooh! That's nice! I didn't notice the "nicely split up" part earlier. Any chance we can get the original docbook inputs that OLS uses for paper submissions? Have you asked Andrew or Craig about this?
I've asked, but OLS was in the process of happening and they were kind of swamped...
Quoted 2 lines
Also, it would be nice to correlate the talk names with the individual PDFs.
I'm working on that: http://kernel.org/doc/ols/2002
I keep getting distracted by new material coming in. Need to shut it out for a bit and focus on processing the existing pile...
Quoted 1 line
Do you want me to solicit inside CELF for a volunteer for this?
Yes please. I'm happy to have volunteers help out with any of this, if I can figure out how to sanely partition it.
I'm not just collating names with papers, I'm also reading the paper and trying to come up with a one page summary. If you look at
http://kernel.org/doc you'll see a (now stale) version of an index skeleton. I'd like to link to all these papers from one big index. And also link the Documentation files into the same index. And http://lwn.net/Kernel/Index/
and Linux Journal's archives and...
That's the hard part. Indexing it. Piling up a big heap is easy.
You'll notice that Documentation has its own 00-index file, which doesn't even refer to the make htmldocs output. The lwn kernel material has an excellent index. The Linux Journal articles have a chronological archive. Every volume of OLS papers starts with a brief index.
And the problem is, every time somebody notices the problem they start a _new_ index that doesn't use any of the others. (And they keep wanting to do it as a wiki, which dooms the project. In my experience wikis aren't stable and only locally versioned. They're not designed to be snapshotted as a coherent whole, which is the _point_ of an index. If you deep-link into a wiki you get 404 errors after a while. They're a good tool for the "piling up information" part, but a bad tool for editorial jobs like organizing and indexing existing information. Wikis are designed to be decentralized, so the left hand doesn't have to know what the right hand is doing, but editorial functions is all about collating, coordinating, and correlating into a coherent whole. Which is nontrivial.)
Quoted 2 lines
We'd probably produce a wiki page of links, based on your list.txt file and the proceedings index.
I have a list.txt file? (Rummages.) Oh, that's actually extracted from Red Hat's 2007 OLS web page (see the link at top, "mirrored from here".)
I was using Red hat's broken-up pages (and pre-broken-up ones I found for 2004) for a while, but I went back and I re-broke up the pages with my scripts (which you'll notice I link to; I want people other than me to be able to reproduce this.) I did it because all the others I broke up have the two OLS boilerplate pages at the _end_ rather than the beginning, and I wanted to be consistent.
And that list.txt page doesn't have the paragraph per article summary (which I can sometimes take from the abstract, but even when there is an abstract it's often not what I need and I read the darn paper anyway to see what interesting stuff's buried in it). I need that in order to index the articles, title isn't enough. Some articles need to get linked from more than one place. (James Bottomley's 2002 scsi paper has good "history of hotplug" material, for example.)
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
RE: Documentation of kernel messages (Summary)
In reply to Rob LandleyQuoted 8 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Rob Landley
Sent: Tuesday, July 10, 2007 12:48 AM
To: H. Peter Anvin
Cc: Kunai, Takashi; [email protected]; Andrew Morton; [email protected]; [email protected]; [email protected]; [email protected]; [email protected];
Quoted 1 line
Quoted 6 lines
[email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: Documentation of kernel messages (Summary)
Quoted 6 lines
In regard to translating kernel messages:
Quoted 3 lines · On Monday 09 July 2007 01:36:31 H. Peter Anvin wrote:
Quoted 2 lines · Takashi
(1) Your kernel development proposal will be greatly supported by Japanese vendor community. At the same time, it needs support from
the
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
kernel communities, as well.
There is a very strong reason for the kernel community to NOT
support
Quoted 6 lines
Quoted 1 line
this: it makes it much harder to deal with bug reports.
Agreed.
There's a proposal in play for translations of documentation, and to recruit "language maintainers" (which is an idea I see that Matt
Mackall came
Quoted 3 lines
up with before I did).
A language maintainer is somebody who can accept a patch in language
X,
Quoted 1 line
translate its description to post on the appropriate english-only list
with a
Quoted 1 line
cc to the appropriate maintainer(s), and translate questions and
reviews
Quoted 1 line
back and forth for the original author until the patch can be merged.
(This
Quoted 1 line
means that documentation translations don't result in a stream of
unmergeable
Quoted 3 lines
patches.)
As for the documentation translations themselves, I note that Eric
Raymond
Quoted 1 line
dissuaded me from actually hosting translations of kernel
documentation on
Quoted 1 line
http://kernel.org/doc due to his experience with translations of his
own
Quoted 3 lines
writings. If he hosts the translations on his website, they never get updated again. But if he makes the contributor host them on their own website, then they sometimes get updated.
If the documents are already maintained together on one site, it won't be very hard to synchronize with kernel.org. Eric is right for the case that separate documents on separate sites.
Quoted 2 lines
For my part, I can't _tell_ when a given translation is out of date
because I
Quoted 1 line
can't read it, and I certainly can't update it. So I agree with Eric
and I'm
Quoted 1 line
linking to sites hosting kernel documentation in other languages
rather than
Quoted 1 line
hosting them myself. I've got a good link to a Japanese site, need to
ping
Quoted 1 line
the contributors of the chinese documentation and see if they have a
site for
Quoted 1 line
it...
Please add a link to http://zh-kernel.org/docs for Chinese translated kernel documentation.
- Leo
Re: Documentation of kernel messages (Summary)
In reply to H. Peter AnvinHi,
Quoted 8 lines · H. Peter Anvin
Quoted 3 lines · Takashi
(1) Your kernel development proposal will be greatly supported by Japanese vendor community. At the same time, it needs support from the kernel communities, as well.
There is a very strong reason for the kernel community to NOT support
this: it makes it much harder to deal with bug reports.
Quoted 8 lines
Like it or not, English is a lingua franca[1] of the technology industry. When I deal with the rest of the kernel community, I use English, instead of my native language (Swedish).
For things that are targetted toward end users, that's a different matter, of course, but kernel messages are inherently directed toward developers and technically sophisticated users.
I strongly agree with you.
Please keep kernel clean.
Re: Documentation of kernel messages (Summary)
In reply to Kunai, TakashiAm Montag, 9. Juli 2007 schrieb Kunai, Takashi:
Quoted 3 lines
(b)printk hashes and (c)Format strings. (a) seems difficult to get supports from kernel developers and (c) lacks uniqueness of each message. Though (b) also lacks uniqueness, adding component id and/or
We have generators for hash functions that guarantee unique results for a set of inputs. They are even GPLed. As you are operating against a known target, that should work.
Regards
Oliver
Re: Documentation of kernel messages (Summary)
Quoted 7 lines · Oliver Neukum
Am Montag, 9. Juli 2007 schrieb Kunai, Takashi:
Quoted 3 lines
(b)printk hashes and (c)Format strings. (a) seems difficult to get supports from kernel developers and (c) lacks uniqueness of each message. Though (b) also lacks uniqueness, adding component id and/or
We have generators for hash functions that guarantee unique results for a set of inputs.
Right, perfect hash generators require the input set to be known *a priori*.
Quoted 2 lines
They are even GPLed. As you are operating against
a known target, that should work.
But, I'm not sure they'd be operating against a known target -- I don't really know what exactly would be hashed, but if it's kernel printk() messages (the format string, obviously), then please remember that new messages would get added all the time, and unless we're also willing to change the hash function every time a printk() gets added to the kernel (whew!) it's going to be a problem -- especially because we don't really want to generate new hash functions for every kernel version / or for every kernel build, because we'd obviously like the same error message to hash to the same value across versions.
If we really care about uniqueness, the only solution I see is to use a strong/cryptographic hash function (say SHA-1, which would never change in the future) that generates the hashes for all the printk() messages that are getting compiled-in at build-time ... but it would be slow, of course.
Satyam
Re: Documentation of kernel messages (Summary)
Am Dienstag, 10. Juli 2007 schrieb Satyam Sharma:
Quoted 9 lines
But, I'm not sure they'd be operating against a known target -- I don't really know what exactly would be hashed, but if it's kernel
printk()messages (the format string, obviously), then please remember that new messages would get added all the time, and unless we're also willing to change the hash function every time aprintk()gets added to the kernel (whew!) it's going to be a problem -- especially because we don't really want to generate new hash functions for every kernel version / or for every kernel build, because we'd obviously like the same error message to hash to the same value across versions.
I suggest exactly that you build a new hash function for every build. Nothing but your translating demon need ever see it. We build a new System.map every time and it doesn't hurt.
Regards
Oliver
Re: [Lf_kernel_messages] Re: Documentation of kernel messages (Summary)
In reply to Rob LandleyQuoted 5 lines · Rob Landley
Personally? No to the second question, which renders the first "do it yourself outside of the tree".
Just a guess, and I don't speak for anyone else here, but I think most of us are waiting to see how long it takes you to lose interest.
Actually, at a brainstorming session at the recent Linux Foundation collaboration summit, there were a number of kernel developers, including James Bottomley, Greg K-H, Cristoph Hellwig (I think), and others, who were actually generally symapthetic to the idea, if done right.
The idea that was tossed about was essentially printk hashes (hashed on message plus filename, with optional reporting of filename+line number), with the messaging catalog being maintained externally. Yes, that will be a major pain for whoever wants to do the translation --- but it puts the onus on the people who would be getting the value for this to do the work.
The other idea that was tossed about was that for subsystems that are using structured reporting (i.e., dev_printk, and sdev_printk in the SCSI subsystem --- guess who suggested that :-), that there is an opportunity to push a lot more information out to userspace for people who want to use this facility to more easily determine when something has bad happened to a particular disk device, for example. This goes a little a beyond the original stated desire for translation support, but it was another idea to explore.
Quoted 4 lines
I find "document messages" to be a horrible idea conceptually, because I think the messages should _be_ documentation. If the message is unclear it should be clarified. "There's too much garbage on the floor, we should laminate it so we can't smell it anymore." Er, no...
There are two separable desires being addressed here. The first is "cute" messages such as "lp0: on fire". Yes, maybe those should be fixed, but sometimes kernel developers *like* cute messages. (Sniff, I'm pretty sure there used to be a "eaten by a grue" printk, but it doesn't seem to be there any more. :-) Fixing these is largely a political problem: "of course printer on fire makes sense".
The second desire is where people want entire paragraphs discsusing possible causes of the messages and recommended responses to be carried out by the data center's 3rd shift operations guy (aka "tape monkey"). That's not something you *ever* want to put in the kernel, and who ever compiles such a long document will probably sell it for $$$ as significant value add. Which is fine, because it is going to cost a lot of $$$ to keep it up to date. :-)
Quoted 7 lines
Quoted 2 lines
printk hashes:
- Additional preprocessor step neededOnly for you guys. Those of us building our own kernels and NOT translating them or producing "the big book of explanations of kernel messages for IBM customers who don't understand them" shouldn't have to run this preprocessor step.
Yep, I suspect it will only be used by distro kernels. I probably wouldn't turn it on myself. :-)
Quoted 9 lines
Quoted 4 lines
- Hard to keep printks and descriptions up-to-date. If using hashes, each typo fix in a printk breaks the link to your documentation.
- Since the developer knows the meaning of a printk best, he probably would be the right person to write the description.Imposing additional requirements on developers isn't going to work unless you reject patches from developers who don't follow your procedures and submit the appropriately filled-out forms with each patch for processing by the central bureaucracy.
Yep, it's going to be up to the translators to keep the message catalogs up to date. The gettext folks have figured out ways of doing fuzzy string matches, and so if you keep the external database of filename, line #, string, hash, and translation(s), when you go to new version of the kernel, it's not that hard to make the fuzzy translations work.
- Ted
Re: [Lf_kernel_messages] Re: Documentation of kernel messages (Summary)
O> Yep, it's going to be up to the translators to keep the message
Quoted 5 lines
catalogs up to date. The gettext folks have figured out ways of doing fuzzy string matches, and so if you keep the external database of filename, line #, string, hash, and translation(s), when you go to new version of the kernel, it's not that hard to make the fuzzy translations work.
The tools can extract the strings if marked _() which is easy work to do and you usually only want to hit a subset of strings anyway [the ones that occur usually].
Having implemented an encheferizer in a much older kernel its worth it even for the humour value of a valspeak kernel (plus some people will never be happy until it says things like
SYS-E-FOO: Incorrect operating system in use
Re: [Lf_kernel_messages] Re: Documentation of kernel messages (Summary)
In reply to Theodore TsoQuoted 6 lines · On Monday 09 July 2007 14:10:07 Theodore Tso wrote:
The idea that was tossed about was essentially printk hashes (hashed on message plus filename, with optional reporting of filename+line number), with the messaging catalog being maintained externally. Yes, that will be a major pain for whoever wants to do the translation --- but it puts the onus on the people who would be getting the value for this to do the work.
Sounds good to me.
Rob
Signature · 2 lines
"One of my most productive days was throwing away 1000 lines of code." - Ken Thompson.
I think my general response to something like this, if it goes in, would be to stop emitting useful kernel log messages in the code I write because having to document it too on top of that is just too much extra work to be worthwhile.
Quoted 4 lines · David Miller
I think my general response to something like this, if it goes in, would be to stop emitting useful kernel log messages in the code I write because having to document it too on top of that is just too much extra work to be worthwhile.
Nobody will force you to document your messages. As I said, there is no need to document self-explaining printks.
Michael