Why no XML in the Kernel?
Can somebody tell me why the Kernel-Development dont wanne have XML is being used in the Kernel??
Regards
Ahmad Reza Cheraghi
__________________________________
Yahoo! Mail - PC Magazine Editors' Choice 2005
http://mail.yahoo.com
Quoted 3 lines · Ahmad Reza Cheraghi
Can somebody tell me why the Kernel-Development dont wanne have XML is being used in the Kernel??
Because nobody has come up with a good reason why it should be. Same as anything that isn't in the kernel.
Nick
Signature · 3 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com
Quoted 11 lines · Nick Piggin
Quoted 3 lines · Ahmad Reza Cheraghi
Can somebody tell me why the Kernel-Development dont wanne have XML is being used in the Kernel??
Because nobody has come up with a good reason why it should be. Same as anything that isn't in the kernel.
Nick
I have a requirement to pass information from the kernel to user space. The information is passed fairly rarely, but over time extra parameters are added. At the moment we just use a struct, but that means that the kernel and the userspace app have to both keep in step. If something like XML was used, we could implement new parameters in the kernel, and the user space could just ignore them, until the user space is upgraded. XML would initially seem a good idea for this, but are there any methods currently used in the kernel that could handle these parameter changes over time.
For example, should the sysfs be used for this?
Any comments?
James
Quoted 7 lines · James Courtier-Dutton
I have a requirement to pass information from the kernel to user space. The information is passed fairly rarely, but over time extra parameters are added. At the moment we just use a struct, but that means that the kernel and the userspace app have to both keep in step. If something like XML was used, we could implement new parameters in the kernel, and the user space could just ignore them, until the user space is upgraded. XML would initially seem a good idea for this, but are there any methods currently used in the kernel that could handle these parameter changes over time.
For example, should the sysfs be used for this?
Any comments?
Yes use sysfs (or procfs if the information is related to a process). Using ASCII text representation, and a single value per file is noramlly the preferred method to do this I think.
Nick
Signature · 4 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com
Hello Nick,
In case /proc or /sys wouldn't be preffered, a simple key value
binary format would do.
First a key/value pair count. Followed by the key/value pairs, which
consist out of the value and key prefixed by their size.
Your user and kernel code could ignore keys they don't regognize.
A better way though, is negotiate what types of structures to pass.
The user space program would support all previous version of the kernel module.
Bas
Quoted 27 lines · Nick Piggin
Quoted 10 lines · James Courtier-Dutton
I have a requirement to pass information from the kernel to user space. The information is passed fairly rarely, but over time extra parameters are added. At the moment we just use a struct, but that means that the kernel and the userspace app have to both keep in step. If something like XML was used, we could implement new parameters in the kernel, and the user space could just ignore them, until the user space is upgraded. XML would initially seem a good idea for this, but are there any methods currently used in the kernel that could handle these parameter changes over time.
Quoted 4 lines
For example, should the sysfs be used for this?
Any comments?
Yes use sysfs (or procfs if the information is related to a process). Using ASCII text representation, and a single value per file is noramlly the preferred method to do this I think.
Nick
--
SUSE Labs, Novell Inc.
Quoted 8 lines
Send instant messages to your online friends http://au.messenger.yahoo.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
--
Bas Westerbaan
http://blog.w-nz.com/
GPG Public Keys: http://w-nz.com/keys/bas.westerbaan.asc
Quoted 11 lines · James Courtier-Dutton
Quoted 8 lines · Nick Piggin
Quoted 2 lines · Ahmad Reza Cheraghi
Can somebody tell me why the Kernel-Development dont wanne have XML is being used in the Kernel??
Because nobody has come up with a good reason why it should be. Same as anything that isn't in the kernel. Nick
I have a requirement to pass information from the kernel to user space. The information is passed fairly rarely, but over time extra parameters are added. At the moment we just use a struct, but that means that the kernel and the userspace app have to both keep in step. If something like XML was used, we could implement new parameters in the kernel, and the user space could just ignore them, until the user space is upgraded. XML would initially seem a good idea for this, but are there any methods currently used in the kernel that could handle these parameter changes over time.
If you only add parameters, why not use a struct with a length field? That can encode version information implicitly; You can assume everything beyond the length is not present in that ABI version. If the interface is based on read(2), then you don't even need the length field in the structure, since everything can be handled by userspace specifying the length it wants as the read size parameter, and the kernel returning only the bytes for the fields it knows about.
- Jim Bruce
Quoted 3 lines
For example, should the sysfs be used for this?
Any comments?
James
Quoted 13 lines · James Courtier-Dutton
Quoted 11 lines · Nick Piggin
Quoted 3 lines · Ahmad Reza Cheraghi
Can somebody tell me why the Kernel-Development dont wanne have XML is being used in the Kernel??
Because nobody has come up with a good reason why it should be. Same as anything that isn't in the kernel.
Nick
Quoted 9 lines
I have a requirement to pass information from the kernel to user space. The information is passed fairly rarely, but over time extra parameters are added. At the moment we just use a struct, but that means that the kernel and the userspace app have to both keep in step. If something like XML was used, we could implement new parameters in the kernel, and the user space could just ignore them, until the user space is upgraded. XML would initially seem a good idea for this, but are there any methods currently used in the kernel that could handle these parameter changes over time.
Yes, look at /proc/meminfo for instance. Everytime you need to return multiple values from a single file, you can easily do it with one key per line using the following syntax :
key: value [value ...]
It's also how SMTP and HTTP works and servers often send data that most clients simply ignore.
Regards,
Willy
Quoted 11 lines · James Courtier-Dutton
Quoted 9 lines · Nick Piggin
Quoted 3 lines · Ahmad Reza Cheraghi
Can somebody tell me why the Kernel-Development dont wanne have XML is being used in the Kernel??
Because nobody has come up with a good reason why it should be. Same as anything that isn't in the kernel.
Quoted 8 lines
Quoted 2 lines
Nick
I have a requirement to pass information from the kernel to user space. The information is passed fairly rarely, but over time extra parameters are added. At the moment we just use a struct, but that means that the kernel and the userspace app have to both keep in step. If something like XML was used, we could implement new parameters in the kernel, and the user space could just ignore them, until the user space is upgraded. XML would initially seem a good idea for this, but are there any methods currently used in the kernel that could handle these parameter changes over time.
For example, should the sysfs be used for this?
Any comments?
For decades people have been solving this with key:value, ignoring unknown keys. And using nice text format eliminates any endian and word size issues.
XML sounds like overkill for problems of this type.
Signature · 3 lines
-bill davidsen ([email protected]) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me
Quoted 2 lines · Ahmad Reza Cheraghi
Can somebody tell me why the Kernel-Development dont wanne have XML is being used in the Kernel??
I think it's mostly because the marketeers haven't got to everyone yet :-) The kernel might be e-enabled, but it's not fully iBuzzwordcompliant.
Besides, having a *fully* XML parser/generator in the kernel is extremely braindead, even if it's called libkernelxml ("I'm not really quite in the kernel really") or something. What's so wrong with really simple files being exposed to userspace? Most of these are less than a single page of data and it's just dandy like that.
Jon.
On 2 Oct 2005, Jon Masters suggested tentatively:
Quoted 5 lines
Besides, having a *fully* XML parser/generator in the kernel is extremely braindead, even if it's called libkernelxml ("I'm not really quite in the kernel really") or something. What's so wrong with really simple files being exposed to userspace? Most of these are less than a single page of data and it's just dandy like that.
I beat off a suggestion last week[1] that the kernel should not bother with any of this /proc and /sys stuff' but just export a single vast heap of XML, because XML is properly structured'.
(As opposed to, oh, a filesystem, which is apparently not structured enough.)
Considerations of ugliness and difficulty of implementing the equivalent of writes to procs files did not shift the twit: but starting top on a busy system and showing said twit the CPU load spikes as /proc/[0-9]* got iterated over, and asking `how severe would this be if *all* of /proc and /sys had to be generated for every single request?' seems to have imparted enough clue.
I have a large notice by my desk at work reading simply `XML is not a panacea'. It's amazing how few people wandering by read it.
[1] by someone who couldn't have implemented it in a month of Sundays, thankfully
Signature · 4 lines
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James NicollQuoted 6 lines · Nix
Considerations of ugliness and difficulty of implementing the equivalent of writes to procs files did not shift the twit: but starting top on a busy system and showing said twit the CPU load spikes as /proc/[0-9]* got iterated over, and asking `how severe would this be if *all* of /proc and /sys had to be generated for every single request?' seems to have imparted enough clue.
Another fun consideration in that area is that XML (or s-exp, or trees, whatever representation you prefer) has nothing to help with dynamic data structures. Exporting snapshots does not work since the real state includes the information about locks being held - without that you can't tell which invariants hold at the moment, since the real ones include lock state. And forcing all locks involved into known state is nowhere near feasible, of course. OTOH, exporting dynamic state including locks and walking the damn thing is
a) not feasible with XML
b) would require giving userland way too much access to locking,
creating a nightmare wrt deadlock potential.
On Mon, 3 Oct 2005, Al Viro moaned:
Quoted 6 lines
Another fun consideration in that area is that XML (or s-exp, or trees, whatever representation you prefer) has nothing to help with dynamic data structures. Exporting snapshots does not work since the real state includes the information about locks being held - without that you can't tell which invariants hold at the moment, since the real ones include lock state.
Oh yes; the only practical way to get the system into a consistent state would be to take out the BKL (the old, non-preemptible variant), generate all that XML (for all of /proc and all of /sys!) and then release it again.
Efficient this is *not*.
(at least, not the loony everything-in-one-big-file variant. Keeping the current smaller files but making them XML is possible, but pointless:
the filesystem already provides the hierarchical structure in /sys, and nothing can make /proc regular, so what's the point of adding an extra layer of hierarchy that serves only to complicate parsing and make it hard for *humans* to use?)
Quoted 4 lines
And forcing all locks involved into known state
is nowhere near feasible, of course. OTOH, exporting dynamic state including locks and walking the damn thing is
a) not feasible with XML
It's feasible, if you don't mind ps(1) becoming a DoS attack, and one running instance of top(1) damn-nearly freezing the system.
It's just not *sane*.
Quoted 2 lines
b) would require giving userland way too much access to locking,
creating a nightmare wrt deadlock potential.
Indeed.
(Current rant: DRM churn, forcing one of abandonment of decent 3D support, or upgrading of the X server to the bleeding-edge, or using an old kernel with known security holes, or becoming enough of a DRI developer to backport the changes, or using nothing but distro kernels <=2.6.11. Most of these are not terribly feasible for me right now. Ah well, my 3D card is total crap anyway. It's just a shame the X server crashes whenever asked to do in-software 3D rendering... time to debug. I thought I might actually get some work done this evening. Fat chance.)
Signature · 4 lines
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James NicollQuoted 37 lines · Nix
On Mon, 3 Oct 2005, Al Viro moaned:
Quoted 6 lines
Another fun consideration in that area is that XML (or s-exp, or trees, whatever representation you prefer) has nothing to help with dynamic data structures. Exporting snapshots does not work since the real state includes the information about locks being held - without that you can't tell which invariants hold at the moment, since the real ones include lock state.
Oh yes; the only practical way to get the system into a consistent state would be to take out the BKL (the old, non-preemptible variant), generate all that XML (for all of /proc and all of /sys!) and then release it again.
Efficient this is *not*.
(at least, not the loony everything-in-one-big-file variant. Keeping the current smaller files but making them XML is possible, but pointless:
the filesystem already provides the hierarchical structure in /sys, and nothing can make /proc regular, so what's the point of adding an extra layer of hierarchy that serves only to complicate parsing and make it hard for *humans* to use?)Quoted 4 lines
And forcing all locks involved into known state
is nowhere near feasible, of course. OTOH, exporting dynamic state including locks and walking the damn thing is
a) not feasible with XMLIt's feasible, if you don't mind ps(1) becoming a DoS attack, and one running instance of top(1) damn-nearly freezing the system.
It's just not *sane*.
Quoted 2 lines
b) would require giving userland way too much access to locking,
creating a nightmare wrt deadlock potential.Indeed.
Quoted 10 lines
(Current rant: DRM churn, forcing one of abandonment of decent 3D support, or upgrading of the X server to the bleeding-edge, or using an old kernel with known security holes, or becoming enough of a DRI developer to backport the changes, or using nothing but distro kernels <=2.6.11. Most of these are not terribly feasible for me right now. Ah well, my 3D card is total crap anyway. It's just a shame the X server crashes whenever asked to do in-software 3D rendering... time to debug. I thought I might actually get some work done this evening. Fat chance.)
...could get rid of all the kernel function codes and just put a XML interpreter inside the kernel. That way, web-page designers could become kernel developers overnight.
Quoted 6 lines
--
`Next: FEMA neglects to take into account the possibility of fire in Old Balsawood Town (currently in its fifth year of drought and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
-
XML inside the kernel is like BASIC inside the kernel. It just doesn't belong there, even though it would work.
Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.55 BogoMips). Warning : 98.36% of all statistics are fiction.
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
Quoted 10 lines
(Current rant: DRM churn, forcing one of abandonment of decent 3D support, or upgrading of the X server to the bleeding-edge, or using an old kernel with known security holes, or becoming enough of a DRI developer to backport the changes, or using nothing but distro kernels <=2.6.11. Most of these are not terribly feasible for me right now. Ah well, my 3D card is total crap anyway. It's just a shame the X server crashes whenever asked to do in-software 3D rendering... time to debug. I thought I might actually get some work done this evening. Fat chance.)
This is just mach64, we don't have mach64 support in the kernel so it has nothing to do with the kernel... I've no idea why mach64 broke but upgrading everything won't fix it.. the kernel-drm and userspace do not need to be kept in any sort of lockstep, and the mach64 dri code hasn't been touched by anyone in > 1 year probably even two.. (I was last person to own one to test it on...)
If you could nail down exactly when it went south then we can fix it... but giving out here won't help anyone..
Dave.
On Thu, 6 Oct 2005, Dave Airlie announced authoritatively:
Quoted 10 lines
Quoted 9 lines
(Current rant: DRM churn, forcing one of abandonment of decent 3D support, or upgrading of the X server to the bleeding-edge, or using an old kernel with known security holes, or becoming enough of a DRI developer to backport the changes, or using nothing but distro kernels <=2.6.11. Most of these are not terribly feasible for me right now. Ah well, my 3D card is total crap anyway. It's just a shame the X server crashes whenever asked to do in-software 3D rendering... time to debug. I thought I might actually get some work done this evening. Fat chance.)
Quoted 2 lines
This is just mach64, we don't have mach64 support in the kernel so it has nothing to do with the kernel... I've no idea why mach64 broke but
I misspoke. Some of the non-DRM API changes around 2.6.12 broke the 6.8.2 mach64 module in DRI CVS; the development version builds again, and nearly works.
Quoted 2 lines
upgrading everything won't fix it.. the kernel-drm and userspace do not need to be kept in any sort of lockstep,
I could swear that I've encountered problems in the past due to upgrading one and not the other, but since I can't actually remember what those problems *were*, I'll shut up.
Quoted 3 lines
and the mach64 dri code
hasn't been touched by anyone in > 1 year probably even two.. (I was last person to own one to test it on...)
I'd not have got a mach64 if I hadn't got the machine free. I hear R9200s are the most useful 3D cards with free drivers around at the
moment: I suppose I should get one. (It's not as though the Mach64's hardware 3D support is significantly faster than software rendering on this 1300MHz Athlon anyway.)
Quoted 2 lines
If you could nail down exactly when it went south then we can fix it... but giving out here won't help anyone..
It was just a random rant: I know that I can't really complain until I've tried the development version, so I've built up 6.8.99.900, which nearly half sort-of works, modulo only fdo bugs #4574 (which bug breaks the mach64 DRM module, but at least it builds and modprobes OK), and #4696 (which kills the X server and is probably a Mesa bug rather than anything Mach64-specific).
No kernel-tree problems, really, and I'm not complaining about the speed of kernel development at all, let alone asking for some mythical and restrictive stable module API. I'm just wondering if I'll ever be able to use DRI for more than a month at a time without something breaking it :( such are the risks of using a card with out-of-tree DRM support.
(Aside: I must say I'm damn impressed by the X server's internal API stability. I screwed up the installation of X.org 6.8.99.900, and for a day or so was unknowingly running the X.org 6.8.2 X server against the 6.8.99.900 modules. Except for problems drawing certain small pixmaps, it seemed to work perfectly well!)
Signature · 4 lines
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James NicollQuoted 8 lines
Quoted 3 lines
This is just mach64, we don't have mach64 support in the kernel so it has nothing to do with the kernel... I've no idea why mach64 broke but
I misspoke. Some of the non-DRM API changes around 2.6.12 broke the 6.8.2 mach64 module in DRI CVS; the development version builds again, and nearly works.
But that's my point if you had a previously working mach64 with 6.8.2 with a DRM from around then, and a kernel upgrade broke the DRM you should be just able to upgrade the DRM to the latest DRM CVS, there isn't any such thing as a 6.8.2 DRM, you'll only cause much more issues trying to fix an issue in the kernel side by also simultaneously upgrading userspace... as you've no fixed working point to try from..
The mach64 drm is in CVS on cvs.freedesktop.org:/cvs/dri and the module is drm..
Dave.
On Thu, 6 Oct 2005, Dave Airlie uttered the following:
Quoted 8 lines
Quoted 3 lines
I misspoke. Some of the non-DRM API changes around 2.6.12 broke the 6.8.2 mach64 module in DRI CVS; the development version builds again, and nearly works.
But that's my point if you had a previously working mach64 with 6.8.2 with a DRM from around then, and a kernel upgrade broke the DRM you should be just able to upgrade the DRM to the latest DRM CVS, there isn't any such thing as a 6.8.2 DRM,
Ah. So, er, the DRM<-> userspace protocol is stable, then?
Looks like I was working on bad assumptions (assuming the DRM and X were tied). I'm not sure where those assumptions came from. Possibly just that they shared a CVS repo, although I'd hope I'd had more evidence than that. I realy can't recall.
Quoted 4 lines
you'll only cause much more
issues trying to fix an issue in the kernel side by also simultaneously upgrading userspace... as you've no fixed working point to try from..
That's OK, the -RC0 X server actually works better than the 6.8.2 one for my application, and this gives me an excuse to report some bugs before 6.9/7.0 is released :)
Thanks for the advice, anyway: I'll certainly bear it in mind in future.
Signature · 4 lines
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James NicollQuoted 7 lines
Ah. So, er, the DRM<-> userspace protocol is stable, then?
Looks like I was working on bad assumptions (assuming the DRM and X were tied). I'm not sure where those assumptions came from. Possibly just that they shared a CVS repo, although I'd hope I'd had more evidence than that. I realy can't recall.
In theory yes, on occasion I do get bugs that break XFree86 4.3, but these are bugs as opposed to design decisions, upgrading the kernel should never require upgrading to a new version of X or anything like that, however upgrading X can sometimes require a newer kernel in order to take advantage of newer drm features.. but X should always work with the older drms...
Dave.
Quoted 16 lines · Dave Airlie
Quoted 8 lines
Ah. So, er, the DRM<-> userspace protocol is stable, then?
Looks like I was working on bad assumptions (assuming the DRM and X were tied). I'm not sure where those assumptions came from. Possibly just that they shared a CVS repo, although I'd hope I'd had more evidence than that. I realy can't recall.
In theory yes, on occasion I do get bugs that break XFree86 4.3, but these are bugs as opposed to design decisions, upgrading the kernel should never require upgrading to a new version of X or anything like that, however upgrading X can sometimes require a newer kernel in order to take advantage of newer drm features.. but X should always work with the older drms...
Quoted 9 lines
Dave.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
How about putting the ability to disable graphics mode in the kernel and moving this capability from X, and saving the video state. Would make kernel debuggers work a hell of a lot better when the damn thing crashes in X in the kernel. At least then the screen won;t be locked up (of course you can type "reboot " from memory while the system is still hung in X).
Jeff
Quoted 8 lines
Quoted 1 line
How about putting the ability to disable graphics mode in the kernel and moving this capability from X, and saving the video state. Would make kernel debuggers work a hell of a lot better when the damn thing crashes in X in the kernel. At least then the screen won;t be locked up (of course you can type "reboot " from memory while the system is still hung in X).
It's been on the todo list for a long while... there's been talks at different events about it, there'll be a talk at LCA from me again about it and what has happened since KS (not a huge amount)....
We've nearly all agreed on a direction, we haven't found anyone with the bandwidth to actually move things in that direction... (or at least no-one has said to me heres some money go do this thing... :-)
Dave.
Quoted 6 lines · Dave Airlie
Quoted 6 lines
How about putting the ability to disable graphics mode in the kernel and moving this capability from X, and saving the video state. Would make kernel debuggers work a hell of a lot better when the damn thing crashes in X in the kernel. At least then the screen won;t be locked up (of course you can type "reboot " from memory while the system is still hung in X).
You don't need to reboot. You can use some tools to restore text mode - svgalib comes with three such tools, that work in large majority of the cases:
textmode (on all platforms), mode3 and vga_reset (i386 only).
Quoted 7 lines
It's been on the todo list for a long while... there's been talks at different events about it, there'll be a talk at LCA from me again about it and what has happened since KS (not a huge amount)....
We've nearly all agreed on a direction, we haven't found anyone with the bandwidth to actually move things in that direction... (or at least no-one has said to me heres some money go do this thing... :-)
I intend to put that part of svgalib in kernel (restoring modes), since it would make the most complicated part of svgalib -- console switch handling -- much simpler. The problem is that svgalib only handles video mode, totally ignoring 2D/3D engines, so it might
Signature · 1 line
Matan Ziv-Av. [email protected]
--Ahmad Reza Cheraghi <[email protected]> wrote (on Sunday, October 02, 2005 02:41:42 -0700):
Quoted 2 lines
Can somebody tell me why the Kernel-Development dont wanne have XML is being used in the Kernel??
Because nobody has yet discovered strong enough weed?
M.
Quoted 10 lines · Martin J. Bligh
--Ahmad Reza Cheraghi <[email protected]> wrote (on Sunday, October 02, 2005 02:41:42 -0700):
Quoted 2 lines
Can somebody tell me why the Kernel-Development dont wanne have XML is being used in the Kernel??
Because nobody has yet discovered strong enough weed?
Well I think _somebody_ has. They've just made the mistake of not sending any to Linus before sending patches.
Send instant messages to your online friends http://au.messenger.yahoo.com