A change to string encoding
The problem with current string encoding is that it is parsable, but non-human readable. It also complicates parsing by requiring 2 different decoding methods to be implemented.
It occurs to me that a URL encoding scheme would also meet the parsing requirements. Additionally:
1. It is always human readable.
2. There is only 1 encoding scheme.
3. Substring matching on encoded strings will always succeed.
URL encoding is just one way to achieve this, and has the advantage of being widely implemented. However, the minimal requirements would be a scheme which encoded only separator characters (whitespace in this case) without the use of those separators.
I'm sure this has been considered before. Given that it's a road I'm considering heading down, what were the reasons for not doing it?
Thanks,
Matt
Signature · 6 lines
Matthew Booth, RHCA, RHCSS Red Hat, Global Professional Services M: +44 (0)7977 267231 GPG ID: D33C3490 GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490
Quoted 2 lines · Matthew Booth
The problem with current string encoding is that it is parsable, but non-human readable.
There are times when it has things that would never be human readable.
Quoted 2 lines
URL encoding is just one way to achieve this, and has the advantage of being widely implemented.
Inside the kernel?
Quoted 2 lines
I'm sure this has been considered before. Given that it's a road I'm considering heading down, what were the reasons for not doing it?
Can you encode data structures in it? The kernel developer at the time wanted something that was either already in the kernel or something that could be implemented in a couple lines of code and something that works for any kind of encoding that needed to be done. So, I think minimal amount of code and maximum flexibility is what drove the decision.
-Steve
Quoted 5 lines · Steve Grubb
Quoted 2 lines · Matthew Booth
The problem with current string encoding is that it is parsable, but non-human readable.
There are times when it has things that would never be human readable.
Do you have an exhaustive list? Off the top of my head I can think of the record which contains a struct sockaddr.
Quoted 4 lines
Quoted 2 lines
URL encoding is just one way to achieve this, and has the advantage of being widely implemented.
Inside the kernel?
No, I mean the format output by the audit daemon.
Quoted 8 lines
Quoted 2 lines
I'm sure this has been considered before. Given that it's a road I'm considering heading down, what were the reasons for not doing it?
Can you encode data structures in it? The kernel developer at the time wanted something that was either already in the kernel or something that could be implemented in a couple lines of code and something that works for any kind of encoding that needed to be done. So, I think minimal amount of code and maximum flexibility is what drove the decision.
Sounds like this falls in to the 'easy to produce' category. We also need to consider how this data is consumed. I'll take this opportunity to suggest again that a binary format produced by the kernel would be no worse than a hex encoded binary format, and a whole lot easier for the audit daemon to mangle into something usable on the output side.
Matt
Quoted 2 lines · Steve Grubb
Can you encode data structures in it? The kernel developer at the time wanted something that was either already in the kernel or something that could be implemented in a couple lines of code and something that works for any kind of encoding that needed to be done. So, I think minimal amount of code and maximum flexibility is what drove the decision.
The comment was *not* about encoding data structures, rather it was about string encoding.
I have provided code in the past which encodes a string according to the ISO C99 standard. It does not tax the kernel, use excessive resources, or is complicated in any sense whatsoever (it's just a per character table lookup) and wraps the result in double quotes.
Yes, this would change the output of the kernel audit data, which does have the potential to break existing user code. However, it's often been stated only the official audit libraries should ever be used to read audit data and if that recommendation still holds then the audit libraries should be capable of gracefully handling either the old or new format providing a transparent transition.
I hope at some point we can start to address this reoccurring issue.
Signature · 4 lines
John Dennis <[email protected]> Looking to carve out IT costs? www.redhat.com/carveoutcosts/
Quoted 11 lines · Matthew Booth
The problem with current string encoding is that it is parsable, but non-human readable. It also complicates parsing by requiring 2 different decoding methods to be implemented.
It occurs to me that a URL encoding scheme would also meet the parsing requirements. Additionally:
1. It is always human readable.
2. There is only 1 encoding scheme.
3. Substring matching on encoded strings will always succeed.
Quoted 7 lines
URL encoding is just one way to achieve this, and has the advantage of being widely implemented. However, the minimal requirements would be a scheme which encoded only separator characters (whitespace in this case) without the use of those separators.
I'm sure this has been considered before. Given that it's a road I'm considering heading down, what were the reasons for not doing it?
It was already discussed here without a conclusion:
http://marc.info/?l=linux-audit&m=120978583018941&w=2
Signature · 3 lines
Tomas Mraz
No matter how far down the wrong road you've gone, turn back.
Turkish proverbQuoted 11 lines · Matthew Booth
The problem with current string encoding is that it is parsable, but non-human readable. It also complicates parsing by requiring 2 different decoding methods to be implemented.
It occurs to me that a URL encoding scheme would also meet the parsing requirements. Additionally:
1. It is always human readable.
2. There is only 1 encoding scheme.
3. Substring matching on encoded strings will always succeed.
Quoted 7 lines
URL encoding is just one way to achieve this, and has the advantage of being widely implemented. However, the minimal requirements would be a scheme which encoded only separator characters (whitespace in this case) without the use of those separators.
I'm sure this has been considered before. Given that it's a road I'm considering heading down, what were the reasons for not doing it?
Lack of code. And history I guess. What we have is fast and easy. Any encoding scheme must meet both of those. It's come up before with the basic agreement that what we have isn't great. It works, is about the best thing you can say about it.
Backwards compatibility is a big issue. Any new code (in the kernel at least) has to allow us to continue outputting the way we do for some time. I've said it before and I'll say it again, I'm willing to entertain a new string encoding system in the kernel but I don't have the time to write it.
There was talk that someone in the IPA project was going to write an audit plugin that would re-encode strings to something they liked, but I haven't seen it.
As long as you have some way to maintain backwards compatibility and have the time to write it, I think just about any other string encoding scheme would make people happier than what we have today...
-Eric