2.6.x BSD Process Accounting w/High UID
Quoted 4 lines
We might, however, put the magic into the implicit padding of
ac_flag(ugly, since this would require an arch dependant definition of stuct acct), or put it into the uppermost three bits ofac_flagitself
First of all, none of this matters much if the format is given a sysctl. The old format is default for now, and the new one is default (only?) in a couple years. Sun appears to have done something like this.
When fixing it, note that a 5-bit binary exponent with denormals would beat the current float format.
Regarding the existing struct though...
Let's take a close look at this. I think there are 2 bytes of padding on all Linux ports, and another 2 available on everything except maybe m68k and/or arm. (that is, ports that will put a u32 on any u16 boundry) Here is the current struct, compactly formatted with 64-bit blocking:
struct linux_acct {
char ac_flag; // Flags
// 1 pad byte
__u16 ac_uid; // Real User ID
__u16 ac_gid; // Real Group ID
__u16 ac_tty; // Control Terminal __u32 ac_btime; // Process Creation Time
comp_t ac_utime; // User Time
comp_t ac_stime; // System Time comp_t ac_etime; // Elapsed Time
comp_t ac_mem; // Average Memory Usage
comp_t ac_io; // Chars Transferred
comp_t ac_rw; // Blocks Read or Written comp_t ac_minflt; // Minor Pagefaults
comp_t ac_majflt; // Major Pagefaults
comp_t ac_swaps; // Number of Swaps// 2 pad bytes (except m68k or arm maybe?)
__u32 ac_exitcode; // Exitcode
// hppa might pad this
char ac_comm[17]; // Command Name
// hppa might pad this
char ac_pad[10]; // Padding Bytes
// 1 pad byte (maybe 6 for hppa)
};
Just a sec... ARRRGH WHY DO PEOPLE LEAVE PADDING AND GENERALLY MIS-ALIGN THINGS ALL THE TIME??????
So there you go. You have a pad byte after ac_flag at a known location, and a pad byte after ac_pad that might move a bit due to mis-alignments above it.
Quoted 4 lines · Albert Cahalan
First of all, none of this matters much if the format is given a sysctl. The old format is default for now, and the new one is default (only?) in a couple years. Sun appears to have done something like this.
Yeah, if we even want to introduce a new syscall for this... But Documentation/Changes is not an empty file, i.e., we require people to upgrade userspace anyways.
Quoted 2 lines
When fixing it, note that a 5-bit binary exponent with denormals would beat the current float format.
Yes, but only by a short head. And, since comp_t is hard-coded into current GNU acct tools, we can't keep source compatibility (not that this matters too much anyways...)
I'm open for suggestions in this direction. Any reasonable ideas to get more precision? (e.g. 16 bit mantissa and 4 bit base-whatever exponent?)
Quoted 11 lines
Regarding the existing struct though...
Let's take a close look at this. I think there are 2 bytes of padding on all Linux ports, and another 2 available on everything except maybe m68k and/or arm. (that is, ports that will put a u32 on any u16 boundry) Here is the current struct, compactly formatted with 64-bit blocking:
struct
linux_acct{
charac_flag; // Flags
// 1 pad byte
Yep, but depending on architecure I think the compiler is free to insert the padding either before or after ac_flags.
[...]
Quoted 3 lines
Just a sec... ARRRGH WHY DO PEOPLE LEAVE PADDING AND GENERALLY MIS-ALIGN THINGS ALL THE TIME??????
Yupp, I changed this for the new suggested binary-incompatible layout.
Quoted 3 lines
So there you go. You have a pad byte after
ac_flagat a known location, and a pad byte afterac_padthat might move a bit due to mis-alignments above it.
Yes, but I'd prefer an arch-independent and prominent position (i.e. one that doesn't perturb alignment at the beginning of the struct)
Tim
Quoted 10 lines · Tim Schmielau
Quoted 4 lines · Albert Cahalan
First of all, none of this matters much if the format is given a sysctl. The old format is default for now, and the new one is default (only?) in a couple years. Sun appears to have done something like this.
Yeah, if we even want to introduce a new syscall for this... But
Documentation/Changesis not an empty file, i.e., we require people to upgrade userspace anyways.
In that case, don't forget to upgrade atop. That's a version of top that uses BSD process accounting to grab the transient processes that wouldn't be seen in /proc before they die.
Quoted 4 lines
Quoted 2 lines
When fixing it, note that a 5-bit binary exponent with denormals would beat the current float format.
Yes, but only by a short head.
It's by 8 bits, with a stable 11-bit fraction instead of a 10-bit to 12-bit variable-size one.
old: 1xxxxxxxxxxyy000000000000000000000
new: 1xxxxxxxxxxx000000000000000000000000000000
That's a 42-bit number instead of a 36-bit one. The old base-8 exponent is wasteful, plus the old format stores the leading 1 instead of using an implied 1 and special exponent for leading 0.
Quoted 6 lines
And, since
comp_tis hard-coded into
current GNU acct tools, we can't keep source compatibility (not that this matters too much anyways...)
I'm open for suggestions in this direction. Any reasonable ideas to get more precision? (e.g. 16 bit mantissa and 4 bit base-whatever exponent?)
a. just what I said
b. 32-bit IEEE float (easy enough to encode by hand) c. raw data -- is the space saving all that critical? d. raw data with gzip-style compression
(note that gzip's deflate algorithm is in the kernel)
Quoted 8 lines
Quoted 8 lines
Regarding the existing struct though...
Let's take a close look at this. I think there are 2 bytes of padding on all Linux ports, and another 2 available on everything except maybe m68k and/or arm. (that is, ports that will put a u32 on any u16 boundry) Here is the current struct, compactly formatted with 64-bit blocking:
Quoted 6 lines
Quoted 3 lines
struct
linux_acct{
charac_flag; // Flags
// 1 pad byteYep, but depending on architecure I think the compiler is free to insert the padding either before or after
ac_flags.
I doubt it. I think the C standard has something to say about this. In any case, I just checked a mix of big-endian and little-endian boxes:
32-bit BE SPARC
64-bit BE SPARC
32-bit BE PowerPC
32-bit LE i386
64-bit LE x86-64
64-bit LE Alpha
In every case, 1-byte padding came after ac_flag.
I'm pretty sure ac_comm is too big as well. It has room for 17 bytes. It needs room for 15, plus a '\0' if you want one.
Quoted 4 lines · Albert Cahalan
In that case, don't forget to upgrade atop. That's a version of top that uses BSD process accounting to grab the transient processes that wouldn't be seen in /proc before they die.
OK, thanks for the reference. I wonder what other packages use BSD accounting.
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines
When fixing it, note that a 5-bit binary exponent with denormals would beat the current float format.
Yes, but only by a short head.
Quoted 7 lines
It's by 8 bits, with a stable 11-bit fraction instead of a 10-bit to 12-bit variable-size one.
old: 1xxxxxxxxxxyy000000000000000000000
new: 1xxxxxxxxxxx000000000000000000000000000000That's a 42-bit number instead of a 36-bit one.
Huh, are we talking about the same format? I think comp_t rather looks like this:
eeemmmmmmmmmmmmm
i.e., three bit (base 8) exponent, followed by 13 bits mantissa. mantissa is right-aligned, so I don't understand why you are talking about denormals at all.
The base 8 exponent indeed wastes up to two bits of the mantissa, but it saves two bits of the exponent, so it's as good in the worst case and gains one bit of precision on average.
The largest representable number (as I see it) is only (2^13-1) * 8^7 = 0x3ffe00000, i.e. a 34 bit number.
What's wrong with my view?
Quoted 3 lines
The old base-8 exponent is wasteful, plus the old format stores the leading 1 instead of using an implied 1 and special exponent for leading 0.
I really don't know which leading 1 you are talking about.
Quoted 5 lines
Quoted 2 lines
get more precision? (e.g. 16 bit mantissa and 4 bit base-whatever exponent?)
a. just what I said
b. 32-bit IEEE float (easy enough to encode by hand)
great idea - it even allows to keep source code compatibility with (at least) the GNU acct tools!
Quoted 1 line
c. raw data -- is the space saving all that critical?
No, but I think it's just not worth expanding struct acct.
Quoted 3 lines
d. raw data with gzip-style compression
(note that gzip's deflate algorithm is in the kernel)
too unpredictable, I think.
e. suddenly pops to my mind: #define AHZ 10 /* ;-) */
but b. seems superior to me as it increases precision as well and we can afford it.
Quoted 5 lines
Quoted 2 lines
Yep, but depending on architecure I think the compiler is free to insert the padding either before or after
ac_flags.I doubt it. I think the C standard has something to say about this.
I'd guess I'll have to update my K&R to ANSI on that...
Quoted 11 lines
In any case, I just checked a mix of
big-endian and little-endian boxes:32-bit BE SPARC
64-bit BE SPARC
32-bit BE PowerPC
32-bit LE i386
64-bit LE x86-64
64-bit LE AlphaIn every case, 1-byte padding came after
ac_flag.
That sounds good. A version magic at the beginning of the struct would allow us to extend it more easily.
Tim
Quoted 1 line · Albert Cahalan
That's a 42-bit number instead of a 36-bit one.
OK, your format clearly wins. Especially since I think that comp_t can only encode a 34-bit number.
But I favor your suggestion of 32-bit IEEE floats even more, as it doesn't need a change to the GNU acct tools.
Sorry for the confusion
Tim
Quoted 6 lines · Tim Schmielau
Quoted 1 line · Albert Cahalan
That's a 42-bit number instead of a 36-bit one.
OK, your format clearly wins. Especially since I think that
comp_tcan only encode a 34-bit number.
That is correct. 42 - 8 != 36
My diagram was right; the math was not.
Quoted 2 lines
But I favor your suggestion of 32-bit IEEE floats even more, as it doesn't need a change to the GNU acct tools.
I'm surprised. Do the tools rely on a #define for this?
Is there a reason to have the whole struct be a power of two? If so, and you don't wish to expand it to 128 bytes, consider packing 3 80-byte records and a 16-byte header into 256 bytes.
Quoted 4 lines · Albert Cahalan
Quoted 2 lines
But I favor your suggestion of 32-bit IEEE floats even more, as it doesn't need a change to the GNU acct tools.
I'm surprised. Do the tools rely on a #define for this?
They check the types with autoconf. In case of a comp_t, the hard-coded conversion routine is called, otherwise the value is just cast to a double.
Quoted 2 lines
Is there a reason to have the whole struct be a
power of two?
Not that I'm aware of. But I hesitate to change the length of the struct since seeking in a file with mixed field lengths would become a nightmare. And why would we want to store a version number in each acct record if we can't mix them afterwards?
So my current plot would be to expand the most interesting fields to floats, using up the padding, and leave the others as comp_t.
Actually, the only field of interest here seems to be ac_etime where, as Arthur Corliss pointed out, also precision matters.
Less impotant candidates might be ac_io, ac_rw and/or ac_minflt.
For ac_utime and ac_stime, on the other hand, 497 days of cpu time for each job seems rather plentyful. Even more, if we'd fix the internal overflow in GNU acct's comp_t conversion routine and have 1988 days at hand.
Tim