procps and new kernel fields

DISCUSSION Dave Hansen 3 participants 4 messages Apr 6, 2009 – Apr 14, 2009 Memory management

Share & tools

Dave Hansen

author

2009-04-06 21:54 UTC #0 lore raw

There was a very brief conversation about this a year ago or so. We've got a ton of newfangled output in /proc/meminfo, but procps ignores most of it. There's also a bunch of types of memory that don't get shown by the various procps commands these days.

	http://marc.info/?l=linux-mm&m=120496901605830&w=2

Novell has integrated that patch into procps which has the side-effect that now reclaimable slab and unstable NFS pages are included in the 'cached' output of vmstat and free.

	https://bugzilla.novell.com/show_bug.cgi?id=405246

The most worrisome side-effect of this change to me is that we can no longer run vmstat or free on two machines and compare their output.

At the same time, we have machines that have dozens of GB of slab objects that are mostly reclaimable. Yet, 'free' and 'vmstat' basically ignore slab. Surely we need to find some way to report on those, especially since we can now break out {un,}reclaimable slab.

We also have "new" memory use like unstable NFS pages. How should we account for those?

I'd love to see an --extended output from things like vmstat. It could include wider output since fitting in 80 columns just isn't that important any more, and my 256GB machine's output really screws up the column alignment. We could also add some information which is in addition to what we already provide in order to account for things like slab more precisely.

-- Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Albert Cahalan

2009-04-07 04:39 UTC #1 lore raw

Quoted 1 line · Dave Hansen

Novell has integrated that patch into procps

...

Quoted 2 lines

The most worrisome side-effect of this change to me is that we can no longer run vmstat or free on two machines and compare their output.

Right. Vendors never consier that. They then expect upstream to accept and support their hack until the end of time.

Quoted 7 lines

At the same time, we have machines that have dozens of GB of slab objects that are mostly reclaimable. Yet, 'free' and 'vmstat' basically ignore slab. Surely we need to find some way to report on those, especially since we can now break out {un,}reclaimable slab.
We also have "new" memory use like unstable NFS pages. How should we account for those?

How should I know? :-)

I've seen memstat fields go away before. This makes me reluctant to bother with data that few people will need or even understand. Why should I believe that these fields are now securely part of the kernel ABI?

Quoted 4 lines

I'd love to see an --extended output from things like vmstat. It could include wider output since fitting in 80 columns just isn't that important any more, and my 256GB machine's output really screws up the column alignment.

80 is still the default xterm width, the default console width, and sometimes the only available width.

I certainly like the idea of allowing options for extra columns and for wider columns.

The current code is too simplistic to handle such changes without becoming an unreadable mess. It really needs to be rewritten in a more serious style, much like ps.

Quoted 3 lines

We could also add some information which is in
addition to what we already provide in order to account for things like slab more precisely.

How do I even explain a slab? What about a slob or slub? A few years from now, will this allocator even exist?

Remember that I need something for the man page, and most of my audience knows almost nothing about programming.

Bart Van Assche

2009-04-07 13:02 UTC #2 lore raw

Quoted 8 lines · Albert Cahalan

Quoted 1 line · Dave Hansen
Novell has integrated that patch into procps
...
Quoted 2 lines
The most worrisome side-effect of this change to me is that we can no longer run vmstat or free on two machines and compare their output.
Right. Vendors never consier that. They then expect upstream to accept and support their hack until the end of time.

The patch that was integrated by this vendor in their procps package was posted on a public mailing list more than a year ago. It would have helped if someone would have commented earlier on that patch.

Quoted 9 lines

Quoted 3 lines
We could also add some information which is in
addition to what we already provide in order to account for things like slab more precisely.
How do I even explain a slab? What about a slob or slub? A few years from now, will this allocator even exist?
Remember that I need something for the man page, and most of my audience knows almost nothing about programming.

It's not the difference between SLAB, SLOB and SLUB that matters here, but the fact that some of the memory allocated by these kernel allocators can be reclaimed. The procps tools currently count reclaimable SLAB / SLOB / SLUB memory as used memory, which is misleading. How can this be explained to someone who is not a programmer ?

Bart.

Dave Hansen

author

2009-04-14 18:34 UTC #3 lore raw

Quoted 7 lines · Bart Van Assche

Quoted 6 lines · Albert Cahalan
Quoted 1 line · Dave Hansen
Novell has integrated that patch into procps
...
Quoted 2 lines
The most worrisome side-effect of this change to me is that we can no longer run vmstat or free on two machines and compare their output.

Quoted 6 lines

Quoted 2 lines
Right. Vendors never consier that. They then expect upstream to accept and support their hack until the end of time.
The patch that was integrated by this vendor in their procps package was posted on a public mailing list more than a year ago. It would have helped if someone would have commented earlier on that patch.

We suck. :)

Quoted 10 lines

Quoted 9 lines
Quoted 3 lines
We could also add some information which is in
addition to what we already provide in order to account for things like slab more precisely.
How do I even explain a slab? What about a slob or slub? A few years from now, will this allocator even exist?
Remember that I need something for the man page, and most of my audience knows almost nothing about programming.

Quoted 6 lines

It's not the difference between SLAB, SLOB and SLUB that matters here, but the fact that some of the memory allocated by these kernel allocators can be reclaimed. The procps tools currently count reclaimable SLAB / SLOB / SLUB memory as used memory, which is misleading. How can this be explained to someone who is not a programmer ?

I actually think it is probably OK to call them "cache". They *are* quite similar to the page cache from a user's perspective. I just have a problem with changing it *now*, though.

Page cache is fundamentally user data. It is verbatim in memory exactly what came off or is going to the filesystem, and gets exposed to userspace directly.

The various sl*bs are fundamentally kernel data. They're never seen by users directly.

So, if I were to write a tool that told users of both slab and page cache, I'd probably say "file cache" and "kernel cache" or something to that effect. It's a bit over-simplified, but I think it gets the point across sufficiently. Personally, I'd probably rather see 'buffers' get collapsed in with 'cache' and get a new column for sl*b.

-- Dave