Skip to contentSkip to conversation

mm documentation

LSF/MM TOPIC Mike Rapoport 3 participants 4 messages May 20, 2021 – May 25, 2021 Memory management

lore
Share & tools
lore
#0 lore raw

Hi,

The mm documentation is, well, not entirely up to date. We can opt for dropping the outdated parts, which would generate a nice negative diffstat, but identifying the outdated documentation requires nearly as much effort as updating it, so I think that making and keeping the docs up to date would be a better option.

I'd like to discuss what can be done process-wise to improve the situation.

Some points I had in mind:

* Pay more attention to docs during review
* Set an expectation level for docs accompanying a changeset
* Spend some cycles to review and update the existing docs
* Spend some more cycles to add new documentation
* Participate in prorams like Google Season of Docs

I'd appreciate a discussion about how we can improve the existing memory management documentation so that a reader can get a coherent view of it, what are the gaps (although they are too many), and what would be the best way to close these gaps.

Signature · 2 lines
Sincerely yours,
Mike.
#1 lore raw
Quoted 17 lines · Mike Rapoport

The mm documentation is, well, not entirely up to date. We can opt for dropping the outdated parts, which would generate a nice negative diffstat, but identifying the outdated documentation requires nearly as much effort as updating it, so I think that making and keeping the docs up to date would be a better option.

I'd like to discuss what can be done process-wise to improve the situation.

Some points I had in mind:

* Pay more attention to docs during review
* Set an expectation level for docs accompanying a changeset
* Spend some cycles to review and update the existing docs
* Spend some more cycles to add new documentation
* Participate in prorams like Google Season of Docs

Quoted 4 lines

I'd appreciate a discussion about how we can improve the existing memory management documentation so that a reader can get a coherent view of it, what are the gaps (although they are too many), and what would be the best way to close these gaps.

I think we have four target audiences for mm documentation,

- Sysadmins/user space programmers who are trying to make their system perform well. They need documentation of the proc/sys/cmdline/... parameters that the MM pays attention to.
- Kernel programmers who are not (and do not wish to be) MM developers. Filesystem developers, device driver developers, networking developers. They need kernel-doc for our exported functions and advice for when to use and not use particular functions/flags.
- Programmers who want to "get started" in the MM area. They may or may not be familiar with Linux internals (perhaps they're moving from another kernel, perhaps their experience is with some other part of Linux; perhaps they have no experience at all). I think these people are probably well served by Mel's book, even if it is a few years old now.
- Each other. The MM is huge these days, and I certainly don't understand how it all interacts with itself. I think we actually do a pretty good job of talking to each other and writing commit messages.

I think it's really the second group where we do the worst job of documentation, but I may be biased. It's certainly where I'm focusing my documentation efforts.

#2 lore raw
Quoted 23 lines · Matthew Wilcox
Quoted 17 lines · Mike Rapoport

The mm documentation is, well, not entirely up to date. We can opt for dropping the outdated parts, which would generate a nice negative diffstat, but identifying the outdated documentation requires nearly as much effort as updating it, so I think that making and keeping the docs up to date would be a better option.

I'd like to discuss what can be done process-wise to improve the situation.

Some points I had in mind:

* Pay more attention to docs during review
* Set an expectation level for docs accompanying a changeset
* Spend some cycles to review and update the existing docs
* Spend some more cycles to add new documentation
* Participate in prorams like Google Season of Docs

Quoted 4 lines

I'd appreciate a discussion about how we can improve the existing memory management documentation so that a reader can get a coherent view of it, what are the gaps (although they are too many), and what would be the best way to close these gaps.

Quoted 5 lines

I think we have four target audiences for mm documentation,

- Sysadmins/user space programmers who are trying to make their system perform well. They need documentation of the proc/sys/cmdline/... parameters that the MM pays attention to.

I'd say this part needs some love. Just a few weeks ago we've discussed how /proc/meminfo description is outdated and there are more examples. Besides, this is probably the most important part to keep up to date.

Quoted 10 lines

- Kernel programmers who are not (and do not wish to be) MM developers. Filesystem developers, device driver developers, networking developers. They need kernel-doc for our exported functions and advice for when to use and not use particular functions/flags.
- Programmers who want to "get started" in the MM area. They may or may not be familiar with Linux internals (perhaps they're moving from another kernel, perhaps their experience is with some other part of Linux; perhaps they have no experience at all). I think these people are probably well served by Mel's book, even if it is a few years old now.

Mel's book is definitely an excellent starting point, but I afraid its age is starting to show. As it was written mostly about 2.4 with some bits about 2.6, there are lot of details that are missing in the book. Some are probably less important because the concept didn't change much (e.g. page table management), but others have considerable effect on our code (e.g migration, compaction, THP).

Quoted 3 lines

- Each other. The MM is huge these days, and I certainly don't understand how it all interacts with itself. I think we actually do a pretty good job of talking to each other and writing commit messages.

IMHO, there is a fifth group: arch developers. I think it is important to have documentation of what core mm expects from an architecture and what is the expected semantics of various low level functions architectures supply.

Quoted 3 lines

I think it's really the second group where we do the worst job of documentation, but I may be biased. It's certainly where I'm focusing my documentation efforts.

I'm not sure it's the worst, but certainly along with the first group it's the most important part.

Signature · 2 lines
Sincerely yours,
Mike.
#3 lore raw
Quoted 46 lines · Mike Rapoport
Quoted 23 lines · Matthew Wilcox
Quoted 17 lines · Mike Rapoport

The mm documentation is, well, not entirely up to date. We can opt for dropping the outdated parts, which would generate a nice negative diffstat, but identifying the outdated documentation requires nearly as much effort as updating it, so I think that making and keeping the docs up to date would be a better option.

I'd like to discuss what can be done process-wise to improve the situation.

Some points I had in mind:

* Pay more attention to docs during review
* Set an expectation level for docs accompanying a changeset
* Spend some cycles to review and update the existing docs
* Spend some more cycles to add new documentation
* Participate in prorams like Google Season of Docs

Quoted 4 lines

I'd appreciate a discussion about how we can improve the existing memory management documentation so that a reader can get a coherent view of it, what are the gaps (although they are too many), and what would be the best way to close these gaps.

Quoted 5 lines

I think we have four target audiences for mm documentation,

- Sysadmins/user space programmers who are trying to make their system perform well. They need documentation of the proc/sys/cmdline/... parameters that the MM pays attention to.

I'd say this part needs some love. Just a few weeks ago we've discussed how /proc/meminfo description is outdated and there are more examples. Besides, this is probably the most important part to keep up to date.

Quoted 10 lines

- Kernel programmers who are not (and do not wish to be) MM developers. Filesystem developers, device driver developers, networking developers. They need kernel-doc for our exported functions and advice for when to use and not use particular functions/flags.
- Programmers who want to "get started" in the MM area. They may or may not be familiar with Linux internals (perhaps they're moving from another kernel, perhaps their experience is with some other part of Linux; perhaps they have no experience at all). I think these people are probably well served by Mel's book, even if it is a few years old now.

Quoted 6 lines

Mel's book is definitely an excellent starting point, but I afraid its age is starting to show. As it was written mostly about 2.4 with some bits about 2.6, there are lot of details that are missing in the book. Some are probably less important because the concept didn't change much (e.g. page table management), but others have considerable effect on our code (e.g migration, compaction, THP).

IMO, updating Mel's book will be helpful. I thought about it but realized that without guidance I can't make good progress.

Quoted 9 lines
Quoted 4 lines

- Each other. The MM is huge these days, and I certainly don't understand how it all interacts with itself. I think we actually do a pretty good job of talking to each other and writing commit messages.

IMHO, there is a fifth group: arch developers. I think it is important to have documentation of what core mm expects from an architecture and what is the expected semantics of various low level functions architectures supply.

Quoted 6 lines
Quoted 3 lines

I think it's really the second group where we do the worst job of documentation, but I may be biased. It's certainly where I'm focusing my documentation efforts.

I'm not sure it's the worst, but certainly along with the first group it's the most important part.

I am interested to contribute to this if agree to take it forward.