RFC: Adding arch-specific user ABI documentation in linux-man
Subsystems (2)
Hi all,
I considering trying to plug some gaps in the arch-specific ABI documentation in the linux man-pages, specifically for arm64 (and possibly arm, where compat means we have some overlap).
For arm64, there are now significant new extensions (Pointer authentication, SVE, MTE etc.) Currently there is some user-facing documentation mixed in with the kernel-facing documentation in the kernel tree, but this situation isn't ideal.
Do you have an opinion on where in the man-pages documentation should be added, and how to structure it?
Affected areas include:
* exec interface
* aux vector, hwcaps
* arch-specific signals
* signal frame
* mmap/mprotect extensions
* prctl calls
* ptrace quirks and extensions
* coredump contents
Not everything has an obvious home in an existing page, and adding specifics for every architecture could make some existing manpages very unwieldy.
I think for some arch features, we really need some "overview" pages
too: just documenting the low-level details is of limited value without some guide as to how to use them together.
Does the following sketch look reasonable?
* man7/arm64.7: new page: overview of arm64-specific ABI extensions
* man7/sve.7 (or man7/arm64-sve.7 or man7/sve.7arm64): new page:
overview of arm64 SVE ABI
* man2/arm64-ptrace.2 (or man2/ptrace.2arm64): new page:
arm64 ptrace extensions
* man2/mmap.2: extend with arm64-specific flags (only two flags, so we add them to the existing man page rather than creating a new one).
etc.
Ideally, I'd like to adopt a pattern that other arches can follow.
Thoughts?
Cheers
---Dave
AW: Adding arch-specific user ABI documentation in linux-man
In reply to a message not in the archiveHi Dave,
you are pointing to an (IMHO) interesting question.
How to document different CPUs ?
Given that an operating system should hide the different CPU's using CPU specific features should be used sparsely at best.
the easy part are adds-on like flags for prctrl etc. simply add it to the page.
Other things should go to a cpu specific pages (can of worms). The problem will be to keep that small but informative. I have no idea about the level of detail (and i have worked with a range of CPUs) that could be interesting for a programmer. An of cause every other CPU now needs also a page.
jm2c
wh
________________________________________
Von: [email protected] <[email protected]> im Auftrag von Dave Martin <[email protected]>
Gesendet: Montag, 4. Mai 2020 17:32:35
An: Michael Kerrisk
Cc: Catalin Marinas; Will Deacon; Vincenzo Frascino; [email protected]; [email protected]; linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
Betreff: RFC: Adding arch-specific user ABI documentation in linux-man
Hi all,
I considering trying to plug some gaps in the arch-specific ABI documentation in the linux man-pages, specifically for arm64 (and possibly arm, where compat means we have some overlap).
For arm64, there are now significant new extensions (Pointer authentication, SVE, MTE etc.) Currently there is some user-facing documentation mixed in with the kernel-facing documentation in the kernel tree, but this situation isn't ideal.
Do you have an opinion on where in the man-pages documentation should be added, and how to structure it?
Affected areas include:
* exec interface
* aux vector, hwcaps
* arch-specific signals
* signal frame
* mmap/mprotect extensions
* prctl calls
* ptrace quirks and extensions
* coredump contents
Not everything has an obvious home in an existing page, and adding specifics for every architecture could make some existing manpages very unwieldy.
I think for some arch features, we really need some "overview" pages
too: just documenting the low-level details is of limited value without some guide as to how to use them together.
Does the following sketch look reasonable?
* man7/arm64.7: new page: overview of arm64-specific ABI extensions
* man7/sve.7 (or man7/arm64-sve.7 or man7/sve.7arm64): new page:
overview of arm64 SVE ABI
* man2/arm64-ptrace.2 (or man2/ptrace.2arm64): new page:
arm64 ptrace extensions
* man2/mmap.2: extend with arm64-specific flags (only two flags, so we add them to the existing man page rather than creating a new one).
etc.
Ideally, I'd like to adopt a pattern that other arches can follow.
Thoughts?
Cheers
---Dave
Hi Dave,
Quoted 36 lines · Dave Martin
I considering trying to plug some gaps in the arch-specific ABI documentation in the linux man-pages, specifically for arm64 (and possibly arm, where compat means we have some overlap).
For arm64, there are now significant new extensions (Pointer authentication, SVE, MTE etc.) Currently there is some user-facing documentation mixed in with the kernel-facing documentation in the kernel tree, but this situation isn't ideal.
Do you have an opinion on where in the man-pages documentation should be added, and how to structure it?
Affected areas include:
* exec interface
* aux vector, hwcaps
* arch-specific signals
* signal frame
* mmap/mprotect extensions
* prctl calls
* ptrace quirks and extensions
* coredump contentsNot everything has an obvious home in an existing page, and adding specifics for every architecture could make some existing manpages very unwieldy.
I think for some arch features, we really need some "overview" pages
too: just documenting the low-level details is of limited value without some guide as to how to use them together.Does the following sketch look reasonable?
Quoted 7 lines
* man7/arm64.7: new page: overview of arm64-specific ABI extensions
* man7/sve.7 (or man7/arm64-sve.7 or man7/sve.7arm64): new page:
overview of arm64 SVE ABI* man2/arm64-ptrace.2 (or man2/ptrace.2arm64): new page:
arm64 ptrace extensions
Michael has been nagging me on and off about that for, what, 10 years now? I would therefore be very much in favour of having our ptrace extensions documented!
We could even put this stuff under Documentation/arm64/man/ if it's deemed too CPU-specific for the man-pages project, but my preference would still be for it to be hosted there alongside all the other man pages.
Will
Re: Adding arch-specific user ABI documentation in linux-man
In reply to a message not in the archiveQuoted 5 lines · Walter Harms
Hi Dave,
you are pointing to an (IMHO) interesting question.
How to document different CPUs ?
Given that an operating system should hide the different CPU's using CPU specific features should be used sparsely at best.
Agreed! But there are real situations where cpu specifics can't be avoided, and having documentation will help people to use those features correctly.
Quoted 1 line
the easy part are adds-on like flags for prctrl etc. simply add it to the page.
For prctl, that makes sense (it's a jumble of arch specifics already).
But would it be considered a problem if the ptrace page, say, becomes 80% arch-specific stuff? That page is cumbersome enough already...
Quoted 4 lines
Other things should go to a cpu specific pages (can of worms). The problem will be to keep that small but informative. I have no idea about the level of detail (and i have worked with a range of CPUs) that could be interesting for a programmer. An of cause every other CPU now needs also a page.
Fair enough.
Cheers
---Dave
Quoted 47 lines · Will Deacon
Hi Dave,
Quoted 36 lines · Dave Martin
I considering trying to plug some gaps in the arch-specific ABI documentation in the linux man-pages, specifically for arm64 (and possibly arm, where compat means we have some overlap).
For arm64, there are now significant new extensions (Pointer authentication, SVE, MTE etc.) Currently there is some user-facing documentation mixed in with the kernel-facing documentation in the kernel tree, but this situation isn't ideal.
Do you have an opinion on where in the man-pages documentation should be added, and how to structure it?
Affected areas include:
* exec interface
* aux vector, hwcaps
* arch-specific signals
* signal frame
* mmap/mprotect extensions
* prctl calls
* ptrace quirks and extensions
* coredump contentsNot everything has an obvious home in an existing page, and adding specifics for every architecture could make some existing manpages very unwieldy.
I think for some arch features, we really need some "overview" pages
too: just documenting the low-level details is of limited value without some guide as to how to use them together.Does the following sketch look reasonable?
Quoted 7 lines
* man7/arm64.7: new page: overview of arm64-specific ABI extensions
* man7/sve.7 (or man7/arm64-sve.7 or man7/sve.7arm64): new page:
overview of arm64 SVE ABI* man2/arm64-ptrace.2 (or man2/ptrace.2arm64): new page:
arm64 ptrace extensions
Quoted 7 lines
Michael has been nagging me on and off about that for, what, 10 years now? I would therefore be very much in favour of having our ptrace extensions documented!
We could even put this stuff under
Documentation/arm64/man/if it's deemed too CPU-specific for the man-pages project, but my preference would still be for it to be hosted there alongside all the other man pages.
Heh, perhaps we could build that into the kernel and mount it somewhere.
Seriously though,
I guess I can start off with straightforward small things for which the documentation has an obvious home (like prctls[*]) and then move on to the bigger stuff like ptrace.
If people start shouting about a page getting too big or messy I can try to split it up.
Make sense?
Cheers
---Dave
[*] "straightforward" was a joke, obviously
Quoted 52 lines · Dave Martin
Quoted 51 lines · Will Deacon
Hi Dave,
Quoted 36 lines · Dave Martin
I considering trying to plug some gaps in the arch-specific ABI documentation in the linux man-pages, specifically for arm64 (and possibly arm, where compat means we have some overlap).
For arm64, there are now significant new extensions (Pointer authentication, SVE, MTE etc.) Currently there is some user-facing documentation mixed in with the kernel-facing documentation in the kernel tree, but this situation isn't ideal.
Do you have an opinion on where in the man-pages documentation should be added, and how to structure it?
Affected areas include:
* exec interface
* aux vector, hwcaps
* arch-specific signals
* signal frame
* mmap/mprotect extensions
* prctl calls
* ptrace quirks and extensions
* coredump contentsNot everything has an obvious home in an existing page, and adding specifics for every architecture could make some existing manpages very unwieldy.
I think for some arch features, we really need some "overview" pages
too: just documenting the low-level details is of limited value without some guide as to how to use them together.Does the following sketch look reasonable?
Quoted 7 lines
* man7/arm64.7: new page: overview of arm64-specific ABI extensions
* man7/sve.7 (or man7/arm64-sve.7 or man7/sve.7arm64): new page:
overview of arm64 SVE ABI* man2/arm64-ptrace.2 (or man2/ptrace.2arm64): new page:
arm64 ptrace extensionsMichael has been nagging me on and off about that for, what, 10 years now? I would therefore be very much in favour of having our ptrace extensions documented!
Quoted 5 lines
Quoted 3 lines
We could even put this stuff under
Documentation/arm64/man/if it's deemed too CPU-specific for the man-pages project, but my preference would still be for it to be hosted there alongside all the other man pages.Heh, perhaps we could build that into the kernel and mount it somewhere.
Argh, don't joke too loudly about things like that!
Quoted 10 lines
Seriously though,
I guess I can start off with straightforward small things for which the documentation has an obvious home (like prctls[*]) and then move on to the bigger stuff like ptrace.
If people start shouting about a page getting too big or messy I can try to split it up.
Make sense?
Sure, works for me.
Will
Quoted 7 lines · Will Deacon
Michael has been nagging me on and off about that for, what, 10 years now? I would therefore be very much in favour of having our ptrace extensions documented!
We could even put this stuff under
Documentation/arm64/man/if it's deemed too CPU-specific for the man-pages project, but my preference would still be for it to be hosted there alongside all the other man pages.
Stuffing random things into the kernel tree is painful for some people.
For example, if you cross-build your kernel, then the stuff in the tools/ subdirectory is totally useless (I think everything except perf) because you can't build it.
Let's stop making the mistake of constantly shoving stuff into the kernel source tree.
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTC broadband for 0.8mile line in suburbia: sync at 10.2Mbps down 587kbps up
Quoted 9 lines · Russell King - ARM Linux admin
Quoted 7 lines · Will Deacon
Michael has been nagging me on and off about that for, what, 10 years now? I would therefore be very much in favour of having our ptrace extensions documented!
We could even put this stuff under
Documentation/arm64/man/if it's deemed too CPU-specific for the man-pages project, but my preference would still be for it to be hosted there alongside all the other man pages.
Quoted 8 lines
Stuffing random things into the kernel tree is painful for some people.
For example, if you cross-build your kernel, then the stuff in the
tools/subdirectory is totally useless (I think everything except perf) because you can't build it.Let's stop making the mistake of constantly shoving stuff into the kernel source tree.
For userspace tools, I'm inclined to agree, but this is just documentation so it shouldn't cause any issues with cross building. But to be clear: I'd still prefer it to be part of the man-pages project, and would only consider it for inclusion in the kernel tree if it was rejected for being too CPU-specific.
Will
Quoted 16 lines · Will Deacon
Quoted 15 lines · Russell King - ARM Linux admin
Quoted 7 lines · Will Deacon
Michael has been nagging me on and off about that for, what, 10 years now? I would therefore be very much in favour of having our ptrace extensions documented!
We could even put this stuff under
Documentation/arm64/man/if it's deemed too CPU-specific for the man-pages project, but my preference would still be for it to be hosted there alongside all the other man pages.Stuffing random things into the kernel tree is painful for some people.
For example, if you cross-build your kernel, then the stuff in the
tools/subdirectory is totally useless (I think everything except perf) because you can't build it.
Quoted 8 lines
Quoted 2 lines
Let's stop making the mistake of constantly shoving stuff into the kernel source tree.
For userspace tools, I'm inclined to agree, but this is just documentation so it shouldn't cause any issues with cross building. But to be clear: I'd still prefer it to be part of the man-pages project, and would only consider it for inclusion in the kernel tree if it was rejected for being too CPU-specific.
I don't think that should be a concern; the man-pages project already contains documentation that is specific to kernel versions, including documentation for interfaces that are architecture specific (such as prctl is a big one, ptrace to a lesser extent.) syscall(2) contains a whole bunch of architecture stuff about the calling convention for syscalls.
Interestingly, I notice that syscall(2) is wrong for arm/OABI. I am not surprised, because that documentation never came my way, and I am the author of the kernel's OABI syscall interface.
It claims:
arch/ABI arg1 arg2 arg3 arg4 arg5 arg6 arg7 Notes
──────────────────────────────────────────────────────────────
arm/OABI a1 a2 a3 a4 v1 v2 v3
whereas, at the time I invented it, I decided that it shall pass arguments in r0 to r6. That's r0 to r6. Not the APCS register names that this document claims. Not everything in OABI is APCS, as illustrated here - APCS passes the first four arguments in a1 to a4, and then the rest on the stack. The OABI syscall interface doesn't do that.
I guess that's what happens when someone else writes interface documentation and doesn't bother to get it reviewed by those who created the interface in the first place.
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTC broadband for 0.8mile line in suburbia: sync at 10.2Mbps down 587kbps up
Hello Dave, et al.
Quoted 5 lines · Dave Martin
Hi all,
I considering trying to plug some gaps in the arch-specific ABI documentation in the linux man-pages, specifically for arm64 (and possibly arm, where compat means we have some overlap).
Sounds good to me.
Quoted 5 lines
For arm64, there are now significant new extensions (Pointer authentication, SVE, MTE etc.) Currently there is some user-facing documentation mixed in with the kernel-facing documentation in the kernel tree, but this situation isn't ideal.
Quoted 7 lines
Do you have an opinion on where in the man-pages documentation should be added, and how to structure it?
Affected areas include:
* exec interface
Not sure what the details are here, so I have no opinion yet. But probably, as additions to execve(2).
Quoted 1 line
* aux vector, hwcaps
==> getauxval(3)
Quoted 2 lines
* arch-specific signals
* signal frame
Not sure what the details are here, so I have no opinion yet.
Quoted 1 line
* mmap/mprotect extensions
See below.
Quoted 1 line
* prctl calls
As additions in prctl(2) would be fine, I think.
Quoted 1 line
* ptrace quirks and extensions
See below.
Quoted 1 line
* coredump contents
Not sure what the details are here, so I have no opinion yet. Possibly as additions to core(5).
Quoted 1 line
Not everything has an obvious home in an existing page,
Yes.
Quoted 3 lines
and adding
specifics for every architecture could make some existing manpages very unwieldy.
Still, I think it's worth adding details, especially for widely used architectures.
Quoted 8 lines
I think for some arch features, we really need some "overview" pages
too: just documenting the low-level details is of limited value without some guide as to how to use them together.Does the following sketch look reasonable?
* man7/arm64.7: new page: overview of arm64-specific ABI extensions
I'm a little unclear on what would land in here, but sounds reasonable in principle.
Quoted 2 lines
* man7/sve.7 (or man7/arm64-sve.7 or man7/sve.7arm64): new page:
overview of arm64 SVE ABI
Sounds reasonable to me.
Quoted 2 lines
* man2/arm64-ptrace.2 (or man2/ptrace.2arm64): new page:
arm64 ptrace extensions
I think maybe better is: ptrace-arm64.2
I'm agnostic about whether there should be a new page, or whether these should be added to ptrace(2). But, we could start with the idea of a new page.
Quoted 2 lines
* man2/mmap.2: extend with arm64-specific flags (only two flags, so we add them to the existing man page rather than creating a new one).
Sounds good to me
Quoted 4 lines
etc.
Ideally, I'd like to adopt a pattern that other arches can follow.
Well, if they do follow. Arch-specific documentation is woefully thin at the moment. I'm not going to worry too much about the right pattern (don't let perfect get in the way of good, yadda, yadda), until I get so much arch-specific documentation that some refactoring may be required. (I'm not going to hold my breath waiting for that to happen ;-).)
Cheers,
Michael
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/
Quoted 4 lines
Quoted 2 lines
* man2/arm64-ptrace.2 (or man2/ptrace.2arm64): new page:
arm64 ptrace extensionsMichael has been nagging me on and off about that for, what, 10 years now?
Perhaps not that long, but a while now ;-).
Quoted 6 lines
I would therefore be very much in favour of having our ptrace extensions documented!
We could even put this stuff under
Documentation/arm64/man/if it's deemed too CPU-specific for the man-pages project, but my preference would still be for it to be hosted there alongside all the other man pages.
Agreed; manual pages is I think a better place.
Cheers,
Michael
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/
Quoted 4 lines
If people start shouting about a page getting too big or messy I can try to split it up.
Make sense?
Yes.
Thanks,
Michael
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/
Quoted 10 lines · Russell King - ARM Linux admin
Quoted 7 lines · Will Deacon
Michael has been nagging me on and off about that for, what, 10 years now? I would therefore be very much in favour of having our ptrace extensions documented!
We could even put this stuff under
Documentation/arm64/man/if it's deemed too CPU-specific for the man-pages project, but my preference would still be for it to be hosted there alongside all the other man pages.Stuffing random things into the kernel tree is painful for some people.
Yes, and too often not easily noticeable for user-space programmers.
Quoted 7 lines
For example, if you cross-build your kernel, then the stuff in the
tools/subdirectory is totally useless (I think everything except perf) because you can't build it.Let's stop making the mistake of constantly shoving stuff into the kernel source tree.
Agrred.
Thanks,
Michael
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/
Quoted 20 lines · Michael Kerrisk (man-pages)
Hello Dave, et al.
Quoted 5 lines · Dave Martin
Hi all,
I considering trying to plug some gaps in the arch-specific ABI documentation in the linux man-pages, specifically for arm64 (and possibly arm, where compat means we have some overlap).
Sounds good to me.
Quoted 9 lines
For arm64, there are now significant new extensions (Pointer authentication, SVE, MTE etc.) Currently there is some user-facing documentation mixed in with the kernel-facing documentation in the kernel tree, but this situation isn't ideal.
Do you have an opinion on where in the man-pages documentation should be added, and how to structure it?
Quoted 6 lines
Quoted 3 lines
Affected areas include:
* exec interface
Not sure what the details are here, so I have no opinion yet. But probably, as additions to execve(2).
Having looked at that page again, probably yes.
This would include things like how arch-specific registers get reset, and how various thread properties are reset/inherited.
Possibly, much of this would be documented by cross-referencing other pages.
Quoted 4 lines
Quoted 2 lines
* aux vector, hwcaps
==> getauxval(3)
Ah, yes. I keep forgetting that because it's provided by libc.
There are a few things missing, but we can add them in there.
Quoted 4 lines
Quoted 2 lines
* arch-specific signals
* signal frameNot sure what the details are here, so I have no opinion yet.
Stuff like architecture-specific si_codes and how they are used.
Signal handlers that poke about in the signal frame need to know how it is structured, and for arm64 this is now quite complex. I want to document some example code for parsing it somewhere, and I feel that will be too much noise for signal(7).
I guess I can try to write something and than we can figure out where to put it.
Quoted 9 lines
Quoted 2 lines
* mmap/mprotect extensions
See below.
Quoted 1 line
* prctl calls
As additions in prctl(2) would be fine, I think.
Quoted 8 lines
Quoted 1 line
* ptrace quirks and extensions
See below.
Quoted 1 line
* coredump contents
Not sure what the details are here, so I have no opinion yet. Possibly as additions to core(5).
Things like arch-specific regsets. Some of that can probabably live in the ptrace page, with cross-references from core(5).
Quoted 10 lines
Quoted 1 line
Not everything has an obvious home in an existing page,
Yes.
Quoted 3 lines
and adding
specifics for every architecture could make some existing manpages very unwieldy.Still, I think it's worth adding details, especially for widely used architectures.
Agreed. If there's a lot of explanation required for somethings which feels out of scope for the core page, then I might still be tempted to push some of that out into a more arch-specific page and cross-reference it. We can play that by ear, though.
Quoted 11 lines
Quoted 8 lines
I think for some arch features, we really need some "overview" pages
too: just documenting the low-level details is of limited value without some guide as to how to use them together.Does the following sketch look reasonable?
* man7/arm64.7: new page: overview of arm64-specific ABI extensions
I'm a little unclear on what would land in here, but sounds reasonable in principle.
I probably won't attempt to add such a page until I find a use for it.
I mainly intended this as an overview of what areas are impacted by the architecture and which other pages to go look at.
Quoted 9 lines
Quoted 2 lines
* man7/sve.7 (or man7/arm64-sve.7 or man7/sve.7arm64): new page:
overview of arm64 SVE ABISounds reasonable to me.
Quoted 2 lines
* man2/arm64-ptrace.2 (or man2/ptrace.2arm64): new page:
arm64 ptrace extensionsI think maybe better is: ptrace-arm64.2
Agreed. That's probably easier to for people to place in their mental map.
Quoted 13 lines
I'm agnostic about whether there should be a new page, or whether these should be added to ptrace(2). But, we could start with the idea of a new page.
Quoted 2 lines
* man2/mmap.2: extend with arm64-specific flags (only two flags, so we add them to the existing man page rather than creating a new one).
Sounds good to me
Quoted 3 lines
etc.
Quoted 8 lines
Quoted 1 line
Ideally, I'd like to adopt a pattern that other arches can follow.
Well, if they do follow. Arch-specific documentation is woefully thin at the moment. I'm not going to worry too much about the right pattern (don't let perfect get in the way of good, yadda, yadda), until I get so much arch-specific documentation that some refactoring may be required. (I'm not going to hold my breath waiting for that to happen ;-).)
Sure. I don't want to do something obviously broken, but it sounds like you're reasonably happy with my approach. In any case I'll start with the simple stuff.
Where it's easy to do so, I may try to trawl up some undocumented non- ARM stuff, but I'm not the expert there...
Cheers
---Dave