Extend zImage format to support notes
RFC 1/1 Extend zImage format to support notes
Hello all,
Currently ELF is supporting notes (see include/linux/elfnote.h) which is used by Xen to know the feature supported by the kernel.
On ARM world, the zImage format is mainly used by the distribution. Rather than ELF, it's currently not possible to add notes on it.
==== Current zImage format ====
The header starts at offset 0x24:
uint32_t magic /* Magic number: 0x016f2818 */
uint32_t start /* absolute load/run zImage address */
uint32_t end /* zImage end address */==== Extension proposed to zImage format ===
The header stays compatible with the current format and starts at offset 0x24:
uint32_t magic0 /* Magic number : 0x16f2818 */
uint32_t start /* absolute load/run zImage address */
uint32_t end /* zImage end address */
uint32_t magic1 /* Magic number : "ARM\x32" */
uint32_t note_start /* Notes section offset */
uint32_t note_end /* Notes sections end */I was thinking to also add a version field in case we might want to extend the format in the future.
The notes section will contains a list of note (see description below).
==== Format of a note ====
The format is compatible with ELF note:
Name size : 4 bytes
Desc size : 4 bytes
Type : 4 bytes: identify the type of the note
Name : variable size, padded to 4 byte boundary
Desc : variable size, padded to 4 byte boundary : contains data related to the note type.
The "Name size", "Desc size" and "Type" fields are interpreted as an integer.
I wrote a quick patch to show what would be the modification on Linux side (see below).
Comments and questions are welcomed.
Sincerely yours,
arch/arm/boot/compressed/head.S +6 -0
@@ -133,6 +138,12 @@ start: .word 0x016f2818 @ Magic numbers to help the loader .word start @ absolute load/run zImage address .word _edata @ zImage end address + .byte 0x41 @ Magic number, "ARM\x32" + .byte 0x52 + .byte 0x4d + .byte 0x32 + .word _note_start @ Notes section offset + .word _note_end @ Notes section end THUMB( .thumb ) 1: ARM_BE8( setend be ) @ go BE8 if compiled for BE8
arch/arm/boot/compressed/vmlinux.lds.in +4 -0
@@ -52,6 +52,10 @@ SECTIONS .got : { *(.got) } _got_end = .; + _note_start = .; + .note : { *(.note*) } + _note_end = .; + /* ensure the zImage file size is always a multiple of 64 bits */ /* (without a dummy byte, ld just ignores the empty section) */ .pad : { BYTE(0); . = ALIGN(8); }
include/linux/elfnote.h +1 -1
@@ -39,7 +39,7 @@* ELFNOTE(XYZCo, 12, .long, 0xdeadbeef) */ #define ELFNOTE_START(name, type, flags) \ -.pushsection .note.name, flags, at note ; \ +.pushsection .note.name, flags,%note ; \ .balign 4 ; \ .long 2f - 1f /* namesz */ ; \ .long 4484f - 3f /* descsz */ ; \
Signature · 1 line
Julien Grall
Quoted 1 line · Julien Grall
Hello all,
Hi Julien,
Quoted 2 lines
Currently ELF is supporting notes (see
include/linux/elfnote.h) which is used by Xen to know the feature supported by the kernel.
What kind of features do you intend for Xen discover through this mechanism on ARM?
Can support for these not be negotiated at run-time (how does this work with other OSs under Xen)?
Quoted 12 lines
On ARM world, the zImage format is mainly used by the distribution. Rather than ELF, it's currently not possible to add notes on it.
==== Current zImage format ====
The header starts at offset
0x24:uint32_t magic /* Magic number: 0x016f2818 */ uint32_t start /* absolute load/run zImage address */ uint32_t end /* zImage end address */==== Extension proposed to zImage format ===
Quoted 7 lines
The header stays compatible with the current format and starts at offset
0x24:uint32_t magic0 /* Magic number : 0x16f2818 */ uint32_t start /* absolute load/run zImage address */ uint32_t end /* zImage end address */ uint32_t magic1 /* Magic number : "ARM\x32" */ uint32_t note_start /* Notes section offset */ uint32_t note_end /* Notes sections end */
On arm64 we seem to have a notes section but no pointer to it in the Image header. I assume you'd consider a notes section useful for 64-bit too?
I have some patches extending the header for arm64 which I intend to post shortly; if we're going to clash there and this is needed, perhaps I can fold in a similar update.
Cheers,
Mark.
Quoted 2 lines · Mark Rutland
On Mon, Apr 07, 2014 at 02:59:53PM +0100, Julien Grall wrote:
Hi Julien,
Hello Mark,
Quoted 5 lines
Quoted 2 lines
Currently ELF is supporting notes (see
include/linux/elfnote.h) which is used by Xen to know the feature supported by the kernel.What kind of features do you intend for Xen discover through this mechanism on ARM?
tl;dr: To know if DOM0 Linux will correctly work when IOMMU is enabled by Xen.
Actually Xen doesn't support IOMMU. To allow DMA-capable devices correctly work, we use an hybrid approach:
- 1:1 mapping for DOM0 memory
- Use swiotlb for DMA on a foreign page (i.e. page which belongs to
another domain).
For the latter, the swiotlb will return a physical address rather than an IPA.
I'm actually working to add SMMU support in Xen for ARM. Once Xen will enable the IOMMU, Linux will have to program DMA with IPA.
As the swiotlb is returning physical address, Linux will have to know whether the device is protected by IOMMU or not. For the former case, Linux shouldn't use swiotlb for this device (see
https://lkml.org/lkml/2014/2/20/351).
Since the Xen ABI is stable for ARM (since Linux 3.13 and Xen 4.4), we want to be able to boot Linux 3.13 on newer version of Xen. For this version and onward (i.e. as long as swiotlb is used by default in DOM0), Xen has to boot with IOMMU disabled.
With the zImage format extension, we will be able to add a note saying:
"I don't use swiotlb by default, you can turn on the IOMMU" once it's case.
Quoted 2 lines
Can support for these not be negotiated at run-time (how does this work with other OSs under Xen)?
We though about runtime approach (see the whole discussion here
https://patches.linaro.org/26034/), but it's not as extensible as notes. In fact, we might need more notes in the future when new feature will be added and requires a DOM0/guest side support.
On x86 world, the dom0/guest features is discovered via ELF notes. I think we should use the same way on ARM when zImage is used to boot DOM0.
Quoted 3 lines
On arm64 we seem to have a notes section but no pointer to it in the Image header. I assume you'd consider a notes section useful for 64-bit too?
Yes, we will need to same section on arm64.
Quoted 3 lines
I have some patches extending the header for arm64 which I intend to post shortly; if we're going to clash there and this is needed, perhaps I can fold in a similar update.
I didn't start to work on arm64 format extension. I wanted to wait for feedback on the zImage format extension for arm32.
I plan to use one (may two) field from the reserved ones.
Sincerely yours,
Signature · 1 line
Julien Grall
Quoted 21 lines · Julien Grall
Quoted 2 lines · Mark Rutland
On Mon, Apr 07, 2014 at 02:59:53PM +0100, Julien Grall wrote:
Hi Julien,Hello Mark,
Quoted 5 lines
Quoted 2 lines
Currently ELF is supporting notes (see
include/linux/elfnote.h) which is used by Xen to know the feature supported by the kernel.What kind of features do you intend for Xen discover through this mechanism on ARM?
tl;dr: To know if DOM0 Linux will correctly work when IOMMU is enabled by Xen.
Actually Xen doesn't support IOMMU. To allow DMA-capable devices correctly work, we use an hybrid approach:
- 1:1 mapping for DOM0 memory
- Use swiotlb for DMA on a foreign page (i.e. page which belongs to
another domain).
Quoted 5 lines
For the latter, the swiotlb will return a physical address rather than an IPA.
I'm actually working to add SMMU support in Xen for ARM. Once Xen will enable the IOMMU, Linux will have to program DMA with IPA.
I'm probably being thick here, but is there anything preventing those foreign pages from being idmapped into DOM0 (at least for the duration of the DMA access) such that there is a 1-1 PA<->IPA mapping?
That would mean that from the point of view of Linux, there's nothing to care about, as every address it knows it can deal with is a valid IPA. Existing kernels would function with the IOMMU(s) on.
I assume I'm missing something fairly obvious :)
Quoted 4 lines
As the swiotlb is returning physical address, Linux will have to know whether the device is protected by IOMMU or not. For the former case, Linux shouldn't use swiotlb for this device (see
https://lkml.org/lkml/2014/2/20/351).
For some reason I can only see the cover letter and not the patches when following that link.
Quoted 4 lines
Since the Xen ABI is stable for ARM (since Linux 3.13 and Xen 4.4), we want to be able to boot Linux 3.13 on newer version of Xen. For this version and onward (i.e. as long as swiotlb is used by default in DOM0), Xen has to boot with IOMMU disabled.
A mentioned above, I think I'm missing some part of the reasoning as to why Linux needs to know the difference.
Quoted 10 lines
With the zImage format extension, we will be able to add a note saying:
"I don't use swiotlb by default, you can turn on the IOMMU" once it's case.Quoted 2 lines
Can support for these not be negotiated at run-time (how does this work with other OSs under Xen)?
We though about runtime approach (see the whole discussion here
https://patches.linaro.org/26034/), but it's not as extensible as notes. In fact, we might need more notes in the future when new feature will be added and requires a DOM0/guest side support.
I can certainly see the usefulness of notes.
Quoted 9 lines
On x86 world, the dom0/guest features is discovered via ELF notes. I think we should use the same way on ARM when zImage is used to boot DOM0.
Quoted 3 lines
On arm64 we seem to have a notes section but no pointer to it in the Image header. I assume you'd consider a notes section useful for 64-bit too?
Yes, we will need to same section on arm64.
If it makes sense for this case then I can see why we would want it on arm64.
Quoted 6 lines
Quoted 3 lines
I have some patches extending the header for arm64 which I intend to post shortly; if we're going to clash there and this is needed, perhaps I can fold in a similar update.
I didn't start to work on arm64 format extension. I wanted to wait for feedback on the zImage format extension for arm32.
Certainly. I just wanted to make sure we weren't going to end up stepping on each others' toes whenever we try to get our patches merged.
Cheers,
Mark.
Hi Mark,
Quoted 10 lines · Mark Rutland
I'm probably being thick here, but is there anything preventing those foreign pages from being idmapped into DOM0 (at least for the duration of the DMA access) such that there is a 1-1 PA<->IPA mapping?
That would mean that from the point of view of Linux, there's nothing to care about, as every address it knows it can deal with is a valid IPA. Existing kernels would function with the IOMMU(s) on.
I assume I'm missing something fairly obvious :)
I didn't though about this solution. It works on midway only when the physical address size is 32 bits.
I didn't yet figure out why. I guess this is because I have a bug in my SMMU driver in Xen.
Quoted 6 lines
Quoted 4 lines
We though about runtime approach (see the whole discussion here
https://patches.linaro.org/26034/), but it's not as extensible as notes. In fact, we might need more notes in the future when new feature will be added and requires a DOM0/guest side support.I can certainly see the usefulness of notes.
[..]
Quoted 2 lines
If it makes sense for this case then I can see why we would want it on arm64.
With the solution of idmapping, the RFC seems irrelevant for now. But we may need to add notes in the future.
I'm wondering if it would be worth to agree on a solution now. Thus, we will be able to add a note quickly when it will be necessary.
Any thoughts?
Sincerely yours,
Signature · 1 line
Julien Grall