Skip to contentSkip to conversation

Re: [Fwd: RE: elf header bits for page migration support......]

Fwd: RE: elf header bits for page migration support...... Jack Carter 3 participants 6 messages May 15, 2005 – May 16, 2005

lore
Share & tools
lore
#0 lore raw
Quoted 27 lines · Ray Bryant

2/2

-------- Original Message --------
Subject: RE: elf header bits for page migration support......
Date: Fri, 13 May 2005 10:54:11 -0700
From: Luck, Tony <[email protected]>
To: Ray Bryant <[email protected]>
CC: <[email protected]>

Quoted 5 lines

The alternative being suggested is to mark the elf header of object files
that should only have shared pages migrated, or that should not be migrated
(e. g. you don't want to migrate /bin/csh all the time.)

No ... I'd like you to replicate shared pages (of /bin/bash ... I personally don't
care what happens to the pages of /bin/csh :-)

Quoted 5 lines

Can I use some bits (I need 2) of p_flags of Elf64_Phdr for this purpose?
If so, which bits?

If not which other fields can I use?

Quoted 9 lines

I think H.J's suggestion of an extra section makes sense ... then you can go
wild with as many bits as you can dream up uses for, rather than constraining
yourself to squeeze into just 2 bits.

-Tony

The fun part of this is that if you introduce a new section and a new segment to point to it (only segments exist at load time) then you need to really muck with the bits because you will be growing the program header (segment table) and the section header as well as putting in a new section.

If this information is to be used by rld it will need to be in some element of the bits that will be loadable and that means the new section needs to be either contiguous to a currently marked (PT_LOAD) segment with it's size increase due to the new section or the new section needs to be in it's own segment which means increasing the number of PT_LOAD segment entries.

In either case whether you have a virtual segment pointing to the new section (remember rld can't see it if it isn't pointed to from somewhere in a loadable segment) or a new loadable segment, you will need to grow the program header (segment table) and there may not be enough space for that without overlapping other elements of the executable.

Of course this doesn't go into the subtleties like whether or not linux kernels and rlds can handle more than 2 loadable segments and if they will melt down when encountering new unknown virtual segments.

You could move the program header and grow it, but some kernels and rlds assume that the program header is in the text segment right after the ELF header. They shouldn't, but they do.

None of this is not very hard if we do it at link time. It gets messy only after because much of the relocation information is lost and people seem to like to be clever in their knowlege of where things begin and end and write code based on those assumptions whether or not those positions have been garaunteed by the ABI.

Just wanted to add the joy of sunshine to this discussion.

Jack

#1 lore raw
Quoted 44 lines · Jack Carter
Quoted 27 lines · Ray Bryant

2/2

-------- Original Message --------
Subject: RE: elf header bits for page migration support......
Date: Fri, 13 May 2005 10:54:11 -0700
From: Luck, Tony <[email protected]>
To: Ray Bryant <[email protected]>
CC: <[email protected]>

Quoted 5 lines

The alternative being suggested is to mark the elf header of object files
that should only have shared pages migrated, or that should not be migrated
(e. g. you don't want to migrate /bin/csh all the time.)

No ... I'd like you to replicate shared pages (of /bin/bash ... I personally don't
care what happens to the pages of /bin/csh :-)

Quoted 5 lines

Can I use some bits (I need 2) of p_flags of Elf64_Phdr for this purpose?
If so, which bits?

If not which other fields can I use?

Quoted 9 lines

I think H.J's suggestion of an extra section makes sense ... then you can go
wild with as many bits as you can dream up uses for, rather than constraining
yourself to squeeze into just 2 bits.

-Tony

The fun part of this is that if you introduce a new section and a new segment to point to it (only segments exist at load time) then you need to really muck with the bits because you will be growing the program header (segment table) and the section header as well as putting in a new section.

Quoted 6 lines

If this information is to be used by rld it will need to be in some element of the bits that will be loadable and that means the new section needs to be either contiguous to a currently marked (PT_LOAD) segment with it's size increase due to the new section or the new section needs to be in it's own segment which means increasing the number of PT_LOAD segment entries.

I am not sure I understand what you are saying. As far as I know, you don't need PT_LOAD for it. The dynamic linker will handle it in program header just fine.

H.J.

#2 lore raw
In reply to Jack Carter
Quoted 64 lines · H. J. Lu
Quoted 52 lines · Jack Carter
Quoted 31 lines · Ray Bryant

2/2

-------- Original Message --------
Subject: RE: elf header bits for page migration support......
Date: Fri, 13 May 2005 10:54:11 -0700
From: Luck, Tony <[email protected]>
To: Ray Bryant <[email protected]>
CC: <[email protected]>

Quoted 7 lines

The alternative being suggested is to mark the elf header of object files
that should only have shared pages migrated, or that should not be migrated
(e. g. you don't want to migrate /bin/csh all the time.)

No ... I'd like you to replicate shared pages (of /bin/bash ... I personally don't
care what happens to the pages of /bin/csh :-)

Quoted 7 lines

Can I use some bits (I need 2) of p_flags of Elf64_Phdr for this purpose?
If so, which bits?

If not which other fields can I use?

Quoted 11 lines

I think H.J's suggestion of an extra section makes sense ... then you can go
wild with as many bits as you can dream up uses for, rather than constraining
yourself to squeeze into just 2 bits.

-Tony

The fun part of this is that if you introduce a new section and a new segment to point to it (only segments exist at load time) then you need to really muck with the bits because you will be growing the program header (segment table) and the section header as well as putting in a new section.

Quoted 8 lines

If this information is to be used by rld it will need to be in some element of the bits that will be loadable and that means the new section needs to be either contiguous to a currently marked (PT_LOAD) segment with it's size increase due to the new section or the new section needs to be in it's own segment which means increasing the number of PT_LOAD segment entries.

Quoted 8 lines

I am not sure I understand what you are saying. As far as I know, you don't need PT_LOAD for it. The dynamic linker will handle it in program header just fine.

H.J.

I was trying to give a number of scenarios, one of which was that the new "section" that was suggested, would be in it's own segment because that way it could be put at the end of the file and not perturb the rest of the executable.

In any case, the situation where you create a new area of data that rld needs to read has to be pointed to. Whether this is a new virtual segment or a new entry in the .dynamic section which by the way would probably be a better place for it.

In either case you will be growing things that may not have room to grow and that's where the problems start to happen.

This not to say it's impossible to do. I did this and much more with pixie on IRIX executables and dsos. It is just that there are a lot of gottchas to contend with and I'm not sure how you content with them in the open source world unless you get it firmly established in the ABI.

Regards,

Jack

#3 lore raw
In reply to Jack Carter
Quoted 84 lines · Jack Carter
Quoted 64 lines · H. J. Lu
Quoted 52 lines · Jack Carter
Quoted 31 lines · Ray Bryant

2/2

-------- Original Message --------
Subject: RE: elf header bits for page migration support......
Date: Fri, 13 May 2005 10:54:11 -0700
From: Luck, Tony <[email protected]>
To: Ray Bryant <[email protected]>
CC: <[email protected]>

Quoted 7 lines

The alternative being suggested is to mark the elf header of object files
that should only have shared pages migrated, or that should not be migrated
(e. g. you don't want to migrate /bin/csh all the time.)

No ... I'd like you to replicate shared pages (of /bin/bash ... I personally don't
care what happens to the pages of /bin/csh :-)

Quoted 7 lines

Can I use some bits (I need 2) of p_flags of Elf64_Phdr for this purpose?
If so, which bits?

If not which other fields can I use?

Quoted 11 lines

I think H.J's suggestion of an extra section makes sense ... then you can go
wild with as many bits as you can dream up uses for, rather than constraining
yourself to squeeze into just 2 bits.

-Tony

The fun part of this is that if you introduce a new section and a new segment to point to it (only segments exist at load time) then you need to really muck with the bits because you will be growing the program header (segment table) and the section header as well as putting in a new section.

Quoted 8 lines

If this information is to be used by rld it will need to be in some element of the bits that will be loadable and that means the new section needs to be either contiguous to a currently marked (PT_LOAD) segment with it's size increase due to the new section or the new section needs to be in it's own segment which means increasing the number of PT_LOAD segment entries.

Quoted 8 lines

I am not sure I understand what you are saying. As far as I know, you don't need PT_LOAD for it. The dynamic linker will handle it in program header just fine.

H.J.

I was trying to give a number of scenarios, one of which was that the new "section" that was suggested, would be in it's own segment because that way it could be put at the end of the file and not perturb the rest of the executable.

In any case, the situation where you create a new area of data that rld needs to read has to be pointed to. Whether this is a new virtual segment or a new entry in the .dynamic section which by the way would probably be a better place for it.

Quoted 8 lines

In either case you will be growing things that may not have room to grow and that's where the problems start to happen.

This not to say it's impossible to do. I did this and much more with pixie on IRIX executables and dsos. It is just that there are a lot of gottchas to contend with and I'm not sure how you content with them in the open source world unless you get it firmly established in the ABI.

I didn't follow the whole thread very closely. But I can say

1. We should use a dummy segement instead of bits in ELF header.
2. ld.so has no problems with more than one PT_LOAD segment. Most of binaries/DSOs have 2: one is readonly and executable, the other is read and write.

If you want to modify ld.so to support your scheme, you can mark it any PT_XXX you want. If you don't need/want to modify ld.so, you can use a new PT_XXX and deal it in the user code. Ld.so will ignore the unknow PT_XXX. The static linker can provide _start/_end symbols for the new PT_XXX segment.

H.J.

#4 lore raw
In reply to Jack Carter
Quoted 8 lines · H. J. Lu

I didn't follow the whole thread very closely. But I can say

1. We should use a dummy segement instead of bits in ELF header.
2. ld.so has no problems with more than one PT_LOAD segment. Most of binaries/DSOs have 2: one is readonly and executable, the other is read and write.

Quoted 9 lines

If you want to modify ld.so to support your scheme, you can mark it any PT_XXX you want. If you don't need/want to modify ld.so, you can use a new PT_XXX and deal it in the user code. Ld.so will ignore the unknow PT_XXX. The static linker can provide _start/_end symbols for the new PT_XXX segment.

H.J.

If we use a new segment at the end of the file (Jack had suggested this), then don't we have to scan the entire file before we find out that the executable has this special section? That makes it hard, because we call mmap() in ld.so and execve() [inside the kernel] as each segment is encountered. So we need to know before that mmap() call is made whether or not the migration attribute needs to be added to that mmap'd region.

Also, I suppose it possible to use the linker to add the special section some other place in the file (?). We would have to then relink every application in the system that could be part of a migratable program. I'm not sure how feasble that might be, but I can investigate if our customer can deal with this.

Signature · 9 lines
Best Regards,
Ray
-----------------------------------------------
                   Ray Bryant
512-453-9679 (work)         512-507-7807 (cell)
[email protected]             [email protected]
The box said: "Requires Windows 98 or better",
            so I installed Linux.
-----------------------------------------------
#5 lore raw
In reply to Jack Carter
Quoted 20 lines · Ray Bryant
Quoted 8 lines · H. J. Lu

I didn't follow the whole thread very closely. But I can say

1. We should use a dummy segement instead of bits in ELF header.
2. ld.so has no problems with more than one PT_LOAD segment. Most of binaries/DSOs have 2: one is readonly and executable, the other is read and write.

Quoted 9 lines

If you want to modify ld.so to support your scheme, you can mark it any PT_XXX you want. If you don't need/want to modify ld.so, you can use a new PT_XXX and deal it in the user code. Ld.so will ignore the unknow PT_XXX. The static linker can provide _start/_end symbols for the new PT_XXX segment.

H.J.

Quoted 6 lines

If we use a new segment at the end of the file (Jack had suggested this), then don't we have to scan the entire file before we find out that the executable has this special section? That makes it hard, because we call mmap() in ld.so and execve() [inside the kernel] as each segment is encountered. So we need to know before that mmap() call is made whether or not the migration attribute needs to be added to that mmap'd region.

All necessary information of any segment should be in the ELF program header. Looking it up is very straight forward in kernel and ld.so.

H.J.