RW and executable hole in page tables on x86_64
Hi,
I've noticed there's a chunk of kernel memory still marked RW and x. See 0xffffffff82956000 below...
---[ High Kernel Mapping ]---
0xffffffff80000000-0xffffffff81000000 16M pmd 0xffffffff81000000-0xffffffff81a00000 10M ro PSE GLB x pmd 0xffffffff81a00000-0xffffffff81e00000 4M ro PSE GLB NX pmd 0xffffffff81e00000-0xffffffff82200000 4M RW GLB NX pte 0xffffffff82200000-0xffffffff82800000 6M RW PSE GLB NX pmd 0xffffffff82800000-0xffffffff82956000 1368K RW GLB NX pte 0xffffffff82956000-0xffffffff82a00000 680K RW GLB x pte
0xffffffff82a00000-0xffffffffa0000000 470M pmd
HPA looked at it for a bit, but it wasn't obvious what was going on. It's after the end of bss. I do note that the two adjacent regions add up to 2MiB. Is this some kind of leftover mapping? What is this region? Is there a sensible place to clean it up?
Thanks,
-Kees
Signature · 1 line
Kees Cook @outflux.net
Quoted 13 lines · Kees Cook
Hi,
I've noticed there's a chunk of kernel memory still marked RW and x. See
0xffffffff82956000below...---[ High Kernel Mapping ]---
0xffffffff80000000-0xffffffff81000000 16M pmd 0xffffffff81000000-0xffffffff81a00000 10M ro PSE GLB x pmd 0xffffffff81a00000-0xffffffff81e00000 4M ro PSE GLB NX pmd 0xffffffff81e00000-0xffffffff82200000 4M RW GLB NX pte 0xffffffff82200000-0xffffffff82800000 6M RW PSE GLB NX pmd 0xffffffff82800000-0xffffffff82956000 1368K RW GLB NX pte 0xffffffff82956000-0xffffffff82a00000 680K RW GLB x pte
Quoted 7 lines
0xffffffff82a00000-0xffffffffa0000000 470M pmd
HPA looked at it for a bit, but it wasn't obvious what was going on. It's after the end of bss. I do note that the two adjacent regions add up to 2MiB. Is this some kind of leftover mapping? What is this region? Is there a sensible place to clean it up?
It looks to be what is left after the 2 MB page for bss is broken up. It doesn't mean it isn't broken, though.
-hpa
Continuing a thread from a year ago...
Quoted 16 lines · H. Peter Anvin
Quoted 15 lines · Kees Cook
Hi,
I've noticed there's a chunk of kernel memory still marked RW and x. See
0xffffffff82956000below...---[ High Kernel Mapping ]---
0xffffffff80000000-0xffffffff81000000 16M pmd 0xffffffff81000000-0xffffffff81a00000 10M ro PSE GLB x pmd 0xffffffff81a00000-0xffffffff81e00000 4M ro PSE GLB NX pmd 0xffffffff81e00000-0xffffffff82200000 4M RW GLB NX pte 0xffffffff82200000-0xffffffff82800000 6M RW PSE GLB NX pmd 0xffffffff82800000-0xffffffff82956000 1368K RW GLB NX pte 0xffffffff82956000-0xffffffff82a00000 680K RW GLB x pte0xffffffff82a00000-0xffffffffa0000000 470M pmd
Quoted 8 lines
Quoted 5 lines
HPA looked at it for a bit, but it wasn't obvious what was going on. It's after the end of bss. I do note that the two adjacent regions add up to 2MiB. Is this some kind of leftover mapping? What is this region? Is there a sensible place to clean it up?
It looks to be what is left after the 2 MB page for bss is broken up. It doesn't mean it isn't broken, though.
It looks like the problem still exists:
---[ High Kernel Mapping ]---
0xffffffff80000000-0xffffffff9ca00000 458M pmd 0xffffffff9ca00000-0xffffffff9d200000 8M ro PSE GLB x pmd 0xffffffff9d200000-0xffffffff9d3f3000 1996K ro GLB x pte 0xffffffff9d3f3000-0xffffffff9d400000 52K ro x pte 0xffffffff9d400000-0xffffffff9d600000 2M ro PSE GLB NX pmd 0xffffffff9d600000-0xffffffff9d7e8000 1952K ro GLB NX pte 0xffffffff9d7e8000-0xffffffff9d800000 96K ro NX pte 0xffffffff9d800000-0xffffffff9d8ff000 1020K RW GLB NX pte 0xffffffff9d8ff000-0xffffffff9da2d000 1208K RW NX pte 0xffffffff9da2d000-0xffffffff9dc00000 1868K RW GLB NX pte 0xffffffff9dc00000-0xffffffff9e600000 10M RW PSE GLB NX pmd 0xffffffff9e600000-0xffffffff9e7f5000 2004K RW GLB NX pte 0xffffffff9e7f5000-0xffffffff9e800000 44K RW GLB x pte
0xffffffff9e800000-0xffffffffc0000000 536M pmd
Still seems to be the bss getting broken up. What is this left-over memory used for? Any pointers to where it happens? I'd really like to kill this area.
-Kees
Signature · 1 line
Kees Cook @outflux.net
Quoted 44 lines · Kees Cook
Continuing a thread from a year ago...
Quoted 16 lines · H. Peter Anvin
Quoted 15 lines · Kees Cook
Hi,
I've noticed there's a chunk of kernel memory still marked RW and x. See
0xffffffff82956000below...---[ High Kernel Mapping ]---
0xffffffff80000000-0xffffffff81000000 16M pmd 0xffffffff81000000-0xffffffff81a00000 10M ro PSE GLB x pmd 0xffffffff81a00000-0xffffffff81e00000 4M ro PSE GLB NX pmd 0xffffffff81e00000-0xffffffff82200000 4M RW GLB NX pte 0xffffffff82200000-0xffffffff82800000 6M RW PSE GLB NX pmd 0xffffffff82800000-0xffffffff82956000 1368K RW GLB NX pte 0xffffffff82956000-0xffffffff82a00000 680K RW GLB x pte0xffffffff82a00000-0xffffffffa0000000 470M pmd
Quoted 8 lines
Quoted 5 lines
HPA looked at it for a bit, but it wasn't obvious what was going on. It's after the end of bss. I do note that the two adjacent regions add up to 2MiB. Is this some kind of leftover mapping? What is this region? Is there a sensible place to clean it up?
It looks to be what is left after the 2 MB page for bss is broken up. It doesn't mean it isn't broken, though.
It looks like the problem still exists:
---[ High Kernel Mapping ]---
0xffffffff80000000-0xffffffff9ca00000 458M pmd 0xffffffff9ca00000-0xffffffff9d200000 8M ro PSE GLB x pmd 0xffffffff9d200000-0xffffffff9d3f3000 1996K ro GLB x pte 0xffffffff9d3f3000-0xffffffff9d400000 52K ro x pte 0xffffffff9d400000-0xffffffff9d600000 2M ro PSE GLB NX pmd 0xffffffff9d600000-0xffffffff9d7e8000 1952K ro GLB NX pte 0xffffffff9d7e8000-0xffffffff9d800000 96K ro NX pte 0xffffffff9d800000-0xffffffff9d8ff000 1020K RW GLB NX pte 0xffffffff9d8ff000-0xffffffff9da2d000 1208K RW NX pte 0xffffffff9da2d000-0xffffffff9dc00000 1868K RW GLB NX pte 0xffffffff9dc00000-0xffffffff9e600000 10M RW PSE GLB NX pmd 0xffffffff9e600000-0xffffffff9e7f5000 2004K RW GLB NX pte 0xffffffff9e7f5000-0xffffffff9e800000 44K RW GLB x pte
Quoted 5 lines
0xffffffff9e800000-0xffffffffc0000000 536M pmd
Still seems to be the bss getting broken up. What is this left-over memory used for? Any pointers to where it happens? I'd really like to kill this area.
mark_rodata_ro()
set_memory_nx(rodata_start, (all_end - rodata_start) >> PAGE_SHIFT);
all_end is _end which is the end of the the __brk section.
That looks indeed like a 2MB page which is split by set_memory_nx().
That reminder of the 2MB is a leftover from cleanup_highmap()
* We limit the mappings to the region from _text to _brk_end. _brk_end
* is rounded up to the 2MB boundary.
So what you see is the reminder between _brk_end and the 2MB boundary.
Now the simple solution is to round up all_end in mark_rodata_ro() to the 2MB boundary, but we should probably get rid of the mapping completely.
Something like:
end = roundup((unsigned long)all_end, PMD_SIZE) - 1;
free_init_pages(all_end + 1, end)
should do the trick.
Thanks,
tglx
Hi Thomas,
Quoted 61 lines · Thomas Gleixner
Quoted 44 lines · Kees Cook
Continuing a thread from a year ago...
Quoted 16 lines · H. Peter Anvin
Quoted 15 lines · Kees Cook
Hi,
I've noticed there's a chunk of kernel memory still marked RW and x. See
0xffffffff82956000below...---[ High Kernel Mapping ]---
0xffffffff80000000-0xffffffff81000000 16M pmd 0xffffffff81000000-0xffffffff81a00000 10M ro PSE GLB x pmd 0xffffffff81a00000-0xffffffff81e00000 4M ro PSE GLB NX pmd 0xffffffff81e00000-0xffffffff82200000 4M RW GLB NX pte 0xffffffff82200000-0xffffffff82800000 6M RW PSE GLB NX pmd 0xffffffff82800000-0xffffffff82956000 1368K RW GLB NX pte 0xffffffff82956000-0xffffffff82a00000 680K RW GLB x pte0xffffffff82a00000-0xffffffffa0000000 470M pmd
Quoted 8 lines
Quoted 5 lines
HPA looked at it for a bit, but it wasn't obvious what was going on. It's after the end of bss. I do note that the two adjacent regions add up to 2MiB. Is this some kind of leftover mapping? What is this region? Is there a sensible place to clean it up?
It looks to be what is left after the 2 MB page for bss is broken up. It doesn't mean it isn't broken, though.
It looks like the problem still exists:
---[ High Kernel Mapping ]---
0xffffffff80000000-0xffffffff9ca00000 458M pmd 0xffffffff9ca00000-0xffffffff9d200000 8M ro PSE GLB x pmd 0xffffffff9d200000-0xffffffff9d3f3000 1996K ro GLB x pte 0xffffffff9d3f3000-0xffffffff9d400000 52K ro x pte 0xffffffff9d400000-0xffffffff9d600000 2M ro PSE GLB NX pmd 0xffffffff9d600000-0xffffffff9d7e8000 1952K ro GLB NX pte 0xffffffff9d7e8000-0xffffffff9d800000 96K ro NX pte 0xffffffff9d800000-0xffffffff9d8ff000 1020K RW GLB NX pte 0xffffffff9d8ff000-0xffffffff9da2d000 1208K RW NX pte 0xffffffff9da2d000-0xffffffff9dc00000 1868K RW GLB NX pte 0xffffffff9dc00000-0xffffffff9e600000 10M RW PSE GLB NX pmd 0xffffffff9e600000-0xffffffff9e7f5000 2004K RW GLB NX pte 0xffffffff9e7f5000-0xffffffff9e800000 44K RW GLB x pteQuoted 5 lines
0xffffffff9e800000-0xffffffffc0000000 536M pmd
Still seems to be the bss getting broken up. What is this left-over memory used for? Any pointers to where it happens? I'd really like to kill this area.
mark_rodata_ro()set_memory_nx(rodata_start, (all_end - rodata_start) >> PAGE_SHIFT);
all_endis_endwhich is the end of the the__brksection.That looks indeed like a 2MB page which is split by
set_memory_nx().That reminder of the 2MB is a leftover from
cleanup_highmap()
Quoted 7 lines
* We limit the mappings to the region from
_textto_brk_end._brk_end
* is rounded up to the 2MB boundary.So what you see is the reminder between
_brk_endand the 2MB boundary.Now the simple solution is to round up
all_endinmark_rodata_ro()to the 2MB boundary,
Thanks! I just managed to work these things out from Peter's earlier hints, just before you sent this. Good to have my guess confirmed. :)
Quoted 10 lines
but we should probably get rid of the mapping
completely.Something like:
end = roundup((unsigned long)all_end, PMD_SIZE) - 1;
free_init_pages(all_end + 1, end)should do the trick.
Ah-ha! Okay, I'll send a new patch that does this. Thanks!
-Kees
Signature · 1 line
Kees Cook @outflux.net