PAT support
A message that is not in the archive (sent off-list or not yet received): [email protected]
Terence Ripperda <[email protected]> writes:
Hi Terence,
Quoted 9 lines
this current patch includes the original PAT support and the new cachemap mechanism. note that the cachemap mechanism does not actually change any caching attributes, it only keeps track of the attributes and tests regions. I think the end idea would be that drivers would use the normal ioremap/change_page_attr/remap_page_range mechanisms like they already do, and these mechanisms would in turn use cachemap to make sure there's no conflicts. I'm completely open to how any specific details should work, and any changes needed to be made.
Looks good for starting. The patch could use some minor cleanup still, but it should be good enough for testing.
As for an interface - i still think it would be cleaner to just call it from change_page_attr(). Then other users only need to call a single function. But that's easily changed.
To make it really useful I think we need ioremap_wrcomb() and support in the bus/pci mmap function (the PCI layer already has ioctls for this, they are just ignored on i386 right). Then the X server could start using it.
Without any users the testing coverage would be probably not too good, but it needs some testing in the real world before it could be merged first. Maybe it could be simply hooked into the AGP driver and into some DRM driver. Then people would start testing it at least.
-Andi
Quoted 2 lines · On Mon, Apr 12, 2004 at 05:01:33PM -0700, [email protected] wrote:
Looks good for starting. The patch could use some minor cleanup still, but it should be good enough for testing.
sure. I don't know all of the kernel style guidelines well enough, so I wouldn't be surprised if some of that was off. Also, I'm not sure if I did the architectural breakdown correctly.
Quoted 3 lines
As for an interface - i still think it would be cleaner to just call it from
change_page_attr(). Then other users only need to call a single function. But that's easily changed.
sure, I'm fine with that. I have a couple of related questions that might be dumb:
we discussed before how change_page_attr takes a struct page *. there are many cases where pci i/o memory is backed by struct pages, but in the majority of x86 desktops, this isn't the case. I'm unclear how these cases would be handled? would change_page_attr be changed to take address/size rather than struct page? would drivers be responsible for recognizing this case and calling a different api in that case (cmap directly)? should x86 be changed such that all memory is covered by struct pages (doubtful)?
a lot of mmaps currently use remap_page_range w/o change_page_attr currently. I had thought that by putting the cmap_request inside of remap_page_range, we would make sure we caught all remappings and didn't miss any potential conflicts. is the preference to have all drivers updated and be responsible for calling change_page_attr before remap_page_range? or perhaps I'm missing the obvious: ioremap calls change_page_attr in the correct cases, so perhaps remap_page_range should call change_page_attr, which would in turn call cmap_request.
the main reason I hadn't added the cmap_request call to change_page_attr was due to us focusing on i/o regions first, then tackling system memory later. I can go ahead and add the call, since cmap_request will currently recognize system memory and skip over it, returning success. I would then still need to figure out how to work change_page_attr and i/o regions, at least on x86.
Quoted 4 lines
To make it really useful I think we need
ioremap_wrcomb()and support in the bus/pci mmap function (the PCI layer already has ioctls for this, they are just ignored on i386 right). Then the X server could start using it.
is this pci_mmap_page_range? I hadn't seen that before (it looks like it was added to i386 in the 2.6 series). it does look like it's plugged into proc_bus_pci_mmap for i386 on 2.6, but perhaps I'm missing something when skimming the code. that should be easy to add (or would be picked up by a change to remap_page_range).
I've added ioremap_wrcomb in my tree, but need to test it still.
Quoted 5 lines
Without any users the testing coverage would be probably not too good, but it needs some testing in the real world before it could be merged first. Maybe it could be simply hooked into the AGP driver and into some DRM driver. Then people would start testing it at least.
sure. I did a quick scan of the code, I see an mmap function in the agpgart code. it looks like some of the drm drivers (ffb/i810/i830) have mmap, but not all of them. I would assume they relied on the agpgart mmap. the agpgart mmap could be updated, based on the decision above (driver vs. change_page_attr/remap_page_range), would that be enough?
Thanks,
Terence
[I put Egbert Eich in cc who may be interested in the X server side of this]
Quoted 9 lines · Terence Ripperda
Quoted 2 lines · On Mon, Apr 12, 2004 at 05:01:33PM -0700, [email protected] wrote:
Looks good for starting. The patch could use some minor cleanup still, but it should be good enough for testing.
sure. I don't know all of the kernel style guidelines well enough, so I wouldn't be surprised if some of that was off. Also, I'm not sure if I did the architectural breakdown correctly.
Quoted 7 lines
Quoted 3 lines
As for an interface - i still think it would be cleaner to just call it from
change_page_attr(). Then other users only need to call a single function. But that's easily changed.sure, I'm fine with that. I have a couple of related questions that might be dumb:
we discussed before how
change_page_attrtakes a struct page *. there are many cases where pci i/o memory is backed by struct pages, but in the majority of x86 desktops, this isn't the case. I'm unclear how these cases would be handled? wouldchange_page_attrbe changed to take address/size rather than struct page? would drivers be responsible for recognizing this case and calling a different api in that case (cmap directly)? should x86 be changed such that all memory is covered by struct pages (doubtful)?
Covering everything with struct page * would be a waste of memory (the mappings tend to be at the end of 4GB and just covering 4GB with struct page costs considerable memory)
I would just change change_page_attr() back to use physical addresses. In fact the original version did that, but someone complained that it may get used for highmem too - but that was never handled (it BUGs right now) and doesn't make sense because highmem is not in the direct mapping.
Quoted 1 line
a lot of mmaps currently use
remap_page_rangew/ochange_page_attrcurrently. I had thought that by putting thecmap_requestinside ofremap_page_range, we would make sure we caught all remappings and didn't miss any potential conflicts. is the preference to have all drivers updated and be responsible for callingchange_page_attrbeforeremap_page_range? or perhaps I'm missing the obvious: ioremap callschange_page_attrin the correct cases, so perhapsremap_page_rangeshould callchange_page_attr, which would in turn callcmap_request.
Normally it's not needed on i386 because the mappings remapped by
remap_page_range are not in the direct mapping (which only maps the first
~900MB) on i386. On x86-64 it is buggy when you have enough memory, because the PCI IO regions below 4GB will be in the direct mapping then. Then it should call change_page_attr when it uses a non caching attribute, agreed.
It could be a problem on x86 too if someone remaps a mapping in the
640k-1MB hole, but that's probably unlikely.
Quoted 2 lines
the main reason I hadn't added the
cmap_requestcall tochange_page_attrwas due to us focusing on i/o regions first, then tackling system memory later. I can go ahead and add the call, sincecmap_requestwill currently recognize system memory and skip over it, returning success. I would then still need to figure out how to workchange_page_attrand i/o regions, at least on x86.
For x86 it's not needed, only for x86-64, but it would be nice to use common behaviour between the two.
Quoted 10 lines
Quoted 6 lines
To make it really useful I think we need
ioremap_wrcomb()and support in the bus/pci mmap function (the PCI layer already has ioctls for this, they are just ignored on i386 right). Then the X server could start using it.is this
pci_mmap_page_range? I hadn't seen that before (it looks like it was added to i386 in the 2.6 series). it does look like it's plugged intoproc_bus_pci_mmapfor i386 on 2.6, but perhaps I'm missing something when skimming the code. that should be easy to add (or would be picked up by a change toremap_page_range).I've added
ioremap_wrcombin my tree, but need to test it still.
Yes. I thought it was already in 2.4?
The pci mmap also has an ioctl - see drivers/pci/proc.c:proc_bus_pci_ioctl - to set write combining. It would be nice if that was just hooked up.
Quoted 7 lines
Quoted 5 lines
Without any users the testing coverage would be probably not too good, but it needs some testing in the real world before it could be merged first. Maybe it could be simply hooked into the AGP driver and into some DRM driver. Then people would start testing it at least.
sure. I did a quick scan of the code, I see an mmap function in the agpgart code. it looks like some of the drm drivers (ffb/i810/i830) have mmap, but not all of them. I would assume they relied on the agpgart mmap. the agpgart mmap could be updated, based on the decision above (driver vs. change_page_attr/remap_page_range), would that be enough?
Yes.
And possible the X server to use /proc/bus/pci/* instead of /dev/mem and then use the wrcombining ioctl instead of /proc/mtrr.
-Andi
Thanks for the great feedback (as always), Andi.
I've gotten started on this, mainly focusing on modifying change_page_attr to use unsigned long instead of struct page*. hit a few snags on the way, but got things figured out. it should be working now, but may still have a bug or two I need to work out. I also moved cmap_request_range into change_page_attr as suggested.
what I wanted to bring up is some of the issues I'm running into that will require some slight adjustments. I wanted to make sure to get feedback on this.
the first is that change_page_attr is basically a one-way operation that just sets the caching for a range of pages. in contrast, the cmap_ api is more of a push/pop approach, with cmap_request_range and cmap_release_range. all callers of change_page_attr generally use it to set a new attribute, then assume they need to restore the attribute back to PAGE_KERNEL (WB). additionally, when splitting/merging large pages, change_page_attr has similar assumptions (that we flip between a default PAGE_KERNEL caching and some other caching).
I haven't thought through all the details yet, but would it be acceptable to consider modifying change_page_attr to also use a push/pop approach? it may also make sense to remove the assumptions about initial state and actually save off what the initial state is, so it can be restored to that state.
the second is in iounmap. this function only takes an address, we don't get the size until we've retrieved the struct vm_struct* by freeing the mapping via remove_vm_area. once we have this size, we can call change_page_attr. but by then, the virtual mapping was already freed in remove_vm_area, so playing with the page tables in change_page_attr causes problems:
remap_area_pte: page already exists
------------[ cut here ]------------ kernel BUG at arch/i386/mm/ioremap.c:37! invalid operand: 0000 [#1] ... Call Trace: [<c011d9c4>] __ioremap+0xbc/0xec [<c0228012>] acpi_os_map_memory+0x2a/0x44 [<c023c11a>] acpi_tb_get_table_header+0x52/0xdc [<c023c076>] acpi_tb_get_table+0x16/0x68 [<c023fb87>] acpi_ut_allocate_owner_id+0x8f/0x9c
...
in this case, I think change_page_attr filled the pte back in after it had been freed by remove_vm_area.
this could be fixed multiple ways: change_page_attr could not touch the page tables at all for i/o regions (would ioremap cover everything? are 4M ptes used on i/o regions?), we could check for the PAGE_PRESENT bit (seems like a hack workaround), I could add an api to retrieve the struct vm_struct* so we could call change_page_attr before calling remove_vm_area. I wanted to check and see what the preferred approach was.
(aside: we only call change_page_attr if we're calling a non-standard ioremap, like ioremap_nocache. I need to move this logic into ioremap, so we get a cmap_ for all ioremaps, regardless of caching type. note that iounmap currently has no idea what caching ioremap requested.)
also, I'm attaching my current progress. note that this is a debug-work-in-progress, so has lots of extra printouts and perhaps some code that would be cleaned up (for example, I duplicated lookup_address -> lookup_phys_address to make some things easier, but that might be cleaned up before long). I also haven't had time to get to all the suggestions, but will get to them.
Thanks,
Terence
Quoted 3 lines · Terence Ripperda
the first is that
change_page_attris basically a one-way operation that just sets the caching for a range of pages. in contrast, thecmap_api is more of a push/pop approach, withcmap_request_rangeandcmap_release_range. all callers ofchange_page_attrgenerally use it to set a new attribute, then assume they need to restore the attribute back toPAGE_KERNEL(WB). additionally, when splitting/merging large pages,change_page_attrhas similar assumptions (that we flip between a defaultPAGE_KERNELcaching and some other caching).I haven't thought through all the details yet, but would it be acceptable to consider modifying
change_page_attrto also use a push/pop approach? it may also make sense to remove the assumptions about initial state and actually save off what the initial state is, so it can be restored to that state.
change_page_attr can change more than just caching attributes, also read/write (e.g. slab debug uses it for that)
At least for the later adding another book keeping data structure may be too expensive. Of course the data structure is only needed for change that set a non standard caching attribute,but having an release function would require a handle to release, forcing it on the others too.
I think I prefer the do/undo model instead of push/pop. That can work with cmaps too. PAGE_KERNEL means no cmap, PAGE_KERNEL_WC and PAGE_KERNEL_NOCACHE get a cmap.
Quoted 1 line
the second is in iounmap. this function only takes an address, we don't get the size until we've retrieved the struct vm_struct* by freeing the mapping via
remove_vm_area. once we have this size, we can callchange_page_attr. but by then, the virtual mapping was already freed inremove_vm_area, so playing with the page tables inchange_page_attrcauses problems:
remove_vm_area() needs to just be split into some worker functions (__remove_vm_area et.al.)
Quoted 2 lines
this could be fixed multiple ways:
change_page_attrcould not touch the page tables at all for i/o regions (would ioremap cover everything? are 4M ptes used on i/o regions?), we could check for thePAGE_PRESENTbit (seems like a hack workaround), I could add an api to retrieve the struct vm_struct* so we could callchange_page_attrbefore callingremove_vm_area. I wanted to check and see what the preferred approach was.
The latest sounds cleanest to me.
Quoted 1 line
(aside: we only call
change_page_attrif we're calling a non-standard ioremap, likeioremap_nocache. I need to move this logic into ioremap, so we get acmap_for all ioremaps, regardless of caching type. note that iounmap currently has no idea what caching ioremap requested.)
Why? no cmap means write back, no?
It's probably an academic case anyways - ioremaps without _nocache should be near always from main memory, which it would ignore.
-Andi
Quoted 2 lines · On Fri, Apr 16, 2004 at 05:42:17PM -0700, [email protected] wrote:
change_page_attrcan change more than just caching attributes, also read/write (e.g. slab debug uses it for that)
ah, ok. sorry, missed that.
Quoted 2 lines
At least for the later adding another book keeping data structure may be too expensive.
makes sense.
Quoted 3 lines
I think I prefer the do/undo model instead of push/pop. That can work with cmaps too.
PAGE_KERNELmeans no cmap,PAGE_KERNEL_WCandPAGE_KERNEL_NOCACHEget a cmap.
but then what is the point of cmap? I would expect a mix of WC and UC mappings to be much less dangerous than a mix of WC/UC and WB. perhaps my mindset is wrong, but it seems allowing ioremap to request a cached mapping is important, and that if that mapping was followed by ioremap_nocached or ioremap_wrcomb, that these subsequent calls should fail.
I did finish implementing your suggestion, that change_page_attr should consider PAGE_KERNEL as a call to cmap_release_range and anything else as a call to cmap_request_range. seemed to work ok, but I'm seeing the acpi table code doing a lot of ioremaps (cached) that are ignored, then iounmaps are causing cmap_release_range calls to complain about not finding the regions. of course in a final version, we'd cut out the debug output, but expecting lots of empty calls to cmap_release_range seems messy.
what if there was a restore_page_attr(unsigned long address, unsigned long numpages) that assumed the pgprot was PAGE_KERNEL. change_page_attr knows to call cmap_request_range and restore_page_attr knows to call cmap_release_range. otherwise they do the same thing, restore_ just inherently uses PAGE_KERNEL for the caching type.
Quoted 2 lines
remove_vm_area()needs to just be split into some worker functions (__remove_vm_areaet.al.)
ok, easily done.
Thanks,
Terence
Quoted 5 lines · Terence Ripperda
Quoted 3 lines
I think I prefer the do/undo model instead of push/pop. That can work with cmaps too.
PAGE_KERNELmeans no cmap,PAGE_KERNEL_WCandPAGE_KERNEL_NOCACHEget a cmap.but then what is the point of cmap? I would expect a mix of WC and UC mappings to be much less dangerous than a mix of WC/UC and WB. perhaps my mindset is wrong, but it seems allowing ioremap to request a cached mapping is important, and that if that mapping was followed by
ioremap_nocachedorioremap_wrcomb, that these subsequent calls should fail.
Hmm, you're right. push/pop is probably better for io-mappings, otherwise we cannot catch existing mappings. This will be needed for user mmap too.
Ignore my previous suggestion on that then please. Sorry for the noise.
-Andi
ok, got this finished up. modified change_page_attr to take a kernel virtual address instead of a page struct. also introduced restore_page_attr, which releases the cmap_range and assumed a pgprot of type PAGE_KERNEL. tweaked the change_page_attr a little to allow virtual addresses, convert from virtual to physical. if you'd prefer any style changes to what I made, feel free to flame me.
I also merged the ioremap functions into a single function (which really just means moving change_page_attr into __ioremap) and made ioremap, ioremap_nocache, and ioremap_wrcomb as inline routines that send the appropriate flags into __ioremap.
fixed an oversight in cmap_convert_flags.
updated all references to change_page_attr to use the correct arguments, but still need to finish the core function changes for x86_64.
between finishing this up and making a first pass at the /proc/bus/pci/* changes, I made some "doh!" realizations:
change_page_attr takes a kernel virtual address currently. this doesn't work well for mmap situations, which have a user virtual address. lookup_address could theoretically be changed to handle this (by switching between kernel & user pgd), but this wouldn't work for 4G/4G kernels.
change_page_attr can't take a physical address, since then it couldn't change_page_attr (change the page table attributes).
a side effect of having cmap_request_range being part of change_page_attr is that the caller must first setup the page tables for the virtual mapping, then check if the caching is ok. it would seem preferrable to use cmap_request_range to check if the caching is ok, setup the page tables, then modify the page tables via change_page_attr (for kernel pages).
I haven't looked too closely yet, but it would seem like all of this could be encapsulated in remap_page_range and ioremap (ie, driver writers would never call cmap_ routines directly). I'll spend a day or two looking into how well this works.
Thanks,
Terence
Quoted 21 lines · On Tue, Apr 20, 2004 at 11:51:12AM -0700, [email protected] wrote:
Quoted 5 lines · Terence Ripperda
Quoted 3 lines
I think I prefer the do/undo model instead of push/pop. That can work with cmaps too.
PAGE_KERNELmeans no cmap,PAGE_KERNEL_WCandPAGE_KERNEL_NOCACHEget a cmap.but then what is the point of cmap? I would expect a mix of WC and UC
mappings to be much less dangerous than a mix of WC/UC and WB. perhaps my mindset is wrong, but it seems allowing ioremap to request a cached mapping is important, and that if that mapping was followed by
ioremap_nocachedorioremap_wrcomb, that these subsequent calls should fail.Hmm, you're right. push/pop is probably better for io-mappings, otherwise
we cannot catch existing mappings. This will be needed for user mmap too.Ignore my previous suggestion on that then please. Sorry for the noise.
-Andi
Quoted 2 lines · Terence Ripperda
ok, got this finished up. modified
change_page_attrto take a kernel virtual address instead of a page struct. also introducedrestore_page_attr, which releases thecmap_rangeand assumed a pgprot of typePAGE_KERNEL. tweaked thechange_page_attra little to allow virtual addresses, convert from virtual to physical. if you'd prefer any style changes to what I made, feel free to flame me.
I leave the style wars to others who are more motivated at nitpicking than me ;-)
actually it doesn't look that bad.
Quoted 11 lines
I also merged the ioremap functions into a single function (which really just means moving
change_page_attrinto__ioremap) and made ioremap,ioremap_nocache, andioremap_wrcombas inline routines that send the appropriate flags into__ioremap.fixed an oversight in
cmap_convert_flags.updated all references to
change_page_attrto use the correct arguments, but still need to finish the core function changes forx86_64.between finishing this up and making a first pass at the
/proc/bus/pci/*changes, I made some "doh!" realizations:
change_page_attrtakes a kernel virtual address currently. this doesn't work well for mmap situations, which have a user virtual address.lookup_addresscould theoretically be changed to handle this (by switching between kernel & user pgd), but this wouldn't work for 4G/4G kernels.
Yes, change_page_attr is only for the kernel direct mapping, nothing more.
All other mapers (pci, /dev/mem, agp, ioremap_*) must take a cmap reference and prevent conflicting maps in the new model.
Quoted 1 line
change_page_attrcan't take a physical address, since then it couldn'tchange_page_attr(change the page table attributes).
It can: there is a 1:1 mapping from physical to kernel virtual address with __va (except highmem, but that's not interesting here because it's not mapped in the kernel and doesn't need to be changed)
actually when user space PAT is supported highmem kmap_* may need to take a look at the PTE or lookup the cmap because it also does a private mapping, but that would get quite expensive and kmap is time critical. My gut feeling would be to just ignore this case for now - it could only happen e.g. when someone does write(fd, mmapfromhardware, ...) and that could be left undefined.
Quoted 1 line
a side effect of having
cmap_request_rangebeing part ofchange_page_attris that the caller must first setup the page tables for the virtual mapping, then check if the caching is ok. it would seem preferrable to usecmap_request_rangeto check if the caching is ok, setup the page tables, then modify the page tables viachange_page_attr(for kernel pages).
True.
Quoted 2 lines
I haven't looked too closely yet, but it would seem like all of this could be encapsulated in
remap_page_rangeand ioremap (ie, driver writers would never callcmap_routines directly). I'll spend a day or two looking into how well this works.
Yes.
But there should be more low level calls too for the few users who cannot use these high level functions for some reason.
Quick review:
set_pat: shouldn't the unknown CPU case error out?
pageattr: i think it should work with PFNs, otherwise you cannot
handle >4GB on i386. while IO mappings are normally <4GB this will be needed for memory and in theory some mapping could be above 4GB.
i'm not sure what lookup_phys_address is good for. Why don't you just use __pa ? Or just pass the physical address ?
+ /* I think this is wrong, i/o region is above highmem? */
+ if (pfn_valid(pfn) && (pfn_to_page(pfn) >= highmem_start_page))
when it doesn't handle cmaps it should just return for any memory >= high_memory
+ if ((c_tmp->end + 1) & 0xfff) BUG();
better use PAGE_MASK instead of hardcoding.
-Andi
Quoted 26 lines · Andi Kleen
Terence Ripperda <[email protected]> writes:
Hi Terence,
Quoted 9 lines
this current patch includes the original PAT support and the new cachemap mechanism. note that the cachemap mechanism does not actually change any caching attributes, it only keeps track of the attributes and tests regions. I think the end idea would be that drivers would use the normal ioremap/change_page_attr/remap_page_range mechanisms like they already do, and these mechanisms would in turn use cachemap to make sure there's no conflicts. I'm completely open to how any specific details should work, and any changes needed to be made.
Looks good for starting. The patch could use some minor cleanup still, but it should be good enough for testing.
As for an interface - i still think it would be cleaner to just call it from
change_page_attr(). Then other users only need to call a single function. But that's easily changed.To make it really useful I think we need
ioremap_wrcomb()and support in the bus/pci mmap function (the PCI layer already has ioctls for this, they are just ignored on i386 right). Then the X server could start using it.
Quoted 5 lines
Without any users the testing coverage would be probably not too good, but it needs some testing in the real world before it could be merged first. Maybe it could be simply hooked into the AGP driver and into some DRM driver. Then people would start testing it at least.
This would also be extremely useful on machines with large amounts of memory, for write-back mappings. With large amounts but odd sized entries it becomes extremely tricky to map all of the memory using mtrrs.
Last I looked the common remedy was to using overlapping mtrrs which the kernel does not understand and that make it impossible for X to setup write-combining MTRRs.
The memory map on x86 shows no hope of getting simpler and mtrrs are getting continually less good at being able to scale so getting PAT support of some kind in the kernel would be extremely good.
Eric
Quoted 4 lines
This would also be extremely useful on machines with large amounts of memory, for write-back mappings. With large amounts but odd sized entries it becomes extremely tricky to map all of the memory using mtrrs.
Yes agreed. I already had vendors complaining about this. But for this it will need some more work - the MTRRs need to be fully converted to PAT and then disabled (because MTRRs have higher priority than PAT). Doing so is a lot more risky than what Terrence's patch does currently though. But longer term we will need it.
Also it will still need to handle overlapping ranges. I suppose it will need some simple rules like: converting from UC to WC is always ok.
-Andi
Quoted 11 lines · Andi Kleen
Quoted 4 lines
This would also be extremely useful on machines with large amounts of memory, for write-back mappings. With large amounts but odd sized entries it becomes extremely tricky to map all of the memory using mtrrs.
Yes agreed. I already had vendors complaining about this. But for this it will need some more work - the MTRRs need to be fully converted to PAT and then disabled (because MTRRs have higher priority than PAT). Doing so is a lot more risky than what Terrence's patch does currently though. But longer term we will need it.
Ugh. You are right. The processors look at the two types and pick the one that caches the least. So PAT can't enable caching :(
Quoted 3 lines
Also it will still need to handle overlapping ranges. I suppose it will need some simple rules like: converting from UC to WC is always ok.
Right.
That plus it should have some additional rules like the e820 map trumps the mtrrs in specifying what is memory so should be cacheable.
Eric