HMM (heterogeneous memory manager) and GPU

Hi,

I would like to attend LSF/MM this year to discuss about HMM (Heterogeneous Memory Manager) and more generaly all topics related to GPU and heterogeneous memory architecture (including persistent memory).

Quoted 6 lines · Jerome Glisse

Hi,

I would like to attend LSF/MM this year to discuss about HMM (Heterogeneous Memory Manager) and more generaly all topics related to GPU and heterogeneous memory architecture (including persistent memory).

How is persistent memory heterogeneous?

I thought it's either in the same cache coherency domain (DAX case) or is not a memory for kernel -- behind block layer.
Do we have yet another option?

Hi,

I would like to attend LSF/MM this year to discuss about HMM (Heterogeneous Memory Manager) and more generaly all topics related to GPU and heterogeneous memory architecture (including persistent memory).

I want to discuss how to move forward with HMM merging and i hope that by MM summit time i will be able to share more informations publicly on devices which rely on HMM.

Quoted 11 lines · Jerome Glisse

Hi,

I would like to attend LSF/MM this year to discuss about HMM (Heterogeneous Memory Manager) and more generaly all topics related to GPU and heterogeneous memory architecture (including persistent memory).

I want to discuss how to move forward with HMM merging and i hope that by MM summit time i will be able to share more informations publicly on devices which rely on HMM.

I mentioned in my request to attend mail, I would like to attend this discussion. I am wondering whether we can split the series further to mmu_notifier bits and then the page table mirroring bits. Can the mmu notifier changes go in early so that we can merge the page table mirroring later ?

Can be page table mirroring bits be built as a kernel module ?

I would like to attend LSF/MM this year to discuss about HMM (Heterogeneous Memory Manager) and more generaly all topics related to GPU and heterogeneous memory architecture (including persistent memory).

I want to discuss how to move forward with HMM merging and i hope that by MM summit time i will be able to share more informations publicly on devices which rely on HMM.

Quoted 9 lines · Jerome Glisse

I would like to attend LSF/MM this year to discuss about HMM (Heterogeneous Memory Manager) and more generaly all topics related to GPU and heterogeneous memory architecture (including persistent memory).

I want to discuss how to move forward with HMM merging and i hope that by MM summit time i will be able to share more informations publicly on devices which rely on HMM.

There are a few related issues here around Shared Virtual Memory, and lifetime management of the associated MM, and the proposal discussed at the Kernel Summit for "off-CPU tasks".

I've hit a situation with the Intel SVM code in 4.4 where the device driver binds a PASID, and also has mmap() functionality on the same file descriptor that the PASID is associated with.

So on process exit, the MM doesn't die because the PASID binding still exists. The VMA of the mmap doesn't die because the MM still exists. So the underlying file remains open because the VMA still exists. And the PASID binding thus doesn't die because the file is still open.

I've posted a patch¹ which moves us closer to the amd_iommu_v2 model, although I'm still *strongly* resisting the temptation to call out into device driver code from the mmu_notifier's release callback.

I would like to attend LSF/MM this year so we can continue to work on those issues — now that we actually have some hardware in the field and a better idea of how we can build a unified access model for SVM across the different IOMMU types.

--
David Woodhouse Open Source Technology Centre
[email protected] Intel Corporation

¹ http://www.spinics.net/lists/linux-mm/msg100230.html

Quoted 6 lines

So on process exit, the MM doesn't die because the PASID binding still exists. The VMA of the mmap doesn't die because the MM still exists. So the underlying file remains open because the VMA still exists. And the PASID binding thus doesn't die because the file is still open.

Why connect the PASID to the FD in the first place ? Why not tie everything to the MM ?

Quoted 3 lines

I've posted a patch¹ which moves us closer to the amd_iommu_v2 model, although I'm still *strongly* resisting the temptation to call out into device driver code from the mmu_notifier's release callback.

You mean you are resisting doing this (taken from amdkfd):

--------------
static const struct mmu_notifier_ops kfd_process_mmu_notifier_ops = { .release = kfd_process_notifier_release,
};

process->mmu_notifier.ops = &kfd_process_mmu_notifier_ops;
-----------

Why, if I may ask ?

Quoted 8 lines · Oded Gabbay

Quoted 6 lines

So on process exit, the MM doesn't die because the PASID binding still exists. The VMA of the mmap doesn't die because the MM still exists. So the underlying file remains open because the VMA still exists. And the PASID binding thus doesn't die because the file is still open.

Why connect the PASID to the FD in the first place ? Why not tie everything to the MM ?

That's actually a question for the device driver in question, of course; it's not the generic SVM support code which chooses *when* to bind/unbind PASIDs. We just provide those functions for the driver to call.

But the answer is that that's the normal resource tracking model. Resources hang off the file and are cleared up when the file is closed.

(And exit_files() is called later than exit_mm()).

Quoted 11 lines

Quoted 3 lines

I've posted a patch¹ which moves us closer to the amd_iommu_v2 model, although I'm still *strongly* resisting the temptation to call out into device driver code from the mmu_notifier's release callback.

You mean you are resisting doing this (taken from amdkfd):

--------------
static const struct mmu_notifier_ops kfd_process_mmu_notifier_ops = { .release = kfd_process_notifier_release,
};

Quoted 4 lines

process->mmu_notifier.ops = &kfd_process_mmu_notifier_ops;
-----------

Why, if I may ask ?

The KISS principle, especially as it relates to device drivers. We just Do Not Want random device drivers being called in that context.

It's OK for amdkfd where you have sufficient clue to deal with it — it's more than "just a device driver".

But when we get discrete devices with PASID support (and the required TLP prefix support in our root ports at last!) we're going to see SVM supported in many more device drivers, and we should make it simple.

Having the mmu_notifier release callback exposed to drivers is going to strongly encourage them to do the WRONG thing, because they need to interact with their hardware and *wait* for the PASID to be entirely retired through the pipeline before they tell the IOMMU to flush it.

The patch at http://www.spinics.net/lists/linux-mm/msg100230.html
addresses this by clearing the PASID from the PASID table (in core IOMMU code) when the process exits so that all subsequent accesses to that PASID then take faults. The device driver can then clean up its binding for that PASID in its own time.

It is a fairly fundamental rule that faulting access to *one* PASID should not adversely affect behaviour for *other* PASIDs, of course.

--
David Woodhouse Open Source Technology Centre
[email protected] Intel Corporation

OK, so I think I got confused up a little, but looking at your code I see that you register SVM for the mm notifier (intel_mm_release), therefore I guess what you meant to say you don't want to call a device driver callback from your mm notifier callback, correct ? (like the amd_iommu_v2 does when it calls ev_state->inv_ctx_cb inside its mn_release)

Because you can't really control what the device driver will do, i.e. if it decides to register itself to the mm notifier in its own code.

And because you don't call the device driver, the driver can/will get errors for using this PASID (since you unbinded it) and the device driver is supposed to handle it. Did I understood that correctly ?

If I understood it correctly, doesn't it confuses between error/fault and normal unbinding ? Won't it be better to actively notify them and indeed *wait* until the device driver cleared its H/W pipeline before "pulling the carpet under their feet" ?

In our case (AMD GPUs), if we have such an error it could make the GPU stuck. That's why we even reset the wavefronts inside the GPU, if we can't gracefully remove the work from the GPU (see kfd_unbind_process_from_device)

In the patch's comment you wrote:
"Hardware designers have confirmed that the resulting 'PASID not present' faults should be handled just as gracefully as 'page not present' faults"

Unless *all* the H/W that is going to use SVM is designed by the same company, I don't think we can say such a thing. And even then, from my experience, H/W designers can be "creative" sometimes.

Quoted 6 lines · Oded Gabbay

OK, so I think I got confused up a little, but looking at your code I see that you register SVM for the mm notifier (intel_mm_release), therefore I guess what you meant to say you don't want to call a device driver callback from your mm notifier callback, correct ? (like the amd_iommu_v2 does when it calls ev_state->inv_ctx_cb inside its mn_release)

Right.

Quoted 2 lines

Because you can't really control what the device driver will do, i.e. if it decides to register itself to the mm notifier in its own code.

Right. I can't *prevent* them from doing it. But I don't need to encourage or facilitate it :)

Quoted 3 lines

And because you don't call the device driver, the driver can/will get errors for using this PASID (since you unbinded it) and the device driver is supposed to handle it. Did I understood that correctly ?

In the case of an unclean exit, yes. In an orderly shutdown of the process, one would hope that the device context is relinquished cleanly rather than the process simply exiting.

And yes, the device and its driver are expected to handle faults. If they don't do that, they are broken :)

Quoted 9 lines

If I understood it correctly, doesn't it confuses between error/fault and normal unbinding ? Won't it be better to actively notify them and indeed *wait* until the device driver cleared its H/W pipeline before "pulling the carpet under their feet" ?

In our case (AMD GPUs), if we have such an error it could make the GPU stuck. That's why we even reset the wavefronts inside the GPU, if we can't gracefully remove the work from the GPU (see kfd_unbind_process_from_device)

But a rogue process can easily trigger faults — just request access to an address that doesn't exist. My conversation with the hardware designers was not about the peculiarities of any specific implementation, but just getting them to confirm my assertion that if a device *doesn't* cleanly handle faults on *one* PASID without screwing over all the *other* PASIDs, then it is utterly broken by design and should never get to production.

I *do* anticipate broken hardware which will crap itself completely when it takes a fault, and have implemented a callback from the fault handler so that the driver gets notified when a fault *happens* (even on a PASID which is still alive), and can prod the broken hardware if it needs to.

But I wasn't expecting it to be the norm.

Quoted 7 lines

In the patch's comment you wrote:
"Hardware designers have confirmed that the resulting 'PASID not present' faults should be handled just as gracefully as 'page not present' faults"

Unless *all* the H/W that is going to use SVM is designed by the same company, I don't think we can say such a thing. And even then, from my experience, H/W designers can be "creative" sometimes.

If we have to turn it into a 'page not present' fault instead of a 'PASID not present' fault, that's easy enough to do by pointing it at a dummy PML4 (the zero page will do).

But I stand by my assertion that any hardware which doesn't handle at least a 'page not present' fault in a given PASID without screwing over all the other users of the hardware is BROKEN.

We could *almost* forgive hardware for stalling when it sees a 'PASID not present' fault. Since that *does* require OS participation.

--
David Woodhouse Open Source Technology Centre
[email protected] Intel Corporation

Quoted 38 lines · David Woodhouse

Quoted 6 lines · Oded Gabbay

OK, so I think I got confused up a little, but looking at your code I see that you register SVM for the mm notifier (intel_mm_release), therefore I guess what you meant to say you don't want to call a device driver callback from your mm notifier callback, correct ? (like the amd_iommu_v2 does when it calls ev_state->inv_ctx_cb inside its mn_release)

Right.

Quoted 2 lines

Because you can't really control what the device driver will do, i.e. if it decides to register itself to the mm notifier in its own code.

Right. I can't *prevent* them from doing it. But I don't need to encourage or facilitate it :)

Quoted 3 lines

And because you don't call the device driver, the driver can/will get errors for using this PASID (since you unbinded it) and the device driver is supposed to handle it. Did I understood that correctly ?

In the case of an unclean exit, yes. In an orderly shutdown of the process, one would hope that the device context is relinquished cleanly rather than the process simply exiting.

And yes, the device and its driver are expected to handle faults. If they don't do that, they are broken :)

Quoted 9 lines

If I understood it correctly, doesn't it confuses between error/fault and normal unbinding ? Won't it be better to actively notify them and indeed *wait* until the device driver cleared its H/W pipeline before "pulling the carpet under their feet" ?

In our case (AMD GPUs), if we have such an error it could make the GPU stuck. That's why we even reset the wavefronts inside the GPU, if we can't gracefully remove the work from the GPU (see kfd_unbind_process_from_device)

Quoted 7 lines

But a rogue process can easily trigger faults — just request access to an address that doesn't exist. My conversation with the hardware designers was not about the peculiarities of any specific implementation, but just getting them to confirm my assertion that if a device *doesn't* cleanly handle faults on *one* PASID without screwing over all the *other* PASIDs, then it is utterly broken by design and should never get to production.

Yes, that is agreed, address errors should not affect the H/W itself, nor other processes.

Quoted 9 lines

I *do* anticipate broken hardware which will crap itself completely when it takes a fault, and have implemented a callback from the fault handler so that the driver gets notified when a fault *happens* (even on a PASID which is still alive), and can prod the broken hardware if it needs to.

But I wasn't expecting it to be the norm.

Yeah, I guess that after a few H/W iterations the "correct" implementation will be the norm.

Quoted 8 lines

Quoted 7 lines

In the patch's comment you wrote:
"Hardware designers have confirmed that the resulting 'PASID not present' faults should be handled just as gracefully as 'page not present' faults"

Unless *all* the H/W that is going to use SVM is designed by the same company, I don't think we can say such a thing. And even then, from my experience, H/W designers can be "creative" sometimes.

Quoted 7 lines

If we have to turn it into a 'page not present' fault instead of a 'PASID not present' fault, that's easy enough to do by pointing it at a dummy PML4 (the zero page will do).

But I stand by my assertion that any hardware which doesn't handle at least a 'page not present' fault in a given PASID without screwing over all the other users of the hardware is BROKEN.

Totally agreed!

Quoted 8 lines

We could *almost* forgive hardware for stalling when it sees a 'PASID not present' fault. Since that *does* require OS participation.

--
David Woodhouse Open Source Technology Centre
[email protected] Intel Corporation

Another, perhaps trivial, question.
When there is an address fault, who handles it ? the SVM driver, or each device driver ?

In other words, is the model the same as (AMD) IOMMU where it binds amd_iommu driver to the IOMMU H/W, and that driver (amd_iommu/v2) is the only one which handles the PPR events ?

If that is the case, then with SVM, how will the device driver be made aware of faults, if the SVM driver won't notify him about them, because it has already severed the connection between PASID and process ?

If the model is that each device driver gets a direct fault notification (via interrupt or some other way) then that is a different story.

Oded

Quoted 4 lines

Another, perhaps trivial, question.
When there is an address fault, who handles it ? the SVM driver, or each device driver ?

Quoted 8 lines

In other words, is the model the same as (AMD) IOMMU where it binds amd_iommu driver to the IOMMU H/W, and that driver (amd_iommu/v2) is the only one which handles the PPR events ?

If that is the case, then with SVM, how will the device driver be made aware of faults, if the SVM driver won't notify him about them, because it has already severed the connection between PASID and process ?

And another question, if I may, aren't you afraid of "false positive" prints to dmesg ? I mean, I'm pretty sure page faults / pasid faults errors will be logged somewhere, probably to dmesg. Aren't you concerned of the users seeing those errors and thinking they may have a bug, while actually the errors were only caused by process termination ?

Or in that case you say that the application is broken, because if it still had something running in the H/W, it should not have closed itself ?

In the ideal case, there's no need for the device driver to get involved at all. When a page isn't found in the page tables, the IOMMU code calls handle_mm_fault() and either populates the page and sends a a 'success' response, or sends an 'invalid fault' response back.

Quoted 4 lines · Oded Gabbay

Quoted 4 lines

Another, perhaps trivial, question.
When there is an address fault, who handles it ? the SVM driver, or each device driver ?

Quoted 8 lines

Quoted 8 lines

In other words, is the model the same as (AMD) IOMMU where it binds amd_iommu driver to the IOMMU H/W, and that driver (amd_iommu/v2) is the only one which handles the PPR events ?

If that is the case, then with SVM, how will the device driver be made aware of faults, if the SVM driver won't notify him about them, because it has already severed the connection between PASID and process ?

In the ideal case, there's no need for the device driver to get involved at all. When a page isn't found in the page tables, the IOMMU code calls handle_mm_fault() and either populates the page and sends a a 'success' response, or sends an 'invalid fault' response back.

To account for broken hardware, we *have* added a callback into the device driver when these faults happen. Ideally it should never be used, of course.

In the case where the process has gone away, the PASID is still assigned and we still hold mm_count on the MM, just not mm_users. This callback into the device driver still occurs if a fault happens during process exit between the exit_mm() and exit_files() stage.

Quoted 6 lines

And another question, if I may, aren't you afraid of "false positive" prints to dmesg ? I mean, I'm pretty sure page faults / pasid faults errors will be logged somewhere, probably to dmesg. Aren't you concerned of the users seeing those errors and thinking they may have a bug, while actually the errors were only caused by process termination ?

If that's the case, it's easy enough to silence them. We are already explicitly testing for the 'defunct mm' case in our fault handler, to prevent us from faulting more pages into an obsolescent MM after its mm_users reaches zero and its page tables are supposed to have been torn down. That's the 'if(!atomic_inc_not_zere(&svm->mm->mm_users)) goto bad_req;' part.

Quoted 3 lines

Or in that case you say that the application is broken, because if it still had something running in the H/W, it should not have closed itself ?

That's also true but it's still nice to avoid confusion. Even if only to disambiguate cause and effect — we don't want people to see PASID faults which were caused by the process crashing, and to think that they might be involved in *causing* that process to crash...

--
David Woodhouse Open Source Technology Centre
[email protected] Intel Corporation

It seems you have most of your bases covered. I'll stop harassing you now :) But in seriousness, its interesting to see the different approaches taken to handling pretty much the same type of H/W (IOMMU).

There are a few related issues here around Shared Virtual Memory, and lifetime management of the associated MM, and the proposal discussed at the Kernel Summit for "off-CPU tasks".

I've hit a situation with the Intel SVM code in 4.4 where the device driver binds a PASID, and also has mmap() functionality on the same file descriptor that the PASID is associated with.

So on process exit, the MM doesn't die because the PASID binding still exists. The VMA of the mmap doesn't die because the MM still exists. So the underlying file remains open because the VMA still exists. And the PASID binding thus doesn't die because the file is still open.

I've posted a patchA1 which moves us closer to the amd_iommu_v2 model, although I'm still *strongly* resisting the temptation to call out into device driver code from the mmu_notifier's release callback.

I would like to attend LSF/MM this year so we can continue to work on those issues a?? now that we actually have some hardware in the field and a better idea of how we can build a unified access model for SVM across the different IOMMU types.