drivers: iommu: Use of add_device()/remove_device() outside of IOMMU groups
PATCH 1/1 drivers: iommu: Use of add_device()/remove_device() outside of IOMMU groups
We've recently been thinking about the best way to attach platform devices in
an IOMMU in a generic way without having to have every platform device
register itself at probe time. We've seen that at least on the tegra IOMMU,
using the platform_bus notifier seems like a viable option (http://lists.linuxfoundation.org/pipermail/iommu/2012-December/004954.html). We also noticed that the IOMMU ops add_device() function is called (indirectly) from a similar bus
notifier and we're thinking that using the bus notifier in the common code
would be better than us rolling our own to do basically the same thing.
The problem, however, is that add_device() seems to be intended for use with IOMMU groups. i.e. After calling add_device() on a device that is should be in a group, dev->iommu_group should have some meaningful value.
I think though, that if the IOMMU groups requirement can be removed, it should be more versatile.
I believe that removing the IOMMU groups dependency can be achieved with the patch below. Currently the intel-iommu seems to be the only implementation using these callbacks.
--- drivers/iommu/intel-iommu.c | 3 ++- drivers/iommu/iommu.c | 2 +- include/linux/iommu.h | 4 ++-- 3 files changed, 5 insertions(+), 4 deletions(-)
drivers/iommu/intel-iommu.c +2 -1
@@ -4217,7 +4217,8 @@ root_bus:static void intel_iommu_remove_device (struct device *dev) { -iommu_group_remove_device (dev); +if (dev->iommu_group) +iommu_group_remove_device (dev); }static struct iommu_ops intel_iommu_ops = {
drivers/iommu/iommu.c +1 -1
@@ -510,7 +510,7 @@ static int iommu_bus_notifier(struct notifier_block *nb,if (ops->add_device)return ops->add_device (dev); }else if (action ==BUS_NOTIFY_DEL_DEVICE ) { -if (ops->remove_device && dev->iommu_group) { +if (ops->remove_device) { ops->remove_device (dev);return 0 ; }
include/linux/iommu.h +2 -2
@@ -74,8 +74,8 @@ enum iommu_attr {* @unmap: unmap a physically contiguous memory region from an iommu domain * @iova_to_phys: translate iova to physical address * @domain_has_cap: domain capabilities query -* @add_device: add device to iommu grouping -* @remove_device: remove device from iommu grouping +* @add_device: add device to iommu grouping or domain +* @remove_device: remove device from iommu grouping or domain * @domain_get_attr: Query domain attributes * @domain_set_attr: Change domain attributes * @pgsize_bitmap: bitmap of supported page sizes
Signature · 1 line
1.7.5.4
A message that is not in the archive (sent off-list or not yet received): 1358244652-12728-1-git-send-email-dhobsong-AlSX/UN32fvPDbFq/[email protected]
Quoted 14 lines · Damian Hobson-Garcia
We've recently been thinking about the best way to attach platform devices in
an IOMMU in a generic way without having to have every platform device
register itself at probe time. We've seen that at least on the tegra IOMMU,
using theplatform_busnotifier seems like a viable option (http://lists.linuxfoundation.org/pipermail/iommu/2012-December/004954.html). We also noticed that the IOMMU opsadd_device()function is called (indirectly) from a similar bus
notifier and we're thinking that using the bus notifier in the common code
would be better than us rolling our own to do basically the same thing.The problem, however, is that
add_device()seems to be intended for use with IOMMU groups. i.e. After callingadd_device()on a device that is should be in a group,dev->iommu_groupshould have some meaningful value.
I think though, that if the IOMMU groups requirement can be removed, it should be more versatile.
Quoted 3 lines
I believe that removing the IOMMU groups dependency can be achieved with the patch below. Currently the intel-iommu seems to be the only implementation using these callbacks.
What stops you from creating a single iommu_group and adding all device to it? Thanks,
Alex
Quoted 49 lines
--- drivers/iommu/intel-iommu.c | 3 ++- drivers/iommu/iommu.c | 2 +- include/linux/iommu.h | 4 ++-- 3 files changed, 5 insertions(+), 4 deletions(-)drivers/iommu/intel-iommu.c +2
-1@@ -4217,7 +4217,8 @@ root_bus:static void intel_iommu_remove_device (struct device *dev) { -iommu_group_remove_device (dev); +if (dev->iommu_group) +iommu_group_remove_device (dev); }static struct iommu_ops intel_iommu_ops = {drivers/iommu/iommu.c +1
-1@@ -510,7 +510,7 @@ static int iommu_bus_notifier(struct notifier_block *nb,if (ops->add_device)return ops->add_device (dev); }else if (action ==BUS_NOTIFY_DEL_DEVICE ) { -if (ops->remove_device && dev->iommu_group) { +if (ops->remove_device) { ops->remove_device (dev);return 0 ; }include/linux/iommu.h +2
-2@@ -74,8 +74,8 @@ enum iommu_attr {* @unmap: unmap a physically contiguous memory region from an iommu domain * @iova_to_phys: translate iova to physical address * @domain_has_cap: domain capabilities query -* @add_device: add device to iommu grouping -* @remove_device: remove device from iommu grouping +* @add_device: add device to iommu grouping or domain +* @remove_device: remove device from iommu grouping or domain * @domain_get_attr: Query domain attributes * @domain_set_attr: Change domain attributes * @pgsize_bitmap: bitmap of supported page sizes
A message that is not in the archive (sent off-list or not yet received): 1358372869.11144.21.camel-xdHQ/5r00wBBDLzU/[email protected]
I mistakenly sent this with the tag [PATCH] when it should have been [RFC]. Updated with this reply.
Hi Alex,
Quoted 15 lines · Alex Williamson
Quoted 14 lines · Damian Hobson-Garcia
We've recently been thinking about the best way to attach platform devices in
an IOMMU in a generic way without having to have every platform device
register itself at probe time. We've seen that at least on the tegra IOMMU,
using theplatform_busnotifier seems like a viable option (http://lists.linuxfoundation.org/pipermail/iommu/2012-December/004954.html). We also noticed that the IOMMU opsadd_device()function is called (indirectly) from a similar bus
notifier and we're thinking that using the bus notifier in the common code
would be better than us rolling our own to do basically the same thing.The problem, however, is that
add_device()seems to be intended for use with IOMMU groups. i.e. After callingadd_device()on a device that is should be in a group,dev->iommu_groupshould have some meaningful value.
I think though, that if the IOMMU groups requirement can be removed, it should be more versatile.
Quoted 8 lines
Quoted 3 lines
I believe that removing the IOMMU groups dependency can be achieved with the patch below. Currently the intel-iommu seems to be the only implementation using these callbacks.
What stops you from creating a single
iommu_groupand adding all device to it? Thanks,Alex
Thanks for your comment.
Nothing stops me, but putting everything in one group doesn't really add any benefit over not using groups at all, does it? It seems like its just adding more complexity(i.e code) to the driver without adding any additional functionality.
Thanks,
Damian
Quoted 53 lines
Quoted 50 lines
--- drivers/iommu/intel-iommu.c | 3 ++- drivers/iommu/iommu.c | 2 +- include/linux/iommu.h | 4 ++-- 3 files changed, 5 insertions(+), 4 deletions(-)drivers/iommu/intel-iommu.c +2
-1@@ -4217,7 +4217,8 @@ root_bus:static void intel_iommu_remove_device (struct device *dev) { -iommu_group_remove_device (dev); +if (dev->iommu_group) +iommu_group_remove_device (dev); }static struct iommu_ops intel_iommu_ops = {drivers/iommu/iommu.c +1
-1@@ -510,7 +510,7 @@ static int iommu_bus_notifier(struct notifier_block *nb,if (ops->add_device)return ops->add_device (dev); }else if (action ==BUS_NOTIFY_DEL_DEVICE ) { -if (ops->remove_device && dev->iommu_group) { +if (ops->remove_device) { ops->remove_device (dev);return 0 ; }include/linux/iommu.h +2
-2@@ -74,8 +74,8 @@ enum iommu_attr {* @unmap: unmap a physically contiguous memory region from an iommu domain * @iova_to_phys: translate iova to physical address * @domain_has_cap: domain capabilities query -* @add_device: add device to iommu grouping -* @remove_device: remove device from iommu grouping +* @add_device: add device to iommu grouping or domain +* @remove_device: remove device from iommu grouping or domain * @domain_get_attr: Query domain attributes * @domain_set_attr: Change domain attributes * @pgsize_bitmap: bitmap of supported page sizes
Signature · 3 lines
Damian Hobson-Garcia IGEL Co.,Ltd http://www.igel.co.jp
A message that is not in the archive (sent off-list or not yet received): 50F76E12.7020607-AlSX/UN32fvPDbFq/[email protected]
Quoted 43 lines
-----Original Message-----
From: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org [mailto:iommu- bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org] On Behalf Of Damian Hobson-Garcia
Sent: Thursday, January 17, 2013 8:51 AM
To: Alex Williamson
Cc: hdk-AlSX/UN32fvPDbFq/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; laurent.pinchart-ryLnwIuWjnjg/[email protected]
Subject: Re: [RFC] drivers: iommu: Use of add_device()/remove_device()
outside of IOMMU groupsI mistakenly sent this with the tag [PATCH] when it should have been [RFC]. Updated with this reply.
Hi Alex,
Quoted 8 lines · Alex Williamson
Quoted 7 lines · Damian Hobson-Garcia
We've recently been thinking about the best way to attach platform devices in an IOMMU in a generic way without having to have every platform device register itself at probe time. We've seen that at least on the tegra IOMMU, using the
platform_busnotifier seems like a viable option
(http://lists.linuxfoundation.org/pipermail/iommu/2012-December/00495
4.html). We also noticed that the IOMMU opsadd_device()function iscalled (indirectly) from a similar bus notifier and we're thinking that using the bus notifier in the common code would be better than us rolling our own to do basically the same thing.
Quoted 4 lines
Quoted 4 lines
The problem, however, is that
add_device()seems to be intended for use with IOMMU groups. i.e. After callingadd_device()on a device that is should be in a group,dev->iommu_groupshould have somemeaningful value.
Quoted 10 lines
Quoted 6 lines
I think though, that if the IOMMU groups requirement can be removed, it should be more versatile.
I believe that removing the IOMMU groups dependency can be achieved with the patch below. Currently the intel-iommu seems to be the only implementation using these callbacks.
What stops you from creating a single
iommu_groupand adding all device to it? Thanks,
Quoted 8 lines
Quoted 1 line
Alex
Thanks for your comment.
Nothing stops me, but putting everything in one group doesn't really add any benefit over not using groups at all, does it? It seems like its just adding more complexity(i.e code) to the driver without adding any additional functionality.
Do you intend to use vfio? That's the specific objective of creating iommu groups.
-Varun
Quoted 27 lines · Damian Hobson-Garcia
I mistakenly sent this with the tag [PATCH] when it should have been [RFC]. Updated with this reply.
Hi Alex,
Quoted 22 lines · Alex Williamson
Quoted 14 lines · Damian Hobson-Garcia
We've recently been thinking about the best way to attach platform devices in
an IOMMU in a generic way without having to have every platform device
register itself at probe time. We've seen that at least on the tegra IOMMU,
using theplatform_busnotifier seems like a viable option (http://lists.linuxfoundation.org/pipermail/iommu/2012-December/004954.html). We also noticed that the IOMMU opsadd_device()function is called (indirectly) from a similar bus
notifier and we're thinking that using the bus notifier in the common code
would be better than us rolling our own to do basically the same thing.The problem, however, is that
add_device()seems to be intended for use with IOMMU groups. i.e. After callingadd_device()on a device that is should be in a group,dev->iommu_groupshould have some meaningful value.
I think though, that if the IOMMU groups requirement can be removed, it should be more versatile.Quoted 3 lines
I believe that removing the IOMMU groups dependency can be achieved with the patch below. Currently the intel-iommu seems to be the only implementation using these callbacks.
What stops you from creating a single
iommu_groupand adding all device to it? Thanks,
Quoted 7 lines
Quoted 1 line
Alex
Thanks for your comment.
Nothing stops me, but putting everything in one group doesn't really add any benefit over not using groups at all, does it? It seems like its just adding more complexity(i.e code) to the driver without adding any additional functionality.
Does your iommu provide any isolation or is it translate-only. As Varun notes, groups enable vfio, so if there's any possibility that you care about userspace drivers for your devices or assigning devices to qemu guests, then groups might be useful. The overhead of groups is really meant to be very minimal for the iommu driver. That said, I guess I have no real objection to generalizing the iommu bus notifier for add/remove. Thanks,
Alex
Quoted 54 lines
Quoted 52 lines
Quoted 49 lines
--- drivers/iommu/intel-iommu.c | 3 ++- drivers/iommu/iommu.c | 2 +- include/linux/iommu.h | 4 ++-- 3 files changed, 5 insertions(+), 4 deletions(-)drivers/iommu/intel-iommu.c +2
-1@@ -4217,7 +4217,8 @@ root_bus:static void intel_iommu_remove_device (struct device *dev) { -iommu_group_remove_device (dev); +if (dev->iommu_group) +iommu_group_remove_device (dev); }static struct iommu_ops intel_iommu_ops = {drivers/iommu/iommu.c +1
-1@@ -510,7 +510,7 @@ static int iommu_bus_notifier(struct notifier_block *nb,if (ops->add_device)return ops->add_device (dev); }else if (action ==BUS_NOTIFY_DEL_DEVICE ) { -if (ops->remove_device && dev->iommu_group) { +if (ops->remove_device) { ops->remove_device (dev);return 0 ; }include/linux/iommu.h +2
-2@@ -74,8 +74,8 @@ enum iommu_attr {* @unmap: unmap a physically contiguous memory region from an iommu domain * @iova_to_phys: translate iova to physical address * @domain_has_cap: domain capabilities query -* @add_device: add device to iommu grouping -* @remove_device: remove device from iommu grouping +* @add_device: add device to iommu grouping or domain +* @remove_device: remove device from iommu grouping or domain * @domain_get_attr: Query domain attributes * @domain_set_attr: Change domain attributes * @pgsize_bitmap: bitmap of supported page sizes
A message that is not in the archive (sent off-list or not yet received): 1358457711.9252.54.camel-85EaTFmN5p//[email protected]
Hi Alex,
Quoted 36 lines · Alex Williamson
Quoted 27 lines · Damian Hobson-Garcia
I mistakenly sent this with the tag [PATCH] when it should have been [RFC]. Updated with this reply.
Hi Alex,
Quoted 22 lines · Alex Williamson
Quoted 14 lines · Damian Hobson-Garcia
We've recently been thinking about the best way to attach platform devices in
an IOMMU in a generic way without having to have every platform device
register itself at probe time. We've seen that at least on the tegra IOMMU,
using theplatform_busnotifier seems like a viable option (http://lists.linuxfoundation.org/pipermail/iommu/2012-December/004954.html). We also noticed that the IOMMU opsadd_device()function is called (indirectly) from a similar bus
notifier and we're thinking that using the bus notifier in the common code
would be better than us rolling our own to do basically the same thing.The problem, however, is that
add_device()seems to be intended for use with IOMMU groups. i.e. After callingadd_device()on a device that is should be in a group,dev->iommu_groupshould have some meaningful value.
I think though, that if the IOMMU groups requirement can be removed, it should be more versatile.Quoted 3 lines
I believe that removing the IOMMU groups dependency can be achieved with the patch below. Currently the intel-iommu seems to be the only implementation using these callbacks.
What stops you from creating a single
iommu_groupand adding all device to it? Thanks,Quoted 7 lines
Quoted 1 line
Alex
Thanks for your comment.
Nothing stops me, but putting everything in one group doesn't really add any benefit over not using groups at all, does it? It seems like its just adding more complexity(i.e code) to the driver without adding any additional functionality.
Quoted 8 lines
Does your iommu provide any isolation or is it translate-only. As Varun notes, groups enable vfio, so if there's any possibility that you care about userspace drivers for your devices or assigning devices to qemu guests, then groups might be useful. The overhead of groups is really meant to be very minimal for the iommu driver. That said, I guess I have no real objection to generalizing the iommu bus notifier for add/remove. Thanks,
Our iommu is used for translation. We plan on using vfio at some point, though how well that fits with the current IOMMU ARM implementation remains to be seen. So considering your and Varun's comments, it looks like we want to be implementing groups after all, though generalizing the add/remove still seems to me like a good idea.
Thanks,
Damian
Quoted 59 lines
Alex
Quoted 54 lines
Quoted 52 lines
Quoted 49 lines
--- drivers/iommu/intel-iommu.c | 3 ++- drivers/iommu/iommu.c | 2 +- include/linux/iommu.h | 4 ++-- 3 files changed, 5 insertions(+), 4 deletions(-)drivers/iommu/intel-iommu.c +2
-1@@ -4217,7 +4217,8 @@ root_bus:static void intel_iommu_remove_device (struct device *dev) { -iommu_group_remove_device (dev); +if (dev->iommu_group) +iommu_group_remove_device (dev); }static struct iommu_ops intel_iommu_ops = {drivers/iommu/iommu.c +1
-1@@ -510,7 +510,7 @@ static int iommu_bus_notifier(struct notifier_block *nb,if (ops->add_device)return ops->add_device (dev); }else if (action ==BUS_NOTIFY_DEL_DEVICE ) { -if (ops->remove_device && dev->iommu_group) { +if (ops->remove_device) { ops->remove_device (dev);return 0 ; }include/linux/iommu.h +2
-2@@ -74,8 +74,8 @@ enum iommu_attr {* @unmap: unmap a physically contiguous memory region from an iommu domain * @iova_to_phys: translate iova to physical address * @domain_has_cap: domain capabilities query -* @add_device: add device to iommu grouping -* @remove_device: remove device from iommu grouping +* @add_device: add device to iommu grouping or domain +* @remove_device: remove device from iommu grouping or domain * @domain_get_attr: Query domain attributes * @domain_set_attr: Change domain attributes * @pgsize_bitmap: bitmap of supported page sizes
Signature · 3 lines
Damian Hobson-Garcia IGEL Co.,Ltd http://www.igel.co.jp