Date: Mon, 20 Mar 2023 15:07:13 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: Nicolin Chen <nicolinc@nvidia.com>
Cc: Robin Murphy <robin.murphy@arm.com>, will@kernel.org,
	eric.auger@redhat.com, kevin.tian@intel.com,
	baolu.lu@linux.intel.com, joro@8bytes.org,
	shameerali.kolothum.thodi@huawei.com, jean-philippe@linaro.org,
	linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v1 14/14] iommu/arm-smmu-v3: Add
 arm_smmu_cache_invalidate_user
Message-ID: <ZBig0fN3l7LsUlQg@nvidia.com>
References: <ZAx2K08L5TIm6r3y@Asurada-Nvidia>
 <ab762cc6-5adf-2515-d9d2-d21d916eb6ad@arm.com>
 <ZBJcS07G3mt7gjkA@Asurada-Nvidia>
 <c753c2a8-024d-5bef-8987-96582084991e@arm.com>
 <ZBOFcenanpRsGFzF@Asurada-Nvidia>
 <ZBe3kxRXf+VbKy+m@Asurada-Nvidia>
 <ZBhbmqprnux4q00i@nvidia.com>
 <ZBh7hSX5hdW4vxwh@Asurada-Nvidia>
 <ZBiDcYwxL7eV1EmQ@nvidia.com>
 <ZBiLSJvtY5UKDJ5l@Asurada-Nvidia>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZBiLSJvtY5UKDJ5l@Asurada-Nvidia>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?OxBTJlIeno4c6awe8+p8eiyzuUzw9/PUnSZUAiwRYcIr56ypHG/vcJ5IxdwV?=
 =?us-ascii?Q?KuCp8F8FYvlYf2r1KSueurchD2orY7movGmBSgunAi0O/tcf3DtJ6ASxXHl7?=
 =?us-ascii?Q?2Z5rl5a8WG+tV/GwlDwvKIldcJp2a5+XDW+PBq8a4MRdNEMtKqJHPHUCNuwz?=
 =?us-ascii?Q?HPSez0RMo4GFet4IRABC1pG5yWWlebn3aiLriMUDbfUCqn+E4Z0a+O7DCfP3?=
 =?us-ascii?Q?3s9EkT2vJbGVMc3GjQnwGpJd5XHk30KyrwGCErHCek1w3td3ZM7sqSpGpANq?=
 =?us-ascii?Q?BDeJw8HZmLOKR3ZRonFXMnlnjEQ4hOctU8DcWOxjqYnrDvwZu635ECHtSc5Q?=
 =?us-ascii?Q?53RWY1tHXbSHBH+L8V2yuE0KmTA+8o8IINz5cvmR2I1q14DzYulaXlkC28aO?=
 =?us-ascii?Q?F7tmLbaEaNHhG9H6hu575WfZfwLUpc6qwu/EJu66SADxAPPwZvuYeHiPuHkY?=
 =?us-ascii?Q?l1hUoU5rGS7AKqFb//1ytl96Vpar0Yq8FNcSUQAczsjOPVO9NvXaBgb8RMYF?=
 =?us-ascii?Q?yKN8pXG2yf3DADy1Kn7i6yhRNsqL95Tln+uiNi46P4hLxzgeVjiHeh594m/V?=
 =?us-ascii?Q?uiarcH0ycwWSokK04UeYh6ieGIIJAaX35nKFelPp8VqotDjQG5s+xaU1cvA8?=
 =?us-ascii?Q?H20h2nxcFXP91i6wYo4at+hjNTLXU4J9EF6KTeloFT/2JbnM5+YoxgX8dacE?=
 =?us-ascii?Q?+rwsTxThFEsuqNTWOowQcxKNRNW6+GVvwGbfK66B7WHBrQaZGrdlvhX1U4iv?=
 =?us-ascii?Q?gTClcDpb3vSCoOwqF5jm21/6hdLtYnGYS5aBqXR7b0pWsZ659OL2dDEzF+T9?=
 =?us-ascii?Q?Hkc7leCDSTeOo1+CqqCTnvZfB9UPfMXh6YF2If5Z+2glFyHb6pdhJmy/oDjx?=
 =?us-ascii?Q?53jEjSq5YUMN1vnZj0wIolTUiDFdkoBdoXfJTq1LuBcw75pAd4HhNE9D4pPk?=
 =?us-ascii?Q?sEysUeFYh63xBj6ZRWE74V+WHCv9olafuZSlOB+ODnGq2dEi4gj/6OqT5ySK?=
 =?us-ascii?Q?MrsyqBGdIqWVA+WhQczQb5ifhiCiHwK+vkR9V3puTJy18z4NUsWNiyDsVnfd?=
 =?us-ascii?Q?xAkLDINa63Hr/oz7FFNJbxQJlGNQHcIXDe9LrtCRLfoa6s9fDMZ5cLy1k+ry?=
 =?us-ascii?Q?ky9WKwAN2WlmnebqL0L5nMYPj62SpxQZIBgbk8SJbQv7j+jqTHZgGdlbtLiM?=
 =?us-ascii?Q?R4ts+hS9yOJeyIapUyEOg2+Xj654r+VYWoq9zYDvgoY1K48PmOunS3TyNz1+?=
 =?us-ascii?Q?n0P/Bnx0P+k4LTYk4sG6noIwZMW45XJB5i77WD7buyIhKnKDLGBLtlZZvmR5?=
 =?us-ascii?Q?kHxxh8y7F2VUec+veQOAILV0UemF2hBTGoDlKMAidoIqQQDDO4aO0r3MizQj?=
 =?us-ascii?Q?QG9eaDQ2NjtqIINwa1zDIHfw4WkLSF4LgzqNpRcJvMLPFaabuxu62F7prDHJ?=
 =?us-ascii?Q?fljz/kAg/kXBIsqytU11LsQHylhuUh4MJPC6x8KmmBsjW2d2f7nqpoVnWApP?=
 =?us-ascii?Q?DfroF/UCqq0HqJ6b6G8ROSenDDgEGVQgIMwZxrHA9YQkVsmhIzcCaQKvP2AU?=
 =?us-ascii?Q?DeCqbx2NdT/mOjTW1WybFZ7Q3cryMhRIC12efcTL?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d811e816-08db-4a05-7ea4-08db296def32
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Mar 2023 18:07:14.5857
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Aa1Y7NSqIv8n8vSQi9sFZlX/46riWkhnZZam06ZoUsHEDDzx3yzxBNfaFcfytM72
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5242

On Mon, Mar 20, 2023 at 09:35:20AM -0700, Nicolin Chen wrote:

> > You need to know what devices the vSID is targetting ang issues
> > invalidations only for those devices.
> 
> I agree with that, yet cannot think of a solution to achieve
> that out of vSID. QEMU code by means of emulating a physical
> SMMU only reads the commands from the queue, without knowing
> which device (vSID) actually sent these commands.

Huh?

CMD_ATC_INV has the SID

Other commands have the ASID.

You never need to cross an ASID to a SID or vice versa.

If the guest is aware of ATS it will issue CMD_ATC_INV with vSIDs, and
the hypervisor just needs to convert vSID to pSID.

Otherwise vSID doesn't matter because it isn't used in the invalidation
API and you are just handling ASIDs that only need the VM_ID scope
applied.

Jason