Skip to contentSkip to conversation

Enable VAS and NX-GZIP support on powerVM

PATCH Superseded Haren Myneni 4 participants 44 messages Apr 17, 2021 – May 11, 2021

Subsystems (4)

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v3 cover Enable VAS and NX-GZIP support on powerVM

This patch series enables VAS / NX-GZIP on powerVM which allows the user space to do copy/paste with the same existing interface that is available on powerNV.

VAS Enablement:
- Get all VAS capabilities using H_QUERY_VAS_CAPABILITIES that are available in the hypervisor. These capabilities tells OS which type of features (credit types such as Default and Quality of Service (QoS)). Also gives specific capabilities for each credit
type: Maximum window credits, Maximum LPAR credits, Target credits in that parition (varies from max LPAR credits based DLPAR operation), whether supports user mode COPY/PASTE and etc.
- Register LPAR VAS operations such as open window. get paste address and close window with the current VAS user space API.
- Open window operation - Use H_ALLOCATE_VAS_WINDOW HCALL to open window and H_MODIFY_VAS_WINDOW HCALL to setup the window with LPAR PID and etc.
- mmap to paste address returned in H_ALLOCATE_VAS_WINDOW HCALL
- To close window, H_DEALLOCATE_VAS_WINDOW HCALL is used to close in the hypervisor.

NX Enablement:
- Get NX capabilities from the the hypervisor which provides Maximum buffer length in a single GZIP request, recommended minimum compression / decompression lengths.
- Register to VAS to enable user space VAS API

Main feature differences with powerNV implementation:
- Each VAS window will be configured with a number of credits which means that many requests can be issues simultaniously on that window. On powerNV, 1K credits are configured per window. Whereas on powerVM, the hypervisor allows 1 credit per window at present.
- The hypervisor introduced 2 different types of credits: Default - Uses normal priority FIFO and Quality of Service (QoS) - Uses high priority FIFO. On powerVM, VAS/NX HW resources are shared across LPARs. The total number of credits available on a system depends on cores configured. We may see more credits are assigned across the system than the NX HW resources can handle. So to avoid NX HW contention, pHyp introduced QoS credits which can be configured by system administration with HMC API. Then the total number of available default credits on LPAR varies based on QoS credits configured.
- On powerNV, windows are allocated on a specific VAS instance and the user space can select VAS instance with the open window ioctl. Since VAS instances can be shared across partitions on powerVM, the hypervisor manages window allocations on different VAS instances. So H_ALLOCATE_VAS_WINDOW allows to select by domain indentifiers (H_HOME_NODE_ASSOCIATIVITY values by cpu). By default the hypervisor selects VAS instance closer to CPU resources that the parition uses. So vas_id in ioctl interface is ignored on powerVM except vas_id=-1 which is used to allocate window based on CPU that the process is executing. This option is needed for process affinity to NUMA node.

The existing applications that linked with libnxz should work as long as the job request length is restricted to req_max_processed_len.

Tested the following patches on P10 successfully with test cases
given: https://github.com/libnxz/power-gzip

Note: The hypervisor supports user mode NX from p10 onwards. Linux
supports user mode VAS/NX on P10 only with radix page tables.

Patches 1- 4: Move the code that is needed for both powerNV and
powerVM to powerpc book3s platform directory
Patch5: Modify vas-window struct to support both and the
related changes.
Patch 6: Define HCALL and the related VAS/NXGZIP specific
structs.

Patch 7:        Define QoS credit flag in window open ioctl
Patch 8:        Implement Allocate, Modify and Deallocate HCALLs
Patch 9:        Retrieve VAS capabilities from the hypervisor
Patch 10;       Implement window operations and integrate with API
Patch 11:       Setup IRQ and NX fault handling
Patch 12;       Add sysfs interface to expose VAS capabilities
Patch 13 - 14:  Make the code common to add NX-GZIP enablement
Patch 15:       Get NX capabilities from the hypervisor
patch 16;       Add sysfs interface to expose NX capabilities

Changes in V2:
- Rebase on 5.12-rc6
- Moved VAS Kconfig changes to arch/powerpc/platform as suggested by Christophe Leroy
- build fix with allyesconfig (reported by kernel test build)

Changes in V3:
- Rebase on 5.12-rc7
- Moved vas-api.c and VAS Kconfig changes to arch/powerpc/platform/book3s as Michael Ellerman suggested

Haren Myneni (16):
powerpc/powernv/vas: Rename register/unregister functions powerpc/vas: Make VAS API powerpc platform independent powerpc/vas: Create take/drop task reference functions powerpc/vas: Move update_csb/dump_crb to common book3s platform powerpc/vas: Define and use common vas_window struct powerpc/pseries/vas: Define VAS/NXGZIP HCALLs and structs powerpc/vas: Define QoS credit flag to allocate window powerpc/pseries/VAS: Implement allocate/modify/deallocate HCALLS powerpc/pseries/vas: Implement to get all capabilities powerpc/pseries/vas: Integrate API with open/close windows powerpc/pseries/vas: Setup IRQ and fault handling powerpc/pseries/vas: sysfs interface to export capabilities crypto/nx: Rename nx-842-pseries file name to nx-common-pseries crypto/nx: Register and unregister VAS interface crypto/nx: Get NX capabilities for GZIP coprocessor type crypto/nx: Add sysfs interface to export NX capabilities

 arch/powerpc/include/asm/hvcall.h             |   7 +
 arch/powerpc/include/asm/vas.h                | 122 +++-
 arch/powerpc/include/uapi/asm/vas-api.h       |   6 +-
 arch/powerpc/platforms/Kconfig                |   1 +
 arch/powerpc/platforms/Makefile               |   1 +
 arch/powerpc/platforms/book3s/Kconfig         |  15 +
 arch/powerpc/platforms/book3s/Makefile        |   2 +
 arch/powerpc/platforms/book3s/vas-api.c       | 485 +++++++++++++
 arch/powerpc/platforms/powernv/Kconfig        |  14 -
 arch/powerpc/platforms/powernv/Makefile       |   2 +-
 arch/powerpc/platforms/powernv/vas-api.c      | 278 --------
 arch/powerpc/platforms/powernv/vas-debug.c    |  12 +-
 arch/powerpc/platforms/powernv/vas-fault.c    | 155 +---
 arch/powerpc/platforms/powernv/vas-trace.h    |   6 +-
 arch/powerpc/platforms/powernv/vas-window.c   | 250 ++++---
 arch/powerpc/platforms/powernv/vas.h          |  42 +-
 arch/powerpc/platforms/pseries/Makefile       |   1 +
 arch/powerpc/platforms/pseries/vas-sysfs.c    | 173 +++++
 arch/powerpc/platforms/pseries/vas.c          | 674 ++++++++++++++++++
 arch/powerpc/platforms/pseries/vas.h          |  98 +++
 drivers/crypto/nx/Kconfig                     |   1 +
 drivers/crypto/nx/Makefile                    |   2 +-
 drivers/crypto/nx/nx-common-powernv.c         |   6 +-
 .../{nx-842-pseries.c => nx-common-pseries.c} | 135 ++++
 24 files changed, 1889 insertions(+), 599 deletions(-)
 create mode 100644 arch/powerpc/platforms/book3s/Kconfig
 create mode 100644 arch/powerpc/platforms/book3s/Makefile
 create mode 100644 arch/powerpc/platforms/book3s/vas-api.c
 delete mode 100644 arch/powerpc/platforms/powernv/vas-api.c
 create mode 100644 arch/powerpc/platforms/pseries/vas-sysfs.c
 create mode 100644 arch/powerpc/platforms/pseries/vas.c
 create mode 100644 arch/powerpc/platforms/pseries/vas.h
 rename drivers/crypto/nx/{nx-842-pseries.c => nx-common-pseries.c} (90%)
Signature · 1 line
2.18.2
#1 lore raw

PATCH v3 1/16 powerpc/powernv/vas: Rename register/unregister functions

powerNV and pseries drivers register / unregister to the corresponding VAS code separately. So rename powerNV VAS API register/unregister functions.

---
 arch/powerpc/include/asm/vas.h           |  6 +++---
 arch/powerpc/platforms/powernv/vas-api.c | 10 +++++-----
 drivers/crypto/nx/nx-common-powernv.c    |  6 +++---
 3 files changed, 11 insertions(+), 11 deletions(-)
arch/powerpc/include/asm/vas.h +3 -3
index e33f80b0ea81..41f73fae7ab8 100644
@@ -170,8 +170,8 @@ int vas_paste_crb(struct vas_window *win, int offset, bool re);
  * Only NX GZIP coprocessor type is supported now, but this API can be
  * used for others in future.
  */
-int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
-				const char *name);
-void vas_unregister_coproc_api(void);
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
+			     const char *name);
+void vas_unregister_api_powernv(void);
 
 #endif /* __ASM_POWERPC_VAS_H */
arch/powerpc/platforms/powernv/vas-api.c +5 -5
index 98ed5d8c5441..72d8ce39e56c 100644
@@ -207,8 +207,8 @@ static struct file_operations coproc_fops = {
  * Supporting only nx-gzip coprocessor type now, but this API code
  * extended to other coprocessor types later.
  */
-int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
-				const char *name)
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
+			     const char *name)
 {
 	int rc = -EINVAL;
 	dev_t devno;
@@ -262,9 +262,9 @@ int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
 	unregister_chrdev_region(coproc_device.devt, 1);
 	return rc;
 }
-EXPORT_SYMBOL_GPL(vas_register_coproc_api);
+EXPORT_SYMBOL_GPL(vas_register_api_powernv);
 
-void vas_unregister_coproc_api(void)
+void vas_unregister_api_powernv(void)
 {
 	dev_t devno;
 
@@ -275,4 +275,4 @@ void vas_unregister_coproc_api(void)
 	class_destroy(coproc_device.class);
 	unregister_chrdev_region(coproc_device.devt, 1);
 }
-EXPORT_SYMBOL_GPL(vas_unregister_coproc_api);
+EXPORT_SYMBOL_GPL(vas_unregister_api_powernv);
drivers/crypto/nx/nx-common-powernv.c +3 -3
index 13c65deda8e9..88d728415bb2 100644
@@ -1090,8 +1090,8 @@ static __init int nx_compress_powernv_init(void)
 		 * normal FIFO priority is assigned for userspace.
 		 * 842 compression is supported only in kernel.
 		 */
-		ret = vas_register_coproc_api(THIS_MODULE, VAS_COP_TYPE_GZIP,
-						"nx-gzip");
+		ret = vas_register_api_powernv(THIS_MODULE, VAS_COP_TYPE_GZIP,
+					       "nx-gzip");
 
 		/*
 		 * GZIP is not supported in kernel right now.
@@ -1127,7 +1127,7 @@ static void __exit nx_compress_powernv_exit(void)
 	 * use. So delete this API use for GZIP engine.
 	 */
 	if (!nx842_ct)
-		vas_unregister_coproc_api();
+		vas_unregister_api_powernv();
 
 	crypto_unregister_alg(&nx842_powernv_alg);
 
Signature · 1 line
2.18.2
reviewer Reviewed-by on v3 1/16
#2 lore raw

Re: [V3 PATCH 01/16] powerpc/powernv/vas: Rename register/unregister functions

Excerpts from Haren Myneni's message of April 18, 2021 7:00 am:

Quoted 4 lines

powerNV and pseries drivers register / unregister to the corresponding VAS code separately. So rename powerNV VAS API register/unregister functions.

The pseries VAS driver will have different calls for registering a coprocessor driver, you mean?

It certainly looks the same

(from patch 13)

	ret = vas_register_api_pseries(THIS_MODULE, VAS_COP_TYPE_GZIP,
				       "nx-gzip");

So I guess it's just a matter of the driver being different enough that there is no benefit to making this call common (and branching to pseries or powernv dynamically).

Quoted 86 lines
---
 arch/powerpc/include/asm/vas.h           |  6 +++---
 arch/powerpc/platforms/powernv/vas-api.c | 10 +++++-----
 drivers/crypto/nx/nx-common-powernv.c    |  6 +++---
 3 files changed, 11 insertions(+), 11 deletions(-)
arch/powerpc/include/asm/vas.h +3 -3
index e33f80b0ea81..41f73fae7ab8 100644
@@ -170,8 +170,8 @@ int vas_paste_crb(struct vas_window *win, int offset, bool re);
  * Only NX GZIP coprocessor type is supported now, but this API can be
  * used for others in future.
  */
-int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
-				const char *name);
-void vas_unregister_coproc_api(void);
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
+			     const char *name);
+void vas_unregister_api_powernv(void);
 
 #endif /* __ASM_POWERPC_VAS_H */
arch/powerpc/platforms/powernv/vas-api.c +5 -5
index 98ed5d8c5441..72d8ce39e56c 100644
@@ -207,8 +207,8 @@ static struct file_operations coproc_fops = {
  * Supporting only nx-gzip coprocessor type now, but this API code
  * extended to other coprocessor types later.
  */
-int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
-				const char *name)
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
+			     const char *name)
 {
 	int rc = -EINVAL;
 	dev_t devno;
@@ -262,9 +262,9 @@ int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
 	unregister_chrdev_region(coproc_device.devt, 1);
 	return rc;
 }
-EXPORT_SYMBOL_GPL(vas_register_coproc_api);
+EXPORT_SYMBOL_GPL(vas_register_api_powernv);
 
-void vas_unregister_coproc_api(void)
+void vas_unregister_api_powernv(void)
 {
 	dev_t devno;
 
@@ -275,4 +275,4 @@ void vas_unregister_coproc_api(void)
 	class_destroy(coproc_device.class);
 	unregister_chrdev_region(coproc_device.devt, 1);
 }
-EXPORT_SYMBOL_GPL(vas_unregister_coproc_api);
+EXPORT_SYMBOL_GPL(vas_unregister_api_powernv);
drivers/crypto/nx/nx-common-powernv.c +3 -3
index 13c65deda8e9..88d728415bb2 100644
@@ -1090,8 +1090,8 @@ static __init int nx_compress_powernv_init(void)
 		 * normal FIFO priority is assigned for userspace.
 		 * 842 compression is supported only in kernel.
 		 */
-		ret = vas_register_coproc_api(THIS_MODULE, VAS_COP_TYPE_GZIP,
-						"nx-gzip");
+		ret = vas_register_api_powernv(THIS_MODULE, VAS_COP_TYPE_GZIP,
+					       "nx-gzip");
 
 		/*
 		 * GZIP is not supported in kernel right now.
@@ -1127,7 +1127,7 @@ static void __exit nx_compress_powernv_exit(void)
 	 * use. So delete this API use for GZIP engine.
 	 */
 	if (!nx842_ct)
-		vas_unregister_coproc_api();
+		vas_unregister_api_powernv();
 
 	crypto_unregister_alg(&nx842_powernv_alg);
 
Signature · 1 line
2.18.2
#3 lore raw

Re: [V3 PATCH 01/16] powerpc/powernv/vas: Rename register/unregister functions

Quoted 12 lines · Nicholas Piggin

Excerpts from Haren Myneni's message of April 18, 2021 7:00 am:

Quoted 4 lines

powerNV and pseries drivers register / unregister to the corresponding
VAS code separately. So rename powerNV VAS API register/unregister functions.

The pseries VAS driver will have different calls for registering a coprocessor driver, you mean?

It certainly looks the same

(from patch 13)

Quoted 8 lines
	ret = vas_register_api_pseries(THIS_MODULE, VAS_COP_TYPE_GZIP,
				       "nx-gzip");

So I guess it's just a matter of the driver being different enough that
there is no benefit to making this call common (and branching to pseries
or powernv dynamically).

Thanks for your review and comments on all patches.

Yes, we have separate drivers nx-common-powernv/nx-common-pseries for powerNV and pseries.
API registeration patch is:
- driver calls platform specific API vas_register_api_powernv/pseries
- Platform specific code calls common API with its vas_user_win_ops vas_register_coproc_api()

Quoted 102 lines
Quoted 99 lines
---
 arch/powerpc/include/asm/vas.h           |  6 +++---
 arch/powerpc/platforms/powernv/vas-api.c | 10 +++++-----
 drivers/crypto/nx/nx-common-powernv.c    |  6 +++---
 3 files changed, 11 insertions(+), 11 deletions(-)
arch/powerpc/include/asm/vas.h +0 -0
index e33f80b0ea81..41f73fae7ab8 100644
@@ -170,8 +170,8 @@ int vas_paste_crb(struct vas_window *win, int
 offset, bool re);
  * Only NX GZIP coprocessor type is supported now, but this API

can be
* used for others in future.
*/
-int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,

arch/powerpc/include/asm/vas.h +1 -2
-				const char *name);
-void vas_unregister_coproc_api(void);
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type

cop_type,

arch/powerpc/include/asm/vas.h +2 -0
+			     const char *name);
+void vas_unregister_api_powernv(void);
 
 #endif /* __ASM_POWERPC_VAS_H */
arch/powerpc/platforms/powernv/vas-api.c +2 -2
index 98ed5d8c5441..72d8ce39e56c 100644
@@ -207,8 +207,8 @@ static struct file_operations coproc_fops = {
  * Supporting only nx-gzip coprocessor type now, but this API code
  * extended to other coprocessor types later.
  */
-int vas_register_coproc_api(struct module *mod, enum vas_cop_type
 cop_type,
-				const char *name)
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type
 cop_type,
+			     const char *name)
 {
arch/powerpc/platforms/powernv/vas-api.c +3 -3
 	int rc = -EINVAL;
 	dev_t devno;
@@ -262,9 +262,9 @@ int vas_register_coproc_api(struct module *mod,
 enum vas_cop_type cop_type,
 	unregister_chrdev_region(coproc_device.devt, 1);
 	return rc;
 }
-EXPORT_SYMBOL_GPL(vas_register_coproc_api);
+EXPORT_SYMBOL_GPL(vas_register_api_powernv);
 
-void vas_unregister_coproc_api(void)
+void vas_unregister_api_powernv(void)
 {
 	dev_t devno;
@@ -275,4 +275,4 @@ void vas_unregister_coproc_api(void)
 	class_destroy(coproc_device.class);
 	unregister_chrdev_region(coproc_device.devt, 1);
 }
-EXPORT_SYMBOL_GPL(vas_unregister_coproc_api);
+EXPORT_SYMBOL_GPL(vas_unregister_api_powernv);
drivers/crypto/nx/nx-common-powernv.c +2 -2
index 13c65deda8e9..88d728415bb2 100644
@@ -1090,8 +1090,8 @@ static __init int
 nx_compress_powernv_init(void)
 		 * normal FIFO priority is assigned for userspace.
 		 * 842 compression is supported only in kernel.
 		 */
-		ret = vas_register_coproc_api(THIS_MODULE,
 VAS_COP_TYPE_GZIP,
-						"nx-gzip");
+		ret = vas_register_api_powernv(THIS_MODULE,
 VAS_COP_TYPE_GZIP,
+					       "nx-gzip");
drivers/crypto/nx/nx-common-powernv.c +1 -1
 		/*
 		 * GZIP is not supported in kernel right now.
@@ -1127,7 +1127,7 @@ static void __exit
 nx_compress_powernv_exit(void)
 	 * use. So delete this API use for GZIP engine.
 	 */
 	if (!nx842_ct)
-		vas_unregister_coproc_api();
+		vas_unregister_api_powernv();
 
 	crypto_unregister_alg(&nx842_powernv_alg);
Signature · 2 lines
-- 
2.18.2
#4 lore raw

PATCH v3 2/16 powerpc/vas: Move VAS API to common book3s platform

In reply to Haren Myneni

Using the same /dev/crypto/nx-gzip interface for both powerNV and pseries. So this patch creates platforms/book3s/ and moves VAS API to that directory. The actual functionality is not changed.

Common interface functions such as open, window open ioctl, mmap and close are moved to arch/powerpc/platforms/book3s/vas-api.c Added hooks to call platform specific code, but the underline powerNV code in these functions is not changed.

---
 arch/powerpc/include/asm/vas.h                | 22 ++++++-
 arch/powerpc/platforms/Kconfig                |  1 +
 arch/powerpc/platforms/Makefile               |  1 +
 arch/powerpc/platforms/book3s/Kconfig         | 15 +++++
 arch/powerpc/platforms/book3s/Makefile        |  2 +
 .../platforms/{powernv => book3s}/vas-api.c   | 64 ++++++++++--------
 arch/powerpc/platforms/powernv/Kconfig        | 14 ----
 arch/powerpc/platforms/powernv/Makefile       |  2 +-
 arch/powerpc/platforms/powernv/vas-window.c   | 66 +++++++++++++++++++
 9 files changed, 143 insertions(+), 44 deletions(-)
 create mode 100644 arch/powerpc/platforms/book3s/Kconfig
 create mode 100644 arch/powerpc/platforms/book3s/Makefile
 rename arch/powerpc/platforms/{powernv => book3s}/vas-api.c (83%)
arch/powerpc/include/asm/vas.h +19 -3
index 41f73fae7ab8..6bbade60d8f4 100644
@@ -5,6 +5,8 @@
 
 #ifndef _ASM_POWERPC_VAS_H
 #define _ASM_POWERPC_VAS_H
+#include <uapi/asm/vas-api.h>
+
 
 struct vas_window;
 
@@ -48,6 +50,16 @@ enum vas_cop_type {
 	VAS_COP_TYPE_MAX,
 };
 
+/*
+ * User space window operations used for powernv and powerVM
+ */
+struct vas_user_win_ops {
+	struct vas_window * (*open_win)(struct vas_tx_win_open_attr *,
+				enum vas_cop_type);
+	u64 (*paste_addr)(void *);
+	int (*close_win)(void *);
+};
+
 /*
  * Receive window attributes specified by the (in-kernel) owner of window.
  */
@@ -161,6 +173,9 @@ int vas_copy_crb(void *crb, int offset);
  * assumed to be true for NX windows.
  */
 int vas_paste_crb(struct vas_window *win, int offset, bool re);
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
+			     const char *name);
+void vas_unregister_api_powernv(void);
 
 /*
  * Register / unregister coprocessor type to VAS API which will be exported
@@ -170,8 +185,9 @@ int vas_paste_crb(struct vas_window *win, int offset, bool re);
  * Only NX GZIP coprocessor type is supported now, but this API can be
  * used for others in future.
  */
-int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
-			     const char *name);
-void vas_unregister_api_powernv(void);
+int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
+			    const char *name,
+			    struct vas_user_win_ops *vops);
+void vas_unregister_coproc_api(void);
 
 #endif /* __ASM_POWERPC_VAS_H */
arch/powerpc/platforms/Kconfig +1 -0
index 7a5e8f4541e3..594544a65b02 100644
@@ -20,6 +20,7 @@ source "arch/powerpc/platforms/embedded6xx/Kconfig"
 source "arch/powerpc/platforms/44x/Kconfig"
 source "arch/powerpc/platforms/40x/Kconfig"
 source "arch/powerpc/platforms/amigaone/Kconfig"
+source "arch/powerpc/platforms/book3s/Kconfig"
 
 config KVM_GUEST
 	bool "KVM Guest support"
arch/powerpc/platforms/Makefile +1 -0
index 143d4417f6cc..0e75d7df387b 100644
@@ -22,3 +22,4 @@ obj-$(CONFIG_PPC_CELL)		+= cell/
 obj-$(CONFIG_PPC_PS3)		+= ps3/
 obj-$(CONFIG_EMBEDDED6xx)	+= embedded6xx/
 obj-$(CONFIG_AMIGAONE)		+= amigaone/
+obj-$(CONFIG_PPC_BOOK3S)	+= book3s/
arch/powerpc/platforms/book3s/Kconfig new file +15 -0
new file mode 100644
index 000000000000..51e14db83a79
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0
+config PPC_VAS
+	bool "IBM Virtual Accelerator Switchboard (VAS)"
+	depends on PPC_POWERNV && PPC_64K_PAGES
+	default y
+	help
+	  This enables support for IBM Virtual Accelerator Switchboard (VAS).
+
+	  VAS allows accelerators in co-processors like NX-GZIP and NX-842
+	  to be accessible to kernel subsystems and user processes.
+	  VAS adapters are found in POWER9 and later based systems.
+	  The user mode NX-GZIP support is added on P9 for powerNV and on
+	  P10 for powerVM.
+
+	  If unsure, say "N".
arch/powerpc/platforms/book3s/Makefile new file +2 -0
new file mode 100644
index 000000000000..e790f1910f61
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_PPC_VAS)	+= vas-api.o
arch/powerpc/platforms/book3s/vas-api.c renamed from arch/powerpc/platforms/powernv/vas-api.c +38 -26

140 lines not on this page · raw message

arch/powerpc/platforms/powernv/Kconfig +0 -14
index 619b093a0657..043eefbbdd28 100644
@@ -33,20 +33,6 @@ config PPC_MEMTRACE
 	  Enabling this option allows for runtime allocation of memory (RAM)
 	  for hardware tracing.
 
-config PPC_VAS
-	bool "IBM Virtual Accelerator Switchboard (VAS)"
-	depends on PPC_POWERNV && PPC_64K_PAGES
-	default y
-	help
-	  This enables support for IBM Virtual Accelerator Switchboard (VAS).
-
-	  VAS allows accelerators in co-processors like NX-GZIP and NX-842
-	  to be accessible to kernel subsystems and user processes.
-
-	  VAS adapters are found in POWER9 based systems.
-
-	  If unsure, say N.
-
 config SCOM_DEBUGFS
 	bool "Expose SCOM controllers via debugfs"
 	depends on DEBUG_FS
arch/powerpc/platforms/powernv/Makefile +1 -1
index 2eb6ae150d1f..c747a1f1d25b 100644
@@ -18,7 +18,7 @@ obj-$(CONFIG_MEMORY_FAILURE)	+= opal-memory-errors.o
 obj-$(CONFIG_OPAL_PRD)	+= opal-prd.o
 obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
 obj-$(CONFIG_PPC_MEMTRACE)	+= memtrace.o
-obj-$(CONFIG_PPC_VAS)	+= vas.o vas-window.o vas-debug.o vas-fault.o vas-api.o
+obj-$(CONFIG_PPC_VAS)	+= vas.o vas-window.o vas-debug.o vas-fault.o
 obj-$(CONFIG_OCXL_BASE)	+= ocxl.o
 obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
 obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
arch/powerpc/platforms/powernv/vas-window.c +66 -0

75 lines not on this page · raw message

Signature · 1 line
2.18.2
#5 lore raw

Re: [PATCH V3 02/16] powerpc/vas: Move VAS API to common book3s platform

Excerpts from Haren Myneni's message of April 18, 2021 7:02 am:

Quoted 3 lines

Using the same /dev/crypto/nx-gzip interface for both powerNV and pseries.

The pseries NX driver will use the powernv VAS API ?

Quoted 7 lines

So this patch creates platforms/book3s/ and moves VAS API to that directory. The actual functionality is not changed.

Common interface functions such as open, window open ioctl, mmap and close are moved to arch/powerpc/platforms/book3s/vas-api.c Added hooks to call platform specific code, but the underline powerNV code in these functions is not changed.

Even so, could you do one patch that just moves, and another that adds the ops struct?

Quoted 8 lines
---
 arch/powerpc/include/asm/vas.h                | 22 ++++++-
 arch/powerpc/platforms/Kconfig                |  1 +
 arch/powerpc/platforms/Makefile               |  1 +
 arch/powerpc/platforms/book3s/Kconfig         | 15 +++++
 arch/powerpc/platforms/book3s/Makefile        |  2 +

The usual place for these would be arch/powerpc/sysdev/vas. E.g., see arch/powerpc/sysdev/xive.

Quoted 393 lines
 .../platforms/{powernv => book3s}/vas-api.c   | 64 ++++++++++--------
 arch/powerpc/platforms/powernv/Kconfig        | 14 ----
 arch/powerpc/platforms/powernv/Makefile       |  2 +-
 arch/powerpc/platforms/powernv/vas-window.c   | 66 +++++++++++++++++++
 9 files changed, 143 insertions(+), 44 deletions(-)
 create mode 100644 arch/powerpc/platforms/book3s/Kconfig
 create mode 100644 arch/powerpc/platforms/book3s/Makefile
 rename arch/powerpc/platforms/{powernv => book3s}/vas-api.c (83%)
arch/powerpc/include/asm/vas.h +19 -3
index 41f73fae7ab8..6bbade60d8f4 100644
@@ -5,6 +5,8 @@
 
 #ifndef _ASM_POWERPC_VAS_H
 #define _ASM_POWERPC_VAS_H
+#include <uapi/asm/vas-api.h>
+
 
 struct vas_window;
 
@@ -48,6 +50,16 @@ enum vas_cop_type {
 	VAS_COP_TYPE_MAX,
 };
 
+/*
+ * User space window operations used for powernv and powerVM
+ */
+struct vas_user_win_ops {
+	struct vas_window * (*open_win)(struct vas_tx_win_open_attr *,
+				enum vas_cop_type);
+	u64 (*paste_addr)(void *);
+	int (*close_win)(void *);
+};
+
 /*
  * Receive window attributes specified by the (in-kernel) owner of window.
  */
@@ -161,6 +173,9 @@ int vas_copy_crb(void *crb, int offset);
  * assumed to be true for NX windows.
  */
 int vas_paste_crb(struct vas_window *win, int offset, bool re);
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
+			     const char *name);
+void vas_unregister_api_powernv(void);
 
 /*
  * Register / unregister coprocessor type to VAS API which will be exported
@@ -170,8 +185,9 @@ int vas_paste_crb(struct vas_window *win, int offset, bool re);
  * Only NX GZIP coprocessor type is supported now, but this API can be
  * used for others in future.
  */
-int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
-			     const char *name);
-void vas_unregister_api_powernv(void);
+int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
+			    const char *name,
+			    struct vas_user_win_ops *vops);
+void vas_unregister_coproc_api(void);
 
 #endif /* __ASM_POWERPC_VAS_H */
arch/powerpc/platforms/Kconfig +1 -0
index 7a5e8f4541e3..594544a65b02 100644
@@ -20,6 +20,7 @@ source "arch/powerpc/platforms/embedded6xx/Kconfig"
 source "arch/powerpc/platforms/44x/Kconfig"
 source "arch/powerpc/platforms/40x/Kconfig"
 source "arch/powerpc/platforms/amigaone/Kconfig"
+source "arch/powerpc/platforms/book3s/Kconfig"
 
 config KVM_GUEST
 	bool "KVM Guest support"
arch/powerpc/platforms/Makefile +1 -0
index 143d4417f6cc..0e75d7df387b 100644
@@ -22,3 +22,4 @@ obj-$(CONFIG_PPC_CELL)		+= cell/
 obj-$(CONFIG_PPC_PS3)		+= ps3/
 obj-$(CONFIG_EMBEDDED6xx)	+= embedded6xx/
 obj-$(CONFIG_AMIGAONE)		+= amigaone/
+obj-$(CONFIG_PPC_BOOK3S)	+= book3s/
arch/powerpc/platforms/book3s/Kconfig new file +15 -0
new file mode 100644
index 000000000000..51e14db83a79
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0
+config PPC_VAS
+	bool "IBM Virtual Accelerator Switchboard (VAS)"
+	depends on PPC_POWERNV && PPC_64K_PAGES
+	default y
+	help
+	  This enables support for IBM Virtual Accelerator Switchboard (VAS).
+
+	  VAS allows accelerators in co-processors like NX-GZIP and NX-842
+	  to be accessible to kernel subsystems and user processes.
+	  VAS adapters are found in POWER9 and later based systems.
+	  The user mode NX-GZIP support is added on P9 for powerNV and on
+	  P10 for powerVM.
+
+	  If unsure, say "N".
arch/powerpc/platforms/book3s/Makefile new file +2 -0
new file mode 100644
index 000000000000..e790f1910f61
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_PPC_VAS)	+= vas-api.o
arch/powerpc/platforms/book3s/vas-api.c renamed from arch/powerpc/platforms/powernv/vas-api.c +38 -26
similarity index 83%
rename from arch/powerpc/platforms/powernv/vas-api.c
rename to arch/powerpc/platforms/book3s/vas-api.c
index 72d8ce39e56c..05d7b99acf41 100644
@@ -4,15 +4,20 @@
  * Copyright (C) 2019 Haren Myneni, IBM Corp
  */
 
+#include <linux/module.h>
 #include <linux/kernel.h>
 #include <linux/device.h>
 #include <linux/cdev.h>
 #include <linux/fs.h>
 #include <linux/slab.h>
 #include <linux/uaccess.h>
+#include <linux/kthread.h>
+#include <linux/sched/signal.h>
+#include <linux/sched/mm.h>
+#include <linux/mmu_context.h>
 #include <asm/vas.h>
+#include <asm/icswx.h>
 #include <uapi/asm/vas-api.h>
-#include "vas.h"
 
 /*
  * The driver creates the device node that can be used as follows:
@@ -42,6 +47,7 @@ static struct coproc_dev {
 	dev_t devt;
 	struct class *class;
 	enum vas_cop_type cop_type;
+	struct vas_user_win_ops *vops;
 } coproc_device;
 
 struct coproc_instance {
@@ -72,11 +78,10 @@ static int coproc_open(struct inode *inode, struct file *fp)
 static int coproc_ioc_tx_win_open(struct file *fp, unsigned long arg)
 {
 	void __user *uptr = (void __user *)arg;
-	struct vas_tx_win_attr txattr = {};
 	struct vas_tx_win_open_attr uattr;
 	struct coproc_instance *cp_inst;
 	struct vas_window *txwin;
-	int rc, vasid;
+	int rc;
 
 	cp_inst = fp->private_data;
 
@@ -93,27 +98,20 @@ static int coproc_ioc_tx_win_open(struct file *fp, unsigned long arg)
 	}
 
 	if (uattr.version != 1) {
-		pr_err("Invalid version\n");
+		pr_err("Invalid window open API version\n");
 		return -EINVAL;
 	}
 
-	vasid = uattr.vas_id;
-
-	vas_init_tx_win_attr(&txattr, cp_inst->coproc->cop_type);
-
-	txattr.lpid = mfspr(SPRN_LPID);
-	txattr.pidr = mfspr(SPRN_PID);
-	txattr.user_win = true;
-	txattr.rsvd_txbuf_count = false;
-	txattr.pswid = false;
-
-	pr_devel("Pid %d: Opening txwin, PIDR %ld\n", txattr.pidr,
-				mfspr(SPRN_PID));
+	if (!cp_inst->coproc->vops && !cp_inst->coproc->vops->open_win) {
+		pr_err("VAS API is not registered\n");
+		return -EACCES;
+	}
 
-	txwin = vas_tx_win_open(vasid, cp_inst->coproc->cop_type, &txattr);
+	txwin = cp_inst->coproc->vops->open_win(&uattr,
+					cp_inst->coproc->cop_type);
 	if (IS_ERR(txwin)) {
-		pr_err("%s() vas_tx_win_open() failed, %ld\n", __func__,
-					PTR_ERR(txwin));
+		pr_err("%s() VAS window open failed, %ld\n", __func__,
+				PTR_ERR(txwin));
 		return PTR_ERR(txwin);
 	}
 
@@ -125,9 +123,14 @@ static int coproc_ioc_tx_win_open(struct file *fp, unsigned long arg)
 static int coproc_release(struct inode *inode, struct file *fp)
 {
 	struct coproc_instance *cp_inst = fp->private_data;
+	int rc = 0;
 
 	if (cp_inst->txwin) {
-		vas_win_close(cp_inst->txwin);
+		if (cp_inst->coproc->vops && cp_inst->coproc->vops->close_win) {
+			rc = cp_inst->coproc->vops->close_win(cp_inst->txwin);
+			if (rc)
+				return rc;
+		}
 		cp_inst->txwin = NULL;
 	}
 
@@ -168,7 +171,17 @@ static int coproc_mmap(struct file *fp, struct vm_area_struct *vma)
 		return -EINVAL;
 	}
 
-	vas_win_paste_addr(txwin, &paste_addr, NULL);
+	if (!cp_inst->coproc->vops && !cp_inst->coproc->vops->paste_addr) {
+		pr_err("%s(): VAS API is not registered\n", __func__);
+		return -EACCES;
+	}
+
+	paste_addr = cp_inst->coproc->vops->paste_addr(txwin);
+	if (!paste_addr) {
+		pr_err("%s(): Window paste address failed\n", __func__);
+		return -EINVAL;
+	}
+
 	pfn = paste_addr >> PAGE_SHIFT;
 
 	/* flags, page_prot from cxl_mmap(), except we want cachable */
@@ -207,8 +220,8 @@ static struct file_operations coproc_fops = {
  * Supporting only nx-gzip coprocessor type now, but this API code
  * extended to other coprocessor types later.
  */
-int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
-			     const char *name)
+int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
+			const char *name, struct vas_user_win_ops *vops)
 {
 	int rc = -EINVAL;
 	dev_t devno;
@@ -230,6 +243,7 @@ int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
 	}
 	coproc_device.class->devnode = coproc_devnode;
 	coproc_device.cop_type = cop_type;
+	coproc_device.vops = vops;
 
 	coproc_fops.owner = mod;
 	cdev_init(&coproc_device.cdev, &coproc_fops);
@@ -262,9 +276,8 @@ int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
 	unregister_chrdev_region(coproc_device.devt, 1);
 	return rc;
 }
-EXPORT_SYMBOL_GPL(vas_register_api_powernv);
 
-void vas_unregister_api_powernv(void)
+void vas_unregister_coproc_api(void)
 {
 	dev_t devno;
 
@@ -275,4 +288,3 @@ void vas_unregister_api_powernv(void)
 	class_destroy(coproc_device.class);
 	unregister_chrdev_region(coproc_device.devt, 1);
 }
-EXPORT_SYMBOL_GPL(vas_unregister_api_powernv);
arch/powerpc/platforms/powernv/Kconfig +0 -14
index 619b093a0657..043eefbbdd28 100644
@@ -33,20 +33,6 @@ config PPC_MEMTRACE
 	  Enabling this option allows for runtime allocation of memory (RAM)
 	  for hardware tracing.
 
-config PPC_VAS
-	bool "IBM Virtual Accelerator Switchboard (VAS)"
-	depends on PPC_POWERNV && PPC_64K_PAGES
-	default y
-	help
-	  This enables support for IBM Virtual Accelerator Switchboard (VAS).
-
-	  VAS allows accelerators in co-processors like NX-GZIP and NX-842
-	  to be accessible to kernel subsystems and user processes.
-
-	  VAS adapters are found in POWER9 based systems.
-
-	  If unsure, say N.
-
 config SCOM_DEBUGFS
 	bool "Expose SCOM controllers via debugfs"
 	depends on DEBUG_FS
arch/powerpc/platforms/powernv/Makefile +1 -1
index 2eb6ae150d1f..c747a1f1d25b 100644
@@ -18,7 +18,7 @@ obj-$(CONFIG_MEMORY_FAILURE)	+= opal-memory-errors.o
 obj-$(CONFIG_OPAL_PRD)	+= opal-prd.o
 obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
 obj-$(CONFIG_PPC_MEMTRACE)	+= memtrace.o
-obj-$(CONFIG_PPC_VAS)	+= vas.o vas-window.o vas-debug.o vas-fault.o vas-api.o
+obj-$(CONFIG_PPC_VAS)	+= vas.o vas-window.o vas-debug.o vas-fault.o
 obj-$(CONFIG_OCXL_BASE)	+= ocxl.o
 obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
 obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
arch/powerpc/platforms/powernv/vas-window.c +66 -0
index 5f5fe63a3d1c..b973dd574b47 100644
@@ -16,6 +16,8 @@
 #include <linux/mmu_context.h>
 #include <asm/switch_to.h>
 #include <asm/ppc-opcode.h>
+#include <asm/vas.h>
+#include <uapi/asm/vas-api.h>
 #include "vas.h"
 #include "copy-paste.h"
 
@@ -1441,3 +1443,67 @@ struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
 
 	return window;
 }
+
+static struct vas_window *vas_user_win_open(struct vas_tx_win_open_attr *uattr,
+				enum vas_cop_type cop_type)
+{
+	struct vas_tx_win_attr txattr = {};
+
+	vas_init_tx_win_attr(&txattr, cop_type);
+
+	txattr.lpid = mfspr(SPRN_LPID);
+	txattr.pidr = mfspr(SPRN_PID);
+	txattr.user_win = true;
+	txattr.rsvd_txbuf_count = false;
+	txattr.pswid = false;
+
+	pr_devel("Pid %d: Opening txwin, PIDR %ld\n", txattr.pidr,
+				mfspr(SPRN_PID));
+
+	return vas_tx_win_open(uattr->vas_id, cop_type, &txattr);
+}
+
+static u64 vas_user_win_paste_addr(void *addr)
+{
+	u64 paste_addr;
+
+	vas_win_paste_addr((struct vas_window *)addr, &paste_addr, NULL);
+
+	return paste_addr;
+}
+
+static int vas_user_win_close(void *addr)
+{
+	struct vas_window *txwin = addr;
+
+	vas_win_close(txwin);
+
+	return 0;
+}
+
+static struct vas_user_win_ops vops =  {
+	.open_win	=	vas_user_win_open,
+	.paste_addr	=	vas_user_win_paste_addr,
+	.close_win	=	vas_user_win_close,
+};
+
+/*
+ * Supporting only nx-gzip coprocessor type now, but this API code
+ * extended to other coprocessor types later.
+ */
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
+				const char *name)
+{
+	int rc;
+
+	rc = vas_register_coproc_api(mod, cop_type, name, &vops);
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(vas_register_api_powernv);
+
+void vas_unregister_api_powernv(void)
+{
+	vas_unregister_coproc_api();
+}
+EXPORT_SYMBOL_GPL(vas_unregister_api_powernv);
Signature · 1 line
2.18.2
#6 lore raw

Re: [PATCH V3 02/16] powerpc/vas: Move VAS API to common book3s platform

Quoted 5 lines · Nicholas Piggin

Excerpts from Haren Myneni's message of April 18, 2021 7:02 am:

Quoted 2 lines

Using the same /dev/crypto/nx-gzip interface for both powerNV and pseries.

The pseries NX driver will use the powernv VAS API ?

Yes, both powerNV and pseries drivers use same VAS API.

Quoted 11 lines
Quoted 8 lines

So this patch creates platforms/book3s/ and moves VAS API to that directory. The actual functionality is not changed.

Common interface functions such as open, window open ioctl, mmap and close are moved to arch/powerpc/platforms/book3s/vas-api.c Added hooks to call platform specific code, but the underline powerNV code in these functions is not changed.

Even so, could you do one patch that just moves, and another that adds the ops struct?

Sure, I will create separate patches.

Quoted 11 lines
Quoted 8 lines
---
 arch/powerpc/include/asm/vas.h                | 22 ++++++-
 arch/powerpc/platforms/Kconfig                |  1 +
 arch/powerpc/platforms/Makefile               |  1 +
 arch/powerpc/platforms/book3s/Kconfig         | 15 +++++
 arch/powerpc/platforms/book3s/Makefile        |  2 +

The usual place for these would be arch/powerpc/sysdev/vas. E.g., see arch/powerpc/sysdev/xive.

It was recommended to create book3s and move common vas-api code.

Quoted 443 lines
Quoted 443 lines
 .../platforms/{powernv => book3s}/vas-api.c   | 64 ++++++++++-----
---
 arch/powerpc/platforms/powernv/Kconfig        | 14 ----
 arch/powerpc/platforms/powernv/Makefile       |  2 +-
 arch/powerpc/platforms/powernv/vas-window.c   | 66
+++++++++++++++++++
 9 files changed, 143 insertions(+), 44 deletions(-)
 create mode 100644 arch/powerpc/platforms/book3s/Kconfig
 create mode 100644 arch/powerpc/platforms/book3s/Makefile
 rename arch/powerpc/platforms/{powernv => book3s}/vas-api.c (83%)
arch/powerpc/include/asm/vas.h +12 -0
index 41f73fae7ab8..6bbade60d8f4 100644
@@ -5,6 +5,8 @@
 
 #ifndef _ASM_POWERPC_VAS_H
 #define _ASM_POWERPC_VAS_H
+#include <uapi/asm/vas-api.h>
+
 
 struct vas_window;
 
@@ -48,6 +50,16 @@ enum vas_cop_type {
 	VAS_COP_TYPE_MAX,
 };
 
+/*
+ * User space window operations used for powernv and powerVM
+ */
+struct vas_user_win_ops {
+	struct vas_window * (*open_win)(struct vas_tx_win_open_attr *,
+				enum vas_cop_type);
+	u64 (*paste_addr)(void *);
+	int (*close_win)(void *);
+};
+
 /*
  * Receive window attributes specified by the (in-kernel) owner of

window.
*/

arch/powerpc/include/asm/vas.h +3 -0
@@ -161,6 +173,9 @@ int vas_copy_crb(void *crb, int offset);
  * assumed to be true for NX windows.
  */
 int vas_paste_crb(struct vas_window *win, int offset, bool re);
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type
 cop_type,
+			     const char *name);
+void vas_unregister_api_powernv(void);
 
 /*

* Register / unregister coprocessor type to VAS API which will be
exported

arch/powerpc/include/asm/vas.h +0 -0
@@ -170,8 +185,9 @@ int vas_paste_crb(struct vas_window *win, int
 offset, bool re);
  * Only NX GZIP coprocessor type is supported now, but this API

can be
* used for others in future.
*/
-int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,

arch/powerpc/include/asm/vas.h +1 -2
-			     const char *name);
-void vas_unregister_api_powernv(void);
+int vas_register_coproc_api(struct module *mod, enum vas_cop_type

cop_type,

arch/powerpc/include/asm/vas.h +3 -0
+			    const char *name,
+			    struct vas_user_win_ops *vops);
+void vas_unregister_coproc_api(void);
 
 #endif /* __ASM_POWERPC_VAS_H */
arch/powerpc/platforms/Kconfig +0 -0
index 7a5e8f4541e3..594544a65b02 100644
@@ -20,6 +20,7 @@ source

"arch/powerpc/platforms/embedded6xx/Kconfig"
source "arch/powerpc/platforms/44x/Kconfig"
source "arch/powerpc/platforms/40x/Kconfig"
source "arch/powerpc/platforms/amigaone/Kconfig"

arch/powerpc/platforms/Kconfig +1 -0
+source "arch/powerpc/platforms/book3s/Kconfig"
 
 config KVM_GUEST
 	bool "KVM Guest support"
arch/powerpc/platforms/Makefile +1 -0
index 143d4417f6cc..0e75d7df387b 100644
@@ -22,3 +22,4 @@ obj-$(CONFIG_PPC_CELL)		+= cell/
 obj-$(CONFIG_PPC_PS3)		+= ps3/
 obj-$(CONFIG_EMBEDDED6xx)	+= embedded6xx/
 obj-$(CONFIG_AMIGAONE)		+= amigaone/
+obj-$(CONFIG_PPC_BOOK3S)	+= book3s/
arch/powerpc/platforms/book3s/Kconfig new file +7 -0
new file mode 100644
index 000000000000..51e14db83a79
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0
+config PPC_VAS
+	bool "IBM Virtual Accelerator Switchboard (VAS)"
+	depends on PPC_POWERNV && PPC_64K_PAGES
+	default y
+	help
+	  This enables support for IBM Virtual Accelerator Switchboard

(VAS).

arch/powerpc/platforms/book3s/Kconfig new file +2 -0
+
+	  VAS allows accelerators in co-processors like NX-GZIP and NX-

842

arch/powerpc/platforms/book3s/Kconfig new file +3 -0
+	  to be accessible to kernel subsystems and user processes.
+	  VAS adapters are found in POWER9 and later based systems.
+	  The user mode NX-GZIP support is added on P9 for powerNV and

on

arch/powerpc/platforms/book3s/Kconfig new file +3 -0
+	  P10 for powerVM.
+
+	  If unsure, say "N".
arch/powerpc/platforms/book3s/Makefile new file +2 -0
new file mode 100644
index 000000000000..e790f1910f61
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_PPC_VAS)	+= vas-api.o
arch/powerpc/platforms/book3s/vas-api.c renamed from arch/powerpc/platforms/powernv/vas-api.c +8 -3
similarity index 83%
rename from arch/powerpc/platforms/powernv/vas-api.c
rename to arch/powerpc/platforms/book3s/vas-api.c
index 72d8ce39e56c..05d7b99acf41 100644
@@ -4,15 +4,20 @@
  * Copyright (C) 2019 Haren Myneni, IBM Corp
  */
 
+#include <linux/module.h>
 #include <linux/kernel.h>
 #include <linux/device.h>
 #include <linux/cdev.h>
 #include <linux/fs.h>
 #include <linux/slab.h>
 #include <linux/uaccess.h>
+#include <linux/kthread.h>
+#include <linux/sched/signal.h>
+#include <linux/sched/mm.h>
+#include <linux/mmu_context.h>
 #include <asm/vas.h>
+#include <asm/icswx.h>
 #include <uapi/asm/vas-api.h>
-#include "vas.h"
 
 /*
  * The driver creates the device node that can be used as follows:
@@ -42,6 +47,7 @@ static struct coproc_dev {
 	dev_t devt;
 	struct class *class;
 	enum vas_cop_type cop_type;
+	struct vas_user_win_ops *vops;
 } coproc_device;
 
 struct coproc_instance {
@@ -72,11 +78,10 @@ static int coproc_open(struct inode *inode,
 struct file *fp)
 static int coproc_ioc_tx_win_open(struct file *fp, unsigned long
 arg)
 {
 	void __user *uptr = (void __user *)arg;
-	struct vas_tx_win_attr txattr = {};
 	struct vas_tx_win_open_attr uattr;
 	struct coproc_instance *cp_inst;
 	struct vas_window *txwin;
-	int rc, vasid;
+	int rc;
 
arch/powerpc/platforms/book3s/vas-api.c renamed from arch/powerpc/platforms/powernv/vas-api.c +15 -17
 	cp_inst = fp->private_data;
@@ -93,27 +98,20 @@ static int coproc_ioc_tx_win_open(struct file
 *fp, unsigned long arg)
 	}
 
 	if (uattr.version != 1) {
-		pr_err("Invalid version\n");
+		pr_err("Invalid window open API version\n");
 		return -EINVAL;
 	}
 
-	vasid = uattr.vas_id;
-
-	vas_init_tx_win_attr(&txattr, cp_inst->coproc->cop_type);
-
-	txattr.lpid = mfspr(SPRN_LPID);
-	txattr.pidr = mfspr(SPRN_PID);
-	txattr.user_win = true;
-	txattr.rsvd_txbuf_count = false;
-	txattr.pswid = false;
-
-	pr_devel("Pid %d: Opening txwin, PIDR %ld\n", txattr.pidr,
-				mfspr(SPRN_PID));
+	if (!cp_inst->coproc->vops && !cp_inst->coproc->vops->open_win) 
 {
+		pr_err("VAS API is not registered\n");
+		return -EACCES;
+	}
 
-	txwin = vas_tx_win_open(vasid, cp_inst->coproc->cop_type,
 &txattr);
+	txwin = cp_inst->coproc->vops->open_win(&uattr,
+					cp_inst->coproc->cop_type);
 	if (IS_ERR(txwin)) {
-		pr_err("%s() vas_tx_win_open() failed, %ld\n",
 __func__,
-					PTR_ERR(txwin));
+		pr_err("%s() VAS window open failed, %ld\n", __func__,
+				PTR_ERR(txwin));
 		return PTR_ERR(txwin);
 	}
@@ -125,9 +123,14 @@ static int coproc_ioc_tx_win_open(struct file
 *fp, unsigned long arg)
 static int coproc_release(struct inode *inode, struct file *fp)
 {
 	struct coproc_instance *cp_inst = fp->private_data;
+	int rc = 0;
 
 	if (cp_inst->txwin) {
-		vas_win_close(cp_inst->txwin);
+		if (cp_inst->coproc->vops && cp_inst->coproc->vops-
 >close_win) {
+			rc = cp_inst->coproc->vops->close_win(cp_inst-
 >txwin);
+			if (rc)
+				return rc;
+		}
arch/powerpc/platforms/book3s/vas-api.c renamed from arch/powerpc/platforms/powernv/vas-api.c +11 -1
 		cp_inst->txwin = NULL;
 	}
@@ -168,7 +171,17 @@ static int coproc_mmap(struct file *fp, struct
 vm_area_struct *vma)
 		return -EINVAL;
 	}
 
-	vas_win_paste_addr(txwin, &paste_addr, NULL);
+	if (!cp_inst->coproc->vops && !cp_inst->coproc->vops-
 >paste_addr) {
+		pr_err("%s(): VAS API is not registered\n", __func__);
+		return -EACCES;
+	}
+
+	paste_addr = cp_inst->coproc->vops->paste_addr(txwin);
+	if (!paste_addr) {
+		pr_err("%s(): Window paste address failed\n",
 __func__);
+		return -EINVAL;
+	}
+
arch/powerpc/platforms/book3s/vas-api.c renamed from arch/powerpc/platforms/powernv/vas-api.c +2 -2
 	pfn = paste_addr >> PAGE_SHIFT;
 
 	/* flags, page_prot from cxl_mmap(), except we want cachable */
@@ -207,8 +220,8 @@ static struct file_operations coproc_fops = {
  * Supporting only nx-gzip coprocessor type now, but this API code
  * extended to other coprocessor types later.
  */
-int vas_register_api_powernv(struct module *mod, enum vas_cop_type
 cop_type,
-			     const char *name)
+int vas_register_coproc_api(struct module *mod, enum vas_cop_type
 cop_type,
+			const char *name, struct vas_user_win_ops
 *vops)

{

arch/powerpc/platforms/book3s/vas-api.c renamed from arch/powerpc/platforms/powernv/vas-api.c +1 -0
 	int rc = -EINVAL;
 	dev_t devno;
@@ -230,6 +243,7 @@ int vas_register_api_powernv(struct module
 *mod, enum vas_cop_type cop_type,
 	}
 	coproc_device.class->devnode = coproc_devnode;
 	coproc_device.cop_type = cop_type;
+	coproc_device.vops = vops;
 
 	coproc_fops.owner = mod;
arch/powerpc/platforms/book3s/vas-api.c renamed from arch/powerpc/platforms/powernv/vas-api.c +1 -3
 	cdev_init(&coproc_device.cdev, &coproc_fops);
@@ -262,9 +276,8 @@ int vas_register_api_powernv(struct module
 *mod, enum vas_cop_type cop_type,
 	unregister_chrdev_region(coproc_device.devt, 1);
 	return rc;
 }
-EXPORT_SYMBOL_GPL(vas_register_api_powernv);
 
-void vas_unregister_api_powernv(void)
+void vas_unregister_coproc_api(void)
 {
 	dev_t devno;
@@ -275,4 +288,3 @@ void vas_unregister_api_powernv(void)
 	class_destroy(coproc_device.class);
 	unregister_chrdev_region(coproc_device.devt, 1);
 }
-EXPORT_SYMBOL_GPL(vas_unregister_api_powernv);
arch/powerpc/platforms/powernv/Kconfig +0 -6
index 619b093a0657..043eefbbdd28 100644
@@ -33,20 +33,6 @@ config PPC_MEMTRACE
 	  Enabling this option allows for runtime allocation of memory
 (RAM)
 	  for hardware tracing.
 
-config PPC_VAS
-	bool "IBM Virtual Accelerator Switchboard (VAS)"
-	depends on PPC_POWERNV && PPC_64K_PAGES
-	default y
-	help
-	  This enables support for IBM Virtual Accelerator Switchboard

(VAS).

arch/powerpc/platforms/powernv/Kconfig +0 -2
-
-	  VAS allows accelerators in co-processors like NX-GZIP and NX-

842

arch/powerpc/platforms/powernv/Kconfig +0 -6
-	  to be accessible to kernel subsystems and user processes.
-
-	  VAS adapters are found in POWER9 based systems.
-
-	  If unsure, say N.
-
 config SCOM_DEBUGFS
 	bool "Expose SCOM controllers via debugfs"
 	depends on DEBUG_FS
arch/powerpc/platforms/powernv/Makefile +0 -0
index 2eb6ae150d1f..c747a1f1d25b 100644
@@ -18,7 +18,7 @@ obj-$(CONFIG_MEMORY_FAILURE)	+= opal-memory-

errors.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-
fault.o vas-api.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-
fault.o
obj-$(CONFIG_OCXL_BASE) += ocxl.o
obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o

arch/powerpc/platforms/powernv/vas-window.c +4 -0
index 5f5fe63a3d1c..b973dd574b47 100644
@@ -16,6 +16,8 @@
 #include <linux/mmu_context.h>
 #include <asm/switch_to.h>
 #include <asm/ppc-opcode.h>
+#include <asm/vas.h>
+#include <uapi/asm/vas-api.h>
 #include "vas.h"
 #include "copy-paste.h"
 
@@ -1441,3 +1443,67 @@ struct vas_window
 *vas_pswid_to_window(struct vas_instance *vinst,
 
 	return window;
 }
+
+static struct vas_window *vas_user_win_open(struct

vas_tx_win_open_attr *uattr,

arch/powerpc/platforms/powernv/vas-window.c +23 -0
+				enum vas_cop_type cop_type)
+{
+	struct vas_tx_win_attr txattr = {};
+
+	vas_init_tx_win_attr(&txattr, cop_type);
+
+	txattr.lpid = mfspr(SPRN_LPID);
+	txattr.pidr = mfspr(SPRN_PID);
+	txattr.user_win = true;
+	txattr.rsvd_txbuf_count = false;
+	txattr.pswid = false;
+
+	pr_devel("Pid %d: Opening txwin, PIDR %ld\n", txattr.pidr,
+				mfspr(SPRN_PID));
+
+	return vas_tx_win_open(uattr->vas_id, cop_type, &txattr);
+}
+
+static u64 vas_user_win_paste_addr(void *addr)
+{
+	u64 paste_addr;
+
+	vas_win_paste_addr((struct vas_window *)addr, &paste_addr,

NULL);

arch/powerpc/platforms/powernv/vas-window.c +24 -0
+
+	return paste_addr;
+}
+
+static int vas_user_win_close(void *addr)
+{
+	struct vas_window *txwin = addr;
+
+	vas_win_close(txwin);
+
+	return 0;
+}
+
+static struct vas_user_win_ops vops =  {
+	.open_win	=	vas_user_win_open,
+	.paste_addr	=	vas_user_win_paste_addr,
+	.close_win	=	vas_user_win_close,
+};
+
+/*
+ * Supporting only nx-gzip coprocessor type now, but this API code
+ * extended to other coprocessor types later.
+ */
+int vas_register_api_powernv(struct module *mod, enum vas_cop_type

cop_type,

arch/powerpc/platforms/powernv/vas-window.c +15 -0
+				const char *name)
+{
+	int rc;
+
+	rc = vas_register_coproc_api(mod, cop_type, name, &vops);
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(vas_register_api_powernv);
+
+void vas_unregister_api_powernv(void)
+{
+	vas_unregister_coproc_api();
+}
+EXPORT_SYMBOL_GPL(vas_unregister_api_powernv);
Signature · 1 line
2.18.2
#7 lore raw

Re: [PATCH V3 02/16] powerpc/vas: Move VAS API to common book3s platform

In reply to Nicholas Piggin
Quoted 27 lines · Nicholas Piggin

Excerpts from Haren Myneni's message of April 18, 2021 7:02 am:

Quoted 3 lines

Using the same /dev/crypto/nx-gzip interface for both powerNV and pseries.

The pseries NX driver will use the powernv VAS API ?

Quoted 7 lines

So this patch creates platforms/book3s/ and moves VAS API to that directory. The actual functionality is not changed.

Common interface functions such as open, window open ioctl, mmap and close are moved to arch/powerpc/platforms/book3s/vas-api.c Added hooks to call platform specific code, but the underline powerNV code in these functions is not changed.

Even so, could you do one patch that just moves, and another that adds the ops struct?

Quoted 8 lines
---
 arch/powerpc/include/asm/vas.h                | 22 ++++++-
 arch/powerpc/platforms/Kconfig                |  1 +
 arch/powerpc/platforms/Makefile               |  1 +
 arch/powerpc/platforms/book3s/Kconfig         | 15 +++++
 arch/powerpc/platforms/book3s/Makefile        |  2 +
Quoted 2 lines

The usual place for these would be arch/powerpc/sysdev/vas. E.g., see arch/powerpc/sysdev/xive.

You're right that is the usual place, but is it a good place? :)

Using platforms/book3s was my suggestion:

https://lore.kernel.org/linuxppc-dev/[email protected]/

But I don't feel that strongly about it, maybe just dumping things in sysdev is easier.

cheers

#8 lore raw

PATCH v3 3/16 powerpc/vas: Create take/drop task reference functions

In reply to Haren Myneni

Take task reference when each window opens and drops during close. This functionality is needed for powerNV and pseries. So this patch defines the existing code as functions in common book3s platform vas-api.c

---
 arch/powerpc/include/asm/vas.h              | 20 ++++++++
 arch/powerpc/platforms/book3s/vas-api.c     | 51 ++++++++++++++++++
 arch/powerpc/platforms/powernv/vas-fault.c  | 10 ++--
 arch/powerpc/platforms/powernv/vas-window.c | 57 ++-------------------
 arch/powerpc/platforms/powernv/vas.h        |  6 +--
 5 files changed, 83 insertions(+), 61 deletions(-)
arch/powerpc/include/asm/vas.h +20 -0
index 6bbade60d8f4..2daaa1a2a9a9 100644
@@ -5,6 +5,9 @@
 
 #ifndef _ASM_POWERPC_VAS_H
 #define _ASM_POWERPC_VAS_H
+#include <linux/sched/mm.h>
+#include <linux/mmu_context.h>
+#include <asm/icswx.h>
 #include <uapi/asm/vas-api.h>
 
 
@@ -60,6 +63,22 @@ struct vas_user_win_ops {
 	int (*close_win)(void *);
 };
 
+struct vas_win_task {
+	struct pid *pid;	/* Thread group ID of owner */
+	struct pid *tgid;	/* Linux process mm_struct */
+	struct mm_struct *mm;	/* Linux process mm_struct */
+};
+
+static inline void vas_drop_reference_task(struct vas_win_task *task)
+{
+	/* Drop references to pid and mm */
+	put_pid(task->pid);
+	if (task->mm) {
+		mm_context_remove_vas_window(task->mm);
+		mmdrop(task->mm);
+	}
+}
+
 /*
  * Receive window attributes specified by the (in-kernel) owner of window.
  */
@@ -190,4 +209,5 @@ int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
 			    struct vas_user_win_ops *vops);
 void vas_unregister_coproc_api(void);
 
+int vas_reference_task(struct vas_win_task *vtask);
 #endif /* __ASM_POWERPC_VAS_H */
arch/powerpc/platforms/book3s/vas-api.c +51 -0

57 lines not on this page · raw message

arch/powerpc/platforms/powernv/vas-fault.c +5 -5
index 3d21fce254b7..a4835cb82c09 100644
@@ -73,7 +73,7 @@ static void update_csb(struct vas_window *window,
 	 * NX user space windows can not be opened for task->mm=NULL
 	 * and faults will not be generated for kernel requests.
 	 */
-	if (WARN_ON_ONCE(!window->mm || !window->user_win))
+	if (WARN_ON_ONCE(!window->task.mm || !window->user_win))
 		return;
 
 	csb_addr = (void __user *)be64_to_cpu(crb->csb_addr);
@@ -92,7 +92,7 @@ static void update_csb(struct vas_window *window,
 	csb.address = crb->stamp.nx.fault_storage_addr;
 	csb.flags = 0;
 
-	pid = window->pid;
+	pid = window->task.pid;
 	tsk = get_pid_task(pid, PIDTYPE_PID);
 	/*
 	 * Process closes send window after all pending NX requests are
@@ -111,7 +111,7 @@ static void update_csb(struct vas_window *window,
 	 * a window and exits without closing it.
 	 */
 	if (!tsk) {
-		pid = window->tgid;
+		pid = window->task.tgid;
 		tsk = get_pid_task(pid, PIDTYPE_PID);
 		/*
 		 * Parent thread (tgid) will be closing window when it
@@ -127,7 +127,7 @@ static void update_csb(struct vas_window *window,
 		return;
 	}
 
-	kthread_use_mm(window->mm);
+	kthread_use_mm(window->task.mm);
 	rc = copy_to_user(csb_addr, &csb, sizeof(csb));
 	/*
 	 * User space polls on csb.flags (first byte). So add barrier
@@ -139,7 +139,7 @@ static void update_csb(struct vas_window *window,
 		smp_mb();
 		rc = copy_to_user(csb_addr, &csb, sizeof(u8));
 	}
-	kthread_unuse_mm(window->mm);
+	kthread_unuse_mm(window->task.mm);
 	put_task_struct(tsk);
 
 	/* Success */
arch/powerpc/platforms/powernv/vas-window.c +5 -52

70 lines not on this page · raw message

arch/powerpc/platforms/powernv/vas.h +2 -4
index c7db3190baca..f7aa2d04cd16 100644
@@ -357,11 +357,9 @@ struct vas_window {
 	bool user_win;		/* True if user space window */
 	void *hvwc_map;		/* HV window context */
 	void *uwc_map;		/* OS/User window context */
-	struct pid *pid;	/* Linux process id of owner */
-	struct pid *tgid;	/* Thread group ID of owner */
-	struct mm_struct *mm;	/* Linux process mm_struct */
 	int wcreds_max;		/* Window credits */
 
+	struct vas_win_task task;
 	char *dbgname;
 	struct dentry *dbgdir;
 
@@ -443,7 +441,7 @@ extern void vas_win_paste_addr(struct vas_window *window, u64 *addr,
 
 static inline int vas_window_pid(struct vas_window *window)
 {
-	return pid_vnr(window->pid);
+	return pid_vnr(window->task.pid);
 }
 
 static inline void vas_log_write(struct vas_window *win, char *name,
Signature · 1 line
2.18.2
#9 lore raw

Re: [V3 PATCH 03/16] powerpc/vas: Create take/drop task reference functions

Excerpts from Haren Myneni's message of April 18, 2021 7:03 am:

Quoted 30 lines

Take task reference when each window opens and drops during close. This functionality is needed for powerNV and pseries. So this patch defines the existing code as functions in common book3s platform vas-api.c

---
 arch/powerpc/include/asm/vas.h              | 20 ++++++++
 arch/powerpc/platforms/book3s/vas-api.c     | 51 ++++++++++++++++++
 arch/powerpc/platforms/powernv/vas-fault.c  | 10 ++--
 arch/powerpc/platforms/powernv/vas-window.c | 57 ++-------------------
 arch/powerpc/platforms/powernv/vas.h        |  6 +--
 5 files changed, 83 insertions(+), 61 deletions(-)
arch/powerpc/include/asm/vas.h +3 -0
index 6bbade60d8f4..2daaa1a2a9a9 100644
@@ -5,6 +5,9 @@
 
 #ifndef _ASM_POWERPC_VAS_H
 #define _ASM_POWERPC_VAS_H
+#include <linux/sched/mm.h>
+#include <linux/mmu_context.h>
+#include <asm/icswx.h>
 #include <uapi/asm/vas-api.h>
 
 

arch/powerpc/include/asm/vas.h struct vas_user_win_ops { · line 70

Quoted 8 lines
arch/powerpc/include/asm/vas.h +5 -0
@@ -60,6 +63,22 @@ struct vas_user_win_ops {
 	int (*close_win)(void *);
 };
 
+struct vas_win_task {
+	struct pid *pid;	/* Thread group ID of owner */
+	struct pid *tgid;	/* Linux process mm_struct */
+	struct mm_struct *mm;	/* Linux process mm_struct */
+};

Looks okay, happy with struct vas_win_task? (and vas_user_win_ops)?

I'd be happier to have everything related to vas windows prefixed with vas_window_ consistently, and have _user be present always for userspace windows, but you have to read and type it.

arch/powerpc/include/asm/vas.h struct vas_user_win_ops { · line 72

Quoted 2 lines
arch/powerpc/include/asm/vas.h +2 -0
+
+static inline void vas_drop_reference_task(struct vas_win_task *task)

This is not dropping a reference task, but a task reference. And it's not really a task reference as far as Linux understands, but a reference on pid (not task) and mm related to an open vas window. And specifically a user window (with corresponding vas_user_win_ops).

Could it be called a 'struct vas_window_user_ref' instead?

Thanks,
Nick

Quoted 7 lines
arch/powerpc/include/asm/vas.h +7 -0
+{
+	/* Drop references to pid and mm */
+	put_pid(task->pid);
+	if (task->mm) {
+		mm_context_remove_vas_window(task->mm);
+		mmdrop(task->mm);
+	}
#10 lore raw

Re: [V3 PATCH 03/16] powerpc/vas: Create take/drop task reference functions

Quoted 44 lines · Nicholas Piggin

Excerpts from Haren Myneni's message of April 18, 2021 7:03 am:

Quoted 32 lines

Take task reference when each window opens and drops during close. This functionality is needed for powerNV and pseries. So this patch defines the existing code as functions in common book3s platform vas-api.c

---
 arch/powerpc/include/asm/vas.h              | 20 ++++++++
 arch/powerpc/platforms/book3s/vas-api.c     | 51
++++++++++++++++++
 arch/powerpc/platforms/powernv/vas-fault.c  | 10 ++--
 arch/powerpc/platforms/powernv/vas-window.c | 57 ++---------------
----
 arch/powerpc/platforms/powernv/vas.h        |  6 +--
 5 files changed, 83 insertions(+), 61 deletions(-)
arch/powerpc/include/asm/vas.h +3 -0
index 6bbade60d8f4..2daaa1a2a9a9 100644
@@ -5,6 +5,9 @@
 
 #ifndef _ASM_POWERPC_VAS_H
 #define _ASM_POWERPC_VAS_H
+#include <linux/sched/mm.h>
+#include <linux/mmu_context.h>
+#include <asm/icswx.h>
 #include <uapi/asm/vas-api.h>
 
 

arch/powerpc/include/asm/vas.h struct vas_user_win_ops { · line 70

Quoted 8 lines
arch/powerpc/include/asm/vas.h +5 -0
@@ -60,6 +63,22 @@ struct vas_user_win_ops {
 	int (*close_win)(void *);
 };
 
+struct vas_win_task {
+	struct pid *pid;	/* Thread group ID of owner */
+	struct pid *tgid;	/* Linux process mm_struct */
+	struct mm_struct *mm;	/* Linux process mm_struct */
+};

Looks okay, happy with struct vas_win_task? (and vas_user_win_ops)?

Quoted 5 lines

I'd be happier to have everything related to vas windows prefixed with
vas_window_ consistently, and have _user be present always for userspace
windows, but you have to read and type it.

I will change to vas_window_task and vas_user_window_ops. struct vas_window is common for both kernel and user space window, but struct vas_window_task is needed only for user space.

Quoted 10 lines

arch/powerpc/include/asm/vas.h struct vas_user_win_ops { · line 73

Quoted 4 lines
arch/powerpc/include/asm/vas.h +2 -0
+
+static inline void vas_drop_reference_task(struct vas_win_task
 *task)

This is not dropping a reference task, but a task reference. And it's
not really a task reference as far as Linux understands, but a reference on pid (not task) and mm related to an open vas window. And specifically a user window (with corresponding vas_user_win_ops).

Yes get and put references to pid and mm. Hence mentioned reference_task to reflect for both pid and mm. Would you prefer vas_put_reference_pid_mm()?

Quoted 2 lines

Could it be called a 'struct vas_window_user_ref' instead?

To support LPM and DLPAR, plannig to add VMA (mapping address) in struct vas_window_task. But I can change it to 'struct vas_window_user_ref' instread of vas_window_task.

Thanks
Haren

Quoted 11 lines

Thanks,
Nick

Quoted 7 lines
arch/powerpc/include/asm/vas.h +7 -0
+{
+	/* Drop references to pid and mm */
+	put_pid(task->pid);
+	if (task->mm) {
+		mm_context_remove_vas_window(task->mm);
+		mmdrop(task->mm);
+	}
#11 lore raw

PATCH v3 4/16 powerpc/vas: Move update_csb/dump_crb to common book3s platform

In reply to Haren Myneni

NX issues an interrupt when sees fault on user space buffer. The kernel processes the fault by updating CSB. This functionality is same for both powerNV and pseries. So this patch moves these functions to common vas-api.c and the actual functionality is not changed.

---
 arch/powerpc/include/asm/vas.h             |   3 +
 arch/powerpc/platforms/book3s/vas-api.c    | 146 ++++++++++++++++++-
 arch/powerpc/platforms/powernv/vas-fault.c | 155 ++-------------------
 3 files changed, 157 insertions(+), 147 deletions(-)
arch/powerpc/include/asm/vas.h +3 -0
index 2daaa1a2a9a9..66bf8fb1a1be 100644
@@ -210,4 +210,7 @@ int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
 void vas_unregister_coproc_api(void);
 
 int vas_reference_task(struct vas_win_task *vtask);
+void vas_update_csb(struct coprocessor_request_block *crb,
+		    struct vas_win_task *vtask);
+void vas_dump_crb(struct coprocessor_request_block *crb);
 #endif /* __ASM_POWERPC_VAS_H */
arch/powerpc/platforms/book3s/vas-api.c +145 -1

158 lines not on this page · raw message

arch/powerpc/platforms/powernv/vas-fault.c +9 -146

173 lines not on this page · raw message

Signature · 1 line
2.18.2
reviewer Reviewed-by on v3 4/16
#12 lore raw

Re: [V3 PATCH 04/16] powerpc/vas: Move update_csb/dump_crb to common book3s platform

Excerpts from Haren Myneni's message of April 18, 2021 7:03 am:

Quoted 2 lines

NX issues an interrupt when sees fault on user space buffer.

If a coprocessor encounters an error translating an address, the VAS will cause an interrupt in the host.

Quoted 181 lines

The
kernel processes the fault by updating CSB. This functionality is same for both powerNV and pseries. So this patch moves these functions to common vas-api.c and the actual functionality is not changed.

---
 arch/powerpc/include/asm/vas.h             |   3 +
 arch/powerpc/platforms/book3s/vas-api.c    | 146 ++++++++++++++++++-
 arch/powerpc/platforms/powernv/vas-fault.c | 155 ++-------------------
 3 files changed, 157 insertions(+), 147 deletions(-)
arch/powerpc/include/asm/vas.h +3 -0
index 2daaa1a2a9a9..66bf8fb1a1be 100644
@@ -210,4 +210,7 @@ int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
 void vas_unregister_coproc_api(void);
 
 int vas_reference_task(struct vas_win_task *vtask);
+void vas_update_csb(struct coprocessor_request_block *crb,
+		    struct vas_win_task *vtask);
+void vas_dump_crb(struct coprocessor_request_block *crb);
 #endif /* __ASM_POWERPC_VAS_H */
arch/powerpc/platforms/book3s/vas-api.c +144 -0
index d98caa734154..dc131b2e4acd 100644
@@ -111,6 +111,150 @@ int vas_reference_task(struct vas_win_task *vtask)
 	return 0;
 }
 
+/*
+ * Update the CSB to indicate a translation error.
+ *
+ * User space will be polling on CSB after the request is issued.
+ * If NX can handle the request without any issues, it updates CSB.
+ * Whereas if NX encounters page fault, the kernel will handle the
+ * fault and update CSB with translation error.
+ *
+ * If we are unable to update the CSB means copy_to_user failed due to
+ * invalid csb_addr, send a signal to the process.
+ */
+void vas_update_csb(struct coprocessor_request_block *crb,
+		    struct vas_win_task *vtask)
+{
+	struct coprocessor_status_block csb;
+	struct kernel_siginfo info;
+	struct task_struct *tsk;
+	void __user *csb_addr;
+	struct pid *pid;
+	int rc;
+
+	/*
+	 * NX user space windows can not be opened for task->mm=NULL
+	 * and faults will not be generated for kernel requests.
+	 */
+	if (WARN_ON_ONCE(!vtask->mm))
+		return;
+
+	csb_addr = (void __user *)be64_to_cpu(crb->csb_addr);
+
+	memset(&csb, 0, sizeof(csb));
+	csb.cc = CSB_CC_FAULT_ADDRESS;
+	csb.ce = CSB_CE_TERMINATION;
+	csb.cs = 0;
+	csb.count = 0;
+
+	/*
+	 * NX operates and returns in BE format as defined CRB struct.
+	 * So saves fault_storage_addr in BE as NX pastes in FIFO and
+	 * expects user space to convert to CPU format.
+	 */
+	csb.address = crb->stamp.nx.fault_storage_addr;
+	csb.flags = 0;
+
+	pid = vtask->pid;
+	tsk = get_pid_task(pid, PIDTYPE_PID);
+	/*
+	 * Process closes send window after all pending NX requests are
+	 * completed. In multi-thread applications, a child thread can
+	 * open a window and can exit without closing it. May be some
+	 * requests are pending or this window can be used by other
+	 * threads later. We should handle faults if NX encounters
+	 * pages faults on these requests. Update CSB with translation
+	 * error and fault address. If csb_addr passed by user space is
+	 * invalid, send SEGV signal to pid saved in window. If the
+	 * child thread is not running, send the signal to tgid.
+	 * Parent thread (tgid) will close this window upon its exit.
+	 *
+	 * pid and mm references are taken when window is opened by
+	 * process (pid). So tgid is used only when child thread opens
+	 * a window and exits without closing it.
+	 */
+	if (!tsk) {
+		pid = vtask->tgid;
+		tsk = get_pid_task(pid, PIDTYPE_PID);
+		/*
+		 * Parent thread (tgid) will be closing window when it
+		 * exits. So should not get here.
+		 */
+		if (WARN_ON_ONCE(!tsk))
+			return;
+	}
+
+	/* Return if the task is exiting. */
+	if (tsk->flags & PF_EXITING) {
+		put_task_struct(tsk);
+		return;
+	}
+
+	kthread_use_mm(vtask->mm);
+	rc = copy_to_user(csb_addr, &csb, sizeof(csb));
+	/*
+	 * User space polls on csb.flags (first byte). So add barrier
+	 * then copy first byte with csb flags update.
+	 */
+	if (!rc) {
+		csb.flags = CSB_V;
+		/* Make sure update to csb.flags is visible now */
+		smp_mb();
+		rc = copy_to_user(csb_addr, &csb, sizeof(u8));
+	}
+	kthread_unuse_mm(vtask->mm);
+	put_task_struct(tsk);
+
+	/* Success */
+	if (!rc)
+		return;
+
+
+	pr_debug("Invalid CSB address 0x%p signalling pid(%d)\n",
+			csb_addr, pid_vnr(pid));
+
+	clear_siginfo(&info);
+	info.si_signo = SIGSEGV;
+	info.si_errno = EFAULT;
+	info.si_code = SEGV_MAPERR;
+	info.si_addr = csb_addr;
+	/*
+	 * process will be polling on csb.flags after request is sent to
+	 * NX. So generally CSB update should not fail except when an
+	 * application passes invalid csb_addr. So an error message will
+	 * be displayed and leave it to user space whether to ignore or
+	 * handle this signal.
+	 */
+	rcu_read_lock();
+	rc = kill_pid_info(SIGSEGV, &info, pid);
+	rcu_read_unlock();
+
+	pr_devel("%s(): pid %d kill_proc_info() rc %d\n", __func__,
+			pid_vnr(pid), rc);
+}
+
+void vas_dump_crb(struct coprocessor_request_block *crb)
+{
+	struct data_descriptor_entry *dde;
+	struct nx_fault_stamp *nx;
+
+	dde = &crb->source;
+	pr_devel("SrcDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+		be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+		dde->count, dde->index, dde->flags);
+
+	dde = &crb->target;
+	pr_devel("TgtDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+		be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+		dde->count, dde->index, dde->flags);
+
+	nx = &crb->stamp.nx;
+	pr_devel("NX Stamp: PSWID 0x%x, FSA 0x%llx, flags 0x%x, FS 0x%x\n",
+		be32_to_cpu(nx->pswid),
+		be64_to_cpu(crb->stamp.nx.fault_storage_addr),
+		nx->flags, nx->fault_status);
+}
+
 static int coproc_open(struct inode *inode, struct file *fp)
 {
 	struct coproc_instance *cp_inst;

arch/powerpc/platforms/book3s/vas-api.c static struct file_operations coproc_fops = { · line 422

Quoted 8 lines
arch/powerpc/platforms/book3s/vas-api.c +1 -1
@@ -272,7 +416,7 @@ static struct file_operations coproc_fops = {
  * extended to other coprocessor types later.
  */
 int vas_register_coproc_api(struct module *mod, enum vas_cop_type cop_type,
-			const char *name, struct vas_user_win_ops *vops)
+			    const char *name, struct vas_user_win_ops *vops)
 {
 	int rc = -EINVAL;
 	dev_t devno;

This change should go back where you added the code. But you've brought back vas_register_corpoc_api? In... patch 2, by the looks which just renames them back. Perhaps think about keeping the vas_register_coproc_api in patch 1 in that case, and just adding the new powernv specific wrapper over it.

But for this patch,

Thanks,
Nick

#14 lore raw

Re: [V3 PATCH 05/16] powerpc/vas: Define and use common vas_window struct

Excerpts from Haren Myneni's message of April 18, 2021 7:04 am:

Quoted 2 lines

Empty email?

Thanks,
Nick

#15 lore raw

Re: [V3 PATCH 05/16] powerpc/vas: Define and use common vas_window struct

Quoted 7 lines · Nicholas Piggin

Excerpts from Haren Myneni's message of April 18, 2021 7:04 am:

Quoted 1 line

Empty email?

Thanks,
Nick

Sorry, My mistake. Here is the patch and will add in V4 series.

[V3 PATCH 05/16] powerpc/vas: Define and use common vas_window struct

Same vas_window struct is used on powerNV and pseries. So this patch changes in struct vas_window to support both platforms and also the corresponding modifications in powerNV vas code.

On powerNV, vas_window is used for both TX and RX windows, whereas only for TX windows on powerVM. So some elements are specific to these platforms.

---
 arch/powerpc/include/asm/vas.h              |  48 ++++++++
 arch/powerpc/platforms/powernv/vas-debug.c  |  12 +-
 arch/powerpc/platforms/powernv/vas-fault.c  |   4 +-
 arch/powerpc/platforms/powernv/vas-trace.h  |   6 +-
 arch/powerpc/platforms/powernv/vas-window.c | 129 +++++++++++---------
 arch/powerpc/platforms/powernv/vas.h        |  38 +-----
 6 files changed, 135 insertions(+), 102 deletions(-)
arch/powerpc/include/asm/vas.h +48 -0
index 66bf8fb1a1be..f928bf4c7e98 100644
@@ -69,6 +69,54 @@ struct vas_win_task {
 	struct mm_struct *mm;	/* Linux process mm_struct */
 };
 
+/*
+ * In-kernel state a VAS window. One per window.
+ * powerVM: Used only for Tx windows.
+ * powerNV: Used for both Tx and Rx windows.
+ */
+struct vas_window {
+	u32 winid;
+	u32 wcreds_max;	/* Window credits */
+	enum vas_cop_type cop;
+	struct vas_win_task task;
+	char *dbgname;
+	struct dentry *dbgdir;
+	union {
+		/* powerNV specific data */
+		struct {
+			void *vinst;	/* points to VAS instance */
+			bool tx_win;	/* True if send window */
+			bool nx_win;	/* True if NX window */
+			bool user_win;	/* True if user space window */
+			void *hvwc_map;	/* HV window context */
+			void *uwc_map;	/* OS/User window context */
+
+			/* Fields applicable only to send windows */
+			void *paste_kaddr;
+			char *paste_addr_name;
+			struct vas_window *rxwin;
+
+			atomic_t num_txwins;	/* Only for receive windows */
+		} pnv;
+		struct {
+			u64 win_addr;	/* Physical paste address */
+			u8 win_type;	/* QoS or Default window */
+			u8 status;
+			u32 complete_irq;	/* Completion interrupt */
+			u32 fault_irq;	/* Fault interrupt */
+			u64 domain[6];	/* Associativity domain Ids */
+					/* this window is allocated */
+			u64 util;
+
+			/* List of windows opened which is used for LPM */
+			struct list_head win_list;
+			u64 flags;
+			char *name;
+			int fault_virq;
+		} lpar;
+	};
+};
+
 static inline void vas_drop_reference_task(struct vas_win_task *task)
 {
 	/* Drop references to pid and mm */
arch/powerpc/platforms/powernv/vas-debug.c +7 -5
index 41fa90d2f4ab..80f735449ab8 100644
@@ -9,6 +9,7 @@
 #include <linux/slab.h>
 #include <linux/debugfs.h>
 #include <linux/seq_file.h>
+#include <asm/vas.h>
 #include "vas.h"
 
 static struct dentry *vas_debugfs;
@@ -33,11 +34,11 @@ static int info_show(struct seq_file *s, void *private)
 	mutex_lock(&vas_mutex);
 
 	/* ensure window is not unmapped */
-	if (!window->hvwc_map)
+	if (!window->pnv.hvwc_map)
 		goto unlock;
 
 	seq_printf(s, "Type: %s, %s\n", cop_to_str(window->cop),
-					window->tx_win ? "Send" : "Receive");
+				window->pnv.tx_win ? "Send" : "Receive");
 	seq_printf(s, "Pid : %d\n", vas_window_pid(window));
 
 unlock:
@@ -60,7 +61,7 @@ static int hvwc_show(struct seq_file *s, void *private)
 	mutex_lock(&vas_mutex);
 
 	/* ensure window is not unmapped */
-	if (!window->hvwc_map)
+	if (!window->pnv.hvwc_map)
 		goto unlock;
 
 	print_reg(s, window, VREG(LPID));
@@ -115,9 +116,10 @@ void vas_window_free_dbgdir(struct vas_window *window)
 
 void vas_window_init_dbgdir(struct vas_window *window)
 {
+	struct vas_instance *vinst = window->pnv.vinst;
 	struct dentry *d;
 
-	if (!window->vinst->dbgdir)
+	if (!vinst->dbgdir)
 		return;
 
 	window->dbgname = kzalloc(16, GFP_KERNEL);
@@ -126,7 +128,7 @@ void vas_window_init_dbgdir(struct vas_window *window)
 
 	snprintf(window->dbgname, 16, "w%d", window->winid);
 
-	d = debugfs_create_dir(window->dbgname, window->vinst->dbgdir);
+	d = debugfs_create_dir(window->dbgname, vinst->dbgdir);
 	window->dbgdir = d;
 
 	debugfs_create_file("info", 0444, d, window, &info_fops);
arch/powerpc/platforms/powernv/vas-fault.c +2 -2
index 2e898eac1bb2..82acede2d892 100644
@@ -152,10 +152,10 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
 			/*
 			 * NX sees faults only with user space windows.
 			 */
-			if (window->user_win)
+			if (window->pnv.user_win)
 				vas_update_csb(crb, &window->task);
 			else
-				WARN_ON_ONCE(!window->user_win);
+				WARN_ON_ONCE(!window->pnv.user_win);
 
 			/*
 			 * Return credit for send window after processing
arch/powerpc/platforms/powernv/vas-trace.h +4 -2
index a449b9f0c12e..843aae37c07a 100644
@@ -95,9 +95,11 @@ TRACE_EVENT(	vas_paste_crb,
 
 		TP_fast_assign(
 			__entry->pid = tsk->pid;
-			__entry->vasid = win->vinst->vas_id;
+			__entry->vasid =
+				((struct vas_instance *)win->pnv.vinst)->vas_id;
 			__entry->winid = win->winid;
-			__entry->paste_kaddr = (unsigned long)win->paste_kaddr
+			__entry->paste_kaddr =
+				(unsigned long)win->pnv.paste_kaddr;
 		),
 
 		TP_printk("pid=%d, vasid=%d, winid=%d, paste_kaddr=0x%016lx\n",
arch/powerpc/platforms/powernv/vas-window.c +69 -60

331 lines not on this page · raw message

arch/powerpc/platforms/powernv/vas.h +5 -33
index f7aa2d04cd16..72ccb2f692cc 100644
@@ -345,34 +345,6 @@ struct vas_instance {
 	struct dentry *dbgdir;
 };
 
-/*
- * In-kernel state a VAS window. One per window.
- */
-struct vas_window {
-	/* Fields common to send and receive windows */
-	struct vas_instance *vinst;
-	int winid;
-	bool tx_win;		/* True if send window */
-	bool nx_win;		/* True if NX window */
-	bool user_win;		/* True if user space window */
-	void *hvwc_map;		/* HV window context */
-	void *uwc_map;		/* OS/User window context */
-	int wcreds_max;		/* Window credits */
-
-	struct vas_win_task task;
-	char *dbgname;
-	struct dentry *dbgdir;
-
-	/* Fields applicable only to send windows */
-	void *paste_kaddr;
-	char *paste_addr_name;
-	struct vas_window *rxwin;
-
-	/* Feilds applicable only to receive windows */
-	enum vas_cop_type cop;
-	atomic_t num_txwins;
-};
-
 /*
  * Container for the hardware state of a window. One per-window.
  *
@@ -449,8 +421,8 @@ static inline void vas_log_write(struct vas_window *win, char *name,
 {
 	if (val)
 		pr_debug("%swin #%d: %s reg %p, val 0x%016llx\n",
-				win->tx_win ? "Tx" : "Rx", win->winid, name,
-				regptr, val);
+				win->pnv.tx_win ? "Tx" : "Rx", win->winid,
+				name, regptr, val);
 }
 
 static inline void write_uwc_reg(struct vas_window *win, char *name,
@@ -458,7 +430,7 @@ static inline void write_uwc_reg(struct vas_window *win, char *name,
 {
 	void *regptr;
 
-	regptr = win->uwc_map + reg;
+	regptr = win->pnv.uwc_map + reg;
 	vas_log_write(win, name, regptr, val);
 
 	out_be64(regptr, val);
@@ -469,7 +441,7 @@ static inline void write_hvwc_reg(struct vas_window *win, char *name,
 {
 	void *regptr;
 
-	regptr = win->hvwc_map + reg;
+	regptr = win->pnv.hvwc_map + reg;
 	vas_log_write(win, name, regptr, val);
 
 	out_be64(regptr, val);
@@ -478,7 +450,7 @@ static inline void write_hvwc_reg(struct vas_window *win, char *name,
 static inline u64 read_hvwc_reg(struct vas_window *win,
 			char *name __maybe_unused, s32 reg)
 {
-	return in_be64(win->hvwc_map+reg);
+	return in_be64(win->pnv.hvwc_map + reg);
 }
 
 /*
Signature · 1 line
2.18.2
#16 lore raw

PATCH v3 6/16 powerpc/pseries/vas: Define VAS/NXGZIP HCALLs and structs

In reply to Haren Myneni

This patch adds HCALLs and other definitions. Also define structs that are used in VAS implementation on powerVM.

---
 arch/powerpc/include/asm/hvcall.h    |  7 ++
 arch/powerpc/include/asm/vas.h       | 28 ++++++++
 arch/powerpc/platforms/pseries/vas.h | 96 ++++++++++++++++++++++++++++
 3 files changed, 131 insertions(+)
 create mode 100644 arch/powerpc/platforms/pseries/vas.h
arch/powerpc/include/asm/hvcall.h +7 -0
index ed6086d57b22..accbb7f6f272 100644
@@ -294,6 +294,13 @@
 #define H_RESIZE_HPT_COMMIT	0x370
 #define H_REGISTER_PROC_TBL	0x37C
 #define H_SIGNAL_SYS_RESET	0x380
+#define	H_ALLOCATE_VAS_WINDOW	0x388
+#define	H_MODIFY_VAS_WINDOW	0x38C
+#define	H_DEALLOCATE_VAS_WINDOW	0x390
+#define	H_QUERY_VAS_WINDOW	0x394
+#define	H_QUERY_VAS_CAPABILITIES	0x398
+#define	H_QUERY_NX_CAPABILITIES	0x39C
+#define	H_GET_NX_FAULT		0x3A0
 #define H_INT_GET_SOURCE_INFO   0x3A8
 #define H_INT_SET_SOURCE_CONFIG 0x3AC
 #define H_INT_GET_SOURCE_CONFIG 0x3B0
arch/powerpc/include/asm/vas.h +28 -0
index f928bf4c7e98..d15784506a54 100644
@@ -179,6 +179,7 @@ struct vas_tx_win_attr {
 	bool rx_win_ord_mode;
 };
 
+#ifdef CONFIG_PPC_POWERNV
 /*
  * Helper to map a chip id to VAS id.
  * For POWER9, this is a 1:1 mapping. In the future this maybe a 1:N
@@ -243,6 +244,33 @@ int vas_paste_crb(struct vas_window *win, int offset, bool re);
 int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
 			     const char *name);
 void vas_unregister_api_powernv(void);
+#endif
+
+#ifdef CONFIG_PPC_PSERIES
+
+/* VAS Capabilities */
+#define VAS_GZIP_QOS_FEAT	0x1
+#define VAS_GZIP_DEF_FEAT	0x2
+#define VAS_GZIP_QOS_FEAT_BIT	(1UL << (63 - VAS_GZIP_QOS_FEAT)) /* Bit 1 */
+#define VAS_GZIP_DEF_FEAT_BIT	(1UL << (63 - VAS_GZIP_DEF_FEAT)) /* Bit 2 */
+
+/* NX Capabilities */
+#define	VAS_NX_GZIP_FEAT	0x1
+#define	VAS_NX_GZIP_FEAT_BIT	(1UL << (63 - VAS_NX_GZIP_FEAT)) /* Bit 1 */
+#define	VAS_DESCR_LEN		8
+
+struct vas_all_capabs_be {
+		__be64  descriptor;
+		__be64  feat_type;
+} __packed __aligned(0x1000);
+
+struct vas_all_capabs {
+	char	name[VAS_DESCR_LEN + 1];
+	u64     descriptor;
+	u64     feat_type;
+};
+
+#endif
 
 /*
  * Register / unregister coprocessor type to VAS API which will be exported
arch/powerpc/platforms/pseries/vas.h new file +96 -0

96 lines not on this page · raw message

Signature · 1 line
2.18.2
#17 lore raw

Re: [V3 PATCH 06/16] powerpc/pseries/vas: Define VAS/NXGZIP HCALLs and structs

Excerpts from Haren Myneni's message of April 18, 2021 7:05 am:

Quoted 17 lines

This patch adds HCALLs and other definitions. Also define structs that are used in VAS implementation on powerVM.

---
 arch/powerpc/include/asm/hvcall.h    |  7 ++
 arch/powerpc/include/asm/vas.h       | 28 ++++++++
 arch/powerpc/platforms/pseries/vas.h | 96 ++++++++++++++++++++++++++++
 3 files changed, 131 insertions(+)
 create mode 100644 arch/powerpc/platforms/pseries/vas.h
arch/powerpc/include/asm/hvcall.h +0 -0
index ed6086d57b22..accbb7f6f272 100644

arch/powerpc/include/asm/hvcall.h line 303

Quoted 10 lines
arch/powerpc/include/asm/hvcall.h +7 -0
@@ -294,6 +294,13 @@
 #define H_RESIZE_HPT_COMMIT	0x370
 #define H_REGISTER_PROC_TBL	0x37C
 #define H_SIGNAL_SYS_RESET	0x380
+#define	H_ALLOCATE_VAS_WINDOW	0x388
+#define	H_MODIFY_VAS_WINDOW	0x38C
+#define	H_DEALLOCATE_VAS_WINDOW	0x390
+#define	H_QUERY_VAS_WINDOW	0x394
+#define	H_QUERY_VAS_CAPABILITIES	0x398
+#define	H_QUERY_NX_CAPABILITIES	0x39C
+#define	H_GET_NX_FAULT		0x3A0

These should be spaces.

Quoted 18 lines
arch/powerpc/include/asm/hvcall.h +0 -0
 #define H_INT_GET_SOURCE_INFO   0x3A8
 #define H_INT_SET_SOURCE_CONFIG 0x3AC
 #define H_INT_GET_SOURCE_CONFIG 0x3B0
arch/powerpc/include/asm/vas.h +1 -0
index f928bf4c7e98..d15784506a54 100644
@@ -179,6 +179,7 @@ struct vas_tx_win_attr {
 	bool rx_win_ord_mode;
 };
 
+#ifdef CONFIG_PPC_POWERNV
 /*
  * Helper to map a chip id to VAS id.
  * For POWER9, this is a 1:1 mapping. In the future this maybe a 1:N
@@ -243,6 +244,33 @@ int vas_paste_crb(struct vas_window *win, int offset, bool re);
 int vas_register_api_powernv(struct module *mod, enum vas_cop_type cop_type,
 			     const char *name);

arch/powerpc/include/asm/vas.h int vas_paste_crb(struct vas_window *win, int offset, bool re); · line 255

Quoted 10 lines
arch/powerpc/include/asm/vas.h +9 -0
 void vas_unregister_api_powernv(void);
+#endif
+
+#ifdef CONFIG_PPC_PSERIES
+
+/* VAS Capabilities */
+#define VAS_GZIP_QOS_FEAT	0x1
+#define VAS_GZIP_DEF_FEAT	0x2
+#define VAS_GZIP_QOS_FEAT_BIT	(1UL << (63 - VAS_GZIP_QOS_FEAT)) /* Bit 1 */
+#define VAS_GZIP_DEF_FEAT_BIT	(1UL << (63 - VAS_GZIP_DEF_FEAT)) /* Bit 2 */

Use PPC_BIT for these.

Quoted 6 lines
arch/powerpc/include/asm/vas.h +6 -0
+
+/* NX Capabilities */
+#define	VAS_NX_GZIP_FEAT	0x1
+#define	VAS_NX_GZIP_FEAT_BIT	(1UL << (63 - VAS_NX_GZIP_FEAT)) /* Bit 1 */
+#define	VAS_DESCR_LEN		8
+

arch/powerpc/include/asm/vas.h int vas_paste_crb(struct vas_window *win, int offset, bool re); · line 271

Quoted 10 lines
arch/powerpc/include/asm/vas.h +10 -0
+struct vas_all_capabs_be {
+		__be64  descriptor;
+		__be64  feat_type;
+} __packed __aligned(0x1000);
+
+struct vas_all_capabs {
+	char	name[VAS_DESCR_LEN + 1];
+	u64     descriptor;
+	u64     feat_type;
+};

You're using _be for the struct that is passed to the hcall, and a non-postfixed one for something the driver uses internally? It seems like buf or buffer, or hv_ prefix is typically used rather than be (host kernel could be BE).

struct hv_query_vas_capabilities_buffer for example.

Does the hcall really require 0x1000 alignment?

Quoted 112 lines
arch/powerpc/include/asm/vas.h +2 -0
+
+#endif
 
 /*
  * Register / unregister coprocessor type to VAS API which will be exported
arch/powerpc/platforms/pseries/vas.h new file +96 -0
new file mode 100644
index 000000000000..208682fffa57
@@ -0,0 +1,96 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright 2020-21 IBM Corp.
+ */
+
+#ifndef _VAS_H
+#define _VAS_H
+#include <asm/vas.h>
+#include <linux/mutex.h>
+#include <linux/stringify.h>
+
+/*
+ * VAS window modify flags
+ */
+#define	VAS_MOD_WIN_CLOSE	(1UL << 63)
+#define	VAS_MOD_WIN_JOBS_KILL	(1UL << (63 - 1))
+#define	VAS_MOD_WIN_DR		(1UL << (63 - 3))
+#define	VAS_MOD_WIN_PR		(1UL << (63 - 4))
+#define	VAS_MOD_WIN_SF		(1UL << (63 - 5))
+#define	VAS_MOD_WIN_TA		(1UL << (63 - 6))
+#define	VAS_MOD_WIN_FLAGS	(VAS_MOD_WIN_JOBS_KILL | VAS_MOD_WIN_DR | \
+				VAS_MOD_WIN_PR | VAS_MOD_WIN_SF)
+
+#define	VAS_WIN_ACTIVE		0x0
+#define	VAS_WIN_CLOSED		0x1
+#define	VAS_WIN_INACTIVE	0x2	/* Inactive due to HW failure */
+/* Process of being modified, deallocated, or quiesced */
+#define	VAS_WIN_MOD_IN_PROCESS	0x3
+
+#define	VAS_COPY_PASTE_USER_MODE	0x00000001
+#define	VAS_COP_OP_USER_MODE		0x00000010
+
+/*
+ * Co-processor feature - GZIP QoS windows or GZIP default windows
+ */
+enum vas_cop_feat_type {
+	VAS_GZIP_QOS_FEAT_TYPE,
+	VAS_GZIP_DEF_FEAT_TYPE,
+	VAS_MAX_FEAT_TYPE,
+};
+
+struct vas_ct_capabs_be {
+	__be64	descriptor;
+	u8	win_type;		/* Default or QoS type */
+	u8	user_mode;
+	__be16	max_lpar_creds;
+	__be16	max_win_creds;
+	union {
+		__be16	reserved;
+		__be16	def_lpar_creds; /* Used for default capabilities */
+	};
+	__be16	target_lpar_creds;
+} __packed __aligned(0x1000);
+
+struct vas_ct_capabs {
+	char		name[VAS_DESCR_LEN + 1];
+	u64		descriptor;
+	u8		win_type;	/* Default or QoS type */
+	u8		user_mode;	/* User mode copy/paste or COP HCALL */
+	u16		max_lpar_creds;	/* Max credits available in LPAR */
+	/* Max credits can be assigned per window */
+	u16		max_win_creds;
+	union {
+		u16	reserved;	/* Used for QoS credit type */
+		u16	def_lpar_creds; /* Used for default credit type */
+	};
+	/* Total LPAR available credits. Can be different from max LPAR */
+	/* credits due to DLPAR operation */
+	atomic_t	target_lpar_creds;
+	atomic_t	used_lpar_creds; /* Used credits so far */
+	u16		avail_lpar_creds; /* Remaining available credits */
+};
+
+struct vas_capabs {
+	struct vas_ct_capabs capab;
+	struct list_head list;
+};
+
+struct vas_win_lpar_be {
+	__be16	version;
+	u8	win_type;
+	u8	status;
+	__be16	credits;	/* No of credits assigned to this window */
+	__be16	reserved;
+	__be32	pid;		/* LPAR Process ID */
+	__be32	tid;		/* LPAR Thread ID */
+	__be64	win_addr;
+	__be32	interrupt;	/* Interrupt when NX request completes */
+	__be32	fault;		/* Interrupt when NX sees fault */
+	/* Associativity Domain Identifiers as returned in */
+	/* H_HOME_NODE_ASSOCIATIVITY */
+	__be64	domain[6];
+	__be64	win_util;	/* Number of bytes processed */
+} __packed __aligned(0x1000);
+
+#endif /* _VAS_H */
Signature · 1 line
2.18.2
#18 lore raw

Re: [V3 PATCH 06/16] powerpc/pseries/vas: Define VAS/NXGZIP HCALLs and structs

Quoted 93 lines · Nicholas Piggin

Excerpts from Haren Myneni's message of April 18, 2021 7:05 am:

Quoted 18 lines

This patch adds HCALLs and other definitions. Also define structs that are used in VAS implementation on powerVM.

---
 arch/powerpc/include/asm/hvcall.h    |  7 ++
 arch/powerpc/include/asm/vas.h       | 28 ++++++++
 arch/powerpc/platforms/pseries/vas.h | 96
++++++++++++++++++++++++++++
 3 files changed, 131 insertions(+)
 create mode 100644 arch/powerpc/platforms/pseries/vas.h
arch/powerpc/include/asm/hvcall.h +0 -0
index ed6086d57b22..accbb7f6f272 100644

arch/powerpc/include/asm/hvcall.h line 303

Quoted 10 lines
arch/powerpc/include/asm/hvcall.h +7 -0
@@ -294,6 +294,13 @@
 #define H_RESIZE_HPT_COMMIT	0x370
 #define H_REGISTER_PROC_TBL	0x37C
 #define H_SIGNAL_SYS_RESET	0x380
+#define	H_ALLOCATE_VAS_WINDOW	0x388
+#define	H_MODIFY_VAS_WINDOW	0x38C
+#define	H_DEALLOCATE_VAS_WINDOW	0x390
+#define	H_QUERY_VAS_WINDOW	0x394
+#define	H_QUERY_VAS_CAPABILITIES	0x398
+#define	H_QUERY_NX_CAPABILITIES	0x39C
+#define	H_GET_NX_FAULT		0x3A0

These should be spaces.

Quoted 31 lines
arch/powerpc/include/asm/hvcall.h +0 -0
 #define H_INT_GET_SOURCE_INFO   0x3A8
 #define H_INT_SET_SOURCE_CONFIG 0x3AC
 #define H_INT_GET_SOURCE_CONFIG 0x3B0
arch/powerpc/include/asm/vas.h +1 -0
index f928bf4c7e98..d15784506a54 100644
@@ -179,6 +179,7 @@ struct vas_tx_win_attr {
 	bool rx_win_ord_mode;
 };
 
+#ifdef CONFIG_PPC_POWERNV
 /*
  * Helper to map a chip id to VAS id.
  * For POWER9, this is a 1:1 mapping. In the future this maybe a

1:N

arch/powerpc/include/asm/vas.h +8 -0
@@ -243,6 +244,33 @@ int vas_paste_crb(struct vas_window *win, int
 offset, bool re);
 int vas_register_api_powernv(struct module *mod, enum vas_cop_type
 cop_type,
 			     const char *name);
 void vas_unregister_api_powernv(void);
+#endif
+
+#ifdef CONFIG_PPC_PSERIES
+
+/* VAS Capabilities */
+#define VAS_GZIP_QOS_FEAT	0x1
+#define VAS_GZIP_DEF_FEAT	0x2
+#define VAS_GZIP_QOS_FEAT_BIT	(1UL << (63 -
Quoted 3 lines

VAS_GZIP_QOS_FEAT)) /* Bit 1 */
+#define VAS_GZIP_DEF_FEAT_BIT (1UL << (63 -
VAS_GZIP_DEF_FEAT)) /* Bit 2 */

Use PPC_BIT for these.

Quoted 7 lines
arch/powerpc/include/asm/vas.h +4 -0
+
+/* NX Capabilities */
+#define	VAS_NX_GZIP_FEAT	0x1
+#define	VAS_NX_GZIP_FEAT_BIT	(1UL << (63 -

VAS_NX_GZIP_FEAT)) /* Bit 1 */

arch/powerpc/include/asm/vas.h +2 -0
+#define	VAS_DESCR_LEN		8
+

arch/powerpc/include/asm/vas.h int vas_paste_crb(struct vas_window *win, int offset, bool re); · line 271

Quoted 10 lines
arch/powerpc/include/asm/vas.h +10 -0
+struct vas_all_capabs_be {
+		__be64  descriptor;
+		__be64  feat_type;
+} __packed __aligned(0x1000);
+
+struct vas_all_capabs {
+	char	name[VAS_DESCR_LEN + 1];
+	u64     descriptor;
+	u64     feat_type;
+};

You're using _be for the struct that is passed to the hcall, and a non-postfixed one for something the driver uses internally? It seems like buf or buffer, or hv_ prefix is typically used rather than be (host
kernel could be BE).

Quoted 1 line

struct hv_query_vas_capabilities_buffer for example.

Can I add like "hv_vas_all_caps_buf" instead of vas_all_capabs_be

Quoted 2 lines

Does the hcall really require 0x1000 alignment?

pAPR user mode NX says "resultBuffer : The logical real address of a size-aligned 4K buffer to store VAS capabilities in" (Section:
H_QUERY_VAS_CAPABILITIES HCALL)

Quoted 122 lines
Quoted 122 lines
arch/powerpc/include/asm/vas.h +2 -0
+
+#endif
 
 /*
  * Register / unregister coprocessor type to VAS API which will be

exported

arch/powerpc/platforms/pseries/vas.h new file +21 -0
new file mode 100644
index 000000000000..208682fffa57
@@ -0,0 +1,96 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright 2020-21 IBM Corp.
+ */
+
+#ifndef _VAS_H
+#define _VAS_H
+#include <asm/vas.h>
+#include <linux/mutex.h>
+#include <linux/stringify.h>
+
+/*
+ * VAS window modify flags
+ */
+#define	VAS_MOD_WIN_CLOSE	(1UL << 63)
+#define	VAS_MOD_WIN_JOBS_KILL	(1UL << (63 - 1))
+#define	VAS_MOD_WIN_DR		(1UL << (63 - 3))
+#define	VAS_MOD_WIN_PR		(1UL << (63 - 4))
+#define	VAS_MOD_WIN_SF		(1UL << (63 - 5))
+#define	VAS_MOD_WIN_TA		(1UL << (63 - 6))
+#define	VAS_MOD_WIN_FLAGS	(VAS_MOD_WIN_JOBS_KILL |

VAS_MOD_WIN_DR | \

arch/powerpc/platforms/pseries/vas.h new file +5 -0
+				VAS_MOD_WIN_PR | VAS_MOD_WIN_SF)
+
+#define	VAS_WIN_ACTIVE		0x0
+#define	VAS_WIN_CLOSED		0x1
+#define	VAS_WIN_INACTIVE	0x2	/* Inactive due to HW

failure */

arch/powerpc/platforms/pseries/vas.h new file +24 -0
+/* Process of being modified, deallocated, or quiesced */
+#define	VAS_WIN_MOD_IN_PROCESS	0x3
+
+#define	VAS_COPY_PASTE_USER_MODE	0x00000001
+#define	VAS_COP_OP_USER_MODE		0x00000010
+
+/*
+ * Co-processor feature - GZIP QoS windows or GZIP default windows
+ */
+enum vas_cop_feat_type {
+	VAS_GZIP_QOS_FEAT_TYPE,
+	VAS_GZIP_DEF_FEAT_TYPE,
+	VAS_MAX_FEAT_TYPE,
+};
+
+struct vas_ct_capabs_be {
+	__be64	descriptor;
+	u8	win_type;		/* Default or QoS type */
+	u8	user_mode;
+	__be16	max_lpar_creds;
+	__be16	max_win_creds;
+	union {
+		__be16	reserved;
+		__be16	def_lpar_creds; /* Used for default

capabilities */

arch/powerpc/platforms/pseries/vas.h new file +9 -0
+	};
+	__be16	target_lpar_creds;
+} __packed __aligned(0x1000);
+
+struct vas_ct_capabs {
+	char		name[VAS_DESCR_LEN + 1];
+	u64		descriptor;
+	u8		win_type;	/* Default or QoS type */
+	u8		user_mode;	/* User mode copy/paste or COP

HCALL */

arch/powerpc/platforms/pseries/vas.h new file +8 -0
+	u16		max_lpar_creds;	/* Max credits available in LPAR */
+	/* Max credits can be assigned per window */
+	u16		max_win_creds;
+	union {
+		u16	reserved;	/* Used for QoS credit type */
+		u16	def_lpar_creds; /* Used for default credit type */
+	};
+	/* Total LPAR available credits. Can be different from max LPAR

*/

arch/powerpc/platforms/pseries/vas.h new file +16 -0
+	/* credits due to DLPAR operation */
+	atomic_t	target_lpar_creds;
+	atomic_t	used_lpar_creds; /* Used credits so far */
+	u16		avail_lpar_creds; /* Remaining available credits */
+};
+
+struct vas_capabs {
+	struct vas_ct_capabs capab;
+	struct list_head list;
+};
+
+struct vas_win_lpar_be {
+	__be16	version;
+	u8	win_type;
+	u8	status;
+	__be16	credits;	/* No of credits assigned to this

window */

arch/powerpc/platforms/pseries/vas.h new file +5 -0
+	__be16	reserved;
+	__be32	pid;		/* LPAR Process ID */
+	__be32	tid;		/* LPAR Thread ID */
+	__be64	win_addr;
+	__be32	interrupt;	/* Interrupt when NX request

completes */

arch/powerpc/platforms/pseries/vas.h new file +8 -0
+	__be32	fault;		/* Interrupt when NX sees fault */
+	/* Associativity Domain Identifiers as returned in */
+	/* H_HOME_NODE_ASSOCIATIVITY */
+	__be64	domain[6];
+	__be64	win_util;	/* Number of bytes processed */
+} __packed __aligned(0x1000);
+
+#endif /* _VAS_H */
Signature · 1 line
2.18.2
#19 lore raw

PATCH v3 7/16 powerpc/vas: Define QoS credit flag to allocate window

In reply to Haren Myneni

pHyp introduces two different type of credits: Default and Quality of service (QoS).

The total number of default credits available on each LPAR depends on CPU resources configured. But these credits can be shared or over-committed across LPARs in shared mode which can result in paste command failure (RMA_busy). To avoid NX HW contention, phyp introduces QoS credit type which makes sure guaranteed access to NX resources. The system admins can assign QoS credits for each LPAR via HMC.

Default credit type is used to allocate a VAS window by default as on powerVM implementation. But the process can pass VAS_WIN_QOS_CREDITS flag with VAS_TX_WIN_OPEN ioctl to open QoS type window.

---
 arch/powerpc/include/uapi/asm/vas-api.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/powerpc/include/uapi/asm/vas-api.h +5 -1
index ebd4b2424785..eb7c8694174f 100644
@@ -13,11 +13,15 @@
 #define VAS_MAGIC	'v'
 #define VAS_TX_WIN_OPEN	_IOW(VAS_MAGIC, 0x20, struct vas_tx_win_open_attr)
 
+/* Flags to VAS TX open window ioctl */
+/* To allocate a window with QoS credit, otherwise default credit is used */
+#define	VAS_WIN_QOS_CREDITS	0x0000000000000001
+
 struct vas_tx_win_open_attr {
 	__u32	version;
 	__s16	vas_id;	/* specific instance of vas or -1 for default */
 	__u16	reserved1;
-	__u64	flags;	/* Future use */
+	__u64	flags;
 	__u64	reserved2[6];
 };
 
Signature · 1 line
2.18.2
#20 lore raw

Re: [V3 PATCH 07/16] powerpc/vas: Define QoS credit flag to allocate window

Excerpts from Haren Myneni's message of April 18, 2021 7:06 am:

Quoted 12 lines

pHyp introduces two different type of credits: Default and Quality of service (QoS).

The total number of default credits available on each LPAR depends on CPU resources configured. But these credits can be shared or over-committed across LPARs in shared mode which can result in paste command failure (RMA_busy). To avoid NX HW contention, phyp introduces QoS credit type which makes sure guaranteed access to NX resources. The system admins can assign QoS credits for each LPAR via HMC.

Quoted 2 lines

Default credit type is used to allocate a VAS window by default as on powerVM implementation. But the process can pass VAS_WIN_QOS_CREDITS

There's some interchanging of pHyp and PowerVM in the series.

PowerVM is probably the better term to use, with uppercase P. Unless you mean PAPR or pseries etc.

I think you can say the PAPR VAS spec has two different types of credits, rather than say a specific hypervisor is introducing them.

Quoted 12 lines

flag with VAS_TX_WIN_OPEN ioctl to open QoS type window.

---
 arch/powerpc/include/uapi/asm/vas-api.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/powerpc/include/uapi/asm/vas-api.h +0 -0
index ebd4b2424785..eb7c8694174f 100644

arch/powerpc/include/uapi/asm/vas-api.h line 20

Quoted 8 lines
arch/powerpc/include/uapi/asm/vas-api.h +4 -0
@@ -13,11 +13,15 @@
 #define VAS_MAGIC	'v'
 #define VAS_TX_WIN_OPEN	_IOW(VAS_MAGIC, 0x20, struct vas_tx_win_open_attr)
 
+/* Flags to VAS TX open window ioctl */
+/* To allocate a window with QoS credit, otherwise default credit is used */
+#define	VAS_WIN_QOS_CREDITS	0x0000000000000001
+
 struct vas_tx_win_open_attr {

Some consistency of naming might help, VAS_TX_WIN_FLAG_QOS_CREDIT.

Quoted 13 lines
arch/powerpc/include/uapi/asm/vas-api.h +1 -1
 	__u32	version;
 	__s16	vas_id;	/* specific instance of vas or -1 for default */
 	__u16	reserved1;
-	__u64	flags;	/* Future use */
+	__u64	flags;
 	__u64	reserved2[6];
 };
 
Signature · 1 line
2.18.2
#21 lore raw

Re: [V3 PATCH 07/16] powerpc/vas: Define QoS credit flag to allocate window

Quoted 16 lines · Nicholas Piggin

Excerpts from Haren Myneni's message of April 18, 2021 7:06 am:

Quoted 11 lines

pHyp introduces two different type of credits: Default and Quality of service (QoS).

The total number of default credits available on each LPAR depends on CPU resources configured. But these credits can be shared or over-committed across LPARs in shared mode which can result in paste command failure (RMA_busy). To avoid NX HW contention, phyp introduces QoS credit type which makes sure guaranteed access to NX resources. The system admins can assign QoS credits for each LPAR via HMC.

Quoted 3 lines

Default credit type is used to allocate a VAS window by default as on powerVM implementation. But the process can pass VAS_WIN_QOS_CREDITS

Quoted 7 lines

There's some interchanging of pHyp and PowerVM in the series.

PowerVM is probably the better term to use, with uppercase P. Unless you mean PAPR or pseries etc.

I think you can say the PAPR VAS spec has two different types of credits, rather than say a specific hypervisor is introducing them.

DEF and QoS credits types are introduced by the hypervisor, not VAS PAPR. We did not have these types on powerNV.

Quoted 18 lines
Quoted 18 lines

flag with VAS_TX_WIN_OPEN ioctl to open QoS type window.

---
 arch/powerpc/include/uapi/asm/vas-api.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/powerpc/include/uapi/asm/vas-api.h +0 -0
index ebd4b2424785..eb7c8694174f 100644
@@ -13,11 +13,15 @@
 #define VAS_MAGIC	'v'
 #define VAS_TX_WIN_OPEN	_IOW(VAS_MAGIC, 0x20, struct
 vas_tx_win_open_attr)
 
Quoted 8 lines

arch/powerpc/include/uapi/asm/vas-api.h line 20

Quoted 6 lines
arch/powerpc/include/uapi/asm/vas-api.h +2 -0
+/* Flags to VAS TX open window ioctl */
+/* To allocate a window with QoS credit, otherwise default credit

is used */

arch/powerpc/include/uapi/asm/vas-api.h +2 -0
+#define	VAS_WIN_QOS_CREDITS	0x0000000000000001
+
 struct vas_tx_win_open_attr {

Some consistency of naming might help, VAS_TX_WIN_FLAG_QOS_CREDIT.

Sure, will change.

Quoted 15 lines
Quoted 15 lines
arch/powerpc/include/uapi/asm/vas-api.h +0 -0
 	__u32	version;
 	__s16	vas_id;	/* specific instance of vas or -1 for

default */

arch/powerpc/include/uapi/asm/vas-api.h +1 -1
 	__u16	reserved1;
-	__u64	flags;	/* Future use */
+	__u64	flags;
 	__u64	reserved2[6];
 };
Signature · 2 lines
-- 
2.18.2
#22 lore raw

PATCH v3 8/16 powerpc/pseries/VAS: Implement allocate/modify/deallocate HCALLS

In reply to Haren Myneni

This patch adds the following HCALLs which are used to allocate, modify and deallocate VAS windows.

H_ALLOCATE_VAS_WINDOW: Allocate VAS window
H_DEALLOCATE_VAS_WINDOW: Close VAS window
H_MODIFY_VAS_WINDOW: Setup window before using

Also adds phyp call (H_QUERY_VAS_CAPABILITIES) to get all VAS capabilities that phyp provides.

---
 arch/powerpc/platforms/pseries/vas.c | 217 +++++++++++++++++++++++++++
 1 file changed, 217 insertions(+)
 create mode 100644 arch/powerpc/platforms/pseries/vas.c
arch/powerpc/platforms/pseries/vas.c new file +217 -0
new file mode 100644
index 000000000000..06960151477c
@@ -0,0 +1,217 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright 2020-21 IBM Corp.
+ */
+
+#define pr_fmt(fmt) "vas: " fmt
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/export.h>
+#include <linux/types.h>
+#include <linux/delay.h>
+#include <linux/slab.h>
+#include <linux/irqdomain.h>
+#include <linux/interrupt.h>
+#include <linux/sched/mm.h>
+#include <linux/mmu_context.h>
+#include <asm/hvcall.h>
+#include <asm/hvconsole.h>
+#include <asm/machdep.h>
+#include <asm/plpar_wrappers.h>
+#include <asm/vas.h>
+#include "vas.h"
+
+#define	VAS_INVALID_WIN_ADDRESS	0xFFFFFFFFFFFFFFFFul
+#define	VAS_DEFAULT_DOMAIN_ID	0xFFFFFFFFFFFFFFFFul
+/* Authority Mask Register (AMR) value is not supported in */
+/* linux implementation. So pass '0' to modify window HCALL */
+#define	VAS_AMR_VALUE	0
+/* phyp allows one credit per window right now */
+#define DEF_WIN_CREDS		1
+
+static int64_t hcall_return_busy_check(int64_t rc)
+{
+	/* Check if we are stalled for some time */
+	if (H_IS_LONG_BUSY(rc)) {
+		msleep(get_longbusy_msecs(rc));
+		rc = H_BUSY;
+	} else if (rc == H_BUSY) {
+		cond_resched();
+	}
+
+	return rc;
+}
+
+/*
+ * Allocate VAS window HCALL
+ */
+static int plpar_vas_allocate_window(struct vas_window *win, u64 *domain,
+				     u8 wintype, u16 credits)
+{
+	long retbuf[PLPAR_HCALL9_BUFSIZE] = {0};
+	int64_t rc;
+
+	do {
+		rc = plpar_hcall9(H_ALLOCATE_VAS_WINDOW, retbuf, wintype,
+				  credits, domain[0], domain[1], domain[2],
+				  domain[3], domain[4], domain[5]);
+
+		rc = hcall_return_busy_check(rc);
+	} while (rc == H_BUSY);
+
+	switch (rc) {
+	case H_SUCCESS:
+		win->winid = retbuf[0];
+		win->lpar.win_addr = retbuf[1];
+		win->lpar.complete_irq = retbuf[2];
+		win->lpar.fault_irq = retbuf[3];
+		if (win->lpar.win_addr == VAS_INVALID_WIN_ADDRESS) {
+			pr_err("HCALL(%x): COPY/PASTE is not supported\n",
+				H_ALLOCATE_VAS_WINDOW);
+			return -ENOTSUPP;
+		}
+		return 0;
+	case H_PARAMETER:
+		pr_err("HCALL(%x): Invalid window type (%u)\n",
+			H_ALLOCATE_VAS_WINDOW, wintype);
+		return -EINVAL;
+	case H_P2:
+		pr_err("HCALL(%x): Credits(%u) exceed maximum window credits\n",
+			H_ALLOCATE_VAS_WINDOW, credits);
+		return -EINVAL;
+	case H_COP_HW:
+		pr_err("HCALL(%x): User-mode COPY/PASTE is not supported\n",
+			H_ALLOCATE_VAS_WINDOW);
+		return -ENOTSUPP;
+	case H_RESOURCE:
+		pr_err("HCALL(%x): LPAR credit limit exceeds window limit\n",
+			H_ALLOCATE_VAS_WINDOW);
+		return -EPERM;
+	case H_CONSTRAINED:
+		pr_err("HCALL(%x): Credits (%u) are not available\n",
+			H_ALLOCATE_VAS_WINDOW, credits);
+		return -EPERM;
+	default:
+		pr_err("HCALL(%x): Unexpected error %lld\n",
+			H_ALLOCATE_VAS_WINDOW, rc);
+		return -EIO;
+	}
+}
+
+/*
+ * Deallocate VAS window HCALL.
+ */
+static int plpar_vas_deallocate_window(u64 winid)
+{
+	int64_t rc;
+
+	do {
+		rc = plpar_hcall_norets(H_DEALLOCATE_VAS_WINDOW, winid);
+
+		rc = hcall_return_busy_check(rc);
+	} while (rc == H_BUSY);
+
+	switch (rc) {
+	case H_SUCCESS:
+		return 0;
+	case H_PARAMETER:
+		pr_err("HCALL(%x): Invalid window ID %llu\n",
+			H_DEALLOCATE_VAS_WINDOW, winid);
+		return -EINVAL;
+	case H_STATE:
+		pr_err("HCALL(%x): Window(%llu): Invalid page table entries\n",
+			H_DEALLOCATE_VAS_WINDOW, winid);
+		return -EPERM;
+	default:
+		pr_err("HCALL(%x): Unexpected error %lld for window(%llu)\n",
+			H_DEALLOCATE_VAS_WINDOW, rc, winid);
+		return -EIO;
+	}
+}
+
+/*
+ * Modify VAS window.
+ * After the window is opened with allocate window HCALL, configure it
+ * with flags and LPAR PID before using.
+ */
+static int plpar_vas_modify_window(struct vas_window *win)
+{
+	int64_t rc;
+	u32 lpid = mfspr(SPRN_PID);
+
+	/*
+	 * AMR value is not supported in Linux implementation
+	 * phyp ignores it if 0 is passed.
+	 */
+	do {
+		rc = plpar_hcall_norets(H_MODIFY_VAS_WINDOW, win->winid,
+					lpid, 0, VAS_MOD_WIN_FLAGS,
+					VAS_AMR_VALUE);
+
+		rc = hcall_return_busy_check(rc);
+	} while (rc == H_BUSY);
+
+	switch (rc) {
+	case H_SUCCESS:
+		return 0;
+	case H_PARAMETER:
+		pr_err("HCALL(%x): Invalid window ID %u\n",
+			H_MODIFY_VAS_WINDOW, win->winid);
+		return -EINVAL;
+	case H_P2:
+		pr_err("HCALL(%x): Window(%d): Invalid LPAR Process ID %u\n",
+			H_MODIFY_VAS_WINDOW, lpid, win->winid);
+		return -EINVAL;
+	case H_P3:
+		/* LPAR thread ID is deprecated on P10 */
+		pr_err("HCALL(%x): Invalid LPAR Thread ID for window(%u)\n",
+			H_MODIFY_VAS_WINDOW, win->winid);
+		return -EINVAL;
+	case H_STATE:
+		pr_err("HCALL(%x): Jobs in progress, Can't modify window(%u)\n",
+			H_MODIFY_VAS_WINDOW, win->winid);
+		return -EBUSY;
+	default:
+		pr_err("HCALL(%x): Unexpected error %lld for window(%u)\n",
+			H_MODIFY_VAS_WINDOW, rc, win->winid);
+		return -EIO;
+	}
+}
+
+/*
+ * This HCALL is used to determine the capabilities that pHyp provides.
+ * @hcall: H_QUERY_VAS_CAPABILITIES or H_QUERY_NX_CAPABILITIES
+ * @query_type: If 0 is passed, phyp returns the overall capabilities
+ *		which provides all feature(s) that are available. Then
+ *		query phyp to get the corresponding capabilities for
+ *		the specific feature.
+ *		Example: H_QUERY_VAS_CAPABILITIES provides VAS GZIP QoS
+ *			and VAS GZIP Default capabilities.
+ *			H_QUERY_NX_CAPABILITIES provides NX GZIP
+ *			capabilities.
+ * @result: Return buffer to save capabilities.
+ */
+int plpar_vas_query_capabilities(const u64 hcall, u8 query_type,
+					u64 result)
+{
+	int64_t rc;
+
+	rc = plpar_hcall_norets(hcall, query_type, result);
+
+	switch (rc) {
+	case H_SUCCESS:
+		return 0;
+	case H_PARAMETER:
+		pr_err("HCALL(%llx): Invalid query type %u\n", hcall,
+			query_type);
+		return -EINVAL;
+	case H_PRIVILEGE:
+		pr_err("HCALL(%llx): Invalid result buffer 0x%llx\n",
+			hcall, result);
+		return -EACCES;
+	default:
+		pr_err("HCALL(%llx): Unexpected error %lld\n", hcall, rc);
+		return -EIO;
+	}
+}
Signature · 1 line
2.18.2
reviewer Reviewed-by on v3 8/16
#23 lore raw

Re: [V3 PATCH 08/16] powerpc/pseries/VAS: Implement allocate/modify/deallocate HCALLS

Excerpts from Haren Myneni's message of April 18, 2021 7:07 am:

Quoted 3 lines

This patch adds the following HCALLs which are used to allocate, modify and deallocate VAS windows.

I don't like to be nitpicky about the language, but this is adding the hcall wrappers. Implementing the hcall would be adding it to KVM. Otherwise looks okay I think.

Quoted 6 lines

H_ALLOCATE_VAS_WINDOW: Allocate VAS window
H_DEALLOCATE_VAS_WINDOW: Close VAS window
H_MODIFY_VAS_WINDOW: Setup window before using

Also adds phyp call (H_QUERY_VAS_CAPABILITIES) to get all VAS capabilities that phyp provides.

"PAPR hcall to get VAS capabilities provided by the hypervisor"

Thanks,
Nick

Quoted 235 lines
---
 arch/powerpc/platforms/pseries/vas.c | 217 +++++++++++++++++++++++++++
 1 file changed, 217 insertions(+)
 create mode 100644 arch/powerpc/platforms/pseries/vas.c
arch/powerpc/platforms/pseries/vas.c new file +217 -0
new file mode 100644
index 000000000000..06960151477c
@@ -0,0 +1,217 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright 2020-21 IBM Corp.
+ */
+
+#define pr_fmt(fmt) "vas: " fmt
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/export.h>
+#include <linux/types.h>
+#include <linux/delay.h>
+#include <linux/slab.h>
+#include <linux/irqdomain.h>
+#include <linux/interrupt.h>
+#include <linux/sched/mm.h>
+#include <linux/mmu_context.h>
+#include <asm/hvcall.h>
+#include <asm/hvconsole.h>
+#include <asm/machdep.h>
+#include <asm/plpar_wrappers.h>
+#include <asm/vas.h>
+#include "vas.h"
+
+#define	VAS_INVALID_WIN_ADDRESS	0xFFFFFFFFFFFFFFFFul
+#define	VAS_DEFAULT_DOMAIN_ID	0xFFFFFFFFFFFFFFFFul
+/* Authority Mask Register (AMR) value is not supported in */
+/* linux implementation. So pass '0' to modify window HCALL */
+#define	VAS_AMR_VALUE	0
+/* phyp allows one credit per window right now */
+#define DEF_WIN_CREDS		1
+
+static int64_t hcall_return_busy_check(int64_t rc)
+{
+	/* Check if we are stalled for some time */
+	if (H_IS_LONG_BUSY(rc)) {
+		msleep(get_longbusy_msecs(rc));
+		rc = H_BUSY;
+	} else if (rc == H_BUSY) {
+		cond_resched();
+	}
+
+	return rc;
+}
+
+/*
+ * Allocate VAS window HCALL
+ */
+static int plpar_vas_allocate_window(struct vas_window *win, u64 *domain,
+				     u8 wintype, u16 credits)
+{
+	long retbuf[PLPAR_HCALL9_BUFSIZE] = {0};
+	int64_t rc;
+
+	do {
+		rc = plpar_hcall9(H_ALLOCATE_VAS_WINDOW, retbuf, wintype,
+				  credits, domain[0], domain[1], domain[2],
+				  domain[3], domain[4], domain[5]);
+
+		rc = hcall_return_busy_check(rc);
+	} while (rc == H_BUSY);
+
+	switch (rc) {
+	case H_SUCCESS:
+		win->winid = retbuf[0];
+		win->lpar.win_addr = retbuf[1];
+		win->lpar.complete_irq = retbuf[2];
+		win->lpar.fault_irq = retbuf[3];
+		if (win->lpar.win_addr == VAS_INVALID_WIN_ADDRESS) {
+			pr_err("HCALL(%x): COPY/PASTE is not supported\n",
+				H_ALLOCATE_VAS_WINDOW);
+			return -ENOTSUPP;
+		}
+		return 0;
+	case H_PARAMETER:
+		pr_err("HCALL(%x): Invalid window type (%u)\n",
+			H_ALLOCATE_VAS_WINDOW, wintype);
+		return -EINVAL;
+	case H_P2:
+		pr_err("HCALL(%x): Credits(%u) exceed maximum window credits\n",
+			H_ALLOCATE_VAS_WINDOW, credits);
+		return -EINVAL;
+	case H_COP_HW:
+		pr_err("HCALL(%x): User-mode COPY/PASTE is not supported\n",
+			H_ALLOCATE_VAS_WINDOW);
+		return -ENOTSUPP;
+	case H_RESOURCE:
+		pr_err("HCALL(%x): LPAR credit limit exceeds window limit\n",
+			H_ALLOCATE_VAS_WINDOW);
+		return -EPERM;
+	case H_CONSTRAINED:
+		pr_err("HCALL(%x): Credits (%u) are not available\n",
+			H_ALLOCATE_VAS_WINDOW, credits);
+		return -EPERM;
+	default:
+		pr_err("HCALL(%x): Unexpected error %lld\n",
+			H_ALLOCATE_VAS_WINDOW, rc);
+		return -EIO;
+	}
+}
+
+/*
+ * Deallocate VAS window HCALL.
+ */
+static int plpar_vas_deallocate_window(u64 winid)
+{
+	int64_t rc;
+
+	do {
+		rc = plpar_hcall_norets(H_DEALLOCATE_VAS_WINDOW, winid);
+
+		rc = hcall_return_busy_check(rc);
+	} while (rc == H_BUSY);
+
+	switch (rc) {
+	case H_SUCCESS:
+		return 0;
+	case H_PARAMETER:
+		pr_err("HCALL(%x): Invalid window ID %llu\n",
+			H_DEALLOCATE_VAS_WINDOW, winid);
+		return -EINVAL;
+	case H_STATE:
+		pr_err("HCALL(%x): Window(%llu): Invalid page table entries\n",
+			H_DEALLOCATE_VAS_WINDOW, winid);
+		return -EPERM;
+	default:
+		pr_err("HCALL(%x): Unexpected error %lld for window(%llu)\n",
+			H_DEALLOCATE_VAS_WINDOW, rc, winid);
+		return -EIO;
+	}
+}
+
+/*
+ * Modify VAS window.
+ * After the window is opened with allocate window HCALL, configure it
+ * with flags and LPAR PID before using.
+ */
+static int plpar_vas_modify_window(struct vas_window *win)
+{
+	int64_t rc;
+	u32 lpid = mfspr(SPRN_PID);
+
+	/*
+	 * AMR value is not supported in Linux implementation
+	 * phyp ignores it if 0 is passed.
+	 */
+	do {
+		rc = plpar_hcall_norets(H_MODIFY_VAS_WINDOW, win->winid,
+					lpid, 0, VAS_MOD_WIN_FLAGS,
+					VAS_AMR_VALUE);
+
+		rc = hcall_return_busy_check(rc);
+	} while (rc == H_BUSY);
+
+	switch (rc) {
+	case H_SUCCESS:
+		return 0;
+	case H_PARAMETER:
+		pr_err("HCALL(%x): Invalid window ID %u\n",
+			H_MODIFY_VAS_WINDOW, win->winid);
+		return -EINVAL;
+	case H_P2:
+		pr_err("HCALL(%x): Window(%d): Invalid LPAR Process ID %u\n",
+			H_MODIFY_VAS_WINDOW, lpid, win->winid);
+		return -EINVAL;
+	case H_P3:
+		/* LPAR thread ID is deprecated on P10 */
+		pr_err("HCALL(%x): Invalid LPAR Thread ID for window(%u)\n",
+			H_MODIFY_VAS_WINDOW, win->winid);
+		return -EINVAL;
+	case H_STATE:
+		pr_err("HCALL(%x): Jobs in progress, Can't modify window(%u)\n",
+			H_MODIFY_VAS_WINDOW, win->winid);
+		return -EBUSY;
+	default:
+		pr_err("HCALL(%x): Unexpected error %lld for window(%u)\n",
+			H_MODIFY_VAS_WINDOW, rc, win->winid);
+		return -EIO;
+	}
+}
+
+/*
+ * This HCALL is used to determine the capabilities that pHyp provides.
+ * @hcall: H_QUERY_VAS_CAPABILITIES or H_QUERY_NX_CAPABILITIES
+ * @query_type: If 0 is passed, phyp returns the overall capabilities
+ *		which provides all feature(s) that are available. Then
+ *		query phyp to get the corresponding capabilities for
+ *		the specific feature.
+ *		Example: H_QUERY_VAS_CAPABILITIES provides VAS GZIP QoS
+ *			and VAS GZIP Default capabilities.
+ *			H_QUERY_NX_CAPABILITIES provides NX GZIP
+ *			capabilities.
+ * @result: Return buffer to save capabilities.
+ */
+int plpar_vas_query_capabilities(const u64 hcall, u8 query_type,
+					u64 result)
+{
+	int64_t rc;
+
+	rc = plpar_hcall_norets(hcall, query_type, result);
+
+	switch (rc) {
+	case H_SUCCESS:
+		return 0;
+	case H_PARAMETER:
+		pr_err("HCALL(%llx): Invalid query type %u\n", hcall,
+			query_type);
+		return -EINVAL;
+	case H_PRIVILEGE:
+		pr_err("HCALL(%llx): Invalid result buffer 0x%llx\n",
+			hcall, result);
+		return -EACCES;
+	default:
+		pr_err("HCALL(%llx): Unexpected error %lld\n", hcall, rc);
+		return -EIO;
+	}
+}
Signature · 1 line
2.18.2
#24 lore raw

PATCH v3 9/16 powerpc/pseries/vas: Implement to get all capabilities

In reply to Haren Myneni

pHyp provides various VAS capabilities such as GZIP default and QoS capabilities which are used to determine total number of credits available in LPAR, maximum window credits, maximum LPAR credits, whether usermode copy/paste is supported, and etc.

So first retrieve overall vas capabilities using H_QUERY_VAS_CAPABILITIES HCALL which tells the specific features that are available. Then retrieve the specific capabilities by using the feature type in H_QUERY_VAS_CAPABILITIES HCALL.

pHyp supports only GZIP default and GZIP QoS capabilities right now.

---
 arch/powerpc/platforms/pseries/vas.c | 130 +++++++++++++++++++++++++++
 1 file changed, 130 insertions(+)
arch/powerpc/platforms/pseries/vas.c +130 -0
index 06960151477c..35946fb02995 100644
@@ -30,6 +30,13 @@
 /* phyp allows one credit per window right now */
 #define DEF_WIN_CREDS		1
 
+static struct vas_all_capabs capabs_all;
+static int copypaste_feat;
+
+struct vas_capabs vcapabs[VAS_MAX_FEAT_TYPE];
+
+DEFINE_MUTEX(vas_pseries_mutex);
+
 static int64_t hcall_return_busy_check(int64_t rc)
 {
 	/* Check if we are stalled for some time */
@@ -215,3 +222,126 @@ int plpar_vas_query_capabilities(const u64 hcall, u8 query_type,
 		return -EIO;
 	}
 }
+
+/*
+ * Get the specific capabilities based on the feature type.
+ * Right now supports GZIP default and GZIP QoS capabilities.
+ */
+static int get_vas_capabilities(u8 feat, enum vas_cop_feat_type type,
+				struct vas_ct_capabs_be *capab_be)
+{
+	struct vas_ct_capabs *capab;
+	struct vas_capabs *vcapab;
+	int rc = 0;
+
+	vcapab = &vcapabs[type];
+	memset(vcapab, 0, sizeof(*vcapab));
+	INIT_LIST_HEAD(&vcapab->list);
+
+	capab = &vcapab->capab;
+
+	rc = plpar_vas_query_capabilities(H_QUERY_VAS_CAPABILITIES, feat,
+					  (u64)virt_to_phys(capab_be));
+	if (rc)
+		return rc;
+
+	capab->user_mode = capab_be->user_mode;
+	if (!(capab->user_mode & VAS_COPY_PASTE_USER_MODE)) {
+		pr_err("User space COPY/PASTE is not supported\n");
+		return -ENOTSUPP;
+	}
+
+	snprintf(capab->name, VAS_DESCR_LEN + 1, "%.8s",
+		 (char *)&capab_be->descriptor);
+	capab->descriptor = be64_to_cpu(capab_be->descriptor);
+	capab->win_type = capab_be->win_type;
+	if (capab->win_type >= VAS_MAX_FEAT_TYPE) {
+		pr_err("Unsupported window type %u\n", capab->win_type);
+		return -EINVAL;
+	}
+	capab->max_lpar_creds = be16_to_cpu(capab_be->max_lpar_creds);
+	capab->max_win_creds = be16_to_cpu(capab_be->max_win_creds);
+	atomic_set(&capab->target_lpar_creds,
+		   be16_to_cpu(capab_be->target_lpar_creds));
+	if (feat == VAS_GZIP_DEF_FEAT) {
+		capab->def_lpar_creds = be16_to_cpu(capab_be->def_lpar_creds);
+
+		if (capab->max_win_creds < DEF_WIN_CREDS) {
+			pr_err("Window creds(%u) > max allowed window creds(%u)\n",
+			       DEF_WIN_CREDS, capab->max_win_creds);
+			return -EINVAL;
+		}
+	}
+
+	copypaste_feat = 1;
+
+	return 0;
+}
+
+static int __init pseries_vas_init(void)
+{
+	struct vas_ct_capabs_be *ct_capabs_be;
+	struct vas_all_capabs_be *capabs_be;
+	int rc;
+
+	/*
+	 * Linux supports user space COPY/PASTE only with Radix
+	 */
+	if (!radix_enabled()) {
+		pr_err("API is supported only with radix page tables\n");
+		return -ENOTSUPP;
+	}
+
+	capabs_be = kmalloc(sizeof(*capabs_be), GFP_KERNEL);
+	if (!capabs_be)
+		return -ENOMEM;
+	/*
+	 * Get VAS overall capabilities by passing 0 to feature type.
+	 */
+	rc = plpar_vas_query_capabilities(H_QUERY_VAS_CAPABILITIES, 0,
+					  (u64)virt_to_phys(capabs_be));
+	if (rc)
+		goto out;
+
+	snprintf(capabs_all.name, VAS_DESCR_LEN, "%.7s",
+		 (char *)&capabs_be->descriptor);
+	capabs_all.descriptor = be64_to_cpu(capabs_be->descriptor);
+	capabs_all.feat_type = be64_to_cpu(capabs_be->feat_type);
+
+	ct_capabs_be = kmalloc(sizeof(*ct_capabs_be), GFP_KERNEL);
+	if (!ct_capabs_be) {
+		rc = -ENOMEM;
+		goto out;
+	}
+	/*
+	 * QOS capabilities available
+	 */
+	if (capabs_all.feat_type & VAS_GZIP_QOS_FEAT_BIT) {
+		rc = get_vas_capabilities(VAS_GZIP_QOS_FEAT,
+					  VAS_GZIP_QOS_FEAT_TYPE, ct_capabs_be);
+
+		if (rc)
+			goto out_ct;
+	}
+	/*
+	 * Default capabilities available
+	 */
+	if (capabs_all.feat_type & VAS_GZIP_DEF_FEAT_BIT) {
+		rc = get_vas_capabilities(VAS_GZIP_DEF_FEAT,
+					  VAS_GZIP_DEF_FEAT_TYPE, ct_capabs_be);
+		if (rc)
+			goto out_ct;
+	}
+
+	if (!copypaste_feat)
+		pr_err("GZIP feature is not supported\n");
+
+	pr_info("GZIP feature is available\n");
+
+out_ct:
+	kfree(ct_capabs_be);
+out:
+	kfree(capabs_be);
+	return rc;
+}
+machine_device_initcall(pseries, pseries_vas_init);
Signature · 1 line
2.18.2
#25 lore raw

Re: [V3 PATCH 09/16] powerpc/pseries/vas: Implement to get all capabilities

Excerpts from Haren Myneni's message of April 18, 2021 7:08 am:

Quoted 6 lines

pHyp provides various VAS capabilities such as GZIP default and QoS capabilities which are used to determine total number of credits available in LPAR, maximum window credits, maximum LPAR credits, whether usermode copy/paste is supported, and etc.

Quoted 6 lines

So first retrieve overall vas capabilities using H_QUERY_VAS_CAPABILITIES HCALL which tells the specific features that are available. Then retrieve the specific capabilities by using the feature type in H_QUERY_VAS_CAPABILITIES HCALL.

pHyp supports only GZIP default and GZIP QoS capabilities right now.

Changelog and title could use a bit of work.

Quoted 11 lines
---
 arch/powerpc/platforms/pseries/vas.c | 130 +++++++++++++++++++++++++++
 1 file changed, 130 insertions(+)
arch/powerpc/platforms/pseries/vas.c +0 -0
index 06960151477c..35946fb02995 100644

arch/powerpc/platforms/pseries/vas.c line 33

Quoted 4 lines
arch/powerpc/platforms/pseries/vas.c +1 -0
@@ -30,6 +30,13 @@
 /* phyp allows one credit per window right now */
 #define DEF_WIN_CREDS		1
 
+static struct vas_all_capabs capabs_all;

Does this name come from PAPR? If not, capabilities or caps are better for readability than capabs.

arch/powerpc/platforms/pseries/vas.c line 34

Quoted 1 line
arch/powerpc/platforms/pseries/vas.c +1 -0
+static int copypaste_feat;

Should be a bool? And what does it mean? copy-paste is a host core capability.

arch/powerpc/platforms/pseries/vas.c line 38

Quoted 4 lines
arch/powerpc/platforms/pseries/vas.c +4 -0
+
+struct vas_capabs vcapabs[VAS_MAX_FEAT_TYPE];
+
+DEFINE_MUTEX(vas_pseries_mutex);

Can these be made static if they're only used here, and export them if a future patch uses them (or add the header declaration now).

Quoted 108 lines
arch/powerpc/platforms/pseries/vas.c +101 -0
+
 static int64_t hcall_return_busy_check(int64_t rc)
 {
 	/* Check if we are stalled for some time */
@@ -215,3 +222,126 @@ int plpar_vas_query_capabilities(const u64 hcall, u8 query_type,
 		return -EIO;
 	}
 }
+
+/*
+ * Get the specific capabilities based on the feature type.
+ * Right now supports GZIP default and GZIP QoS capabilities.
+ */
+static int get_vas_capabilities(u8 feat, enum vas_cop_feat_type type,
+				struct vas_ct_capabs_be *capab_be)
+{
+	struct vas_ct_capabs *capab;
+	struct vas_capabs *vcapab;
+	int rc = 0;
+
+	vcapab = &vcapabs[type];
+	memset(vcapab, 0, sizeof(*vcapab));
+	INIT_LIST_HEAD(&vcapab->list);
+
+	capab = &vcapab->capab;
+
+	rc = plpar_vas_query_capabilities(H_QUERY_VAS_CAPABILITIES, feat,
+					  (u64)virt_to_phys(capab_be));
+	if (rc)
+		return rc;
+
+	capab->user_mode = capab_be->user_mode;
+	if (!(capab->user_mode & VAS_COPY_PASTE_USER_MODE)) {
+		pr_err("User space COPY/PASTE is not supported\n");
+		return -ENOTSUPP;
+	}
+
+	snprintf(capab->name, VAS_DESCR_LEN + 1, "%.8s",
+		 (char *)&capab_be->descriptor);
+	capab->descriptor = be64_to_cpu(capab_be->descriptor);
+	capab->win_type = capab_be->win_type;
+	if (capab->win_type >= VAS_MAX_FEAT_TYPE) {
+		pr_err("Unsupported window type %u\n", capab->win_type);
+		return -EINVAL;
+	}
+	capab->max_lpar_creds = be16_to_cpu(capab_be->max_lpar_creds);
+	capab->max_win_creds = be16_to_cpu(capab_be->max_win_creds);
+	atomic_set(&capab->target_lpar_creds,
+		   be16_to_cpu(capab_be->target_lpar_creds));
+	if (feat == VAS_GZIP_DEF_FEAT) {
+		capab->def_lpar_creds = be16_to_cpu(capab_be->def_lpar_creds);
+
+		if (capab->max_win_creds < DEF_WIN_CREDS) {
+			pr_err("Window creds(%u) > max allowed window creds(%u)\n",
+			       DEF_WIN_CREDS, capab->max_win_creds);
+			return -EINVAL;
+		}
+	}
+
+	copypaste_feat = 1;
+
+	return 0;
+}
+
+static int __init pseries_vas_init(void)
+{
+	struct vas_ct_capabs_be *ct_capabs_be;
+	struct vas_all_capabs_be *capabs_be;
+	int rc;
+
+	/*
+	 * Linux supports user space COPY/PASTE only with Radix
+	 */
+	if (!radix_enabled()) {
+		pr_err("API is supported only with radix page tables\n");
+		return -ENOTSUPP;
+	}
+
+	capabs_be = kmalloc(sizeof(*capabs_be), GFP_KERNEL);
+	if (!capabs_be)
+		return -ENOMEM;
+	/*
+	 * Get VAS overall capabilities by passing 0 to feature type.
+	 */
+	rc = plpar_vas_query_capabilities(H_QUERY_VAS_CAPABILITIES, 0,
+					  (u64)virt_to_phys(capabs_be));
+	if (rc)
+		goto out;
+
+	snprintf(capabs_all.name, VAS_DESCR_LEN, "%.7s",
+		 (char *)&capabs_be->descriptor);
+	capabs_all.descriptor = be64_to_cpu(capabs_be->descriptor);
+	capabs_all.feat_type = be64_to_cpu(capabs_be->feat_type);
+
+	ct_capabs_be = kmalloc(sizeof(*ct_capabs_be), GFP_KERNEL);
+	if (!ct_capabs_be) {
+		rc = -ENOMEM;
+		goto out;
+	}
+	/*
+	 * QOS capabilities available
+	 */
+	if (capabs_all.feat_type & VAS_GZIP_QOS_FEAT_BIT) {
+		rc = get_vas_capabilities(VAS_GZIP_QOS_FEAT,
+					  VAS_GZIP_QOS_FEAT_TYPE, ct_capabs_be);
+
+		if (rc)
+			goto out_ct;

arch/powerpc/platforms/pseries/vas.c int plpar_vas_query_capabilities(const u64 hcall, u8 query_type, · line 334

Quoted 10 lines
arch/powerpc/platforms/pseries/vas.c +10 -0
+	}
+	/*
+	 * Default capabilities available
+	 */
+	if (capabs_all.feat_type & VAS_GZIP_DEF_FEAT_BIT) {
+		rc = get_vas_capabilities(VAS_GZIP_DEF_FEAT,
+					  VAS_GZIP_DEF_FEAT_TYPE, ct_capabs_be);
+		if (rc)
+			goto out_ct;
+	}

Using the same buffer for two hcalls? Do they fill in different parts of it?

arch/powerpc/platforms/pseries/vas.c int plpar_vas_query_capabilities(const u64 hcall, u8 query_type, · line 337

Quoted 3 lines
arch/powerpc/platforms/pseries/vas.c +3 -0
+
+	if (!copypaste_feat)
+		pr_err("GZIP feature is not supported\n");

This is dead code AFAIKS, because errors will always branch to out.

Thanks,
Nick

Quoted 15 lines
arch/powerpc/platforms/pseries/vas.c +10 -0
+
+	pr_info("GZIP feature is available\n");
+
+out_ct:
+	kfree(ct_capabs_be);
+out:
+	kfree(capabs_be);
+	return rc;
+}
+machine_device_initcall(pseries, pseries_vas_init);
Signature · 1 line
2.18.2
#26 lore raw

Re: [V3 PATCH 09/16] powerpc/pseries/vas: Implement to get all capabilities

Quoted 32 lines · Nicholas Piggin

Excerpts from Haren Myneni's message of April 18, 2021 7:08 am:

Quoted 5 lines

pHyp provides various VAS capabilities such as GZIP default and QoS capabilities which are used to determine total number of credits available in LPAR, maximum window credits, maximum LPAR credits, whether usermode copy/paste is supported, and etc.

Quoted 8 lines

So first retrieve overall vas capabilities using H_QUERY_VAS_CAPABILITIES HCALL which tells the specific features that
are available. Then retrieve the specific capabilities by using the feature type in H_QUERY_VAS_CAPABILITIES HCALL.

pHyp supports only GZIP default and GZIP QoS capabilities right now.

Changelog and title could use a bit of work.

Quoted 15 lines
---
 arch/powerpc/platforms/pseries/vas.c | 130
+++++++++++++++++++++++++++
 1 file changed, 130 insertions(+)
arch/powerpc/platforms/pseries/vas.c +0 -0
index 06960151477c..35946fb02995 100644
@@ -30,6 +30,13 @@
 /* phyp allows one credit per window right now */
 #define DEF_WIN_CREDS		1
 
Quoted 5 lines

arch/powerpc/platforms/pseries/vas.c line 33

Quoted 1 line
arch/powerpc/platforms/pseries/vas.c +1 -0
+static struct vas_all_capabs capabs_all;

Does this name come from PAPR? If not, capabilities or caps are better
for readability than capabs.

No capabs is not from PAPR, I just added capabs as short name for capabilities. I will change it to caps as suggested.

Quoted 5 lines

arch/powerpc/platforms/pseries/vas.c line 34

Quoted 2 lines
arch/powerpc/platforms/pseries/vas.c +1 -0
+static int copypaste_feat;

Should be a bool? And what does it mean? copy-paste is a host core capability.

copypaste_feat can be bool. But is it used to export user space API if reading all capabilities are successful.

Quoted 135 lines

arch/powerpc/platforms/pseries/vas.c line 38

Quoted 5 lines
arch/powerpc/platforms/pseries/vas.c +4 -0
+
+struct vas_capabs vcapabs[VAS_MAX_FEAT_TYPE];
+
+DEFINE_MUTEX(vas_pseries_mutex);

Can these be made static if they're only used here, and export them if a
future patch uses them (or add the header declaration now).

Quoted 124 lines
arch/powerpc/platforms/pseries/vas.c +7 -0
+
 static int64_t hcall_return_busy_check(int64_t rc)
 {
 	/* Check if we are stalled for some time */
@@ -215,3 +222,126 @@ int plpar_vas_query_capabilities(const u64
 hcall, u8 query_type,
 		return -EIO;
 	}
 }
+
+/*
+ * Get the specific capabilities based on the feature type.
+ * Right now supports GZIP default and GZIP QoS capabilities.
+ */
+static int get_vas_capabilities(u8 feat, enum vas_cop_feat_type

type,

arch/powerpc/platforms/pseries/vas.c +13 -0
+				struct vas_ct_capabs_be *capab_be)
+{
+	struct vas_ct_capabs *capab;
+	struct vas_capabs *vcapab;
+	int rc = 0;
+
+	vcapab = &vcapabs[type];
+	memset(vcapab, 0, sizeof(*vcapab));
+	INIT_LIST_HEAD(&vcapab->list);
+
+	capab = &vcapab->capab;
+
+	rc = plpar_vas_query_capabilities(H_QUERY_VAS_CAPABILITIES,

feat,

arch/powerpc/platforms/pseries/vas.c +16 -0
+					  (u64)virt_to_phys(capab_be));
+	if (rc)
+		return rc;
+
+	capab->user_mode = capab_be->user_mode;
+	if (!(capab->user_mode & VAS_COPY_PASTE_USER_MODE)) {
+		pr_err("User space COPY/PASTE is not supported\n");
+		return -ENOTSUPP;
+	}
+
+	snprintf(capab->name, VAS_DESCR_LEN + 1, "%.8s",
+		 (char *)&capab_be->descriptor);
+	capab->descriptor = be64_to_cpu(capab_be->descriptor);
+	capab->win_type = capab_be->win_type;
+	if (capab->win_type >= VAS_MAX_FEAT_TYPE) {
+		pr_err("Unsupported window type %u\n", capab-
Quoted 1 line

win_type);

arch/powerpc/platforms/pseries/vas.c +8 -0
+		return -EINVAL;
+	}
+	capab->max_lpar_creds = be16_to_cpu(capab_be->max_lpar_creds);
+	capab->max_win_creds = be16_to_cpu(capab_be->max_win_creds);
+	atomic_set(&capab->target_lpar_creds,
+		   be16_to_cpu(capab_be->target_lpar_creds));
+	if (feat == VAS_GZIP_DEF_FEAT) {
+		capab->def_lpar_creds = be16_to_cpu(capab_be-
Quoted 1 line

def_lpar_creds);

arch/powerpc/platforms/pseries/vas.c +3 -0
+
+		if (capab->max_win_creds < DEF_WIN_CREDS) {
+			pr_err("Window creds(%u) > max allowed window

creds(%u)\n",

arch/powerpc/platforms/pseries/vas.c +21 -0
+			       DEF_WIN_CREDS, capab->max_win_creds);
+			return -EINVAL;
+		}
+	}
+
+	copypaste_feat = 1;
+
+	return 0;
+}
+
+static int __init pseries_vas_init(void)
+{
+	struct vas_ct_capabs_be *ct_capabs_be;
+	struct vas_all_capabs_be *capabs_be;
+	int rc;
+
+	/*
+	 * Linux supports user space COPY/PASTE only with Radix
+	 */
+	if (!radix_enabled()) {
+		pr_err("API is supported only with radix page

tables\n");

arch/powerpc/platforms/pseries/vas.c +11 -0
+		return -ENOTSUPP;
+	}
+
+	capabs_be = kmalloc(sizeof(*capabs_be), GFP_KERNEL);
+	if (!capabs_be)
+		return -ENOMEM;
+	/*
+	 * Get VAS overall capabilities by passing 0 to feature type.
+	 */
+	rc = plpar_vas_query_capabilities(H_QUERY_VAS_CAPABILITIES, 0,
+					  (u64)virt_to_phys(capabs_be))

;

arch/powerpc/platforms/pseries/vas.c +19 -0
+	if (rc)
+		goto out;
+
+	snprintf(capabs_all.name, VAS_DESCR_LEN, "%.7s",
+		 (char *)&capabs_be->descriptor);
+	capabs_all.descriptor = be64_to_cpu(capabs_be->descriptor);
+	capabs_all.feat_type = be64_to_cpu(capabs_be->feat_type);
+
+	ct_capabs_be = kmalloc(sizeof(*ct_capabs_be), GFP_KERNEL);
+	if (!ct_capabs_be) {
+		rc = -ENOMEM;
+		goto out;
+	}
+	/*
+	 * QOS capabilities available
+	 */
+	if (capabs_all.feat_type & VAS_GZIP_QOS_FEAT_BIT) {
+		rc = get_vas_capabilities(VAS_GZIP_QOS_FEAT,
+					  VAS_GZIP_QOS_FEAT_TYPE,

ct_capabs_be);

arch/powerpc/platforms/pseries/vas.c +10 -0
+
+		if (rc)
+			goto out_ct;
+	}
+	/*
+	 * Default capabilities available
+	 */
+	if (capabs_all.feat_type & VAS_GZIP_DEF_FEAT_BIT) {
+		rc = get_vas_capabilities(VAS_GZIP_DEF_FEAT,
+					  VAS_GZIP_DEF_FEAT_TYPE,
Quoted 8 lines

arch/powerpc/platforms/pseries/vas.c int plpar_vas_query_capabilities(const u64 hcall, u8 query_type, · line 325

Quoted 4 lines

ct_capabs_be);

arch/powerpc/platforms/pseries/vas.c +3 -0
+		if (rc)
+			goto out_ct;
+	}

Using the same buffer for two hcalls? Do they fill in different parts of
it?

Yes, ct_capabs_be buffer is passed to HCALL to get capabilities for DEF and QOS types.

Thanks
Haren

Quoted 25 lines

arch/powerpc/platforms/pseries/vas.c int plpar_vas_query_capabilities(const u64 hcall, u8 query_type, · line 337

Quoted 4 lines
arch/powerpc/platforms/pseries/vas.c +3 -0
+
+	if (!copypaste_feat)
+		pr_err("GZIP feature is not supported\n");

This is dead code AFAIKS, because errors will always branch to out.

Thanks,
Nick

Quoted 15 lines
arch/powerpc/platforms/pseries/vas.c +10 -0
+
+	pr_info("GZIP feature is available\n");
+
+out_ct:
+	kfree(ct_capabs_be);
+out:
+	kfree(capabs_be);
+	return rc;
+}
+machine_device_initcall(pseries, pseries_vas_init);
Signature · 1 line
2.18.2
#27 lore raw

PATCH v3 10/16 powerpc/pseries/vas: Integrate API with open/close windows

In reply to Haren Myneni

This patch adds VAS window allocatioa/close with the corresponding HCALLs. Also changes to integrate with the existing user space VAS API and provide register/unregister functions to NX pseries driver.

The driver register function is used to create the user space interface (/dev/crypto/nx-gzip) and unregister to remove this entry.

The user space process opens this device node and makes an ioctl to allocate VAS window. The close interface is used to deallocate window.

---
 arch/powerpc/include/asm/vas.h          |   5 +
 arch/powerpc/platforms/book3s/Kconfig   |   2 +-
 arch/powerpc/platforms/pseries/Makefile |   1 +
 arch/powerpc/platforms/pseries/vas.c    | 212 ++++++++++++++++++++++++
 4 files changed, 219 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/vas.h +5 -0
index d15784506a54..aa1974aba27e 100644
@@ -270,6 +270,11 @@ struct vas_all_capabs {
 	u64     feat_type;
 };
 
+int plpar_vas_query_capabilities(const u64 hcall, u8 query_type,
+				 u64 result);
+int vas_register_api_pseries(struct module *mod,
+			     enum vas_cop_type cop_type, const char *name);
+void vas_unregister_api_pseries(void);
 #endif
 
 /*
arch/powerpc/platforms/book3s/Kconfig +1 -1
index 51e14db83a79..bed21449e8e5 100644
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 config PPC_VAS
 	bool "IBM Virtual Accelerator Switchboard (VAS)"
-	depends on PPC_POWERNV && PPC_64K_PAGES
+	depends on (PPC_POWERNV || PPC_PSERIES) && PPC_64K_PAGES
 	default y
 	help
 	  This enables support for IBM Virtual Accelerator Switchboard (VAS).
arch/powerpc/platforms/pseries/Makefile +1 -0
index c8a2b0b05ac0..4cda0ef87be0 100644
@@ -30,3 +30,4 @@ obj-$(CONFIG_PPC_SVM)		+= svm.o
 obj-$(CONFIG_FA_DUMP)		+= rtas-fadump.o
 
 obj-$(CONFIG_SUSPEND)		+= suspend.o
+obj-$(CONFIG_PPC_VAS)		+= vas.o
arch/powerpc/platforms/pseries/vas.c +212 -0

218 lines not on this page · raw message

Signature · 1 line
2.18.2
#28 lore raw

Re: [V3 PATCH 10/16] powerpc/pseries/vas: Integrate API with open/close windows

Excerpts from Haren Myneni's message of April 18, 2021 7:08 am:

Quoted 156 lines

This patch adds VAS window allocatioa/close with the corresponding HCALLs. Also changes to integrate with the existing user space VAS API and provide register/unregister functions to NX pseries driver.

The driver register function is used to create the user space interface (/dev/crypto/nx-gzip) and unregister to remove this entry.

The user space process opens this device node and makes an ioctl to allocate VAS window. The close interface is used to deallocate window.

---
 arch/powerpc/include/asm/vas.h          |   5 +
 arch/powerpc/platforms/book3s/Kconfig   |   2 +-
 arch/powerpc/platforms/pseries/Makefile |   1 +
 arch/powerpc/platforms/pseries/vas.c    | 212 ++++++++++++++++++++++++
 4 files changed, 219 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/vas.h +5 -0
index d15784506a54..aa1974aba27e 100644
@@ -270,6 +270,11 @@ struct vas_all_capabs {
 	u64     feat_type;
 };
 
+int plpar_vas_query_capabilities(const u64 hcall, u8 query_type,
+				 u64 result);
+int vas_register_api_pseries(struct module *mod,
+			     enum vas_cop_type cop_type, const char *name);
+void vas_unregister_api_pseries(void);
 #endif
 
 /*
arch/powerpc/platforms/book3s/Kconfig +1 -1
index 51e14db83a79..bed21449e8e5 100644
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 config PPC_VAS
 	bool "IBM Virtual Accelerator Switchboard (VAS)"
-	depends on PPC_POWERNV && PPC_64K_PAGES
+	depends on (PPC_POWERNV || PPC_PSERIES) && PPC_64K_PAGES
 	default y
 	help
 	  This enables support for IBM Virtual Accelerator Switchboard (VAS).
arch/powerpc/platforms/pseries/Makefile +1 -0
index c8a2b0b05ac0..4cda0ef87be0 100644
@@ -30,3 +30,4 @@ obj-$(CONFIG_PPC_SVM)		+= svm.o
 obj-$(CONFIG_FA_DUMP)		+= rtas-fadump.o
 
 obj-$(CONFIG_SUSPEND)		+= suspend.o
+obj-$(CONFIG_PPC_VAS)		+= vas.o
arch/powerpc/platforms/pseries/vas.c +90 -0
index 35946fb02995..0ade0d6d728f 100644
@@ -222,6 +222,218 @@ int plpar_vas_query_capabilities(const u64 hcall, u8 query_type,
 		return -EIO;
 	}
 }
+EXPORT_SYMBOL_GPL(plpar_vas_query_capabilities);
+
+/*
+ * Allocate window and setup IRQ mapping.
+ */
+static int allocate_setup_window(struct vas_window *txwin,
+				 u64 *domain, u8 wintype)
+{
+	int rc;
+
+	rc = plpar_vas_allocate_window(txwin, domain, wintype, DEF_WIN_CREDS);
+	if (rc)
+		return rc;
+
+	txwin->wcreds_max = DEF_WIN_CREDS;
+
+	return 0;
+}
+
+static struct vas_window *vas_allocate_window(struct vas_tx_win_open_attr *uattr,
+					      enum vas_cop_type cop_type)
+{
+	long domain[PLPAR_HCALL9_BUFSIZE] = {VAS_DEFAULT_DOMAIN_ID};
+	struct vas_ct_capabs *ct_capab;
+	struct vas_capabs *capabs;
+	struct vas_window *txwin;
+	int rc;
+
+	txwin = kzalloc(sizeof(*txwin), GFP_KERNEL);
+	if (!txwin)
+		return ERR_PTR(-ENOMEM);
+
+	/*
+	 * A VAS window can have many credits which means that many
+	 * requests can be issued simultaneously. But phyp restricts
+	 * one credit per window.
+	 * phyp introduces 2 different types of credits:
+	 * Default credit type (Uses normal priority FIFO):
+	 *	A limited number of credits are assigned to partitions
+	 *	based on processor entitlement. But these credits may be
+	 *	over-committed on a system depends on whether the CPUs
+	 *	are in shared or dedicated modes - that is, more requests
+	 *	may be issued across the system than NX can service at
+	 *	once which can result in paste command failure (RMA_busy).
+	 *	Then the process has to resend requests or fall-back to
+	 *	SW compression.
+	 * Quality of Service (QoS) credit type (Uses high priority FIFO):
+	 *	To avoid NX HW contention, the system admins can assign
+	 *	QoS credits for each LPAR so that this partition is
+	 *	guaranteed access to NX resources. These credits are
+	 *	assigned to partitions via the HMC.
+	 *	Refer PAPR for more information.
+	 *
+	 * Allocate window with QoS credits if user requested. Otherwise
+	 * default credits are used.
+	 */
+	if (uattr->flags & VAS_WIN_QOS_CREDITS)
+		capabs = &vcapabs[VAS_GZIP_QOS_FEAT_TYPE];
+	else
+		capabs = &vcapabs[VAS_GZIP_DEF_FEAT_TYPE];
+
+	ct_capab = &capabs->capab;
+
+	if (atomic_inc_return(&ct_capab->used_lpar_creds) >
+			atomic_read(&ct_capab->target_lpar_creds)) {
+		pr_err("Credits are not available to allocate window\n");
+		rc = -EINVAL;
+		goto out;
+	}
+
+	/*
+	 * The user space is requesting to allocate a window on a VAS
+	 * instance (or chip) where the process is executing.
+	 * On powerVM, domain values are passed to pHyp to select chip /
+	 * VAS instance. Useful if the process is affinity to NUMA node.
+	 * pHyp selects VAS instance if VAS_DEFAULT_DOMAIN_ID (-1) is
+	 * passed for domain values.
+	 */
+	if (uattr->vas_id == -1) {
+		/*
+		 * To allocate VAS window, pass same domain values returned
+		 * from this HCALL.
+		 */
+		rc = plpar_hcall9(H_HOME_NODE_ASSOCIATIVITY, domain,
+				  VPHN_FLAG_VCPU, smp_processor_id());
+		if (rc != H_SUCCESS) {
+			pr_err("HCALL(%x): failed with ret(%d)\n",
+			       H_HOME_NODE_ASSOCIATIVITY, rc);
+			goto out;
+		}

arch/powerpc/platforms/pseries/vas.c int plpar_vas_query_capabilities(const u64 hcall, u8 query_type, · line 324

Quoted 10 lines
arch/powerpc/platforms/pseries/vas.c +10 -0
+	}
+
+	/*
+	 * Allocate / Deallocate window HCALLs and setup / free IRQs
+	 * have to be protected with mutex. Otherwise, since IRQ is freed
+	 * after deallocate HCALL, may see the case where window ID and
+	 * fault interrupt could be reused before free IRQ (for the old
+	 * window) in kernel. It can result in setup IRQ fail for the
+	 * new window.
+	 */

It's a bit difficult to understand that comment.

The window deallocate is protected with the mutex, then the mutex gets dropped. Some time later presumably the IRQ gets freed.

What prevents the window ID from being reused in between?

Thanks,
Nick

Quoted 120 lines
arch/powerpc/platforms/pseries/vas.c +112 -0
+	mutex_lock(&vas_pseries_mutex);
+	rc = allocate_setup_window(txwin, (u64 *)&domain[0],
+				   ct_capab->win_type);
+	mutex_unlock(&vas_pseries_mutex);
+	if (rc)
+		goto out;
+
+	/*
+	 * Modify window and it is ready to use.
+	 */
+	rc = plpar_vas_modify_window(txwin);
+	if (!rc)
+		rc = vas_reference_task(&txwin->task);
+	if (rc)
+		goto out_free;
+
+	txwin->lpar.win_type = ct_capab->win_type;
+	mutex_lock(&vas_pseries_mutex);
+	list_add(&txwin->lpar.win_list, &capabs->list);
+	mutex_unlock(&vas_pseries_mutex);
+
+	return txwin;
+
+out_free:
+	plpar_vas_deallocate_window(txwin->winid);
+out:
+	atomic_dec(&ct_capab->used_lpar_creds);
+	kfree(txwin);
+	return ERR_PTR(rc);
+}
+
+static u64 vas_paste_address(void *addr)
+{
+	struct vas_window *win = addr;
+
+	return win->lpar.win_addr;
+}
+
+static int deallocate_free_window(struct vas_window *win)
+{
+	int rc = 0;
+
+	rc = plpar_vas_deallocate_window(win->winid);
+	if (!rc)
+		kfree(win->lpar.name);
+
+	return rc;
+}
+
+static int vas_deallocate_window(void *addr)
+{
+	struct vas_window *win = (struct vas_window *)addr;
+	struct vas_ct_capabs *capabs;
+	int rc = 0;
+
+	if (!win)
+		return -EINVAL;
+
+	/* Should not happen */
+	if (win->lpar.win_type >= VAS_MAX_FEAT_TYPE) {
+		pr_err("Window (%u): Invalid window type %u\n",
+				win->winid, win->lpar.win_type);
+		return -EINVAL;
+	}
+
+	capabs = &vcapabs[win->lpar.win_type].capab;
+	mutex_lock(&vas_pseries_mutex);
+	rc = deallocate_free_window(win);
+	if (rc) {
+		mutex_unlock(&vas_pseries_mutex);
+		return rc;
+	}
+
+	list_del(&win->lpar.win_list);
+	atomic_dec(&capabs->used_lpar_creds);
+	mutex_unlock(&vas_pseries_mutex);
+
+	vas_drop_reference_task(&win->task);
+
+	kfree(win);
+	return 0;
+}
+
+static struct vas_user_win_ops vops_pseries = {
+	.open_win	= vas_allocate_window,	/* Open and configure window */
+	.paste_addr	= vas_paste_address,	/* To do copy/paste */
+	.close_win	= vas_deallocate_window, /* Close window */
+};
+
+/*
+ * Supporting only nx-gzip coprocessor type now, but this API code
+ * extended to other coprocessor types later.
+ */
+int vas_register_api_pseries(struct module *mod, enum vas_cop_type cop_type,
+			     const char *name)
+{
+	int rc;
+
+	if (!copypaste_feat)
+		return -ENOTSUPP;
+
+	rc = vas_register_coproc_api(mod, cop_type, name, &vops_pseries);
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(vas_register_api_pseries);
+
+void vas_unregister_api_pseries(void)
+{
+	vas_unregister_coproc_api();
+}
+EXPORT_SYMBOL_GPL(vas_unregister_api_pseries);
 
 /*
  * Get the specific capabilities based on the feature type.
Signature · 1 line
2.18.2
#29 lore raw

Re: [V3 PATCH 10/16] powerpc/pseries/vas: Integrate API with open/close windows

In reply to Haren Myneni

Excerpts from Haren Myneni's message of April 18, 2021 7:08 am:

arch/powerpc/platforms/pseries/vas.c int plpar_vas_query_capabilities(const u64 hcall, u8 query_type, · line 369

Quoted 7 lines
arch/powerpc/platforms/pseries/vas.c +7 -0
+static int deallocate_free_window(struct vas_window *win)
+{
+	int rc = 0;
+
+	rc = plpar_vas_deallocate_window(win->winid);
+	if (!rc)
+		kfree(win->lpar.name);

Oh, did this kfree sneak in here? The allocation appears in patch 11 I think.

Thanks,
Nick

#30 lore raw

PATCH v3 11/16 powerpc/pseries/vas: Setup IRQ and fault handling

In reply to Haren Myneni

When NX sees a fault on the user space buffer, generates a fault interrupt and pHyp forwards that interrupt to OS. Then the kernel makes H_GET_NX_FAULT HCALL to retrieve the fault CRB information.

This patch adds changes to setup IRQ per each window and handles fault by updating CSB.

---
 arch/powerpc/platforms/pseries/vas.c | 111 ++++++++++++++++++++++++++-
 1 file changed, 110 insertions(+), 1 deletion(-)
arch/powerpc/platforms/pseries/vas.c +110 -1
index 0ade0d6d728f..2106eca0862a 100644
@@ -224,6 +224,62 @@ int plpar_vas_query_capabilities(const u64 hcall, u8 query_type,
 }
 EXPORT_SYMBOL_GPL(plpar_vas_query_capabilities);
 
+/*
+ * HCALL to get fault CRB from pHyp.
+ */
+static int plpar_get_nx_fault(u32 winid, u64 buffer)
+{
+	int64_t rc;
+
+	rc = plpar_hcall_norets(H_GET_NX_FAULT, winid, buffer);
+
+	switch (rc) {
+	case H_SUCCESS:
+		return 0;
+	case H_PARAMETER:
+		pr_err("HCALL(%x): Invalid window ID %u\n", H_GET_NX_FAULT,
+		       winid);
+		return -EINVAL;
+	case H_STATE:
+		pr_err("HCALL(%x): No outstanding faults for window ID %u\n",
+		       H_GET_NX_FAULT, winid);
+		return -EINVAL;
+	case H_PRIVILEGE:
+		pr_err("HCALL(%x): Window(%u): Invalid fault buffer 0x%llx\n",
+		       H_GET_NX_FAULT, winid, buffer);
+		return -EACCES;
+	default:
+		pr_err("HCALL(%x): Unexpected error %lld for window(%u)\n",
+		       H_GET_NX_FAULT, rc, winid);
+		return -EIO;
+	}
+}
+
+/*
+ * Handle the fault interrupt.
+ * When the fault interrupt is received for each window, query pHyp to get
+ * the fault CRB on the specific fault. Then process the CRB by updating
+ * CSB or send signal if the user space CSB is invalid.
+ * Note: pHyp forwards an interrupt for each fault request. So one fault
+ *	CRB to process for each H_GET_NX_FAULT HCALL.
+ */
+irqreturn_t pseries_vas_fault_thread_fn(int irq, void *data)
+{
+	struct vas_window *txwin = data;
+	struct coprocessor_request_block crb;
+	struct vas_win_task *tsk;
+	int rc;
+
+	rc = plpar_get_nx_fault(txwin->winid, (u64)virt_to_phys(&crb));
+	if (!rc) {
+		tsk = &txwin->task;
+		vas_dump_crb(&crb);
+		vas_update_csb(&crb, tsk);
+	}
+
+	return IRQ_HANDLED;
+}
+
 /*
  * Allocate window and setup IRQ mapping.
  */
@@ -235,10 +291,51 @@ static int allocate_setup_window(struct vas_window *txwin,
 	rc = plpar_vas_allocate_window(txwin, domain, wintype, DEF_WIN_CREDS);
 	if (rc)
 		return rc;
+	/*
+	 * On powerVM, pHyp setup and forwards the fault interrupt per
+	 * window. So the IRQ setup and fault handling will be done for
+	 * each open window separately.
+	 */
+	txwin->lpar.fault_virq = irq_create_mapping(NULL,
+						    txwin->lpar.fault_irq);
+	if (!txwin->lpar.fault_virq) {
+		pr_err("Failed irq mapping %d\n", txwin->lpar.fault_irq);
+		rc = -EINVAL;
+		goto out_win;
+	}
+
+	txwin->lpar.name = kasprintf(GFP_KERNEL, "vas-win-%d", txwin->winid);
+	if (!txwin->lpar.name) {
+		rc = -ENOMEM;
+		goto out_irq;
+	}
+
+	rc = request_threaded_irq(txwin->lpar.fault_virq, NULL,
+				  pseries_vas_fault_thread_fn, IRQF_ONESHOT,
+				  txwin->lpar.name, txwin);
+	if (rc) {
+		pr_err("VAS-Window[%d]: Request IRQ(%u) failed with %d\n",
+		       txwin->winid, txwin->lpar.fault_virq, rc);
+		goto out_free;
+	}
 
 	txwin->wcreds_max = DEF_WIN_CREDS;
 
 	return 0;
+out_free:
+	kfree(txwin->lpar.name);
+out_irq:
+	irq_dispose_mapping(txwin->lpar.fault_virq);
+out_win:
+	plpar_vas_deallocate_window(txwin->winid);
+	return rc;
+}
+
+static inline void free_irq_setup(struct vas_window *txwin)
+{
+	free_irq(txwin->lpar.fault_virq, txwin);
+	irq_dispose_mapping(txwin->lpar.fault_virq);
+	kfree(txwin->lpar.name);
 }
 
 static struct vas_window *vas_allocate_window(struct vas_tx_win_open_attr *uattr,
@@ -346,6 +443,11 @@ static struct vas_window *vas_allocate_window(struct vas_tx_win_open_attr *uattr
 	return txwin;
 
 out_free:
+	/*
+	 * Window is not operational. Free IRQ before closing
+	 * window so that do not have to hold mutex.
+	 */
+	free_irq_setup(txwin);
 	plpar_vas_deallocate_window(txwin->winid);
 out:
 	atomic_dec(&ct_capab->used_lpar_creds);
@@ -364,9 +466,16 @@ static int deallocate_free_window(struct vas_window *win)
 {
 	int rc = 0;
 
+	/*
+	 * Free IRQ after executing H_DEALLOCATE_VAS_WINDOW HCALL
+	 * to close the window. pHyp waits for all requests including
+	 * faults are processed before closing the window - Means all
+	 * credits are returned. In the case of fault request, credit
+	 * is returned after OS issues H_GET_NX_FAULT HCALL.
+	 */
 	rc = plpar_vas_deallocate_window(win->winid);
 	if (!rc)
-		kfree(win->lpar.name);
+		free_irq_setup(win);
 
 	return rc;
 }
Signature · 1 line
2.18.2
#31 lore raw

PATCH v3 12/16 powerpc/pseries/vas: sysfs interface to export capabilities

In reply to Haren Myneni

pHyp provides GZIP default and GZIP QoS capabilities which gives the total number of credits are available in LPAR. This patch creates sysfs entries and exports LPAR credits, the currently used and the available credits for each feature.

/sys/kernel/vas/VasCaps/VDefGzip: (default GZIP capabilities)
avail_lpar_creds /* Available credits to use */ target_lpar_creds /* Total credits available which can be
/* changed with DLPAR operation */
used_lpar_creds /* Used credits */

/sys/kernel/vas/VasCaps/VQosGzip (QoS GZIP capabilities)
avail_lpar_creds
target_lpar_creds
used_lpar_creds

---
 arch/powerpc/platforms/pseries/Makefile    |   2 +-
 arch/powerpc/platforms/pseries/vas-sysfs.c | 173 +++++++++++++++++++++
 arch/powerpc/platforms/pseries/vas.c       |   6 +
 arch/powerpc/platforms/pseries/vas.h       |   2 +
 4 files changed, 182 insertions(+), 1 deletion(-)
 create mode 100644 arch/powerpc/platforms/pseries/vas-sysfs.c
arch/powerpc/platforms/pseries/Makefile +1 -1
index 4cda0ef87be0..e24093bebc0b 100644
@@ -30,4 +30,4 @@ obj-$(CONFIG_PPC_SVM)		+= svm.o
 obj-$(CONFIG_FA_DUMP)		+= rtas-fadump.o
 
 obj-$(CONFIG_SUSPEND)		+= suspend.o
-obj-$(CONFIG_PPC_VAS)		+= vas.o
+obj-$(CONFIG_PPC_VAS)		+= vas.o vas-sysfs.o
arch/powerpc/platforms/pseries/vas-sysfs.c new file +173 -0

173 lines not on this page · raw message

arch/powerpc/platforms/pseries/vas.c +6 -0
index 2106eca0862a..ac06fa09900c 100644
@@ -594,6 +594,10 @@ static int get_vas_capabilities(u8 feat, enum vas_cop_feat_type type,
 		}
 	}
 
+	rc = sysfs_add_vas_capabs(capab);
+	if (rc)
+		return rc;
+
 	copypaste_feat = 1;
 
 	return 0;
@@ -629,6 +633,8 @@ static int __init pseries_vas_init(void)
 	capabs_all.descriptor = be64_to_cpu(capabs_be->descriptor);
 	capabs_all.feat_type = be64_to_cpu(capabs_be->feat_type);
 
+	sysfs_pseries_vas_init(&capabs_all);
+
 	ct_capabs_be = kmalloc(sizeof(*ct_capabs_be), GFP_KERNEL);
 	if (!ct_capabs_be) {
 		rc = -ENOMEM;
arch/powerpc/platforms/pseries/vas.h +2 -0
index 208682fffa57..e64677f03f08 100644
@@ -93,4 +93,6 @@ struct vas_win_lpar_be {
 	__be64	win_util;	/* Number of bytes processed */
 } __packed __aligned(0x1000);
 
+int sysfs_add_vas_capabs(struct vas_ct_capabs *capabs);
+int __init sysfs_pseries_vas_init(struct vas_all_capabs *vas_caps);
 #endif /* _VAS_H */
Signature · 1 line
2.18.2
#32 lore raw

Re: [V3 PATCH 12/16] powerpc/pseries/vas: sysfs interface to export capabilities

Excerpts from Haren Myneni's message of April 18, 2021 7:10 am:

Quoted 11 lines

pHyp provides GZIP default and GZIP QoS capabilities which gives the total number of credits are available in LPAR. This patch creates sysfs entries and exports LPAR credits, the currently used and the available credits for each feature.

/sys/kernel/vas/VasCaps/VDefGzip: (default GZIP capabilities)
avail_lpar_creds /* Available credits to use */ target_lpar_creds /* Total credits available which can be
/* changed with DLPAR operation */
used_lpar_creds /* Used credits */

/sys/kernel/ is not an appropriate directory to put it in. Also camel case is not thought very highly of these days.

And s/capabs/caps/g applies here (and all other patches).

Thanks,
Nick

#33 lore raw

Re: [V3 PATCH 12/16] powerpc/pseries/vas: sysfs interface to export capabilities

Quoted 12 lines · Nicholas Piggin

Excerpts from Haren Myneni's message of April 18, 2021 7:10 am:

Quoted 10 lines

pHyp provides GZIP default and GZIP QoS capabilities which gives the total number of credits are available in LPAR. This patch creates sysfs entries and exports LPAR credits, the currently used and the available credits for each feature.

/sys/kernel/vas/VasCaps/VDefGzip: (default GZIP capabilities)
avail_lpar_creds /* Available credits to use */ target_lpar_creds /* Total credits available which can be
/* changed with DLPAR operation */
used_lpar_creds /* Used credits */

Quoted 3 lines

/sys/kernel/ is not an appropriate directory to put it in. Also camel
case is not thought very highly of these days.

These capabilities are VAS specific ones (powerpc kernel), not for the indicidual coprocessor. Not sure where to add in sysfs.

Quoted 2 lines

And s/capabs/caps/g applies here (and all other patches).

Thanks, will change.

Quoted 3 lines

Thanks,
Nick

#34 lore raw

PATCH v3 13/16 crypto/nx: Rename nx-842-pseries file name to nx-common-pseries

In reply to Haren Myneni

Rename nx-842-pseries.c to nx-common-pseries.c to add code for new GZIP compression type. The actual functionality is not changed in this patch.

---
 drivers/crypto/nx/Makefile                                  | 2 +-
 drivers/crypto/nx/{nx-842-pseries.c => nx-common-pseries.c} | 0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename drivers/crypto/nx/{nx-842-pseries.c => nx-common-pseries.c} (100%)
drivers/crypto/nx/Makefile +1 -1
index bc89a20e5d9d..d00181a26dd6 100644
@@ -14,5 +14,5 @@ nx-crypto-objs := nx.o \
 obj-$(CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES) += nx-compress-pseries.o nx-compress.o
 obj-$(CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV) += nx-compress-powernv.o nx-compress.o
 nx-compress-objs := nx-842.o
-nx-compress-pseries-objs := nx-842-pseries.o
+nx-compress-pseries-objs := nx-common-pseries.o
 nx-compress-powernv-objs := nx-common-powernv.o
drivers/crypto/nx/nx-common-pseries.c renamed from drivers/crypto/nx/nx-842-pseries.c +0 -0
similarity index 100%
rename from drivers/crypto/nx/nx-842-pseries.c
rename to drivers/crypto/nx/nx-common-pseries.c
Signature · 1 line
2.18.2
maintainer Acked-by on v3 13/16
#35 lore raw

Re: [V3 PATCH 13/16] crypto/nx: Rename nx-842-pseries file name to nx-common-pseries

drivers/crypto/nx/nx-common-pseries.c

Quoted 27 lines · Haren Myneni

Rename nx-842-pseries.c to nx-common-pseries.c to add code for new GZIP compression type. The actual functionality is not changed in this patch.

---
 drivers/crypto/nx/Makefile                                  | 2 +-
 drivers/crypto/nx/{nx-842-pseries.c => nx-common-pseries.c} | 0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename drivers/crypto/nx/{nx-842-pseries.c => nx-common-pseries.c} (100%)
drivers/crypto/nx/Makefile +1 -1
index bc89a20e5d9d..d00181a26dd6 100644
@@ -14,5 +14,5 @@ nx-crypto-objs := nx.o \
 obj-$(CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES) += nx-compress-pseries.o nx-compress.o
 obj-$(CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV) += nx-compress-powernv.o nx-compress.o
 nx-compress-objs := nx-842.o
-nx-compress-pseries-objs := nx-842-pseries.o
+nx-compress-pseries-objs := nx-common-pseries.o
 nx-compress-powernv-objs := nx-common-powernv.o
drivers/crypto/nx/nx-common-pseries.c renamed from drivers/crypto/nx/nx-842-pseries.c +0 -0
similarity index 100%
rename from drivers/crypto/nx/nx-842-pseries.c
rename to drivers/crypto/nx/nx-common-pseries.c
Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#36 lore raw

PATCH v3 14/16 crypto/nx: Register and unregister VAS interface

In reply to Haren Myneni

Changes to create /dev/crypto/nx-gzip interface with VAS register and to remove this interface with VAS unregister.

---
 drivers/crypto/nx/Kconfig             | 1 +
 drivers/crypto/nx/nx-common-pseries.c | 9 +++++++++
 2 files changed, 10 insertions(+)
drivers/crypto/nx/Kconfig +1 -0
index 23e3d0160e67..2a35e0e785bd 100644
@@ -29,6 +29,7 @@ if CRYPTO_DEV_NX_COMPRESS
 config CRYPTO_DEV_NX_COMPRESS_PSERIES
 	tristate "Compression acceleration support on pSeries platform"
 	depends on PPC_PSERIES && IBMVIO
+	depends on PPC_VAS
 	default y
 	help
 	  Support for PowerPC Nest (NX) compression acceleration. This
drivers/crypto/nx/nx-common-pseries.c +9 -0
index cc8dd3072b8b..9a40fca8a9e6 100644
@@ -9,6 +9,7 @@
  */
 
 #include <asm/vio.h>
+#include <asm/vas.h>
 
 #include "nx-842.h"
 #include "nx_csbcpb.h" /* struct nx_csbcpb */
@@ -1101,6 +1102,12 @@ static int __init nx842_pseries_init(void)
 		return ret;
 	}
 
+	ret = vas_register_api_pseries(THIS_MODULE, VAS_COP_TYPE_GZIP,
+				       "nx-gzip");
+
+	if (ret)
+		pr_err("NX-GZIP is not supported. Returned=%d\n", ret);
+
 	return 0;
 }
 
@@ -1111,6 +1118,8 @@ static void __exit nx842_pseries_exit(void)
 	struct nx842_devdata *old_devdata;
 	unsigned long flags;
 
+	vas_unregister_api_pseries();
+
 	crypto_unregister_alg(&nx842_pseries_alg);
 
 	spin_lock_irqsave(&devdata_mutex, flags);
Signature · 1 line
2.18.2
maintainer Acked-by on v3 14/16
#37 lore raw

Re: [V3 PATCH 14/16] crypto/nx: Register and unregister VAS interface

Quoted 9 lines · Haren Myneni

Changes to create /dev/crypto/nx-gzip interface with VAS register and to remove this interface with VAS unregister.

---
 drivers/crypto/nx/Kconfig             | 1 +
 drivers/crypto/nx/nx-common-pseries.c | 9 +++++++++
 2 files changed, 10 insertions(+)
Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#38 lore raw

PATCH v3 15/16 crypto/nx: Get NX capabilities for GZIP coprocessor type

In reply to Haren Myneni

phyp provides NX capabilities which gives recommended minimum compression / decompression length and maximum request buffer size in bytes.

Changes to get NX overall capabilities which points to the specific features phyp supports. Then retrieve NXGZIP specific capabilities.

---
 drivers/crypto/nx/nx-common-pseries.c | 83 +++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
drivers/crypto/nx/nx-common-pseries.c +83 -0
index 9a40fca8a9e6..49224870d05e 100644
@@ -9,6 +9,7 @@
  */
 
 #include <asm/vio.h>
+#include <asm/hvcall.h>
 #include <asm/vas.h>
 
 #include "nx-842.h"
@@ -20,6 +21,24 @@ MODULE_DESCRIPTION("842 H/W Compression driver for IBM Power processors");
 MODULE_ALIAS_CRYPTO("842");
 MODULE_ALIAS_CRYPTO("842-nx");
 
+struct nx_ct_capabs_be {
+	__be64	descriptor;
+	__be64	req_max_processed_len;	/* Max bytes in one GZIP request */
+	__be64	min_compress_len;	/* Min compression size in bytes */
+	__be64	min_decompress_len;	/* Min decompression size in bytes */
+} __packed __aligned(0x1000);
+
+struct nx_ct_capabs {
+	char	name[VAS_DESCR_LEN + 1];
+	u64	descriptor;
+	u64	req_max_processed_len;	/* Max bytes in one GZIP request */
+	u64	min_compress_len;	/* Min compression in bytes */
+	u64	min_decompress_len;	/* Min decompression in bytes */
+};
+
+u64 capab_feat = 0;
+struct nx_ct_capabs nx_ct_capab;
+
 static struct nx842_constraints nx842_pseries_constraints = {
 	.alignment =	DDE_BUFFER_ALIGN,
 	.multiple =	DDE_BUFFER_LAST_MULT,
@@ -1066,6 +1085,66 @@ static void nx842_remove(struct vio_dev *viodev)
 	kfree(old_devdata);
 }
 
+/*
+ * Get NX capabilities from pHyp.
+ * Only NXGZIP capabilities are available right now and these values
+ * are available through sysfs.
+ */
+static void __init nxct_get_capabilities(void)
+{
+	struct vas_all_capabs_be *capabs_be;
+	struct nx_ct_capabs_be *nxc_be;
+	int rc;
+
+	capabs_be = kmalloc(sizeof(*capabs_be), GFP_KERNEL);
+	if (!capabs_be)
+		return;
+	/*
+	 * Get NX overall capabilities with feature type=0
+	 */
+	rc = plpar_vas_query_capabilities(H_QUERY_NX_CAPABILITIES, 0,
+					  (u64)virt_to_phys(capabs_be));
+	if (rc)
+		goto out;
+
+	capab_feat = be64_to_cpu(capabs_be->feat_type);
+	/*
+	 * NX-GZIP feature available
+	 */
+	if (capab_feat & VAS_NX_GZIP_FEAT_BIT) {
+		nxc_be = kmalloc(sizeof(*nxc_be), GFP_KERNEL);
+		if (!nxc_be)
+			goto out;
+		/*
+		 * Get capabilities for NX-GZIP feature
+		 */
+		rc = plpar_vas_query_capabilities(H_QUERY_NX_CAPABILITIES,
+						  VAS_NX_GZIP_FEAT,
+						  (u64)virt_to_phys(nxc_be));
+	} else {
+		pr_err("NX-GZIP feature is not available\n");
+		rc = -EINVAL;
+	}
+
+	if (!rc) {
+		snprintf(nx_ct_capab.name, VAS_DESCR_LEN + 1, "%.8s",
+			 (char *)&nxc_be->descriptor);
+		nx_ct_capab.descriptor = be64_to_cpu(nxc_be->descriptor);
+		nx_ct_capab.req_max_processed_len =
+				be64_to_cpu(nxc_be->req_max_processed_len);
+		nx_ct_capab.min_compress_len =
+				be64_to_cpu(nxc_be->min_compress_len);
+		nx_ct_capab.min_decompress_len =
+				be64_to_cpu(nxc_be->min_decompress_len);
+	} else {
+		capab_feat = 0;
+	}
+
+	kfree(nxc_be);
+out:
+	kfree(capabs_be);
+}
+
 static const struct vio_device_id nx842_vio_driver_ids[] = {
 	{"ibm,compression-v1", "ibm,compression"},
 	{"", ""},
@@ -1093,6 +1172,10 @@ static int __init nx842_pseries_init(void)
 		return -ENOMEM;
 
 	RCU_INIT_POINTER(devdata, new_devdata);
+	/*
+	 * Get NX capabilities from pHyp which is used for NX-GZIP.
+	 */
+	nxct_get_capabilities();
 
 	ret = vio_register_driver(&nx842_vio_driver);
 	if (ret) {
Signature · 1 line
2.18.2
maintainer Acked-by on v3 15/16
#39 lore raw

Re: [V3 PATCH 15/16] crypto/nx: Get NX capabilities for GZIP coprocessor type

Quoted 5 lines · Haren Myneni

phyp provides NX capabilities which gives recommended minimum compression / decompression length and maximum request buffer size in bytes.

Quoted 7 lines

Changes to get NX overall capabilities which points to the specific features phyp supports. Then retrieve NXGZIP specific capabilities.

---
 drivers/crypto/nx/nx-common-pseries.c | 83 +++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#40 lore raw

Re: [V3 PATCH 15/16] crypto/nx: Get NX capabilities for GZIP coprocessor type

In reply to Haren Myneni

Excerpts from Haren Myneni's message of April 18, 2021 7:12 am:

Quoted 26 lines

phyp provides NX capabilities which gives recommended minimum compression / decompression length and maximum request buffer size in bytes.

Changes to get NX overall capabilities which points to the specific features phyp supports. Then retrieve NXGZIP specific capabilities.

---
 drivers/crypto/nx/nx-common-pseries.c | 83 +++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
drivers/crypto/nx/nx-common-pseries.c +1 -0
index 9a40fca8a9e6..49224870d05e 100644
@@ -9,6 +9,7 @@
  */
 
 #include <asm/vio.h>
+#include <asm/hvcall.h>
 #include <asm/vas.h>
 
 #include "nx-842.h"

drivers/crypto/nx/nx-common-pseries.c MODULE_DESCRIPTION("842 H/W Compression driver for IBM Power processors"); · line 24

Quoted 4 lines
drivers/crypto/nx/nx-common-pseries.c +1 -0
@@ -20,6 +21,24 @@ MODULE_DESCRIPTION("842 H/W Compression driver for IBM Power processors");
 MODULE_ALIAS_CRYPTO("842");
 MODULE_ALIAS_CRYPTO("842-nx");
 
+struct nx_ct_capabs_be {

What does "ct" mean? I've seen it in a few other places too.

Quoted 5 lines
drivers/crypto/nx/nx-common-pseries.c +5 -0
+	__be64	descriptor;
+	__be64	req_max_processed_len;	/* Max bytes in one GZIP request */
+	__be64	min_compress_len;	/* Min compression size in bytes */
+	__be64	min_decompress_len;	/* Min decompression size in bytes */
+} __packed __aligned(0x1000);

drivers/crypto/nx/nx-common-pseries.c MODULE_DESCRIPTION("842 H/W Compression driver for IBM Power processors"); · line 39

Quoted 10 lines
drivers/crypto/nx/nx-common-pseries.c +10 -0
+
+struct nx_ct_capabs {
+	char	name[VAS_DESCR_LEN + 1];
+	u64	descriptor;
+	u64	req_max_processed_len;	/* Max bytes in one GZIP request */
+	u64	min_compress_len;	/* Min compression in bytes */
+	u64	min_decompress_len;	/* Min decompression in bytes */
+};
+
+u64 capab_feat = 0;

Why is this here and not a local variable?

drivers/crypto/nx/nx-common-pseries.c MODULE_DESCRIPTION("842 H/W Compression driver for IBM Power processors"); · line 40

Quoted 1 line
drivers/crypto/nx/nx-common-pseries.c +1 -0
+struct nx_ct_capabs nx_ct_capab;

It's okay and generally better to use the same name as the struct name in this situation, i.e.,

"struct nx_ct_capabs nx_ct_capabs"

(modulo static / caps / etc)

Thanks,
Nick

#41 lore raw

Re: [V3 PATCH 15/16] crypto/nx: Get NX capabilities for GZIP coprocessor type

Quoted 32 lines · Nicholas Piggin

Excerpts from Haren Myneni's message of April 18, 2021 7:12 am:

Quoted 31 lines

phyp provides NX capabilities which gives recommended minimum compression / decompression length and maximum request buffer size in bytes.

Changes to get NX overall capabilities which points to the specific features phyp supports. Then retrieve NXGZIP specific capabilities.

---
 drivers/crypto/nx/nx-common-pseries.c | 83
+++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
drivers/crypto/nx/nx-common-pseries.c +1 -0
index 9a40fca8a9e6..49224870d05e 100644
@@ -9,6 +9,7 @@
  */
 
 #include <asm/vio.h>
+#include <asm/hvcall.h>
 #include <asm/vas.h>
 
 #include "nx-842.h"
@@ -20,6 +21,24 @@ MODULE_DESCRIPTION("842 H/W Compression driver
 for IBM Power processors");
 MODULE_ALIAS_CRYPTO("842");
 MODULE_ALIAS_CRYPTO("842-nx");
 
Quoted 3 lines

drivers/crypto/nx/nx-common-pseries.c MODULE_DESCRIPTION("842 H/W Compression driver for IBM Power processors"); · line 24

Quoted 1 line
drivers/crypto/nx/nx-common-pseries.c +1 -0
+struct nx_ct_capabs_be {

What does "ct" mean? I've seen it in a few other places too.

ct means coprocessor type such as 842 and GZIP. phyp provides only GZIP capabilities right now. But this struct may be extended to other types in future.

Quoted 13 lines
Quoted 13 lines
drivers/crypto/nx/nx-common-pseries.c +2 -0
+	__be64	descriptor;
+	__be64	req_max_processed_len;	/* Max bytes in one GZIP

request */

drivers/crypto/nx/nx-common-pseries.c +1 -0
+	__be64	min_compress_len;	/* Min compression size in

bytes */

drivers/crypto/nx/nx-common-pseries.c +1 -0
+	__be64	min_decompress_len;	/* Min decompression size

in bytes */

drivers/crypto/nx/nx-common-pseries.c +5 -0
+} __packed __aligned(0x1000);
+
+struct nx_ct_capabs {
+	char	name[VAS_DESCR_LEN + 1];
+	u64	descriptor;
Quoted 8 lines

drivers/crypto/nx/nx-common-pseries.c MODULE_DESCRIPTION("842 H/W Compression driver for IBM Power processors"); · line 39

Quoted 6 lines
drivers/crypto/nx/nx-common-pseries.c +6 -0
+	u64	req_max_processed_len;	/* Max bytes in one GZIP request */
+	u64	min_compress_len;	/* Min compression in bytes */
+	u64	min_decompress_len;	/* Min decompression in bytes */
+};
+
+u64 capab_feat = 0;

Why is this here and not a local variable?

capab_feat is used to add / delete sysfs entries.

Quoted 10 lines
Quoted 2 lines

+struct nx_ct_capabs nx_ct_capab;

It's okay and generally better to use the same name as the struct name
in this situation, i.e.,

"struct nx_ct_capabs nx_ct_capabs"

(modulo static / caps / etc)

Sure, will change.

Quoted 3 lines

Thanks,
Nick

#42 lore raw

PATCH v3 16/16 crypto/nx: Add sysfs interface to export NX capabilities

In reply to Haren Myneni

Changes to export the following NXGZIP capabilities through sysfs:

/sys/devices/vio/ibm,compression-v1/NxGzCaps:
min_compress_len /*Recommended minimum compress length in bytes*/ min_decompress_len /*Recommended minimum decompress length in bytes*/ req_max_processed_len /* Maximum number of bytes processed in one
request */

---
 drivers/crypto/nx/nx-common-pseries.c | 43 +++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
drivers/crypto/nx/nx-common-pseries.c +43 -0
index 49224870d05e..cc258d2c6475 100644
@@ -962,6 +962,36 @@ static struct attribute_group nx842_attribute_group = {
 	.attrs = nx842_sysfs_entries,
 };
 
+#define	nxct_capab_read(_name)						\
+static ssize_t nxct_##_name##_show(struct device *dev,			\
+			struct device_attribute *attr, char *buf)	\
+{									\
+	return sprintf(buf, "%lld\n", nx_ct_capab._name);		\
+}
+
+#define NXCT_ATTR_RO(_name)						\
+	nxct_capab_read(_name);						\
+	static struct device_attribute dev_attr_##_name = __ATTR(_name,	\
+						0444,			\
+						nxct_##_name##_show,	\
+						NULL);
+
+NXCT_ATTR_RO(req_max_processed_len);
+NXCT_ATTR_RO(min_compress_len);
+NXCT_ATTR_RO(min_decompress_len);
+
+static struct attribute *nxct_capab_sysfs_entries[] = {
+	&dev_attr_req_max_processed_len.attr,
+	&dev_attr_min_compress_len.attr,
+	&dev_attr_min_decompress_len.attr,
+	NULL,
+};
+
+static struct attribute_group nxct_capab_attr_group = {
+	.name	=	nx_ct_capab.name,
+	.attrs	=	nxct_capab_sysfs_entries,
+};
+
 static struct nx842_driver nx842_pseries_driver = {
 	.name =		KBUILD_MODNAME,
 	.owner =	THIS_MODULE,
@@ -1051,6 +1081,16 @@ static int nx842_probe(struct vio_dev *viodev,
 		goto error;
 	}
 
+	if (capab_feat) {
+		if (sysfs_create_group(&viodev->dev.kobj,
+					&nxct_capab_attr_group)) {
+			dev_err(&viodev->dev,
+				"Could not create sysfs NX capability entries\n");
+			ret = -1;
+			goto error;
+		}
+	}
+
 	return 0;
 
 error_unlock:
@@ -1070,6 +1110,9 @@ static void nx842_remove(struct vio_dev *viodev)
 	pr_info("Removing IBM Power 842 compression device\n");
 	sysfs_remove_group(&viodev->dev.kobj, &nx842_attribute_group);
 
+	if (capab_feat)
+		sysfs_remove_group(&viodev->dev.kobj, &nxct_capab_attr_group);
+
 	crypto_unregister_alg(&nx842_pseries_alg);
 
 	spin_lock_irqsave(&devdata_mutex, flags);
Signature · 1 line
2.18.2
maintainer Acked-by on v3 16/16
#43 lore raw

Re: [V3 PATCH 16/16] crypto/nx: Add sysfs interface to export NX capabilities

Quoted 9 lines · Haren Myneni

Changes to export the following NXGZIP capabilities through sysfs:

/sys/devices/vio/ibm,compression-v1/NxGzCaps:
min_compress_len /*Recommended minimum compress length in bytes*/ min_decompress_len /*Recommended minimum decompress length in bytes*/ req_max_processed_len /* Maximum number of bytes processed in one
request */

Quoted 4 lines
---
 drivers/crypto/nx/nx-common-pseries.c | 43 +++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt