FF-A VM to VM support
v4 cover FF-A VM to VM support
This patch serie adds support to use FF-A between VM for communications using indirect messages.
It adds a Kconfig parameter to enable this feature and marks it as experimental as for now there is no system to restrict communication rights between VM using this system.
It also adds support to use VM to VM communications using FF-A even if there is no firmware support for FF-A. As this feature enables FF-A support in all cases, we also introduce a new command line parameter to allow the user to force which tee is to be used between FF-A and Optee to have a solution to enable optee support if FF-A VM to VM is enabled.
Changes since v3:
- reintroduce firmare v1.0 support in partinfo
- fix a possible TOC/TOU issue in indirect message handling
- typos and small fixes
Changes since v2:
- Rework partition_info_get implementation
- Taint Xen and display a message when VM to VM is enabled
- Various fixes explained in each patch
Changes since v1 (rfc):
- add a tee command line parameter
- use IS_ENABLED instead of ifdef when possible
- rebase on latest staging
Bertrand Marquis (5):
xen/arm: Create tee command line parameter xen/arm: ffa: Introduce VM to VM support xen/arm: ffa: Add buffer full notification support xen/arm: ffa: Add indirect message between VM xen/arm: ffa: Enable VM to VM without firmware
docs/misc/xen-command-line.pandoc | 14 ++ xen/arch/arm/include/asm/tee/tee.h | 4 + xen/arch/arm/tee/Kconfig | 11 ++ xen/arch/arm/tee/ffa.c | 24 ++- xen/arch/arm/tee/ffa_msg.c | 110 ++++++++++-- xen/arch/arm/tee/ffa_notif.c | 140 ++++++++------- xen/arch/arm/tee/ffa_partinfo.c | 274 +++++++++++++++++++---------- xen/arch/arm/tee/ffa_private.h | 35 +++- xen/arch/arm/tee/tee.c | 31 ++++ 9 files changed, 474 insertions(+), 169 deletions(-)
Signature · 1 line
2.47.1
PATCH v4 1/5 xen/arm: Create tee command line parameter
Add a new command line parameter "tee=" to be used to explicitly select what tee mediator is to be used by Xen and fail if it does not exist or the probe function for it failed.
Without specifying which tee is to be used, Xen will use the first one for which the probe function succeeds which depends on the order of the mediator list which depends on the compiler.
Using the command line argument, it is now possible to explicit request a specific TEE mediator and panic on boot if it is not available.
---
Changes in v4:
- None
Changes in v3:
- Properly classify tee as arm specific (Jan) Changes in v2:
- Patch introduced to add a command line selection of the TEE
--- docs/misc/xen-command-line.pandoc | 14 ++++++++++++++ xen/arch/arm/include/asm/tee/tee.h | 4 ++++ xen/arch/arm/tee/tee.c | 31 ++++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+)
docs/misc/xen-command-line.pandoc +14 -0
@@ -2651,6 +2651,20 @@ Specify the per-cpu trace buffer size in pages. Flag to enable TSC deadline as the APIC timer mode. +### tee (arm) +> `= <string>` + +Specify the TEE mediator to be probed and use. + +The default behaviour is to probe all supported TEEs supported by Xen and use +the first one successfully probed. When this parameter is passed, Xen will +probe only the TEE mediator passed as argument and boot will fail if this +mediator is not properly probed or if the requested TEE is not supported by +Xen. + +This parameter can be set to `optee` of `ffa` if the corresponding mediators +are compiled in. + ### tevt_mask > `= <integer>`
xen/arch/arm/include/asm/tee/tee.h +4 -0
@@ -55,6 +55,9 @@ struct tee_mediator_desc {/* Printable name of the TEE. */ const char *name; +/* Command line name of the TEE (to be used with tee= cmdline option) */ +const char *cmdline_name; +/* Mediator callbacks as described above. */ const struct tee_mediator_ops *ops;
@@ -77,6 +80,7 @@ void tee_free_domain_ctx(struct domain *d); static const struct tee_mediator_desc __tee_desc_##_name __used \ __section(".teemediator.info") = { \ .name = _namestr, \ + .cmdline_name = #_name, \ .ops = _ops, \ .tee_type = _type \ }
xen/arch/arm/tee/tee.c +31 -0
@@ -19,12 +19,17 @@#include <xen/errno.h> #include <xen/init.h> #include <xen/types.h> +#include <xen/param.h> #include <asm/tee/tee.h> extern const struct tee_mediator_desc _steemediator[], _eteemediator[];static const struct tee_mediator_desc __read_mostly *cur_mediator; +/* Select the TEE mediator using a name on command line. */ +static char __initdata opt_mediator[16 ] ="" ; +string_param ("tee" , opt_mediator); +/* * TODO: Add function to alter Dom0 DTB, so we can properly describe * present TEE.
@@ -81,14 +86,40 @@ static int __init tee_init(void) {const struct tee_mediator_desc *desc; +if (strcmp (opt_mediator,"" ) ) +printk (XENLOG_INFO "TEE Mediator %s selected from command line\n" , + opt_mediator); + +/* +* When a specific TEE is selected using the 'tee=' command line +* argument, we panic if the probe fails or if the requested TEE is not +* supported. +*/ for ( desc = _steemediator; desc != _eteemediator; desc++ ) { +if (strcmp (opt_mediator,"" ) && +strncmp (opt_mediator, desc->cmdline_name,sizeof (opt_mediator)) ) +continue ; +if ( desc->ops->probe () ) {printk (XENLOG_INFO "Using TEE mediator for %s\n" , desc->name); cur_mediator = desc;return 0 ; } +else if (strcmp (opt_mediator,"" ) ) + { +panic ("TEE mediator %s from command line probe failed\n" , + opt_mediator); +return -EFAULT ; + } + } + +if (strcmp (opt_mediator,"" ) ) + { +panic ("TEE Mediator %s from command line not supported\n" , + opt_mediator); +return -EINVAL ; }return 0 ;
Signature · 1 line
2.47.1
Re: [PATCH v4 1/5] xen/arm: Create tee command line parameter
Hi Bertrand,
Quoted 19 lines · Bertrand Marquis
Add a new command line parameter "tee=" to be used to explicitly select what tee mediator is to be used by Xen and fail if it does not exist or the probe function for it failed.
Without specifying which tee is to be used, Xen will use the first one for which the probe function succeeds which depends on the order of the mediator list which depends on the compiler.
Using the command line argument, it is now possible to explicit request a specific TEE mediator and panic on boot if it is not available.Signed-off-by Bertrand Marquis <[email protected]>---
Changes in v4:
- None
Changes in v3:
- Properly classify tee as arm specific (Jan) Changes in v2:
- Patch introduced to add a command line selection of the TEE
Quoted 5 lines
--- docs/misc/xen-command-line.pandoc | 14 ++++++++++++++ xen/arch/arm/include/asm/tee/tee.h | 4 ++++ xen/arch/arm/tee/tee.c | 31 ++++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+)
Cheers,
Jens
Quoted 114 lines
docs/misc/xen-command-line.pandoc +14
-0@@ -2651,6 +2651,20 @@ Specify the per-cpu trace buffer size in pages. Flag to enable TSC deadline as the APIC timer mode. +### tee (arm) +> `= <string>` + +Specify the TEE mediator to be probed and use. + +The default behaviour is to probe all supported TEEs supported by Xen and use +the first one successfully probed. When this parameter is passed, Xen will +probe only the TEE mediator passed as argument and boot will fail if this +mediator is not properly probed or if the requested TEE is not supported by +Xen. + +This parameter can be set to `optee` of `ffa` if the corresponding mediators +are compiled in. + ### tevt_mask > `= <integer>`xen/arch/arm/include/asm/tee/tee.h +4
-0@@ -55,6 +55,9 @@ struct tee_mediator_desc {/* Printable name of the TEE. */ const char *name; +/* Command line name of the TEE (to be used with tee= cmdline option) */ +const char *cmdline_name; +/* Mediator callbacks as described above. */ const struct tee_mediator_ops *ops;@@ -77,6 +80,7 @@ void tee_free_domain_ctx(struct domain *d); static const struct tee_mediator_desc __tee_desc_##_name __used \ __section(".teemediator.info") = { \ .name = _namestr, \ + .cmdline_name = #_name, \ .ops = _ops, \ .tee_type = _type \ }xen/arch/arm/tee/tee.c +31
-1@@ -19,12 +19,17 @@#include <xen/errno.h> #include <xen/init.h> #include <xen/types.h> +#include <xen/param.h> #include <asm/tee/tee.h> extern const struct tee_mediator_desc _steemediator[], _eteemediator[];static const struct tee_mediator_desc __read_mostly *cur_mediator; +/* Select the TEE mediator using a name on command line. */ +static char __initdata opt_mediator[16 ] ="" ; +string_param ("tee" , opt_mediator); +/* * TODO: Add function to alter Dom0 DTB, so we can properly describe * present TEE. @@ -81,14 +86,40 @@ static int __init tee_init(void) {const struct tee_mediator_desc *desc; +if (strcmp (opt_mediator,"" ) ) +printk (XENLOG_INFO "TEE Mediator %s selected from command line\n" , + opt_mediator); + +/* +* When a specific TEE is selected using the 'tee=' command line +* argument, we panic if the probe fails or if the requested TEE is not +* supported. +*/ for ( desc = _steemediator; desc != _eteemediator; desc++ ) { +if (strcmp (opt_mediator,"" ) && +strncmp (opt_mediator, desc->cmdline_name,sizeof (opt_mediator)) ) +continue ; +if ( desc->ops->probe () ) {printk (XENLOG_INFO "Using TEE mediator for %s\n" , desc->name); cur_mediator = desc;return 0 ; } +else if (strcmp (opt_mediator,"" ) ) + { +panic ("TEE mediator %s from command line probe failed\n" , + opt_mediator); +return -EFAULT ; + } + } + +if (strcmp (opt_mediator,"" ) ) + { +panic ("TEE Mediator %s from command line not supported\n" , + opt_mediator); +return -EINVAL ; }return 0 ; --2.47.1
Re: [PATCH v4 1/5] xen/arm: Create tee command line parameter
In reply to Bertrand MarquisHi Bertrand,
Quoted 29 lines · Bertrand Marquis
Add a new command line parameter "tee=" to be used to explicitly select what tee mediator is to be used by Xen and fail if it does not exist or the probe function for it failed.
Without specifying which tee is to be used, Xen will use the first one for which the probe function succeeds which depends on the order of the mediator list which depends on the compiler.
Using the command line argument, it is now possible to explicit request a specific TEE mediator and panic on boot if it is not available.Signed-off-by Bertrand Marquis <[email protected]>---
Changes in v4:
- None
Changes in v3:
- Properly classify tee as arm specific (Jan) Changes in v2:
- Patch introduced to add a command line selection of the TEE--- docs/misc/xen-command-line.pandoc | 14 ++++++++++++++ xen/arch/arm/include/asm/tee/tee.h | 4 ++++ xen/arch/arm/tee/tee.c | 31 ++++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+)docs/misc/xen-command-line.pandoc +0
-0
docs/misc/xen-command-line.pandoc Specify the per-cpu trace buffer size in pages. · line 2659
Quoted 9 lines
docs/misc/xen-command-line.pandoc +6
-0@@ -2651,6 +2651,20 @@ Specify the per-cpu trace buffer size in pages. Flag to enable TSC deadline as the APIC timer mode. +### tee (arm) +> `= <string>` + +Specify the TEE mediator to be probed and use. + +The default behaviour is to probe all supported TEEs supported by Xen and use
typo: I think there is one too many "supported". Maybe keep the one after TEEs?
docs/misc/xen-command-line.pandoc Specify the per-cpu trace buffer size in pages. · line 2665
Quoted 6 lines
docs/misc/xen-command-line.pandoc +6
-0+the first one successfully probed. When this parameter is passed, Xen will +probe only the TEE mediator passed as argument and boot will fail if this +mediator is not properly probed or if the requested TEE is not supported by +Xen. + +This parameter can be set to `optee` of `ffa` if the corresponding mediators
typo: s/of/or/
Quoted 32 lines
docs/misc/xen-command-line.pandoc +2
-0+are compiled in. + ### tevt_mask > `= <integer>`xen/arch/arm/include/asm/tee/tee.h +4
-0@@ -55,6 +55,9 @@ struct tee_mediator_desc {/* Printable name of the TEE. */ const char *name; +/* Command line name of the TEE (to be used with tee= cmdline option) */ +const char *cmdline_name; +/* Mediator callbacks as described above. */ const struct tee_mediator_ops *ops;@@ -77,6 +80,7 @@ void tee_free_domain_ctx(struct domain *d); static const struct tee_mediator_desc __tee_desc_##_name __used \ __section(".teemediator.info") = { \ .name = _namestr, \ + .cmdline_name = #_name, \ .ops = _ops, \ .tee_type = _type \ }xen/arch/arm/tee/tee.c +0
-0
xen/arch/arm/tee/tee.c line 22
Quoted 4 lines
xen/arch/arm/tee/tee.c +1
-0@@ -19,12 +19,17 @@#include <xen/errno.h> #include <xen/init.h> #include <xen/types.h> +#include <xen/param.h>
Coding style: The includes are order. So this wants to be moved before xen/types.h
Quoted 14 lines
xen/arch/arm/tee/tee.c +4
-0#include <asm/tee/tee.h> extern const struct tee_mediator_desc _steemediator[], _eteemediator[];static const struct tee_mediator_desc __read_mostly *cur_mediator; +/* Select the TEE mediator using a name on command line. */ +static char __initdata opt_mediator[16 ] ="" ; +string_param ("tee" , opt_mediator); +/* * TODO: Add function to alter Dom0 DTB, so we can properly describe * present TEE.
xen/arch/arm/tee/tee.c static int __init tee_init(void) · line 89
Quoted 4 lines
xen/arch/arm/tee/tee.c +1
-0@@ -81,14 +86,40 @@ static int __init tee_init(void) {const struct tee_mediator_desc *desc; +if (strcmp (opt_mediator,"" ) )
You are using 'strcmp(opt_mediator, "")' several time in the code. This should never changed, so can we introduce a boolean within the function to indicate whether opt_mediator is set?
Quoted 25 lines
xen/arch/arm/tee/tee.c +17
-0+printk (XENLOG_INFO "TEE Mediator %s selected from command line\n" , + opt_mediator); + +/* +* When a specific TEE is selected using the 'tee=' command line +* argument, we panic if the probe fails or if the requested TEE is not +* supported. +*/ for ( desc = _steemediator; desc != _eteemediator; desc++ ) { +if (strcmp (opt_mediator,"" ) && +strncmp (opt_mediator, desc->cmdline_name,sizeof (opt_mediator)) ) + continue;> + if ( desc->ops->probe() ) {printk (XENLOG_INFO "Using TEE mediator for %s\n" , desc->name); cur_mediator = desc;return 0 ; } + else if ( strcmp(opt_mediator, "") ) + { +panic ("TEE mediator %s from command line probe failed\n" , + opt_mediator); +return -EFAULT ; + }
xen/arch/arm/tee/tee.c static int __init tee_init(void) · line 125
Quoted 10 lines
xen/arch/arm/tee/tee.c +7
-0+ } + +if (strcmp (opt_mediator,"" ) ) + { +panic ("TEE Mediator %s from command line not supported\n" , + opt_mediator); +return -EINVAL ; }return 0 ;
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v4 1/5] xen/arm: Create tee command line parameter
Hi Julien,
Quoted 33 lines · Julien Grall
Hi Bertrand,
Quoted 28 lines · Bertrand Marquis
Add a new command line parameter "tee=" to be used to explicitly select what tee mediator is to be used by Xen and fail if it does not exist or the probe function for it failed.
Without specifying which tee is to be used, Xen will use the first one for which the probe function succeeds which depends on the order of the mediator list which depends on the compiler.
Using the command line argument, it is now possible to explicit request a specific TEE mediator and panic on boot if it is not available.Signed-off-by Bertrand Marquis <[email protected]>---
Changes in v4:
- None
Changes in v3:
- Properly classify tee as arm specific (Jan) Changes in v2:
- Patch introduced to add a command line selection of the TEE--- docs/misc/xen-command-line.pandoc | 14 ++++++++++++++ xen/arch/arm/include/asm/tee/tee.h | 4 ++++ xen/arch/arm/tee/tee.c | 31 ++++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+)docs/misc/xen-command-line.pandoc +0
-0@@ -2651,6 +2651,20 @@ Specify the per-cpu trace buffer size in pages. Flag to enable TSC deadline as the APIC timer mode. +### tee (arm)
Quoted 8 lines
docs/misc/xen-command-line.pandoc Specify the per-cpu trace buffer size in pages. · line 2659
Quoted 5 lines
docs/misc/xen-command-line.pandoc +5
-0+> `= <string>` + +Specify the TEE mediator to be probed and use. + +The default behaviour is to probe all supported TEEs supported by Xen and usetypo: I think there is one too many "supported". Maybe keep the one after TEEs?
ack
Quoted 9 lines
docs/misc/xen-command-line.pandoc Specify the per-cpu trace buffer size in pages. · line 2665
Quoted 7 lines
docs/misc/xen-command-line.pandoc +6
-0+the first one successfully probed. When this parameter is passed, Xen will +probe only the TEE mediator passed as argument and boot will fail if this +mediator is not properly probed or if the requested TEE is not supported by +Xen. + +This parameter can be set to `optee` of `ffa` if the corresponding mediatorstypo: s/of/or/
ack
Quoted 28 lines
Quoted 28 lines
docs/misc/xen-command-line.pandoc +2
-0+are compiled in. + ### tevt_mask > `= <integer>` diff --git a/xen/arch/arm/include/asm/tee/tee.h b/xen/arch/arm/include/asm/tee/tee.hindex
0169fd746bcd..15d664e28dce100644xen/arch/arm/include/asm/tee/tee.h +3
-0@@ -55,6 +55,9 @@ struct tee_mediator_desc {/* Printable name of the TEE. */ const char *name; +/* Command line name of the TEE (to be used with tee= cmdline option) */ + const char *cmdline_name; +/* Mediator callbacks as described above. */ const struct tee_mediator_ops *ops; @@ -77,6 +80,7 @@ void tee_free_domain_ctx(struct domain *d); static const struct tee_mediator_desc __tee_desc_##_name __used \ __section(".teemediator.info") = { \ .name = _namestr, \ + .cmdline_name = #_name, \ .ops = _ops, \ .tee_type = _type \ }xen/arch/arm/tee/tee.c +0
-1--- a/xen/arch/arm/tee/tee.c
Quoted 8 lines
xen/arch/arm/tee/tee.c line 22
Quoted 6 lines
+++
b/xen/arch/arm/tee/tee.cxen/arch/arm/tee/tee.c +1
-0@@ -19,12 +19,17 @@#include <xen/errno.h> #include <xen/init.h> #include <xen/types.h> +#include <xen/param.h> Coding style: The includes are order. So this wants to be moved before
xen/types.h
ack
Quoted 17 lines
Quoted 12 lines
xen/arch/arm/tee/tee.c +3
-0#include <asm/tee/tee.h> extern const struct tee_mediator_desc _steemediator[], _eteemediator[];static const struct tee_mediator_desc __read_mostly *cur_mediator; +/* Select the TEE mediator using a name on command line. */ +static char __initdata opt_mediator[16] = ""; +string_param ("tee" , opt_mediator); +/* * TODO: Add function to alter Dom0 DTB, so we can properly describe * present TEE. xen/arch/arm/tee/tee.c static int __init tee_init(void) · line 88
Quoted 3 lines
xen/arch/arm/tee/tee.c +0
-0@@ -81,14 +86,40 @@ static int __init tee_init(void) {const struct tee_mediator_desc *desc; + if ( strcmp(opt_mediator, "") )You are using '
strcmp(opt_mediator, "")' several time in the code. This should never changed, so can we introduce a boolean within the function to indicate whetheropt_mediatoris set?
Very good point, I will do that.
Quoted 27 lines
Quoted 27 lines
xen/arch/arm/tee/tee.c +18
-0+printk (XENLOG_INFO "TEE Mediator %s selected from command line\n" , + opt_mediator); + +/* +* When a specific TEE is selected using the 'tee=' command line +* argument, we panic if the probe fails or if the requested TEE is not +* supported. +*/ for ( desc = _steemediator; desc != _eteemediator; desc++ ) { +if (strcmp (opt_mediator,"" ) && +strncmp (opt_mediator, desc->cmdline_name,sizeof (opt_mediator)) ) + continue;> + if ( desc->ops->probe() ) {printk (XENLOG_INFO "Using TEE mediator for %s\n" , desc->name); cur_mediator = desc;return 0 ; } + else if ( strcmp(opt_mediator, "") ) + { +panic ("TEE mediator %s from command line probe failed\n" , + opt_mediator); +return -EFAULT ; + } + }
Quoted 8 lines
xen/arch/arm/tee/tee.c static int __init tee_init(void) · line 125
Quoted 8 lines
xen/arch/arm/tee/tee.c +6
-0+ +if (strcmp (opt_mediator,"" ) ) + { +panic ("TEE Mediator %s from command line not supported\n" , + opt_mediator); +return -EINVAL ; }return 0 ;
Thanks a lot for the review.
Cheers
Bertrand
Quoted 5 lines
Cheers,
Signature · 1 line
Julien Grall
PATCH v4 2/5 xen/arm: ffa: Introduce VM to VM support
In reply to Bertrand MarquisCreate a CONFIG_FFA_VM_TO_VM parameter to activate FFA communication between VMs.
When activated list VMs in the system with FF-A support in part_info_get.
When VM to VM is activated, Xen will be tainted as Insecure and a message is displayed to the user during the boot as there is no filtering of VMs in FF-A so any VM can communicate or see any other VM in the system.
WARNING: There is no filtering for now and all VMs are listed !!
---
Changes in v4:
- properly handle SPMC version 1.0 header size case in partinfo_get
- switch to local counting variables instead of *pointer += 1 form
- coding style issue with missing spaces in if ()
Changes in v3:
- break partinfo_get in several sub functions to make the implementation easier to understand and lock handling easier
- rework implementation to check size along the way and prevent previous implementation limits which had to check that the number of VMs or SPs did not change
- taint Xen as INSECURE when VM to VM is enabled
Changes in v2:
- Switch ifdef to IS_ENABLED
- dom was not switched to d as requested by Jan because there is already a variable d pointing to the current domain and it must not be shadowed.
--- xen/arch/arm/tee/Kconfig | 11 ++ xen/arch/arm/tee/ffa.c | 12 ++ xen/arch/arm/tee/ffa_partinfo.c | 274 +++++++++++++++++++++----------- xen/arch/arm/tee/ffa_private.h | 12 ++ 4 files changed, 218 insertions(+), 91 deletions(-)
xen/arch/arm/tee/Kconfig +11 -0
@@ -28,5 +28,16 @@ config FFA [1] https://developer.arm.com/documentation/den0077/latest +config FFA_VM_TO_VM + bool "Enable FF-A between VMs (UNSUPPORTED)" if UNSUPPORTED + default n + depends on FFA + help + This option enables to use FF-A between VMs. + This is experimental and there is no access control so any + guest can communicate with any other guest. + + If unsure, say N. + endmenu
xen/arch/arm/tee/ffa.c +12 -0
@@ -464,6 +464,18 @@ static bool ffa_probe(void)printk (XENLOG_INFO "ARM FF-A Mediator version %u.%u\n" ,FFA_MY_VERSION_MAJOR ,FFA_MY_VERSION_MINOR ); +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + { +/* +* When FFA VM to VM is enabled, the current implementation does not +* offer any way to limit which VM can communicate with which VM using +* FF-A. +* Signal this in the xen console and taint the system as insecure. +* TODO: Introduce a solution to limit what a VM can do through FFA. +*/ +printk (XENLOG_ERR "ffa: VM to VM is enabled, system is insecure !!\n" ); +add_taint (TAINT_MACHINE_INSECURE ); + }/* * psci_init_smccc() updates this value with what's reported by EL-3 * or secure world.
xen/arch/arm/tee/ffa_partinfo.c +183 -91
319 lines not on this page · raw message
xen/arch/arm/tee/ffa_private.h +12 -0
@@ -187,6 +187,18 @@*/ #define FFA_PARTITION_INFO_GET_COUNT_FLAG BIT(0, U) +/* +* Partition properties we give for a normal world VM: +* - can send direct message but not receive them +* - can handle indirect messages +* - can receive notifications +* 32/64 bit flag is set depending on the VM +*/ +#define FFA_PART_VM_PROP (FFA_PART_PROP_DIRECT_REQ_SEND | \ + FFA_PART_PROP_INDIRECT_MSGS | \ + FFA_PART_PROP_RECV_NOTIF | \ + FFA_PART_PROP_IS_PE_ID) +/* Flags used in calls to FFA_NOTIFICATION_GET interface */ #define FFA_NOTIF_FLAG_BITMAP_SP BIT(0, U)#define FFA_NOTIF_FLAG_BITMAP_VM BIT(1, U)
Signature · 1 line
2.47.1
Re: [PATCH v4 2/5] xen/arm: ffa: Introduce VM to VM support
Hi,
Quoted 112 lines · Bertrand Marquis
Create a
CONFIG_FFA_VM_TO_VMparameter to activate FFA communication between VMs.
When activated list VMs in the system with FF-A support inpart_info_get.When VM to VM is activated, Xen will be tainted as Insecure and a message is displayed to the user during the boot as there is no filtering of VMs in FF-A so any VM can communicate or see any other VM in the system.
WARNING: There is no filtering for now and all VMs are listed !!
Signed-off-by Bertrand Marquis <[email protected]>---
Changes in v4:
- properly handle SPMC version 1.0 header size case inpartinfo_get
- switch to local counting variables instead of *pointer += 1 form
- coding style issue with missing spaces in if ()
Changes in v3:
- breakpartinfo_getin several sub functions to make the implementation easier to understand and lock handling easier
- rework implementation to check size along the way and prevent previous implementation limits which had to check that the number of VMs or SPs did not change
- taint Xen as INSECURE when VM to VM is enabled
Changes in v2:
- Switch ifdef toIS_ENABLED
- dom was not switched to d as requested by Jan because there is already a variable d pointing to the current domain and it must not be shadowed.
---xen/arch/arm/tee/Kconfig | 11 ++ xen/arch/arm/tee/ffa.c | 12 ++ xen/arch/arm/tee/ffa_partinfo.c | 274 +++++++++++++++++++++----------- xen/arch/arm/tee/ffa_private.h | 12 ++ 4 files changed, 218 insertions(+), 91 deletions(-)xen/arch/arm/tee/Kconfig +11
-0@@ -28,5 +28,16 @@ config FFA [1] https://developer.arm.com/documentation/den0077/latest +config FFA_VM_TO_VM + bool "Enable FF-A between VMs (UNSUPPORTED)" if UNSUPPORTED + default n + depends on FFA + help + This option enables to use FF-A between VMs. + This is experimental and there is no access control so any + guest can communicate with any other guest. + + If unsure, say N. +endmenu
xen/arch/arm/tee/ffa.c +12
-0@@ -464,6 +464,18 @@ static bool ffa_probe(void)printk (XENLOG_INFO "ARM FF-A Mediator version %u.%u\n" ,FFA_MY_VERSION_MAJOR ,FFA_MY_VERSION_MINOR ); +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + { +/* +* When FFA VM to VM is enabled, the current implementation does not +* offer any way to limit which VM can communicate with which VM using +* FF-A. +* Signal this in the xen console and taint the system as insecure. +* TODO: Introduce a solution to limit what a VM can do through FFA. +*/ +printk (XENLOG_ERR "ffa: VM to VM is enabled, system is insecure !!\n" ); +add_taint (TAINT_MACHINE_INSECURE ); + }/* * psci_init_smccc() updates this value with what's reported by EL-3 * or secure world. xen/arch/arm/tee/ffa_partinfo.c +19
-1@@ -63,9 +63,156 @@ static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags,return ret; } -void ffa_handle_partition_info_get (struct cpu_user_regs *regs) +static int32_t ffa_get_sp_count (uint32_t *uuid,uint32_t *sp_count) +{ +uint32_t src_size; + +return ffa_partition_info_get (uuid,FFA_PARTITION_INFO_GET_COUNT_FLAG , + sp_count, &src_size); +} + +static int32_t ffa_get_sp_partinfo (uint32_t *uuid,uint32_t *sp_count, +void *dst_buf,void *end_buf, +uint32_t dst_size) {int32_t ret; +uint32_t src_size, real_sp_count; +void *src_buf = ffa_rx; +uint32_t count =0 ; + +/* Do we have a RX buffer with the SPMC */ +if ( !ffa_rx ) +return FFA_RET_DENIED ; +
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 96
Quoted 10 lines
xen/arch/arm/tee/ffa_partinfo.c +10
-0+/* We need to use the RX buffer to receive the list */ +spin_lock (&ffa_rx_buffer_lock); + + ret =ffa_partition_info_get (uuid,0 , &real_sp_count, &src_size); +if ( ret ) + goto out; + +/* We now own the RX buffer */ + +/* We only support a 1.1 firmware version */
This comment should have been removed.
I will fix it on next version of might be possible to do on commit if there are no further comments here.
Cheers
Bertrand
Jens Wiklander
Reviewed-byRe: [PATCH v4 2/5] xen/arm: ffa: Introduce VM to VM support
Hi Bertrand,
Quoted 40 lines · Bertrand Marquis
Hi,
Quoted 37 lines · Bertrand Marquis
Create a
CONFIG_FFA_VM_TO_VMparameter to activate FFA communication between VMs.
When activated list VMs in the system with FF-A support inpart_info_get.When VM to VM is activated, Xen will be tainted as Insecure and a message is displayed to the user during the boot as there is no filtering of VMs in FF-A so any VM can communicate or see any other VM in the system.
WARNING: There is no filtering for now and all VMs are listed !!
Signed-off-by Bertrand Marquis <[email protected]>---
Changes in v4:
- properly handle SPMC version 1.0 header size case inpartinfo_get
- switch to local counting variables instead of *pointer += 1 form
- coding style issue with missing spaces in if ()
Changes in v3:
- breakpartinfo_getin several sub functions to make the implementation easier to understand and lock handling easier
- rework implementation to check size along the way and prevent previous implementation limits which had to check that the number of VMs or SPs did not change
- taint Xen as INSECURE when VM to VM is enabled
Changes in v2:
- Switch ifdef toIS_ENABLED
- dom was not switched to d as requested by Jan because there is already a variable d pointing to the current domain and it must not be shadowed.
---xen/arch/arm/tee/Kconfig | 11 ++ xen/arch/arm/tee/ffa.c | 12 ++ xen/arch/arm/tee/ffa_partinfo.c | 274 +++++++++++++++++++++----------- xen/arch/arm/tee/ffa_private.h | 12 ++ 4 files changed, 218 insertions(+), 91 deletions(-)
With the comment you mention below fixed, please add:
Cheers,
Jens
Quoted 93 lines
Quoted 75 lines
xen/arch/arm/tee/Kconfig +11
-0@@ -28,5 +28,16 @@ config FFA [1] https://developer.arm.com/documentation/den0077/latest +config FFA_VM_TO_VM + bool "Enable FF-A between VMs (UNSUPPORTED)" if UNSUPPORTED + default n + depends on FFA + help + This option enables to use FF-A between VMs. + This is experimental and there is no access control so any + guest can communicate with any other guest. + + If unsure, say N. +endmenu
xen/arch/arm/tee/ffa.c +12
-0@@ -464,6 +464,18 @@ static bool ffa_probe(void)printk (XENLOG_INFO "ARM FF-A Mediator version %u.%u\n" ,FFA_MY_VERSION_MAJOR ,FFA_MY_VERSION_MINOR ); +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + { +/* +* When FFA VM to VM is enabled, the current implementation does not +* offer any way to limit which VM can communicate with which VM using +* FF-A. +* Signal this in the xen console and taint the system as insecure. +* TODO: Introduce a solution to limit what a VM can do through FFA. +*/ +printk (XENLOG_ERR "ffa: VM to VM is enabled, system is insecure !!\n" ); +add_taint (TAINT_MACHINE_INSECURE ); + }/* * psci_init_smccc() updates this value with what's reported by EL-3 * or secure world. xen/arch/arm/tee/ffa_partinfo.c +19
-1@@ -63,9 +63,156 @@ static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags,return ret; } -void ffa_handle_partition_info_get (struct cpu_user_regs *regs) +static int32_t ffa_get_sp_count (uint32_t *uuid,uint32_t *sp_count) +{ +uint32_t src_size; + +return ffa_partition_info_get (uuid,FFA_PARTITION_INFO_GET_COUNT_FLAG , + sp_count, &src_size); +} + +static int32_t ffa_get_sp_partinfo (uint32_t *uuid,uint32_t *sp_count, +void *dst_buf,void *end_buf, +uint32_t dst_size) {int32_t ret; +uint32_t src_size, real_sp_count; +void *src_buf = ffa_rx; +uint32_t count =0 ; + +/* Do we have a RX buffer with the SPMC */ +if ( !ffa_rx ) +return FFA_RET_DENIED ; +xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 96
Quoted 10 lines
xen/arch/arm/tee/ffa_partinfo.c +10
-0+/* We need to use the RX buffer to receive the list */ +spin_lock (&ffa_rx_buffer_lock); + + ret =ffa_partition_info_get (uuid,0 , &real_sp_count, &src_size); +if ( ret ) + goto out; + +/* We now own the RX buffer */ + +/* We only support a 1.1 firmware version */ This comment should have been removed.
I will fix it on next version of might be possible to do on commit if there are no further comments here.Cheers
Bertrand
Re: [PATCH v4 2/5] xen/arm: ffa: Introduce VM to VM support
In reply to Bertrand MarquisHi Bertrand,
I didn't dig in the spec and neither some of the callers. So I will mainly focus on the implementation from Xen PoV.
Quoted 117 lines · Bertrand Marquis
Create a
CONFIG_FFA_VM_TO_VMparameter to activate FFA communication between VMs.
When activated list VMs in the system with FF-A support inpart_info_get.When VM to VM is activated, Xen will be tainted as Insecure and a message is displayed to the user during the boot as there is no filtering of VMs in FF-A so any VM can communicate or see any other VM in the system.
WARNING: There is no filtering for now and all VMs are listed !!
Signed-off-by Bertrand Marquis <[email protected]>---
Changes in v4:
- properly handle SPMC version 1.0 header size case inpartinfo_get
- switch to local counting variables instead of *pointer += 1 form
- coding style issue with missing spaces in if ()
Changes in v3:
- breakpartinfo_getin several sub functions to make the implementation easier to understand and lock handling easier
- rework implementation to check size along the way and prevent previous implementation limits which had to check that the number of VMs or SPs did not change
- taint Xen as INSECURE when VM to VM is enabled
Changes in v2:
- Switch ifdef toIS_ENABLED
- dom was not switched to d as requested by Jan because there is already a variable d pointing to the current domain and it must not be shadowed.--- xen/arch/arm/tee/Kconfig | 11 ++ xen/arch/arm/tee/ffa.c | 12 ++ xen/arch/arm/tee/ffa_partinfo.c | 274 +++++++++++++++++++++----------- xen/arch/arm/tee/ffa_private.h | 12 ++ 4 files changed, 218 insertions(+), 91 deletions(-)xen/arch/arm/tee/Kconfig +11
-0@@ -28,5 +28,16 @@ config FFA [1] https://developer.arm.com/documentation/den0077/latest +config FFA_VM_TO_VM + bool "Enable FF-A between VMs (UNSUPPORTED)" if UNSUPPORTED + default n + depends on FFA + help + This option enables to use FF-A between VMs. + This is experimental and there is no access control so any + guest can communicate with any other guest. + + If unsure, say N. + endmenuxen/arch/arm/tee/ffa.c +12
-0@@ -464,6 +464,18 @@ static bool ffa_probe(void)printk (XENLOG_INFO "ARM FF-A Mediator version %u.%u\n" ,FFA_MY_VERSION_MAJOR ,FFA_MY_VERSION_MINOR ); +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + { +/* +* When FFA VM to VM is enabled, the current implementation does not +* offer any way to limit which VM can communicate with which VM using +* FF-A. +* Signal this in the xen console and taint the system as insecure. +* TODO: Introduce a solution to limit what a VM can do through FFA. +*/ +printk (XENLOG_ERR "ffa: VM to VM is enabled, system is insecure !!\n" ); +add_taint (TAINT_MACHINE_INSECURE ); + }/* * psci_init_smccc() updates this value with what's reported by EL-3 * or secure world. xen/arch/arm/tee/ffa_partinfo.c +26
-1@@ -63,9 +63,156 @@ static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags,return ret; } -void ffa_handle_partition_info_get (struct cpu_user_regs *regs) +static int32_t ffa_get_sp_count (uint32_t *uuid,uint32_t *sp_count) +{ +uint32_t src_size; + +return ffa_partition_info_get (uuid,FFA_PARTITION_INFO_GET_COUNT_FLAG , + sp_count, &src_size); +} + +static int32_t ffa_get_sp_partinfo (uint32_t *uuid,uint32_t *sp_count, +void *dst_buf,void *end_buf, +uint32_t dst_size) {int32_t ret; +uint32_t src_size, real_sp_count; +void *src_buf = ffa_rx; +uint32_t count =0 ; + +/* Do we have a RX buffer with the SPMC */ +if ( !ffa_rx ) +return FFA_RET_DENIED ; + +/* We need to use the RX buffer to receive the list */ +spin_lock (&ffa_rx_buffer_lock); + + ret =ffa_partition_info_get (uuid,0 , &real_sp_count, &src_size); +if ( ret ) + goto out; +
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 103
Quoted 10 lines
xen/arch/arm/tee/ffa_partinfo.c +10
-0+/* We now own the RX buffer */ + +/* We only support a 1.1 firmware version */ +if ( src_size <sizeof (struct ffa_partition_info_1_0 ) ) + { + ret =FFA_RET_NOT_SUPPORTED ; + goto out_release; + } + +for (uint32_t sp_num =0 ; sp_num < real_sp_count; sp_num++ )
What's the upper limit of real_sp_count? Asking just in case we need to handle preemption.
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 110
Quoted 7 lines
xen/arch/arm/tee/ffa_partinfo.c +7
-0+ { +struct ffa_partition_info_1_1 *fpi = src_buf; + +/* filter out SP not following bit 15 convention if any */ +if (FFA_ID_IS_SECURE (fpi->id) ) + { +if ( dst_buf + dst_size > end_buf )
Can "dst_buf + dst_size" overflow?
Also, NIT: can you add parenthese to make clear about precedence?
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 116
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+ { + ret =FFA_RET_NO_MEMORY ; + goto out_release; + } + +memcpy (dst_buf, src_buf,MIN (src_size, dst_size));
What's the maximum size of src_size and dst_size?
Quoted 10 lines
xen/arch/arm/tee/ffa_partinfo.c +10
-0+if ( dst_size > src_size ) +memset (dst_buf + src_size,0 , dst_size - src_size); + + dst_buf += dst_size; + count++; + } + + src_buf += src_size; + } +
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 136
Quoted 10 lines
xen/arch/arm/tee/ffa_partinfo.c +10
-0+ *sp_count = count; + +out_release: +ffa_hyp_rx_release (); +out: +spin_unlock (&ffa_rx_buffer_lock); +return ret; +} + +static uint32_t ffa_get_vm_count (void )
Is this meant to be called often? If so, can we instead have a counter that will be incremented when the vTEE is first initialized and then decremented when the VM is destroyed?
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 139
Quoted 3 lines
xen/arch/arm/tee/ffa_partinfo.c +3
-0+{ +struct domain *d = current->domain; +struct domain *dom;
NIT: "d" and "dom" are a bit too close. Could we rename "d" with "curr_d"?
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 142
Quoted 3 lines
xen/arch/arm/tee/ffa_partinfo.c +3
-0+uint32_t vm_count =0 ; + +/* Count the number of VM with FF-A support */
This comment implies this is including the current VM. But ...
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 148
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+rcu_read_lock (&domlist_read_lock); +for_each_domain ( dom ) + { +struct ffa_ctx *vm = dom->arch.tee; + +if ( dom != d && vm !=NULL && vm->guest_vers !=0 )
... here you are excluding it. Also, it sounds like this is support by the OS rather than the VM itself. Is that correct?
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 151
Quoted 3 lines
xen/arch/arm/tee/ffa_partinfo.c +3
-0+ vm_count++;> + } +rcu_read_unlock (&domlist_read_lock); +> + return vm_count;
OOI, I guess this value is just used as an hint? Asking because the number of domains can change at any point.
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 165
Quoted 12 lines
xen/arch/arm/tee/ffa_partinfo.c +12
-0+} + +static int32_t ffa_get_vm_partinfo (uint32_t *vm_count,void *dst_buf, +void *end_buf,uint32_t dst_size) +{ +struct domain *d = current->domain; +struct domain *dom; +int32_t ret =FFA_RET_OK ; +uint32_t count =0 ; + +rcu_read_lock (&domlist_read_lock); +for_each_domain ( dom )
We can have quite a lot of domains in the system. So how can we ensure this is not hogging a pCPU?
I would be ok to delay the work, but we need a TODO so we remember to address it before this is security supported.
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+ { +struct ffa_ctx *vm = dom->arch.tee; + +/* +* we do not add the VM calling to the list and only VMs with +* FF-A support
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags,
Quoted 10 lines
xen/arch/arm/tee/ffa_partinfo.c +10
-0+*/ +if ( dom != d && vm !=NULL && vm->guest_vers !=0 ) + { +/* +* We do not have UUID info for VMs so use +* the 1.0 structure so that we set UUIDs to +* zero using memset +*/ +struct ffa_partition_info_1_0 srcvm; +> + if ( dst_buf + dst_size > end_buf )
Same question as the other similar check.
Quoted 213 lines
xen/arch/arm/tee/ffa_partinfo.c +68
-90+ { + ret =FFA_RET_NO_MEMORY ; + goto out; + } + + srcvm.id =ffa_get_vm_id (dom); + srcvm.execution_context = dom->max_vcpus; + srcvm.partition_properties =FFA_PART_VM_PROP ; +if (is_64bit_domain (dom) ) + srcvm.partition_properties |=FFA_PART_PROP_AARCH64_STATE ; + +memcpy (dst_buf, &srcvm,MIN (sizeof (srcvm), dst_size)); + +if ( dst_size >sizeof (srcvm) ) +memset (dst_buf +sizeof (srcvm),0 , + dst_size -sizeof (srcvm)); + + dst_buf += dst_size; + count++; + } + } + *vm_count = count; + +out: +rcu_read_unlock (&domlist_read_lock); + +return ret; +} + + +void ffa_handle_partition_info_get (struct cpu_user_regs *regs) +{ +int32_t ret =FFA_RET_OK ;struct domain *d = current->domain;struct ffa_ctx *ctx = d->arch.tee;uint32_t flags =get_user_reg (regs,5 );@@ -75,9 +222,9 @@ void ffa_handle_partition_info_get(struct cpu_user_regs *regs)get_user_reg (regs,3 ), get_user_reg(regs, 4), }; -uint32_t src_size, dst_size; -void *dst_buf; -uint32_t ffa_sp_count =0 ; +uint32_t dst_size =0 ; +void *dst_buf, *end_buf; +uint32_t ffa_vm_count =0 , ffa_sp_count =0 ;/* * If the guest is v1.0, he does not get back the entry size so we must @@ -89,118 +236,63 @@ void ffa_handle_partition_info_get(struct cpu_user_regs *regs)else dst_size =sizeof (struct ffa_partition_info_1_1 ); -/* -* FF-A v1.0 has w5 MBZ while v1.1 allows -* FFA_PARTITION_INFO_GET_COUNT_FLAG to be non-zero. -* -* FFA_PARTITION_INFO_GET_COUNT is only using registers and not the -* rxtx buffer so do the partition_info_get directly. -*/ -if ( flags ==FFA_PARTITION_INFO_GET_COUNT_FLAG && - ctx->guest_vers ==FFA_VERSION_1_1 ) +/* Only count requested */ +if ( flags ) { -if (ffa_fw_supports_fid (FFA_PARTITION_INFO_GET ) ) - ret =ffa_partition_info_get (uuid, flags, &ffa_sp_count, - &src_size); -else - ret =FFA_RET_OK ; +/* +* FF-A v1.0 has w5 MBZ while v1.1 allows +* FFA_PARTITION_INFO_GET_COUNT_FLAG to be non-zero. +*/ +if ( ctx->guest_vers ==FFA_VERSION_1_0 || + flags !=FFA_PARTITION_INFO_GET_COUNT_FLAG ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out; + } - goto out; - } +if (ffa_fw_supports_fid (FFA_PARTITION_INFO_GET ) ) + { + ret =ffa_get_sp_count (uuid, &ffa_sp_count); +if ( ret ) + goto out; + } -if ( flags ) - { - ret =FFA_RET_INVALID_PARAMETERS ; - goto out; - } +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + ffa_vm_count =ffa_get_vm_count (); -if ( !ffa_fw_supports_fid (FFA_PARTITION_INFO_GET ) ) - { -/* Just give an empty partition list to the caller */ - ret =FFA_RET_OK ; goto out; } +/* Get the RX buffer to write the list of partitions */ ret =ffa_rx_acquire (d);if ( ret !=FFA_RET_OK ) goto out; dst_buf = ctx->rx; + end_buf = ctx->rx + ctx->page_count *FFA_PAGE_SIZE ; -if ( !ffa_rx ) +if (ffa_fw_supports_fid (FFA_PARTITION_INFO_GET ) ) { - ret =FFA_RET_DENIED ; - goto out_rx_release; - } - -spin_lock (&ffa_rx_buffer_lock); - - ret =ffa_partition_info_get (uuid,0 , &ffa_sp_count, &src_size); - -if ( ret ) - goto out_rx_hyp_unlock; + ret =ffa_get_sp_partinfo (uuid, &ffa_sp_count, dst_buf, end_buf, + dst_size); -/* -* ffa_partition_info_get() succeeded so we now own the RX buffer we -* share with the SPMC. We must give it back using ffa_hyp_rx_release() -* once we've copied the content. -*/ +if ( ret ) + goto out_rx_release; -/* we cannot have a size smaller than 1.0 structure */ -if ( src_size <sizeof (struct ffa_partition_info_1_0 ) ) - { - ret =FFA_RET_NOT_SUPPORTED ; - goto out_rx_hyp_release; + dst_buf += ffa_sp_count * dst_size; } -if ( ctx->page_count *FFA_PAGE_SIZE < ffa_sp_count * dst_size ) - { - ret =FFA_RET_NO_MEMORY ; - goto out_rx_hyp_release; - } +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + ret =ffa_get_vm_partinfo (&ffa_vm_count, dst_buf, end_buf, dst_size); -if ( ffa_sp_count >0 ) - { -uint32_t n, n_limit = ffa_sp_count; -void *src_buf = ffa_rx; - -/* copy the secure partitions info */ -for ( n =0 ; n < n_limit; n++ ) - { -struct ffa_partition_info_1_1 *fpi = src_buf; - -/* filter out SP not following bit 15 convention if any */ -if (FFA_ID_IS_SECURE (fpi->id) ) - { -memcpy (dst_buf, src_buf, dst_size); - dst_buf += dst_size; - } -else - ffa_sp_count--; - - src_buf += src_size; - } - } - -out_rx_hyp_release: -ffa_hyp_rx_release (); -out_rx_hyp_unlock: -spin_unlock (&ffa_rx_buffer_lock); out_rx_release: -/* -* The calling VM RX buffer only contains data to be used by the VM if the -* call was successful, in which case the VM has to release the buffer -* once it has used the data. -* If something went wrong during the call, we have to release the RX -* buffer back to the SPMC as the VM will not do it. -*/ -if ( ret !=FFA_RET_OK ) +if ( ret )ffa_rx_release (d); out:if ( ret )ffa_set_regs_error (regs, ret);else -ffa_set_regs_success (regs, ffa_sp_count, dst_size); +ffa_set_regs_success (regs, ffa_sp_count + ffa_vm_count, dst_size); } static int32_t ffa_direct_req_send_vm(uint16_t sp_id, uint16_t vm_id,xen/arch/arm/tee/ffa_private.h +5
-0@@ -187,6 +187,18 @@*/ #define FFA_PARTITION_INFO_GET_COUNT_FLAG BIT(0, U) +/* +* Partition properties we give for a normal world VM: +* - can send direct message but not receive them +* - can handle indirect messages +* - can receive notifications
xen/arch/arm/tee/ffa_private.h line 204
Quoted 10 lines
xen/arch/arm/tee/ffa_private.h +7
-0+* 32/64 bit flag is set depending on the VM +*/ +#define FFA_PART_VM_PROP (FFA_PART_PROP_DIRECT_REQ_SEND | \ + FFA_PART_PROP_INDIRECT_MSGS | \ + FFA_PART_PROP_RECV_NOTIF | \ + FFA_PART_PROP_IS_PE_ID) +/* Flags used in calls to FFA_NOTIFICATION_GET interface */ #define FFA_NOTIF_FLAG_BITMAP_SP BIT(0, U)#define FFA_NOTIF_FLAG_BITMAP_VM BIT(1, U)
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v4 2/5] xen/arm: ffa: Introduce VM to VM support
Hi Julien,
Quoted 5 lines · Julien Grall
Hi Bertrand,
I didn't dig in the spec and neither some of the callers. So I will mainly focus on the implementation from Xen PoV.
Perfectly ok I think, Jens review should cover the spec point of view.
Quoted 114 lines · Bertrand Marquis
Quoted 112 lines
Create a
CONFIG_FFA_VM_TO_VMparameter to activate FFA communication between VMs.
When activated list VMs in the system with FF-A support inpart_info_get. When VM to VM is activated, Xen will be tainted as Insecure and a message is displayed to the user during the boot as there is no filtering of VMs in FF-A so any VM can communicate or see any other VM in the system.
WARNING: There is no filtering for now and all VMs are listed !!Signed-off-by Bertrand Marquis <[email protected]>---
Changes in v4:
- properly handle SPMC version 1.0 header size case inpartinfo_get
- switch to local counting variables instead of *pointer += 1 form
- coding style issue with missing spaces in if ()
Changes in v3:
- breakpartinfo_getin several sub functions to make the implementation easier to understand and lock handling easier
- rework implementation to check size along the way and prevent previous implementation limits which had to check that the number of VMs or SPs did not change
- taint Xen as INSECURE when VM to VM is enabled
Changes in v2:
- Switch ifdef toIS_ENABLED
- dom was not switched to d as requested by Jan because there is already a variable d pointing to the current domain and it must not be shadowed.--- xen/arch/arm/tee/Kconfig | 11 ++ xen/arch/arm/tee/ffa.c | 12 ++ xen/arch/arm/tee/ffa_partinfo.c | 274 +++++++++++++++++++++----------- xen/arch/arm/tee/ffa_private.h | 12 ++ 4 files changed, 218 insertions(+), 91 deletions(-)xen/arch/arm/tee/Kconfig +10
-0@@ -28,5 +28,16 @@ config FFA [1] https://developer.arm.com/documentation/den0077/latest +config FFA_VM_TO_VM + bool "Enable FF-A between VMs (UNSUPPORTED)" if UNSUPPORTED + default n + depends on FFA + help + This option enables to use FF-A between VMs. + This is experimental and there is no access control so any + guest can communicate with any other guest. + + If unsure, say N. + endmenu diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.cindex
3bbdd7168a6b..e41ab5f8ada6100644xen/arch/arm/tee/ffa.c +11
-0@@ -464,6 +464,18 @@ static bool ffa_probe(void)printk (XENLOG_INFO "ARM FF-A Mediator version %u.%u\n" ,FFA_MY_VERSION_MAJOR ,FFA_MY_VERSION_MINOR ); + if ( IS_ENABLED(CONFIG_FFA_VM_TO_VM) ) + { +/* +* When FFA VM to VM is enabled, the current implementation does not +* offer any way to limit which VM can communicate with which VM using +* FF-A. +* Signal this in the xen console and taint the system as insecure. +* TODO: Introduce a solution to limit what a VM can do through FFA. +*/ +printk (XENLOG_ERR "ffa: VM to VM is enabled, system is insecure !!\n" ); +add_taint (TAINT_MACHINE_INSECURE ); + }/* * psci_init_smccc() updates this value with what's reported by EL-3 * or secure world. xen/arch/arm/tee/ffa_partinfo.c +30
-0@@ -63,9 +63,156 @@ static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags,return ret; } -void ffa_handle_partition_info_get(struct cpu_user_regs *regs) +static int32_t ffa_get_sp_count(uint32_t *uuid, uint32_t *sp_count) +{ +uint32_t src_size; + +return ffa_partition_info_get (uuid,FFA_PARTITION_INFO_GET_COUNT_FLAG , + sp_count, &src_size); +} + +static int32_t ffa_get_sp_partinfo (uint32_t *uuid,uint32_t *sp_count, +void *dst_buf,void *end_buf, +uint32_t dst_size) {int32_t ret; +uint32_t src_size, real_sp_count; +void *src_buf = ffa_rx; +uint32_t count =0 ; + +/* Do we have a RX buffer with the SPMC */ +if ( !ffa_rx ) +return FFA_RET_DENIED ; + +/* We need to use the RX buffer to receive the list */ +spin_lock (&ffa_rx_buffer_lock); + + ret =ffa_partition_info_get (uuid,0 , &real_sp_count, &src_size); +if ( ret ) + goto out; + +/* We now own the RX buffer */ + +/* We only support a 1.1 firmware version */ +if ( src_size <sizeof (struct ffa_partition_info_1_0 ) )
Quoted 8 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 103
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+ { + ret =FFA_RET_NOT_SUPPORTED ; + goto out_release; + } + +for (uint32_t sp_num =0 ; sp_num < real_sp_count; sp_num++ )What's the upper limit of
real_sp_count? Asking just in case we need to handle preemption.
In theory that would be 32k but in practice the number of entries we can receive is limited by the size of the exchange area we have with the secure world.
This area is currently defined to be one page and each entry is 8 byte in the case where firmware is 1.0 (24 bytes otherwise).
This is an upper limit of 500 entries.
Now I do think this is completely unrealistic to imagine a secure world with 500 SPs so If you are ok I would rather define an upper limit in Xen (I would say 64 SPs) that can be changed in the code (through a define).
This call currently does not support preemption in the FF-A spec (and that is something i will check for future versions) so I would have no solution to "continue" it.
Would the "define" solution be acceptable for now ?
Quoted 10 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 110
Quoted 8 lines
xen/arch/arm/tee/ffa_partinfo.c +7
-0+ { +struct ffa_partition_info_1_1 *fpi = src_buf; + +/* filter out SP not following bit 15 convention if any */ +if (FFA_ID_IS_SECURE (fpi->id) ) + { +if ( dst_buf + dst_size > end_buf )Can "
dst_buf+dst_size" overflow?
Well it would mean that the memory corresponding to the buffer in Xen is mapped at a virtual address at the end of the 64bit space. dst_size is set in the code (see ffa_handle_partition_info_get) so can only be 8 or 24.
Probability is low but cannot be ruled out so I will modify the code to check for overflow.
Quoted 2 lines
Also, NIT: can you add parenthese to make clear about precedence?
ack
Quoted 9 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 116
Quoted 7 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+ { + ret =FFA_RET_NO_MEMORY ; + goto out_release; + } + +memcpy (dst_buf, src_buf,MIN (src_size, dst_size));What's the maximum size of
src_sizeanddst_size?
dst_size is set in the code (8 or 24) so there should be no issue here. With the overflow checked upper we should be safe here.
Quoted 15 lines
Quoted 15 lines
xen/arch/arm/tee/ffa_partinfo.c +14
-0+if ( dst_size > src_size ) +memset (dst_buf + src_size,0 , dst_size - src_size); + + dst_buf += dst_size; + count++; + } + + src_buf += src_size; + } + + *sp_count = count; + +out_release: +ffa_hyp_rx_release ();
Quoted 8 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 136
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+out: +spin_unlock (&ffa_rx_buffer_lock); +return ret; +} + +static uint32_t ffa_get_vm_count (void )Is this meant to be called often? If so, can we instead have a counter that will be incremented when the vTEE is first initialized and then decremented when the VM is destroyed?
As of now we could have a global counter that we increase or decrease when a domain version is negociated and when a domain is destroyed.
We could also have some kind of global save of the result to be returned to a guest.
But I did not do that because:
- cpu time required to update the list would be taken out an FF-A call for FFA_VERSION of a VM which would require a global lock to protect the data
- when we will have filtering the data will be per VM (which would make the initial update more complex)
- incoming we have a notion of UUID and filtering depending on the requested UUID which will make the global value only useable in a limited number of cases.
I have 2 pending series on top of this one which would have to remove such optimisations.
At the end this is definitely not something supposed to call often (linux driver is calling it once during init).
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 139
Quoted 4 lines
xen/arch/arm/tee/ffa_partinfo.c +3
-0+{ +struct domain *d = current->domain; +struct domain *dom;NIT: "d" and "dom" are a bit too close. Could we rename "d" with "
curr_d"?
i will go with curr_d dest_d to make this clearer.
Quoted 7 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 142
Quoted 4 lines
xen/arch/arm/tee/ffa_partinfo.c +3
-0+uint32_t vm_count =0 ; + +/* Count the number of VM with FF-A support */ This comment implies this is including the current VM. But ...
Quoted 8 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 148
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+rcu_read_lock (&domlist_read_lock); +for_each_domain ( dom ) + { +struct ffa_ctx *vm = dom->arch.tee; + +if ( dom != d && vm !=NULL && vm->guest_vers !=0 )... here you are excluding it. Also, it sounds like this is support by the OS rather than the VM itself. Is that correct?
I have a comment to explain that one in a different serie that i will put here.
Basically before 1.2, the spec was a bit blurry on if or not the result should include the calling VM and in fact Linux driver (before 1.2) was ending with an error if its own data was included in the result hence this filter.
I will add a comment for that.
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 151
Quoted 4 lines
xen/arch/arm/tee/ffa_partinfo.c +3
-0+ vm_count++;> + } +rcu_read_unlock (&domlist_read_lock); +> + return vm_count;OOI, I guess this value is just used as an hint? Asking because the number of domains can change at any point.
Definitely yes. The data is what it is when called but anything could change after.
This is mostly used as hint by callers.
Quoted 16 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 165
Quoted 13 lines
xen/arch/arm/tee/ffa_partinfo.c +12
-0+} + +static int32_t ffa_get_vm_partinfo (uint32_t *vm_count,void *dst_buf, +void *end_buf,uint32_t dst_size) +{ +struct domain *d = current->domain; +struct domain *dom; +int32_t ret =FFA_RET_OK ; +uint32_t count =0 ; + +rcu_read_lock (&domlist_read_lock); +for_each_domain ( dom )We can have quite a lot of domains in the system. So how can we ensure this is not hogging a pCPU?
Quoted 1 line
I would be ok to delay the work, but we need a TODO so we remember to address it before this is security supported.
Definitely something not addressed in the spec for now that needs addressing (as explained earlier). We have other calls that can be "cut" and this one will need to support it in the future.
For now there is not much we can do and limit the number of SPs we allow will not really solve the issue in the VM to VM case so I will add a big TODO as you suggest.
Quoted 18 lines
Quoted 7 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+ { +struct ffa_ctx *vm = dom->arch.tee; + +/* +* we do not add the VM calling to the list and only VMs with +* FF-A support xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags,
Quoted 10 lines
xen/arch/arm/tee/ffa_partinfo.c +10
-0+*/ +if ( dom != d && vm !=NULL && vm->guest_vers !=0 ) + { +/* +* We do not have UUID info for VMs so use +* the 1.0 structure so that we set UUIDs to +* zero using memset +*/ +struct ffa_partition_info_1_0 srcvm; +> + if ( dst_buf + dst_size > end_buf )
Quoted 1 line
Same question as the other similar check.
Same answer and i will add an overflow check.
Quoted 204 lines
Quoted 204 lines
xen/arch/arm/tee/ffa_partinfo.c +67
-81+ { + ret =FFA_RET_NO_MEMORY ; + goto out; + } + + srcvm.id =ffa_get_vm_id (dom); + srcvm.execution_context = dom->max_vcpus; + srcvm.partition_properties =FFA_PART_VM_PROP ; +if (is_64bit_domain (dom) ) + srcvm.partition_properties |=FFA_PART_PROP_AARCH64_STATE ; + +memcpy (dst_buf, &srcvm,MIN (sizeof (srcvm), dst_size)); + +if ( dst_size >sizeof (srcvm) ) +memset (dst_buf +sizeof (srcvm),0 , + dst_size -sizeof (srcvm)); + + dst_buf += dst_size; + count++; + } + } + *vm_count = count; + +out: +rcu_read_unlock (&domlist_read_lock); + +return ret; +} + + +void ffa_handle_partition_info_get (struct cpu_user_regs *regs) +{ +int32_t ret =FFA_RET_OK ;struct domain *d = current->domain;struct ffa_ctx *ctx = d->arch.tee;uint32_t flags =get_user_reg (regs,5 );@@ -75,9 +222,9 @@ void ffa_handle_partition_info_get(struct cpu_user_regs *regs)get_user_reg (regs,3 ), get_user_reg(regs, 4), }; -uint32_t src_size, dst_size; -void *dst_buf; -uint32_t ffa_sp_count =0 ; +uint32_t dst_size =0 ; +void *dst_buf, *end_buf; +uint32_t ffa_vm_count =0 , ffa_sp_count =0 ;/* * If the guest is v1.0, he does not get back the entry size so we must @@ -89,118 +236,63 @@ void ffa_handle_partition_info_get(struct cpu_user_regs *regs)else dst_size =sizeof (struct ffa_partition_info_1_1 ); -/* -* FF-A v1.0 has w5 MBZ while v1.1 allows -* FFA_PARTITION_INFO_GET_COUNT_FLAG to be non-zero. -* -* FFA_PARTITION_INFO_GET_COUNT is only using registers and not the -* rxtx buffer so do the partition_info_get directly. -*/ - if ( flags == FFA_PARTITION_INFO_GET_COUNT_FLAG && - ctx->guest_vers == FFA_VERSION_1_1 ) +/* Only count requested */ + if ( flags ) { -if (ffa_fw_supports_fid (FFA_PARTITION_INFO_GET ) ) - ret =ffa_partition_info_get (uuid, flags, &ffa_sp_count, - &src_size); -else - ret =FFA_RET_OK ; +/* +* FF-A v1.0 has w5 MBZ while v1.1 allows +* FFA_PARTITION_INFO_GET_COUNT_FLAG to be non-zero. +*/ +if ( ctx->guest_vers ==FFA_VERSION_1_0 || + flags !=FFA_PARTITION_INFO_GET_COUNT_FLAG ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out; + } - goto out; - } +if (ffa_fw_supports_fid (FFA_PARTITION_INFO_GET ) ) + { + ret =ffa_get_sp_count (uuid, &ffa_sp_count); +if ( ret ) + goto out; + } - if ( flags ) - { - ret =FFA_RET_INVALID_PARAMETERS ; - goto out; - } +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + ffa_vm_count =ffa_get_vm_count (); - if ( !ffa_fw_supports_fid(FFA_PARTITION_INFO_GET) ) - { -/* Just give an empty partition list to the caller */ - ret =FFA_RET_OK ; goto out; } +/* Get the RX buffer to write the list of partitions */ ret =ffa_rx_acquire (d);if ( ret !=FFA_RET_OK ) goto out; dst_buf = ctx->rx; + end_buf = ctx->rx + ctx->page_count *FFA_PAGE_SIZE ; - if ( !ffa_rx ) +if (ffa_fw_supports_fid (FFA_PARTITION_INFO_GET ) ) { - ret =FFA_RET_DENIED ; - goto out_rx_release; - } - -spin_lock (&ffa_rx_buffer_lock); - - ret =ffa_partition_info_get (uuid,0 , &ffa_sp_count, &src_size); - -if ( ret ) - goto out_rx_hyp_unlock; + ret =ffa_get_sp_partinfo (uuid, &ffa_sp_count, dst_buf, end_buf, + dst_size); -/* -* ffa_partition_info_get() succeeded so we now own the RX buffer we -* share with the SPMC. We must give it back using ffa_hyp_rx_release() -* once we've copied the content. -*/ +if ( ret ) +goto out_rx_release; - /* we cannot have a size smaller than 1.0 structure */ - if ( src_size < sizeof(struct ffa_partition_info_1_0) ) - { - ret = FFA_RET_NOT_SUPPORTED; - goto out_rx_hyp_release; + dst_buf += ffa_sp_count * dst_size; } - if ( ctx->page_count * FFA_PAGE_SIZE < ffa_sp_count * dst_size ) - { - ret =FFA_RET_NO_MEMORY ; - goto out_rx_hyp_release; - } +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + ret =ffa_get_vm_partinfo (&ffa_vm_count, dst_buf, end_buf, dst_size); - if ( ffa_sp_count > 0 ) - { -uint32_t n, n_limit = ffa_sp_count; -void *src_buf = ffa_rx; - -/* copy the secure partitions info */ -for ( n =0 ; n < n_limit; n++ ) - { -struct ffa_partition_info_1_1 *fpi = src_buf; - -/* filter out SP not following bit 15 convention if any */ -if (FFA_ID_IS_SECURE (fpi->id) ) - { -memcpy (dst_buf, src_buf, dst_size); - dst_buf += dst_size; - } -else - ffa_sp_count--; - - src_buf += src_size; - } - } - -out_rx_hyp_release: -ffa_hyp_rx_release (); -out_rx_hyp_unlock: -spin_unlock (&ffa_rx_buffer_lock); out_rx_release: -/* -* The calling VM RX buffer only contains data to be used by the VM if the -* call was successful, in which case the VM has to release the buffer -* once it has used the data. -* If something went wrong during the call, we have to release the RX -* buffer back to the SPMC as the VM will not do it. -*/ -if ( ret !=FFA_RET_OK ) +if ( ret )ffa_rx_release (d); out:if ( ret )ffa_set_regs_error (regs, ret);else -ffa_set_regs_success (regs, ffa_sp_count, dst_size); +ffa_set_regs_success (regs, ffa_sp_count + ffa_vm_count, dst_size); } static int32_t ffa_direct_req_send_vm(uint16_t sp_id, uint16_t vm_id,xen/arch/arm/tee/ffa_private.h +8
-0@@ -187,6 +187,18 @@*/ #define FFA_PARTITION_INFO_GET_COUNT_FLAG BIT(0, U) +/* +* Partition properties we give for a normal world VM: +* - can send direct message but not receive them +* - can handle indirect messages +* - can receive notifications +* 32/64 bit flag is set depending on the VM +*/ +#define FFA_PART_VM_PROP (FFA_PART_PROP_DIRECT_REQ_SEND | \ + FFA_PART_PROP_INDIRECT_MSGS | \
Quoted 8 lines
xen/arch/arm/tee/ffa_private.h line 204
Quoted 6 lines
xen/arch/arm/tee/ffa_private.h +3
-0+ FFA_PART_PROP_RECV_NOTIF | \ + FFA_PART_PROP_IS_PE_ID) +/* Flags used in calls to FFA_NOTIFICATION_GET interface */ #define FFA_NOTIF_FLAG_BITMAP_SP BIT(0, U)#define FFA_NOTIF_FLAG_BITMAP_VM BIT(1, U)Cheers,
Thanks a lot those are very useful comments and feedback for the code and the spec :-)
Cheers
Bertrand
Quoted 3 lines
Signature · 2 lines
-- Julien Grall
Re: [PATCH v4 2/5] xen/arm: ffa: Introduce VM to VM support
Hi Bertrand,
Quoted 42 lines · Bertrand Marquis
Quoted 25 lines · Julien Grall
Quoted 24 lines
xen/arch/arm/tee/ffa_partinfo.c +22
-0+static int32_t ffa_get_sp_partinfo (uint32_t *uuid,uint32_t *sp_count, +void *dst_buf,void *end_buf, +uint32_t dst_size) {int32_t ret; +uint32_t src_size, real_sp_count; +void *src_buf = ffa_rx; +uint32_t count =0 ; + +/* Do we have a RX buffer with the SPMC */ +if ( !ffa_rx ) +return FFA_RET_DENIED ; + +/* We need to use the RX buffer to receive the list */ +spin_lock (&ffa_rx_buffer_lock); + + ret =ffa_partition_info_get (uuid,0 , &real_sp_count, &src_size); +if ( ret ) + goto out; + +/* We now own the RX buffer */ + +/* We only support a 1.1 firmware version */ +if ( src_size <sizeof (struct ffa_partition_info_1_0 ) )Quoted 8 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 103
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+ { + ret =FFA_RET_NOT_SUPPORTED ; + goto out_release; + } + +for (uint32_t sp_num =0 ; sp_num < real_sp_count; sp_num++ )What's the upper limit of
real_sp_count? Asking just in case we need to handle preemption.In theory that would be 32k but in practice the number of entries we can receive is limited by the size of the exchange area we have with the secure world.
This area is currently defined to be one page and each entry is 8 byte in the case where firmware is 1.0 (24 bytes otherwise).
This is an upper limit of 500 entries.
Quoted 8 lines
Now I do think this is completely unrealistic to imagine a secure world with 500 SPs so If you are ok I would rather define an upper limit in Xen (I would say 64 SPs) that can be changed in the code (through a define).
This call currently does not support preemption in the FF-A spec (and that is something i will check for future versions) so I would have no solution to "continue" it.
Would the "define" solution be acceptable for now ?
I think the define solution is acceptable for now and possibly longer. It is an easy way to avoid dealing with preemption.
[...]
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
+static
uint32_tffa_get_vm_count(void)Is this meant to be called often? If so, can we instead have a counter that will be incremented when the vTEE is first initialized and then decremented when the VM is destroyed?
As of now we could have a global counter that we increase or decrease when a domain version is negociated and when a domain is destroyed.
Quoted 7 lines
We could also have some kind of global save of the result to be returned to a guest.
But I did not do that because:
- cpu time required to update the list would be taken out an FF-A call forFFA_VERSIONof a VM which would require a global lock to protect the data
I would have thought an atomic counter would be sufficient. Is there anything you had in mind?
Quoted 11 lines
- when we will have filtering the data will be per VM (which would make the initial update more complex)
- incoming we have a notion of UUID and filtering depending on the requested UUID which will make the global value only useable in a limited number of cases.I have 2 pending series on top of this one which would have to remove such optimisations.
At the end this is definitely not something supposed to call often (linux driver is calling it once during init).
I think it was a mistake for me to asked whether this is called often or not. When it comes to security, what matter is whether a malicious guest could indirectly call ffa_get_vm_count() and delay any work on the pCPU (Xen is not preemptible).
We don't have to resolve this now. But this will need to be addressed before we can we consider FFA security supported. So we should at least add it in the list of issue at the top of the file.
Quoted 24 lines
Quoted 7 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 139
Quoted 5 lines
xen/arch/arm/tee/ffa_partinfo.c +3
-0+{ +struct domain *d = current->domain; +struct domain *dom;NIT: "d" and "dom" are a bit too close. Could we rename "d" with "
curr_d"?i will go with
curr_ddest_dto make this clearer.Quoted 14 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 142
Quoted 4 lines
xen/arch/arm/tee/ffa_partinfo.c +3
-0+uint32_t vm_count =0 ; + +/* Count the number of VM with FF-A support */ This comment implies this is including the current VM. But ...
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+rcu_read_lock (&domlist_read_lock); +for_each_domain ( dom ) + { +struct ffa_ctx *vm = dom->arch.tee; + +if ( dom != d && vm !=NULL && vm->guest_vers !=0 )
Quoted 6 lines
Quoted 1 line
... here you are excluding it. Also, it sounds like this is support by the OS rather than the VM itself. Is that correct?
I have a comment to explain that one in a different serie that i will put here.
Quoted 1 line
Basically before 1.2, the spec was a bit blurry on if or not the result should include the
calling VM and in fact Linux driver (before 1.2) was ending with an error if its own data was included in the result hence this filter.
Thanks for the clarification. Just to clarify...
Quoted 2 lines
I will add a comment for that.
... will the comment be added in this patch?
Quoted 11 lines
Quoted 7 lines
Review
Quoted 5 lines
(quoted diff) +3
-0+ vm_count++;> + } + rcu_read_unlock(&domlist_read_lock); +> + return vm_count;OOI, I guess this value is just used as an hint? Asking because the number of domains can change at any point.
Definitely yes. The data is what it is when called but anything could change after.
This is mostly used as hint by callers.
Does this mean we would always return a fixed number? Asking because this would solve nicely the preemption problem in ffa_get_vm_count().
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v4 2/5] xen/arm: ffa: Introduce VM to VM support
Hi Julien,
Quoted 49 lines · Julien Grall
Hi Bertrand,
Quoted 43 lines · Bertrand Marquis
Quoted 25 lines · Julien Grall
Quoted 24 lines
xen/arch/arm/tee/ffa_partinfo.c +22
-0+static int32_t ffa_get_sp_partinfo (uint32_t *uuid,uint32_t *sp_count, +void *dst_buf,void *end_buf, +uint32_t dst_size) {int32_t ret; +uint32_t src_size, real_sp_count; +void *src_buf = ffa_rx; +uint32_t count =0 ; + +/* Do we have a RX buffer with the SPMC */ +if ( !ffa_rx ) +return FFA_RET_DENIED ; + +/* We need to use the RX buffer to receive the list */ +spin_lock (&ffa_rx_buffer_lock); + + ret =ffa_partition_info_get (uuid,0 , &real_sp_count, &src_size); +if ( ret ) + goto out; + +/* We now own the RX buffer */ + +/* We only support a 1.1 firmware version */ +if ( src_size <sizeof (struct ffa_partition_info_1_0 ) )Quoted 8 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 103
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+ { + ret =FFA_RET_NOT_SUPPORTED ; + goto out_release; + } + +for (uint32_t sp_num =0 ; sp_num < real_sp_count; sp_num++ )What's the upper limit of
real_sp_count? Asking just in case we need to handle preemption.In theory that would be 32k but in practice the number of entries we can receive is limited by the size of the exchange area we have with the secure world. This area is currently defined to be one page and each entry is 8 byte in the case where firmware is 1.0 (24 bytes otherwise).
This is an upper limit of 500 entries.
Now I do think this is completely unrealistic to imagine a secure world with 500 SPs so If you are ok I would rather define an upper limit in Xen (I would say 64 SPs) that can be changed in the code (through a define).
This call currently does not support preemption in the FF-A spec (and that is something i will check for future versions) so I would have no solution to "continue" it.
Quoted 3 lines
Quoted 1 line
Would the "define" solution be acceptable for now ?
I think the define solution is acceptable for now and possibly longer. It is an easy way to avoid dealing with preemption.
I will do something with a default value of 64 which should fit for now (and maybe longer).
Quoted 16 lines
[...]
Quoted 11 lines
Quoted 3 lines
Quoted 1 line
+static
uint32_tffa_get_vm_count(void)Is this meant to be called often? If so, can we instead have a counter that will be incremented when the vTEE is first initialized and then decremented when the VM is destroyed?
As of now we could have a global counter that we increase or decrease when a domain version is negociated and when a domain is destroyed. We could also have some kind of global save of the result to be returned to a guest.
But I did not do that because:
- cpu time required to update the list would be taken out an FF-A call forFFA_VERSIONof a VM which would require a global lock to protect the data
Quoted 1 line
I would have thought an atomic counter would be sufficient. Is there anything you had in mind?
Very good idea. I only need to increase it when an FFA_VERSION has been negociated and decrease on VM destruction.
I will do that.
Quoted 11 lines
Quoted 10 lines
- when we will have filtering the data will be per VM (which would make the initial update more complex)
- incoming we have a notion of UUID and filtering depending on the requested UUID which will make the global value only useable in a limited number of cases.
I have 2 pending series on top of this one which would have to remove such optimisations.
At the end this is definitely not something supposed to call often (linux driver is calling it once during init).
Quoted 3 lines
I think it was a mistake for me to asked whether this is called often or not. When it comes to security, what matter is whether a malicious guest could indirectly call
ffa_get_vm_count()and delay any work on the pCPU (Xen is not preemptible).We don't have to resolve this now. But this will need to be addressed before we can we consider FFA security supported. So we should at least add it in the list of issue at the top of the file.
No this is really useful and I will fix it and also the partition info part because:
- all information is read only and known when the VM is created (VM ID, max vcpus and 64bit or not)
- I do not need anything else than that to build the result
- If we can prevent to take a lock this will make the code better.
So I will:
- add an atomic to count the number of VMs with FF-A
- create a chain list of VM FF-A contexts
- add a context to the chain when a version is negociated
- put the infos i need in the ffa ctx structure
- just go through the list to build the partinfo result
Quoted 25 lines
Quoted 25 lines
Quoted 7 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 139
Quoted 5 lines
xen/arch/arm/tee/ffa_partinfo.c +3
-0+{ +struct domain *d = current->domain; +struct domain *dom;NIT: "d" and "dom" are a bit too close. Could we rename "d" with "
curr_d"?i will go with
curr_ddest_dto make this clearer.Quoted 7 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 142
Quoted 4 lines
xen/arch/arm/tee/ffa_partinfo.c +3
-0+uint32_t vm_count =0 ; + +/* Count the number of VM with FF-A support */ This comment implies this is including the current VM. But ...
Quoted 8 lines
xen/arch/arm/tee/ffa_partinfo.c static int32_t ffa_partition_info_get(uint32_t *uuid, uint32_t flags, · line 148
Quoted 6 lines
xen/arch/arm/tee/ffa_partinfo.c +6
-0+rcu_read_lock (&domlist_read_lock); +for_each_domain ( dom ) + { +struct ffa_ctx *vm = dom->arch.tee; + +if ( dom != d && vm !=NULL && vm->guest_vers !=0 )... here you are excluding it. Also, it sounds like this is support by the OS rather than the VM itself. Is that correct?
I have a comment to explain that one in a different serie that i will put here.
Quoted 1 line
Basically before 1.2, the spec was a bit blurry on if or not the result should include the
Quoted 8 lines
Quoted 2 lines
calling VM and in fact Linux driver (before 1.2) was ending with an error if its own data was included in the result hence this filter.
Thanks for the clarification. Just to clarify...
Quoted 1 line
I will add a comment for that.
... will the comment be added in this patch?
Yes i will add the comment in the next version of the patch.
Quoted 11 lines
Quoted 9 lines
Quoted 7 lines
Review
Quoted 5 lines
(quoted diff) +3
-0+ vm_count++;> + } + rcu_read_unlock(&domlist_read_lock); +> + return vm_count;OOI, I guess this value is just used as an hint? Asking because the number of domains can change at any point.
Definitely yes. The data is what it is when called but anything could change after. This is mostly used as hint by callers.
Does this mean we would always return a fixed number? Asking because this would solve nicely the preemption problem in
ffa_get_vm_count().
Well the number is changing when VMs are created or destroyed but as explain earlier i will modify the design to prevent the lock using an atomic and a chain list.
Now the problem of the potential high number of VMs still stand so I will add a todo for that and discuss with others to check if this could be solved in the FF-A spec in the feature.
Thanks a lot for the comments.
I will work on v4 for when you get back :-)
Cheers
Bertrand
PATCH v4 3/5 xen/arm: ffa: Add buffer full notification support
In reply to Bertrand MarquisAdd support to raise a Rx buffer full notification to a VM. This function will be used for indirect message support between VM and is only activated if CONFIG_FFA_VM_TO_VM is selected.
Even if there are 32 framework notifications possible, right now only one is defined so the implementation is simplified to only handle the buffer full notification using a boolean. If other framework notifications have to be supported one day, the design will have to be modified to handle it properly.
---
Changes in v4:
- Add Jens R-b
Changes in v3:
- introduce a vm_pending boolean to track if VM notifications are pending and allow to decorelate pending secure notifications from pending vm ones
- remove ifdef around boolean entries for notifications and make use of IS_ENABLED instead of ifdefs when possible
- Fix notification number signaled to VMs for buffer full to use the proper GUEST_FFA_NOTIF_PEND_INTR_ID instead of the identifier received from the SPMC.
- Move back into this patch ffa_private.h part which was wrongly in the patch for indirect messages between VM
Changes in v2:
- Switch ifdef to IS_ENABLED when possible
--- xen/arch/arm/tee/ffa_notif.c | 36 ++++++++++++++++++++++++++++------ xen/arch/arm/tee/ffa_private.h | 23 +++++++++++++++++++++- 2 files changed, 52 insertions(+), 7 deletions(-)
xen/arch/arm/tee/ffa_notif.c +30 -6
@@ -69,6 +69,7 @@ void ffa_handle_notification_info_get(struct cpu_user_regs *regs) {struct domain *d = current->domain;struct ffa_ctx *ctx = d->arch.tee; +bool notif_pending;if ( !notif_enabled ) {
@@ -76,7 +77,11 @@ void ffa_handle_notification_info_get(struct cpu_user_regs *regs)return ; } -if (test_and_clear_bool (ctx->notif.secure_pending) ) + notif_pending =test_and_clear_bool (ctx->notif.secure_pending); +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + notif_pending |=test_and_clear_bool (ctx->notif.vm_pending); + +if ( notif_pending ) {/* A pending global notification for the guest */ ffa_set_regs(regs, FFA_SUCCESS_64, 0,
@@ -93,6 +98,7 @@ void ffa_handle_notification_info_get(struct cpu_user_regs *regs)void ffa_handle_notification_get (struct cpu_user_regs *regs) {struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee;uint32_t recv =get_user_reg (regs,1 );uint32_t flags =get_user_reg (regs,2 );uint32_t w2 =0 ;
@@ -132,11 +138,7 @@ void ffa_handle_notification_get(struct cpu_user_regs *regs)*/ if ( ( flags &FFA_NOTIF_FLAG_BITMAP_SP ) && ( flags &FFA_NOTIF_FLAG_BITMAP_SPM ) ) - { -struct ffa_ctx *ctx = d->arch.tee; - -ACCESS_ONCE (ctx->notif.secure_pending) = false; - } +ACCESS_ONCE (ctx->notif.secure_pending) = false;arm_smccc_1_2_smc (&arg, &resp); e =ffa_get_ret_code (&resp);
@@ -156,6 +158,14 @@ void ffa_handle_notification_get(struct cpu_user_regs *regs) w6 = resp.a6; } +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) && + flags &FFA_NOTIF_FLAG_BITMAP_HYP && +test_and_clear_bool (ctx->notif.buff_full_pending) ) + { +ACCESS_ONCE (ctx->notif.vm_pending) = false; + w7 =FFA_NOTIF_RX_BUFFER_FULL ; + } +ffa_set_regs (regs,FFA_SUCCESS_32 ,0 , w2, w3, w4, w5, w6, w7); }
@@ -178,6 +188,20 @@ int ffa_handle_notification_set(struct cpu_user_regs *regs) bitmap_hi); } +#ifdef CONFIG_FFA_VM_TO_VM +void ffa_raise_rx_buffer_full (struct domain *d) +{ +struct ffa_ctx *ctx = d->arch.tee; + +if ( !ctx ) +return ; + +ACCESS_ONCE (ctx->notif.buff_full_pending) = true; +if ( !test_and_set_bool (ctx->notif.vm_pending) ) +vgic_inject_irq (d, d->vcpu[0 ],GUEST_FFA_NOTIF_PEND_INTR_ID , true); +} +#endif +/* * Extract a 16-bit ID (index n) from the successful return value from * FFA_NOTIFICATION_INFO_GET_64 or FFA_NOTIFICATION_INFO_GET_32. IDs are
xen/arch/arm/tee/ffa_private.h +22 -1
@@ -210,6 +210,8 @@#define FFA_NOTIF_INFO_GET_ID_COUNT_SHIFT 7#define FFA_NOTIF_INFO_GET_ID_COUNT_MASK 0x1F +#define FFA_NOTIF_RX_BUFFER_FULL BIT(0, U) +/* Feature IDs used with FFA_FEATURES */ #define FFA_FEATURE_NOTIF_PEND_INTR 0x1U#define FFA_FEATURE_SCHEDULE_RECV_INTR 0x2U
@@ -295,9 +297,20 @@ struct ffa_mem_region {struct ffa_ctx_notif {/* * True if domain is reported by FFA_NOTIFICATION_INFO_GET to have -* pending global notifications. +* pending notifications from the secure world. */ bool secure_pending; + +/* +* True if domain is reported by FFA_NOTIFICATION_INFO_GET to have +* pending notifications from VMs (including framework ones). +*/ +bool vm_pending; + +/* +* True if domain has buffer full notification pending +*/ +bool buff_full_pending; };struct ffa_ctx {
@@ -369,6 +382,14 @@ void ffa_handle_notification_info_get(struct cpu_user_regs *regs);void ffa_handle_notification_get (struct cpu_user_regs *regs);int ffa_handle_notification_set (struct cpu_user_regs *regs); +#ifdef CONFIG_FFA_VM_TO_VM +void ffa_raise_rx_buffer_full (struct domain *d); +#else +static inline void ffa_raise_rx_buffer_full (struct domain *d) +{ +} +#endif +void ffa_handle_msg_send_direct_req (struct cpu_user_regs *regs,uint32_t fid);int32_t ffa_handle_msg_send2 (struct cpu_user_regs *regs);
Signature · 1 line
2.47.1
PATCH v4 4/5 xen/arm: ffa: Add indirect message between VM
In reply to Bertrand MarquisAdd support for indirect messages between VMs. This is only enabled if CONFIG_FFA_VM_TO_VM is selected.
---
Changes in v4:
- Use a local copy of the message header to prevent a TOC/TOU possible issue when using the payload size
Changes in v3:
- Move vm to vm indirect message handling in a sub function to simplify lock handling and make implementation easier to read
Changes in v2:
- Switch ifdef to IS_ENABLED
--- xen/arch/arm/tee/ffa_msg.c | 110 ++++++++++++++++++++++++++++++++----- 1 file changed, 96 insertions(+), 14 deletions(-)
xen/arch/arm/tee/ffa_msg.c +96 -14
@@ -88,43 +88,125 @@ out: resp.a7 & mask); } +static int32_t ffa_msg_send2_vm (uint16_t dst_id, +struct ffa_part_msg_rxtx *src_msg) +{ +struct domain *dst_d; +struct ffa_ctx *dst_ctx; +struct ffa_part_msg_rxtx *dst_msg; +int err; +int32_t ret; + +if ( dst_id ==0 ) +/* FF-A ID 0 is the hypervisor, this is not valid */ +return FFA_RET_INVALID_PARAMETERS ; + +/* This is also checking that dest is not src */ + err =rcu_lock_live_remote_domain_by_id (dst_id -1 , &dst_d); +if ( err ) +return FFA_RET_INVALID_PARAMETERS ; + +if ( dst_d->arch.tee ==NULL ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out_unlock; + } + + dst_ctx = dst_d->arch.tee; +if ( !dst_ctx->guest_vers ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out_unlock; + } + +/* we need to have enough space in the destination buffer */ +if ( dst_ctx->page_count *FFA_PAGE_SIZE < + (sizeof (struct ffa_part_msg_rxtx ) + src_msg->msg_size) ) + { + ret =FFA_RET_NO_MEMORY ; + goto out_unlock; + } + +/* This also checks that destination has set a Rx buffer */ + ret =ffa_rx_acquire (dst_d); +if ( ret ) + goto out_unlock; + + dst_msg = dst_ctx->rx; + +/* prepare destination header */ + dst_msg->flags =0 ; + dst_msg->reserved =0 ; + dst_msg->msg_offset =sizeof (struct ffa_part_msg_rxtx ); + dst_msg->send_recv_id = src_msg->send_recv_id; + dst_msg->msg_size = src_msg->msg_size; + +memcpy (dst_ctx->rx +sizeof (struct ffa_part_msg_rxtx ), + ((void *)src_msg) + src_msg->msg_offset, src_msg->msg_size); + +/* receiver rx buffer will be released by the receiver*/ + +out_unlock: +rcu_unlock_domain (dst_d); +if ( !ret ) +ffa_raise_rx_buffer_full (dst_d); + +return ret; +} +int32_t ffa_handle_msg_send2 (struct cpu_user_regs *regs) {struct domain *src_d = current->domain;struct ffa_ctx *src_ctx = src_d->arch.tee; -const struct ffa_part_msg_rxtx *src_msg; +struct ffa_part_msg_rxtx src_msg;uint16_t dst_id, src_id;int32_t ret; -if ( !ffa_fw_supports_fid (FFA_MSG_SEND2 ) ) -return FFA_RET_NOT_SUPPORTED ; -if ( !spin_trylock (&src_ctx->tx_lock) )return FFA_RET_BUSY ; - src_msg = src_ctx->tx; - src_id = src_msg->send_recv_id >>16 ; - dst_id = src_msg->send_recv_id &GENMASK (15 ,0 ); +/* create a copy of the message header */ +memcpy (&src_msg, src_ctx->tx,sizeof (src_msg)); + + src_id = src_msg.send_recv_id >>16 ; + dst_id = src_msg.send_recv_id &GENMASK (15 ,0 ); -if ( src_id !=ffa_get_vm_id (src_d) || !FFA_ID_IS_SECURE (dst_id) ) +if ( src_id !=ffa_get_vm_id (src_d) ) { ret =FFA_RET_INVALID_PARAMETERS ; - goto out_unlock_tx; + goto out; }/* check source message fits in buffer */ if ( src_ctx->page_count *FFA_PAGE_SIZE < - src_msg->msg_offset + src_msg->msg_size || - src_msg->msg_offset <sizeof (struct ffa_part_msg_rxtx ) ) + src_msg.msg_offset + src_msg.msg_size || + src_msg.msg_offset <sizeof (struct ffa_part_msg_rxtx ) ) { ret =FFA_RET_INVALID_PARAMETERS ; - goto out_unlock_tx; + goto out; } - ret =ffa_simple_call (FFA_MSG_SEND2 , +if (FFA_ID_IS_SECURE (dst_id) ) + { +/* Message for a secure partition */ +if ( !ffa_fw_supports_fid (FFA_MSG_SEND2 ) ) + { + ret =FFA_RET_NOT_SUPPORTED ; + goto out; + } + + ret =ffa_simple_call (FFA_MSG_SEND2 , ((uint32_t )ffa_get_vm_id (src_d)) <<16 ,0 ,0 ,0 ); + } +else if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + { +/* Message for a VM */ + ret =ffa_msg_send2_vm (dst_id, &src_msg); + } +else + ret =FFA_RET_INVALID_PARAMETERS ; -out_unlock_tx: +out:spin_unlock (&src_ctx->tx_lock);return ret; }
Signature · 1 line
2.47.1
Re: [PATCH v4 4/5] xen/arm: ffa: Add indirect message between VM
Hi Bertrand,
Quoted 14 lines · Bertrand Marquis
Add support for indirect messages between VMs. This is only enabled if
CONFIG_FFA_VM_TO_VMis selected.Signed-off-by Bertrand Marquis <[email protected]>---
Changes in v4:
- Use a local copy of the message header to prevent a TOC/TOU possible issue when using the payload size
Changes in v3:
- Move vm to vm indirect message handling in a sub function to simplify lock handling and make implementation easier to read
Changes in v2:
- Switch ifdef toIS_ENABLED
Quoted 3 lines
--- xen/arch/arm/tee/ffa_msg.c | 110 ++++++++++++++++++++++++++++++++----- 1 file changed, 96 insertions(+), 14 deletions(-)
Cheers,
Jens
Quoted 148 lines
xen/arch/arm/tee/ffa_msg.c +96
-15@@ -88,43 +88,125 @@ out: resp.a7 & mask); } +static int32_t ffa_msg_send2_vm (uint16_t dst_id, +struct ffa_part_msg_rxtx *src_msg) +{ +struct domain *dst_d; +struct ffa_ctx *dst_ctx; +struct ffa_part_msg_rxtx *dst_msg; +int err; +int32_t ret; + +if ( dst_id ==0 ) +/* FF-A ID 0 is the hypervisor, this is not valid */ +return FFA_RET_INVALID_PARAMETERS ; + +/* This is also checking that dest is not src */ + err =rcu_lock_live_remote_domain_by_id (dst_id -1 , &dst_d); +if ( err ) +return FFA_RET_INVALID_PARAMETERS ; + +if ( dst_d->arch.tee ==NULL ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out_unlock; + } + + dst_ctx = dst_d->arch.tee; +if ( !dst_ctx->guest_vers ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out_unlock; + } + +/* we need to have enough space in the destination buffer */ +if ( dst_ctx->page_count *FFA_PAGE_SIZE < + (sizeof (struct ffa_part_msg_rxtx ) + src_msg->msg_size) ) + { + ret =FFA_RET_NO_MEMORY ; + goto out_unlock; + } + +/* This also checks that destination has set a Rx buffer */ + ret =ffa_rx_acquire (dst_d); +if ( ret ) + goto out_unlock; + + dst_msg = dst_ctx->rx; + +/* prepare destination header */ + dst_msg->flags =0 ; + dst_msg->reserved =0 ; + dst_msg->msg_offset =sizeof (struct ffa_part_msg_rxtx ); + dst_msg->send_recv_id = src_msg->send_recv_id; + dst_msg->msg_size = src_msg->msg_size; + +memcpy (dst_ctx->rx +sizeof (struct ffa_part_msg_rxtx ), + ((void *)src_msg) + src_msg->msg_offset, src_msg->msg_size); + +/* receiver rx buffer will be released by the receiver*/ + +out_unlock: +rcu_unlock_domain (dst_d); +if ( !ret ) +ffa_raise_rx_buffer_full (dst_d); + +return ret; +} +int32_t ffa_handle_msg_send2 (struct cpu_user_regs *regs) {struct domain *src_d = current->domain;struct ffa_ctx *src_ctx = src_d->arch.tee; -const struct ffa_part_msg_rxtx *src_msg; +struct ffa_part_msg_rxtx src_msg;uint16_t dst_id, src_id;int32_t ret; -if ( !ffa_fw_supports_fid (FFA_MSG_SEND2 ) ) -return FFA_RET_NOT_SUPPORTED ; -if ( !spin_trylock (&src_ctx->tx_lock) )return FFA_RET_BUSY ; - src_msg = src_ctx->tx; - src_id = src_msg->send_recv_id >>16 ; - dst_id = src_msg->send_recv_id &GENMASK (15 ,0 ); +/* create a copy of the message header */ +memcpy (&src_msg, src_ctx->tx,sizeof (src_msg)); + + src_id = src_msg.send_recv_id >>16 ; + dst_id = src_msg.send_recv_id &GENMASK (15 ,0 ); -if ( src_id !=ffa_get_vm_id (src_d) || !FFA_ID_IS_SECURE (dst_id) ) +if ( src_id !=ffa_get_vm_id (src_d) ) { ret =FFA_RET_INVALID_PARAMETERS ; - goto out_unlock_tx; + goto out; }/* check source message fits in buffer */ if ( src_ctx->page_count *FFA_PAGE_SIZE < - src_msg->msg_offset + src_msg->msg_size || - src_msg->msg_offset <sizeof (struct ffa_part_msg_rxtx ) ) + src_msg.msg_offset + src_msg.msg_size || + src_msg.msg_offset <sizeof (struct ffa_part_msg_rxtx ) ) { ret =FFA_RET_INVALID_PARAMETERS ; - goto out_unlock_tx; + goto out; } - ret =ffa_simple_call (FFA_MSG_SEND2 , +if (FFA_ID_IS_SECURE (dst_id) ) + { +/* Message for a secure partition */ +if ( !ffa_fw_supports_fid (FFA_MSG_SEND2 ) ) + { + ret =FFA_RET_NOT_SUPPORTED ; + goto out; + } + + ret =ffa_simple_call (FFA_MSG_SEND2 , ((uint32_t )ffa_get_vm_id (src_d)) <<16 ,0 ,0 ,0 ); + } +else if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + { +/* Message for a VM */ + ret =ffa_msg_send2_vm (dst_id, &src_msg); + } +else + ret =FFA_RET_INVALID_PARAMETERS ; -out_unlock_tx: +out:spin_unlock (&src_ctx->tx_lock);return ret; } --2.47.1
PATCH v4 5/5 xen/arm: ffa: Enable VM to VM without firmware
In reply to Bertrand MarquisWhen VM to VM support is activated and there is no suitable FF-A support in the firmware, enable FF-A support for VMs to allow using it for VM to VM communications.
If there is OP-TEE running in the secure world and using the non FF-A communication system, having CONFIG_FFA_VM_TO_VM could be non functional (if optee is probed first) or OP-TEE could be non functional (if FF-A is probed first) so it is not recommended to activate the configuration option for such systems.
To make buffer full notification work between VMs when there is no firmware, rework the notification handling and modify the global flag to only be used as check for firmware notification support instead.
---
Changes in v4:
- Fix Optee to OP-TEE in commit message
- Add Jens R-b
Changes in v3:
- fix typos in commit message
- add spaces around <<
- move notification id fix back into buffer full patch
- fix | position in if
Changes in v2:
- replace ifdef with IS_ENABLED when possible
--- xen/arch/arm/tee/ffa.c | 12 +++- xen/arch/arm/tee/ffa_notif.c | 104 ++++++++++++++++------------------- 2 files changed, 59 insertions(+), 57 deletions(-)
xen/arch/arm/tee/ffa.c +11 -1
@@ -324,8 +324,9 @@ static int ffa_domain_init(struct domain *d)struct ffa_ctx *ctx;int ret; -if ( !ffa_fw_version ) +if ( !IS_ENABLED (CONFIG_FFA_VM_TO_VM ) && !ffa_fw_version )return -ENODEV ; +/* * We are using the domain_id + 1 as the FF-A ID for VMs as FF-A ID 0 is * reserved for the hypervisor and we only support secure endpoints using
@@ -561,6 +562,15 @@ err_no_fw:bitmap_zero (ffa_fw_abi_supported,FFA_ABI_BITMAP_SIZE );printk (XENLOG_WARNING "ARM FF-A No firmware support\n" ); +if (IS_ENABLED (CONFIG_FFA_VM_TO_VM ) ) + { +INIT_LIST_HEAD (&ffa_teardown_head); +init_timer (&ffa_teardown_timer, ffa_teardown_timer_callback,NULL ,0 ); + +printk (XENLOG_INFO "ARM FF-A only available between VMs\n" ); +return true; + } +return false; }
xen/arch/arm/tee/ffa_notif.c +48 -56
184 lines not on this page · raw message
Signature · 1 line
2.47.1
Re: [PATCH v4 5/5] xen/arm: ffa: Enable VM to VM without firmware
Hi Bertrand,
Quoted 3 lines · Bertrand Marquis
When VM to VM support is activated and there is no suitable FF-A support in the firmware, enable FF-A support for VMs to allow using it for VM to VM communications.
tee/ and the callbacks associated are meant to be used for mediatiors. My current interpretation ist this is only meant to interpose between a guest and physical resources. Here you are extending the meaning to "virtual TEE". I am sort of ok with that but ...
Quoted 5 lines
If there is OP-TEE running in the secure world and using the non FF-A communication system, having
CONFIG_FFA_VM_TO_VMcould be non functional (if optee is probed first) or OP-TEE could be non functional (if FF-A is probed first) so it is not recommended to activate the configuration option for such systems.
... this part is concerning me. You should be able to build with CONFIG_FFA_VM_TO_VM and still boot when OP-TEE is present on the system. This is not too critical now as this is tech preview but this is definitely a blocker for making FFA supported. Can this be mentioned at the top of the ffa.c file (which already contains existing blocker)?
Also, given this would expose a fully virtual TEE, we should be able to have a system where you have some VMs talking FFA and some using the physical OPTEE (or another TEE). Whether we want to support it is a different question but this design would prevent it. Is this intended?
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v4 5/5] xen/arm: ffa: Enable VM to VM without firmware
Quoted 9 lines · Julien Grall
Hi Bertrand,
Quoted 3 lines · Bertrand Marquis
When VM to VM support is activated and there is no suitable FF-A support in the firmware, enable FF-A support for VMs to allow using it for VM to VM communications.
tee/ and the callbacks associated are meant to be used for mediatiors. My current interpretation ist this is only meant to interpose between a guest and physical resources. Here you are extending the meaning to "virtual TEE". I am sort of ok with that but ...
Quoted 7 lines
Quoted 5 lines
If there is OP-TEE running in the secure world and using the non FF-A communication system, having
CONFIG_FFA_VM_TO_VMcould be non functional (if optee is probed first) or OP-TEE could be non functional (if FF-A is probed first) so it is not recommended to activate the configuration option for such systems.... this part is concerning me. You should be able to build with
CONFIG_FFA_VM_TO_VMand still boot when OP-TEE is present on the system. This is not too critical now as this is tech preview but this is definitely a blocker for making FFA supported. Can this be mentioned at the top of theffa.cfile (which already contains existing blocker)?
Just to clarify, I meant without having the OP-TEE folks to force the TEE on the command line because this is a bit of a hack...
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v4 5/5] xen/arm: ffa: Enable VM to VM without firmware
In reply to Julien GrallHi Julien,
Quoted 10 lines · Julien Grall
Hi Bertrand,
Quoted 3 lines · Bertrand Marquis
When VM to VM support is activated and there is no suitable FF-A support in the firmware, enable FF-A support for VMs to allow using it for VM to VM communications.
tee/ and the callbacks associated are meant to be used for mediatiors. My current interpretation ist this is only meant to interpose between a guest and physical resources. Here you are extending the meaning to "virtual TEE". I am sort of ok with that but ...
I see what you mean but FF-A will not only be used to communicate with TEE (even if the primary use case right now is this one, including have it in a VM instead of the secure world).
Quoted 8 lines
Quoted 6 lines
If there is OP-TEE running in the secure world and using the non FF-A communication system, having
CONFIG_FFA_VM_TO_VMcould be non functional (if optee is probed first) or OP-TEE could be non functional (if FF-A is probed first) so it is not recommended to activate the configuration option for such systems.... this part is concerning me. You should be able to build with
CONFIG_FFA_VM_TO_VMand still boot when OP-TEE is present on the system. This is not too critical now as this is tech preview but this is definitely a blocker for making FFA supported. Can this be mentioned at the top of theffa.cfile (which already contains existing blocker)?
OP-TEE supports FF-A and in fact should be switched to using it by default as it allows it to run in parallel of other TEEs in the secure world or have FF-A compliant SPs running on top of OP-TEE. More and more you will see FF-A popping up as a recommended (or required) part of the firmware features.
So the only reason to use OP-TEE without FF-A is if you have an old OP-TEE in which case your firmware will not support FF-A and using it between VMs is probably not required.
Quoted 2 lines
Also, given this would expose a fully virtual TEE, we should be able to have a system where you have some VMs talking FFA and some using the physical OPTEE (or another TEE). Whether we want to support it is a different question but this design would prevent it. Is this intended?
Right now i would say this is definitely not something we need as part of the tech preview as anybody using this feature in Xen would use an OP-TEE with FF-A support.
But from Xen point of view, we should (if we can) support running on old systems with an existing OP-TEE but still use FF-A between VMs.
This has some consequences on how the tee mediator and FF-A support is designed and I will definitely give it some thoughts (primary idea would be to decouple FF-A with secure as mediator to FF-A between VMs somehow).
For the review side of things, am I right to understand from your comments that this ok for now as tech-preview ?
Cheers
Bertrand
Quoted 6 lines
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v4 5/5] xen/arm: ffa: Enable VM to VM without firmware
Hi Bertrand,
Quoted 19 lines · Bertrand Marquis
Quoted 10 lines · Julien Grall
Hi Bertrand,
Quoted 3 lines · Bertrand Marquis
When VM to VM support is activated and there is no suitable FF-A support in the firmware, enable FF-A support for VMs to allow using it for VM to VM communications.
tee/ and the callbacks associated are meant to be used for mediatiors. My current interpretation ist this is only meant to interpose between a guest and physical resources. Here you are extending the meaning to "virtual TEE". I am sort of ok with that but ...
I see what you mean but FF-A will not only be used to communicate with TEE (even if the primary use case right now is this one, including have it in a VM instead of the secure world).
Quoted 7 lines
Quoted 6 lines
If there is OP-TEE running in the secure world and using the non FF-A communication system, having
CONFIG_FFA_VM_TO_VMcould be non functional (if optee is probed first) or OP-TEE could be non functional (if FF-A is probed first) so it is not recommended to activate the configuration option for such systems.
Quoted 6 lines
Quoted 1 line
... this part is concerning me. You should be able to build with
CONFIG_FFA_VM_TO_VMand still boot when OP-TEE is present on the system. This is not too critical now as this is tech preview but this is definitely a blocker for making FFA supported. Can this be mentioned at the top of theffa.cfile (which already contains existing blocker)?OP-TEE supports FF-A and in fact should be switched to using it by default as it allows it to run in parallel of other TEEs in the secure world or have FF-A compliant SPs running on top of OP-TEE. More and more you will see FF-A popping up as a recommended (or required) part of the firmware features.
So the only reason to use OP-TEE without FF-A is if you have an old OP-TEE in which case your firmware will not support FF-A and using it between VMs is probably not required.
Thanks for the clarification. I know we only support OP-TEE in Xen today, but do you know what will happen for the other TEEs? Will they adopt FF-A?
Quoted 7 lines
Quoted 3 lines
Also, given this would expose a fully virtual TEE, we should be able to have a system where you have some VMs talking FFA and some using the physical OPTEE (or another TEE). Whether we want to support it is a different question but this design would prevent it. Is this intended?
Right now i would say this is definitely not something we need as part of the tech preview as anybody using this feature in Xen would use an OP-TEE with FF-A support.
But from Xen point of view, we should (if we can) support running on old systems with an existing OP-TEE but still use FF-A between VMs.
This has some consequences on how the tee mediator and FF-A support is designed and I will definitely give it some thoughts (primary idea would be to decouple FF-A with secure as mediator to FF-A between VMs somehow).
I am not sure we need to decouple anything. Today, we can already specify the type of the TEE used by a given VM (see tee_type). But we are enforcing the TEE type match the one of the current mediator.
So what if we allow multiple mediator to run and when the domain is initialized we select the correct medatior/ops for the VM?
For simplification, we could even hardocode FF-A as the second mediator.
Quoted 2 lines
For the review side of things, am I right to understand from your comments that this ok for now as tech-preview ?
Yes I am happy with the approach for now.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v4 5/5] xen/arm: ffa: Enable VM to VM without firmware
Hi Julien,
Quoted 23 lines · Julien Grall
Hi Bertrand,
Quoted 18 lines · Bertrand Marquis
Quoted 10 lines · Julien Grall
Hi Bertrand,
Quoted 3 lines · Bertrand Marquis
When VM to VM support is activated and there is no suitable FF-A support in the firmware, enable FF-A support for VMs to allow using it for VM to VM communications.
tee/ and the callbacks associated are meant to be used for mediatiors. My current interpretation ist this is only meant to interpose between a guest and physical resources. Here you are extending the meaning to "virtual TEE". I am sort of ok with that but ...
I see what you mean but FF-A will not only be used to communicate with TEE (even if the primary use case right now is this one, including have it in a VM instead of the secure world).
Quoted 7 lines
Quoted 6 lines
If there is OP-TEE running in the secure world and using the non FF-A communication system, having
CONFIG_FFA_VM_TO_VMcould be non functional (if optee is probed first) or OP-TEE could be non functional (if FF-A is probed first) so it is not recommended to activate the configuration option for such systems.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
... this part is concerning me. You should be able to build with
CONFIG_FFA_VM_TO_VMand still boot when OP-TEE is present on the system. This is not too critical now as this is tech preview but this is definitely a blocker for making FFA supported. Can this be mentioned at the top of theffa.cfile (which already contains existing blocker)?OP-TEE supports FF-A and in fact should be switched to using it by default as it allows it to run in parallel of other TEEs in the secure world or have FF-A compliant SPs running on top of OP-TEE. More and more you will see FF-A popping up as a recommended (or required) part of the firmware features.
So the only reason to use OP-TEE without FF-A is if you have an old OP-TEE in which case your firmware will not support FF-A and using it between VMs is probably not required.Thanks for the clarification. I know we only support OP-TEE in Xen today, but do you know what will happen for the other TEEs? Will they adopt FF-A?
On Arm the idea is to make them adopt FF-A yes and it will be part of System Ready recommendations in the future.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Also, given this would expose a fully virtual TEE, we should be able to have a system where you have some VMs talking FFA and some using the physical OPTEE (or another TEE). Whether we want to support it is a different question but this design would prevent it. Is this intended?
Right now i would say this is definitely not something we need as part of the tech preview as anybody using this feature in Xen would use an OP-TEE with FF-A support.
But from Xen point of view, we should (if we can) support running on old systems with an existing OP-TEE but still use FF-A between VMs.
This has some consequences on how the tee mediator and FF-A support is designed and I will definitely give it some thoughts (primary idea would be to decouple FF-A with secure as mediator to FF-A between VMs somehow).I am not sure we need to decouple anything. Today, we can already specify the type of the TEE used by a given VM (see
tee_type). But we are enforcing the TEE type match the one of the current mediator.
Yes for VMs this has to be specified explicitly, this was the idea behind the command line parameter for Xen to.
Quoted 2 lines
So what if we allow multiple mediator to run and when the domain is initialized we select the correct medatior/ops for the VM?
Right now a VM gets the mediator selected by configuration if it is available.
I do not think we should make it automatic as there might be good reasons to not allow to access a TEE from some VMs.
Quoted 2 lines
For simplification, we could even hardocode FF-A as the second mediator.
That could be a long term solution yes but we definitely need to solve the access rights first. As long as VMs can use FF-A to communicate with any other VMs or TEEs, the current approach is the most secure one.
Quoted 4 lines
Quoted 2 lines
For the review side of things, am I right to understand from your comments that this ok for now as tech-preview ?
Yes I am happy with the approach for now.
Great thanks.
Cheers
Bertrand
Quoted 6 lines
Cheers,
Signature · 1 line
Julien Grall