Xen FF-A mediator
v11 cover Xen FF-A mediator
Hi,
[Parts of the patchset has been merged already, see the changelog below for details.]
This patch set adds an FF-A [1] mediator to the TEE mediator framework already present in Xen. The FF-A mediator implements the subset of the FF-A 1.1 specification needed to communicate with OP-TEE using FF-A as transport mechanism instead of SMC/HVC as with the TEE mediator. It allows a similar design in OP-TEE as with the TEE mediator where OP-TEE presents one virtual partition of itself to each guest in Xen.
The FF-A mediator is generic in the sense it has nothing OP-TEE specific except that only the subset needed for OP-TEE is implemented so far. The hooks needed to inform OP-TEE that a guest is created or destroyed are part of the FF-A specification.
It should be possible to extend the FF-A mediator to implement a larger portion of the FF-A 1.1 specification without breaking with the way OP-TEE is communicated with here. So it should be possible to support any TEE or Secure Partition using FF-A as transport with this mediator.
The patches are also available at https://github.com/jenswi-linaro/xen
branch "xen_ffa_v11".
With help from Bertrand I've integrated this in a test setup with OP-TEE. Please check prerequisites at
https://optee.readthedocs.io/en/latest/building/prerequisites.html
My setup is duplicated using:
repo init -u https://github.com/jenswi-linaro/manifest.git -m qemu_v8.xml \
-b qemu_xen_ffa
repo sync -j8
cd build
make -j8 toolchains
make -j8 all
make run-only
Test in dom0 with for instance:
xtest 1004
at the prompt.
To start up a domu and connect to it do:
cd /mnt/host/build/qemu_v8/xen
xl create guest_ffa.cfg
xl console domu
Then test as usual with "xtest 1004".
The setup uses the branch "ffa" from https://github.com/jenswi-linaro/xen.
That's currently the same as the "xen_ffa_v11" branch, but the "ffa" branch may change later as I update for a new version of the patch set.
[1] https://developer.arm.com/documentation/den0077/latest
Thanks,
Jens
v10->v11:
- Rebased on staging and dropping the following patches that already have been merged:
- xen: Introduce arch_domain_teardown()
- xen/arm: add TEE teardown to arch_domain_teardown()
- xen/arm: smccc: add support for SMCCCv1.2 extended input/output registers
- xen/arm: tee: add a primitive FF-A mediator
- xen/arm: ffa: add remaining SMC function IDs
- xen/arm: ffa: add flags for FFA_PARTITION_INFO_GET
- xen/arm: ffa: add defines for framework direct request/response messages
- xen/arm: ffa: enforce dependency on 4k pages
- xen/arm: ffa: add support for FFA_ID_GET
- Dropping "xen/arm: ffa: support sharing large memory ranges" since it's quite complicated and FF-A still works for OP-TEE
- Changelog in each of the patches
v9->v10:
Changelog in the the patches for two changed patches:
- "xen: Introduce arch_domain_teardown()"
- "xen/arm: add TEE teardown to arch_domain_teardown()"
v8->v9:
* Patch "xen/arm: ffa: add remaining SMC function IDs"
- mention the documentation number used
- Adding Reviewed-by: Henry Wang <[email protected]>
* Patch "xen/arm: ffa: add flags for FFA_PARTITION_INFO_GET"
- Adding a reference to the documentation
- Renaming FFA_PART_PROP_IS_MASK to FFA_PART_PROP_IS_TYPE_MASK
- Adding Reviewed-by: Henry Wang <[email protected]>
* Patch "xen/arm: ffa: add defines for framework direct request/response messages"
- add a reference to the documentation
- Adding Reviewed-by: Henry Wang <[email protected]>
* Patch "docs: add Arm FF-A mediator"
- Updating the wording as requested.
- Adding Reviewed-by: Henry Wang <[email protected]>
- Fixing a few typos and adding a reference to the code for a list of limitations
* Patch "xen/arm: ffa: list current limitations"
- Fixing a couple of typos
- Adding Reviewed-by: Henry Wang <[email protected]>
- Removing an invalid limitation of FFA_RXTX_MAP_*
* Patch "xen/arm: ffa: add defines for sharing memory"
- Fixing a few typos
- Adding a reference to the documentation
- Adding Reviewed-by: Henry Wang <[email protected]>
- Updating FFA_MAX_SHM_PAGE_COUNT to have a safe margin to TEEC_CONFIG_SHAREDMEM_MAX_SIZE
* Patch "xen/arm: ffa: add direct request support"
- Fixing a coding style issue
- Adding Reviewed-by: Henry Wang <[email protected]>
- Updating the TODO comment about features.
* Patch "xen/arm: ffa: send guest events to Secure Partitions"
- Adding a TODO comment and a limit on the loop in ffa_direct_req_send_vm()
* Patch "xen/arm: ffa: support mapping guest RX/TX buffers"
- Change FFA_MAX_RXTX_PAGE_COUNT to 1 with a TODO for a larger value
- Replace !p2m_is_ram(t) test with t != p2m_ram_rw
- Rename the introduced field tx_is_free to tx_is_free
* Patch "xen/arm: ffa: support sharing memory"
- Fixing read_atomic() issue in get_shm_pages()
- Fixing style issue in free_ffa_shm_mem()
- Replace !p2m_is_ram(t) test with t != p2m_ram_rw in get_shm_pages()
* Patch "tools: add Arm FF-A mediator"
- Adding Reviewed-by: Henry Wang <[email protected]>
- Adding Acked-by: Anthony PERARD <[email protected]>
- Moving the patch to just before the docs patch.
* Adding Reviewed-by: Henry Wang <[email protected] to:
- "xen/arm: ffa: enforce dependency on 4k pages"
- "xen/arm: move regpair_to_uint64() and uint64_to_regpair() to regs.h"
- "xen/arm: ffa: add ABI structs for sharing memory"
* Adding "xen: Introduce arch_domain_teardown()" by Andrew Cooper
* Adding "xen/arm: add TEE teardown to arch_domain_teardown()"
based on a patch from Andrew Cooper
v7->v8:
* Adding "xen/arm: ffa: list current limitations" as requested
* Adding tags to "xen/arm: smccc: add support for SMCCCv1.2 extended input/output registers"
* Patch "xen/arm: tee: add a primitive FF-A mediator":
- Changing license for ffa.h and ffa.c to GPL-2.0-only
- Avoiding IS_ENABLED() in the constant FFA_NR_FUNCS
- Accepting version 1.1 SPMC only to keep things simple
- Removes 32bit and only supports 64bit to keep things simple
* Patch "tools: add Arm FF-A mediator"
- Adding Reviewed-by: Bertrand Marquis <[email protected]>
- Adding LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE_FFA for the "ffa" value in arch_arm.tee
* Patch "docs: add Arm FF-A mediator"
- Fixing a spell error
- Moving the patch last in the series
* Patch "xen/arm: ffa: add remaining SMC function IDs"
- Adding Reviewed-by: Bertrand Marquis <[email protected]>
- Renaming the define FFA_MSG_RUN to FFA_RUN to match the specification
* Patch "xen/arm: ffa: add flags for FFA_PARTITION_INFO_GET"
- Updating the comment describing the flags for FFA_PARTITION_INFO_GET
* Patch "xen/arm: ffa: add defines for framework direct request/response messages"
- Updating the comment describing the flags for MSG_SEND_DIRECT_REQ/RESP
* Patch "xen/arm: ffa: enforce dependency on 4k pages"
- Updating title of patch
- Adding Reviewed-by: Bertrand Marquis <[email protected]>
* Patch "xen/arm: ffa: add support for FFA_ID_GET"
- In ffa_domain_init(), check that domain_id isn't greater than UINT16_MAX to avoid a future potential integer overflow in get_vm_id()
* Patch "xen/arm: ffa: add direct request support"
- Move preemption (interrupted) parts to a separate patch "xen/arm: ffa:
support preemption of SP during direct request"
- Remove loop in handle_msg_send_direct_req() to return eventual errors back to the VM instead of the SP.
* Patch "xen/arm: ffa: map SPMC rx/tx buffers"
- Adding a FFA_RXTX_PAGE_COUNT define instead of using 1 directly
* New patch "xen/arm: ffa: support preemption of SP during direct request"
* Patch "xen/arm: ffa: send guest events to Secure Partitions"
- Replacing unsigned int with uint16_t for subscr_vm_created_count and subscr_vm_destroyed_count plus the needed range check to see that they don't overflow.
* Patch "xen/arm: ffa: support mapping guest RX/TX buffers"
- Limit the number of pages in VM RX/TX buffers to 32 using a new FFA_MAX_RXTX_PAGE_COUNT define.
* Patch "xen/arm: ffa: support guest FFA_PARTITION_INFO_GET"
- Renaming tx_is_mine to rx_is_free as requested
- Simplified the FFA_PARTITION_INFO_GET_COUNT_FLAG check in handle_partition_info_get()
- Adding a comment on ownership of the RX buffer
- Adding the patch "xen/arm: ffa: improve lock granularity" to address parts of the locking concerns.
* Patch "xen/arm: move regpair_to_uint64() and uint64_to_regpair() to regs.h"
- Adding Reviewed-by: Bertrand Marquis <[email protected]>
* Patch "xen/arm: ffa: add defines for sharing memory"
- Fixing reference for FFA_NORMAL_MEM_REG_ATTR and FFA_MEM_ACC_RW
- Updating descirption for FFA_MAX_SHM_PAGE_COUNT
* Patch "xen/arm: ffa: add ABI structs for sharing memory"
- Changing name of the "global_handle" memeber in struct ffa_mem_transaction_* to "handle".
* Patch "xen/arm: ffa: support sharing memory"
- Use FFA_MEM_SHARE_64 only since we changed to only suporting ARM_64.
- Rename struct ffa_mem_transaction_x to struct ffa_mem_transaction_int as requested.
- Adding a check that shm->page_count isn't 0 before calling share_shm()
- Masking return value from FFA_MEM_FRAG_RX to avoid an implic cast to the int32_t returned by ffa_mem_share().
* Patch "xen/arm: ffa: add support to reclaim shared memory"
- Adding Reviewed-by: Bertrand Marquis <[email protected]>
* Patch "xen/arm: ffa: support sharing large memory ranges"
- Adding commetns for struct ffa_ctx
- Cleaning up and removing the fragmentation state if handle_mem_frag_tx() detects an error.
* Adding "xen/arm: ffa: improve lock granularity" to address some of the locking concerns.
v6->v7:
* Split some of the larger patches into smaller patches for easier review. For instance, the v6 patch "xen/arm: add a primitive FF-A mediator" has been replaced with:
- "xen/arm: add a primitive FF-A mediator"
- "tools: add Arm FF-A mediator"
- "docs: add Arm FF-A mediator"
- "xen/arm: ffa: add remaining SMC function IDs"
* Some small fixes in the error path for handle_mem_share()
* Switched to SPDX for license in new files.
* Fixed comment style issues in
"xen/arm: smccc: add support for SMCCCv1.2 extended input/output registers"
* Made FFA support UNSUPPORTED in "xen/arm: add a primitive FF-A mediator"
* Replaced ffa_get_call_count() with FFA_NR_FUNCS
* Update the FFA_MAX_SHM_PAGE_COUNT with a formula instead of a value.
* Replaced XEN_ARM_FLAGS_FFA with XEN_DOMCTL_CONFIG_TEE_FFA to minimize impact on struct xen_arch_domainconfig. This works because the FF-A mediator and the OP-TEE mediator will not be used at the same time in by a guest.
* Replaced "ffa" boolean in the guest config with a new "ffa" value to the enumeration "tee_type".
* Integrated the FF-A mediator in the TEE mediator framework instead of being its own.
* Rebased on staging as of 2023-02-16
v5->v6:
* Updated "xen/arm: move regpair_to_uint64() and uint64_to_regpair() to regs.h" commit message and moved the patch right before the patch which needs it. Applied Michal Orzel's R-B tag.
* Renamed the guest configuration option "ffa_enabled" to "ffa" and updated the description.
* More tools update in "xen/arm: add a primitive FF-A mediator" with the "ffa" option, including golang and ocaml.
* Update ffa_domain_init() to return an error if communication with the SPMC can't be established.
* Factored out a ffa_domain_destroy() from ffa_relinquish_resources().
* Added ffa_get_call_count() to give an accurate number of FF-A function, updated in each patch as new FF-A functions are added.
* Added a flags field in struct xen_arch_domainconfig that replaces the ffa_enabled field.
* Made check_mandatory_feature() __init
* Replaced a few printk() calls with gprintk() where needed.
* Rebased on staging as of 2022-09-14
V4->v5:
* Added "xen/arm: move regpair_to_uint64() and uint64_to_regpair() to regs.h"
* Added documentation for the "ffa_enabled" guest config flag
* Changed to GPL license for xen/arch/arm/ffa.c
* Added __read_mostly and const where applicable
* Added more describing comments in the code
* Moved list of shared memory object ("ffa_mem_list") into the guest context as they are guest specific
* Simplified a few of the simple wrapper functions for SMC to SPMC
* Added a BUILD_BUG_ON(PAGE_SIZE != FFA_PAGE_SIZE) since the mediator currently depends on the page size to be same as FFA_PAGE_SIZE (4k).
* Added max number of shared memory object per guest and max number of size of each shared memory object
* Added helper macros to calculate offsets of different FF-A data structures in the communication buffer instead of relying on pointer arithmetic
* Addressed style issues and other comments
* Broke the commit "xen/arm: add FF-A mediator" into multiple parts, trying to add a few features at a time as requested
* Added a missing call to rxtx_unmap() in ffa_relinquish_resources()
* Assignment of "ffa_enabled" is kept as is until I have something definitive on the type etc.
* Tested with CONFIG_DEBUG=y
v3->v4:
* Missed v3 and sent a v4 instead by mistake.
v2->v3:
* Generates offsets into struct arm_smccc_1_2_regs with asm-offsets.c in order to avoid hard coded offsets in the assembly function arm_smccc_1_2_smc()
* Adds an entry in SUPPORT.md on the FF-A status
* Adds a configuration variable "ffa_enabled" to tell if FF-A should be enabled for a particular domu guest
* Moves the ffa_frag_list for fragmented memory share requests into struct ffa_ctx instead to keep it per guest in order to avoid mixups and simplify locking
* Adds a spinlock to struct ffa_ctx for per guest locking
* Addressing style issues and suggestions
* Uses FFA_FEATURES to check that all the needed features are available before initializing the mediator
* Rebased on staging as of 2022-06-20
v1->v2:
* Rebased on staging to resolve some merge conflicts as requested
Jens Wiklander (14):
xen/arm: ffa: add direct request support
xen/arm: ffa: map SPMC rx/tx buffers
xen/arm: ffa: send guest events to Secure Partitions xen/arm: ffa: support mapping guest RX/TX buffers xen/arm: ffa: support guest FFA_PARTITION_INFO_GET xen/arm: move regpair_to_uint64() and uint64_to_regpair() to regs.h xen/arm: ffa: add defines for sharing memory
xen/arm: ffa: add ABI structs for sharing memory xen/arm: ffa: support sharing memory
xen/arm: ffa: add support to reclaim shared memory xen/arm: ffa: improve lock granularity
xen/arm: ffa: list current limitations
tools: add Arm FF-A mediator
docs: add Arm FF-A mediator
CHANGELOG.md | 3 +- SUPPORT.md | 9 + docs/man/xl.cfg.5.pod.in | 15 + tools/include/libxl.h | 5 + tools/libs/light/libxl_arm.c | 3 + tools/libs/light/libxl_types.idl | 3 +- xen/arch/arm/include/asm/regs.h | 12 + xen/arch/arm/tee/ffa.c | 1411 ++++++++++++++++++++++++++++++ xen/arch/arm/tee/optee.c | 11 - 9 files changed, 1459 insertions(+), 13 deletions(-)
Signature · 1 line
2.34.1
PATCH v11 1/14 xen/arm: ffa: add direct request support
Adds support for sending a FF-A direct request. Checks that the SP also supports handling a 32-bit direct request. 64-bit direct requests are not used by the mediator itself so there is not need to check for that.
---v10->v11
- In handle_msg_send_direct_req(), replaced the "arg" with "resp" at the
default: label in the switch statement.
--- xen/arch/arm/tee/ffa.c | 113 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+)
xen/arch/arm/tee/ffa.c +113 -0
@@ -181,6 +181,56 @@ static bool ffa_get_version(uint32_t *vers)return true; } +static int32_t get_ffa_ret_code (const struct arm_smccc_1_2_regs *resp) +{ +switch ( resp->a0 ) + { +case FFA_ERROR : +if ( resp->a2 ) +return resp->a2; +else +return FFA_RET_NOT_SUPPORTED ; +case FFA_SUCCESS_32 : +case FFA_SUCCESS_64 : +return FFA_RET_OK ; +default : +return FFA_RET_NOT_SUPPORTED ; + } +} + +static int32_t ffa_simple_call (uint32_t fid,register_t a1,register_t a2, +register_t a3,register_t a4) +{ +const struct arm_smccc_1_2_regs arg = { + .a0 = fid, + .a1 = a1, + .a2 = a2, + .a3 = a3, + .a4 = a4, + }; +struct arm_smccc_1_2_regs resp; + +arm_smccc_1_2_smc (&arg, &resp); + +return get_ffa_ret_code (&resp); +} + +static int32_t ffa_features (uint32_t id) +{ +return ffa_simple_call (FFA_FEATURES , id,0 ,0 ,0 ); +} + +static bool check_mandatory_feature (uint32_t id) +{ +int32_t ret =ffa_features (id); + +if ( ret ) +printk (XENLOG_ERR "ffa: mandatory feature id %#x missing: error %d\n" , + id, ret); + +return !ret; +} +static uint16_t get_vm_id (const struct domain *d) {/* +1 since 0 is reserved for the hypervisor in FF-A */
@@ -222,6 +272,57 @@ static void handle_version(struct cpu_user_regs *regs)set_regs (regs, vers,0 ,0 ,0 ,0 ,0 ,0 ,0 ); } +static void handle_msg_send_direct_req (struct cpu_user_regs *regs,uint32_t fid) +{ +struct arm_smccc_1_2_regs arg = { .a0 = fid, }; +struct arm_smccc_1_2_regs resp = { }; +struct domain *d = current->domain; +uint32_t src_dst; +uint64_t mask; + +if (smccc_is_conv_64 (fid) ) + mask =GENMASK_ULL (63 ,0 ); +else + mask =GENMASK_ULL (31 ,0 ); + + src_dst =get_user_reg (regs,1 ); +if ( (src_dst >>16 ) !=get_vm_id (d) ) + { + resp.a0 =FFA_ERROR ; + resp.a2 =FFA_RET_INVALID_PARAMETERS ; + goto out; + } + + arg.a1 = src_dst; + arg.a2 =get_user_reg (regs,2 ) & mask; + arg.a3 =get_user_reg (regs,3 ) & mask; + arg.a4 =get_user_reg (regs,4 ) & mask; + arg.a5 =get_user_reg (regs,5 ) & mask; + arg.a6 =get_user_reg (regs,6 ) & mask; + arg.a7 =get_user_reg (regs,7 ) & mask; + +arm_smccc_1_2_smc (&arg, &resp); +switch ( resp.a0 ) + { +case FFA_ERROR : +case FFA_SUCCESS_32 : +case FFA_SUCCESS_64 : +case FFA_MSG_SEND_DIRECT_RESP_32 : +case FFA_MSG_SEND_DIRECT_RESP_64 : +break ; +default : +/* Bad fid, report back to the caller. */ +memset (&resp,0 ,sizeof (resp)); + resp.a0 =FFA_ERROR ; + resp.a1 = src_dst; + resp.a2 =FFA_RET_ABORTED ; + } + +out: +set_regs (regs, resp.a0, resp.a1 & mask, resp.a2 & mask, resp.a3 & mask, + resp.a4 & mask, resp.a5 & mask, resp.a6 & mask, resp.a7 & mask); +} +static bool ffa_handle_call (struct cpu_user_regs *regs) {uint32_t fid =get_user_reg (regs,0 );
@@ -239,6 +340,10 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)case FFA_ID_GET :set_regs_success (regs,get_vm_id (d),0 );return true; +case FFA_MSG_SEND_DIRECT_REQ_32 : +case FFA_MSG_SEND_DIRECT_REQ_64 : +handle_msg_send_direct_req (regs, fid); +return true;default :gprintk (XENLOG_ERR ,"ffa: unhandled fid 0x%x\n" , fid);
@@ -331,6 +436,14 @@ static bool ffa_probe(void)printk (XENLOG_INFO "ARM FF-A Firmware version %u.%u\n" , major_vers, minor_vers); +/* +* At the moment domains must support the same features used by Xen. +* TODO: Rework the code to allow domain to use a subset of the +* features supported. +*/ +if ( !check_mandatory_feature (FFA_MSG_SEND_DIRECT_REQ_32 ) ) +return false; + ffa_version = vers;return true;
Signature · 1 line
2.34.1
Re: [XEN PATCH v11 01/14] xen/arm: ffa: add direct request support
Hi Jens,
Quoted 8 lines · Jens Wiklander
Adds support for sending a FF-A direct request. Checks that the SP also supports handling a 32-bit direct request. 64-bit direct requests are not used by the mediator itself so there is not need to check for that.
Signed-off-by Jens Wiklander <[email protected]>Reviewed-by Henry Wang <[email protected]>
Cheers
Bertrand
Quoted 156 lines
---
v10->v11
- Inhandle_msg_send_direct_req(), replaced the "arg" with "resp" at the
default: label in the switch statement.
---xen/arch/arm/tee/ffa.c| 113 +++++++++++++++++++++++++++++++++++++++++1 file changed, 113 insertions(+)xen/arch/arm/tee/ffa.c +113
-0@@ -181,6 +181,56 @@ static bool ffa_get_version(uint32_t *vers)return true; } +static int32_t get_ffa_ret_code (const struct arm_smccc_1_2_regs *resp) +{ +switch ( resp->a0 ) + { +case FFA_ERROR : +if ( resp->a2 ) +return resp->a2; +else +return FFA_RET_NOT_SUPPORTED ; +case FFA_SUCCESS_32 : +case FFA_SUCCESS_64 : +return FFA_RET_OK ; +default : +return FFA_RET_NOT_SUPPORTED ; + } +} + +static int32_t ffa_simple_call (uint32_t fid,register_t a1,register_t a2, +register_t a3,register_t a4) +{ +const struct arm_smccc_1_2_regs arg = { + .a0 = fid, + .a1 = a1, + .a2 = a2, + .a3 = a3, + .a4 = a4, + }; +struct arm_smccc_1_2_regs resp; + +arm_smccc_1_2_smc (&arg, &resp); + +return get_ffa_ret_code (&resp); +} + +static int32_t ffa_features (uint32_t id) +{ +return ffa_simple_call (FFA_FEATURES , id,0 ,0 ,0 ); +} + +static bool check_mandatory_feature (uint32_t id) +{ +int32_t ret =ffa_features (id); + +if ( ret ) +printk (XENLOG_ERR "ffa: mandatory feature id %#x missing: error %d\n" , + id, ret); + +return !ret; +} +static uint16_t get_vm_id (const struct domain *d) {/* +1 since 0 is reserved for the hypervisor in FF-A */ @@ -222,6 +272,57 @@ static void handle_version(struct cpu_user_regs *regs)set_regs (regs, vers,0 ,0 ,0 ,0 ,0 ,0 ,0 ); } +static void handle_msg_send_direct_req (struct cpu_user_regs *regs,uint32_t fid) +{ +struct arm_smccc_1_2_regs arg = { .a0 = fid, }; +struct arm_smccc_1_2_regs resp = { }; +struct domain *d = current->domain; +uint32_t src_dst; +uint64_t mask; + +if (smccc_is_conv_64 (fid) ) + mask =GENMASK_ULL (63 ,0 ); +else + mask =GENMASK_ULL (31 ,0 ); + + src_dst =get_user_reg (regs,1 ); +if ( (src_dst >>16 ) !=get_vm_id (d) ) + { + resp.a0 =FFA_ERROR ; + resp.a2 =FFA_RET_INVALID_PARAMETERS ; + goto out; + } + + arg.a1 = src_dst; + arg.a2 =get_user_reg (regs,2 ) & mask; + arg.a3 =get_user_reg (regs,3 ) & mask; + arg.a4 =get_user_reg (regs,4 ) & mask; + arg.a5 =get_user_reg (regs,5 ) & mask; + arg.a6 =get_user_reg (regs,6 ) & mask; + arg.a7 =get_user_reg (regs,7 ) & mask; + +arm_smccc_1_2_smc (&arg, &resp); +switch ( resp.a0 ) + { +case FFA_ERROR : +case FFA_SUCCESS_32 : +case FFA_SUCCESS_64 : +case FFA_MSG_SEND_DIRECT_RESP_32 : +case FFA_MSG_SEND_DIRECT_RESP_64 : +break ; +default : +/* Bad fid, report back to the caller. */ +memset (&resp,0 ,sizeof (resp)); + resp.a0 =FFA_ERROR ; + resp.a1 = src_dst; + resp.a2 =FFA_RET_ABORTED ; + } + +out: +set_regs (regs, resp.a0, resp.a1 & mask, resp.a2 & mask, resp.a3 & mask, + resp.a4 & mask, resp.a5 & mask, resp.a6 & mask, resp.a7 & mask); +} +static bool ffa_handle_call (struct cpu_user_regs *regs) {uint32_t fid =get_user_reg (regs,0 );@@ -239,6 +340,10 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)case FFA_ID_GET :set_regs_success (regs,get_vm_id (d),0 );return true; +case FFA_MSG_SEND_DIRECT_REQ_32 : +case FFA_MSG_SEND_DIRECT_REQ_64 : +handle_msg_send_direct_req (regs, fid); +return true;default :gprintk (XENLOG_ERR ,"ffa: unhandled fid 0x%x\n" , fid);@@ -331,6 +436,14 @@ static bool ffa_probe(void)printk (XENLOG_INFO "ARM FF-A Firmware version %u.%u\n" , major_vers, minor_vers); +/* +* At the moment domains must support the same features used by Xen. +* TODO: Rework the code to allow domain to use a subset of the +* features supported. +*/ +if ( !check_mandatory_feature (FFA_MSG_SEND_DIRECT_REQ_32 ) ) +return false; + ffa_version = vers;return true;Signature · 1 line
2.34.1
PATCH v11 2/14 xen/arm: ffa: map SPMC rx/tx buffers
In reply to Jens WiklanderWhen initializing the FF-A mediator map the RX and TX buffers shared with the SPMC.
These buffer are later used to to transmit data that cannot be passed in registers only.
Adds a check that the SP supports the needed FF-A features FFA_RXTX_MAP_64 and FFA_RXTX_UNMAP.
---v10->v11
- Adding Reviewed-by: Bertrand Marquis <[email protected]>
--- xen/arch/arm/tee/ffa.c | 50 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 49 insertions(+), 1 deletion(-)
xen/arch/arm/tee/ffa.c +49 -1
@@ -12,6 +12,7 @@#include <xen/errno.h> #include <xen/init.h> #include <xen/lib.h> +#include <xen/mm.h> #include <xen/sched.h> #include <xen/sizes.h> #include <xen/types.h>
@@ -67,6 +68,12 @@*/ #define FFA_PAGE_SIZE SZ_4K +/* +* The number of pages used for each of the RX and TX buffers shared with +* the SPMC. +*/ +#define FFA_RXTX_PAGE_COUNT 1 +/* * Flags and field values used for the MSG_SEND_DIRECT_REQ/RESP: * BIT(31): Framework or partition message
@@ -161,6 +168,13 @@ struct ffa_ctx {/* Negotiated FF-A version to use with the SPMC */ static uint32_t __ro_after_init ffa_version; +/* +* Our rx/tx buffers shared with the SPMC. FFA_RXTX_PAGE_COUNT is the +* number of pages used in each of these buffers. +*/ +static void *ffa_rx__read_mostly ; +static void *ffa_tx__read_mostly ; +static bool ffa_get_version (uint32_t *vers) {const struct arm_smccc_1_2_regs arg = {
@@ -231,6 +245,12 @@ static bool check_mandatory_feature(uint32_t id)return !ret; } +static int32_t ffa_rxtx_map (paddr_t tx_addr,paddr_t rx_addr, +uint32_t page_count) +{ +return ffa_simple_call (FFA_RXTX_MAP_64 , tx_addr, rx_addr, page_count,0 ); +} +static uint16_t get_vm_id (const struct domain *d) {/* +1 since 0 is reserved for the hypervisor in FF-A */
@@ -394,6 +414,7 @@ static int ffa_relinquish_resources(struct domain *d)static bool ffa_probe (void ) {uint32_t vers; +int e;unsigned int major_vers;unsigned int minor_vers;
@@ -441,12 +462,39 @@ static bool ffa_probe(void)* TODO: Rework the code to allow domain to use a subset of the * features supported. */ -if ( !check_mandatory_feature (FFA_MSG_SEND_DIRECT_REQ_32 ) ) +if ( + !check_mandatory_feature (FFA_RXTX_MAP_64 ) || + !check_mandatory_feature (FFA_RXTX_UNMAP ) || + !check_mandatory_feature (FFA_MSG_SEND_DIRECT_REQ_32 ) ) +return false; + + ffa_rx =alloc_xenheap_pages (get_order_from_pages (FFA_RXTX_PAGE_COUNT ),0 ); +if ( !ffa_rx )return false; + ffa_tx =alloc_xenheap_pages (get_order_from_pages (FFA_RXTX_PAGE_COUNT ),0 ); +if ( !ffa_tx ) + goto err_free_ffa_rx; + + e =ffa_rxtx_map (__pa (ffa_tx),__pa (ffa_rx),FFA_RXTX_PAGE_COUNT ); +if ( e ) + { +printk (XENLOG_ERR "ffa: Failed to map rxtx: error %d\n" , e); + goto err_free_ffa_tx; + } ffa_version = vers;return true; + +err_free_ffa_tx: +free_xenheap_pages (ffa_tx,0 ); + ffa_tx =NULL ; +err_free_ffa_rx: +free_xenheap_pages (ffa_rx,0 ); + ffa_rx =NULL ; + ffa_version =0 ; + +return false; } static const struct tee_mediator_ops ffa_ops =
Signature · 1 line
2.34.1
PATCH v11 3/14 xen/arm: ffa: send guest events to Secure Partitions
In reply to Jens WiklanderThe FF-A specification defines framework messages sent as direct requests when certain events occurs. For instance when a VM (guest) is created or destroyed. Only SPs which have subscribed to these events will receive them. An SP can subscribe to these messages in its partition properties.
Adds a check that the SP supports the needed FF-A features FFA_PARTITION_INFO_GET and FFA_RX_RELEASE.
The partition properties of each SP is retrieved with FFA_PARTITION_INFO_GET which returns the information in our RX buffer. Using FFA_PARTITION_INFO_GET changes the owner of the RX buffer to the caller (us), so once we're done with the buffer it must be released using FFA_RX_RELEASE before another call can be made.
---v10->v11:
- Addressing comments and fixing a few style issues
- Fixing how is_in_subscr_list() is used, it's supposed to take an sp_id, not a vm_id.
--- xen/arch/arm/tee/ffa.c | 234 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 233 insertions(+), 1 deletion(-)
xen/arch/arm/tee/ffa.c +233 -1
@@ -160,14 +160,33 @@#define FFA_MSG_SEND 0x8400006EU#define FFA_MSG_POLL 0x8400006AU +/* Partition information descriptor */ +struct ffa_partition_info_1_1 { +uint16_t id; +uint16_t execution_context; +uint32_t partition_properties; +uint8_t uuid[16 ]; +}; +struct ffa_ctx {/* FF-A version used by the guest */ uint32_t guest_vers; +/* +* Number of SPs that we have sent a VM created signal to, used in +* ffa_domain_teardown() to know which SPs need to be signalled. +*/ +uint16_t create_signal_count; };/* Negotiated FF-A version to use with the SPMC */ static uint32_t __ro_after_init ffa_version; +/* SPs subscribing to VM_CREATE and VM_DESTROYED events */ +static uint16_t *subscr_vm_created__read_mostly ; +static uint16_t subscr_vm_created_count__read_mostly ; +static uint16_t *subscr_vm_destroyed__read_mostly ; +static uint16_t subscr_vm_destroyed_count__read_mostly ; +/* * Our rx/tx buffers shared with the SPMC. FFA_RXTX_PAGE_COUNT is the * number of pages used in each of these buffers.
@@ -251,6 +270,87 @@ static int32_t ffa_rxtx_map(paddr_t tx_addr, paddr_t rx_addr,return ffa_simple_call (FFA_RXTX_MAP_64 , tx_addr, rx_addr, page_count,0 ); } +static int32_t ffa_partition_info_get (uint32_t w1,uint32_t w2,uint32_t w3, +uint32_t w4,uint32_t w5, +uint32_t *count) +{ +const struct arm_smccc_1_2_regs arg = { + .a0 =FFA_PARTITION_INFO_GET , + .a1 = w1, + .a2 = w2, + .a3 = w3, + .a4 = w4, + .a5 = w5, + }; +struct arm_smccc_1_2_regs resp; +uint32_t ret; + +arm_smccc_1_2_smc (&arg, &resp); + + ret =get_ffa_ret_code (&resp); +if ( !ret ) + *count = resp.a2; + +return ret; +} + +static int32_t ffa_rx_release (void ) +{ +return ffa_simple_call (FFA_RX_RELEASE ,0 ,0 ,0 ,0 ); +} + +static int32_t ffa_direct_req_send_vm (uint16_t sp_id,uint16_t vm_id, +uint8_t msg) +{ +uint32_t exp_resp =FFA_MSG_FLAG_FRAMEWORK ; +unsigned int retry_count =0 ; +int32_t res; + +if ( msg ==FFA_MSG_SEND_VM_CREATED ) + exp_resp |=FFA_MSG_RESP_VM_CREATED ; +else if ( msg ==FFA_MSG_SEND_VM_DESTROYED ) + exp_resp |=FFA_MSG_RESP_VM_DESTROYED ; +else +return FFA_RET_INVALID_PARAMETERS ; + +do { +const struct arm_smccc_1_2_regs arg = { + .a0 =FFA_MSG_SEND_DIRECT_REQ_32 , + .a1 = sp_id, + .a2 =FFA_MSG_FLAG_FRAMEWORK | msg, + .a5 = vm_id, + }; +struct arm_smccc_1_2_regs resp; + +arm_smccc_1_2_smc (&arg, &resp); +if ( resp.a0 !=FFA_MSG_SEND_DIRECT_RESP_32 || resp.a2 != exp_resp ) + { +/* +* This is an invalid response, likely due to some error in the +* implementation of the ABI. +*/ +return FFA_RET_INVALID_PARAMETERS ; + } + res = resp.a3; +if ( ++retry_count >10 ) + { +/* +* TODO +* FFA_RET_INTERRUPTED means that the SPMC has a pending +* non-secure interrupt, we need a way of delivering that +* non-secure interrupt. +* FFA_RET_RETRY is the SP telling us that it's temporarily +* blocked from handling the direct request, we need a generic +* way to deal with this. +* For now in both cases, give up after a few retries. +*/ +return res; + } + }while ( res ==FFA_RET_INTERRUPTED || res ==FFA_RET_RETRY ); + +return res; +} +static uint16_t get_vm_id (const struct domain *d) {/* +1 since 0 is reserved for the hypervisor in FF-A */
@@ -374,6 +474,8 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)static int ffa_domain_init (struct domain *d) {struct ffa_ctx *ctx; +unsigned int n; +int32_t res;if ( !ffa_version )return -ENODEV ;
@@ -390,17 +492,68 @@ static int ffa_domain_init(struct domain *d) d->arch.tee = ctx; +for ( n =0 ; n < subscr_vm_created_count; n++ ) + { + res =ffa_direct_req_send_vm (subscr_vm_created[n],get_vm_id (d), +FFA_MSG_SEND_VM_CREATED ); +if ( res ) + { +printk (XENLOG_ERR "ffa: Failed to report creation of vm_id %u to %u: res %d\n" , +get_vm_id (d), subscr_vm_created[n], res); + ctx->create_signal_count = n; +return -EIO ; + } + } + ctx->create_signal_count = subscr_vm_created_count; +return 0 ; } +static bool is_in_subscr_list (const uint16_t *subscr,uint16_t start, +uint16_t end,uint16_t sp_id) +{ +unsigned int n; + +for ( n = start; n < end; n++ ) + { +if ( subscr[n] == sp_id ) +return true; + } + +return false; +} +/* This function is supposed to undo what ffa_domain_init() has done */ static int ffa_domain_teardown (struct domain *d) {struct ffa_ctx *ctx = d->arch.tee; +unsigned int n; +int32_t res;if ( !ctx )return 0 ; + +for ( n =0 ; n < subscr_vm_destroyed_count; n++ ) + { +/* +* Skip SPs subscribed to the VM created event that never was +* notified of the VM creation due to an error during +* ffa_domain_init(). +*/ +if (is_in_subscr_list (subscr_vm_created, ctx->create_signal_count, + subscr_vm_created_count, + subscr_vm_destroyed[n]) ) +continue ; + + res =ffa_direct_req_send_vm (subscr_vm_destroyed[n],get_vm_id (d), +FFA_MSG_SEND_VM_DESTROYED ); + +if ( res ) +printk (XENLOG_ERR "ffa: Failed to report destruction of vm_id %u to %u: res %d\n" , +get_vm_id (d), subscr_vm_destroyed[n], res); + } +XFREE (d->arch.tee);return 0 ;
@@ -411,6 +564,81 @@ static int ffa_relinquish_resources(struct domain *d)return 0 ; } +static void uninit_subscribers (void ) +{ + subscr_vm_created_count =0 ; + subscr_vm_destroyed_count =0 ; +XFREE (subscr_vm_created); +XFREE (subscr_vm_destroyed); +} + +static bool init_subscribers (struct ffa_partition_info_1_1 *fpi,uint16_t count) +{ +uint16_t n; +uint16_t c_pos; +uint16_t d_pos; + + subscr_vm_created_count =0 ; + subscr_vm_destroyed_count =0 ; +for ( n =0 ; n < count; n++ ) + { +if ( fpi[n].partition_properties &FFA_PART_PROP_NOTIF_CREATED ) + subscr_vm_created_count++; +if ( fpi[n].partition_properties &FFA_PART_PROP_NOTIF_DESTROYED ) + subscr_vm_destroyed_count++; + } + +if ( subscr_vm_created_count ) + subscr_vm_created =xzalloc_array (uint16_t, subscr_vm_created_count); +if ( subscr_vm_destroyed_count ) + subscr_vm_destroyed =xzalloc_array (uint16_t, + subscr_vm_destroyed_count); +if ( (subscr_vm_created_count && !subscr_vm_created) || + (subscr_vm_destroyed_count && !subscr_vm_destroyed) ) + { +printk (XENLOG_ERR "ffa: Failed to allocate subscription lists\n" ); +uninit_subscribers (); +return false; + } + +for ( c_pos =0 , d_pos =0 , n =0 ; n < count; n++ ) + { +if ( fpi[n].partition_properties &FFA_PART_PROP_NOTIF_CREATED ) + subscr_vm_created[c_pos++] = fpi[n].id; +if ( fpi[n].partition_properties &FFA_PART_PROP_NOTIF_DESTROYED ) + subscr_vm_destroyed[d_pos++] = fpi[n].id; + } + +return true; +} + +static bool init_sps (void ) +{ +bool ret = false; +uint32_t count; +int e; + + e =ffa_partition_info_get (0 ,0 ,0 ,0 ,0 , &count); +if ( e ) + { +printk (XENLOG_ERR "ffa: Failed to get list of SPs: %d\n" , e); + goto out; + } + +if ( count >=UINT16_MAX ) + { +printk (XENLOG_ERR "ffa: Impossible number of SPs: %u\n" , count); + goto out; + } + + ret =init_subscribers (ffa_rx, count); + +out: +ffa_rx_release (); + +return ret; +} +static bool ffa_probe (void ) {uint32_t vers;
@@ -462,7 +690,8 @@ static bool ffa_probe(void)* TODO: Rework the code to allow domain to use a subset of the * features supported. */ -if ( +if ( !check_mandatory_feature (FFA_PARTITION_INFO_GET ) || + !check_mandatory_feature (FFA_RX_RELEASE ) || !check_mandatory_feature (FFA_RXTX_MAP_64 ) || !check_mandatory_feature (FFA_RXTX_UNMAP ) || !check_mandatory_feature (FFA_MSG_SEND_DIRECT_REQ_32 ) )
@@ -484,6 +713,9 @@ static bool ffa_probe(void) } ffa_version = vers; +if ( !init_sps () ) + goto err_free_ffa_tx; +return true; err_free_ffa_tx:
Signature · 1 line
2.34.1
Re: [XEN PATCH v11 03/14] xen/arm: ffa: send guest events to Secure Partitions
Hi Jens,
Quoted 11 lines · Jens Wiklander
The FF-A specification defines framework messages sent as direct requests when certain events occurs. For instance when a VM (guest) is created or destroyed. Only SPs which have subscribed to these events will receive them. An SP can subscribe to these messages in its partition properties.
Adds a check that the SP supports the needed FF-A features
FFA_PARTITION_INFO_GETandFFA_RX_RELEASE.
Quoted 7 lines
The partition properties of each SP is retrieved with
FFA_PARTITION_INFO_GETwhich returns the information in our RX buffer. UsingFFA_PARTITION_INFO_GETchanges the owner of the RX buffer to the caller (us), so once we're done with the buffer it must be released usingFFA_RX_RELEASEbefore another call can be made.Signed-off-by Jens Wiklander <[email protected]>
One NIT after which might be fixed on commit.
With that fixed:
Quoted 180 lines
---
v10->v11:
- Addressing comments and fixing a few style issues
- Fixing howis_in_subscr_list()is used, it's supposed to take ansp_id, not avm_id.
---xen/arch/arm/tee/ffa.c| 234 ++++++++++++++++++++++++++++++++++++++++-1 file changed, 233 insertions(+), 1 deletion(-)xen/arch/arm/tee/ffa.c +130
-0@@ -160,14 +160,33 @@#define FFA_MSG_SEND 0x8400006EU#define FFA_MSG_POLL 0x8400006AU +/* Partition information descriptor */ +struct ffa_partition_info_1_1 { +uint16_t id; +uint16_t execution_context; +uint32_t partition_properties; +uint8_t uuid[16 ]; +}; +struct ffa_ctx {/* FF-A version used by the guest */ uint32_t guest_vers; +/* +* Number of SPs that we have sent a VM created signal to, used in +* ffa_domain_teardown() to know which SPs need to be signalled. +*/ +uint16_t create_signal_count; };/* Negotiated FF-A version to use with the SPMC */ static uint32_t __ro_after_init ffa_version; +/* SPs subscribing to VM_CREATE and VM_DESTROYED events */ +static uint16_t *subscr_vm_created__read_mostly ; +static uint16_t subscr_vm_created_count__read_mostly ; +static uint16_t *subscr_vm_destroyed__read_mostly ; +static uint16_t subscr_vm_destroyed_count__read_mostly ; +/* * Our rx/tx buffers shared with the SPMC. FFA_RXTX_PAGE_COUNT is the * number of pages used in each of these buffers. @@ -251,6 +270,87 @@ static int32_t ffa_rxtx_map(paddr_t tx_addr, paddr_t rx_addr,return ffa_simple_call (FFA_RXTX_MAP_64 , tx_addr, rx_addr, page_count,0 ); } +static int32_t ffa_partition_info_get (uint32_t w1,uint32_t w2,uint32_t w3, +uint32_t w4,uint32_t w5, +uint32_t *count) +{ +const struct arm_smccc_1_2_regs arg = { + .a0 =FFA_PARTITION_INFO_GET , + .a1 = w1, + .a2 = w2, + .a3 = w3, + .a4 = w4, + .a5 = w5, + }; +struct arm_smccc_1_2_regs resp; +uint32_t ret; + +arm_smccc_1_2_smc (&arg, &resp); + + ret =get_ffa_ret_code (&resp); +if ( !ret ) + *count = resp.a2; + +return ret; +} + +static int32_t ffa_rx_release (void ) +{ +return ffa_simple_call (FFA_RX_RELEASE ,0 ,0 ,0 ,0 ); +} + +static int32_t ffa_direct_req_send_vm (uint16_t sp_id,uint16_t vm_id, +uint8_t msg) +{ +uint32_t exp_resp =FFA_MSG_FLAG_FRAMEWORK ; +unsigned int retry_count =0 ; +int32_t res; + +if ( msg ==FFA_MSG_SEND_VM_CREATED ) + exp_resp |=FFA_MSG_RESP_VM_CREATED ; +else if ( msg ==FFA_MSG_SEND_VM_DESTROYED ) + exp_resp |=FFA_MSG_RESP_VM_DESTROYED ; +else +return FFA_RET_INVALID_PARAMETERS ; + +do { +const struct arm_smccc_1_2_regs arg = { + .a0 =FFA_MSG_SEND_DIRECT_REQ_32 , + .a1 = sp_id, + .a2 =FFA_MSG_FLAG_FRAMEWORK | msg, + .a5 = vm_id, + }; +struct arm_smccc_1_2_regs resp; + +arm_smccc_1_2_smc (&arg, &resp); +if ( resp.a0 !=FFA_MSG_SEND_DIRECT_RESP_32 || resp.a2 != exp_resp ) + { +/* +* This is an invalid response, likely due to some error in the +* implementation of the ABI. +*/ +return FFA_RET_INVALID_PARAMETERS ; + } + res = resp.a3; +if ( ++retry_count >10 ) + { +/* +* TODO +* FFA_RET_INTERRUPTED means that the SPMC has a pending +* non-secure interrupt, we need a way of delivering that +* non-secure interrupt. +* FFA_RET_RETRY is the SP telling us that it's temporarily +* blocked from handling the direct request, we need a generic +* way to deal with this. +* For now in both cases, give up after a few retries. +*/ +return res; + } + }while ( res ==FFA_RET_INTERRUPTED || res ==FFA_RET_RETRY ); + +return res; +} +static uint16_t get_vm_id (const struct domain *d) {/* +1 since 0 is reserved for the hypervisor in FF-A */ @@ -374,6 +474,8 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)static int ffa_domain_init (struct domain *d) {struct ffa_ctx *ctx; +unsigned int n; +int32_t res;if ( !ffa_version )return -ENODEV ;@@ -390,17 +492,68 @@ static int ffa_domain_init(struct domain *d) d->arch.tee = ctx; +for ( n =0 ; n < subscr_vm_created_count; n++ ) + { + res =ffa_direct_req_send_vm (subscr_vm_created[n],get_vm_id (d), +FFA_MSG_SEND_VM_CREATED ); +if ( res ) + { +printk (XENLOG_ERR "ffa: Failed to report creation of vm_id %u to %u: res %d\n" , +get_vm_id (d), subscr_vm_created[n], res); + ctx->create_signal_count = n; +return -EIO ; + } + } + ctx->create_signal_count = subscr_vm_created_count; +return 0 ; } +static bool is_in_subscr_list (const uint16_t *subscr,uint16_t start, +uint16_t end,uint16_t sp_id) +{ +unsigned int n; + +for ( n = start; n < end; n++ ) + { +if ( subscr[n] == sp_id ) +return true; + } + +return false; +} +/* This function is supposed to undo what ffa_domain_init() has done */
xen/arch/arm/tee/ffa.c static int ffa_domain_init(struct domain *d) · line 536
Quoted 10 lines
xen/arch/arm/tee/ffa.c +3
-0static int ffa_domain_teardown (struct domain *d) {struct ffa_ctx *ctx = d->arch.tee; +unsigned int n; +int32_t res;if ( !ctx )return 0 ; +
NIT: this extra line should be removed.
@Julien: are you ok to fix that one on commit ?
Quoted 124 lines
xen/arch/arm/tee/ffa.c +100
-1+for ( n =0 ; n < subscr_vm_destroyed_count; n++ ) + { +/* +* Skip SPs subscribed to the VM created event that never was +* notified of the VM creation due to an error during +* ffa_domain_init(). +*/ +if (is_in_subscr_list (subscr_vm_created, ctx->create_signal_count, + subscr_vm_created_count, + subscr_vm_destroyed[n]) ) +continue ; + + res =ffa_direct_req_send_vm (subscr_vm_destroyed[n],get_vm_id (d), +FFA_MSG_SEND_VM_DESTROYED ); + +if ( res ) +printk (XENLOG_ERR "ffa: Failed to report destruction of vm_id %u to %u: res %d\n" , +get_vm_id (d), subscr_vm_destroyed[n], res); + } +XFREE (d->arch.tee);return 0 ;@@ -411,6 +564,81 @@ static int ffa_relinquish_resources(struct domain *d)return 0 ; } +static void uninit_subscribers (void ) +{ + subscr_vm_created_count =0 ; + subscr_vm_destroyed_count =0 ; +XFREE (subscr_vm_created); +XFREE (subscr_vm_destroyed); +} + +static bool init_subscribers (struct ffa_partition_info_1_1 *fpi,uint16_t count) +{ +uint16_t n; +uint16_t c_pos; +uint16_t d_pos; + + subscr_vm_created_count =0 ; + subscr_vm_destroyed_count =0 ; +for ( n =0 ; n < count; n++ ) + { +if ( fpi[n].partition_properties &FFA_PART_PROP_NOTIF_CREATED ) + subscr_vm_created_count++; +if ( fpi[n].partition_properties &FFA_PART_PROP_NOTIF_DESTROYED ) + subscr_vm_destroyed_count++; + } + +if ( subscr_vm_created_count ) + subscr_vm_created =xzalloc_array (uint16_t, subscr_vm_created_count); +if ( subscr_vm_destroyed_count ) + subscr_vm_destroyed =xzalloc_array (uint16_t, + subscr_vm_destroyed_count); +if ( (subscr_vm_created_count && !subscr_vm_created) || + (subscr_vm_destroyed_count && !subscr_vm_destroyed) ) + { +printk (XENLOG_ERR "ffa: Failed to allocate subscription lists\n" ); +uninit_subscribers (); +return false; + } + +for ( c_pos =0 , d_pos =0 , n =0 ; n < count; n++ ) + { +if ( fpi[n].partition_properties &FFA_PART_PROP_NOTIF_CREATED ) + subscr_vm_created[c_pos++] = fpi[n].id; +if ( fpi[n].partition_properties &FFA_PART_PROP_NOTIF_DESTROYED ) + subscr_vm_destroyed[d_pos++] = fpi[n].id; + } + +return true; +} + +static bool init_sps (void ) +{ +bool ret = false; +uint32_t count; +int e; + + e =ffa_partition_info_get (0 ,0 ,0 ,0 ,0 , &count); +if ( e ) + { +printk (XENLOG_ERR "ffa: Failed to get list of SPs: %d\n" , e); + goto out; + } + +if ( count >=UINT16_MAX ) + { +printk (XENLOG_ERR "ffa: Impossible number of SPs: %u\n" , count); + goto out; + } + + ret =init_subscribers (ffa_rx, count); + +out: +ffa_rx_release (); + +return ret; +} +static bool ffa_probe (void ) {uint32_t vers;@@ -462,7 +690,8 @@ static bool ffa_probe(void)* TODO: Rework the code to allow domain to use a subset of the * features supported. */ -if ( +if ( !check_mandatory_feature (FFA_PARTITION_INFO_GET ) || + !check_mandatory_feature (FFA_RX_RELEASE ) || !check_mandatory_feature (FFA_RXTX_MAP_64 ) || !check_mandatory_feature (FFA_RXTX_UNMAP ) || !check_mandatory_feature (FFA_MSG_SEND_DIRECT_REQ_32 ) )@@ -484,6 +713,9 @@ static bool ffa_probe(void) } ffa_version = vers; +if ( !init_sps () ) + goto err_free_ffa_tx; +return true;
Quoted 4 lines
err_free_ffa_tx:Signature · 1 line
2.34.1
Cheers
Bertrand
Re: [XEN PATCH v11 03/14] xen/arm: ffa: send guest events to Secure Partitions
Quoted 1 line · Bertrand Marquis
Hi Jens,
Hi,
Quoted 8 lines
Quoted 8 lines · Jens Wiklander
/* This function is supposed to undo what
ffa_domain_init()has done */ static intffa_domain_teardown(struct domain *d){ struct ffa_ctx *ctx = d->arch.tee;(quoted diff) +2
-0+ unsigned int n; + int32_t res;
Quoted 8 lines
Quoted 4 lines
if ( !ctx ) return 0;+
NIT: this extra line should be removed.
@Julien: are you ok to fix that one on commit ?
Yes I can do it.
CHeers,
Signature · 1 line
Julien Grall
PATCH v11 4/14 xen/arm: ffa: support mapping guest RX/TX buffers
In reply to Jens WiklanderAdds support in the mediator to map and unmap the RX and TX buffers provided by the guest using the two FF-A functions FFA_RXTX_MAP and FFA_RXTX_UNMAP.
These buffer are later used to transmit data that cannot be passed in registers only.
---v10->v11
- Fixing a typo in a comment and adding a couple of newlines as requested
--- xen/arch/arm/tee/ffa.c | 140 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+)
xen/arch/arm/tee/ffa.c +140 -0
@@ -74,6 +74,12 @@*/ #define FFA_RXTX_PAGE_COUNT 1 +/* +* Limit the number of pages RX/TX buffers guests can map. +* TODO support a larger number. +*/ +#define FFA_MAX_RXTX_PAGE_COUNT 1 +/* * Flags and field values used for the MSG_SEND_DIRECT_REQ/RESP: * BIT(31): Framework or partition message
@@ -169,6 +175,12 @@ struct ffa_partition_info_1_1 { };struct ffa_ctx { +void *rx; +const void *tx; +struct page_info *rx_pg; +struct page_info *tx_pg; +/* Number of 4kB pages in each of rx/rx_pg and tx/tx_pg */ +unsigned int page_count;/* FF-A version used by the guest */ uint32_t guest_vers;/*
@@ -176,6 +188,7 @@ struct ffa_ctx {* ffa_domain_teardown() to know which SPs need to be signalled. */ uint16_t create_signal_count; +bool rx_is_free; };/* Negotiated FF-A version to use with the SPMC */
@@ -371,6 +384,11 @@ static void set_regs(struct cpu_user_regs *regs, register_t v0, register_t v1,set_user_reg (regs,7 , v7); } +static void set_regs_error (struct cpu_user_regs *regs,uint32_t error_code) +{ +set_regs (regs,FFA_ERROR ,0 , error_code,0 ,0 ,0 ,0 ,0 ); +} +static void set_regs_success (struct cpu_user_regs *regs,uint32_t w2,uint32_t w3) {
@@ -392,6 +410,108 @@ static void handle_version(struct cpu_user_regs *regs)set_regs (regs, vers,0 ,0 ,0 ,0 ,0 ,0 ,0 ); } +static uint32_t handle_rxtx_map (uint32_t fid,register_t tx_addr, +register_t rx_addr,uint32_t page_count) +{ +uint32_t ret =FFA_RET_INVALID_PARAMETERS ; +struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee; +struct page_info *tx_pg; +struct page_info *rx_pg; +p2m_type_t t; +void *rx; +void *tx; + +if ( !smccc_is_conv_64 (fid) ) + { +/* +* Calls using the 32-bit calling convention must ignore the upper +* 32 bits in the argument registers. +*/ + tx_addr &=UINT32_MAX ; + rx_addr &=UINT32_MAX ; + } + +if ( page_count >FFA_MAX_RXTX_PAGE_COUNT ) + { +printk (XENLOG_ERR "ffa: RXTX_MAP: error: %u pages requested (limit %u)\n" , + page_count,FFA_MAX_RXTX_PAGE_COUNT ); +return FFA_RET_NOT_SUPPORTED ; + } + +/* Already mapped */ +if ( ctx->rx ) +return FFA_RET_DENIED ; + + tx_pg =get_page_from_gfn (d,gfn_x (gaddr_to_gfn (tx_addr)), &t,P2M_ALLOC ); +if ( !tx_pg ) +return FFA_RET_INVALID_PARAMETERS ; + +/* Only normal RW RAM for now */ +if ( t != p2m_ram_rw ) + goto err_put_tx_pg; + + rx_pg =get_page_from_gfn (d,gfn_x (gaddr_to_gfn (rx_addr)), &t,P2M_ALLOC ); +if ( !tx_pg ) + goto err_put_tx_pg; + +/* Only normal RW RAM for now */ +if ( t != p2m_ram_rw ) + goto err_put_rx_pg; + + tx =__map_domain_page_global (tx_pg); +if ( !tx ) + goto err_put_rx_pg; + + rx =__map_domain_page_global (rx_pg); +if ( !rx ) + goto err_unmap_tx; + + ctx->rx = rx; + ctx->tx = tx; + ctx->rx_pg = rx_pg; + ctx->tx_pg = tx_pg; + ctx->page_count = page_count; + ctx->rx_is_free = true; +return FFA_RET_OK ; + +err_unmap_tx: +unmap_domain_page_global (tx); +err_put_rx_pg: +put_page (rx_pg); +err_put_tx_pg: +put_page (tx_pg); + +return ret; +} + +static void rxtx_unmap (struct ffa_ctx *ctx) +{ +unmap_domain_page_global (ctx->rx); +unmap_domain_page_global (ctx->tx); +put_page (ctx->rx_pg); +put_page (ctx->tx_pg); + ctx->rx =NULL ; + ctx->tx =NULL ; + ctx->rx_pg =NULL ; + ctx->tx_pg =NULL ; + ctx->page_count =0 ; + ctx->rx_is_free = false; +} + +static uint32_t handle_rxtx_unmap (void ) +{ +struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee; + +if ( !ctx->rx ) +return FFA_RET_INVALID_PARAMETERS ; + +rxtx_unmap (ctx); + +return FFA_RET_OK ; +} +static void handle_msg_send_direct_req (struct cpu_user_regs *regs,uint32_t fid) {struct arm_smccc_1_2_regs arg = { .a0 = fid, };
@@ -448,6 +568,7 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)uint32_t fid =get_user_reg (regs,0 );struct domain *d = current->domain;struct ffa_ctx *ctx = d->arch.tee; +int e;if ( !ctx )return false;
@@ -460,6 +581,22 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)case FFA_ID_GET :set_regs_success (regs,get_vm_id (d),0 );return true; +case FFA_RXTX_MAP_32 : +case FFA_RXTX_MAP_64 : + e =handle_rxtx_map (fid,get_user_reg (regs,1 ),get_user_reg (regs,2 ), +get_user_reg (regs,3 )); +if ( e ) +set_regs_error (regs, e); +else +set_regs_success (regs,0 ,0 ); +return true; +case FFA_RXTX_UNMAP : + e =handle_rxtx_unmap (); +if ( e ) +set_regs_error (regs, e); +else +set_regs_success (regs,0 ,0 ); +return true;case FFA_MSG_SEND_DIRECT_REQ_32 :case FFA_MSG_SEND_DIRECT_REQ_64 :handle_msg_send_direct_req (regs, fid);
@@ -554,6 +691,9 @@ static int ffa_domain_teardown(struct domain *d) get_vm_id(d), subscr_vm_destroyed[n], res); } +if ( ctx->rx ) +rxtx_unmap (ctx); +XFREE (d->arch.tee);return 0 ;
Signature · 1 line
2.34.1
Re: [XEN PATCH v11 04/14] xen/arm: ffa: support mapping guest RX/TX buffers
Hi Jens,
Quoted 10 lines · Jens Wiklander
Adds support in the mediator to map and unmap the RX and TX buffers provided by the guest using the two FF-A functions
FFA_RXTX_MAPandFFA_RXTX_UNMAP.These buffer are later used to transmit data that cannot be passed in registers only.
Signed-off-by Jens Wiklander <[email protected]>
Cheers
Bertrand
Quoted 210 lines
---
v10->v11
- Fixing a typo in a comment and adding a couple of newlines as requested
---xen/arch/arm/tee/ffa.c| 140 +++++++++++++++++++++++++++++++++++++++++1 file changed, 140 insertions(+)xen/arch/arm/tee/ffa.c +140
-0@@ -74,6 +74,12 @@*/ #define FFA_RXTX_PAGE_COUNT 1 +/* +* Limit the number of pages RX/TX buffers guests can map. +* TODO support a larger number. +*/ +#define FFA_MAX_RXTX_PAGE_COUNT 1 +/* * Flags and field values used for the MSG_SEND_DIRECT_REQ/RESP: * BIT(31): Framework or partition message @@ -169,6 +175,12 @@ struct ffa_partition_info_1_1 { };struct ffa_ctx { +void *rx; +const void *tx; +struct page_info *rx_pg; +struct page_info *tx_pg; +/* Number of 4kB pages in each of rx/rx_pg and tx/tx_pg */ +unsigned int page_count;/* FF-A version used by the guest */ uint32_t guest_vers;/* @@ -176,6 +188,7 @@ struct ffa_ctx {* ffa_domain_teardown() to know which SPs need to be signalled. */ uint16_t create_signal_count; +bool rx_is_free; };/* Negotiated FF-A version to use with the SPMC */ @@ -371,6 +384,11 @@ static void set_regs(struct cpu_user_regs *regs, register_t v0, register_t v1,set_user_reg (regs,7 , v7); } +static void set_regs_error (struct cpu_user_regs *regs,uint32_t error_code) +{ +set_regs (regs,FFA_ERROR ,0 , error_code,0 ,0 ,0 ,0 ,0 ); +} +static void set_regs_success (struct cpu_user_regs *regs,uint32_t w2,uint32_t w3) {@@ -392,6 +410,108 @@ static void handle_version(struct cpu_user_regs *regs)set_regs (regs, vers,0 ,0 ,0 ,0 ,0 ,0 ,0 ); } +static uint32_t handle_rxtx_map (uint32_t fid,register_t tx_addr, +register_t rx_addr,uint32_t page_count) +{ +uint32_t ret =FFA_RET_INVALID_PARAMETERS ; +struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee; +struct page_info *tx_pg; +struct page_info *rx_pg; +p2m_type_t t; +void *rx; +void *tx; + +if ( !smccc_is_conv_64 (fid) ) + { +/* +* Calls using the 32-bit calling convention must ignore the upper +* 32 bits in the argument registers. +*/ + tx_addr &=UINT32_MAX ; + rx_addr &=UINT32_MAX ; + } + +if ( page_count >FFA_MAX_RXTX_PAGE_COUNT ) + { +printk (XENLOG_ERR "ffa: RXTX_MAP: error: %u pages requested (limit %u)\n" , + page_count,FFA_MAX_RXTX_PAGE_COUNT ); +return FFA_RET_NOT_SUPPORTED ; + } + +/* Already mapped */ +if ( ctx->rx ) +return FFA_RET_DENIED ; + + tx_pg =get_page_from_gfn (d,gfn_x (gaddr_to_gfn (tx_addr)), &t,P2M_ALLOC ); +if ( !tx_pg ) +return FFA_RET_INVALID_PARAMETERS ; + +/* Only normal RW RAM for now */ +if ( t != p2m_ram_rw ) + goto err_put_tx_pg; + + rx_pg =get_page_from_gfn (d,gfn_x (gaddr_to_gfn (rx_addr)), &t,P2M_ALLOC ); +if ( !tx_pg ) + goto err_put_tx_pg; + +/* Only normal RW RAM for now */ +if ( t != p2m_ram_rw ) + goto err_put_rx_pg; + + tx =__map_domain_page_global (tx_pg); +if ( !tx ) + goto err_put_rx_pg; + + rx =__map_domain_page_global (rx_pg); +if ( !rx ) + goto err_unmap_tx; + + ctx->rx = rx; + ctx->tx = tx; + ctx->rx_pg = rx_pg; + ctx->tx_pg = tx_pg; + ctx->page_count = page_count; + ctx->rx_is_free = true; +return FFA_RET_OK ; + +err_unmap_tx: +unmap_domain_page_global (tx); +err_put_rx_pg: +put_page (rx_pg); +err_put_tx_pg: +put_page (tx_pg); + +return ret; +} + +static void rxtx_unmap (struct ffa_ctx *ctx) +{ +unmap_domain_page_global (ctx->rx); +unmap_domain_page_global (ctx->tx); +put_page (ctx->rx_pg); +put_page (ctx->tx_pg); + ctx->rx =NULL ; + ctx->tx =NULL ; + ctx->rx_pg =NULL ; + ctx->tx_pg =NULL ; + ctx->page_count =0 ; + ctx->rx_is_free = false; +} + +static uint32_t handle_rxtx_unmap (void ) +{ +struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee; + +if ( !ctx->rx ) +return FFA_RET_INVALID_PARAMETERS ; + +rxtx_unmap (ctx); + +return FFA_RET_OK ; +} +static void handle_msg_send_direct_req (struct cpu_user_regs *regs,uint32_t fid) {struct arm_smccc_1_2_regs arg = { .a0 = fid, };@@ -448,6 +568,7 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)uint32_t fid =get_user_reg (regs,0 );struct domain *d = current->domain;struct ffa_ctx *ctx = d->arch.tee; +int e;if ( !ctx )return false;@@ -460,6 +581,22 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)case FFA_ID_GET :set_regs_success (regs,get_vm_id (d),0 );return true; +case FFA_RXTX_MAP_32 : +case FFA_RXTX_MAP_64 : + e =handle_rxtx_map (fid,get_user_reg (regs,1 ),get_user_reg (regs,2 ), +get_user_reg (regs,3 )); +if ( e ) +set_regs_error (regs, e); +else +set_regs_success (regs,0 ,0 ); +return true; +case FFA_RXTX_UNMAP : + e =handle_rxtx_unmap (); +if ( e ) +set_regs_error (regs, e); +else +set_regs_success (regs,0 ,0 ); +return true;case FFA_MSG_SEND_DIRECT_REQ_32 :case FFA_MSG_SEND_DIRECT_REQ_64 :handle_msg_send_direct_req (regs, fid);@@ -554,6 +691,9 @@ static int ffa_domain_teardown(struct domain *d) get_vm_id(d), subscr_vm_destroyed[n], res); } +if ( ctx->rx ) +rxtx_unmap (ctx); +XFREE (d->arch.tee);return 0 ;Signature · 1 line
2.34.1
PATCH v11 5/14 xen/arm: ffa: support guest FFA_PARTITION_INFO_GET
In reply to Jens WiklanderAdds support in the mediator to handle FFA_PARTITION_INFO_GET requests from a guest. The requests are forwarded to the SPMC and the response is translated according to the FF-A version in use by the guest.
Using FFA_PARTITION_INFO_GET changes the owner of the RX buffer to the caller (the guest in this case), so once it is done with the buffer it must be released using FFA_RX_RELEASE before another call can be made.
---v10->v11
- Fixing a typo in a comment and adding another comment as requested
--- xen/arch/arm/tee/ffa.c | 134 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 134 insertions(+)
xen/arch/arm/tee/ffa.c +134 -0
@@ -166,7 +166,18 @@#define FFA_MSG_SEND 0x8400006EU#define FFA_MSG_POLL 0x8400006AU +/* +* Structs below ending with _1_0 are defined in FF-A-1.0-REL and +* structs ending with _1_1 are defined in FF-A-1.1-REL0. +*/ +/* Partition information descriptor */ +struct ffa_partition_info_1_0 { +uint16_t id; +uint16_t execution_context; +uint32_t partition_properties; +}; +struct ffa_partition_info_1_1 {uint16_t id;uint16_t execution_context;
@@ -189,6 +200,7 @@ struct ffa_ctx {*/ uint16_t create_signal_count;bool rx_is_free; +spinlock_t lock; };/* Negotiated FF-A version to use with the SPMC */
@@ -203,9 +215,15 @@ static uint16_t subscr_vm_destroyed_count __read_mostly;/* * Our rx/tx buffers shared with the SPMC. FFA_RXTX_PAGE_COUNT is the * number of pages used in each of these buffers. +* +* The RX buffer is protected from concurrent usage with ffa_rx_buffer_lock. +* Note that the SPMC is also tracking the ownership of our RX buffer so +* for calls which uses our RX buffer to deliver a result we must call +* ffa_rx_release() to let the SPMC know that we're done with the buffer. */ static void *ffa_rx__read_mostly ;static void *ffa_tx__read_mostly ; +static DEFINE_SPINLOCK (ffa_rx_buffer_lock);static bool ffa_get_version (uint32_t *vers) {
@@ -512,6 +530,103 @@ static uint32_t handle_rxtx_unmap(void)return FFA_RET_OK ; } +static int32_t handle_partition_info_get (uint32_t w1,uint32_t w2,uint32_t w3, +uint32_t w4,uint32_t w5, +uint32_t *count) +{ +int32_t ret =FFA_RET_DENIED ; +struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee; + +/* +* FF-A v1.0 has w5 MBZ while v1.1 allows +* FFA_PARTITION_INFO_GET_COUNT_FLAG to be non-zero. +* +* FFA_PARTITION_INFO_GET_COUNT is only using registers and not the +* rxtx buffer so do the partition_info_get directly. +*/ +if ( w5 ==FFA_PARTITION_INFO_GET_COUNT_FLAG && + ctx->guest_vers ==FFA_VERSION_1_1 ) +return ffa_partition_info_get (w1, w2, w3, w4, w5, count); +if ( w5 ) +return FFA_RET_INVALID_PARAMETERS ; + +if ( !ffa_rx ) +return FFA_RET_DENIED ; + +spin_lock (&ctx->lock); +if ( !ctx->page_count || !ctx->rx_is_free ) + goto out; +spin_lock (&ffa_rx_buffer_lock); + ret =ffa_partition_info_get (w1, w2, w3, w4, w5, count); +if ( ret ) + goto out_rx_buf_unlock; +/* +* ffa_partition_info_get() succeeded so we now own the RX buffer we +* share with the SPMC. We must give it back using ffa_rx_release() +* once we've copied the content. +*/ + +if ( ctx->guest_vers ==FFA_VERSION_1_0 ) + { +size_t n; +struct ffa_partition_info_1_1 *src = ffa_rx; +struct ffa_partition_info_1_0 *dst = ctx->rx; + +if ( ctx->page_count *FFA_PAGE_SIZE < *count *sizeof (*dst) ) + { + ret =FFA_RET_NO_MEMORY ; + goto out_rx_release; + } + +for ( n =0 ; n < *count; n++ ) + { + dst[n].id = src[n].id; + dst[n].execution_context = src[n].execution_context; + dst[n].partition_properties = src[n].partition_properties; + } + } +else + { +size_t sz = *count *sizeof (struct ffa_partition_info_1_1 ); + +if ( ctx->page_count *FFA_PAGE_SIZE < sz ) + { + ret =FFA_RET_NO_MEMORY ; + goto out_rx_release; + } + + +memcpy (ctx->rx, ffa_rx, sz); + } + ctx->rx_is_free = false; +out_rx_release: +ffa_rx_release (); +out_rx_buf_unlock: +spin_unlock (&ffa_rx_buffer_lock); +out: +spin_unlock (&ctx->lock); + +return ret; +} + +static int32_t handle_rx_release (void ) +{ +int32_t ret =FFA_RET_DENIED ; +struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee; + +spin_lock (&ctx->lock); +if ( !ctx->page_count || ctx->rx_is_free ) + goto out; + ret =FFA_RET_OK ; + ctx->rx_is_free = true; +out: +spin_unlock (&ctx->lock); + +return ret; +} +static void handle_msg_send_direct_req (struct cpu_user_regs *regs,uint32_t fid) {struct arm_smccc_1_2_regs arg = { .a0 = fid, };
@@ -568,6 +683,7 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)uint32_t fid =get_user_reg (regs,0 );struct domain *d = current->domain;struct ffa_ctx *ctx = d->arch.tee; +uint32_t count;int e;if ( !ctx )
@@ -597,6 +713,24 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)else set_regs_success (regs,0 ,0 );return true; +case FFA_PARTITION_INFO_GET : + e =handle_partition_info_get (get_user_reg (regs,1 ), +get_user_reg (regs,2 ), +get_user_reg (regs,3 ), +get_user_reg (regs,4 ), +get_user_reg (regs,5 ), &count); +if ( e ) +set_regs_error (regs, e); +else +set_regs_success (regs, count,0 ); +return true; +case FFA_RX_RELEASE : + e =handle_rx_release (); +if ( e ) +set_regs_error (regs, e); +else +set_regs_success (regs,0 ,0 ); +return true;case FFA_MSG_SEND_DIRECT_REQ_32 :case FFA_MSG_SEND_DIRECT_REQ_64 :handle_msg_send_direct_req (regs, fid);
Signature · 1 line
2.34.1
Re: [XEN PATCH v11 05/14] xen/arm: ffa: support guest FFA_PARTITION_INFO_GET
Hi Jens,
Quoted 11 lines · Jens Wiklander
Adds support in the mediator to handle
FFA_PARTITION_INFO_GETrequests from a guest. The requests are forwarded to the SPMC and the response is translated according to the FF-A version in use by the guest.Using
FFA_PARTITION_INFO_GETchanges the owner of the RX buffer to the caller (the guest in this case), so once it is done with the buffer it must be released usingFFA_RX_RELEASEbefore another call can be made.Signed-off-by Jens Wiklander <[email protected]>
Cheers
Bertrand
Quoted 194 lines
---
v10->v11
- Fixing a typo in a comment and adding another comment as requested
---xen/arch/arm/tee/ffa.c| 134 +++++++++++++++++++++++++++++++++++++++++1 file changed, 134 insertions(+)xen/arch/arm/tee/ffa.c +134
-0@@ -166,7 +166,18 @@#define FFA_MSG_SEND 0x8400006EU#define FFA_MSG_POLL 0x8400006AU +/* +* Structs below ending with _1_0 are defined in FF-A-1.0-REL and +* structs ending with _1_1 are defined in FF-A-1.1-REL0. +*/ +/* Partition information descriptor */ +struct ffa_partition_info_1_0 { +uint16_t id; +uint16_t execution_context; +uint32_t partition_properties; +}; +struct ffa_partition_info_1_1 {uint16_t id;uint16_t execution_context;@@ -189,6 +200,7 @@ struct ffa_ctx {*/ uint16_t create_signal_count;bool rx_is_free; +spinlock_t lock; };/* Negotiated FF-A version to use with the SPMC */ @@ -203,9 +215,15 @@ static uint16_t subscr_vm_destroyed_count __read_mostly;/* * Our rx/tx buffers shared with the SPMC. FFA_RXTX_PAGE_COUNT is the * number of pages used in each of these buffers. +* +* The RX buffer is protected from concurrent usage with ffa_rx_buffer_lock. +* Note that the SPMC is also tracking the ownership of our RX buffer so +* for calls which uses our RX buffer to deliver a result we must call +* ffa_rx_release() to let the SPMC know that we're done with the buffer. */ static void *ffa_rx__read_mostly ;static void *ffa_tx__read_mostly ; +static DEFINE_SPINLOCK (ffa_rx_buffer_lock);static bool ffa_get_version (uint32_t *vers) {@@ -512,6 +530,103 @@ static uint32_t handle_rxtx_unmap(void)return FFA_RET_OK ; } +static int32_t handle_partition_info_get (uint32_t w1,uint32_t w2,uint32_t w3, +uint32_t w4,uint32_t w5, +uint32_t *count) +{ +int32_t ret =FFA_RET_DENIED ; +struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee; + +/* +* FF-A v1.0 has w5 MBZ while v1.1 allows +* FFA_PARTITION_INFO_GET_COUNT_FLAG to be non-zero. +* +* FFA_PARTITION_INFO_GET_COUNT is only using registers and not the +* rxtx buffer so do the partition_info_get directly. +*/ +if ( w5 ==FFA_PARTITION_INFO_GET_COUNT_FLAG && + ctx->guest_vers ==FFA_VERSION_1_1 ) +return ffa_partition_info_get (w1, w2, w3, w4, w5, count); +if ( w5 ) +return FFA_RET_INVALID_PARAMETERS ; + +if ( !ffa_rx ) +return FFA_RET_DENIED ; + +spin_lock (&ctx->lock); +if ( !ctx->page_count || !ctx->rx_is_free ) + goto out; +spin_lock (&ffa_rx_buffer_lock); + ret =ffa_partition_info_get (w1, w2, w3, w4, w5, count); +if ( ret ) + goto out_rx_buf_unlock; +/* +* ffa_partition_info_get() succeeded so we now own the RX buffer we +* share with the SPMC. We must give it back using ffa_rx_release() +* once we've copied the content. +*/ + +if ( ctx->guest_vers ==FFA_VERSION_1_0 ) + { +size_t n; +struct ffa_partition_info_1_1 *src = ffa_rx; +struct ffa_partition_info_1_0 *dst = ctx->rx; + +if ( ctx->page_count *FFA_PAGE_SIZE < *count *sizeof (*dst) ) + { + ret =FFA_RET_NO_MEMORY ; + goto out_rx_release; + } + +for ( n =0 ; n < *count; n++ ) + { + dst[n].id = src[n].id; + dst[n].execution_context = src[n].execution_context; + dst[n].partition_properties = src[n].partition_properties; + } + } +else + { +size_t sz = *count *sizeof (struct ffa_partition_info_1_1 ); + +if ( ctx->page_count *FFA_PAGE_SIZE < sz ) + { + ret =FFA_RET_NO_MEMORY ; + goto out_rx_release; + } + + +memcpy (ctx->rx, ffa_rx, sz); + } + ctx->rx_is_free = false; +out_rx_release: +ffa_rx_release (); +out_rx_buf_unlock: +spin_unlock (&ffa_rx_buffer_lock); +out: +spin_unlock (&ctx->lock); + +return ret; +} + +static int32_t handle_rx_release (void ) +{ +int32_t ret =FFA_RET_DENIED ; +struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee; + +spin_lock (&ctx->lock); +if ( !ctx->page_count || ctx->rx_is_free ) + goto out; + ret =FFA_RET_OK ; + ctx->rx_is_free = true; +out: +spin_unlock (&ctx->lock); + +return ret; +} +static void handle_msg_send_direct_req (struct cpu_user_regs *regs,uint32_t fid) {struct arm_smccc_1_2_regs arg = { .a0 = fid, };@@ -568,6 +683,7 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)uint32_t fid =get_user_reg (regs,0 );struct domain *d = current->domain;struct ffa_ctx *ctx = d->arch.tee; +uint32_t count;int e;if ( !ctx )@@ -597,6 +713,24 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)else set_regs_success (regs,0 ,0 );return true; +case FFA_PARTITION_INFO_GET : + e =handle_partition_info_get (get_user_reg (regs,1 ), +get_user_reg (regs,2 ), +get_user_reg (regs,3 ), +get_user_reg (regs,4 ), +get_user_reg (regs,5 ), &count); +if ( e ) +set_regs_error (regs, e); +else +set_regs_success (regs, count,0 ); +return true; +case FFA_RX_RELEASE : + e =handle_rx_release (); +if ( e ) +set_regs_error (regs, e); +else +set_regs_success (regs,0 ,0 ); +return true;case FFA_MSG_SEND_DIRECT_REQ_32 :case FFA_MSG_SEND_DIRECT_REQ_64 :handle_msg_send_direct_req (regs, fid);Signature · 1 line
2.34.1
PATCH v11 6/14 xen/arm: move regpair_to_uint64() and uint64_to_regpair() to regs.h
In reply to Jens WiklanderMoves the two helper functions regpair_to_uint64() and uint64_to_regpair() from xen/arch/arm/tee/optee.c to the common arm specific regs.h. This enables reuse of these functions in the FF-A mediator in a subsequent patch.
---v10->v11
- No changes
--- xen/arch/arm/include/asm/regs.h | 12 ++++++++++++ xen/arch/arm/tee/optee.c | 11 ----------- 2 files changed, 12 insertions(+), 11 deletions(-)
xen/arch/arm/include/asm/regs.h +12 -0
@@ -60,6 +60,18 @@ static inline bool guest_mode(const struct cpu_user_regs *r)register_t get_user_reg (struct cpu_user_regs *regs,int reg);void set_user_reg (struct cpu_user_regs *regs,int reg,register_t value); +static inline uint64_t regpair_to_uint64 (register_t reg0,register_t reg1) +{ +return ((uint64_t )reg0 <<32 ) | (uint32_t )reg1; +} + +static inline void uint64_to_regpair (register_t *reg0,register_t *reg1, +uint64_t val) +{ + *reg0 = val >>32 ; + *reg1 = (uint32_t )val; +} +#endif #endif /* __ARM_REGS_H__ */
xen/arch/arm/tee/optee.c +0 -11
@@ -273,17 +273,6 @@ static int optee_domain_teardown(struct domain *d)return 0 ; } -static uint64_t regpair_to_uint64 (register_t reg0,register_t reg1) -{ -return ((uint64_t )reg0 <<32 ) | (uint32_t )reg1; -} - -static void uint64_to_regpair (register_t *reg0,register_t *reg1,uint64_t val) -{ - *reg0 = val >>32 ; - *reg1 = (uint32_t )val; -} -static struct page_info *get_domain_ram_page (gfn_t gfn) {struct page_info *page;
Signature · 1 line
2.34.1
PATCH v11 7/14 xen/arm: ffa: add defines for sharing memory
In reply to Jens WiklanderAccording to DEN0077A version 1.1 REL0
- Section 10.9.2 Memory region handle, page 167
- Table 10.18 at page 175
- Table 10.15 at page 168
- Section 10.11.4 Flags usage, page 184-187
add defines needed for sharing using the function FFA_MEM_SHARE and friends.
Also add limits for how many shared buffers that a guest can have at once and how large a shared buffer can be at most.
---v10->v11
- Adding Reviewed-by: Bertrand Marquis <[email protected]>
--- xen/arch/arm/tee/ffa.c | 58 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+)
xen/arch/arm/tee/ffa.c +58 -0
@@ -5,6 +5,14 @@* Arm Firmware Framework for ARMv8-A (FF-A) mediator * * Copyright (C) 2023 Linaro Limited +* +* References: +* FF-A-1.0-REL: FF-A specification version 1.0 available at +* https://developer.arm.com/documentation/den0077/a +* FF-A-1.1-REL0: FF-A specification version 1.1 available at +* https://developer.arm.com/documentation/den0077/e +* TEEC-1.0C: TEE Client API Specification version 1.0c available at +* https://globalplatform.org/specs-library/tee-client-api-specification/ */ #include <xen/bitops.h>
@@ -80,6 +88,56 @@*/ #define FFA_MAX_RXTX_PAGE_COUNT 1 +/* +* Limit for shared buffer size. Please note that this define limits +* number of pages. +* +* FF-A doesn't have any direct requirements on GlobalPlatform or vice +* versa, but an implementation can very well use FF-A in order to provide +* a GlobalPlatform interface on top. +* +* Global Platform specification for TEE requires that any TEE +* implementation should allow to share buffers with size of at least +* 512KB, defined in TEEC-1.0C page 24, Table 4-1, +* TEEC_CONFIG_SHAREDMEM_MAX_SIZE. +* Due to overhead which can be hard to predict exactly, double this number +* to give a safe margin. +*/ +#define FFA_MAX_SHM_PAGE_COUNT (2 * SZ_512K / FFA_PAGE_SIZE) + +/* +* Limits the number of shared buffers that guest can have at once. This +* is to prevent case, when guests trick XEN into exhausting its own +* memory by allocating many small buffers. This value has been chosen +* arbitrarily. +*/ +#define FFA_MAX_SHM_COUNT 32 + +/* FF-A-1.1-REL0 section 10.9.2 Memory region handle, page 167 */ +#define FFA_HANDLE_HYP_FLAG BIT(63, ULL) +#define FFA_HANDLE_INVALID 0xffffffffffffffffULL + +/* +* Memory attributes: Normal memory, Write-Back cacheable, Inner shareable +* Defined in FF-A-1.1-REL0 Table 10.18 at page 175. +*/ +#define FFA_NORMAL_MEM_REG_ATTR 0x2fU +/* +* Memory access permissions: Read-write +* Defined in FF-A-1.1-REL0 Table 10.15 at page 168. +*/ +#define FFA_MEM_ACC_RW 0x2U + +/* FF-A-1.1-REL0 section 10.11.4 Flags usage, page 184-187 */ +/* Clear memory before mapping in receiver */ +#define FFA_MEMORY_REGION_FLAG_CLEAR BIT(0, U) +/* Relayer may time slice this operation */ +#define FFA_MEMORY_REGION_FLAG_TIME_SLICE BIT(1, U) +/* Clear memory after receiver relinquishes it */ +#define FFA_MEMORY_REGION_FLAG_CLEAR_RELINQUISH BIT(2, U) +/* Share memory transaction */ +#define FFA_MEMORY_REGION_TRANSACTION_TYPE_SHARE (1U << 3) +/* * Flags and field values used for the MSG_SEND_DIRECT_REQ/RESP: * BIT(31): Framework or partition message
Signature · 1 line
2.34.1
PATCH v11 8/14 xen/arm: ffa: add ABI structs for sharing memory
In reply to Jens WiklanderAdds the ABI structs used by function FFA_MEM_SHARE and friends for sharing memory.
---v10->v11
- No changes
--- xen/arch/arm/tee/ffa.c | 69 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+)
xen/arch/arm/tee/ffa.c +69 -0
@@ -243,6 +243,75 @@ struct ffa_partition_info_1_1 {uint8_t uuid[16 ]; }; +/* Constituent memory region descriptor */ +struct ffa_address_range { +uint64_t address; +uint32_t page_count; +uint32_t reserved; +}; + +/* Composite memory region descriptor */ +struct ffa_mem_region { +uint32_t total_page_count; +uint32_t address_range_count; +uint64_t reserved; +struct ffa_address_range address_range_array[]; +}; + +/* Memory access permissions descriptor */ +struct ffa_mem_access_perm { +uint16_t endpoint_id; +uint8_t perm; +uint8_t flags; +}; + +/* Endpoint memory access descriptor */ +struct ffa_mem_access { +struct ffa_mem_access_perm access_perm; +uint32_t region_offs; +uint64_t reserved; +}; + +/* Lend, donate or share memory transaction descriptor */ +struct ffa_mem_transaction_1_0 { +uint16_t sender_id; +uint8_t mem_reg_attr; +uint8_t reserved0; +uint32_t flags; +uint64_t handle; +uint64_t tag; +uint32_t reserved1; +uint32_t mem_access_count; +struct ffa_mem_access mem_access_array[]; +}; + +struct ffa_mem_transaction_1_1 { +uint16_t sender_id; +uint16_t mem_reg_attr; +uint32_t flags; +uint64_t handle; +uint64_t tag; +uint32_t mem_access_size; +uint32_t mem_access_count; +uint32_t mem_access_offs; +uint8_t reserved[12 ]; +}; + +/* Endpoint RX/TX descriptor */ +struct ffa_endpoint_rxtx_descriptor_1_0 { +uint16_t sender_id; +uint16_t reserved; +uint32_t rx_range_count; +uint32_t tx_range_count; +}; + +struct ffa_endpoint_rxtx_descriptor_1_1 { +uint16_t sender_id; +uint16_t reserved; +uint32_t rx_region_offs; +uint32_t tx_region_offs; +}; +struct ffa_ctx {void *rx;const void *tx;
Signature · 1 line
2.34.1
PATCH v11 9/14 xen/arm: ffa: support sharing memory
In reply to Jens WiklanderAdds support for a guest to share memory with an SP using FFA_MEM_SHARE. Only memory regions small enough to be shared with a single call to FFA_MEM_SHARE are supported.
With this commit we have a FF-A version 1.1 [1] mediator able to communicate with a Secure Partition in secure world using shared memory. The secure world must use FF-A version 1.1, but the guest is free to use version 1.0 or version 1.1.
Adds a check that the SP supports the needed FF-A features FFA_MEM_SHARE_64 or FFA_MEM_SHARE_32.
[1] https://developer.arm.com/documentation/den0077/latest
---v10->v11
- Adding Reviewed-by: Bertrand Marquis <[email protected]>
--- xen/arch/arm/tee/ffa.c | 486 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 486 insertions(+)
xen/arch/arm/tee/ffa.c +486 -0
@@ -297,6 +297,38 @@ struct ffa_mem_transaction_1_1 {uint8_t reserved[12 ]; }; +/* Calculate offset of struct ffa_mem_access from start of buffer */ +#define MEM_ACCESS_OFFSET (access_idx) \ + ( sizeof(struct ffa_mem_transaction_1_1) + \ + ( access_idx ) * sizeof(struct ffa_mem_access) ) + +/* Calculate offset of struct ffa_mem_region from start of buffer */ +#define REGION_OFFSET (access_count, region_idx) \ + ( MEM_ACCESS_OFFSET(access_count) + \ + ( region_idx ) * sizeof(struct ffa_mem_region) ) + +/* Calculate offset of struct ffa_address_range from start of buffer */ +#define ADDR_RANGE_OFFSET (access_count, region_count, range_idx) \ + ( REGION_OFFSET(access_count, region_count) + \ + ( range_idx ) * sizeof(struct ffa_address_range) ) + +/* +* The parts needed from struct ffa_mem_transaction_1_0 or struct +* ffa_mem_transaction_1_1, used to provide an abstraction of difference in +* data structures between version 1.0 and 1.1. This is just an internal +* interface and can be changed without changing any ABI. +*/ +struct ffa_mem_transaction_int { +uint16_t sender_id; +uint8_t mem_reg_attr; +uint8_t flags; +uint8_t mem_access_size; +uint8_t mem_access_count; +uint16_t mem_access_offs; +uint64_t handle; +uint64_t tag; +}; +/* Endpoint RX/TX descriptor */ struct ffa_endpoint_rxtx_descriptor_1_0 {uint16_t sender_id;
@@ -327,9 +359,22 @@ struct ffa_ctx {*/ uint16_t create_signal_count;bool rx_is_free; +/* Used shared memory objects, struct ffa_shm_mem */ +struct list_head shm_list; +/* Number of allocated shared memory object */ +unsigned int shm_count;spinlock_t lock; }; +struct ffa_shm_mem { +struct list_head list; +uint16_t sender_id; +uint16_t ep_id;/* endpoint, the one lending */ +uint64_t handle;/* FFA_HANDLE_INVALID if not set yet */ +unsigned int page_count; +struct page_info *pages[]; +}; +/* Negotiated FF-A version to use with the SPMC */ static uint32_t __ro_after_init ffa_version;
@@ -351,6 +396,7 @@ static uint16_t subscr_vm_destroyed_count __read_mostly;static void *ffa_rx__read_mostly ;static void *ffa_tx__read_mostly ;static DEFINE_SPINLOCK (ffa_rx_buffer_lock); +static DEFINE_SPINLOCK (ffa_tx_buffer_lock);static bool ffa_get_version (uint32_t *vers) {
@@ -457,6 +503,41 @@ static int32_t ffa_rx_release(void)return ffa_simple_call (FFA_RX_RELEASE ,0 ,0 ,0 ,0 ); } +static int32_t ffa_mem_share (uint32_t tot_len,uint32_t frag_len, +register_t addr,uint32_t pg_count, +uint64_t *handle) +{ +struct arm_smccc_1_2_regs arg = { + .a0 =FFA_MEM_SHARE_64 , + .a1 = tot_len, + .a2 = frag_len, + .a3 = addr, + .a4 = pg_count, + }; +struct arm_smccc_1_2_regs resp; + +arm_smccc_1_2_smc (&arg, &resp); + +switch ( resp.a0 ) + { +case FFA_ERROR : +if ( resp.a2 ) +return resp.a2; +else +return FFA_RET_NOT_SUPPORTED ; +case FFA_SUCCESS_32 : + *handle =regpair_to_uint64 (resp.a3, resp.a2); +return FFA_RET_OK ; +case FFA_MEM_FRAG_RX : + *handle =regpair_to_uint64 (resp.a2, resp.a1); +if ( resp.a3 >INT32_MAX )/* Impossible value */ +return FFA_RET_ABORTED ; +return resp.a3 &INT32_MAX ; +default : +return FFA_RET_NOT_SUPPORTED ; + } +} +static int32_t ffa_direct_req_send_vm (uint16_t sp_id,uint16_t vm_id,uint8_t msg) {
@@ -805,6 +886,403 @@ out: resp.a4 & mask, resp.a5 & mask, resp.a6 & mask, resp.a7 & mask); } +/* +* Gets all page and assigns them to the supplied shared memory object. If +* this function fails then the caller is still expected to call +* put_shm_pages() as a cleanup. +*/ +static int get_shm_pages (struct domain *d,struct ffa_shm_mem *shm, +const struct ffa_address_range *range, +uint32_t range_count,unsigned int start_page_idx, +unsigned int *last_page_idx) +{ +unsigned int pg_idx = start_page_idx; +gfn_t gfn; +unsigned int n; +unsigned int m; +p2m_type_t t; +uint64_t addr; +uint64_t page_count; + +for ( n =0 ; n < range_count; n++ ) + { + page_count =read_atomic (&range[n].page_count); + addr =read_atomic (&range[n].address); +for ( m =0 ; m < page_count; m++ ) + { +if ( pg_idx >= shm->page_count ) +return FFA_RET_INVALID_PARAMETERS ; + + gfn =gaddr_to_gfn (addr + m *FFA_PAGE_SIZE ); + shm->pages[pg_idx] =get_page_from_gfn (d,gfn_x (gfn), &t, +P2M_ALLOC ); +if ( !shm->pages[pg_idx] ) +return FFA_RET_DENIED ; +/* Only normal RW RAM for now */ +if ( t != p2m_ram_rw ) +return FFA_RET_DENIED ; + pg_idx++; + } + } + + *last_page_idx = pg_idx; + +return FFA_RET_OK ; +} + +static void put_shm_pages (struct ffa_shm_mem *shm) +{ +unsigned int n; + +for ( n =0 ; n < shm->page_count && shm->pages[n]; n++ ) + { +put_page (shm->pages[n]); + shm->pages[n] =NULL ; + } +} + +static struct ffa_shm_mem *alloc_ffa_shm_mem (struct ffa_ctx *ctx, +unsigned int page_count) +{ +struct ffa_shm_mem *shm; + +if ( page_count >=FFA_MAX_SHM_PAGE_COUNT || + ctx->shm_count >=FFA_MAX_SHM_COUNT ) +return NULL ; + + shm = xzalloc_flex_struct(struct ffa_shm_mem, pages, page_count); +if ( shm ) + { + ctx->shm_count++; + shm->page_count = page_count; + } + +return shm; +} + +static void free_ffa_shm_mem (struct ffa_ctx *ctx,struct ffa_shm_mem *shm) +{ +if ( !shm ) +return ; + +ASSERT (ctx->shm_count >0 ); + ctx->shm_count--; +put_shm_pages (shm); +xfree (shm); +} + +static void init_range (struct ffa_address_range *addr_range, +paddr_t pa) +{ +memset (addr_range,0 ,sizeof (*addr_range)); + addr_range->address = pa; + addr_range->page_count =1 ; +} + +/* +* This function uses the ffa_tx buffer to transmit the memory transaction +* descriptor. The function depends ffa_tx_buffer_lock to be used to guard +* the buffer from concurrent use. +*/ +static int share_shm (struct ffa_shm_mem *shm) +{ +const uint32_t max_frag_len =FFA_RXTX_PAGE_COUNT *FFA_PAGE_SIZE ; +struct ffa_mem_access *mem_access_array; +struct ffa_mem_transaction_1_1 *descr; +struct ffa_address_range *addr_range; +struct ffa_mem_region *region_descr; +const unsigned int region_count =1 ; +void *buf = ffa_tx; +uint32_t frag_len; +uint32_t tot_len; +paddr_t last_pa; +unsigned int n; +paddr_t pa; + +ASSERT (spin_is_locked (&ffa_tx_buffer_lock)); +ASSERT (shm->page_count); + + descr = buf; +memset (descr,0 ,sizeof (*descr)); + descr->sender_id = shm->sender_id; + descr->handle = shm->handle; + descr->mem_reg_attr =FFA_NORMAL_MEM_REG_ATTR ; + descr->mem_access_count =1 ; + descr->mem_access_size =sizeof (*mem_access_array); + descr->mem_access_offs =MEM_ACCESS_OFFSET (0 ); + + mem_access_array = buf + descr->mem_access_offs; +memset (mem_access_array,0 ,sizeof (*mem_access_array)); + mem_access_array[0 ].access_perm.endpoint_id = shm->ep_id; + mem_access_array[0 ].access_perm.perm =FFA_MEM_ACC_RW ; + mem_access_array[0 ].region_offs =REGION_OFFSET (descr->mem_access_count,0 ); + + region_descr = buf + mem_access_array[0 ].region_offs; +memset (region_descr,0 ,sizeof (*region_descr)); + region_descr->total_page_count = shm->page_count; + + region_descr->address_range_count =1 ; + last_pa =page_to_maddr (shm->pages[0 ]); +for ( n =1 ; n < shm->page_count; last_pa = pa, n++ ) + { + pa =page_to_maddr (shm->pages[n]); +if ( last_pa +FFA_PAGE_SIZE == pa ) +continue ; + region_descr->address_range_count++; + } + + tot_len =ADDR_RANGE_OFFSET (descr->mem_access_count, region_count, + region_descr->address_range_count); +if ( tot_len > max_frag_len ) +return FFA_RET_NOT_SUPPORTED ; + + addr_range = region_descr->address_range_array; + frag_len =ADDR_RANGE_OFFSET (descr->mem_access_count, region_count,1 ); + last_pa =page_to_maddr (shm->pages[0 ]); +init_range (addr_range, last_pa); +for ( n =1 ; n < shm->page_count; last_pa = pa, n++ ) + { + pa =page_to_maddr (shm->pages[n]); +if ( last_pa +FFA_PAGE_SIZE == pa ) + { + addr_range->page_count++; +continue ; + } + + frag_len +=sizeof (*addr_range); + addr_range++; +init_range (addr_range, pa); + } + +return ffa_mem_share (tot_len, frag_len,0 ,0 , &shm->handle); +} + +static int read_mem_transaction (uint32_t ffa_vers,const void *buf,size_t blen, +struct ffa_mem_transaction_int *trans) +{ +uint16_t mem_reg_attr; +uint32_t flags; +uint32_t count; +uint32_t offs; +uint32_t size; + +if ( ffa_vers >=FFA_VERSION_1_1 ) + { +const struct ffa_mem_transaction_1_1 *descr; + +if ( blen <sizeof (*descr) ) +return FFA_RET_INVALID_PARAMETERS ; + + descr = buf; + trans->sender_id = descr->sender_id; + mem_reg_attr = descr->mem_reg_attr; + flags = descr->flags; + trans->handle = descr->handle; + trans->tag = descr->tag; + + count = descr->mem_access_count; + size = descr->mem_access_size; + offs = descr->mem_access_offs; + } +else + { +const struct ffa_mem_transaction_1_0 *descr; + +if ( blen <sizeof (*descr) ) +return FFA_RET_INVALID_PARAMETERS ; + + descr = buf; + trans->sender_id = descr->sender_id; + mem_reg_attr = descr->mem_reg_attr; + flags = descr->flags; + trans->handle = descr->handle; + trans->tag = descr->tag; + + count = descr->mem_access_count; + size =sizeof (struct ffa_mem_access ); + offs = offsetof(struct ffa_mem_transaction_1_0 , mem_access_array); + } +/* +* Make sure that "descr" which is shared with the guest isn't accessed +* again after this point. +*/ +barrier (); + +/* +* We're doing a rough check to see that no information is lost when +* tranfering the values into a struct ffa_mem_transaction_int below. +* The fields in struct ffa_mem_transaction_int are wide enough to hold +* any valid value so being out of range means that something is wrong. +*/ +if ( mem_reg_attr >UINT8_MAX || flags >UINT8_MAX || size >UINT8_MAX || + count >UINT8_MAX || offs >UINT16_MAX ) +return FFA_RET_INVALID_PARAMETERS ; + +/* Check that the endpoint memory access descriptor array fits */ +if ( size * count + offs > blen ) +return FFA_RET_INVALID_PARAMETERS ; + + trans->mem_reg_attr = mem_reg_attr; + trans->flags = flags; + trans->mem_access_size = size; + trans->mem_access_count = count; + trans->mem_access_offs = offs; + +return 0 ; +} + +static void handle_mem_share (struct cpu_user_regs *regs) +{ +uint32_t tot_len =get_user_reg (regs,1 ); +uint32_t frag_len =get_user_reg (regs,2 ); +uint64_t addr =get_user_reg (regs,3 ); +uint32_t page_count =get_user_reg (regs,4 ); +const struct ffa_mem_region *region_descr; +const struct ffa_mem_access *mem_access; +struct ffa_mem_transaction_int trans; +struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee; +struct ffa_shm_mem *shm =NULL ; +unsigned int last_page_idx =0 ; +register_t handle_hi =0 ; +register_t handle_lo =0 ; +int ret =FFA_RET_DENIED ; +uint32_t range_count; +uint32_t region_offs; + +/* +* We're only accepting memory transaction descriptors via the rx/tx +* buffer. +*/ +if ( addr ) + { + ret =FFA_RET_NOT_SUPPORTED ; + goto out_set_ret; + } + +/* Check that fragment length doesn't exceed total length */ +if ( frag_len > tot_len ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out_set_ret; + } + +/* We currently only support a single fragment */ +if ( frag_len != tot_len ) + { + ret =FFA_RET_NOT_SUPPORTED ; + goto out_set_ret; + } + +spin_lock (&ctx->lock); + +if ( frag_len > ctx->page_count *FFA_PAGE_SIZE ) + goto out_unlock; + + ret =read_mem_transaction (ctx->guest_vers, ctx->tx, frag_len, &trans); +if ( ret ) + goto out_unlock; + +if ( trans.mem_reg_attr !=FFA_NORMAL_MEM_REG_ATTR ) + { + ret =FFA_RET_NOT_SUPPORTED ; + goto out_unlock; + } + +/* Only supports sharing it with one SP for now */ +if ( trans.mem_access_count !=1 ) + { + ret =FFA_RET_NOT_SUPPORTED ; + goto out_unlock; + } + +if ( trans.sender_id !=get_vm_id (d) ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out_unlock; + } + +/* Check that it fits in the supplied data */ +if ( trans.mem_access_offs + trans.mem_access_size > frag_len ) + goto out_unlock; + + mem_access = ctx->tx + trans.mem_access_offs; +if (read_atomic (&mem_access->access_perm.perm) !=FFA_MEM_ACC_RW ) + { + ret =FFA_RET_NOT_SUPPORTED ; + goto out_unlock; + } + + region_offs =read_atomic (&mem_access->region_offs); +if (sizeof (*region_descr) + region_offs > frag_len ) + { + ret =FFA_RET_NOT_SUPPORTED ; + goto out_unlock; + } + + region_descr = ctx->tx + region_offs; + range_count =read_atomic (®ion_descr->address_range_count); + page_count =read_atomic (®ion_descr->total_page_count); + +if ( !page_count ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out_unlock; + } + + shm =alloc_ffa_shm_mem (ctx, page_count); +if ( !shm ) + { + ret =FFA_RET_NO_MEMORY ; + goto out_unlock; + } + shm->sender_id = trans.sender_id; + shm->ep_id =read_atomic (&mem_access->access_perm.endpoint_id); + +/* +* Check that the Composite memory region descriptor fits. +*/ +if (sizeof (*region_descr) + region_offs + + range_count *sizeof (struct ffa_address_range ) > frag_len ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out; + } + + ret =get_shm_pages (d, shm, region_descr->address_range_array, range_count, +0 , &last_page_idx); +if ( ret ) + goto out; +if ( last_page_idx != shm->page_count ) + { + ret =FFA_RET_INVALID_PARAMETERS ; + goto out; + } + +/* Note that share_shm() uses our tx buffer */ +spin_lock (&ffa_tx_buffer_lock); + ret =share_shm (shm); +spin_unlock (&ffa_tx_buffer_lock); +if ( ret ) + goto out; + +list_add_tail (&shm->list, &ctx->shm_list); + +uint64_to_regpair (&handle_hi, &handle_lo, shm->handle); + +out: +if ( ret ) +free_ffa_shm_mem (ctx, shm); +out_unlock: +spin_unlock (&ctx->lock); + +out_set_ret: +if ( ret ==0 ) +set_regs_success (regs, handle_lo, handle_hi); +else +set_regs_error (regs, ret); +} +static bool ffa_handle_call (struct cpu_user_regs *regs) {uint32_t fid =get_user_reg (regs,0 );
@@ -862,6 +1340,10 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)case FFA_MSG_SEND_DIRECT_REQ_64 :handle_msg_send_direct_req (regs, fid);return true; +case FFA_MEM_SHARE_32 : +case FFA_MEM_SHARE_64 : +handle_mem_share (regs); +return true;default :gprintk (XENLOG_ERR ,"ffa: unhandled fid 0x%x\n" , fid);
@@ -904,6 +1386,8 @@ static int ffa_domain_init(struct domain *d) } ctx->create_signal_count = subscr_vm_created_count; +INIT_LIST_HEAD (&ctx->shm_list); +return 0 ; }
@@ -1094,7 +1578,9 @@ static bool ffa_probe(void)if ( !check_mandatory_feature (FFA_PARTITION_INFO_GET ) || !check_mandatory_feature (FFA_RX_RELEASE ) || !check_mandatory_feature (FFA_RXTX_MAP_64 ) || + !check_mandatory_feature (FFA_MEM_SHARE_64 ) || !check_mandatory_feature (FFA_RXTX_UNMAP ) || + !check_mandatory_feature (FFA_MEM_SHARE_32 ) || !check_mandatory_feature (FFA_MSG_SEND_DIRECT_REQ_32 ) )return false;
Signature · 1 line
2.34.1
PATCH v11 10/14 xen/arm: ffa: add support to reclaim shared memory
In reply to Jens WiklanderAdds support to reclaim memory previously shared with FFA_MEM_SHARE.
A memory region that doesn't need to be shared any longer can be reclaimed with FFA_MEM_RECLAIM once the SP doesn't use it any longer. This is checked by the SPMC and not in control of the mediator.
Adds a check that the SP supports the needed FF-A feature FFA_MEM_RECLAIM.
---v10->v11
- No changes
--- xen/arch/arm/tee/ffa.c | 53 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+)
xen/arch/arm/tee/ffa.c +53 -0
@@ -538,6 +538,12 @@ static int32_t ffa_mem_share(uint32_t tot_len, uint32_t frag_len, } } +static int32_t ffa_mem_reclaim (uint32_t handle_lo,uint32_t handle_hi, +uint32_t flags) +{ +return ffa_simple_call (FFA_MEM_RECLAIM , handle_lo, handle_hi, flags,0 ); +} +static int32_t ffa_direct_req_send_vm (uint16_t sp_id,uint16_t vm_id,uint8_t msg) {
@@ -1283,6 +1289,43 @@ out_set_ret:set_regs_error (regs, ret); } +static int handle_mem_reclaim (uint64_t handle,uint32_t flags) +{ +struct domain *d = current->domain; +struct ffa_ctx *ctx = d->arch.tee; +struct ffa_shm_mem *shm; +register_t handle_hi; +register_t handle_lo; +int ret; + +spin_lock (&ctx->lock); +list_for_each_entry (shm, &ctx->shm_list, list) + { +if ( shm->handle == handle ) + goto found_it; + } + shm =NULL ; + ret =FFA_RET_INVALID_PARAMETERS ; + goto out; +found_it: + +uint64_to_regpair (&handle_hi, &handle_lo, handle); + ret =ffa_mem_reclaim (handle_lo, handle_hi, flags); +if ( ret ) + { + shm =NULL ; + goto out; + } + +list_del (&shm->list); + +out: +free_ffa_shm_mem (ctx, shm); +spin_unlock (&ctx->lock); + +return ret; +} +static bool ffa_handle_call (struct cpu_user_regs *regs) {uint32_t fid =get_user_reg (regs,0 );
@@ -1344,6 +1387,15 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)case FFA_MEM_SHARE_64 :handle_mem_share (regs);return true; +case FFA_MEM_RECLAIM : + e =handle_mem_reclaim (regpair_to_uint64 (get_user_reg (regs,2 ), +get_user_reg (regs,1 )), +get_user_reg (regs,3 )); +if ( e ) +set_regs_error (regs, e); +else +set_regs_success (regs,0 ,0 ); +return true;default :gprintk (XENLOG_ERR ,"ffa: unhandled fid 0x%x\n" , fid);
@@ -1581,6 +1633,7 @@ static bool ffa_probe(void) !check_mandatory_feature (FFA_MEM_SHARE_64 ) || !check_mandatory_feature (FFA_RXTX_UNMAP ) || !check_mandatory_feature (FFA_MEM_SHARE_32 ) || + !check_mandatory_feature (FFA_MEM_RECLAIM ) || !check_mandatory_feature(FFA_MSG_SEND_DIRECT_REQ_32) ) return false;
Signature · 1 line
2.34.1
PATCH v11 11/14 xen/arm: ffa: improve lock granularity
In reply to Jens WiklanderThe single lock in struct ffa_ctx is complemented with rx_lock and tx_lock.
The old lock is used for small critical sections, like increasing shm_count or adding another shm to shm_list.
rx_lock and tx_lock are only acquired using spin_trylock() which for well-behaving guests should always succeed. Guests using the RX and TX buffers are expected to serialize accesses before doing the FF-A request.
---v10->v11
- No changes except resolving conflicts since "xen/arm: ffa: support sharing large memory ranges" was dropped.
--- xen/arch/arm/tee/ffa.c | 107 +++++++++++++++++++++++++++++------------ 1 file changed, 77 insertions(+), 30 deletions(-)
xen/arch/arm/tee/ffa.c +77 -30
@@ -363,6 +363,13 @@ struct ffa_ctx {struct list_head shm_list;/* Number of allocated shared memory object */ unsigned int shm_count; +/* +* tx_lock is used to serialize access to tx +* rx_lock is used to serialize access to rx +* lock is used for the rest in this struct +*/ +spinlock_t tx_lock; +spinlock_t rx_lock;spinlock_t lock; };
@@ -768,7 +775,9 @@ static int32_t handle_partition_info_get(uint32_t w1, uint32_t w2, uint32_t w3,if ( !ffa_rx )return FFA_RET_DENIED ; -spin_lock (&ctx->lock); +if ( !spin_trylock (&ctx->rx_lock) ) +return FFA_RET_BUSY ; +if ( !ctx->page_count || !ctx->rx_is_free ) goto out;spin_lock (&ffa_rx_buffer_lock);
@@ -819,7 +828,7 @@ out_rx_release: out_rx_buf_unlock:spin_unlock (&ffa_rx_buffer_lock); out: -spin_unlock (&ctx->lock); +spin_unlock (&ctx->rx_lock);return ret; }
@@ -830,13 +839,15 @@ static int32_t handle_rx_release(void)struct domain *d = current->domain;struct ffa_ctx *ctx = d->arch.tee; -spin_lock (&ctx->lock); +if ( !spin_trylock (&ctx->rx_lock) ) +return FFA_RET_BUSY ; +if ( !ctx->page_count || ctx->rx_is_free ) goto out; ret =FFA_RET_OK ; ctx->rx_is_free = true; out: -spin_unlock (&ctx->lock); +spin_unlock (&ctx->rx_lock);return ret; }
@@ -947,21 +958,43 @@ static void put_shm_pages(struct ffa_shm_mem *shm) } } +static bool inc_ctx_shm_count (struct ffa_ctx *ctx) +{ +bool ret = true; + +spin_lock (&ctx->lock); +if (ctx->shm_count >=FFA_MAX_SHM_COUNT ) + ret = false; +else + ctx->shm_count++; +spin_unlock (&ctx->lock); + +return ret; +} + +static void dec_ctx_shm_count (struct ffa_ctx *ctx) +{ +spin_lock (&ctx->lock); +ASSERT (ctx->shm_count >0 ); + ctx->shm_count--; +spin_unlock (&ctx->lock); +} +static struct ffa_shm_mem *alloc_ffa_shm_mem (struct ffa_ctx *ctx,unsigned int page_count) {struct ffa_shm_mem *shm; -if ( page_count >=FFA_MAX_SHM_PAGE_COUNT || - ctx->shm_count >=FFA_MAX_SHM_COUNT ) +if ( page_count >=FFA_MAX_SHM_PAGE_COUNT ) +return NULL ; +if ( !inc_ctx_shm_count (ctx) )return NULL ; shm = xzalloc_flex_struct(struct ffa_shm_mem, pages, page_count);if ( shm ) - { - ctx->shm_count++; shm->page_count = page_count; - } +else +dec_ctx_shm_count (ctx);return shm; }
@@ -971,8 +1004,7 @@ static void free_ffa_shm_mem(struct ffa_ctx *ctx, struct ffa_shm_mem *shm)if ( !shm )return ; -ASSERT (ctx->shm_count >0 ); - ctx->shm_count--; +dec_ctx_shm_count (ctx);put_shm_pages (shm);xfree (shm); }
@@ -1180,7 +1212,11 @@ static void handle_mem_share(struct cpu_user_regs *regs) goto out_set_ret; } -spin_lock (&ctx->lock); +if ( !spin_trylock (&ctx->tx_lock) ) + { + ret =FFA_RET_BUSY ; + goto out_set_ret; + }if ( frag_len > ctx->page_count *FFA_PAGE_SIZE ) goto out_unlock;
@@ -1272,7 +1308,9 @@ static void handle_mem_share(struct cpu_user_regs *regs)if ( ret ) goto out; +spin_lock (&ctx->lock);list_add_tail (&shm->list, &ctx->shm_list); +spin_unlock (&ctx->lock);uint64_to_regpair (&handle_hi, &handle_lo, shm->handle);
@@ -1280,7 +1318,7 @@ out:if ( ret )free_ffa_shm_mem (ctx, shm); out_unlock: -spin_unlock (&ctx->lock); +spin_unlock (&ctx->tx_lock); out_set_ret:if ( ret ==0 )
@@ -1289,6 +1327,18 @@ out_set_ret:set_regs_error (regs, ret); } +/* Must only be called with ctx->lock held */ +static struct ffa_shm_mem *find_shm_mem (struct ffa_ctx *ctx,uint64_t handle) +{ +struct ffa_shm_mem *shm; + +list_for_each_entry (shm, &ctx->shm_list, list) +if ( shm->handle == handle ) +return shm; + +return NULL ; +} +static int handle_mem_reclaim (uint64_t handle,uint32_t flags) {struct domain *d = current->domain;
@@ -1299,29 +1349,26 @@ static int handle_mem_reclaim(uint64_t handle, uint32_t flags)int ret;spin_lock (&ctx->lock); -list_for_each_entry (shm, &ctx->shm_list, list) - { -if ( shm->handle == handle ) - goto found_it; - } - shm =NULL ; - ret =FFA_RET_INVALID_PARAMETERS ; - goto out; -found_it: + shm =find_shm_mem (ctx, handle); +if ( shm ) +list_del (&shm->list); +spin_unlock (&ctx->lock); +if ( !shm ) +return FFA_RET_INVALID_PARAMETERS ;uint64_to_regpair (&handle_hi, &handle_lo, handle); ret =ffa_mem_reclaim (handle_lo, handle_hi, flags); +if ( ret ) { - shm =NULL ; - goto out; +spin_lock (&ctx->lock); +list_add_tail (&shm->list, &ctx->shm_list); +spin_unlock (&ctx->lock); + } +else + { +free_ffa_shm_mem (ctx, shm); } - -list_del (&shm->list); - -out: -free_ffa_shm_mem (ctx, shm); -spin_unlock (&ctx->lock);return ret; }
Signature · 1 line
2.34.1
Re: [XEN PATCH v11 11/14] xen/arm: ffa: improve lock granularity
Hi Jens,
Quoted 7 lines · Jens Wiklander
The single lock in struct
ffa_ctxis complemented withrx_lockandtx_lock.The old lock is used for small critical sections, like increasing
shm_countor adding another shm toshm_list.
Quoted 6 lines
rx_lockandtx_lockare only acquired usingspin_trylock()which for well-behaving guests should always succeed. Guests using the RX and TX buffers are expected to serialize accesses before doing the FF-A request.Signed-off-by Jens Wiklander <[email protected]>
Cheers
Bertrand
Quoted 221 lines
---
v10->v11
- No changes except resolving conflicts since "xen/arm: ffa: support sharing large memory ranges" was dropped.
---xen/arch/arm/tee/ffa.c| 107 +++++++++++++++++++++++++++++------------1 file changed, 77 insertions(+), 30 deletions(-)xen/arch/arm/tee/ffa.c +10
-1@@ -363,6 +363,13 @@ struct ffa_ctx {struct list_head shm_list;/* Number of allocated shared memory object */ unsigned int shm_count; +/* +* tx_lock is used to serialize access to tx +* rx_lock is used to serialize access to rx +* lock is used for the rest in this struct +*/ +spinlock_t tx_lock; +spinlock_t rx_lock;spinlock_t lock; };@@ -768,7 +775,9 @@ static int32_t handle_partition_info_get(uint32_t w1, uint32_t w2, uint32_t w3,if ( !ffa_rx )return FFA_RET_DENIED ; -spin_lock (&ctx->lock); +if ( !spin_trylock (&ctx->rx_lock) ) +return FFA_RET_BUSY ; +if ( !ctx->page_count || !ctx->rx_is_free ) goto out;spin_lock (&ffa_rx_buffer_lock);@@ -819,7 +828,7 @@ out_rx_release:
out_rx_buf_unlock:spin_unlock(&ffa_rx_buffer_lock);out:
xen/arch/arm/tee/ffa.c +1
-1-spin_unlock (&ctx->lock); +spin_unlock (&ctx->rx_lock);return ret;}
xen/arch/arm/tee/ffa.c +3
-1@@ -830,13 +839,15 @@ static int32_t handle_rx_release(void)struct domain *d = current->domain;struct ffa_ctx *ctx = d->arch.tee; -spin_lock (&ctx->lock); +if ( !spin_trylock (&ctx->rx_lock) ) +return FFA_RET_BUSY ; +if ( !ctx->page_count || ctx->rx_is_free ) goto out; ret =FFA_RET_OK ; ctx->rx_is_free = true;out:
xen/arch/arm/tee/ffa.c +1
-1-spin_unlock (&ctx->lock); +spin_unlock (&ctx->rx_lock);return ret;}
xen/arch/arm/tee/ffa.c +35
-8@@ -947,21 +958,43 @@ static void put_shm_pages(struct ffa_shm_mem *shm) } } +static bool inc_ctx_shm_count (struct ffa_ctx *ctx) +{ +bool ret = true; + +spin_lock (&ctx->lock); +if (ctx->shm_count >=FFA_MAX_SHM_COUNT ) + ret = false; +else + ctx->shm_count++; +spin_unlock (&ctx->lock); + +return ret; +} + +static void dec_ctx_shm_count (struct ffa_ctx *ctx) +{ +spin_lock (&ctx->lock); +ASSERT (ctx->shm_count >0 ); + ctx->shm_count--; +spin_unlock (&ctx->lock); +} +static struct ffa_shm_mem *alloc_ffa_shm_mem (struct ffa_ctx *ctx,unsigned int page_count) {struct ffa_shm_mem *shm; -if ( page_count >=FFA_MAX_SHM_PAGE_COUNT || - ctx->shm_count >=FFA_MAX_SHM_COUNT ) +if ( page_count >=FFA_MAX_SHM_PAGE_COUNT ) +return NULL ; +if ( !inc_ctx_shm_count (ctx) )return NULL ; shm = xzalloc_flex_struct(struct ffa_shm_mem, pages, page_count);if ( shm ) - { - ctx->shm_count++; shm->page_count = page_count; - } +else +dec_ctx_shm_count (ctx);return shm; }@@ -971,8 +1004,7 @@ static void free_ffa_shm_mem(struct ffa_ctx *ctx, struct ffa_shm_mem *shm)if ( !shm )return ; -ASSERT (ctx->shm_count >0 ); - ctx->shm_count--; +dec_ctx_shm_count (ctx);put_shm_pages (shm);xfree (shm); }@@ -1180,7 +1212,11 @@ static void handle_mem_share(struct cpu_user_regs *regs) goto out_set_ret; } -spin_lock (&ctx->lock); +if ( !spin_trylock (&ctx->tx_lock) ) + { + ret =FFA_RET_BUSY ; + goto out_set_ret; + }if ( frag_len > ctx->page_count *FFA_PAGE_SIZE ) goto out_unlock;@@ -1272,7 +1308,9 @@ static void handle_mem_share(struct cpu_user_regs *regs)if ( ret ) goto out; +spin_lock (&ctx->lock);list_add_tail (&shm->list, &ctx->shm_list); +spin_unlock (&ctx->lock);uint64_to_regpair (&handle_hi, &handle_lo, shm->handle);@@ -1280,7 +1318,7 @@ out:if ( ret )free_ffa_shm_mem (ctx, shm);
out_unlock:xen/arch/arm/tee/ffa.c +1
-1-spin_unlock (&ctx->lock); +spin_unlock (&ctx->tx_lock);
out_set_ret:if ( ret == 0)xen/arch/arm/tee/ffa.c +26
-17@@ -1289,6 +1327,18 @@ out_set_ret:set_regs_error (regs, ret); } +/* Must only be called with ctx->lock held */ +static struct ffa_shm_mem *find_shm_mem (struct ffa_ctx *ctx,uint64_t handle) +{ +struct ffa_shm_mem *shm; + +list_for_each_entry (shm, &ctx->shm_list, list) +if ( shm->handle == handle ) +return shm; + +return NULL ; +} +static int handle_mem_reclaim (uint64_t handle,uint32_t flags) {struct domain *d = current->domain;@@ -1299,29 +1349,26 @@ static int handle_mem_reclaim(uint64_t handle, uint32_t flags)int ret;spin_lock (&ctx->lock); -list_for_each_entry (shm, &ctx->shm_list, list) - { -if ( shm->handle == handle ) - goto found_it; - } - shm =NULL ; - ret =FFA_RET_INVALID_PARAMETERS ; - goto out; -found_it: + shm =find_shm_mem (ctx, handle); +if ( shm ) +list_del (&shm->list); +spin_unlock (&ctx->lock); +if ( !shm ) +return FFA_RET_INVALID_PARAMETERS ;uint64_to_regpair (&handle_hi, &handle_lo, handle); ret =ffa_mem_reclaim (handle_lo, handle_hi, flags); +if ( ret ) { - shm =NULL ; - goto out; +spin_lock (&ctx->lock); +list_add_tail (&shm->list, &ctx->shm_list); +spin_unlock (&ctx->lock); + } +else + { +free_ffa_shm_mem (ctx, shm); } - -list_del (&shm->list); - -out: -free_ffa_shm_mem (ctx, shm); -spin_unlock (&ctx->lock);return ret; }Signature · 1 line
2.34.1
PATCH v11 12/14 xen/arm: ffa: list current limitations
In reply to Jens WiklanderAdds comments with a list of unsupported FF-A interfaces and limitations in the implemented FF-A interfaces.
---v10->v11
- No changes
--- xen/arch/arm/tee/ffa.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+)
xen/arch/arm/tee/ffa.c +31 -0
@@ -13,6 +13,37 @@* https://developer.arm.com/documentation/den0077/e * TEEC-1.0C: TEE Client API Specification version 1.0c available at * https://globalplatform.org/specs-library/tee-client-api-specification/ +* +* Notes on the the current implementation. +* +* Unsupported FF-A interfaces: +* o FFA_MSG_POLL and FFA_MSG_SEND - deprecated in FF-A-1.1-REL0 +* o FFA_MEM_RETRIEVE_* - Used when sharing memory from an SP to a VM +* o FFA_MEM_DONATE_* and FFA_MEM_LEND_* - Used when tranferring ownership +* or access of a memory region +* o FFA_MSG_SEND2 and FFA_MSG_WAIT - Used for indirect messaging +* o FFA_MSG_YIELD +* o FFA_INTERRUPT - Used to report preemption +* o FFA_RUN +* +* Limitations in the implemented FF-A interfaces: +* o FFA_RXTX_MAP_*: +* - Maps only one 4k page as RX and TX buffers +* - Doesn't support forwarding this call on behalf of an endpoint +* o FFA_MEM_SHARE_*: only supports sharing +* - from a VM to an SP +* - with one borrower +* - with the memory transaction descriptor in the RX/TX buffer +* - normal memory +* - at most 512 kB large memory regions +* - at most 32 shared memory regions per guest +* o FFA_MSG_SEND_DIRECT_REQ: +* - only supported from a VM to an SP +* +* There are some large locked sections with ffa_tx_buffer_lock and +* ffa_rx_buffer_lock. Especially the ffa_tx_buffer_lock spinlock used +* around share_shm() is a very large locked section which can let one VM +* affect another VM. */ #include <xen/bitops.h>
Signature · 1 line
2.34.1
Re: [XEN PATCH v11 12/14] xen/arm: ffa: list current limitations
Hi Jens,
Quoted 7 lines · Jens Wiklander
Adds comments with a list of unsupported FF-A interfaces and limitations in the implemented FF-A interfaces.
Signed-off-by Jens Wiklander <[email protected]>Reviewed-by Henry Wang <[email protected]>
Cheers
Bertrand
Quoted 52 lines
---
v10->v11
- No changes
---xen/arch/arm/tee/ffa.c| 31 +++++++++++++++++++++++++++++++1 file changed, 31 insertions(+)xen/arch/arm/tee/ffa.c +31
-0@@ -13,6 +13,37 @@* https://developer.arm.com/documentation/den0077/e * TEEC-1.0C: TEE Client API Specification version 1.0c available at * https://globalplatform.org/specs-library/tee-client-api-specification/ +* +* Notes on the the current implementation. +* +* Unsupported FF-A interfaces: +* o FFA_MSG_POLL and FFA_MSG_SEND - deprecated in FF-A-1.1-REL0 +* o FFA_MEM_RETRIEVE_* - Used when sharing memory from an SP to a VM +* o FFA_MEM_DONATE_* and FFA_MEM_LEND_* - Used when tranferring ownership +* or access of a memory region +* o FFA_MSG_SEND2 and FFA_MSG_WAIT - Used for indirect messaging +* o FFA_MSG_YIELD +* o FFA_INTERRUPT - Used to report preemption +* o FFA_RUN +* +* Limitations in the implemented FF-A interfaces: +* o FFA_RXTX_MAP_*: +* - Maps only one 4k page as RX and TX buffers +* - Doesn't support forwarding this call on behalf of an endpoint +* o FFA_MEM_SHARE_*: only supports sharing +* - from a VM to an SP +* - with one borrower +* - with the memory transaction descriptor in the RX/TX buffer +* - normal memory +* - at most 512 kB large memory regions +* - at most 32 shared memory regions per guest +* o FFA_MSG_SEND_DIRECT_REQ: +* - only supported from a VM to an SP +* +* There are some large locked sections with ffa_tx_buffer_lock and +* ffa_rx_buffer_lock. Especially the ffa_tx_buffer_lock spinlock used +* around share_shm() is a very large locked section which can let one VM +* affect another VM. */ #include <xen/bitops.h> Signature · 1 line
2.34.1
PATCH v11 13/14 tools: add Arm FF-A mediator
In reply to Jens WiklanderAdds a new "ffa" value to the Enumeration "tee_type" to indicate if a guest is trusted to use FF-A.
---v10->v11
- No changes
--- tools/include/libxl.h | 5 +++++ tools/libs/light/libxl_arm.c | 3 +++ tools/libs/light/libxl_types.idl | 3 ++- 3 files changed, 10 insertions(+), 1 deletion(-)
tools/include/libxl.h +5 -0
@@ -283,6 +283,11 @@*/ #define LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE 1 +/* +* arch_arm.tee field in libxl_domain_build_info has ffa value. +*/ +#define LIBXL_HAVE_BUILDINFO_ARCH_ARM_TEE_FFA 1 +/* * libxl_domain_build_info has the arch_arm.sve_vl field. */
tools/libs/light/libxl_arm.c +3 -0
@@ -207,6 +207,9 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,case LIBXL_TEE_TYPE_OPTEE : config->arch.tee_type =XEN_DOMCTL_CONFIG_TEE_OPTEE ;break ; +case LIBXL_TEE_TYPE_FFA : + config->arch.tee_type =XEN_DOMCTL_CONFIG_TEE_FFA ; +break ;default :LOG (ERROR ,"Unknown TEE type %d" , d_config->b_info.tee);
tools/libs/light/libxl_types.idl +2 -1
@@ -520,7 +520,8 @@ libxl_gic_version = Enumeration("gic_version", [ libxl_tee_type = Enumeration("tee_type", [ (0, "none"), - (1, "optee") + (1, "optee"), + (2, "ffa"), ], init_val = "LIBXL_TEE_TYPE_NONE") libxl_sve_type = Enumeration("sve_type", [
Signature · 1 line
2.34.1
Re: [XEN PATCH v11 13/14] tools: add Arm FF-A mediator
Quoted 6 lines · Jens Wiklander
Adds a new "ffa" value to the Enumeration "
tee_type" to indicate if a guest is trusted to use FF-A.Signed-off-by Jens Wiklander <[email protected]>Reviewed-by Bertrand Marquis <[email protected]>Reviewed-by Henry Wang <[email protected]>
Thanks,
Signature · 1 line
Anthony PERARD
PATCH v11 14/14 docs: add Arm FF-A mediator
In reply to Jens WiklanderDescribes a FF-A version 1.1 [1] mediator to communicate with a Secure Partition in secure world.
[1] https://developer.arm.com/documentation/den0077/latest
---v10->v11
- Add a CHANGELOG.md entry as requested
--- CHANGELOG.md | 3 ++- SUPPORT.md | 9 +++++++++ docs/man/xl.cfg.5.pod.in | 15 +++++++++++++++ 3 files changed, 26 insertions(+), 1 deletion(-)
CHANGELOG.md +2 -1
@@ -24,7 +24,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - xl/libxl can customize SMBIOS strings for HVM guests. - Add support for AVX512-FP16 on x86. - On Arm, Xen supports guests running SVE/SVE2 instructions. (Tech Preview) - + - On Arm, add suport for Firmware Framework for Arm A-profile (FF-A) Mediator + (Tech Preview) ## [4.17.0](https://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=RELEASE-4.17.0) - 2022-12-12
SUPPORT.md +9 -0
@@ -840,6 +840,15 @@ that covers the DMA of the device to be passed through. No support for QEMU backends in a 16K or 64K domain. +### ARM: Firmware Framework for Arm A-profile (FF-A) Mediator + + Status, Arm64: Tech Preview + +There are still some code paths where a vCPU may hog a pCPU longer than +necessary. The FF-A mediator is not yet implemented for Arm32. Part of the +FF-A specification is not supported, see the top comment in +xen/arch/arm/tee/ffa.c for limitations. + ### ARM: Guest Device Tree support Status: Supported
docs/man/xl.cfg.5.pod.in +15 -0
@@ -1656,6 +1656,21 @@ in OP-TEE. This feature is a B<technology preview>. +=item B<ffa> + +B<Arm only.> Allow a guest to communicate via FF-A with Secure Partitions +(SP), default false. + +Currently only a small subset of the FF-A specification is supported. Just +enough to communicate with OP-TEE. In general only direct messaging and +sharing memory with one SP. More advanced use cases where memory might be +shared or donated to multiple SPs are not supported. + +See L<https://developer.arm.com/documentation/den0077/latest> for more +information about FF-A. + +This feature is a B<technology preview>. + =back =back
Signature · 1 line
2.34.1
Re: [XEN PATCH v11 14/14] docs: add Arm FF-A mediator
Quoted 6 lines · Jens Wiklander
Describes a FF-A version 1.1 [1] mediator to communicate with a Secure Partition in secure world.
[1] https://developer.arm.com/documentation/den0077/latest
Signed-off-by Jens Wiklander <[email protected]>Reviewed-by Henry Wang <[email protected]>
Thanks,
Signature · 1 line
Anthony PERARD
Re: [XEN PATCH v11 14/14] docs: add Arm FF-A mediator
In reply to Jens WiklanderHi Jens,
Quoted 8 lines · Jens Wiklander
Describes a FF-A version 1.1 [1] mediator to communicate with a Secure Partition in secure world.
[1] https://developer.arm.com/documentation/den0077/latest
Signed-off-by Jens Wiklander <[email protected]>Reviewed-by Henry Wang <[email protected]>
Cheers
Bertrand
Quoted 72 lines
---
v10->v11
- Add a CHANGELOG.md entry as requested
---CHANGELOG.md | 3 ++- SUPPORT.md | 9 +++++++++ docs/man/xl.cfg.5.pod.in | 15 +++++++++++++++ 3 files changed, 26 insertions(+), 1 deletion(-)CHANGELOG.md +2
-1@@ -24,7 +24,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - xl/libxl can customize SMBIOS strings for HVM guests. - Add support for AVX512-FP16 on x86. - On Arm, Xen supports guests running SVE/SVE2 instructions. (Tech Preview) - + - On Arm, add suport for Firmware Framework for Arm A-profile (FF-A) Mediator + (Tech Preview) ## [4.17.0](https://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=RELEASE-4.17.0) - 2022-12-12SUPPORT.md +0
-0@@ -840,6 +840,15 @@ that covers the DMA of the device to be passed through.No support for QEMU backends in a 16K or 64K domain.
SUPPORT.md +9
-0+### ARM: Firmware Framework for Arm A-profile (FF-A) Mediator + + Status, Arm64: Tech Preview + +There are still some code paths where a vCPU may hog a pCPU longer than +necessary. The FF-A mediator is not yet implemented for Arm32. Part of the +FF-A specification is not supported, see the top comment in +xen/arch/arm/tee/ffa.c for limitations. +### ARM: Guest Device Tree support
Status: Supported
docs/man/xl.cfg.5.pod.in +0
-0@@ -1656,6 +1656,21 @@ in OP-TEE.This feature is a B<technology preview>.
+=item B<ffa>
docs/man/xl.cfg.5.pod.in +14
-0+ +B<Arm only.> Allow a guest to communicate via FF-A with Secure Partitions +(SP), default false. + +Currently only a small subset of the FF-A specification is supported. Just +enough to communicate with OP-TEE. In general only direct messaging and +sharing memory with one SP. More advanced use cases where memory might be +shared or donated to multiple SPs are not supported. + +See L<https://developer.arm.com/documentation/den0077/latest> for more +information about FF-A. + +This feature is a B<technology preview>. +=back
=back
Signature · 1 line
2.34.1
Hi Jens,
Quoted 7 lines · Jens Wiklander
Jens Wiklander (14):
xen/arm: ffa: add direct request support
xen/arm: ffa: map SPMC rx/tx buffers
xen/arm: ffa: send guest events to Secure Partitions xen/arm: ffa: support mapping guest RX/TX buffers xen/arm: ffa: support guestFFA_PARTITION_INFO_GETxen/arm: moveregpair_to_uint64()anduint64_to_regpair()toregs.h
Quoted 8 lines
xen/arm: ffa: add defines for sharing memory xen/arm: ffa: add ABI structs for sharing memory xen/arm: ffa: support sharing memory xen/arm: ffa: add support to reclaim shared memory xen/arm: ffa: improve lock granularity xen/arm: ffa: list current limitations
tools: add Arm FF-A mediator
docs: add Arm FF-A mediator
The series is now committed. Note, I had to resolve a context conflict in the CHANGELOG.md.
Cheers,
Signature · 1 line
Julien Grall
Hi Julien,
Quoted 20 lines · Julien Grall
Hi Jens,
Quoted 7 lines · Jens Wiklander
Jens Wiklander (14):
xen/arm: ffa: add direct request support
xen/arm: ffa: map SPMC rx/tx buffers
xen/arm: ffa: send guest events to Secure Partitions xen/arm: ffa: support mapping guest RX/TX buffers xen/arm: ffa: support guestFFA_PARTITION_INFO_GETxen/arm: moveregpair_to_uint64()anduint64_to_regpair()toregs.hQuoted 8 lines
xen/arm: ffa: add defines for sharing memory xen/arm: ffa: add ABI structs for sharing memory xen/arm: ffa: support sharing memory xen/arm: ffa: add support to reclaim shared memory xen/arm: ffa: improve lock granularity xen/arm: ffa: list current limitations
tools: add Arm FF-A mediator
docs: add Arm FF-A mediator
Quoted 2 lines
The series is now committed. Note, I had to resolve a context conflict in the CHANGELOG.md.
Thanks for the update, and a big thank you to all of you who have helped me to get this into a committable state.
Cheers,
Jens
Quoted 5 lines
Cheers,
--
Julien Grall