Argo: hypervisor-mediated interdomain communication
v5 cover Argo: hypervisor-mediated interdomain communication
Version five of this patch series:
* Changes are primarily addressing feedback from the v4 series reviews. Many points noted on the invididual commit posts.
* Critical sections have been shrunk, with allocations and frees pulled outside where possible, reordering logic within hypercall ops.
* A new ring hash function implemented, derived from the djb2 string hash function.
* Flags returned by the notify op have been simplified.
* Now uses a single argo boot parameter, taking a list:
- top level boolean to enable/disable Argo
- mac-permissive option to enable/disable wildcard rings
- command line doc edit: no "CONFIG_ARGO" but refers to build config
* Switched to use the standard list data structures used by Xen's common code.
* Further removal of uses of fixed-width types.
* Added a new patch to add Argo to the MAINTAINERS file.
Christopher Clark (15):
argo: Introduce the Kconfig option to govern inclusion of Argo
argo: introduce the argo_op hypercall boilerplate
argo: define argo_dprintk for subsystem debugging
argo: init, destroy and soft-reset, with enable command line opt
errno: add POSIX error codes EMSGSIZE, ECONNREFUSED to the ABI xen/arm: introduce guest_handle_for_field()
argo: implement the register op
argo: implement the unregister op
argo: implement the sendv op; evtchn: expose send_guest_global_virq
argo: implement the notify op
xsm, argo: XSM control for argo register
xsm, argo: XSM control for argo message send operation xsm, argo: XSM control for any access to argo by a domain xsm, argo: notify: don't describe rings that cannot be sent to
MAINTAINERS: add new section for Argo and self as maintainer
MAINTAINERS | 8 + docs/misc/xen-command-line.pandoc | 22 + tools/flask/policy/modules/guest_features.te | 7 + xen/arch/x86/guest/hypercall_page.S | 2 +- xen/arch/x86/hvm/hypercall.c | 3 + xen/arch/x86/hypercall.c | 3 + xen/arch/x86/pv/hypercall.c | 3 + xen/common/Kconfig | 19 + xen/common/Makefile | 3 +- xen/common/argo.c | 2281 ++++++++++++++++++++++++++ xen/common/compat/argo.c | 62 + xen/common/domain.c | 9 + xen/common/event_channel.c | 2 +- xen/include/Makefile | 1 + xen/include/asm-arm/guest_access.h | 3 + xen/include/public/argo.h | 280 ++++ xen/include/public/errno.h | 2 + xen/include/public/xen.h | 4 +- xen/include/xen/argo.h | 44 + xen/include/xen/event.h | 7 + xen/include/xen/hypercall.h | 9 + xen/include/xen/sched.h | 5 + xen/include/xlat.lst | 8 + xen/include/xsm/dummy.h | 25 + xen/include/xsm/xsm.h | 31 + xen/xsm/dummy.c | 6 + xen/xsm/flask/hooks.c | 41 +- xen/xsm/flask/policy/access_vectors | 16 + xen/xsm/flask/policy/security_classes | 1 + 29 files changed, 2899 insertions(+), 8 deletions(-) create mode 100644 xen/common/argo.c create mode 100644 xen/common/compat/argo.c create mode 100644 xen/include/public/argo.h create mode 100644 xen/include/xen/argo.h
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 1/15 argo: Introduce the Kconfig option to govern inclusion of Argo
Defines CONFIG_ARGO when enabled. Default: disabled.
When the Kconfig option is enabled, the Argo hypercall implementation will be included, allowing use of the hypervisor-mediated interdomain communication mechanism.
Argo is implemented for x86 and ARM hardware platforms.
Availability of the option depends on EXPERT and Argo is currently an experimental feature.
---
v3 added Jan's Ack
v2 #01 feedback, Jan: replace def_bool/prompt with bool
v1 #02 feedback, Jan: default Kconfig off, use EXPERT, fix whitespace
xen/common/Kconfig | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
xen/common/Kconfig +19 -0
@@ -202,6 +202,25 @@ config LATE_HWDOM If unsure, say N. +config ARGO + bool "Argo: hypervisor-mediated interdomain communication" if EXPERT = "y" + ---help--- + Enables a hypercall for domains to ask the hypervisor to perform + data transfer of messages between domains. + + This allows communication channels to be established that do not + require any shared memory between domains; the hypervisor is the + entity that each domain interacts with. The hypervisor is able to + enforce Mandatory Access Control policy over the communication. + + If XSM_FLASK is enabled, XSM policy can govern which domains may + communicate via the Argo system. + + This feature does nothing if the "argo" boot parameter is not present. + Argo is disabled at runtime by default. + + If unsure, say N. + menu "Schedulers" visible if EXPERT = "y"
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 2/15 argo: introduce the argo_op hypercall boilerplate
In reply to Christopher ClarkPresence is gated upon CONFIG_ARGO.
Registers the hypercall previously reserved for this. Takes 5 arguments, does nothing and returns -ENOSYS.
Will be avoiding a compat ABI by using fixed-size types in hypercall ops so HYPERCALL, rather than COMPAT_CALL, is the correct macro for the hypercall tables.
Even though handles will be used for (up to) two of the arguments to the hypercall, there will be no need for any XLAT_* translation functions because the referenced data structures have been constructed to be exactly the same size and bit pattern on both 32-bit and 64-bit guests, and padded to be integer multiples of 32 bits in size. This means that the same copy_to_guest and copy_from_guest logic can be relied upon to perform as required without any further intervention. Testing communication with 32 and 64 bit guests has confirmed this works as intended.
---
v2 Copyright line: add 2019
v2 feedback #3 Jan: drop "message" from argo_message_op v2 feedback #3 Jan: add Acked-by
v1 feedback #15 Jan: handle upper-halves of hypercall args v1 feedback #15 Jan: use unsigned where negative values impossible
xen/arch/x86/guest/hypercall_page.S | 2 +- xen/arch/x86/hvm/hypercall.c | 3 +++ xen/arch/x86/hypercall.c | 3 +++ xen/arch/x86/pv/hypercall.c | 3 +++ xen/common/Makefile | 1 + xen/common/argo.c | 28 ++++++++++++++++++++++++++++ xen/include/public/xen.h | 2 +- xen/include/xen/hypercall.h | 9 +++++++++ 8 files changed, 49 insertions(+), 2 deletions(-) create mode 100644 xen/common/argo.c
xen/arch/x86/guest/hypercall_page.S +1 -1
@@ -59,7 +59,7 @@ DECLARE_HYPERCALL(sysctl) DECLARE_HYPERCALL(domctl) DECLARE_HYPERCALL(kexec_op) DECLARE_HYPERCALL(tmem_op) -DECLARE_HYPERCALL(xc_reserved_op) +DECLARE_HYPERCALL(argo_op) DECLARE_HYPERCALL(xenpmu_op) DECLARE_HYPERCALL(arch_0)
xen/arch/x86/hvm/hypercall.c +3 -0
@@ -134,6 +134,9 @@ static const hypercall_table_t hvm_hypercall_table[] = {#ifdef CONFIG_TMEM HYPERCALL(tmem_op),#endif +#ifdef CONFIG_ARGO + HYPERCALL(argo_op), +#endif COMPAT_CALL(platform_op),#ifdef CONFIG_PV COMPAT_CALL(mmuext_op),
xen/arch/x86/hypercall.c +3 -0
@@ -64,6 +64,9 @@ const hypercall_args_t hypercall_args_table[NR_hypercalls] =ARGS (domctl,1 ),ARGS (kexec_op,2 ),ARGS (tmem_op,1 ), +#ifdef CONFIG_ARGO +ARGS (argo_op,5 ), +#endifARGS (xenpmu_op,2 ), #ifdef CONFIG_HVM ARGS(hvm_op, 2),
xen/arch/x86/pv/hypercall.c +3 -0
@@ -77,6 +77,9 @@ const hypercall_table_t pv_hypercall_table[] = {#ifdef CONFIG_TMEM HYPERCALL(tmem_op),#endif +#ifdef CONFIG_ARGO + HYPERCALL(argo_op), +#endif HYPERCALL(xenpmu_op),#ifdef CONFIG_HVM HYPERCALL(hvm_op),
xen/common/Makefile +1 -0
@@ -1,3 +1,4 @@ +obj-$(CONFIG_ARGO) += argo.o obj-y += bitmap.o obj-y += bsearch.o obj-$(CONFIG_CORE_PARKING) += core_parking.o
xen/common/argo.c new file +28 -0
@@ -0,0 +1,28 @@ +/****************************************************************************** +* Argo : Hypervisor-Mediated data eXchange +* +* Derived from v4v, the version 2 of v2v. +* +* Copyright (c) 2010, Citrix Systems +* Copyright (c) 2018-2019 BAE Systems +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +*/ + +#include <xen/errno.h> +#include <xen/guest_access.h> + +long +do_argo_op (unsigned int cmd,XEN_GUEST_HANDLE_PARAM (void ) arg1, +XEN_GUEST_HANDLE_PARAM (void ) arg2,unsigned long arg3, +unsigned long arg4) +{ +return -ENOSYS ; +}
xen/include/public/xen.h +1 -1
@@ -118,7 +118,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);#define __HYPERVISOR_domctl 36#define __HYPERVISOR_kexec_op 37#define __HYPERVISOR_tmem_op 38 -#define __HYPERVISOR_xc_reserved_op 39/* reserved for XenClient */ +#define __HYPERVISOR_argo_op 39#define __HYPERVISOR_xenpmu_op 40#define __HYPERVISOR_dm_op 41
xen/include/xen/hypercall.h +9 -0
@@ -136,6 +136,15 @@ do_tmem_op( XEN_GUEST_HANDLE_PARAM(tmem_op_t) uops);#endif +#ifdef CONFIG_ARGO +extern long do_argo_op ( +unsigned int cmd, +XEN_GUEST_HANDLE_PARAM (void ) arg1, +XEN_GUEST_HANDLE_PARAM (void ) arg2, +unsigned long arg3, +unsigned long arg4); +#endif +extern long do_xenoprof_op (int op,XEN_GUEST_HANDLE_PARAM (void ) arg);
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 3/15 argo: define argo_dprintk for subsystem debugging
In reply to Christopher ClarkA convenience for working on development of the argo subsystem:
setting a #define variable enables additional debug messages.
---
v3 added Roger's Reviewed-by
v3 added Jan's Ack
v2 #03 feedback, Jan: fix ifdef/define confusion error v1 #04 feedback, Jan: fix dprintk implementation
xen/common/argo.c | 9 +++++++++ 1 file changed, 9 insertions(+)
xen/common/argo.c +9 -0
@@ -19,6 +19,15 @@#include <xen/errno.h> #include <xen/guest_access.h> +/* Change this to #define ARGO_DEBUG here to enable more debug messages */ +#undef ARGO_DEBUG + +#ifdef ARGO_DEBUG +#define argo_dprintk(format, args...) printk("argo: " format, ## args ) +#else +#define argo_dprintk (format, ... ) ((void)0) +#endif + long do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, XEN_GUEST_HANDLE_PARAM(void) arg2, unsigned long arg3,
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 4/15 argo: init, destroy and soft-reset, with enable command line opt
In reply to Christopher ClarkInitialises basic data structures and performs teardown of argo state for domain shutdown.
Inclusion of the Argo implementation is dependent on CONFIG_ARGO.
Introduces a new Xen command line parameter 'argo': bool to enable/disable the argo hypercall. Defaults to disabled.
New headers:
public/argo.h: with definions of addresses and ring structure, including indexes for atomic update for communication between domain and hypervisor.
xen/argo.h: to expose the hooks for integration into domain lifecycle:
argo_init: per-domain init of argo data structures for domain_create.
argo_destroy: teardown for domain_destroy and the error exit
path of domain_create.
argo_soft_reset: reset of domain state for domain_soft_reset.
Adds a new field to struct domain: struct argo_domain *argo;
In accordance with recent work on _domain_destroy, argo_destroy is idempotent. It will tear down: all rings registered by this domain, all rings where this domain is the single sender (ie. specified partner, non-wildcard rings), and all pending notifications where this domain is awaiting signal about available space in the rings of other domains.
A count will be maintained of the number of rings that a domain has registered in order to limit it below the fixed maximum limit defined here.
Macros are defined to verify the internal locking state within the argo implementation. The macros are ASSERTed on entry to functions to validate and document the required lock state prior to calling.
The hash function for the hashtables that hold ring state is derived from the string hashing function djb2 (http://www.cse.yorku.ca/~oz/hash.html)
by Daniel J. Bernstein. Basic testing with a limited number of domains and ports has shown reasonable distribution for the table size.
The software license on the public header is the BSD license, standard procedure for the public Xen headers. The public header was originally posted under a GPL license at: [1]:
https://lists.xenproject.org/archives/html/xen-devel/2013-05/msg02710.html
The following ACK by Lars Kurth is to confirm that only people being employees of Citrix contributed to the header files in the series posted at
[1] and that thus the copyright of the files in question is fully owned by Citrix. The ACK also confirms that Citrix is happy for the header files to be published under a BSD license in this series (which is based on [1]).
---
v4 Jan: amend the command line doc text referring to build configuration v4 feedback: use standard data structures as per common code
v4 Jan: replace hash_index with djb2-derived hash algorithm
v4 Andrew: switch argo command line option to list argo=<bool>
v4: removed note to remove argo_destroy from domain_kill (test shows issue) v4 #04 Roger: drop unneeded init of ring_count in argo_domain_init v4 #04 Roger: replace if (ring_info->mfns) with ASSERTs in ring_unmap v4 #04 Roger: rewrite the locking verification macros
v4 #04 Roger: make L1 lock description comment clearer about R(L1) and W(L1) v4 Andrew: fix split of dprintk in ring_map_info across v4 commits
v3 #04 Andrew: use xzalloc for struct argo_domain in argo_init v3 #04 Andrew: reference CONFIG_ARGO in the command line documentation v3 #07 Jan: rename ring_find_info to find_ring_info
v3 #04 Andrew: don't truncate args do_argo_op printk
v3 #07 Jan: fix numeric entries in printk format strings
v3 #10 Roger: move find functions to top of file and drop prototypes v3 #04 Jan: meld compat check for hypercall arg types
v3 #04 Roger/Jan: make lock names clearer and assert their state v3 #04 Jan: port -> aport with type; distinguish argo port from evtchn v3 #04 Jan: reorder call to argo_init_domain in argo_init
v3 #04 Jan: ring_remove_mfns: zero count before freeing arrays v3 #04 Jason/Roger: soft_reset: can assume reinit is ok if d->argo set v3 #04 Roger: remove unused and confusing d->argo_lock
v3 #04 Roger: add simple inlines in xen/argo.h, drop ifdef CONFIG_ARGO v3 #04 Roger: simpler return -EOPNOTSUPP in do_argo_op
v3 #04 Roger: add const to domain arg to ring_remove_info
v3 #04 Roger: use XFREE
v3 #04 Roger: newline fix in wildcard_pending_list_remove
v3 #04 Roger: mfn_mapping: void* instead of uint8_t*
v3 #04 Roger: drop npages struct member in argo_ring_info; use len v3 #04 Roger/Jan: drop many fixed width types in internal structs v3 #04 Jason/Jan: drop pad and fixed width type in pending_ent struct v3 #04 Eric: moved ring_find_info from register op into this commit v3 moved hash_index function, nospec include from register op to this commit v3 moved XEN_ARGO_DOMID_ANY defn from register op into this commit v3 added #include <xen/sched.h> to <xen/argo.h> for domain struct defn v3 feedback #04 Roger: reorder #includes to alphabetical order v3 Added Ross's Reviewed-by.
v2 rewrite locking explanation comment
v2 header copyright line now includes 2019
v2 self: use ring_info backpointer in pending_ent to maintain npending v2 self: rename all_rings_remove_info to domain_rings_remove_all
v2 feedback Jan: drop cookie, implement teardown
v2 self: add npending to track number of pending entries per ring v2 self: amend comment on locking; drop section comments
v2 cookie_eq: test low bits first and use likely on high bits
v2 self: OVERHAUL
v2 self: s/argo_pending_ent/pending_ent/g
v2 self: drop pending_remove_ent, inline at single call site
v1 feedback Roger, Jan: drop argo prefix on static functions
v2 #4 Lars: add Acked-by and details to commit message.
v2 feedback #9 Jan: document argo boot opt in xen-command-line.markdown v2 bugfix: xsm use in soft-reset prior to introduction
v2 feedback #9 Jan: drop 'message' from do_argo_message_op
v1 #5 feedback Paul: init/destroy unsigned, brackets and whitespace fixes v1 #5 feedback Paul: Use mfn_eq for comparing mfns.
v1 #5 feedback Paul: init/destroy : use currd
v1 #6 (#5) feedback Jan: init/destroy: s/ENOSYS/EOPNOTSUPP/
v1 #6 feedback Paul: Folded patch 6 into patch 5.
v1 #6 feedback Jan: drop opt_argo_enabled initializer
v1 $6 feedback Jan: s/ENOSYS/EOPNOTSUPP/g and drop useless dprintk v1. #5 feedback Paul: change the license on public header to BSD
- ack from Lars at Citrix.
v1. self, Jan: drop unnecessary xen include from sched.h
v1. self, Jan: drop inclusion of public argo.h in private one
v1. self, Jan: add include of public argo.h to argo.c
v1. self, Jan: drop fwd decl of argo_domain in priv header
v1. Paul/self/Jan: add data structures to xlat.lst and compat/argo.h to Makefile v1. self: removed allocation of event channel since switching to VIRQ v1. self: drop types.h include from private argo.h
v1: reorder public argo include position
v1: #13 feedback Jan: public namespace: prefix with xen
v1: self: rename pending ent "id" to "domain_id"
v1: self: add domain_cookie to ent struct
v1. #15 feedback Jan: make cmd unsigned
v1. #15 feedback Jan: make i loop variable unsigned
v1: self: adjust dprintks in init, destroy
v1: #18 feedback Jan: meld max ring count limit
v1: self: use type not struct in public defn, affects compat gen header
v1: feedback #15 Jan: handle upper-halves of hypercall args
v1: add comment explaining the 'magic' field
v1: self + Jan feedback: implement soft reset
v1: feedback #13 Roger: use ASSERT_UNREACHABLE
docs/misc/xen-command-line.pandoc | 15 + xen/common/Makefile | 2 +- xen/common/argo.c | 617 +++++++++++++++++++++++++++++++++++++- xen/common/compat/argo.c | 23 ++ xen/common/domain.c | 9 + xen/include/Makefile | 1 + xen/include/public/argo.h | 64 ++++ xen/include/xen/argo.h | 44 +++ xen/include/xen/sched.h | 5 + xen/include/xlat.lst | 2 + 10 files changed, 780 insertions(+), 2 deletions(-) create mode 100644 xen/common/compat/argo.c create mode 100644 xen/include/public/argo.h create mode 100644 xen/include/xen/argo.h
docs/misc/xen-command-line.pandoc +15 -0
@@ -182,6 +182,21 @@ Permit Xen to use "Always Running APIC Timer" support on compatible hardware in combination with cpuidle. This option is only expected to be useful for developers wishing Xen to fall back to older timing methods on newer hardware. +### argo + = List of [ <bool> ] + +Controls for the Argo hypervisor-mediated interdomain communication service. + +The functionality that this option controls is only available when Xen has been +compiled with the build setting for Argo enabled in the build configuration. + +Argo is a interdomain communication mechanism, where Xen acts as the central +point of authority. Guests may register memory rings to recieve messages, +query the status of other domains, and send messages by hypercall, all subject +to appropriate auditing by Xen. + +* An overall boolean acts as a global control. Argo is disabled by default. + ### asid (x86) > `= <boolean>`
xen/common/Makefile +1 -1
@@ -71,7 +71,7 @@ lzo-y := lzo lzo-$(CONFIG_TMEM) := obj-bin-$(CONFIG_X86) += $(foreach n,decompress bunzip2 unxz unlzma $(lzo-y) unlzo unlz4 earlycpio,$(n).init.o) -obj-$(CONFIG_COMPAT) += $(addprefix compat/,domain.o kernel.o memory.o multicall.o xlat.o) +obj-$(CONFIG_COMPAT) += $(addprefix compat/,argo.o domain.o kernel.o memory.o multicall.o xlat.o) tmem-y := tmem.o tmem_xen.o tmem_control.o tmem-$(CONFIG_COMPAT) += compat/tmem_xen.o
xen/common/argo.c +616 -1
634 lines not on this page · raw message
xen/common/compat/argo.c new file +23 -0
@@ -0,0 +1,23 @@ +/****************************************************************************** +* Argo : Hypervisor-Mediated data eXchange +* +* Copyright (c) 2018, BAE Systems +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +*/ + +#include <xen/lib.h> + +#include <public/argo.h> + +#include <compat/argo.h> + +CHECK_argo_addr; +CHECK_argo_ring;
xen/common/domain.c +9 -0
@@ -32,6 +32,7 @@#include <xen/grant_table.h> #include <xen/xenoprof.h> #include <xen/irq.h> +#include <xen/argo.h> #include <asm/debugger.h> #include <asm/p2m.h> #include <asm/processor.h>
@@ -277,6 +278,8 @@ static void _domain_destroy(struct domain *d)xfree (d->pbuf); +argo_destroy (d); +rangeset_domain_destroy (d);free_cpumask_var (d->dirty_cpumask);
@@ -445,6 +448,9 @@ struct domain *domain_create(domid_t domid, goto fail; init_status |= INIT_gnttab; +if ( (err =argo_init (d)) !=0 ) + goto fail; + err = -ENOMEM ; d->pbuf =xzalloc_array (char,DOMAIN_PBUF_SIZE );
@@ -717,6 +723,7 @@ int domain_kill(struct domain *d)if ( d->is_dying != DOMDYING_alive )return domain_kill (d); d->is_dying = DOMDYING_dying; +argo_destroy (d);evtchn_destroy (d);gnttab_release_mappings (d);tmem_destroy (d->tmem_client);
@@ -1175,6 +1182,8 @@ int domain_soft_reset(struct domain *d)grant_table_warn_active_grants (d); +argo_soft_reset (d); + for_each_vcpu ( d, v ) {set_xen_guest_handle (runstate_guest (v),NULL );
xen/include/Makefile +1 -0
@@ -5,6 +5,7 @@ ifneq ($(CONFIG_COMPAT),) compat-arch-$(CONFIG_X86) := x86_32 headers-y := \ + compat/argo.h \ compat/callback.h \ compat/elfnote.h \ compat/event_channel.h \
xen/include/public/argo.h new file +64 -0
64 lines not on this page · raw message
xen/include/xen/argo.h new file +44 -0
44 lines not on this page · raw message
xen/include/xen/sched.h +5 -0
@@ -490,6 +490,11 @@ struct domain unsigned int guest_request_enabled : 1; unsigned int guest_request_sync : 1; } monitor; + +#ifdef CONFIG_ARGO +/* Argo interdomain communication support */ +struct argo_domain *argo; +#endif };/* Protect updates/reads (resp.) of domain_list and domain_hash. */
xen/include/xlat.lst +2 -0
@@ -148,3 +148,5 @@ ? flask_setenforce xsm/flask_op.h ! flask_sid_context xsm/flask_op.h ? flask_transition xsm/flask_op.h +? argo_addr argo.h +? argo_ring argo.h
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 04/15] argo: init, destroy and soft-reset, with enable command line opt
Quoted 49 lines · Christopher Clark
Initialises basic data structures and performs teardown of argo state for domain shutdown.
Inclusion of the Argo implementation is dependent on
CONFIG_ARGO.Introduces a new Xen command line parameter 'argo': bool to enable/disable the argo hypercall. Defaults to disabled.
New headers:
public/argo.h: with definions of addresses and ring structure, including indexes for atomic update for communication between domain and hypervisor.
xen/argo.h: to expose the hooks for integration into domain lifecycle:
argo_init: per-domain init of argo data structures fordomain_create.
argo_destroy: teardown fordomain_destroyand the error exit
path ofdomain_create.
argo_soft_reset: reset of domain state fordomain_soft_reset.Adds a new field to struct domain: struct
argo_domain*argo;In accordance with recent work on
_domain_destroy,argo_destroyis idempotent. It will tear down: all rings registered by this domain, all rings where this domain is the single sender (ie. specified partner, non-wildcard rings), and all pending notifications where this domain is awaiting signal about available space in the rings of other domains.A count will be maintained of the number of rings that a domain has registered in order to limit it below the fixed maximum limit defined here.
Macros are defined to verify the internal locking state within the argo implementation. The macros are ASSERTed on entry to functions to validate and document the required lock state prior to calling.
The hash function for the hashtables that hold ring state is derived from the string hashing function djb2 (http://www.cse.yorku.ca/~oz/hash.html)
by Daniel J. Bernstein. Basic testing with a limited number of domains and ports has shown reasonable distribution for the table size.The software license on the public header is the BSD license, standard procedure for the public Xen headers. The public header was originally posted under a GPL license at: [1]:
https://lists.xenproject.org/archives/html/xen-devel/2013-05/msg02710.htmlThe following ACK by Lars Kurth is to confirm that only people being employees of Citrix contributed to the header files in the series posted at
[1] and that thus the copyright of the files in question is fully owned by Citrix. The ACK also confirms that Citrix is happy for the header files to be published under a BSD license in this series (which is based on [1]).
Quoted 3 lines
Signed-off-by Christopher Clark <[email protected]>Acked-by Lars Kurth <[email protected]>Reviewed-by Ross Philipson <[email protected]>
Thanks.
I've got some nits below, but it's purely cosmetic changes to make the code cleaner.
Quoted 114 lines
---
v4 Jan: amend the command line doc text referring to build configuration v4 feedback: use standard data structures as per common code
v4 Jan: replacehash_indexwith djb2-derived hash algorithm
v4 Andrew: switch argo command line option to list argo=<bool>
v4: removed note to removeargo_destroyfromdomain_kill(test shows issue) v4 #04 Roger: drop unneeded init ofring_countinargo_domain_initv4 #04 Roger: replace if (ring_info->mfns) with ASSERTs inring_unmapv4 #04 Roger: rewrite the locking verification macros
v4 #04 Roger: make L1 lock description comment clearer about R(L1) and W(L1) v4 Andrew: fix split of dprintk inring_map_infoacross v4 commitsv3 #04 Andrew: use xzalloc for struct
argo_domaininargo_initv3 #04 Andrew: referenceCONFIG_ARGOin the command line documentation v3 #07 Jan: renamering_find_infotofind_ring_info
v3 #04 Andrew: don't truncate argsdo_argo_opprintk
v3 #07 Jan: fix numeric entries in printk format strings
v3 #10 Roger: move find functions to top of file and drop prototypes v3 #04 Jan: meld compat check for hypercall arg types
v3 #04 Roger/Jan: make lock names clearer and assert their state v3 #04 Jan: port -> aport with type; distinguish argo port from evtchn v3 #04 Jan: reorder call toargo_init_domaininargo_init
v3 #04 Jan:ring_remove_mfns: zero count before freeing arrays v3 #04 Jason/Roger:soft_reset: can assume reinit is ok ifd->argoset v3 #04 Roger: remove unused and confusingd->argo_lock
v3 #04 Roger: add simple inlines inxen/argo.h, drop ifdefCONFIG_ARGOv3 #04 Roger: simpler return-EOPNOTSUPPindo_argo_op
v3 #04 Roger: add const to domain arg toring_remove_info
v3 #04 Roger: use XFREE
v3 #04 Roger: newline fix inwildcard_pending_list_remove
v3 #04 Roger:mfn_mapping: void* instead of uint8_t*
v3 #04 Roger: drop npages struct member inargo_ring_info; use len v3 #04 Roger/Jan: drop many fixed width types in internal structs v3 #04 Jason/Jan: drop pad and fixed width type inpending_entstruct v3 #04 Eric: movedring_find_infofrom register op into this commit v3 movedhash_indexfunction, nospec include from register op to this commit v3 movedXEN_ARGO_DOMID_ANYdefn from register op into this commit v3 added #include <xen/sched.h> to <xen/argo.h> for domain struct defn v3 feedback #04 Roger: reorder #includes to alphabetical order v3 Added Ross's Reviewed-by.v2 rewrite locking explanation comment
v2 header copyright line now includes 2019
v2 self: usering_infobackpointer inpending_entto maintain npending v2 self: renameall_rings_remove_infotodomain_rings_remove_all
v2 feedback Jan: drop cookie, implement teardown
v2 self: add npending to track number of pending entries per ring v2 self: amend comment on locking; drop section comments
v2cookie_eq: test low bits first and use likely on high bits
v2 self: OVERHAUL
v2 self: s/argo_pending_ent/pending_ent/g
v2 self: droppending_remove_ent, inline at single call site
v1 feedback Roger, Jan: drop argo prefix on static functions
v2 #4 Lars: add Acked-by and details to commit message.
v2 feedback #9 Jan: document argo boot opt in xen-command-line.markdown v2 bugfix: xsm use in soft-reset prior to introduction
v2 feedback #9 Jan: drop 'message' fromdo_argo_message_op
v1 #5 feedback Paul: init/destroy unsigned, brackets and whitespace fixes v1 #5 feedback Paul: Usemfn_eqfor comparing mfns.
v1 #5 feedback Paul: init/destroy : use currd
v1 #6 (#5) feedback Jan: init/destroy: s/ENOSYS/EOPNOTSUPP/
v1 #6 feedback Paul: Folded patch 6 into patch 5.
v1 #6 feedback Jan: dropopt_argo_enabledinitializer
v1 $6 feedback Jan: s/ENOSYS/EOPNOTSUPP/g and drop useless dprintk v1. #5 feedback Paul: change the license on public header to BSD
- ack from Lars at Citrix.
v1. self, Jan: drop unnecessary xen include fromsched.h
v1. self, Jan: drop inclusion of publicargo.hin private one
v1. self, Jan: add include of publicargo.htoargo.c
v1. self, Jan: drop fwd decl ofargo_domainin priv header
v1. Paul/self/Jan: add data structures to xlat.lst andcompat/argo.htoMakefilev1. self: removed allocation of event channel since switching to VIRQ v1. self: droptypes.hinclude from privateargo.h
v1: reorder public argo include position
v1: #13 feedback Jan: public namespace: prefix with xen
v1: self: rename pending ent "id" to "domain_id"
v1: self: adddomain_cookieto ent struct
v1. #15 feedback Jan: make cmd unsigned
v1. #15 feedback Jan: make i loop variable unsigned
v1: self: adjust dprintks in init, destroy
v1: #18 feedback Jan: meld max ring count limit
v1: self: use type not struct in public defn, affects compat gen header
v1: feedback #15 Jan: handle upper-halves of hypercall args
v1: add comment explaining the 'magic' field
v1: self + Jan feedback: implement soft reset
v1: feedback #13 Roger: useASSERT_UNREACHABLEdocs/misc/xen-command-line.pandoc | 15 + xen/common/Makefile | 2 +- xen/common/argo.c | 617 +++++++++++++++++++++++++++++++++++++- xen/common/compat/argo.c | 23 ++ xen/common/domain.c | 9 + xen/include/Makefile | 1 + xen/include/public/argo.h | 64 ++++ xen/include/xen/argo.h | 44 +++ xen/include/xen/sched.h | 5 + xen/include/xlat.lst | 2 + 10 files changed, 780 insertions(+), 2 deletions(-) create mode 100644 xen/common/compat/argo.c create mode 100644 xen/include/public/argo.h create mode 100644 xen/include/xen/argo.hdocs/misc/xen-command-line.pandoc +4
-0@@ -182,6 +182,21 @@ Permit Xen to use "Always Running APIC Timer" support on compatible hardware in combination with cpuidle. This option is only expected to be useful for developers wishing Xen to fall back to older timing methods on newer hardware. +### argo + = List of [ <bool> ] + +Controls for the Argo hypervisor-mediated interdomain communication service.
docs/misc/xen-command-line.pandoc Permit Xen to use "Always Running APIC Timer" support on compatible hardware · line 198
Quoted 10 lines
docs/misc/xen-command-line.pandoc +10
-0+ +The functionality that this option controls is only available when Xen has been +compiled with the build setting for Argo enabled in the build configuration. + +Argo is a interdomain communication mechanism, where Xen acts as the central +point of authority. Guests may register memory rings to recieve messages, +query the status of other domains, and send messages by hypercall, all subject +to appropriate auditing by Xen. + +* An overall boolean acts as a global control. Argo is disabled by default.
I'm not sure it's worth adding a list item for the boolean, I would just add the "Argo is disabled by default" to the first paragraph.
[...]
docs/misc/xen-command-line.pandoc Permit Xen to use "Always Running APIC Timer" support on compatible hardware
Quoted 11 lines
docs/misc/xen-command-line.pandoc +11
-0+static struct argo_ring_info * +find_ring_info(const struct domain *d, const struct argo_ring_id *id) +{ + struct list_head *cursor, *bucket; + + ASSERT(LOCKING_Read_rings_L2(d)); + + /* List is not modified here. Search and return the match if found. */ + bucket = &d->argo->ring_hash[hash_index(id)]; + + for ( cursor = bucket->next; cursor != bucket; cursor = cursor->next )
Why are you open-coding list_for_each here?
You might also consider using list_for_each_entry, so that you can avoid the list_entry call below.
Quoted 19 lines
docs/misc/xen-command-line.pandoc +19
-0+ { + struct argo_ring_info *ring_info = + list_entry(cursor, struct argo_ring_info, node); + const struct argo_ring_id *cmpid = &ring_info->id; + + if ( cmpid->aport == id->aport && + cmpid->domain_id == id->domain_id && + cmpid->partner_id == id->partner_id ) + { + argo_dprintk("found ring_info for ring(%u:%x %u)\n", + id->domain_id, id->aport, id->partner_id); + return ring_info; + } + } + argo_dprintk("no ring_info for ring(%u:%x %u)\n", + id->domain_id, id->aport, id->partner_id); + + return NULL; +}
[...]
docs/misc/xen-command-line.pandoc
Quoted 10 lines
docs/misc/xen-command-line.pandoc +10
-0+static void +pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) +{ + struct list_head *ring_pending = &ring_info->pending; + struct pending_ent *ent; + + ASSERT(LOCKING_L3(d, ring_info)); + + /* Delete all pending notifications from this ring's list. */ + while ( !list_empty(ring_pending) )
Nit: you could use list_first_entry_or_null that joins the list_empty and list_entry calls.
Quoted 16 lines
docs/misc/xen-command-line.pandoc +16
-0+ { + ent = list_entry(ring_pending->next, struct pending_ent, node); + + /* For wildcard rings, remove each from their wildcard list too. */ + if ( ring_info->id.partner_id == XEN_ARGO_DOMID_ANY ) + wildcard_pending_list_remove(ent->domain_id, ent); + list_del(&ent->node); + xfree(ent); + } + ring_info->npending = 0; +} + +static void +wildcard_rings_pending_remove(struct domain *d) +{ + struct list_head *wildcard_head;
docs/misc/xen-command-line.pandoc
Quoted 10 lines
docs/misc/xen-command-line.pandoc +10
-0+ + ASSERT(LOCKING_Write_L1); + + /* Delete all pending signals to the domain about wildcard rings. */ + wildcard_head = &d->argo->wildcard_pend_list; + + while ( !list_empty(wildcard_head) ) + { + struct pending_ent *ent = + list_entry(wildcard_head->next, struct pending_ent, node);
Same here regarding the usage of list_first_entry_or_null.
Quoted 13 lines
docs/misc/xen-command-line.pandoc +13
-0+ + /* + * The ent->node deleted here, and the npending value decreased, + * belong to the ring_info of another domain, which is why this + * function requires holding W(L1): + * it implies the L3 lock that protects that ring_info struct. + */ + ent->ring_info->npending--; + list_del(&ent->node); + list_del(&ent->wildcard_node); + xfree(ent); + } +}
[...]
Quoted 5 lines
docs/misc/xen-command-line.pandoc +5
-0+static void +domain_rings_remove_all(struct domain *d) +{ + unsigned int i; + struct argo_ring_info *ring_info;
docs/misc/xen-command-line.pandoc
Quoted 10 lines
docs/misc/xen-command-line.pandoc +10
-0+ + ASSERT(LOCKING_Write_rings_L2(d)); + + for ( i = 0; i < ARGO_HASHTABLE_SIZE; ++i ) + { + struct list_head *bucket = &d->argo->ring_hash[i]; + + while ( !list_empty(bucket) ) + { + ring_info = list_entry(bucket->next, struct argo_ring_info, node);
list_first_entry_or_null
Quoted 17 lines
docs/misc/xen-command-line.pandoc +17
-0+ ring_remove_info(d, ring_info); + } + } + d->argo->ring_count = 0; +} + +/* + * Tear down all rings of other domains where src_d domain is the partner. + * (ie. it is the single domain that can send to those rings.) + * This will also cancel any pending notifications about those rings. + */ +static void +partner_rings_remove(struct domain *src_d) +{ + unsigned int i; + struct argo_send_info *send_info; + struct argo_ring_info *ring_info;
docs/misc/xen-command-line.pandoc
Quoted 10 lines
docs/misc/xen-command-line.pandoc +10
-0+ struct domain *dst_d; + + ASSERT(LOCKING_Write_L1); + + for ( i = 0; i < ARGO_HASHTABLE_SIZE; ++i ) + { + struct list_head *cursor, *bucket = &src_d->argo->send_hash[i]; + + /* Remove all ents from the send list. Take each off their ring list. */ + for ( cursor = bucket->next; cursor != bucket; cursor = cursor->next )
Another open-coded version of list_for_each, see my comments on the instances above.
docs/misc/xen-command-line.pandoc
Quoted 2 lines
docs/misc/xen-command-line.pandoc +2
-0+ { + send_info = list_entry(cursor, struct argo_send_info, node);
send_info should be defined here to reduce it's scope.
docs/misc/xen-command-line.pandoc
Quoted 5 lines
docs/misc/xen-command-line.pandoc +5
-0+ + dst_d = get_domain_by_id(send_info->id.domain_id); + if ( dst_d && dst_d->argo ) + { + ring_info = find_ring_info(dst_d, &send_info->id);
ring_info should be defined here.
Quoted 9 lines
docs/misc/xen-command-line.pandoc +9
-0+ if ( ring_info ) + { + ring_remove_info(dst_d, ring_info); + dst_d->argo->ring_count--; + } + else + ASSERT_UNREACHABLE(); + } + else
docs/misc/xen-command-line.pandoc
Quoted 10 lines
docs/misc/xen-command-line.pandoc +10
-0+ ASSERT_UNREACHABLE(); + + if ( dst_d ) + put_domain(dst_d); + + list_del(&send_info->node); + xfree(send_info); + } + } +}
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 04/15] argo: init, destroy and soft-reset, with enable command line opt
Quoted 82 lines · Roger Pau Monné
Quoted 49 lines · Christopher Clark
Initialises basic data structures and performs teardown of argo state for domain shutdown.
Inclusion of the Argo implementation is dependent on
CONFIG_ARGO.Introduces a new Xen command line parameter 'argo': bool to enable/disable the argo hypercall. Defaults to disabled.
New headers:
public/argo.h: with definions of addresses and ring structure, including indexes for atomic update for communication between domain and hypervisor.
xen/argo.h: to expose the hooks for integration into domain lifecycle:
argo_init: per-domain init of argo data structures fordomain_create.
argo_destroy: teardown fordomain_destroyand the error exit
path ofdomain_create.
argo_soft_reset: reset of domain state fordomain_soft_reset.Adds a new field to struct domain: struct
argo_domain*argo;In accordance with recent work on
_domain_destroy,argo_destroyis idempotent. It will tear down: all rings registered by this domain, all rings where this domain is the single sender (ie. specified partner, non-wildcard rings), and all pending notifications where this domain is awaiting signal about available space in the rings of other domains.A count will be maintained of the number of rings that a domain has registered in order to limit it below the fixed maximum limit defined here.
Macros are defined to verify the internal locking state within the argo implementation. The macros are ASSERTed on entry to functions to validate and document the required lock state prior to calling.
The hash function for the hashtables that hold ring state is derived from the string hashing function djb2 (http://www.cse.yorku.ca/~oz/hash.html)
by Daniel J. Bernstein. Basic testing with a limited number of domains and ports has shown reasonable distribution for the table size.The software license on the public header is the BSD license, standard procedure for the public Xen headers. The public header was originally posted under a GPL license at: [1]:
https://lists.xenproject.org/archives/html/xen-devel/2013-05/msg02710.htmlThe following ACK by Lars Kurth is to confirm that only people being employees of Citrix contributed to the header files in the series posted at
[1] and that thus the copyright of the files in question is fully owned by Citrix. The ACK also confirms that Citrix is happy for the header files to be published under a BSD license in this series (which is based on [1]).Quoted 3 lines
Signed-off-by Christopher Clark <[email protected]>Acked-by Lars Kurth <[email protected]>Reviewed-by Ross Philipson <[email protected]>Thanks.
Reviewed-by Roger Pau Monné <[email protected]>I've got some nits below, but it's purely cosmetic changes to make the code cleaner.
Quoted 20 lines
---
docs/misc/xen-command-line.pandoc +9
-0@@ -182,6 +182,21 @@ Permit Xen to use "Always Running APIC Timer" support on compatible hardware in combination with cpuidle. This option is only expected to be useful for developers wishing Xen to fall back to older timing methods on newer hardware. +### argo + = List of [ <bool> ] + +Controls for the Argo hypervisor-mediated interdomain communication service. + +The functionality that this option controls is only available when Xen has been +compiled with the build setting for Argo enabled in the build configuration. + +Argo is a interdomain communication mechanism, where Xen acts as the central
Quoted 8 lines
docs/misc/xen-command-line.pandoc Permit Xen to use "Always Running APIC Timer" support on compatible hardware · line 198
Quoted 5 lines
docs/misc/xen-command-line.pandoc +5
-0+point of authority. Guests may register memory rings to recieve messages, +query the status of other domains, and send messages by hypercall, all subject +to appropriate auditing by Xen. + +* An overall boolean acts as a global control. Argo is disabled by default.I'm not sure it's worth adding a list item for the boolean, I would just add the "Argo is disabled by default" to the first paragraph.
ack, thanks, done.
Quoted 15 lines
[...]
docs/misc/xen-command-line.pandoc
Quoted 11 lines
docs/misc/xen-command-line.pandoc +11
-0+static struct argo_ring_info * +find_ring_info(const struct domain *d, const struct argo_ring_id *id) +{ + struct list_head *cursor, *bucket; + + ASSERT(LOCKING_Read_rings_L2(d)); + + /* List is not modified here. Search and return the match if found. */ + bucket = &d->argo->ring_hash[hash_index(id)]; + + for ( cursor = bucket->next; cursor != bucket; cursor = cursor->next )Why are you open-coding
list_for_eachhere?
see response to cover letter
Quoted 2 lines
You might also consider using
list_for_each_entry, so that you can avoid thelist_entrycall below.
Ack, have done that.
Quoted 21 lines
Quoted 20 lines
docs/misc/xen-command-line.pandoc +19
-0+ { + struct argo_ring_info *ring_info = + list_entry(cursor, struct argo_ring_info, node); + const struct argo_ring_id *cmpid = &ring_info->id; + + if ( cmpid->aport == id->aport && + cmpid->domain_id == id->domain_id && + cmpid->partner_id == id->partner_id ) + { + argo_dprintk("found ring_info for ring(%u:%x %u)\n", + id->domain_id, id->aport, id->partner_id); + return ring_info; + } + } + argo_dprintk("no ring_info for ring(%u:%x %u)\n", + id->domain_id, id->aport, id->partner_id); + + return NULL; +}[...]
Quoted 13 lines
docs/misc/xen-command-line.pandoc
Quoted 10 lines
docs/misc/xen-command-line.pandoc +10
-0+static void +pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) +{ + struct list_head *ring_pending = &ring_info->pending; + struct pending_ent *ent; + + ASSERT(LOCKING_L3(d, ring_info)); + + /* Delete all pending notifications from this ring's list. */ + while ( !list_empty(ring_pending) )Nit: you could use
list_first_entry_or_nullthat joins thelist_emptyandlist_entrycalls.
There are no existing users of list_first_entry_or_null anywhere in Xen, and applying it to this loop results in an assignment within the while condition, which also appears to be very rare construct within Xen, so I just used the list_for_each_entry_safe macro.
Quoted 21 lines
Quoted 21 lines
docs/misc/xen-command-line.pandoc +20
-0+ { + ent = list_entry(ring_pending->next, struct pending_ent, node); + + /* For wildcard rings, remove each from their wildcard list too. */ + if ( ring_info->id.partner_id == XEN_ARGO_DOMID_ANY ) + wildcard_pending_list_remove(ent->domain_id, ent); + list_del(&ent->node); + xfree(ent); + } + ring_info->npending = 0; +} + +static void +wildcard_rings_pending_remove(struct domain *d) +{ + struct list_head *wildcard_head; + + ASSERT(LOCKING_Write_L1); + + /* Delete all pending signals to the domain about wildcard rings. */
Quoted 8 lines
docs/misc/xen-command-line.pandoc
Quoted 6 lines
docs/misc/xen-command-line.pandoc +6
-0+ wildcard_head = &d->argo->wildcard_pend_list; + + while ( !list_empty(wildcard_head) ) + { + struct pending_ent *ent = + list_entry(wildcard_head->next, struct pending_ent, node);Same here regarding the usage of
list_first_entry_or_null.
list_for_each_entry_safe.
Quoted 55 lines
Quoted 14 lines
docs/misc/xen-command-line.pandoc +13
-0+ + /* + * The ent->node deleted here, and the npending value decreased, + * belong to the ring_info of another domain, which is why this + * function requires holding W(L1): + * it implies the L3 lock that protects that ring_info struct. + */ + ent->ring_info->npending--; + list_del(&ent->node); + list_del(&ent->wildcard_node); + xfree(ent); + } +}[...]
Quoted 5 lines
docs/misc/xen-command-line.pandoc +5
-0+static void +domain_rings_remove_all(struct domain *d) +{ + unsigned int i; + struct argo_ring_info *ring_info;docs/misc/xen-command-line.pandoc
Quoted 10 lines
docs/misc/xen-command-line.pandoc +10
-0+ + ASSERT(LOCKING_Write_rings_L2(d)); + + for ( i = 0; i < ARGO_HASHTABLE_SIZE; ++i ) + { + struct list_head *bucket = &d->argo->ring_hash[i]; + + while ( !list_empty(bucket) ) + { + ring_info = list_entry(bucket->next, struct argo_ring_info, node);
list_first_entry_or_nullQuoted 22 lines
docs/misc/xen-command-line.pandoc +22
-0+ ring_remove_info(d, ring_info); + } + } + d->argo->ring_count = 0; +} + +/* + * Tear down all rings of other domains where src_d domain is the partner. + * (ie. it is the single domain that can send to those rings.) + * This will also cancel any pending notifications about those rings. + */ +static void +partner_rings_remove(struct domain *src_d) +{ + unsigned int i; + struct argo_send_info *send_info; + struct argo_ring_info *ring_info; + struct domain *dst_d; + + ASSERT(LOCKING_Write_L1); + + for ( i = 0; i < ARGO_HASHTABLE_SIZE; ++i )
Quoted 8 lines
docs/misc/xen-command-line.pandoc
Quoted 5 lines
docs/misc/xen-command-line.pandoc +5
-0+ { + struct list_head *cursor, *bucket = &src_d->argo->send_hash[i]; + + /* Remove all ents from the send list. Take each off their ring list. */ + for ( cursor = bucket->next; cursor != bucket; cursor = cursor->next )Another open-coded version of
list_for_each, see my comments on the instances above.
list_for_each_entry_safe
Quoted 5 lines
docs/misc/xen-command-line.pandoc
Quoted 3 lines
docs/misc/xen-command-line.pandoc +2
-0+ { + send_info = list_entry(cursor, struct argo_send_info, node);
send_infoshould be defined here to reduce it's scope.
send_info is now iterating in the list_for_each_entry_safe macro, so its position here is retained since now required.
Quoted 8 lines
docs/misc/xen-command-line.pandoc
Quoted 6 lines
docs/misc/xen-command-line.pandoc +5
-0+ + dst_d = get_domain_by_id(send_info->id.domain_id); + if ( dst_d && dst_d->argo ) + { + ring_info = find_ring_info(dst_d, &send_info->id);
ring_infoshould be defined here.
ack, done.
Quoted 12 lines
Quoted 12 lines
docs/misc/xen-command-line.pandoc +11
-0+ if ( ring_info ) + { + ring_remove_info(dst_d, ring_info); + dst_d->argo->ring_count--; + } + else + ASSERT_UNREACHABLE(); + } + else + ASSERT_UNREACHABLE(); +
Quoted 8 lines
docs/misc/xen-command-line.pandoc
Quoted 8 lines
docs/misc/xen-command-line.pandoc +8
-0+ if ( dst_d ) + put_domain(dst_d); + + list_del(&send_info->node); + xfree(send_info); + } + } +}
thanks,
Christopher
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 04/15] argo: init, destroy and soft-reset, with enable command line opt
Quoted 17 lines
Quoted 1 line
Quoted 1 line · On 31.01.19 at 05:06, <[email protected]> wrote:
Quoted 14 lines · Roger Pau Monné
xen/common/argo.c line 392
Quoted 10 lines · Christopher Clark
xen/common/argo.c +10
-0+static void +pending_remove_all (const struct domain *d,struct argo_ring_info *ring_info) +{ +struct list_head *ring_pending = &ring_info->pending; +struct pending_ent *ent; + +ASSERT (LOCKING_L3 (d, ring_info)); + +/* Delete all pending notifications from this ring's list. */ +while ( !list_empty (ring_pending) )Nit: you could use
list_first_entry_or_nullthat joins thelist_emptyandlist_entrycalls.
Quoted 4 lines
There are no existing users of
list_first_entry_or_nullanywhere in Xen, and applying it to this loop results in an assignment within the while condition, which also appears to be very rare construct within Xen, so I just used thelist_for_each_entry_safemacro.
I'm not fully following why lack of use of a construct elsewhere in the tree would be a reason not to use it here. If you were to use only constructs already in use in common code, you should also not have had a need to e.g. introduce for Arm (and use) guest_handle_for_field().
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 5/15 errno: add POSIX error codes EMSGSIZE, ECONNREFUSED to the ABI
In reply to Christopher ClarkEMSGSIZE: Argo's sendv operation will return EMSGSIZE when an excess amount of data, across all iovs, has been supplied, exceeding either the statically configured maximum size of a transmittable message, or the (variable) size of the ring registered by the destination domain.
ECONNREFUSED: Argo's register operation will return ECONNREFUSED if a ring is being registered to communicate with a specific remote domain that does exist but is not argo-enabled.
These codes are described by POSIX here:
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/errno.h.html
EMSGSIZE : "Message too large"
ECONNREFUSED : "Connection refused".
The numeric values assigned to each are taken from Linux, as is the case for the existing error codes.
EMSGSIZE : 90
ECONNREFUSED : 111
--- xen/include/public/errno.h | 2 ++ 1 file changed, 2 insertions(+)
xen/include/public/errno.h +2 -0
@@ -102,6 +102,7 @@ XEN_ERRNO(EILSEQ, 84) /* Illegal byte sequence */ XEN_ERRNO(ERESTART, 85) /* Interrupted system call should be restarted */#endif XEN_ERRNO(ENOTSOCK, 88) /* Socket operation on non-socket */ +XEN_ERRNO(EMSGSIZE, 90) /* Message too large. */ XEN_ERRNO(EOPNOTSUPP, 95) /* Operation not supported on transport endpoint */ XEN_ERRNO(EADDRINUSE, 98) /* Address already in use */XEN_ERRNO (EADDRNOTAVAIL ,99 )/* Cannot assign requested address */
@@ -109,6 +110,7 @@ XEN_ERRNO(ENOBUFS, 105) /* No buffer space available */ XEN_ERRNO(EISCONN, 106) /* Transport endpoint is already connected */ XEN_ERRNO(ENOTCONN, 107) /* Transport endpoint is not connected */ XEN_ERRNO(ETIMEDOUT, 110) /* Connection timed out */ +XEN_ERRNO(ECONNREFUSED, 111) /* Connection refused */#undef XEN_ERRNO#endif /* XEN_ERRNO */
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 6/15 xen/arm: introduce guest_handle_for_field()
In reply to Christopher ClarkARM port of c/s bb544585: "introduce guest_handle_for_field()"
This helper turns a field of a GUEST_HANDLE into a GUEST_HANDLE.
---
v3: Added Stefano's Reviewed-by
v2: Added Paul's Reviewed-by
xen/include/asm-arm/guest_access.h | 3 +++ 1 file changed, 3 insertions(+)
xen/include/asm-arm/guest_access.h +3 -0
@@ -63,6 +63,9 @@ int access_guest_memory_by_ipa(struct domain *d, paddr_t ipa, void *buf, _y; \ }) +#define guest_handle_for_field (hnd, type, fld) \ + ((XEN_GUEST_HANDLE(type)) { &(hnd).p->fld }) +#define guest_handle_from_ptr (ptr, type) \ ((XEN_GUEST_HANDLE_PARAM(type)) { (type *)ptr })#define const_guest_handle_from_ptr (ptr, type) \
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 7/15 argo: implement the register op
In reply to Christopher ClarkThe register op is used by a domain to register a region of memory for receiving messages from either a specified other domain, or, if specifying a wildcard, any domain.
This operation creates a mapping within Xen's private address space that will remain resident for the lifetime of the ring. In subsequent commits, the hypervisor will use this mapping to copy data from a sending domain into this registered ring, making it accessible to the domain that registered the ring to receive data.
Wildcard any-sender rings are default disabled and registration will be refused with EPERM unless they have been specifically enabled with the new mac-permissive flag that is added to the argo boot option here. The reason why the default for wildcard rings is 'deny' is that there is currently no means to protect the ring from DoS by a noisy domain spamming the ring, affecting other domains ability to send to it. This will be addressed with XSM policy controls in subsequent work.
Since denying access to any-sender rings is a significant functional constraint, the new option "mac-permissive" for the argo bootparam enables overriding this. eg: "argo=1,mac-permissive=1"
The p2m type of the memory supplied by the guest for the ring must be p2m_ram_rw and the memory will be pinned as PGT_writable_page while the ring is registered.
xen_argo_gfn_t type is defined and is 64-bit on all architectures which assists with avoiding the need for compat code to translate hypercall args. This hypercall op and its interface currently only supports 4K-sized pages.
---
v4 v3#07 Jan: shrink critical sections in register_ring
v4 v3#07 Jan: revise register flag MASK in header, note 32-bitness of args v4 feedback: use standard data structures per common code, not loop macros v4 Andrew: use the single argo command line option list
v4 #07 Jan: rewrite find_ring_mfn to use check_get_page_from_gfn v4 #07 Roger: add FIXME to ring_map_page for vmap contiguous ring mapping
v3 #07 Jan: comment: minimum ring size is based on minimum-sized message v3 #04 Andrew: reference CONFIG_ARGO in the command line documentation v3 #07 Jan: register_ring: fold else, if into else-if to drop indent v3 #07 Jan: remove no longer used guest_handle_is_aligned macros
v3 #07 Jan: remove dead code from find_ring_mfns
v3 #07 Jan: fix format string indention in printks
v3 #07 Jan: remove redundant bounds check on npage in find_ring_mfns v3 #08 self/Roger: improve dprintk output in find_ring_info like find_send_info v3 #07 Jan: rename ring_find_info to find_ring_info
v3 #07 Jan: use array_index_nospec in ring_map_page
v3 #07 Jan: fix numeric entries in printk format strings
v3 #7 Jan: drop unneeded parentheses from ROUNDUP_MESSAGE defn
v3 #10 Roger: move find functions to top of file and drop prototypes v3 #03 meld compat check for hypercall arg register struct
v3 #04 Roger/Jan: make lock names clearer and assert their state
v3 #04 Jan: port -> aport with type; distinguish argo port from evtchn v3 feedback #07 Eric: fix header max ring size comment units
v3 feedback #04 Roger: mfn_mapping: void* instead of uint8_t*
v3 use %u for printing unsigned ints in find_ring_mfns
v3 feedback #04 Jan: uint32_t -> unsigned int for npage in register_ring v3 feedback #04 Roger: drop npages struct member, calculate from len v3 : register_ring: uint32_t -> unsigned int for private_tx_ptr
v3 feedback Roger/Jan: ASSERT currd is current->domain or use 'd' variable name v3 feedback #07 Roger: use opt_argo_mac_permissive : a boolean opt v3 feedback #04 Roger: reorder #includes to alphabetical order
v3 feedback #07 Roger: drop comment re: Intel EPT/AMD NPT for write-only mapping v3 feedback #07 Roger: drop ptr arithmetic in update_tx_ptr, use ring struct cast v3 feedback #07 Roger: drop newline in ring_map_page
v3 feedback #07 Roger: drop unneeded null check before xfree
v3 feedback #07 Roger: use return and drop out label in register_ring v3 Stefano: add 4K page constraint to header file comment & commit msg v3 Julien/Stefano: 4K granularity ok: use 64-bit gfns in register interface
v2 self: disallow ring resize via reregister
v2 feedback Jan: drop cookie, implement teardown
v2 feedback Jan: drop message from argo_message_op
v2 self: move hash_index function below locking comment
v2 self: OVERHAUL
v2 self/Jan: remove use of magic verification field and tidy up v2 self: merge max and min ring size check clauses
v2 feedback v1#13 Roger: use OS-supplied roundup; drop from public header v2 feedback #9, Jan: use the argo-mac bootparam at point of introduction v2 feedback #9, Jan: rename boot opt variable to comply with convention v2 feedback #9, Jan: rename the argo_mac bootparam to argo-mac v2 feedback #9 Jan: document argo boot opt in xen-command-line.markdown v1,2 feedback Jan/Roger/Paul: drop errno returning guest access functions v1 feedback Roger, Jan: drop argo prefix on static functions v1 feedback Roger: s/pfn/gfn/ and retire always-64-bit type v2. feedback Jan: document the argo-mac boot opt
v2. feedback Jan: simplify re-register, drop mappings
v1 #13 feedback Jan: revise use of guest_handle_okay vs __copy ops
v1 #13 feedback, Jan: register op : s/ECONNREFUSED/ESRCH/
v1 #5 (#13) feedback Paul: register op: use currd in do_message_op v1 #13 feedback, Paul: register op: use mfn_eq comparator
v1 #5 (#13) feedback Paul: register op: use currd in argo_register_ring v1 #13 feedback Paul: register op: whitespace, unsigned, bounds check v1 #13 feedback Paul: use of hex in limit constant definition v1 #13 feedback Paul, register op: set nmfns on loop termination v1 #13 feedback Paul: register op: do/while -> gotos, reindent v1 argo_ring_map_page: drop uint32_t for unsigned int
v1. #13 feedback Julien: use page descriptors instead of gpfns.
- adds ABI support for pages with different granularity.
v1 feedback #13, Paul: adjust log level of message
v1 feedback #13, Paul: use gprintk for guest-triggered warning v1 feedback #13, Paul: gprintk and XENLOG_DEBUG for ring registration v1 feedback #13, Paul: use gprintk for errs in argo_ring_map_page v1 feedback #13, Paul: use ENOMEM if global mapping fails
v1 feedback Paul: overflow check before shift
v1: add define for copy_field_to_guest_errno
v1: fix gprintk use for ARM as its defn dislikes split format strings
v1: use copy_field_to_guest_errno
v1 feedback #13, Jan: argo_hash_fn: no inline, rename, change type v1 feedback #13, Paul, Jan: EFAULT -> ENOMEM in argo_ring_map_page v1 feedback #13, Jan: rename page var in argo_ring_map_page
v1 feedback #13, Jan: switch uint8_t* to void* and drop cast
v1 feedback #13, Jan: switch memory barrier to smp_wmb
v1 feedback #13, Jan: make 'ring' comment comply with single-line style v1 feedback #13, Jan: use xzalloc_array, drop loop NULL init
v1 feedback #13, Jan: init bool with false rather than 0
v1 feedback #13 Jan: use __copy; define and use __copy_field_to_guest_errno v1 feedback #13, Jan: use xzalloc, drop individual init zeroes v1 feedback #13, Jan: prefix public namespace with xen
v1 feedback #13, Jan: blank line after op case in do_argo_message_op v1 self: reflow comment in argo_ring_map_page to within 80 char len v1 feedback #13, Roger: use true not 1 in assign to update_tx_ptr bool v1 feedback #21, Jan: fold in the array_index_nospec hash function guards v1 feedback #18, Jan: fold the max ring count limit into the series v1 self: use unsigned long type for XEN_ARGO_REGISTER_FLAG_MASK
v1: feedback #15 Jan: handle upper-halves of hypercall args
v1. feedback #13 Jan: add comment re: page alignment
v1. self: confirm ring magic presence in supplied page array v1. feedback #13 Jan: add comment re: minimum ring size
v1. feedback #13 Roger: use ASSERT_UNREACHABLE
v1. feedback Roger: add comment to hash function
docs/misc/xen-command-line.pandoc | 9 +- xen/common/argo.c | 467 ++++++++++++++++++++++++++++++++++++++ xen/common/compat/argo.c | 1 + xen/include/public/argo.h | 73 ++++++ xen/include/xlat.lst | 1 + 5 files changed, 550 insertions(+), 1 deletion(-)
docs/misc/xen-command-line.pandoc +8 -1
@@ -183,7 +183,7 @@ in combination with cpuidle. This option is only expected to be useful for developers wishing Xen to fall back to older timing methods on newer hardware. ### argo - = List of [ <bool> ] + = List of [ <bool>, mac-permissive=<bool> ] Controls for the Argo hypervisor-mediated interdomain communication service.
@@ -197,6 +197,13 @@ to appropriate auditing by Xen. * An overall boolean acts as a global control. Argo is disabled by default. +* The `mac-permissive` boolean controls whether wildcard receive rings may be + registered (`mac-permissive=1`) or may not be registered + (`mac-permissive=0`). + + This option is disabled by default, to protect domains from a DoS by a + buggy or malicious other domain spamming the ring. + ### asid (x86) > `= <boolean>`
xen/common/argo.c +467 -0
512 lines not on this page · raw message
xen/common/compat/argo.c +1 -0
@@ -20,4 +20,5 @@#include <compat/argo.h> CHECK_argo_addr; +CHECK_argo_register_ring; CHECK_argo_ring;
xen/include/public/argo.h +73 -0
86 lines not on this page · raw message
xen/include/xlat.lst +1 -0
@@ -150,3 +150,4 @@ ? flask_transition xsm/flask_op.h ? argo_addr argo.h ? argo_ring argo.h +? argo_register_ring argo.h
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 07/15] argo: implement the register op
Quoted 26 lines · Christopher Clark
The register op is used by a domain to register a region of memory for receiving messages from either a specified other domain, or, if specifying a wildcard, any domain.
This operation creates a mapping within Xen's private address space that will remain resident for the lifetime of the ring. In subsequent commits, the hypervisor will use this mapping to copy data from a sending domain into this registered ring, making it accessible to the domain that registered the ring to receive data.
Wildcard any-sender rings are default disabled and registration will be refused with EPERM unless they have been specifically enabled with the new mac-permissive flag that is added to the argo boot option here. The reason why the default for wildcard rings is 'deny' is that there is currently no means to protect the ring from DoS by a noisy domain spamming the ring, affecting other domains ability to send to it. This will be addressed with XSM policy controls in subsequent work.
Since denying access to any-sender rings is a significant functional constraint, the new option "mac-permissive" for the argo bootparam enables overriding this. eg: "argo=1,mac-permissive=1"
The p2m type of the memory supplied by the guest for the ring must be
p2m_ram_rwand the memory will be pinned asPGT_writable_pagewhile the ring is registered.
Quoted 5 lines
xen_argo_gfn_ttype is defined and is 64-bit on all architectures which assists with avoiding the need for compat code to translate hypercall args. This hypercall op and its interface currently only supports 4K-sized pages.Signed-off-by Christopher Clark <[email protected]>
Just some nits that can be taken care of later.
Quoted 67 lines
xen/common/argo.c +67
-0+static int +find_ring_mfns (struct domain *d,struct argo_ring_info *ring_info, +const unsigned int npage, +XEN_GUEST_HANDLE_PARAM (xen_argo_gfn_t) gfn_hnd, +const unsigned int len) +{ +unsigned int i; +int ret =0 ; +mfn_t *mfns; +void **mfn_mapping; + +ASSERT (LOCKING_Write_rings_L2 (d)); + +if ( ring_info->mfns ) + { +/* Ring already existed: drop the previous mapping. */ +gprintk (XENLOG_INFO ,"argo: vm%u re-register existing ring " +"(vm%u:%x vm%u) clears mapping\n" , + d->domain_id, ring_info->id.domain_id, + ring_info->id.aport, ring_info->id.partner_id); + +ring_remove_mfns (d, ring_info); +ASSERT (!ring_info->mfns); + } + + mfns =xmalloc_array (mfn_t, npage); +if ( !mfns ) +return -ENOMEM ; + +for ( i =0 ; i < npage; i++ ) + mfns[i] =INVALID_MFN ; + + mfn_mapping = xzalloc_array(void *, npage); +if ( !mfn_mapping ) + { +xfree (mfns); +return -ENOMEM ; + } + + ring_info->mfns = mfns; + ring_info->mfn_mapping = mfn_mapping; + +for ( i =0 ; i < npage; i++ ) + { +xen_argo_gfn_t argo_gfn; +mfn_t mfn; + + ret =__copy_from_guest_offset (&argo_gfn, gfn_hnd, i,1 ) ? -EFAULT :0 ; +if ( ret ) +break ; + + ret =find_ring_mfn (d,_gfn (argo_gfn), &mfn); +if ( ret ) + { +gprintk (XENLOG_ERR ,"argo: vm%u: invalid gfn %" PRI_gfn" " +"r:(vm%u:%x vm%u) %p %u/%u\n" , + d->domain_id,gfn_x (_gfn (argo_gfn)), + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id, ring_info, i, npage); +break ; + } + + ring_info->mfns[i] = mfn; + +argo_dprintk ("%u: %" PRI_gfn" -> %" PRI_mfn"\n" , + i,gfn_x (_gfn (argo_gfn)),mfn_x (ring_info->mfns[i])); + }
xen/common/argo.c partner_rings_remove(struct domain *src_d) · line 727
Quoted 10 lines
xen/common/argo.c +10
-0+ + ring_info->nmfns = i; + +if ( ret ) +ring_remove_mfns (d, ring_info); +else + { +ASSERT (ring_info->nmfns ==NPAGES_RING (len)); + + gprintk(XENLOG_DEBUG, "argo: vm%u ring (vm%u:%x vm%u) %p "
Nit: this likely wants to be an argo_dprintk?
Quoted 127 lines
xen/common/argo.c +127
-0+"mfn_mapping %p len %u nmfns %u\n" , + d->domain_id, ring_info->id.domain_id, + ring_info->id.aport, ring_info->id.partner_id, ring_info, + ring_info->mfn_mapping, ring_info->len, ring_info->nmfns); + } + +return ret; +} + +static long +register_ring (struct domain *currd, +XEN_GUEST_HANDLE_PARAM (xen_argo_register_ring_t) reg_hnd, +XEN_GUEST_HANDLE_PARAM (xen_argo_gfn_t) gfn_hnd, +unsigned int npage,bool fail_exist) +{ +xen_argo_register_ring_t reg; +struct argo_ring_id ring_id; +void *map_ringp; +xen_argo_ring_t *ringp; +struct argo_ring_info *ring_info, *new_ring_info =NULL ; +struct argo_send_info *send_info =NULL ; +struct domain *dst_d =NULL ; +int ret =0 ; +unsigned int private_tx_ptr; + +ASSERT (currd == current->domain); + +if (copy_from_guest (®, reg_hnd,1 ) ) +return -EFAULT ; + +/* +* A ring must be large enough to transmit messages, so requires space for: +* * 1 message header, plus +* * 1 payload slot (payload is always rounded to a multiple of 16 bytes) +* for the message payload to be written into, plus +* * 1 more slot, so that the ring cannot be filled to capacity with a +* single minimum-size message -- see the logic in ringbuf_insert -- +* allowing for this ensures that there can be space remaining when a +* message is present. +* The above determines the minimum acceptable ring size. +*/ +if ( (reg.len < (sizeof (struct xen_argo_ring_message_header ) + +ROUNDUP_MESSAGE (1 ) +ROUNDUP_MESSAGE (1 ))) || + (reg.len >XEN_ARGO_MAX_RING_SIZE ) || + (reg.len !=ROUNDUP_MESSAGE (reg.len)) || + (NPAGES_RING (reg.len) != npage) || + (reg.pad !=0 ) ) +return -EINVAL ; + + ring_id.partner_id = reg.partner_id; + ring_id.aport = reg.aport; + ring_id.domain_id = currd->domain_id; + +if ( reg.partner_id ==XEN_ARGO_DOMID_ANY ) + { +if ( !opt_argo_mac_permissive ) +return -EPERM ; + } +else + { + dst_d =get_domain_by_id (reg.partner_id); +if ( !dst_d ) + { +argo_dprintk ("!dst_d, ESRCH\n" ); +return -ESRCH ; + } + + send_info = xzalloc(struct argo_send_info); +if ( !send_info ) + { + ret = -ENOMEM ; + goto out; + } + send_info->id = ring_id; + } + +/* +* Common case is that the ring doesn't already exist, so do the alloc here +* before picking up any locks. +*/ + new_ring_info = xzalloc(struct argo_ring_info); +if ( !new_ring_info ) + { + ret = -ENOMEM ; + goto out; + } + +read_lock (&L1_global_argo_rwlock); + +if ( !currd->argo ) + { + ret = -ENODEV ; + goto out_unlock; + } + +if ( dst_d && !dst_d->argo ) + { +argo_dprintk ("!dst_d->argo, ECONNREFUSED\n" ); + ret = -ECONNREFUSED ; + goto out_unlock; + } + +write_lock (&currd->argo->rings_L2_rwlock); + +if ( currd->argo->ring_count >=MAX_RINGS_PER_DOMAIN ) + { + ret = -ENOSPC ; + goto out_unlock2; + } + + ring_info =find_ring_info (currd, &ring_id); +if ( !ring_info ) + { + ring_info = new_ring_info; + new_ring_info =NULL ; + +spin_lock_init (&ring_info->L3_lock); + + ring_info->id = ring_id; +INIT_LIST_HEAD (&ring_info->pending); + +list_add (&ring_info->node, + &currd->argo->ring_hash[hash_index (&ring_info->id)]); + +gprintk (XENLOG_DEBUG ,"argo: vm%u registering ring (vm%u:%x vm%u)\n" , + currd->domain_id, ring_id.domain_id, ring_id.aport, + ring_id.partner_id);
xen/common/argo.c partner_rings_remove(struct domain *src_d) · line 864
Quoted 10 lines
xen/common/argo.c +10
-0+ } +else if ( ring_info->len ) + { +/* +* If the caller specified that the ring must not already exist, +* fail at attempt to add a completed ring which already exists. +*/ +if ( fail_exist ) + { +argo_dprintk ("disallowed reregistration of existing ring\n" );
And this should likely be gprintk with error type?
I think the pattern of using gprintk for error messages and argo_dprintk for verbose information is correct, but there are a couple of oddities that can be fixed later.
Quoted 18 lines
xen/common/argo.c +18
-0+ ret = -EEXIST ; + goto out_unlock2; + } + +if ( ring_info->len != reg.len ) + { +/* +* Change of ring size could result in entries on the pending +* notifications list that will never trigger. +* Simple blunt solution: disallow ring resize for now. +* TODO: investigate enabling ring resize. +*/ +gprintk (XENLOG_ERR ,"argo: vm%u attempted to change ring size " +"(vm%u:%x vm%u)\n" , + currd->domain_id, ring_id.domain_id, ring_id.aport, + ring_id.partner_id); +/* +* Could return EINVAL here, but if the ring didn't already
xen/common/argo.c partner_rings_remove(struct domain *src_d) · line 892
Quoted 10 lines
xen/common/argo.c +10
-0+* exist then the arguments would have been valid, so: EEXIST. +*/ + ret = -EEXIST ; + goto out_unlock2; + } + +gprintk (XENLOG_DEBUG , +"argo: vm%u re-registering existing ring (vm%u:%x vm%u)\n" , + currd->domain_id, ring_id.domain_id, ring_id.aport, + ring_id.partner_id);
This again would better be argo_dprintk IMO.
[...]
xen/common/argo.c do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, · line 998
Quoted 13 lines
xen/common/argo.c +9
-0@@ -552,6 +987,38 @@ do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,switch (cmd) { +case XEN_ARGO_OP_register_ring: + { +XEN_GUEST_HANDLE_PARAM (xen_argo_register_ring_t) reg_hnd = +guest_handle_cast (arg1, xen_argo_register_ring_t); +XEN_GUEST_HANDLE_PARAM (xen_argo_gfn_t) gfn_hnd = +guest_handle_cast (arg2, xen_argo_gfn_t); +/* arg3 is npage */ +/* arg4 is flags */ +bool fail_exist = arg4 &XEN_ARGO_REGISTER_FLAG_FAIL_EXIST ;
Nit: I would add a:
BUILD_BUG_ON(!IS_ALIGNED(XEN_ARGO_MAX_RING_SIZE, PAGE_SIZE));
xen/common/argo.c do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, · line 1004
Quoted 5 lines
xen/common/argo.c +5
-0+if (unlikely (arg3 > (XEN_ARGO_MAX_RING_SIZE >>PAGE_SHIFT )) ) + { + rc = -EINVAL ; +break ; + }
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 07/15] argo: implement the register op
Quoted 109 lines · Roger Pau Monné
Quoted 26 lines · Christopher Clark
The register op is used by a domain to register a region of memory for receiving messages from either a specified other domain, or, if specifying a wildcard, any domain.
This operation creates a mapping within Xen's private address space that will remain resident for the lifetime of the ring. In subsequent commits, the hypervisor will use this mapping to copy data from a sending domain into this registered ring, making it accessible to the domain that registered the ring to receive data.
Wildcard any-sender rings are default disabled and registration will be refused with EPERM unless they have been specifically enabled with the new mac-permissive flag that is added to the argo boot option here. The reason why the default for wildcard rings is 'deny' is that there is currently no means to protect the ring from DoS by a noisy domain spamming the ring, affecting other domains ability to send to it. This will be addressed with XSM policy controls in subsequent work.
Since denying access to any-sender rings is a significant functional constraint, the new option "mac-permissive" for the argo bootparam enables overriding this. eg: "argo=1,mac-permissive=1"
The p2m type of the memory supplied by the guest for the ring must be
p2m_ram_rwand the memory will be pinned asPGT_writable_pagewhile the ring is registered.Quoted 5 lines
xen_argo_gfn_ttype is defined and is 64-bit on all architectures which assists with avoiding the need for compat code to translate hypercall args. This hypercall op and its interface currently only supports 4K-sized pages.Signed-off-by Christopher Clark <[email protected]>Reviewed-by Roger Pau Mooné <[email protected]>Just some nits that can be taken care of later.
Quoted 71 lines
xen/common/argo.c +71
-0+static int +find_ring_mfns (struct domain *d,struct argo_ring_info *ring_info, +const unsigned int npage, +XEN_GUEST_HANDLE_PARAM (xen_argo_gfn_t) gfn_hnd, +const unsigned int len) +{ +unsigned int i; +int ret =0 ; +mfn_t *mfns; +void **mfn_mapping; + +ASSERT (LOCKING_Write_rings_L2 (d)); + +if ( ring_info->mfns ) + { +/* Ring already existed: drop the previous mapping. */ +gprintk (XENLOG_INFO ,"argo: vm%u re-register existing ring " +"(vm%u:%x vm%u) clears mapping\n" , + d->domain_id, ring_info->id.domain_id, + ring_info->id.aport, ring_info->id.partner_id); + +ring_remove_mfns (d, ring_info); +ASSERT (!ring_info->mfns); + } + + mfns =xmalloc_array (mfn_t, npage); +if ( !mfns ) +return -ENOMEM ; + +for ( i =0 ; i < npage; i++ ) + mfns[i] =INVALID_MFN ; + + mfn_mapping = xzalloc_array(void *, npage); +if ( !mfn_mapping ) + { +xfree (mfns); +return -ENOMEM ; + } + + ring_info->mfns = mfns; + ring_info->mfn_mapping = mfn_mapping; + +for ( i =0 ; i < npage; i++ ) + { +xen_argo_gfn_t argo_gfn; +mfn_t mfn; + + ret =__copy_from_guest_offset (&argo_gfn, gfn_hnd, i,1 ) ? -EFAULT :0 ; +if ( ret ) +break ; + + ret =find_ring_mfn (d,_gfn (argo_gfn), &mfn); +if ( ret ) + { +gprintk (XENLOG_ERR ,"argo: vm%u: invalid gfn %" PRI_gfn" " +"r:(vm%u:%x vm%u) %p %u/%u\n" , + d->domain_id,gfn_x (_gfn (argo_gfn)), + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id, ring_info, i, npage); +break ; + } + + ring_info->mfns[i] = mfn; + +argo_dprintk ("%u: %" PRI_gfn" -> %" PRI_mfn"\n" , + i,gfn_x (_gfn (argo_gfn)),mfn_x (ring_info->mfns[i])); + } + + ring_info->nmfns = i; + +if ( ret )
Quoted 8 lines
xen/common/argo.c partner_rings_remove(struct domain *src_d) · line 727
Quoted 6 lines
xen/common/argo.c +6
-0+ring_remove_mfns (d, ring_info); + else + { +ASSERT (ring_info->nmfns ==NPAGES_RING (len)); + + gprintk(XENLOG_DEBUG, "argo: vm%u ring (vm%u:%x vm%u) %p "Nit: this likely wants to be an
argo_dprintk?
There are not many instances in the Argo code where gprintk(XENLOG_DEBUG is used, but it's intentional here, because argo_dprintk needs a recompile to enable it, whereas gprintk does not.
Ring registration is non-datapath and the message is potentially useful when diagnosing a deployed system.
Quoted 132 lines
Quoted 132 lines
xen/common/argo.c +131
-0+"mfn_mapping %p len %u nmfns %u\n" , + d->domain_id, ring_info->id.domain_id, + ring_info->id.aport, ring_info->id.partner_id, ring_info, + ring_info->mfn_mapping, ring_info->len, ring_info->nmfns); + } + +return ret; +} + +static long +register_ring (struct domain *currd, +XEN_GUEST_HANDLE_PARAM (xen_argo_register_ring_t) reg_hnd, +XEN_GUEST_HANDLE_PARAM (xen_argo_gfn_t) gfn_hnd, +unsigned int npage,bool fail_exist) +{ +xen_argo_register_ring_t reg; +struct argo_ring_id ring_id; +void *map_ringp; +xen_argo_ring_t *ringp; +struct argo_ring_info *ring_info, *new_ring_info =NULL ; +struct argo_send_info *send_info =NULL ; +struct domain *dst_d =NULL ; +int ret =0 ; +unsigned int private_tx_ptr; + +ASSERT (currd == current->domain); + +if (copy_from_guest (®, reg_hnd,1 ) ) +return -EFAULT ; + +/* +* A ring must be large enough to transmit messages, so requires space for: +* * 1 message header, plus +* * 1 payload slot (payload is always rounded to a multiple of 16 bytes) +* for the message payload to be written into, plus +* * 1 more slot, so that the ring cannot be filled to capacity with a +* single minimum-size message -- see the logic in ringbuf_insert -- +* allowing for this ensures that there can be space remaining when a +* message is present. +* The above determines the minimum acceptable ring size. +*/ +if ( (reg.len < (sizeof (struct xen_argo_ring_message_header ) + +ROUNDUP_MESSAGE (1 ) +ROUNDUP_MESSAGE (1 ))) || + (reg.len >XEN_ARGO_MAX_RING_SIZE ) || + (reg.len !=ROUNDUP_MESSAGE (reg.len)) || + (NPAGES_RING (reg.len) != npage) || + (reg.pad !=0 ) ) +return -EINVAL ; + + ring_id.partner_id = reg.partner_id; + ring_id.aport = reg.aport; + ring_id.domain_id = currd->domain_id; + +if ( reg.partner_id ==XEN_ARGO_DOMID_ANY ) + { +if ( !opt_argo_mac_permissive ) +return -EPERM ; + } +else + { + dst_d =get_domain_by_id (reg.partner_id); +if ( !dst_d ) + { +argo_dprintk ("!dst_d, ESRCH\n" ); +return -ESRCH ; + } + + send_info = xzalloc(struct argo_send_info); +if ( !send_info ) + { + ret = -ENOMEM ; + goto out; + } + send_info->id = ring_id; + } + +/* +* Common case is that the ring doesn't already exist, so do the alloc here +* before picking up any locks. +*/ + new_ring_info = xzalloc(struct argo_ring_info); +if ( !new_ring_info ) + { + ret = -ENOMEM ; + goto out; + } + +read_lock (&L1_global_argo_rwlock); + +if ( !currd->argo ) + { + ret = -ENODEV ; + goto out_unlock; + } + +if ( dst_d && !dst_d->argo ) + { +argo_dprintk ("!dst_d->argo, ECONNREFUSED\n" ); + ret = -ECONNREFUSED ; + goto out_unlock; + } + +write_lock (&currd->argo->rings_L2_rwlock); + +if ( currd->argo->ring_count >=MAX_RINGS_PER_DOMAIN ) + { + ret = -ENOSPC ; + goto out_unlock2; + } + + ring_info =find_ring_info (currd, &ring_id); +if ( !ring_info ) + { + ring_info = new_ring_info; + new_ring_info =NULL ; + +spin_lock_init (&ring_info->L3_lock); + + ring_info->id = ring_id; +INIT_LIST_HEAD (&ring_info->pending); + +list_add (&ring_info->node, + &currd->argo->ring_hash[hash_index (&ring_info->id)]); + +gprintk (XENLOG_DEBUG ,"argo: vm%u registering ring (vm%u:%x vm%u)\n" , + currd->domain_id, ring_id.domain_id, ring_id.aport, + ring_id.partner_id); + } +else if ( ring_info->len ) + { +/*
Quoted 8 lines
xen/common/argo.c partner_rings_remove(struct domain *src_d) · line 864
Quoted 6 lines
xen/common/argo.c +6
-0+* If the caller specified that the ring must not already exist, +* fail at attempt to add a completed ring which already exists. +*/ +if ( fail_exist ) + { +argo_dprintk ("disallowed reregistration of existing ring\n" );And this should likely be gprintk with error type?
ack, yes, thanks.
Quoted 27 lines
I think the pattern of using gprintk for error messages and
argo_dprintkfor verbose information is correct, but there are a couple of oddities that can be fixed later.Quoted 22 lines
xen/common/argo.c +22
-0+ ret = -EEXIST ; + goto out_unlock2; + } + +if ( ring_info->len != reg.len ) + { +/* +* Change of ring size could result in entries on the pending +* notifications list that will never trigger. +* Simple blunt solution: disallow ring resize for now. +* TODO: investigate enabling ring resize. +*/ +gprintk (XENLOG_ERR ,"argo: vm%u attempted to change ring size " +"(vm%u:%x vm%u)\n" , + currd->domain_id, ring_id.domain_id, ring_id.aport, + ring_id.partner_id); +/* +* Could return EINVAL here, but if the ring didn't already +* exist then the arguments would have been valid, so: EEXIST. +*/ + ret = -EEXIST ; + goto out_unlock2;
Quoted 8 lines
xen/common/argo.c partner_rings_remove(struct domain *src_d) · line 892
Quoted 6 lines
xen/common/argo.c +6
-0+ } + +gprintk (XENLOG_DEBUG , +"argo: vm%u re-registering existing ring (vm%u:%x vm%u)\n" , + currd->domain_id, ring_id.domain_id, ring_id.aport, + ring_id.partner_id);This again would better be
argo_dprintkIMO.
similar to above: message useful when deployed, not just for argo development.
Quoted 18 lines
[...]
xen/common/argo.c do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, · line 998
Quoted 13 lines
xen/common/argo.c +9
-0@@ -552,6 +987,38 @@ do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,switch (cmd) { +case XEN_ARGO_OP_register_ring: + { +XEN_GUEST_HANDLE_PARAM (xen_argo_register_ring_t) reg_hnd = +guest_handle_cast (arg1, xen_argo_register_ring_t); +XEN_GUEST_HANDLE_PARAM (xen_argo_gfn_t) gfn_hnd = +guest_handle_cast (arg2, xen_argo_gfn_t); +/* arg3 is npage */ +/* arg4 is flags */ +bool fail_exist = arg4 &XEN_ARGO_REGISTER_FLAG_FAIL_EXIST ;Nit: I would add a:
Quoted 1 line
BUILD_BUG_ON(!IS_ALIGNED(XEN_ARGO_MAX_RING_SIZE, PAGE_SIZE));
ack
thanks,
Christopher
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 8/15 argo: implement the unregister op
In reply to Christopher ClarkTakes a single argument: a handle to the ring unregistration struct, which specifies the port and partner domain id or wildcard.
The ring's entry is removed from the hashtable of registered rings; any entries for pending notifications are removed; and the ring is unmapped from Xen's address space.
If the ring had been registered to communicate with a single specified domain (ie. a non-wildcard ring) then the partner domain state is removed from the partner domain's argo send_info hash table.
---
The logic in unregister_ring got pretty heavily reordered in this version of the patch, to shrink the critical sections. I'm happy with the result. I've added a couple of ASSERT_UNREACHABLEs where it seems appropriate.
v4 # Jan: shrink the critical sections in unregister
v4 : use standard data structures as per common code
v4 #08 Roger: skip send_info lookup for wildcard rings
v4: add ASSERT_UNREACHABLE for missing sender domain or send_info
v4: reduce indentation by using goto
v4: add unlikely to currd->argo check
v4 #08 Jan: move put_domain outside L2 critical section
v4: include ring data in debug output when ring not found
v3 #08 Jan: pull xfree out of exclusive critical sections in unregister_ring v3 #08 Jan: rename send_find_info to find_send_info
v3 #07 Jan: rename ring_find_info to find_ring_info
v3 #08 Roger: use return and remove the out label in unregister_ring v3 #08 Roger: better debug output in send_find_info
v3 #10 Roger: move find functions to top of file and drop prototypes v3 #04 Jan: meld compat check for unregister_ring struct
v3 #04 Roger/Jan: make lock names clearer and assert their state v3 #04 Jan: port -> aport with type; distinguish argo port from evtchn v3 feedback Roger/Jan: ASSERT currd is current->domain or use 'd' variable name v3 feedback #07 Roger: const the argo_ring_id structs in send_find_info v2 feedback Jan: drop cookie, implement teardown
v2 feedback Jan: drop message from argo_message_op
v2 self: OVERHAUL
v2 self: reorder logic to shorten critical section
v1 #13 feedback Jan: revise use of guest_handle_okay vs __copy ops v1 feedback Roger, Jan: drop argo prefix on static functions
v1,2 feedback Jan/Roger/Paul: drop errno returning guest access functions v1 #5 (#14) feedback Paul: use currd in do_argo_message_op
v1 #5 (#14) feedback Paul: full use currd in argo_unregister_ring v1 #13 (#14) feedback Paul: replace do/while with goto; reindent v1 self: add blank lines in unregister case in do_argo_message_op
v1: #13 feedback Jan: public namespace: prefix with xen
v1: #13 feedback Jan: blank line after op case in do_argo_message_op
v1: #14 feedback Jan: replace domain id override with validation
v1: #18 feedback Jan: meld the ring count limit into the series
v1: feedback #15 Jan: verify zero in unused hypercall args
xen/common/argo.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++ xen/common/compat/argo.c | 1 + xen/include/public/argo.h | 19 +++++++ xen/include/xlat.lst | 1 + 4 files changed, 147 insertions(+)
xen/common/argo.c +126 -0
@@ -43,6 +43,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_argo_addr_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_gfn_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_register_ring_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_ring_t); +DEFINE_XEN_GUEST_HANDLE (xen_argo_unregister_ring_t);static bool __read_mostly opt_argo;static bool __read_mostly opt_argo_mac_permissive;
@@ -351,6 +352,37 @@ find_ring_info(const struct domain *d, const struct argo_ring_id *id)return NULL ; } +static struct argo_send_info * +find_send_info (const struct domain *d,const struct argo_ring_id *id) +{ +struct list_head *cursor, *bucket; + +ASSERT (LOCKING_send_L2 (d)); + +/* List is not modified here. Search and return the match if found. */ + bucket = &d->argo->send_hash[hash_index (id)]; + +for ( cursor = bucket->next; cursor != bucket; cursor = cursor->next ) + { +struct argo_send_info *send_info = + list_entry(cursor, struct argo_send_info, node); +const struct argo_ring_id *cmpid = &send_info->id; + +if ( cmpid->aport == id->aport && + cmpid->domain_id == id->domain_id && + cmpid->partner_id == id->partner_id ) + { +argo_dprintk ("found send_info for ring(%u:%x %u)\n" , + id->domain_id, id->aport, id->partner_id); +return send_info; + } + } +argo_dprintk ("no send_info for ring(%u:%x %u)\n" , + id->domain_id, id->aport, id->partner_id); + +return NULL ; +} +static void ring_unmap (const struct domain *d,struct argo_ring_info *ring_info) {
@@ -735,6 +767,85 @@ find_ring_mfns(struct domain *d, struct argo_ring_info *ring_info, }static long +unregister_ring (struct domain *currd, +XEN_GUEST_HANDLE_PARAM (xen_argo_unregister_ring_t) unreg_hnd) +{ +xen_argo_unregister_ring_t unreg; +struct argo_ring_id ring_id; +struct argo_ring_info *ring_info =NULL ; +struct argo_send_info *send_info =NULL ; +struct domain *dst_d =NULL ; + +ASSERT (currd == current->domain); + +if (copy_from_guest (&unreg, unreg_hnd,1 ) ) +return -EFAULT ; + +if ( unreg.pad ) +return -EINVAL ; + + ring_id.partner_id = unreg.partner_id; + ring_id.aport = unreg.aport; + ring_id.domain_id = currd->domain_id; + +read_lock (&L1_global_argo_rwlock); + +if (unlikely (!currd->argo) ) + { +read_unlock (&L1_global_argo_rwlock); +return -ENODEV ; + } + +write_lock (&currd->argo->rings_L2_rwlock); + + ring_info =find_ring_info (currd, &ring_id); +if ( !ring_info ) + goto out; + +ring_remove_info (currd, ring_info); + currd->argo->ring_count--; + +if ( ring_id.partner_id ==XEN_ARGO_DOMID_ANY ) + goto out; + + dst_d =get_domain_by_id (ring_id.partner_id); +if ( !dst_d || !dst_d->argo ) + { +ASSERT_UNREACHABLE (); + goto out; + } + +spin_lock (&dst_d->argo->send_L2_lock); + + send_info =find_send_info (dst_d, &ring_id); +if ( send_info ) +list_del (&send_info->node); +else +ASSERT_UNREACHABLE (); + +spin_unlock (&dst_d->argo->send_L2_lock); + + out: +write_unlock (&currd->argo->rings_L2_rwlock); + +read_unlock (&L1_global_argo_rwlock); + +if ( dst_d ) +put_domain (dst_d); + +xfree (send_info); + +if ( !ring_info ) + { +argo_dprintk ("unregister_ring: no ring_info found for ring(%u:%x %u)\n" , + ring_id.domain_id, ring_id.aport, ring_id.partner_id); +return -ENOENT ; + } + +return 0 ; +} + +static long register_ring(struct domain *currd, XEN_GUEST_HANDLE_PARAM(xen_argo_register_ring_t) reg_hnd, XEN_GUEST_HANDLE_PARAM(xen_argo_gfn_t) gfn_hnd,
@@ -1019,6 +1130,21 @@ do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,break ; } +case XEN_ARGO_OP_unregister_ring: + { +XEN_GUEST_HANDLE_PARAM (xen_argo_unregister_ring_t) unreg_hnd = +guest_handle_cast (arg1, xen_argo_unregister_ring_t); + +if (unlikely ((!guest_handle_is_null (arg2)) || arg3 || arg4) ) + { + rc = -EINVAL ; +break ; + } + + rc =unregister_ring (currd, unreg_hnd); +break ; + } +default : rc = -EOPNOTSUPP ;break ;
xen/common/compat/argo.c +1 -0
@@ -22,3 +22,4 @@ CHECK_argo_addr; CHECK_argo_register_ring; CHECK_argo_ring; +CHECK_argo_unregister_ring;
xen/include/public/argo.h +19 -0
@@ -79,6 +79,13 @@ typedef struct xen_argo_register_ringuint32_t len; } xen_argo_register_ring_t; +typedef struct xen_argo_unregister_ring +{ +xen_argo_port_t aport; +domid_t partner_id; +uint16_t pad; +}xen_argo_unregister_ring_t ; +/* Messages on the ring are padded to a multiple of this size. */ #define XEN_ARGO_MSG_SLOT_SIZE 0x10
@@ -134,4 +141,16 @@ struct xen_argo_ring_message_header#define XEN_ARGO_REGISTER_FLAG_MASK XEN_ARGO_REGISTER_FLAG_FAIL_EXIST#endif +/* +* XEN_ARGO_OP_unregister_ring +* +* Unregister a previously-registered ring, ending communication. +* +* arg1: XEN_GUEST_HANDLE(xen_argo_unregister_ring_t) +* arg2: NULL +* arg3: 0 (ZERO) +* arg4: 0 (ZERO) +*/ +#define XEN_ARGO_OP_unregister_ring 2 +#endif
xen/include/xlat.lst +1 -0
@@ -151,3 +151,4 @@ ? argo_addr argo.h ? argo_ring argo.h ? argo_register_ring argo.h +? argo_unregister_ring argo.h
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 08/15] argo: implement the unregister op
Quoted 7 lines · Christopher Clark
Takes a single argument: a handle to the ring unregistration struct, which specifies the port and partner domain id or wildcard.
The ring's entry is removed from the hashtable of registered rings; any entries for pending notifications are removed; and the ring is unmapped from Xen's address space.
Quoted 5 lines
If the ring had been registered to communicate with a single specified domain (ie. a non-wildcard ring) then the partner domain state is removed from the partner domain's argo
send_infohash table.Signed-off-by Christopher Clark <[email protected]>
LGTM:
Just one nit about the open-coded list_for_each.
Quoted 17 lines
xen/common/argo.c +2
-0@@ -43,6 +43,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_argo_addr_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_gfn_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_register_ring_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_ring_t); +DEFINE_XEN_GUEST_HANDLE (xen_argo_unregister_ring_t);static bool __read_mostly opt_argo;static bool __read_mostly opt_argo_mac_permissive;@@ -351,6 +352,37 @@ find_ring_info(const struct domain *d, const struct argo_ring_id *id)return NULL ; } +static struct argo_send_info *
xen/common/argo.c find_ring_info(const struct domain *d, const struct argo_ring_id *id) · line 365
Quoted 10 lines
xen/common/argo.c +10
-0+find_send_info(const struct domain *d, const struct argo_ring_id *id) +{ + struct list_head *cursor, *bucket; + +ASSERT (LOCKING_send_L2 (d)); + +/* List is not modified here. Search and return the match if found. */ + bucket = &d->argo->send_hash[hash_index (id)]; + +for ( cursor = bucket->next; cursor != bucket; cursor = cursor->next )
list_for_each
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 9/15 argo: implement the sendv op; evtchn: expose send_guest_global_virq
In reply to Christopher Clarksendv operation is invoked to perform a synchronous send of buffers contained in iovs to a remote domain's registered ring.
It takes:
* A destination address (domid, port) for the ring to send to. It performs a most-specific match lookup, to allow for wildcard.
* A source address, used to inform the destination of where to reply.
* The address of an array of iovs containing the data to send
* .. and the length of that array of iovs
* and a 32-bit message type, available to communicate message context data (eg. kernel-to-kernel, separate from the application data).
If insufficient space exists in the destination ring, it will return -EAGAIN and Xen will notify the caller when sufficient space becomes available.
Accesses to the ring indices are appropriately atomic. The rings are mapped into Xen's private address space to write as needed and the mappings are retained for later use.
Notifications are sent to guests via VIRQ and send_guest_global_virq is exposed in the change to enable argo to call it. VIRQ_ARGO_MESSAGE is claimed from the VIRQ previously reserved for this purpose (#11).
The VIRQ notification method is used rather than sending events using evtchn functions directly because:
* no current event channel type is an exact fit for the intended behaviour. ECS_IPI is closest, but it disallows migration to other VCPUs which is not necessarily a requirement for Argo.
* at the point of argo_init, allocation of an event channel is complicated by none of the guest VCPUs being initialized yet and the event channel logic expects that a valid event channel has a present VCPU.
* at the point of signalling a notification, the VIRQ logic is already
defensive: if d->vcpu[0] is NULL, the notification is just silently dropped, whereas the evtchn_send logic is not so defensive: vcpu[0] must not be NULL, otherwise a null pointer dereference occurs.
Using a VIRQ removes the need for the guest to query to determine which event channel notifications will be delivered on. This is also likely to simplify establishing future L0/L1 nested hypervisor argo communication.
---
v4 Jan: remove use of fixed-width types from iov_count, ringbuf_insert v4 #07 Jan: shrink critical sections in sendv
v3 #07 Jan: header: note 32-bitness of hypercall message tuype arg v4 : use standard data structures as per common code
v4 self: bugfix memcpy_to_guest_ring: head_len must check (offset + len) v4 #09 Roger: drop MESSAGE from VIRQ_ARGO_MESSAGE
v3 #07 Jan: rename ring_find_info* to find_ring_info*
v3 #07 Jan: fix numeric entries in printk format strings
v3 #10 Roger: move find functions to top of file and drop prototypes v3 #04 Jan: meld compat struct checking for hypercall args
v3 #04 Roger/Jan: make lock names clearer and assert their state v3 #04 Jan: port -> aport with type; distinguish argo port from evtchn v3 feedback #09 Eric: fix len & offset sanity check in memcpy_to_guest_ring v3 feedback #04 Roger: newline fix in wildcard_pending_list_insert v3 feedback #04 Roger: drop npages struct member, calculate from len v3 #09 Roger: simplify EFAULT return in memcpy_to_guest_ring
v3 #09 Roger: add newline before return in get_sanitized_ring v3 #09 Roger: replace while with for loop in iov_count
v3 #09 Roger: drop 0 in struct init in ringbuf_insert
v3 #09 Roger: comment for XEN_ARGO_MAXIOV: warn of stack overflow risk v3 #09 Roger: simplify while loop: for instead in ringbuf_insert v3 #09 Roger: drop out label for returns in ringbuf_insert
v3 #09 Roger: drop newline in pending_queue
v3 #09 Roger: replace second goto label with error path unlock in sendv v3 #09 Jason: check iov_len vs MAX_ARGO_MESSAGE_SIZE in iov_count v3 #09 Jason: check padding is zeroed in sendv op
v3 #09 Jason: memcpy_to_guest_ring: simpler code with better loop
v2 self: use ring_info backpointer in pending_ent to maintain npending v2 feedback Jan: drop cookie, implement teardown
v2 self: pending_queue: reap stale ents when in need of space
v2 self: pending_requeue: reclaim ents for stale domains v2.feedback Jan: only override sender domid if DOMID_ANY
v2 feedback Jan: drop message from argo_message_op
v2 self: check npending vs maximum limit
v2 self: get_sanitized_ring instead of get_rx_ptr
v2 feedback v1#13 Jan: remove double read from ringbuf insert, lower MAX_IOV v2 self: make iov_count const
v2 self: iov_count : return EMSGSIZE for message too big
v2 self: OVERHAUL
v2 self: s/argo_pending_ent/pending_ent/g
v2 feedback v1#13 Roger: use OS-supplied roundup; drop from public header v1,2 feedback Jan/Roger/Paul: drop errno returning guest access functions v1 feedback Roger, Jan: drop argo prefix on static functions
v1 feedback #13 Jan: drop guest_handle_okay when using copy_from_guest
- reorder do_argo_op logic
v2 self: add _hnd suffix to iovs variable name to indicate guest handle type v2 self: replace use of XEN_GUEST_HANDLE_NULL with two existing macros
v1 #15 feedback, Jan: sendv op : s/ECONNREFUSED/ESRCH/
v1 #5 (#15) feedback Paul: sendv: use currd in do_argo_message_op
v1 #13 (#15) feedback Paul: sendv op: do/while reindent only
v1 #13 (#15) feedback Paul: sendv op: do/while: argo_ringbuf_insert to goto style v1 #13 (#15) feedback Paul: sendv op: do/while: reindent only again v1 #13 (#15) feedback Paul: sendv op: do/while : goto
v1 #15 feedback Paul: sendv op: make page var: unsigned
v1 #15 feedback Paul: sendv op: new local var for PAGE_SIZE - offset v1 #8 feedback Jan: XEN_GUEST_HANDLE : C89 compliance
v1 rebase after switching register op from pfns to page descriptors v1 self: move iov DEFINE_XEN_GUEST_HANDLE out of public header into argo.c v1 #13 (#15) feedback Paul: fix loglevel for guest-triggered messages v1 : add compat xlat.lst entries
v1 self: switched notification to send_guest_global_virq instead of event
v1: fix gprintk use for ARM as its defn dislikes split format strings
v1: init len variable to satisfy ARM compiler initialized checking v1 #13 feedback Jan: rename page var
v1:#14 feedback Jan: uint8_t* -> void*
v1: #13 feedback Jan: public namespace: prefix with xen
v1: #13 feedback Jan: blank line after case op in do_argo_message_op
v1: #15 feedback Jan: add comments explaining why the writes don't overrun
v1: self: add ASSERT to support comment that overrun cannot happen
v1: self: fail on short writes where guest manipulated the iov_lens
v1: self: rename ent id to domain_id
v1: self: add moan for iov rewrite
v1. feedback #15 Jan: require the pad bits are zero
v1. feedback #15 Jan: drop NULL check in argo_signal_domain as now using VIRQ v1. self: store domain_cookie in pending ent
v1. feedback #15 Jan: use unsigned where possible
v1. feedback Jan: use handle type for iov_base in public iov interface v1. self: log whenever visible error occurs
v1 feedback #15, Jan: drop unnecessary mb
v1 self: only update internal tx_ptr if able to return success
and update the visible tx_ptr
v1 self: log on failure to map ring to update visible tx_ptr
v1 feedback #15 Jan: add comment re: notification size policy
v1 self/Roger? remove errant space after sizeof
v1. feedback #15 Jan: require iov pad be zero
v1. self: rename iov_base to iov_hnd for handle in public iov interface
v1: feedback #15 Jan: handle upper-halves of hypercall args; changes some
types in function signatures to match.
v1: self: add dprintk to sendv
v1: self: add debug output to argo_iov_count
v1. feedback #14 Jan: blank line before return in argo_iov_count
v1 feedback #15 Jan: verify src id, not override
xen/common/argo.c | 636 +++++++++++++++++++++++++++++++++++++++++++++ xen/common/compat/argo.c | 19 ++ xen/common/event_channel.c | 2 +- xen/include/public/argo.h | 60 +++++ xen/include/public/xen.h | 2 +- xen/include/xen/event.h | 7 + xen/include/xlat.lst | 2 + 7 files changed, 726 insertions(+), 2 deletions(-)
xen/common/argo.c +636 -0
@@ -30,10 +30,15 @@#include <public/argo.h> #define MAX_RINGS_PER_DOMAIN 128U +#define MAX_PENDING_PER_RING 32U/* All messages on the ring are padded to a multiple of the slot size. */ #define ROUNDUP_MESSAGE (a) ROUNDUP((a), XEN_ARGO_MSG_SLOT_SIZE) +/* The maximum size of a message that may be sent on the largest Argo ring. */ +#define MAX_ARGO_MESSAGE_SIZE ((XEN_ARGO_MAX_RING_SIZE) - \ + (sizeof(struct xen_argo_ring_message_header)) - ROUNDUP_MESSAGE(1)) +/* Number of PAGEs needed to hold a ring of a given size in bytes */ #define NPAGES_RING (ring_len) \ (ROUNDUP((ROUNDUP_MESSAGE(ring_len) + sizeof(xen_argo_ring_t)), PAGE_SIZE) \
@@ -41,8 +46,10 @@DEFINE_XEN_GUEST_HANDLE (xen_argo_addr_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_gfn_t); +DEFINE_XEN_GUEST_HANDLE (xen_argo_iov_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_register_ring_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_ring_t); +DEFINE_XEN_GUEST_HANDLE (xen_argo_send_addr_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_unregister_ring_t);static bool __read_mostly opt_argo;
@@ -352,6 +359,28 @@ find_ring_info(const struct domain *d, const struct argo_ring_id *id)return NULL ; } +static struct argo_ring_info * +find_ring_info_by_match (const struct domain *d,xen_argo_port_t aport, +domid_t partner_id) +{ +struct argo_ring_id id; +struct argo_ring_info *ring_info; + +ASSERT (LOCKING_Read_rings_L2 (d)); + + id.aport = aport; + id.domain_id = d->domain_id; + id.partner_id = partner_id; + + ring_info =find_ring_info (d, &id); +if ( ring_info ) +return ring_info; + + id.partner_id =XEN_ARGO_DOMID_ANY ; + +return find_ring_info (d, &id); +} +static struct argo_send_info *find_send_info (const struct domain *d,const struct argo_ring_id *id) {
@@ -384,6 +413,14 @@ find_send_info(const struct domain *d, const struct argo_ring_id *id) }static void +signal_domain (struct domain *d) +{ +argo_dprintk ("signalling domid:%u\n" , d->domain_id); + +send_guest_global_virq (d,VIRQ_ARGO ); +} + +static void ring_unmap (const struct domain *d,struct argo_ring_info *ring_info) {unsigned int i;
@@ -477,6 +514,390 @@ update_tx_ptr(const struct domain *d, struct argo_ring_info *ring_info,smp_wmb (); } +static int +memcpy_to_guest_ring (const struct domain *d,struct argo_ring_info *ring_info, +unsigned int offset, +const void *src,XEN_GUEST_HANDLE (uint8_t ) src_hnd, +unsigned int len) +{ +unsigned int mfns_index = offset >>PAGE_SHIFT ; +void *dst; +int ret; +unsigned int src_offset =0 ; + +ASSERT (LOCKING_L3 (d, ring_info)); + + offset &= ~PAGE_MASK ; + +if ( len + offset >XEN_ARGO_MAX_RING_SIZE ) +return -EFAULT ; + +while ( len ) + { +unsigned int head_len = (offset + len) >PAGE_SIZE ?PAGE_SIZE - offset + : len; + + ret =ring_map_page (d, ring_info, mfns_index, &dst); +if ( ret ) +return ret; + +if ( src ) + { +memcpy (dst + offset, src + src_offset, head_len); + src_offset += head_len; + } +else + { +if (copy_from_guest (dst + offset, src_hnd, head_len) ) +return -EFAULT ; + +guest_handle_add_offset (src_hnd, head_len); + } + + mfns_index++; + len -= head_len; + offset =0 ; + } + +return 0 ; +} + +/* +* Use this with caution: rx_ptr is under guest control and may be bogus. +* See get_sanitized_ring for a safer alternative. +*/ +static int +get_rx_ptr (const struct domain *d,struct argo_ring_info *ring_info, +uint32_t *rx_ptr) +{ +void *src; +xen_argo_ring_t *ringp; +int ret; + +ASSERT (LOCKING_L3 (d, ring_info)); + +if ( !ring_info->nmfns || ring_info->nmfns <NPAGES_RING (ring_info->len) ) +return -EINVAL ; + + ret =ring_map_page (d, ring_info,0 , &src); +if ( ret ) +return ret; + + ringp = (xen_argo_ring_t *)src; + + *rx_ptr =read_atomic (&ringp->rx_ptr); + +return 0 ; +} + +/* +* get_sanitized_ring creates a modified copy of the ring pointers where +* the rx_ptr is rounded up to ensure it is aligned, and then ring +* wrap is handled. Simplifies safe use of the rx_ptr for available +* space calculation. +*/ +static int +get_sanitized_ring (const struct domain *d,xen_argo_ring_t *ring, +struct argo_ring_info *ring_info) +{ +uint32_t rx_ptr; +int ret; + +ASSERT (LOCKING_L3 (d, ring_info)); + + ret =get_rx_ptr (d, ring_info, &rx_ptr); +if ( ret ) +return ret; + + ring->tx_ptr = ring_info->tx_ptr; + + rx_ptr =ROUNDUP_MESSAGE (rx_ptr); +if ( rx_ptr >= ring_info->len ) + rx_ptr =0 ; + + ring->rx_ptr = rx_ptr; + +return 0 ; +} + +/* +* iov_count returns its count on success via an out variable to avoid +* potential for a negative return value to be used incorrectly +* (eg. coerced into an unsigned variable resulting in a large incorrect value) +*/ +static int +iov_count (const xen_argo_iov_t *piov,unsigned long niov, +unsigned int *count) +{ +unsigned int sum_iov_lens =0 ; + +if ( niov >XEN_ARGO_MAXIOV ) +return -EINVAL ; + +for ( ; niov--; piov++ ) + { +/* valid iovs must have the padding field set to zero */ +if ( piov->pad ) + { +argo_dprintk ("invalid iov: padding is not zero\n" ); +return -EINVAL ; + } + +/* check each to protect sum against integer overflow */ +if ( piov->iov_len >MAX_ARGO_MESSAGE_SIZE ) + { +argo_dprintk ("invalid iov_len: too big (%u)>%llu\n" , + piov->iov_len,MAX_ARGO_MESSAGE_SIZE ); +return -EINVAL ; + } + + sum_iov_lens += piov->iov_len; + +/* +* Again protect sum from integer overflow +* and ensure total msg size will be within bounds. +*/ +if ( sum_iov_lens >MAX_ARGO_MESSAGE_SIZE ) + { +argo_dprintk ("invalid iov series: total message too big\n" ); +return -EMSGSIZE ; + } + } + + *count = sum_iov_lens; + +return 0 ; +} + +static int +ringbuf_insert (const struct domain *d,struct argo_ring_info *ring_info, +const struct argo_ring_id *src_id, +XEN_GUEST_HANDLE_PARAM (xen_argo_iov_t) iovs_hnd, +unsigned long niov,uint32_t message_type, +unsigned long *out_len) +{ +xen_argo_ring_t ring; +struct xen_argo_ring_message_header mh = { }; +int sp, ret; +unsigned int len =0 ; +xen_argo_iov_t iovs[XEN_ARGO_MAXIOV ]; +xen_argo_iov_t *piov; +XEN_GUEST_HANDLE (uint8_t ) NULL_hnd = +guest_handle_from_param (guest_handle_from_ptr (NULL , uint8_t), uint8_t); + +ASSERT (LOCKING_L3 (d, ring_info)); + + ret =__copy_from_guest (iovs, iovs_hnd, niov) ? -EFAULT :0 ; +if ( ret ) +return ret; + +/* +* Obtain the total size of data to transmit -- sets the 'len' variable +* -- and sanity check that the iovs conform to size and number limits. +* Enforced below: no more than 'len' bytes of guest data +* (plus the message header) will be sent in this operation. +*/ + ret =iov_count (iovs, niov, &len); +if ( ret ) +return ret; + +/* +* Size bounds check against ring size and static maximum message limit. +* The message must not fill the ring; there must be at least one slot +* remaining so we can distinguish a full ring from an empty one. +*/ +if ( ((ROUNDUP_MESSAGE (len) + +sizeof (struct xen_argo_ring_message_header )) >= ring_info->len) || + (len >MAX_ARGO_MESSAGE_SIZE ) ) +return -EMSGSIZE ; + + ret =get_sanitized_ring (d, &ring, ring_info); +if ( ret ) +return ret; + +argo_dprintk ("ring.tx_ptr=%u ring.rx_ptr=%u ring len=%u" +" ring_info->tx_ptr=%u\n" , + ring.tx_ptr, ring.rx_ptr, ring_info->len, ring_info->tx_ptr); + +if ( ring.rx_ptr == ring.tx_ptr ) + sp = ring_info->len; +else + { + sp = ring.rx_ptr - ring.tx_ptr; +if ( sp <0 ) + sp += ring_info->len; + } + +/* +* Size bounds check against currently available space in the ring. +* Again: the message must not fill the ring leaving no space remaining. +*/ +if ( (ROUNDUP_MESSAGE (len) + +sizeof (struct xen_argo_ring_message_header )) >= sp ) + { +argo_dprintk ("EAGAIN\n" ); +return -EAGAIN ; + } + + mh.len = len +sizeof (struct xen_argo_ring_message_header ); + mh.source.aport = src_id->aport; + mh.source.domain_id = src_id->domain_id; + mh.message_type = message_type; + +/* +* For this copy to the guest ring, tx_ptr is always 16-byte aligned +* and the message header is 16 bytes long. +*/ +BUILD_BUG_ON ( +sizeof (struct xen_argo_ring_message_header ) !=ROUNDUP_MESSAGE (1 )); + +/* +* First data write into the destination ring: fixed size, message header. +* This cannot overrun because the available free space (value in 'sp') +* is checked above and must be at least this size. +*/ + ret =memcpy_to_guest_ring (d, ring_info, + ring.tx_ptr +sizeof (xen_argo_ring_t), + &mh, NULL_hnd,sizeof (mh)); +if ( ret ) + { +gprintk (XENLOG_ERR , +"argo: failed to write message header to ring (vm%u:%x vm%u)\n" , + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +return ret; + } + + ring.tx_ptr +=sizeof (mh); +if ( ring.tx_ptr == ring_info->len ) + ring.tx_ptr =0 ; + +for ( piov = iovs; niov--; piov++ ) + { +XEN_GUEST_HANDLE_64 (uint8_t ) buf_hnd = piov->iov_hnd; +unsigned int iov_len = piov->iov_len; + +/* If no data is provided in this iov, moan and skip on to the next */ +if ( !iov_len ) + { +gprintk (XENLOG_ERR , +"argo: no data iov_len=0 iov_hnd=%p ring (vm%u:%x vm%u)\n" , + buf_hnd.p, ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +continue ; + } + +if (unlikely (!guest_handle_okay (buf_hnd, iov_len)) ) + { +gprintk (XENLOG_ERR , +"argo: bad iov handle [%p, %u] (vm%u:%x vm%u)\n" , + buf_hnd.p, iov_len, + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +return -EFAULT ; + } + + sp = ring_info->len - ring.tx_ptr; + +/* Check: iov data size versus free space at the tail of the ring */ +if ( iov_len > sp ) + { +/* +* Second possible data write: ring-tail-wrap-write. +* Populate the ring tail and update the internal tx_ptr to handle +* wrapping at the end of ring. +* Size of data written here: sp +* which is the exact full amount of free space available at the +* tail of the ring, so this cannot overrun. +*/ + ret =memcpy_to_guest_ring (d, ring_info, + ring.tx_ptr +sizeof (xen_argo_ring_t), +NULL , buf_hnd, sp); +if ( ret ) + { +gprintk (XENLOG_ERR , +"argo: failed to copy {%p, %d} (vm%u:%x vm%u)\n" , + buf_hnd.p, sp, + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +return ret; + } + + ring.tx_ptr =0 ; + iov_len -= sp; +guest_handle_add_offset (buf_hnd, sp); + +ASSERT (iov_len <= ring_info->len); + } + +/* +* Third possible data write: all data remaining for this iov. +* Size of data written here: iov_len +* +* Case 1: if the ring-tail-wrap-write above was performed, then +* iov_len has been decreased by 'sp' and ring.tx_ptr is zero. +* +* We know from checking the result of iov_count: +* len + sizeof(message_header) <= ring_info->len +* We also know that len is the total of summing all iov_lens, so: +* iov_len <= len +* so by transitivity: +* iov_len <= len <= (ring_info->len - sizeof(msgheader)) +* and therefore: +* (iov_len + sizeof(msgheader) <= ring_info->len) && +* (ring.tx_ptr == 0) +* so this write cannot overrun here. +* +* Case 2: ring-tail-wrap-write above was not performed +* -> so iov_len is the guest-supplied value and: (iov_len <= sp) +* ie. less than available space at the tail of the ring: +* so this write cannot overrun. +*/ + ret =memcpy_to_guest_ring (d, ring_info, + ring.tx_ptr +sizeof (xen_argo_ring_t), +NULL , buf_hnd, iov_len); +if ( ret ) + { +gprintk (XENLOG_ERR , +"argo: failed to copy [%p, %u] (vm%u:%x vm%u)\n" , + buf_hnd.p, iov_len, ring_info->id.domain_id, + ring_info->id.aport, ring_info->id.partner_id); + +return ret; + } + + ring.tx_ptr += iov_len; + +if ( ring.tx_ptr == ring_info->len ) + ring.tx_ptr =0 ; + } + + ring.tx_ptr =ROUNDUP_MESSAGE (ring.tx_ptr); + +if ( ring.tx_ptr >= ring_info->len ) + ring.tx_ptr -= ring_info->len; + +update_tx_ptr (d, ring_info, ring.tx_ptr); + +/* +* At this point (and also on an error exit paths from this function) it is +* possible to unmap the ring_info, ie: +* ring_unmap(d, ring_info); +* but performance should be improved by not doing so, and retaining +* the mapping. +* An XSM policy control over level of confidentiality required +* versus performance cost could be added to decide that here. +*/ + + *out_len = len; + +return ret; +} +static void wildcard_pending_list_remove (domid_t domain_id,struct pending_ent *ent) {
@@ -497,6 +918,25 @@ wildcard_pending_list_remove(domid_t domain_id, struct pending_ent *ent) }static void +wildcard_pending_list_insert (domid_t domain_id,struct pending_ent *ent) +{ +struct domain *d =get_domain_by_id (domain_id); + +if ( !d ) +return ; + +ASSERT (LOCKING_Read_L1); + +if ( d->argo ) + { +spin_lock (&d->argo->wildcard_L2_lock); +list_add (&ent->wildcard_node, &d->argo->wildcard_pend_list); +spin_unlock (&d->argo->wildcard_L2_lock); + } +put_domain (d); +} + +static void pending_remove_all (const struct domain *d,struct argo_ring_info *ring_info) {struct list_head *ring_pending = &ring_info->pending;
@@ -518,6 +958,70 @@ pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) ring_info->npending =0 ; } +static int +pending_queue (const struct domain *d,struct argo_ring_info *ring_info, +domid_t src_id,unsigned int len) +{ +struct pending_ent *ent; + +ASSERT (LOCKING_L3 (d, ring_info)); + +if ( ring_info->npending >=MAX_PENDING_PER_RING ) +return -ENOSPC ; + + ent = xmalloc(struct pending_ent); +if ( !ent ) +return -ENOMEM ; + + ent->len = len; + ent->domain_id = src_id; + ent->ring_info = ring_info; + +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) +wildcard_pending_list_insert (src_id, ent); +list_add (&ent->node, &ring_info->pending); + ring_info->npending++; + +return 0 ; +} + +static int +pending_requeue (const struct domain *d,struct argo_ring_info *ring_info, +domid_t src_id,unsigned int len) +{ +struct list_head *cursor, *head; + +ASSERT (LOCKING_L3 (d, ring_info)); + +/* List structure is not modified here. Update len in a match if found. */ + head = &ring_info->pending; + +for ( cursor = head->next; cursor != head; cursor = cursor->next ) + { + struct pending_ent *ent = list_entry(cursor, struct pending_ent, node); + +if ( ent->domain_id == src_id ) + { +/* +* Reuse an existing queue entry for a notification rather than add +* another. If the existing entry is waiting for a smaller size than +* the current message then adjust the record to wait for the +* current (larger) size to be available before triggering a +* notification. +* This assists the waiting sender by ensuring that whenever a +* notification is triggered, there is sufficient space available +* for (at least) any one of the messages awaiting transmission. +*/ +if ( ent->len < len ) + ent->len = len; + +return 0 ; + } + } + +return pending_queue (d, ring_info, src_id, len); +} +static void wildcard_rings_pending_remove (struct domain *d) {
@@ -1082,6 +1586,91 @@ register_ring(struct domain *currd,return ret; } +static long +sendv (struct domain *src_d,const xen_argo_addr_t *src_addr, +const xen_argo_addr_t *dst_addr, +XEN_GUEST_HANDLE_PARAM (xen_argo_iov_t) iovs_hnd,unsigned long niov, +uint32_t message_type) +{ +struct domain *dst_d =NULL ; +struct argo_ring_id src_id; +struct argo_ring_info *ring_info; +int ret =0 ; +unsigned long len =0 ; + +ASSERT (src_d->domain_id == src_addr->domain_id); + +argo_dprintk ("sendv: (%u:%x)->(%u:%x) niov:%lu iov:%p type:%u\n" , + src_addr->domain_id, src_addr->aport, + dst_addr->domain_id, dst_addr->aport, + niov, iovs_hnd.p, message_type); + + src_id.aport = src_addr->aport; + src_id.domain_id = src_d->domain_id; + src_id.partner_id = dst_addr->domain_id; + + dst_d =get_domain_by_id (dst_addr->domain_id); +if ( !dst_d ) +return -ESRCH ; + +read_lock (&L1_global_argo_rwlock); + +if ( !src_d->argo ) + { + ret = -ENODEV ; + goto out_unlock; + } + +if ( !dst_d->argo ) + { +argo_dprintk ("!dst_d->argo, ECONNREFUSED\n" ); + ret = -ECONNREFUSED ; + goto out_unlock; + } + +read_lock (&dst_d->argo->rings_L2_rwlock); + + ring_info =find_ring_info_by_match (dst_d, dst_addr->aport, + src_id.domain_id); +if ( !ring_info ) + { +gprintk (XENLOG_ERR , +"argo: vm%u connection refused, src (vm%u:%x) dst (vm%u:%x)\n" , + current->domain->domain_id, src_id.domain_id, src_id.aport, + dst_addr->domain_id, dst_addr->aport); + + ret = -ECONNREFUSED ; + } +else + { +spin_lock (&ring_info->L3_lock); + + ret =ringbuf_insert (dst_d, ring_info, &src_id, iovs_hnd, niov, + message_type, &len); +if ( ret == -EAGAIN ) + { +argo_dprintk ("argo_ringbuf_sendv failed, EAGAIN\n" ); +/* requeue to issue a notification when space is there */ + ret =pending_requeue (dst_d, ring_info, src_id.domain_id, len); + } + +spin_unlock (&ring_info->L3_lock); + } + +read_unlock (&dst_d->argo->rings_L2_rwlock); + + out_unlock: +read_unlock (&L1_global_argo_rwlock); + +if ( ret >=0 ) +signal_domain (dst_d); + +if ( dst_d ) +put_domain (dst_d); + +return ( ret <0 ) ? ret : len; +} + long do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, XEN_GUEST_HANDLE_PARAM(void) arg2, unsigned long arg3,
@@ -1145,6 +1734,53 @@ do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,break ; } +case XEN_ARGO_OP_sendv: + { +xen_argo_send_addr_t send_addr; + +XEN_GUEST_HANDLE_PARAM (xen_argo_send_addr_t) send_addr_hnd = +guest_handle_cast (arg1, xen_argo_send_addr_t); +XEN_GUEST_HANDLE_PARAM (xen_argo_iov_t) iovs_hnd = +guest_handle_cast (arg2, xen_argo_iov_t); +/* arg3 is niov */ +/* arg4 is message_type. Must be a 32-bit value. */ + + rc =copy_from_guest (&send_addr, send_addr_hnd,1 ) ? -EFAULT :0 ; +if ( rc ) +break ; + +/* +* Check padding is zeroed. Reject niov above limit or message_types +* that are outside 32 bit range. +*/ +if (unlikely (send_addr.src.pad || send_addr.dst.pad || + (arg3 >XEN_ARGO_MAXIOV ) || (arg4 & ~0xffffffffUL )) ) + { + rc = -EINVAL ; +break ; + } + +if ( send_addr.src.domain_id ==XEN_ARGO_DOMID_ANY ) + send_addr.src.domain_id = currd->domain_id; + +/* No domain is currently authorized to send on behalf of another */ +if (unlikely (send_addr.src.domain_id != currd->domain_id) ) + { + rc = -EPERM ; +break ; + } + +/* +* Check access to the whole array here so we can use the faster __copy +* operations to read each element later. +*/ +if (unlikely (!guest_handle_okay (iovs_hnd, arg3)) ) +break ; + + rc =sendv (currd, &send_addr.src, &send_addr.dst, iovs_hnd, arg3, arg4); +break ; + } +default : rc = -EOPNOTSUPP ;break ;
xen/common/compat/argo.c +19 -0
@@ -23,3 +23,22 @@ CHECK_argo_addr; CHECK_argo_register_ring; CHECK_argo_ring; CHECK_argo_unregister_ring; + +/* +* Disable strict type checking in this compat validation macro for the +* following struct checks because it cannot handle fields within structs that +* have types that differ in the compat versus non-compat structs. +* Replace it with a field size check which is sufficient here. +*/ + +#undef CHECK_FIELD_COMMON_ +#define CHECK_FIELD_COMMON_ (k, name, n, f) \ +static inline int __maybe_unused name(k xen_ ## n *x, k compat_ ## n *c) \ +{ \ + BUILD_BUG_ON(offsetof(k xen_ ## n, f) != \ + offsetof(k compat_ ## n, f)); \ + return sizeof(x->f) == sizeof(c->f); \ +} + +CHECK_argo_send_addr; +CHECK_argo_iov;
xen/common/event_channel.c +1 -1
@@ -746,7 +746,7 @@ void send_guest_vcpu_virq(struct vcpu *v, uint32_t virq)spin_unlock_irqrestore (&v->virq_lock, flags); } -static void send_guest_global_virq (struct domain *d,uint32_t virq) +void send_guest_global_virq (struct domain *d,uint32_t virq) {unsigned long flags;int port;
xen/include/public/argo.h +60 -0
76 lines not on this page · raw message
xen/include/public/xen.h +1 -1
@@ -178,7 +178,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);#define VIRQ_CON_RING 8/* G. (DOM0) Bytes received on console */ #define VIRQ_PCPU_STATE 9/* G. (DOM0) PCPU state changed */ #define VIRQ_MEM_EVENT 10/* G. (DOM0) A memory event has occurred */ -#define VIRQ_XC_RESERVED 11/* G. Reserved for XenClient */ +#define VIRQ_ARGO 11/* G. Argo interdomain message notification */ #define VIRQ_ENOMEM 12/* G. (DOM0) Low on heap memory */ #define VIRQ_XENPMU 13/* V. PMC interrupt */
xen/include/xen/event.h +7 -0
@@ -29,6 +29,13 @@ void send_guest_vcpu_virq(struct vcpu *v, uint32_t virq);void send_global_virq (uint32_t virq);/* +* send_guest_global_virq: +* @d: Domain to which VIRQ should be sent +* @virq: Virtual IRQ number (VIRQ_*), must be global +*/ +void send_guest_global_virq (struct domain *d,uint32_t virq); + +/* * sent_global_virq_handler: Set a global VIRQ handler. * @d: New target domain for this VIRQ * @virq: Virtual IRQ number (VIRQ_*), must be global
xen/include/xlat.lst +2 -0
@@ -152,3 +152,5 @@ ? argo_ring argo.h ? argo_register_ring argo.h ? argo_unregister_ring argo.h +? argo_iov argo.h +? argo_send_addr argo.h
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
Quoted 16 lines · Christopher Clark
sendv operation is invoked to perform a synchronous send of buffers contained in iovs to a remote domain's registered ring.
It takes:
* A destination address (domid, port) for the ring to send to. It performs a most-specific match lookup, to allow for wildcard.
* A source address, used to inform the destination of where to reply.
* The address of an array of iovs containing the data to send
* .. and the length of that array of iovs
* and a 32-bit message type, available to communicate message context data (eg. kernel-to-kernel, separate from the application data).If insufficient space exists in the destination ring, it will return
-EAGAINand Xen will notify the caller when sufficient space becomes available.
Quoted 6 lines
Accesses to the ring indices are appropriately atomic. The rings are mapped into Xen's private address space to write as needed and the mappings are retained for later use.
Notifications are sent to guests via VIRQ and
send_guest_global_virqis exposed in the change to enable argo to call it.VIRQ_ARGO_MESSAGEis
^ VIRQ_ARGO
Quoted 5 lines
claimed from the VIRQ previously reserved for this purpose (#11).
The VIRQ notification method is used rather than sending events using evtchn functions directly because:
Quoted 8 lines
* no current event channel type is an exact fit for the intended behaviour.
ECS_IPIis closest, but it disallows migration to other VCPUs which is not necessarily a requirement for Argo.* at the point of
argo_init, allocation of an event channel is complicated by none of the guest VCPUs being initialized yet and the event channel logic expects that a valid event channel has a present VCPU.
IMO iff you wanted to use event channels those _must_ be setup by the guest, ie: the guest argo driver would load, allocate an event channel and then tell the hypervisor about the event channel that should be used for argo notifications.
Quoted 12 lines
xen/common/argo.c +12
-0+static int +memcpy_to_guest_ring (const struct domain *d,struct argo_ring_info *ring_info, +unsigned int offset, +const void *src,XEN_GUEST_HANDLE (uint8_t ) src_hnd, +unsigned int len) +{ +unsigned int mfns_index = offset >>PAGE_SHIFT ; +void *dst; +int ret; +unsigned int src_offset =0 ; + +ASSERT (LOCKING_L3 (d, ring_info));
xen/common/argo.c update_tx_ptr(const struct domain *d, struct argo_ring_info *ring_info, · line 538
Quoted 10 lines
xen/common/argo.c +10
-0+ + offset &= ~PAGE_MASK ; + +if ( len + offset >XEN_ARGO_MAX_RING_SIZE ) +return -EFAULT ; + +while ( len ) + { +unsigned int head_len = (offset + len) >PAGE_SIZE ?PAGE_SIZE - offset + : len;
IMO that would be clearer as:
head_len = min(PAGE_SIZE - offset, len);
But anyway, this should go away when you move to using vmap.
[...]
Quoted 30 lines
xen/common/argo.c +30
-0+static int +ringbuf_insert (const struct domain *d,struct argo_ring_info *ring_info, +const struct argo_ring_id *src_id, +XEN_GUEST_HANDLE_PARAM (xen_argo_iov_t) iovs_hnd, +unsigned long niov,uint32_t message_type, +unsigned long *out_len) +{ +xen_argo_ring_t ring; +struct xen_argo_ring_message_header mh = { }; +int sp, ret; +unsigned int len =0 ; +xen_argo_iov_t iovs[XEN_ARGO_MAXIOV ]; +xen_argo_iov_t *piov; +XEN_GUEST_HANDLE (uint8_t ) NULL_hnd = +guest_handle_from_param (guest_handle_from_ptr (NULL , uint8_t), uint8_t); + +ASSERT (LOCKING_L3 (d, ring_info)); + + ret =__copy_from_guest (iovs, iovs_hnd, niov) ? -EFAULT :0 ; +if ( ret ) +return ret; + +/* +* Obtain the total size of data to transmit -- sets the 'len' variable +* -- and sanity check that the iovs conform to size and number limits. +* Enforced below: no more than 'len' bytes of guest data +* (plus the message header) will be sent in this operation. +*/ + ret =iov_count (iovs, niov, &len); +if ( ret )
xen/common/argo.c update_tx_ptr(const struct domain *d, struct argo_ring_info *ring_info, · line 711
Quoted 10 lines
xen/common/argo.c +10
-0+return ret; + +/* +* Size bounds check against ring size and static maximum message limit. +* The message must not fill the ring; there must be at least one slot +* remaining so we can distinguish a full ring from an empty one. +*/ +if ( ((ROUNDUP_MESSAGE (len) + +sizeof (struct xen_argo_ring_message_header )) >= ring_info->len) || + (len >MAX_ARGO_MESSAGE_SIZE ) )
len is already checked to be <= MAX_ARGO_MESSAGE_SIZE in iov_count where it gets set, this is redundant.
Quoted 63 lines
xen/common/argo.c +63
-0+return -EMSGSIZE ; + + ret =get_sanitized_ring (d, &ring, ring_info); +if ( ret ) +return ret; + +argo_dprintk ("ring.tx_ptr=%u ring.rx_ptr=%u ring len=%u" +" ring_info->tx_ptr=%u\n" , + ring.tx_ptr, ring.rx_ptr, ring_info->len, ring_info->tx_ptr); + +if ( ring.rx_ptr == ring.tx_ptr ) + sp = ring_info->len; +else + { + sp = ring.rx_ptr - ring.tx_ptr; +if ( sp <0 ) + sp += ring_info->len; + } + +/* +* Size bounds check against currently available space in the ring. +* Again: the message must not fill the ring leaving no space remaining. +*/ +if ( (ROUNDUP_MESSAGE (len) + +sizeof (struct xen_argo_ring_message_header )) >= sp ) + { +argo_dprintk ("EAGAIN\n" ); +return -EAGAIN ; + } + + mh.len = len +sizeof (struct xen_argo_ring_message_header ); + mh.source.aport = src_id->aport; + mh.source.domain_id = src_id->domain_id; + mh.message_type = message_type; + +/* +* For this copy to the guest ring, tx_ptr is always 16-byte aligned +* and the message header is 16 bytes long. +*/ +BUILD_BUG_ON ( +sizeof (struct xen_argo_ring_message_header ) !=ROUNDUP_MESSAGE (1 )); + +/* +* First data write into the destination ring: fixed size, message header. +* This cannot overrun because the available free space (value in 'sp') +* is checked above and must be at least this size. +*/ + ret =memcpy_to_guest_ring (d, ring_info, + ring.tx_ptr +sizeof (xen_argo_ring_t), + &mh, NULL_hnd,sizeof (mh)); +if ( ret ) + { +gprintk (XENLOG_ERR , +"argo: failed to write message header to ring (vm%u:%x vm%u)\n" , + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +return ret; + } + + ring.tx_ptr +=sizeof (mh); +if ( ring.tx_ptr == ring_info->len ) + ring.tx_ptr =0 ;
xen/common/argo.c update_tx_ptr(const struct domain *d, struct argo_ring_info *ring_info, · line 784
Quoted 10 lines
xen/common/argo.c +10
-0+ +for ( piov = iovs; niov--; piov++ ) + { +XEN_GUEST_HANDLE_64 (uint8_t ) buf_hnd = piov->iov_hnd; +unsigned int iov_len = piov->iov_len; + +/* If no data is provided in this iov, moan and skip on to the next */ +if ( !iov_len ) + { + gprintk(XENLOG_ERR,
This should likely be WARN or INFO, since it's not an error?
Quoted 88 lines
xen/common/argo.c +88
-0+"argo: no data iov_len=0 iov_hnd=%p ring (vm%u:%x vm%u)\n" , + buf_hnd.p, ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +continue ; + } + +if (unlikely (!guest_handle_okay (buf_hnd, iov_len)) ) + { +gprintk (XENLOG_ERR , +"argo: bad iov handle [%p, %u] (vm%u:%x vm%u)\n" , + buf_hnd.p, iov_len, + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +return -EFAULT ; + } + + sp = ring_info->len - ring.tx_ptr; + +/* Check: iov data size versus free space at the tail of the ring */ +if ( iov_len > sp ) + { +/* +* Second possible data write: ring-tail-wrap-write. +* Populate the ring tail and update the internal tx_ptr to handle +* wrapping at the end of ring. +* Size of data written here: sp +* which is the exact full amount of free space available at the +* tail of the ring, so this cannot overrun. +*/ + ret =memcpy_to_guest_ring (d, ring_info, + ring.tx_ptr +sizeof (xen_argo_ring_t), +NULL , buf_hnd, sp); +if ( ret ) + { +gprintk (XENLOG_ERR , +"argo: failed to copy {%p, %d} (vm%u:%x vm%u)\n" , + buf_hnd.p, sp, + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +return ret; + } + + ring.tx_ptr =0 ; + iov_len -= sp; +guest_handle_add_offset (buf_hnd, sp); + +ASSERT (iov_len <= ring_info->len); + } + +/* +* Third possible data write: all data remaining for this iov. +* Size of data written here: iov_len +* +* Case 1: if the ring-tail-wrap-write above was performed, then +* iov_len has been decreased by 'sp' and ring.tx_ptr is zero. +* +* We know from checking the result of iov_count: +* len + sizeof(message_header) <= ring_info->len +* We also know that len is the total of summing all iov_lens, so: +* iov_len <= len +* so by transitivity: +* iov_len <= len <= (ring_info->len - sizeof(msgheader)) +* and therefore: +* (iov_len + sizeof(msgheader) <= ring_info->len) && +* (ring.tx_ptr == 0) +* so this write cannot overrun here. +* +* Case 2: ring-tail-wrap-write above was not performed +* -> so iov_len is the guest-supplied value and: (iov_len <= sp) +* ie. less than available space at the tail of the ring: +* so this write cannot overrun. +*/ + ret =memcpy_to_guest_ring (d, ring_info, + ring.tx_ptr +sizeof (xen_argo_ring_t), +NULL , buf_hnd, iov_len); +if ( ret ) + { +gprintk (XENLOG_ERR , +"argo: failed to copy [%p, %u] (vm%u:%x vm%u)\n" , + buf_hnd.p, iov_len, ring_info->id.domain_id, + ring_info->id.aport, ring_info->id.partner_id); + +return ret; + } +
xen/common/argo.c update_tx_ptr(const struct domain *d, struct argo_ring_info *ring_info, · line 882
Quoted 10 lines
xen/common/argo.c +10
-0+ ring.tx_ptr += iov_len; + +if ( ring.tx_ptr == ring_info->len ) + ring.tx_ptr =0 ; + } + + ring.tx_ptr =ROUNDUP_MESSAGE (ring.tx_ptr); + +if ( ring.tx_ptr >= ring_info->len ) + ring.tx_ptr -= ring_info->len;
You seem to handle the wrapping after each possible write, so I think the above is not needed? Maybe it should be an assert instead?
Quoted 80 lines
xen/common/argo.c +66
-0+ +update_tx_ptr (d, ring_info, ring.tx_ptr); + +/* +* At this point (and also on an error exit paths from this function) it is +* possible to unmap the ring_info, ie: +* ring_unmap(d, ring_info); +* but performance should be improved by not doing so, and retaining +* the mapping. +* An XSM policy control over level of confidentiality required +* versus performance cost could be added to decide that here. +*/ + + *out_len = len; + +return ret; +} +static void wildcard_pending_list_remove (domid_t domain_id,struct pending_ent *ent) {@@ -497,6 +918,25 @@ wildcard_pending_list_remove(domid_t domain_id, struct pending_ent *ent) }static void +wildcard_pending_list_insert (domid_t domain_id,struct pending_ent *ent) +{ +struct domain *d =get_domain_by_id (domain_id); + +if ( !d ) +return ; + +ASSERT (LOCKING_Read_L1); + +if ( d->argo ) + { +spin_lock (&d->argo->wildcard_L2_lock); +list_add (&ent->wildcard_node, &d->argo->wildcard_pend_list); +spin_unlock (&d->argo->wildcard_L2_lock); + } +put_domain (d); +} + +static void pending_remove_all (const struct domain *d,struct argo_ring_info *ring_info) {struct list_head *ring_pending = &ring_info->pending;@@ -518,6 +958,70 @@ pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) ring_info->npending =0 ; } +static int +pending_queue (const struct domain *d,struct argo_ring_info *ring_info, +domid_t src_id,unsigned int len) +{ +struct pending_ent *ent; + +ASSERT (LOCKING_L3 (d, ring_info)); + +if ( ring_info->npending >=MAX_PENDING_PER_RING ) +return -ENOSPC ; + + ent = xmalloc(struct pending_ent); +if ( !ent ) +return -ENOMEM ; + + ent->len = len; + ent->domain_id = src_id; + ent->ring_info = ring_info; + +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) +wildcard_pending_list_insert (src_id, ent); +list_add (&ent->node, &ring_info->pending); + ring_info->npending++; + +return 0 ; +} + +static int +pending_requeue(const struct domain *d, struct argo_ring_info *ring_info,
xen/common/argo.c pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) · line 999
Quoted 10 lines
xen/common/argo.c +10
-0+ domid_t src_id, unsigned int len) +{ +struct list_head *cursor, *head; + +ASSERT (LOCKING_L3 (d, ring_info)); + +/* List structure is not modified here. Update len in a match if found. */ + head = &ring_info->pending; + +for ( cursor = head->next; cursor != head; cursor = cursor->next )
list_for_each_entry
Quoted 18 lines
xen/common/argo.c +11
-0long do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, XEN_GUEST_HANDLE_PARAM(void) arg2, unsigned long arg3,@@ -1145,6 +1734,53 @@ do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,break ; } +case XEN_ARGO_OP_sendv: + { +xen_argo_send_addr_t send_addr; + +XEN_GUEST_HANDLE_PARAM (xen_argo_send_addr_t) send_addr_hnd = +guest_handle_cast (arg1, xen_argo_send_addr_t); +XEN_GUEST_HANDLE_PARAM (xen_argo_iov_t) iovs_hnd = +guest_handle_cast (arg2, xen_argo_iov_t); +/* arg3 is niov */ +/* arg4 is message_type. Must be a 32-bit value. */ +
xen/common/argo.c do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, · line 1757
Quoted 10 lines
xen/common/argo.c +10
-0+ rc =copy_from_guest (&send_addr, send_addr_hnd,1 ) ? -EFAULT :0 ; +if ( rc ) +break ; + +/* +* Check padding is zeroed. Reject niov above limit or message_types +* that are outside 32 bit range. +*/ +if (unlikely (send_addr.src.pad || send_addr.dst.pad || + (arg3 >XEN_ARGO_MAXIOV ) || (arg4 & ~0xffffffffUL )) )
arg4 & (GB(4) - 1)
Is clearer IMO, or:
arg4 > UINT32_MAX
Quoted 10 lines
xen/common/argo.c +10
-0+ { + rc = -EINVAL ; +break ; + } + +if ( send_addr.src.domain_id ==XEN_ARGO_DOMID_ANY ) + send_addr.src.domain_id = currd->domain_id; + +/* No domain is currently authorized to send on behalf of another */ +if (unlikely (send_addr.src.domain_id != currd->domain_id) )
xen/common/argo.c do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, · line 1777
Quoted 10 lines
xen/common/argo.c +10
-0+ { + rc = -EPERM ; +break ; + } + +/* +* Check access to the whole array here so we can use the faster __copy +* operations to read each element later. +*/ +if (unlikely (!guest_handle_okay (iovs_hnd, arg3)) )
You need to set rc to EFAULT here, because the call to copy_from_guest has set it to 0.
Alternatively you can change the call above to be:
if ( copy_from_guest(&send_addr, send_addr_hnd, 1) )
return -EFAULT;
So rc doesn't get set to 0 on success.
With those taken care of:
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
Quoted 69 lines · Roger Pau Monné
Quoted 16 lines · Christopher Clark
sendv operation is invoked to perform a synchronous send of buffers contained in iovs to a remote domain's registered ring.
It takes:
* A destination address (domid, port) for the ring to send to. It performs a most-specific match lookup, to allow for wildcard.
* A source address, used to inform the destination of where to reply.
* The address of an array of iovs containing the data to send
* .. and the length of that array of iovs
* and a 32-bit message type, available to communicate message context data (eg. kernel-to-kernel, separate from the application data).If insufficient space exists in the destination ring, it will return
-EAGAINand Xen will notify the caller when sufficient space becomes available.Quoted 6 lines
Accesses to the ring indices are appropriately atomic. The rings are mapped into Xen's private address space to write as needed and the mappings are retained for later use.
Notifications are sent to guests via VIRQ and
send_guest_global_virqis exposed in the change to enable argo to call it.VIRQ_ARGO_MESSAGEis^
VIRQ_ARGOQuoted 5 lines
claimed from the VIRQ previously reserved for this purpose (#11).
The VIRQ notification method is used rather than sending events using evtchn functions directly because:
Quoted 8 lines
* no current event channel type is an exact fit for the intended behaviour.
ECS_IPIis closest, but it disallows migration to other VCPUs which is not necessarily a requirement for Argo.* at the point of
argo_init, allocation of an event channel is complicated by none of the guest VCPUs being initialized yet and the event channel logic expects that a valid event channel has a present VCPU.IMO iff you wanted to use event channels those _must_ be setup by the guest, ie: the guest argo driver would load, allocate an event channel and then tell the hypervisor about the event channel that should be used for argo notifications.
Quoted 12 lines
xen/common/argo.c +12
-0+static int +memcpy_to_guest_ring (const struct domain *d,struct argo_ring_info *ring_info, +unsigned int offset, +const void *src,XEN_GUEST_HANDLE (uint8_t ) src_hnd, +unsigned int len) +{ +unsigned int mfns_index = offset >>PAGE_SHIFT ; +void *dst; +int ret; +unsigned int src_offset =0 ; + +ASSERT (LOCKING_L3 (d, ring_info));xen/common/argo.c update_tx_ptr(const struct domain *d, struct argo_ring_info *ring_info, · line 538
Quoted 10 lines
xen/common/argo.c +10
-0+ + offset &= ~PAGE_MASK ; + +if ( len + offset >XEN_ARGO_MAX_RING_SIZE ) +return -EFAULT ; + +while ( len ) + { +unsigned int head_len = (offset + len) >PAGE_SIZE ?PAGE_SIZE - offset + : len;IMO that would be clearer as:
Quoted 1 line
head_len=min(PAGE_SIZE - offset, len);
You're right that the calculated result should be the same, but I've left this unchanged because I think the reason for using that value (ie. intent) is clearer in the form it has:
it's not about trying to find the smallest amount of data to write, it's about only writing up to the PAGE_SIZE boundary, starting at offset.
Quoted 39 lines
But anyway, this should go away when you move to using vmap.
[...]
Quoted 35 lines
xen/common/argo.c +35
-0+static int +ringbuf_insert (const struct domain *d,struct argo_ring_info *ring_info, +const struct argo_ring_id *src_id, +XEN_GUEST_HANDLE_PARAM (xen_argo_iov_t) iovs_hnd, +unsigned long niov,uint32_t message_type, +unsigned long *out_len) +{ +xen_argo_ring_t ring; +struct xen_argo_ring_message_header mh = { }; +int sp, ret; +unsigned int len =0 ; +xen_argo_iov_t iovs[XEN_ARGO_MAXIOV ]; +xen_argo_iov_t *piov; +XEN_GUEST_HANDLE (uint8_t ) NULL_hnd = +guest_handle_from_param (guest_handle_from_ptr (NULL , uint8_t), uint8_t); + +ASSERT (LOCKING_L3 (d, ring_info)); + + ret =__copy_from_guest (iovs, iovs_hnd, niov) ? -EFAULT :0 ; +if ( ret ) +return ret; + +/* +* Obtain the total size of data to transmit -- sets the 'len' variable +* -- and sanity check that the iovs conform to size and number limits. +* Enforced below: no more than 'len' bytes of guest data +* (plus the message header) will be sent in this operation. +*/ + ret =iov_count (iovs, niov, &len); +if ( ret ) +return ret; + +/* +* Size bounds check against ring size and static maximum message limit. +* The message must not fill the ring; there must be at least one slot
Quoted 8 lines
xen/common/argo.c update_tx_ptr(const struct domain *d, struct argo_ring_info *ring_info, · line 711
Quoted 5 lines
xen/common/argo.c +5
-0+* remaining so we can distinguish a full ring from an empty one. +*/ +if ( ((ROUNDUP_MESSAGE (len) + +sizeof (struct xen_argo_ring_message_header )) >= ring_info->len) || + (len >MAX_ARGO_MESSAGE_SIZE ) )len is already checked to be <=
MAX_ARGO_MESSAGE_SIZEiniov_countwhere it gets set, this is redundant.
ack, removed, thanks.
Quoted 68 lines
Quoted 68 lines
xen/common/argo.c +67
-0+return -EMSGSIZE ; + + ret =get_sanitized_ring (d, &ring, ring_info); +if ( ret ) +return ret; + +argo_dprintk ("ring.tx_ptr=%u ring.rx_ptr=%u ring len=%u" +" ring_info->tx_ptr=%u\n" , + ring.tx_ptr, ring.rx_ptr, ring_info->len, ring_info->tx_ptr); + +if ( ring.rx_ptr == ring.tx_ptr ) + sp = ring_info->len; +else + { + sp = ring.rx_ptr - ring.tx_ptr; +if ( sp <0 ) + sp += ring_info->len; + } + +/* +* Size bounds check against currently available space in the ring. +* Again: the message must not fill the ring leaving no space remaining. +*/ +if ( (ROUNDUP_MESSAGE (len) + +sizeof (struct xen_argo_ring_message_header )) >= sp ) + { +argo_dprintk ("EAGAIN\n" ); +return -EAGAIN ; + } + + mh.len = len +sizeof (struct xen_argo_ring_message_header ); + mh.source.aport = src_id->aport; + mh.source.domain_id = src_id->domain_id; + mh.message_type = message_type; + +/* +* For this copy to the guest ring, tx_ptr is always 16-byte aligned +* and the message header is 16 bytes long. +*/ +BUILD_BUG_ON ( +sizeof (struct xen_argo_ring_message_header ) !=ROUNDUP_MESSAGE (1 )); + +/* +* First data write into the destination ring: fixed size, message header. +* This cannot overrun because the available free space (value in 'sp') +* is checked above and must be at least this size. +*/ + ret =memcpy_to_guest_ring (d, ring_info, + ring.tx_ptr +sizeof (xen_argo_ring_t), + &mh, NULL_hnd,sizeof (mh)); +if ( ret ) + { +gprintk (XENLOG_ERR , +"argo: failed to write message header to ring (vm%u:%x vm%u)\n" , + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +return ret; + } + + ring.tx_ptr +=sizeof (mh); +if ( ring.tx_ptr == ring_info->len ) + ring.tx_ptr =0 ; + +for ( piov = iovs; niov--; piov++ ) + { +XEN_GUEST_HANDLE_64 (uint8_t ) buf_hnd = piov->iov_hnd;
Quoted 8 lines
xen/common/argo.c update_tx_ptr(const struct domain *d, struct argo_ring_info *ring_info, · line 784
Quoted 6 lines
xen/common/argo.c +6
-0+unsigned int iov_len = piov->iov_len; + +/* If no data is provided in this iov, moan and skip on to the next */ +if ( !iov_len ) + { + gprintk(XENLOG_ERR,This should likely be WARN or INFO, since it's not an error?
Yes, changed to WARNING, ack.
Quoted 94 lines
Quoted 94 lines
xen/common/argo.c +93
-0+"argo: no data iov_len=0 iov_hnd=%p ring (vm%u:%x vm%u)\n" , + buf_hnd.p, ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +continue ; + } + +if (unlikely (!guest_handle_okay (buf_hnd, iov_len)) ) + { +gprintk (XENLOG_ERR , +"argo: bad iov handle [%p, %u] (vm%u:%x vm%u)\n" , + buf_hnd.p, iov_len, + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +return -EFAULT ; + } + + sp = ring_info->len - ring.tx_ptr; + +/* Check: iov data size versus free space at the tail of the ring */ +if ( iov_len > sp ) + { +/* +* Second possible data write: ring-tail-wrap-write. +* Populate the ring tail and update the internal tx_ptr to handle +* wrapping at the end of ring. +* Size of data written here: sp +* which is the exact full amount of free space available at the +* tail of the ring, so this cannot overrun. +*/ + ret =memcpy_to_guest_ring (d, ring_info, + ring.tx_ptr +sizeof (xen_argo_ring_t), +NULL , buf_hnd, sp); +if ( ret ) + { +gprintk (XENLOG_ERR , +"argo: failed to copy {%p, %d} (vm%u:%x vm%u)\n" , + buf_hnd.p, sp, + ring_info->id.domain_id, ring_info->id.aport, + ring_info->id.partner_id); + +return ret; + } + + ring.tx_ptr =0 ; + iov_len -= sp; +guest_handle_add_offset (buf_hnd, sp); + +ASSERT (iov_len <= ring_info->len); + } + +/* +* Third possible data write: all data remaining for this iov. +* Size of data written here: iov_len +* +* Case 1: if the ring-tail-wrap-write above was performed, then +* iov_len has been decreased by 'sp' and ring.tx_ptr is zero. +* +* We know from checking the result of iov_count: +* len + sizeof(message_header) <= ring_info->len +* We also know that len is the total of summing all iov_lens, so: +* iov_len <= len +* so by transitivity: +* iov_len <= len <= (ring_info->len - sizeof(msgheader)) +* and therefore: +* (iov_len + sizeof(msgheader) <= ring_info->len) && +* (ring.tx_ptr == 0) +* so this write cannot overrun here. +* +* Case 2: ring-tail-wrap-write above was not performed +* -> so iov_len is the guest-supplied value and: (iov_len <= sp) +* ie. less than available space at the tail of the ring: +* so this write cannot overrun. +*/ + ret =memcpy_to_guest_ring (d, ring_info, + ring.tx_ptr +sizeof (xen_argo_ring_t), +NULL , buf_hnd, iov_len); +if ( ret ) + { +gprintk (XENLOG_ERR , +"argo: failed to copy [%p, %u] (vm%u:%x vm%u)\n" , + buf_hnd.p, iov_len, ring_info->id.domain_id, + ring_info->id.aport, ring_info->id.partner_id); + +return ret; + } + + ring.tx_ptr += iov_len; + +if ( ring.tx_ptr == ring_info->len ) + ring.tx_ptr =0 ; + }
Quoted 8 lines
xen/common/argo.c update_tx_ptr(const struct domain *d, struct argo_ring_info *ring_info, · line 882
Quoted 5 lines
xen/common/argo.c +5
-0+ + ring.tx_ptr =ROUNDUP_MESSAGE (ring.tx_ptr); + +if ( ring.tx_ptr >= ring_info->len ) + ring.tx_ptr -= ring_info->len;You seem to handle the wrapping after each possible write, so I think the above is not needed? Maybe it should be an assert instead?
The wrap handling is necesssary due to that ROUNDUP_MESSAGE immediately above it.
I've added a new comment to make it a bit clearer:
Finished writing data from all iovs into the ring: now need to round up tx_ptr to align to the next message boundary, and then wrap if necessary.
Quoted 85 lines
Quoted 85 lines
xen/common/argo.c +70
-0+ +update_tx_ptr (d, ring_info, ring.tx_ptr); + +/* +* At this point (and also on an error exit paths from this function) it is +* possible to unmap the ring_info, ie: +* ring_unmap(d, ring_info); +* but performance should be improved by not doing so, and retaining +* the mapping. +* An XSM policy control over level of confidentiality required +* versus performance cost could be added to decide that here. +*/ + + *out_len = len; + +return ret; +} +static void wildcard_pending_list_remove (domid_t domain_id,struct pending_ent *ent) {@@ -497,6 +918,25 @@ wildcard_pending_list_remove(domid_t domain_id, struct pending_ent *ent) }static void +wildcard_pending_list_insert (domid_t domain_id,struct pending_ent *ent) +{ +struct domain *d =get_domain_by_id (domain_id); + +if ( !d ) +return ; + +ASSERT (LOCKING_Read_L1); + +if ( d->argo ) + { +spin_lock (&d->argo->wildcard_L2_lock); +list_add (&ent->wildcard_node, &d->argo->wildcard_pend_list); +spin_unlock (&d->argo->wildcard_L2_lock); + } +put_domain (d); +} + +static void pending_remove_all (const struct domain *d,struct argo_ring_info *ring_info) {struct list_head *ring_pending = &ring_info->pending;@@ -518,6 +958,70 @@ pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) ring_info->npending =0 ; } +static int +pending_queue (const struct domain *d,struct argo_ring_info *ring_info, +domid_t src_id,unsigned int len) +{ +struct pending_ent *ent; + +ASSERT (LOCKING_L3 (d, ring_info)); + +if ( ring_info->npending >=MAX_PENDING_PER_RING ) +return -ENOSPC ; + + ent = xmalloc(struct pending_ent); +if ( !ent ) +return -ENOMEM ; + + ent->len = len; + ent->domain_id = src_id; + ent->ring_info = ring_info; + +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) +wildcard_pending_list_insert (src_id, ent); +list_add (&ent->node, &ring_info->pending); + ring_info->npending++; + +return 0 ; +} + +static int +pending_requeue (const struct domain *d,struct argo_ring_info *ring_info, +domid_t src_id,unsigned int len) +{ +struct list_head *cursor, *head; +
Quoted 8 lines
xen/common/argo.c pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) · line 999
Quoted 6 lines
xen/common/argo.c +6
-0+ASSERT (LOCKING_L3 (d, ring_info)); + +/* List structure is not modified here. Update len in a match if found. */ + head = &ring_info->pending; + +for ( cursor = head->next; cursor != head; cursor = cursor->next )
list_for_each_entry
ack
Quoted 32 lines
Quoted 19 lines
xen/common/argo.c +11
-0long do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, XEN_GUEST_HANDLE_PARAM(void) arg2, unsigned long arg3,@@ -1145,6 +1734,53 @@ do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,break ; } +case XEN_ARGO_OP_sendv: + { +xen_argo_send_addr_t send_addr; + +XEN_GUEST_HANDLE_PARAM (xen_argo_send_addr_t) send_addr_hnd = +guest_handle_cast (arg1, xen_argo_send_addr_t); +XEN_GUEST_HANDLE_PARAM (xen_argo_iov_t) iovs_hnd = +guest_handle_cast (arg2, xen_argo_iov_t); +/* arg3 is niov */ +/* arg4 is message_type. Must be a 32-bit value. */ +xen/common/argo.c do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, · line 1757
Quoted 10 lines
xen/common/argo.c +10
-0+ rc =copy_from_guest (&send_addr, send_addr_hnd,1 ) ? -EFAULT :0 ; +if ( rc ) +break ; + +/* +* Check padding is zeroed. Reject niov above limit or message_types +* that are outside 32 bit range. +*/ +if (unlikely (send_addr.src.pad || send_addr.dst.pad || + (arg3 >XEN_ARGO_MAXIOV ) || (arg4 & ~0xffffffffUL )) )arg4 & (GB(4) - 1)
Quoted 3 lines
Is clearer IMO, or:
arg4 >
UINT32_MAX
I've left the code unchanged, as the mask constant is used multiple places elsewhere in Xen. UINT32_MAX is only used as a threshold value.
Quoted 16 lines
Quoted 16 lines
xen/common/argo.c +15
-0+ { + rc = -EINVAL ; +break ; + } + +if ( send_addr.src.domain_id ==XEN_ARGO_DOMID_ANY ) + send_addr.src.domain_id = currd->domain_id; + +/* No domain is currently authorized to send on behalf of another */ +if (unlikely (send_addr.src.domain_id != currd->domain_id) ) + { + rc = -EPERM ; +break ; + } +
Quoted 8 lines
xen/common/argo.c do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, · line 1777
Quoted 5 lines
xen/common/argo.c +5
-0+/* +* Check access to the whole array here so we can use the faster __copy +* operations to read each element later. +*/ +if (unlikely (!guest_handle_okay (iovs_hnd, arg3)) )You need to set rc to EFAULT here, because the call to
copy_from_guesthas set it to 0.
ack.
Quoted 11 lines
Alternatively you can change the call above to be:
if (
copy_from_guest(&send_addr, send_addr_hnd, 1))return -EFAULT;So rc doesn't get set to 0 on success.
With those taken care of:
Reviewed-by Roger Pau Monné <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
Quoted 33 lines · Christopher Clark
Quoted 32 lines · Roger Pau Monné
Quoted 17 lines · Christopher Clark
do_argo_op(unsigned int cmd,
XEN_GUEST_HANDLE_PARAM(void)arg1,
XEN_GUEST_HANDLE_PARAM(void)arg2, unsigned long arg3,xen/common/argo.c +11
-0@@ -1145,6 +1734,53 @@ do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,break ; } +case XEN_ARGO_OP_sendv: + { +xen_argo_send_addr_t send_addr; + +XEN_GUEST_HANDLE_PARAM (xen_argo_send_addr_t) send_addr_hnd = +guest_handle_cast (arg1, xen_argo_send_addr_t); +XEN_GUEST_HANDLE_PARAM (xen_argo_iov_t) iovs_hnd = +guest_handle_cast (arg2, xen_argo_iov_t); +/* arg3 is niov */ +/* arg4 is message_type. Must be a 32-bit value. */ +xen/common/argo.c do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, · line 1757
Quoted 10 lines
xen/common/argo.c +10
-0+ rc =copy_from_guest (&send_addr, send_addr_hnd,1 ) ? -EFAULT :0 ; +if ( rc ) +break ; + +/* +* Check padding is zeroed. Reject niov above limit or message_types +* that are outside 32 bit range. +*/ +if (unlikely (send_addr.src.pad || send_addr.dst.pad || + (arg3 >XEN_ARGO_MAXIOV ) || (arg4 & ~0xffffffffUL )) )arg4 & (GB(4) - 1)
Quoted 6 lines
Quoted 3 lines
Is clearer IMO, or:
arg4 >
UINT32_MAXI've left the code unchanged, as the mask constant is used multiple places elsewhere in Xen.
UINT32_MAXis only used as a threshold value.
The fact that others parts of the code could be improved is not an excuse to follow suit. I'm having a hard time believing that you find "arg4 & ~0xffffffffUL" easier to read than "arg4 & ~(GB(4) - 1)" or even "arg4 >= GB(4)".
IMO it's much more likely to miss an 'f' in the first construct, and thus get the value wrong and introduce a bug.
Anyway, this is your code, so I'm not going to insist.
Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
Quoted 20 lines
Quoted 1 line
Quoted 1 line · On 31.01.19 at 11:18, <[email protected]> wrote:
Quoted 9 lines · Christopher Clark
Quoted 8 lines · Roger Pau Monné
Quoted 6 lines · Christopher Clark
xen/common/argo.c +6
-0+/* +* Check padding is zeroed. Reject niov above limit or message_types +* that are outside 32 bit range. +*/ +if (unlikely (send_addr.src.pad || send_addr.dst.pad || + (arg3 >XEN_ARGO_MAXIOV ) || (arg4 & ~0xffffffffUL )) )Quoted 8 lines
Quoted 5 lines
arg4 & (GB(4) - 1)
Is clearer IMO, or:
arg4 >
UINT32_MAXI've left the code unchanged, as the mask constant is used multiple places elsewhere in Xen.
UINT32_MAXis only used as a threshold value.
Quoted 7 lines
The fact that others parts of the code could be improved is not an excuse to follow suit. I'm having a hard time believing that you find "arg4 & ~0xffffffffUL" easier to read than "arg4 & ~(GB(4) - 1)" or even "arg4 >= GB(4)".
IMO it's much more likely to miss an 'f' in the first construct, and thus get the value wrong and introduce a bug.
I agree with this last statement, but I'm having trouble to see how message _type_ is related to a size construct like GB(4) is. I see only UINT32_MAX as a viable alternative for something that's not expressing the size of anything.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
Quoted 25 lines · Jan Beulich
Quoted 25 lines
Quoted 1 line
Quoted 1 line · On 31.01.19 at 11:18, <[email protected]> wrote:
Quoted 9 lines · Christopher Clark
Quoted 8 lines · Roger Pau Monné
Quoted 6 lines · Christopher Clark
xen/common/argo.c +6
-0+/* +* Check padding is zeroed. Reject niov above limit or message_types +* that are outside 32 bit range. +*/ +if (unlikely (send_addr.src.pad || send_addr.dst.pad || + (arg3 >XEN_ARGO_MAXIOV ) || (arg4 & ~0xffffffffUL )) )Quoted 8 lines
Quoted 5 lines
arg4 & (GB(4) - 1)
Is clearer IMO, or:
arg4 >
UINT32_MAXI've left the code unchanged, as the mask constant is used multiple places elsewhere in Xen.
UINT32_MAXis only used as a threshold value.The fact that others parts of the code could be improved is not an excuse to follow suit. I'm having a hard time believing that you find "arg4 & ~0xffffffffUL" easier to read than "arg4 & ~(GB(4) - 1)" or even "arg4 >= GB(4)".
Quoted 7 lines
Quoted 2 lines
IMO it's much more likely to miss an 'f' in the first construct, and thus get the value wrong and introduce a bug.
I agree with this last statement, but I'm having trouble to see how message _type_ is related to a size construct like GB(4) is. I see only
UINT32_MAXas a viable alternative for something that's not expressing the size of anything.
I've suggested the GB construct as an alternative because the comment above mentions the 32bit range. IMO anything that avoids using 0xffffffffUL is fine.
Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
Quoted 19 lines · Roger Pau Monné
Quoted 17 lines · Jan Beulich
Quoted 17 lines
Quoted 1 line
Quoted 1 line · On 31.01.19 at 11:18, <[email protected]> wrote:
Quoted 15 lines · Christopher Clark
Quoted 13 lines · Roger Pau Monné
xen/common/argo.c do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1, · line 1757
Quoted 6 lines · Christopher Clark
xen/common/argo.c +6
-0+/* +* Check padding is zeroed. Reject niov above limit or message_types +* that are outside 32 bit range. +*/ +if (unlikely (send_addr.src.pad || send_addr.dst.pad || + (arg3 >XEN_ARGO_MAXIOV ) || (arg4 & ~0xffffffffUL )) )arg4 & (GB(4) - 1)
Is clearer IMO, or:
arg4 >
UINT32_MAX
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
I've left the code unchanged, as the mask constant is used multiple places elsewhere in Xen.
UINT32_MAXis only used as a threshold value.The fact that others parts of the code could be improved is not an excuse to follow suit. I'm having a hard time believing that you find "arg4 & ~0xffffffffUL" easier to read than "arg4 & ~(GB(4) - 1)" or even "arg4 >= GB(4)".
Below, I propose an alternative way of achieving our correctness and readability goals.
On the topic of readability, this self-contained definition does stand out: ~0xffffffffUL,
encouraging caution and careful counting of 'f's. However, no other source files are involved, making the code independent of changes in (macro) definitions in other files.
In comparison, to understand GB, I have find the external definition, and then parse this:
#define GB(_gb) (_AC(_gb, ULL) << 30)
(which seems to have a different type? ULL vs UL?) and then find and understand this, in another file:
#ifdef __ASSEMBLY__
#define _AC(X,Y) X
#define _AT(T,X) X
#else
#define __AC(X,Y) (X##Y) #define _AC(X,Y) __AC(X,Y) #define _AT(T,X) ((T)(X))
#endif
so I'm saying: it's at least somewhat arguable which is easier to understand. Regardless, I think there's a better option than either.
Quoted 11 lines
Quoted 7 lines
Quoted 2 lines
IMO it's much more likely to miss an 'f' in the first construct, and thus get the value wrong and introduce a bug.
I agree with this last statement, but I'm having trouble to see how message _type_ is related to a size construct like GB(4) is. I see only
UINT32_MAXas a viable alternative for something that's not expressing the size of anything.I've suggested the GB construct as an alternative because the comment above mentions the 32bit range. IMO anything that avoids using
0xffffffffULis fine.
Jan and Andrew have employed a useful technique in recent changes where such a test was required. This could work:
(arg4 != (uint32_t)arg4))
It is self-contained, readable and clearly expresses the intent of the check being performed. I have tested a series with this applied, and have it ready to post if you approve.
Christopher
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 09/15] argo: implement the sendv op; evtchn: expose send_guest_global_virq
Quoted 70 lines · Christopher Clark
Quoted 19 lines · Roger Pau Monné
Quoted 17 lines · Jan Beulich
Quoted 17 lines
Quoted 1 line
Quoted 1 line · On 31.01.19 at 11:18, <[email protected]> wrote:
Quoted 15 lines · Christopher Clark
Quoted 13 lines · Roger Pau Monné
Review
Quoted 6 lines · Christopher Clark
(quoted diff) +6
-0+ /* + * Check padding is zeroed. Reject niov above limit or message_types + * that are outside 32 bit range. + */ + if ( unlikely(send_addr.src.pad || send_addr.dst.pad || + (arg3 > XEN_ARGO_MAXIOV) || (arg4 & ~0xffffffffUL)) )arg4 & (GB(4) - 1)
Is clearer IMO, or:
arg4 >
UINT32_MAXQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
I've left the code unchanged, as the mask constant is used multiple places elsewhere in Xen.
UINT32_MAXis only used as a threshold value.The fact that others parts of the code could be improved is not an excuse to follow suit. I'm having a hard time believing that you find "arg4 & ~0xffffffffUL" easier to read than "arg4 & ~(GB(4) - 1)" or even "arg4 >= GB(4)".
Below, I propose an alternative way of achieving our correctness and readability goals.
On the topic of readability, this self-contained definition does stand out: ~0xffffffffUL,
encouraging caution and careful counting of 'f's. However, no other source files are involved, making the code independent of changes in (macro) definitions in other files.In comparison, to understand GB, I have find the external definition, and then parse this:
#define
GB(_gb)(_AC(_gb, ULL)<< 30)(which seems to have a different type? ULL vs UL?) and then find and understand this, in another file:
#ifdef
__ASSEMBLY__
#define_AC(X,Y)X
#define_AT(T,X)X
#else#define __AC(X,Y) (X##Y) #define _AC(X,Y) __AC(X,Y) #define _AT(T,X) ((T)(X))#endif
so I'm saying: it's at least somewhat arguable which is easier to understand. Regardless, I think there's a better option than either.
Quoted 11 lines
Quoted 7 lines
Quoted 2 lines
IMO it's much more likely to miss an 'f' in the first construct, and thus get the value wrong and introduce a bug.
I agree with this last statement, but I'm having trouble to see how message _type_ is related to a size construct like GB(4) is. I see only
UINT32_MAXas a viable alternative for something that's not expressing the size of anything.I've suggested the GB construct as an alternative because the comment above mentions the 32bit range. IMO anything that avoids using
0xffffffffULis fine.
Quoted 8 lines
Jan and Andrew have employed a useful technique in recent changes where such a test was required. This could work:
(arg4 != (uint32_t)arg4))
It is self-contained, readable and clearly expresses the intent of the check being performed. I have tested a series with this applied, and have it ready to post if you approve.
Yes, that's fine. As said in v7 anything that doesn't involve an open-coded mask is fine with me.
Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 10/15 argo: implement the notify op
In reply to Christopher ClarkQueries for data about space availability in registered rings and causes notification to be sent when space has become available.
The hypercall op populates a supplied data structure with information about ring state and if insufficient space is currently available in a given ring, the hypervisor will record the domain's expressed interest and notify it when it observes that space has become available.
Checks for free space occur when this notify op is invoked, so it may be intentionally invoked with no data structure to populate
(ie. a NULL argument) to trigger such a check and consequent notifications.
Limit the maximum number of notify requests in a single operation to a simple fixed limit of 256.
---
v4 #10 Roger: consolidate notify flags; infer pending notify if needed v4 bugfix: take L3 before accessing ring_info in fill_ring_data v4 #10 Roger: shorten notify flag names: drop _DATA_F
v4 #10 self/Roger: fill_ring_data: check pending_requeue error code v4 : use standard data structures as per common code
v4 #10 Roger: lower indentation in fill_ring_data by using goto v4 #10 Roger: reword the XEN_ARGO_RING_DATA_F_SUFFICIENT comment v4 fix location of a FIXME that was incorrectly moved by this later commit
v3 #07 Jan: fix format string indention in printks
v3 (general) Jan: drop fixed width types for ringbuf_payload_space v3 #07 Jan: rename ring_find_info_by_match to find_ring_info_by_match v3 #07 Jan: fix numeric entries in printk format strings
v3: ringbuf_payload_space: simpler return 0 if get_sanitized_ring fails v3 #10 Roger: simplify ringbuf_payload_space for empty rings
v3 #10 Roger: ringbuf_payload_space: add comment to explain how ret < INT32_MAX v3 #10 Roger: drop out label, use return -EFAULT in fill_ring_data v3 #10 Roger: add newline in signal_domid
v3 #10 Roger: move find functions to top of file and drop prototypes v3 #04 Jan: meld the compat hypercall arg checking
v3 #04 Roger/Jan: make lock names clearer and assert their state v3 #04 Jan: port -> aport with type; distinguish argo port from evtchn v3 self: drop braces in foreach of notify_check_pending
v3 feedback Roger/Jan: ASSERT currd is current->domain or use 'd' variable name
v2 feedback Jan: drop cookie, implement teardown
v2 notify: add flag to indicate ring is shared
v2 argument name for fill_ring_data arg is now currd
v2 self: check ring size vs request and flag error rather than queue signal v2 feedback Jan: drop 'message' from 'argo_message_op'
v2 self: simplify signal_domid, drop unnecessary label + goto
v2 self: skip the cookie check in pending_cancel
v2 self: implement npending limit on number of pending entries
v1 feedback #16 Jan: sanitize_ring in ringbuf_payload_space
v2 self: inline fill_ring_data_array
v2 self: avoid retesting dst_d for put_domain
v2 self/Jan: remove use of magic verification field and tidy up v1 feedback #16 Jan: remove testing of magic in guest-supplied structure v2 self: s/argo_pending_ent/pending_ent/g
v2 feedback v1#13 Roger: use OS-supplied roundup; drop from public header v1,2 feedback Jan/Roger/Paul: drop errno returning guest access functions v1 feedback Roger, Jan: drop argo prefix on static functions
v2 self: reduce indentation via goto out if arg NULL
v1 feedback #13 Jan: resolve checking of array handle and use of __copy v1 #5 (#16) feedback Paul: notify op: use currd in do_argo_message_op v1 #5 (#16) feedback Paul: notify op: use currd in argo_notify
v1 #5 (#16) feedback Paul: notify op: use currd in argo_notify_check_pending v1 #5 (#16) feedback Paul: notify op: use currd in argo_fill_ring_data_array v1 #13 (#16) feedback Paul: notify op: do/while: reindent only
v1 #13 (#16) feedback Paul: notify op: do/while: goto
v1 : add compat xlat.lst entries
v1: add definition for copy_field_from_guest_errno
v1 #13 feedback Jan: make 'ring data' comment comply with single-line style v1 feedback #13 Jan: use __copy; so define and use __copy_field_to_guest_errno
v1: #13 feedback Jan: public namespace: prefix with xen
v1: #13 feedback Jan: add blank line after case in do_argo_message_op
v1: self: rename ent id to domain_id
v1: self: ent id-> domain_id
v1: self: drop signal if domain_cookie mismatches
v1. feedback #15 Jan: make loop i unsigned
v1. self: drop unnecessary mb() in argo_notify_check_pending
v1. self: add blank line
v1 #16 feedback Jan: const domain arg to +argo_fill_ring_data
v1. feedback #15 Jan: check unusued hypercall args are zero
v1 feedback #16 Jan: add comment on space available signal policy v1. feedback #16 Jan: move declr, drop braces, lower indent
v1. feedback #18 Jan: meld the resource limits into the main commit v1. feedback #16 Jan: clarify use of magic field
v1. self: use single copy to read notify ring data struct
v1: argo_fill_ring_data: fix dprintk types for port field
v1: self: use %x for printing port as per other print sites
v1. feedback Jan: add comments explaining ring full vs empty
v1. following Jan: fix argo_ringbuf_payload_space calculation for empty ring
xen/common/argo.c | 371 ++++++++++++++++++++++++++++++++++++++++++++++ xen/common/compat/argo.c | 18 +++ xen/include/public/argo.h | 64 ++++++++ xen/include/xlat.lst | 2 + 4 files changed, 455 insertions(+)
xen/common/argo.c +371 -0
@@ -30,6 +30,7 @@#include <public/argo.h> #define MAX_RINGS_PER_DOMAIN 128U +#define MAX_NOTIFY_COUNT 256U#define MAX_PENDING_PER_RING 32U/* All messages on the ring are padded to a multiple of the slot size. */
@@ -49,6 +50,8 @@ DEFINE_XEN_GUEST_HANDLE(xen_argo_gfn_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_iov_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_register_ring_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_ring_t); +DEFINE_XEN_GUEST_HANDLE (xen_argo_ring_data_t); +DEFINE_XEN_GUEST_HANDLE (xen_argo_ring_data_ent_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_send_addr_t);DEFINE_XEN_GUEST_HANDLE (xen_argo_unregister_ring_t);
@@ -421,6 +424,18 @@ signal_domain(struct domain *d) }static void +signal_domid (domid_t domain_id) +{ +struct domain *d =get_domain_by_id (domain_id); + +if ( !d ) +return ; + +signal_domain (d); +put_domain (d); +} + +static void ring_unmap (const struct domain *d,struct argo_ring_info *ring_info) {unsigned int i;
@@ -620,6 +635,66 @@ get_sanitized_ring(const struct domain *d, xen_argo_ring_t *ring,return 0 ; } +static unsigned int +ringbuf_payload_space (const struct domain *d,struct argo_ring_info *ring_info) +{ +xen_argo_ring_t ring; +unsigned int len; +int ret; + +ASSERT (LOCKING_L3 (d, ring_info)); + + len = ring_info->len; +if ( !len ) +return 0 ; + +if (get_sanitized_ring (d, &ring, ring_info) ) +return 0 ; + +argo_dprintk ("sanitized ringbuf_payload_space: tx_ptr=%u rx_ptr=%u\n" , + ring.tx_ptr, ring.rx_ptr); + +/* +* rx_ptr == tx_ptr means that the ring has been emptied. +* See message size checking logic in the entry to ringbuf_insert which +* ensures that there is always one message slot of size ROUNDUP_MESSAGE(1) +* left available, preventing a ring from being entirely filled. +* This ensures that matching ring indexes always indicate an empty ring +* and never a full one. +*/ + ret = ring.rx_ptr - ring.tx_ptr; +if ( ret <=0 ) + ret += len; + +/* +* In a sanitized ring, we can rely on: +* (rx_ptr < ring_info->len) && +* (tx_ptr < ring_info->len) && +* (ring_info->len <= XEN_ARGO_MAX_RING_SIZE) +* +* and since: XEN_ARGO_MAX_RING_SIZE < INT32_MAX +* therefore right here: ret < INT32_MAX +* and we are safe to return it as a unsigned value from this function. +* The subtractions below cannot increase its value. +*/ + +/* +* The maximum size payload for a message that will be accepted is: +* (the available space between the ring indexes) +* minus (space for a message header) +* minus (space for one message slot) +* since ringbuf_insert requires that one message slot be left +* unfilled, to avoid filling the ring to capacity and confusing a full +* ring with an empty one. +* Since the ring indexes are sanitized, the value in ret is aligned, so +* the simple subtraction here works to return the aligned value needed: +*/ + ret -=sizeof (struct xen_argo_ring_message_header ); + ret -=ROUNDUP_MESSAGE (1 ); + +return (ret <0 ) ?0 : ret; +} +/* * iov_count returns its count on success via an out variable to avoid * potential for a negative return value to be used incorrectly
@@ -958,6 +1033,71 @@ pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) ring_info->npending =0 ; } +static void +pending_notify (struct list_head *to_notify) +{ +ASSERT (LOCKING_Read_L1); + +/* Sending signals for all ents in this list, draining until it is empty. */ +while ( !list_empty (to_notify) ) + { +struct pending_ent *ent = + list_entry(to_notify->next, struct pending_ent, node); + +list_del (&ent->node); +signal_domid (ent->domain_id); +xfree (ent); + } +} + +static void +pending_find (const struct domain *d,struct argo_ring_info *ring_info, +unsigned int payload_space,struct list_head *to_notify) +{ +struct list_head *cursor, *pending_head; + +ASSERT (LOCKING_Read_rings_L2 (d)); + +/* +* TODO: Current policy here is to signal _all_ of the waiting domains +* interested in sending a message of size less than payload_space. +* +* This is likely to be suboptimal, since once one of them has added +* their message to the ring, there may well be insufficient room +* available for any of the others to transmit, meaning that they were +* woken in vain, which created extra work just to requeue their wait. +* +* Retain this simple policy for now since it at least avoids starving a +* domain of available space notifications because of a policy that only +* notified other domains instead. Improvement may be possible; +* investigation required. +*/ +spin_lock (&ring_info->L3_lock); + +/* Remove matching ents from the ring list, and add them to "to_notify" */ + pending_head = &ring_info->pending; + cursor = pending_head->next; + +while ( cursor != pending_head ) + { + struct pending_ent *ent = list_entry(cursor, struct pending_ent, node); + + cursor = cursor->next; + +if ( payload_space >= ent->len ) + { +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) +wildcard_pending_list_remove (ent->domain_id, ent); + +list_del (&ent->node); + ring_info->npending--; +list_add (&ent->node, to_notify); + } + } + +spin_unlock (&ring_info->L3_lock); +} +static int pending_queue (const struct domain *d,struct argo_ring_info *ring_info,domid_t src_id,unsigned int len)
@@ -1023,6 +1163,36 @@ pending_requeue(const struct domain *d, struct argo_ring_info *ring_info, }static void +pending_cancel (const struct domain *d,struct argo_ring_info *ring_info, +domid_t src_id) +{ +struct list_head *cursor, *pending_head; + +ASSERT (LOCKING_L3 (d, ring_info)); + +/* Remove all ents where domain_id matches src_id from the ring's list. */ + pending_head = &ring_info->pending; + cursor = pending_head->next; + +while ( cursor != pending_head ) + { + struct pending_ent *ent = list_entry(cursor, struct pending_ent, node); + + cursor = cursor->next; + +if ( ent->domain_id == src_id ) + { +/* For wildcard rings, remove each from their wildcard list too. */ +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) +wildcard_pending_list_remove (ent->domain_id, ent); +list_del (&ent->node); +xfree (ent); + ring_info->npending--; + } + } +} + +static void wildcard_rings_pending_remove (struct domain *d) {struct list_head *wildcard_head;
@@ -1158,6 +1328,86 @@ partner_rings_remove(struct domain *src_d) }static int +fill_ring_data (const struct domain *currd, +XEN_GUEST_HANDLE (xen_argo_ring_data_ent_t) data_ent_hnd) +{ +xen_argo_ring_data_ent_t ent; +struct domain *dst_d; +struct argo_ring_info *ring_info; +int ret =0 ; + +ASSERT (currd == current->domain); +ASSERT (LOCKING_Read_L1); + +if (__copy_from_guest (&ent, data_ent_hnd,1 ) ) +return -EFAULT ; + +argo_dprintk ("fill_ring_data: ent.ring.domain=%u,ent.ring.aport=%x\n" , + ent.ring.domain_id, ent.ring.aport); + + ent.flags =0 ; + + dst_d =get_domain_by_id (ent.ring.domain_id); +if ( !dst_d || !dst_d->argo ) + goto out; + +read_lock (&dst_d->argo->rings_L2_rwlock); + + ring_info =find_ring_info_by_match (dst_d, ent.ring.aport, + currd->domain_id); +if ( ring_info ) + { +unsigned int space_avail; + + ent.flags |=XEN_ARGO_RING_EXISTS ; + +spin_lock (&ring_info->L3_lock); + + ent.max_message_size = ring_info->len - +sizeof (struct xen_argo_ring_message_header ) - +ROUNDUP_MESSAGE (1 ); + +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) + ent.flags |=XEN_ARGO_RING_SHARED ; + + space_avail =ringbuf_payload_space (dst_d, ring_info); + +argo_dprintk ("fill_ring_data: aport=%x space_avail=%u" +" space_wanted=%u\n" , + ring_info->id.aport, space_avail, ent.space_required); + +/* Do not queue a notification for an unachievable size */ +if ( ent.space_required > ent.max_message_size ) + ent.flags |=XEN_ARGO_RING_EMSGSIZE ; +else if ( space_avail >= ent.space_required ) + { +pending_cancel (dst_d, ring_info, currd->domain_id); + ent.flags |=XEN_ARGO_RING_SUFFICIENT ; + } +else + ret =pending_requeue (dst_d, ring_info, currd->domain_id, + ent.space_required); + +spin_unlock (&ring_info->L3_lock); + +if ( space_avail == ent.max_message_size ) + ent.flags |=XEN_ARGO_RING_EMPTY ; + + } +read_unlock (&dst_d->argo->rings_L2_rwlock); + + out: +if ( dst_d ) +put_domain (dst_d); + +if ( !ret && (__copy_field_to_guest (data_ent_hnd, &ent, flags) || +__copy_field_to_guest (data_ent_hnd, &ent, max_message_size)) ) +return -EFAULT ; + +return ret; +} + +static int find_ring_mfn (struct domain *d,gfn_t gfn,mfn_t *mfn) {struct page_info *page;
@@ -1586,6 +1836,112 @@ register_ring(struct domain *currd,return ret; } +static void +notify_ring (const struct domain *d,struct argo_ring_info *ring_info, +struct list_head *to_notify) +{ +unsigned int space; + +ASSERT (LOCKING_Read_rings_L2 (d)); + +spin_lock (&ring_info->L3_lock); + +if ( ring_info->len ) + space =ringbuf_payload_space (d, ring_info); +else + space =0 ; + +spin_unlock (&ring_info->L3_lock); + +if ( space ) +pending_find (d, ring_info, space, to_notify); +} + +static void +notify_check_pending (struct domain *d) +{ +unsigned int i; +LIST_HEAD (to_notify); + +ASSERT (LOCKING_Read_L1); + +read_lock (&d->argo->rings_L2_rwlock); + +/* Walk all rings, call notify_ring on each to populate to_notify list */ +for ( i =0 ; i <ARGO_HASHTABLE_SIZE ; i++ ) + { +struct list_head *cursor, *bucket = &d->argo->ring_hash[i]; +struct argo_ring_info *ring_info; + +for ( cursor = bucket->next; cursor != bucket; cursor = cursor->next ) + { + ring_info = list_entry(cursor, struct argo_ring_info, node); +notify_ring (d, ring_info, &to_notify); + } + } + +read_unlock (&d->argo->rings_L2_rwlock); + +if ( !list_empty (&to_notify) ) +pending_notify (&to_notify); +} + +static long +notify (struct domain *currd, +XEN_GUEST_HANDLE_PARAM (xen_argo_ring_data_t) ring_data_hnd) +{ +XEN_GUEST_HANDLE (xen_argo_ring_data_ent_t) ent_hnd; +xen_argo_ring_data_t ring_data; +int ret =0 ; + +ASSERT (currd == current->domain); + +read_lock (&L1_global_argo_rwlock); + +if ( !currd->argo ) + { +argo_dprintk ("!d->argo, ENODEV\n" ); + ret = -ENODEV ; + goto out; + } + +notify_check_pending (currd); + +if (guest_handle_is_null (ring_data_hnd) ) + goto out; + + ret =copy_from_guest (&ring_data, ring_data_hnd,1 ) ? -EFAULT :0 ; +if ( ret ) + goto out; + +if ( ring_data.nent >MAX_NOTIFY_COUNT ) + { +gprintk (XENLOG_ERR ,"argo: notify entry count(%u) exceeds max(%u)\n" , + ring_data.nent,MAX_NOTIFY_COUNT ); + ret = -EACCES ; + goto out; + } + + ent_hnd =guest_handle_for_field (ring_data_hnd, + xen_argo_ring_data_ent_t, data[0 ]); +if (unlikely (!guest_handle_okay (ent_hnd, ring_data.nent)) ) + { + ret = -EFAULT ; + goto out; + } + +while ( !ret && ring_data.nent-- ) + { + ret =fill_ring_data (currd, ent_hnd); +guest_handle_add_offset (ent_hnd,1 ); + } + + out: +read_unlock (&L1_global_argo_rwlock); + +return ret; +} + static long sendv(struct domain *src_d, const xen_argo_addr_t *src_addr, const xen_argo_addr_t *dst_addr,
@@ -1781,6 +2137,21 @@ do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,break ; } +case XEN_ARGO_OP_notify: + { +XEN_GUEST_HANDLE_PARAM (xen_argo_ring_data_t) ring_data_hnd = +guest_handle_cast (arg1, xen_argo_ring_data_t); + +if (unlikely ((!guest_handle_is_null (arg2)) || arg3 || arg4) ) + { + rc = -EINVAL ; +break ; + } + + rc =notify (currd, ring_data_hnd); +break ; + } +default : rc = -EOPNOTSUPP ;break ;
xen/common/compat/argo.c +18 -0
@@ -41,4 +41,22 @@ static inline int __maybe_unused name(k xen_ ## n *x, k compat_ ## n *c) \ } CHECK_argo_send_addr; +CHECK_argo_ring_data_ent; CHECK_argo_iov; + +/* +* Disable sizeof type checking for the following struct checks because +* these structs have fields of types that differ in the compat vs non-compat +* structs with variable size which prevents the size check validation. +*/ + +#undef CHECK_FIELD_COMMON_ +#define CHECK_FIELD_COMMON_ (k, name, n, f) \ +static inline int __maybe_unused name(k xen_ ## n *x, k compat_ ## n *c) \ +{ \ + BUILD_BUG_ON(offsetof(k xen_ ## n, f) != \ + offsetof(k compat_ ## n, f)); \ + return 1; \ +} + +CHECK_argo_ring_data;
xen/include/public/argo.h +64 -0
74 lines not on this page · raw message
xen/include/xlat.lst +2 -0
@@ -154,3 +154,5 @@ ? argo_unregister_ring argo.h ? argo_iov argo.h ? argo_send_addr argo.h +? argo_ring_data_ent argo.h +? argo_ring_data argo.h
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 10/15] argo: implement the notify op
Quoted 8 lines · Christopher Clark
Queries for data about space availability in registered rings and causes notification to be sent when space has become available.
The hypercall op populates a supplied data structure with information about ring state and if insufficient space is currently available in a given ring, the hypervisor will record the domain's expressed interest and notify it when it observes that space has become available.
Quoted 8 lines
Checks for free space occur when this notify op is invoked, so it may be intentionally invoked with no data structure to populate
(ie. a NULL argument) to trigger such a check and consequent notifications.Limit the maximum number of notify requests in a single operation to a simple fixed limit of 256.
Signed-off-by Christopher Clark <[email protected]>
LGTM, but I would like to see the open-coded versions of the list_ macros fixed:
Quoted 4 lines
xen/common/argo.c +0
-0
[...]
xen/common/argo.c pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) · line 1045
Quoted 10 lines
xen/common/argo.c +10
-0+static void +pending_notify (struct list_head *to_notify) +{ +ASSERT (LOCKING_Read_L1); + +/* Sending signals for all ents in this list, draining until it is empty. */ +while ( !list_empty (to_notify) ) + { +struct pending_ent *ent = + list_entry(to_notify->next, struct pending_ent, node);
list_first_entry_or_null
Quoted 30 lines
xen/common/argo.c +30
-0+ +list_del (&ent->node); +signal_domid (ent->domain_id); +xfree (ent); + } +} + +static void +pending_find (const struct domain *d,struct argo_ring_info *ring_info, +unsigned int payload_space,struct list_head *to_notify) +{ +struct list_head *cursor, *pending_head; + +ASSERT (LOCKING_Read_rings_L2 (d)); + +/* +* TODO: Current policy here is to signal _all_ of the waiting domains +* interested in sending a message of size less than payload_space. +* +* This is likely to be suboptimal, since once one of them has added +* their message to the ring, there may well be insufficient room +* available for any of the others to transmit, meaning that they were +* woken in vain, which created extra work just to requeue their wait. +* +* Retain this simple policy for now since it at least avoids starving a +* domain of available space notifications because of a policy that only +* notified other domains instead. Improvement may be possible; +* investigation required. +*/ +spin_lock (&ring_info->L3_lock);
xen/common/argo.c pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) · line 1085
Quoted 10 lines
xen/common/argo.c +10
-0+ +/* Remove matching ents from the ring list, and add them to "to_notify" */ + pending_head = &ring_info->pending; + cursor = pending_head->next; + +while ( cursor != pending_head ) + { + struct pending_ent *ent = list_entry(cursor, struct pending_ent, node); + + cursor = cursor->next;
list_for_each_entry_safe?
Quoted 28 lines
xen/common/argo.c +21
-0+ +if ( payload_space >= ent->len ) + { +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) +wildcard_pending_list_remove (ent->domain_id, ent); + +list_del (&ent->node); + ring_info->npending--; +list_add (&ent->node, to_notify); + } + } + +spin_unlock (&ring_info->L3_lock); +} +static int pending_queue (const struct domain *d,struct argo_ring_info *ring_info,domid_t src_id,unsigned int len)@@ -1023,6 +1163,36 @@ pending_requeue(const struct domain *d, struct argo_ring_info *ring_info, }static void +pending_cancel (const struct domain *d,struct argo_ring_info *ring_info, +domid_t src_id) +{ +struct list_head *cursor, *pending_head; + +ASSERT (LOCKING_L3 (d, ring_info));
xen/common/argo.c pending_requeue(const struct domain *d, struct argo_ring_info *ring_info, · line 1181
Quoted 10 lines
xen/common/argo.c +10
-0+ +/* Remove all ents where domain_id matches src_id from the ring's list. */ + pending_head = &ring_info->pending; + cursor = pending_head->next; + +while ( cursor != pending_head ) + { + struct pending_ent *ent = list_entry(cursor, struct pending_ent, node); + + cursor = cursor->next;
list_for_each_entry_safe
Quoted 136 lines
xen/common/argo.c +122
-0+ +if ( ent->domain_id == src_id ) + { +/* For wildcard rings, remove each from their wildcard list too. */ +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) +wildcard_pending_list_remove (ent->domain_id, ent); +list_del (&ent->node); +xfree (ent); + ring_info->npending--; + } + } +} + +static void wildcard_rings_pending_remove (struct domain *d) {struct list_head *wildcard_head;@@ -1158,6 +1328,86 @@ partner_rings_remove(struct domain *src_d) }static int +fill_ring_data (const struct domain *currd, +XEN_GUEST_HANDLE (xen_argo_ring_data_ent_t) data_ent_hnd) +{ +xen_argo_ring_data_ent_t ent; +struct domain *dst_d; +struct argo_ring_info *ring_info; +int ret =0 ; + +ASSERT (currd == current->domain); +ASSERT (LOCKING_Read_L1); + +if (__copy_from_guest (&ent, data_ent_hnd,1 ) ) +return -EFAULT ; + +argo_dprintk ("fill_ring_data: ent.ring.domain=%u,ent.ring.aport=%x\n" , + ent.ring.domain_id, ent.ring.aport); + + ent.flags =0 ; + + dst_d =get_domain_by_id (ent.ring.domain_id); +if ( !dst_d || !dst_d->argo ) + goto out; + +read_lock (&dst_d->argo->rings_L2_rwlock); + + ring_info =find_ring_info_by_match (dst_d, ent.ring.aport, + currd->domain_id); +if ( ring_info ) + { +unsigned int space_avail; + + ent.flags |=XEN_ARGO_RING_EXISTS ; + +spin_lock (&ring_info->L3_lock); + + ent.max_message_size = ring_info->len - +sizeof (struct xen_argo_ring_message_header ) - +ROUNDUP_MESSAGE (1 ); + +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) + ent.flags |=XEN_ARGO_RING_SHARED ; + + space_avail =ringbuf_payload_space (dst_d, ring_info); + +argo_dprintk ("fill_ring_data: aport=%x space_avail=%u" +" space_wanted=%u\n" , + ring_info->id.aport, space_avail, ent.space_required); + +/* Do not queue a notification for an unachievable size */ +if ( ent.space_required > ent.max_message_size ) + ent.flags |=XEN_ARGO_RING_EMSGSIZE ; +else if ( space_avail >= ent.space_required ) + { +pending_cancel (dst_d, ring_info, currd->domain_id); + ent.flags |=XEN_ARGO_RING_SUFFICIENT ; + } +else + ret =pending_requeue (dst_d, ring_info, currd->domain_id, + ent.space_required); + +spin_unlock (&ring_info->L3_lock); + +if ( space_avail == ent.max_message_size ) + ent.flags |=XEN_ARGO_RING_EMPTY ; + + } +read_unlock (&dst_d->argo->rings_L2_rwlock); + + out: +if ( dst_d ) +put_domain (dst_d); + +if ( !ret && (__copy_field_to_guest (data_ent_hnd, &ent, flags) || +__copy_field_to_guest (data_ent_hnd, &ent, max_message_size)) ) +return -EFAULT ; + +return ret; +} + +static int find_ring_mfn (struct domain *d,gfn_t gfn,mfn_t *mfn) {struct page_info *page;@@ -1586,6 +1836,112 @@ register_ring(struct domain *currd,return ret; } +static void +notify_ring (const struct domain *d,struct argo_ring_info *ring_info, +struct list_head *to_notify) +{ +unsigned int space; + +ASSERT (LOCKING_Read_rings_L2 (d)); + +spin_lock (&ring_info->L3_lock); + +if ( ring_info->len ) + space =ringbuf_payload_space (d, ring_info); +else + space =0 ; + +spin_unlock (&ring_info->L3_lock); + +if ( space ) +pending_find (d, ring_info, space, to_notify); +} + +static void +notify_check_pending (struct domain *d) +{ +unsigned int i; +LIST_HEAD (to_notify); + +ASSERT (LOCKING_Read_L1);
xen/common/argo.c register_ring(struct domain *currd, · line 1876
Quoted 10 lines
xen/common/argo.c +10
-0+ +read_lock (&d->argo->rings_L2_rwlock); + +/* Walk all rings, call notify_ring on each to populate to_notify list */ +for ( i =0 ; i <ARGO_HASHTABLE_SIZE ; i++ ) + { +struct list_head *cursor, *bucket = &d->argo->ring_hash[i]; +struct argo_ring_info *ring_info; + +for ( cursor = bucket->next; cursor != bucket; cursor = cursor->next )
list_for_each_entry
Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 10/15] argo: implement the notify op
Quoted 29 lines · Roger Pau Monné
Quoted 8 lines · Christopher Clark
Queries for data about space availability in registered rings and causes notification to be sent when space has become available.
The hypercall op populates a supplied data structure with information about ring state and if insufficient space is currently available in a given ring, the hypervisor will record the domain's expressed interest and notify it when it observes that space has become available.
Quoted 8 lines
Checks for free space occur when this notify op is invoked, so it may be intentionally invoked with no data structure to populate
(ie. a NULL argument) to trigger such a check and consequent notifications.Limit the maximum number of notify requests in a single operation to a simple fixed limit of 256.
Signed-off-by Christopher Clark <[email protected]>LGTM, but I would like to see the open-coded versions of the
list_macros fixed:Reviewed-by Roger Pau Monné <[email protected]>Quoted 4 lines
xen/common/argo.c +0
-0[...]
Quoted 12 lines
xen/common/argo.c pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) · line 1045
Quoted 10 lines
xen/common/argo.c +10
-0+static void +pending_notify (struct list_head *to_notify) +{ +ASSERT (LOCKING_Read_L1); + +/* Sending signals for all ents in this list, draining until it is empty. */ +while ( !list_empty (to_notify) ) + { +struct pending_ent *ent = + list_entry(to_notify->next, struct pending_ent, node);
list_first_entry_or_null
(same as earlier message: list_first_entry_or_null is not used by Xen)
list_for_each_entry_safe
Quoted 35 lines
Quoted 35 lines
xen/common/argo.c +34
-0+ +list_del (&ent->node); +signal_domid (ent->domain_id); +xfree (ent); + } +} + +static void +pending_find (const struct domain *d,struct argo_ring_info *ring_info, +unsigned int payload_space,struct list_head *to_notify) +{ +struct list_head *cursor, *pending_head; + +ASSERT (LOCKING_Read_rings_L2 (d)); + +/* +* TODO: Current policy here is to signal _all_ of the waiting domains +* interested in sending a message of size less than payload_space. +* +* This is likely to be suboptimal, since once one of them has added +* their message to the ring, there may well be insufficient room +* available for any of the others to transmit, meaning that they were +* woken in vain, which created extra work just to requeue their wait. +* +* Retain this simple policy for now since it at least avoids starving a +* domain of available space notifications because of a policy that only +* notified other domains instead. Improvement may be possible; +* investigation required. +*/ +spin_lock (&ring_info->L3_lock); + +/* Remove matching ents from the ring list, and add them to "to_notify" */ + pending_head = &ring_info->pending; + cursor = pending_head->next;
Quoted 8 lines
xen/common/argo.c pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) · line 1085
Quoted 6 lines
xen/common/argo.c +6
-0+ +while ( cursor != pending_head ) + { + struct pending_ent *ent = list_entry(cursor, struct pending_ent, node); + + cursor = cursor->next;
list_for_each_entry_safe?
ack
Quoted 33 lines
Quoted 33 lines
xen/common/argo.c +25
-0+ +if ( payload_space >= ent->len ) + { +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) +wildcard_pending_list_remove (ent->domain_id, ent); + +list_del (&ent->node); + ring_info->npending--; +list_add (&ent->node, to_notify); + } + } + +spin_unlock (&ring_info->L3_lock); +} +static int pending_queue (const struct domain *d,struct argo_ring_info *ring_info,domid_t src_id,unsigned int len)@@ -1023,6 +1163,36 @@ pending_requeue(const struct domain *d, struct argo_ring_info *ring_info, }static void +pending_cancel (const struct domain *d,struct argo_ring_info *ring_info, +domid_t src_id) +{ +struct list_head *cursor, *pending_head; + +ASSERT (LOCKING_L3 (d, ring_info)); + +/* Remove all ents where domain_id matches src_id from the ring's list. */ + pending_head = &ring_info->pending; + cursor = pending_head->next;
Quoted 8 lines
xen/common/argo.c pending_remove_all(const struct domain *d, struct argo_ring_info *ring_info) · line 1085
Quoted 6 lines
xen/common/argo.c +6
-0+ +while ( cursor != pending_head ) + { + struct pending_ent *ent = list_entry(cursor, struct pending_ent, node); + + cursor = cursor->next;
list_for_each_entry_safe
ack
Quoted 141 lines
Quoted 141 lines
xen/common/argo.c +126
-0+ +if ( ent->domain_id == src_id ) + { +/* For wildcard rings, remove each from their wildcard list too. */ +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) +wildcard_pending_list_remove (ent->domain_id, ent); +list_del (&ent->node); +xfree (ent); + ring_info->npending--; + } + } +} + +static void wildcard_rings_pending_remove (struct domain *d) {struct list_head *wildcard_head;@@ -1158,6 +1328,86 @@ partner_rings_remove(struct domain *src_d) }static int +fill_ring_data (const struct domain *currd, +XEN_GUEST_HANDLE (xen_argo_ring_data_ent_t) data_ent_hnd) +{ +xen_argo_ring_data_ent_t ent; +struct domain *dst_d; +struct argo_ring_info *ring_info; +int ret =0 ; + +ASSERT (currd == current->domain); +ASSERT (LOCKING_Read_L1); + +if (__copy_from_guest (&ent, data_ent_hnd,1 ) ) +return -EFAULT ; + +argo_dprintk ("fill_ring_data: ent.ring.domain=%u,ent.ring.aport=%x\n" , + ent.ring.domain_id, ent.ring.aport); + + ent.flags =0 ; + + dst_d =get_domain_by_id (ent.ring.domain_id); +if ( !dst_d || !dst_d->argo ) + goto out; + +read_lock (&dst_d->argo->rings_L2_rwlock); + + ring_info =find_ring_info_by_match (dst_d, ent.ring.aport, + currd->domain_id); +if ( ring_info ) + { +unsigned int space_avail; + + ent.flags |=XEN_ARGO_RING_EXISTS ; + +spin_lock (&ring_info->L3_lock); + + ent.max_message_size = ring_info->len - +sizeof (struct xen_argo_ring_message_header ) - +ROUNDUP_MESSAGE (1 ); + +if ( ring_info->id.partner_id ==XEN_ARGO_DOMID_ANY ) + ent.flags |=XEN_ARGO_RING_SHARED ; + + space_avail =ringbuf_payload_space (dst_d, ring_info); + +argo_dprintk ("fill_ring_data: aport=%x space_avail=%u" +" space_wanted=%u\n" , + ring_info->id.aport, space_avail, ent.space_required); + +/* Do not queue a notification for an unachievable size */ +if ( ent.space_required > ent.max_message_size ) + ent.flags |=XEN_ARGO_RING_EMSGSIZE ; +else if ( space_avail >= ent.space_required ) + { +pending_cancel (dst_d, ring_info, currd->domain_id); + ent.flags |=XEN_ARGO_RING_SUFFICIENT ; + } +else + ret =pending_requeue (dst_d, ring_info, currd->domain_id, + ent.space_required); + +spin_unlock (&ring_info->L3_lock); + +if ( space_avail == ent.max_message_size ) + ent.flags |=XEN_ARGO_RING_EMPTY ; + + } +read_unlock (&dst_d->argo->rings_L2_rwlock); + + out: +if ( dst_d ) +put_domain (dst_d); + +if ( !ret && (__copy_field_to_guest (data_ent_hnd, &ent, flags) || +__copy_field_to_guest (data_ent_hnd, &ent, max_message_size)) ) +return -EFAULT ; + +return ret; +} + +static int find_ring_mfn (struct domain *d,gfn_t gfn,mfn_t *mfn) {struct page_info *page;@@ -1586,6 +1836,112 @@ register_ring(struct domain *currd,return ret; } +static void +notify_ring (const struct domain *d,struct argo_ring_info *ring_info, +struct list_head *to_notify) +{ +unsigned int space; + +ASSERT (LOCKING_Read_rings_L2 (d)); + +spin_lock (&ring_info->L3_lock); + +if ( ring_info->len ) + space =ringbuf_payload_space (d, ring_info); +else + space =0 ; + +spin_unlock (&ring_info->L3_lock); + +if ( space ) +pending_find (d, ring_info, space, to_notify); +} + +static void +notify_check_pending (struct domain *d) +{ +unsigned int i; +LIST_HEAD (to_notify); + +ASSERT (LOCKING_Read_L1); + +read_lock (&d->argo->rings_L2_rwlock); + +/* Walk all rings, call notify_ring on each to populate to_notify list */
Quoted 8 lines
xen/common/argo.c register_ring(struct domain *currd, · line 1876
Quoted 6 lines
xen/common/argo.c +6
-0+for ( i =0 ; i <ARGO_HASHTABLE_SIZE ; i++ ) + { +struct list_head *cursor, *bucket = &d->argo->ring_hash[i]; +struct argo_ring_info *ring_info; + +for ( cursor = bucket->next; cursor != bucket; cursor = cursor->next )
list_for_each_entry
list_for_each_entry_safe
thanks
Christopher
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 11/15 xsm, argo: XSM control for argo register
In reply to Christopher ClarkXSM controls for argo ring registration with two distinct cases, where the ring being registered is:
1) Single source: registering a ring for communication to receive messages from a specified single other domain. Default policy: allow.
2) Any source: registering a ring for communication to receive messages
from any, or all, other domains (ie. wildcard).
Default policy: deny, with runtime policy configuration via bootparam.
This commit modifies the signature of core XSM hook functions in order to apply 'const' to arguments, needed in order for 'const' to be accepted in signature of functions that invoke them.
---
v3 Daniel/Jan: add to the default xsm policy for the register op v3 hoist opt_argo_mac_permissive check to allow default policy to match non-XSM v3 was: Acked-by: Daniel De Graaf <[email protected]>
v3 Add Daniel's Acked-by ; note minor changes required for v4
v3 feedback #07 Roger: use opt_argo_mac_permissive : a boolean opt v2 feedback #9 Jan: refactor to use argo-mac bootparam at point of introduction v1 feedback Paul: replace use of strncmp with strcmp
v1 feedback #16 Jan: apply const to function signatures
v1 feedback #14 Jan: add blank line before return in parse_argo_mac_param
tools/flask/policy/modules/guest_features.te | 6 ++++++ xen/common/argo.c | 11 +++++++++-- xen/include/xsm/dummy.h | 14 ++++++++++++++ xen/include/xsm/xsm.h | 19 +++++++++++++++++++ xen/xsm/dummy.c | 4 ++++ xen/xsm/flask/hooks.c | 27 ++++++++++++++++++++++++--- xen/xsm/flask/policy/access_vectors | 11 +++++++++++ xen/xsm/flask/policy/security_classes | 1 + 8 files changed, 88 insertions(+), 5 deletions(-)
tools/flask/policy/modules/guest_features.te +6 -0
@@ -5,6 +5,12 @@ allow domain_type xen_t:xen tmem_op; # pmu_ctrl is for) allow domain_type xen_t:xen2 pmu_use; +# Allow all domains: +# to register single-sender (unicast) rings to partner with any domain; and +# to register any-sender (wildcard) rings that can be sent to by any domain. +allow domain_type xen_t:argo { register_any_source }; +allow domain_type domain_type:argo { register_single_source }; + # Allow guest console output to the serial console. This is used by PV Linux # and stub domains for early boot output, so don't audit even when we deny it. # Without XSM, this is enabled only if the Xen was compiled in debug mode.
xen/common/argo.c +9 -2
@@ -26,6 +26,7 @@#include <xen/nospec.h> #include <xen/sched.h> #include <xen/time.h> +#include <xsm/xsm.h> #include <public/argo.h>
@@ -1645,8 +1646,10 @@ register_ring(struct domain *currd,if ( reg.partner_id ==XEN_ARGO_DOMID_ANY ) { -if ( !opt_argo_mac_permissive ) -return -EPERM ; + ret = opt_argo_mac_permissive ?xsm_argo_register_any_source (currd) : + -EPERM ; +if ( ret ) +return ret; }else {
@@ -1657,6 +1660,10 @@ register_ring(struct domain *currd,return -ESRCH ; } + ret =xsm_argo_register_single_source (currd, dst_d); +if ( ret ) + goto out; + send_info = xzalloc(struct argo_send_info);if ( !send_info ) {
xen/include/xsm/dummy.h +14 -0
@@ -720,6 +720,20 @@ static XSM_INLINE int xsm_dm_op(XSM_DEFAULT_ARG struct domain *d)#endif /* CONFIG_X86 */ +#ifdef CONFIG_ARGO +static XSM_INLINE int xsm_argo_register_single_source(struct domain *d, +struct domain *t) +{ +return 0 ; +} + +static XSM_INLINE int xsm_argo_register_any_source(struct domain *d) +{ +return 0 ; +} + +#endif /* CONFIG_ARGO */ +#include <public/version.h> static XSM_INLINE int xsm_xen_version (XSM_DEFAULT_ARG uint32_t op) {
xen/include/xsm/xsm.h +19 -0
@@ -181,6 +181,11 @@ struct xsm_operations {#endif int (*xen_version) (uint32_t cmd);int (*domain_resource_map) (struct domain *d); +#ifdef CONFIG_ARGO +int (*argo_register_single_source) (const struct domain *d, +const struct domain *t); +int (*argo_register_any_source) (const struct domain *d); +#endif };#ifdef CONFIG_XSM
@@ -698,6 +703,20 @@ static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d)return xsm_ops->domain_resource_map (d); } +#ifdef CONFIG_ARGO +static inline xsm_argo_register_single_source(const struct domain *d, + const struct domain *t) +{ + return xsm_ops->argo_register_single_source(d, t); +} + +static inline xsm_argo_register_any_source(const struct domain *d) +{ +return xsm_ops->argo_register_any_source (d); +} + +#endif /* CONFIG_ARGO */ +#endif /* XSM_NO_WRAPPERS */ #ifdef CONFIG_MULTIBOOT
xen/xsm/dummy.c +4 -0
@@ -152,4 +152,8 @@ void __init xsm_fixup_ops (struct xsm_operations *ops)#endif set_to_dummy_if_null (ops, xen_version);set_to_dummy_if_null (ops, domain_resource_map); +#ifdef CONFIG_ARGO +set_to_dummy_if_null (ops, argo_register_single_source); +set_to_dummy_if_null (ops, argo_register_any_source); +#endif }
xen/xsm/flask/hooks.c +24 -3
@@ -36,13 +36,14 @@#include <objsec.h> #include <conditional.h> -static u32 domain_sid (struct domain *dom) +static u32 domain_sid (const struct domain *dom) {struct domain_security_struct *dsec = dom->ssid;return dsec->sid; } -static u32 domain_target_sid (struct domain *src,struct domain *dst) +static u32 domain_target_sid (const struct domain *src, +const struct domain *dst) {struct domain_security_struct *ssec = src->ssid;struct domain_security_struct *dsec = dst->ssid;
@@ -58,7 +59,8 @@ static u32 evtchn_sid(const struct evtchn *chn)return chn->ssid.flask_sid; } -static int domain_has_perm (struct domain *dom1,struct domain *dom2, +static int domain_has_perm (const struct domain *dom1, +const struct domain *dom2,u16 class,u32 perms) {u32 ssid, tsid;
@@ -1717,6 +1719,21 @@ static int flask_domain_resource_map(struct domain *d)return current_has_perm (d,SECCLASS_DOMAIN2 ,DOMAIN2__RESOURCE_MAP ); } +#ifdef CONFIG_ARGO +static int flask_argo_register_single_source (const struct domain *d, +const struct domain *t) +{ +return domain_has_perm (d, t,SECCLASS_ARGO , +ARGO__REGISTER_SINGLE_SOURCE ); +} + +static int flask_argo_register_any_source (const struct domain *d) +{ +return avc_has_perm (domain_sid (d),SECINITSID_XEN ,SECCLASS_ARGO , +ARGO__REGISTER_ANY_SOURCE ,NULL ); +} +#endif +long do_flask_op (XEN_GUEST_HANDLE_PARAM (xsm_op_t) u_flask_op);int compat_flask_op (XEN_GUEST_HANDLE_PARAM (xsm_op_t) u_flask_op);
@@ -1851,6 +1868,10 @@ static struct xsm_operations flask_ops = {#endif .xen_version = flask_xen_version, .domain_resource_map = flask_domain_resource_map, +#ifdef CONFIG_ARGO + .argo_register_single_source = flask_argo_register_single_source, + .argo_register_any_source = flask_argo_register_any_source, +#endif };void __init flask_init (const void *policy_buffer,size_t policy_size)
xen/xsm/flask/policy/access_vectors +11 -0
@@ -531,3 +531,14 @@ class version # Xen build id xen_build_id } + +# Class argo is used to describe the Argo interdomain communication system. +class argo +{ + # Domain requesting registration of a communication ring + # to receive messages from a specific other domain. + register_single_source + # Domain requesting registration of a communication ring + # to receive messages from any other domain. + register_any_source +}
xen/xsm/flask/policy/security_classes +1 -0
@@ -19,5 +19,6 @@ class event class grant class security class version +class argo # FLASK
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 12/15 xsm, argo: XSM control for argo message send operation
In reply to Christopher ClarkDefault policy: allow.
---
v3 Daniel/Jan: add to the default xsm policy for the send op v3 Add Daniel's Acked-by
v2: reordered commit sequence to after sendv implementation v1 feedback Jan #16: apply const to function signatures v1 version was: Reviewed-by: Paul Durrant <[email protected]>
tools/flask/policy/modules/guest_features.te | 7 ++++--- xen/common/argo.c | 11 +++++++++++ xen/include/xsm/dummy.h | 6 ++++++ xen/include/xsm/xsm.h | 6 ++++++ xen/xsm/dummy.c | 1 + xen/xsm/flask/hooks.c | 7 +++++++ xen/xsm/flask/policy/access_vectors | 2 ++ 7 files changed, 37 insertions(+), 3 deletions(-)
tools/flask/policy/modules/guest_features.te +4 -3
@@ -6,10 +6,11 @@ allow domain_type xen_t:xen tmem_op; allow domain_type xen_t:xen2 pmu_use; # Allow all domains: -# to register single-sender (unicast) rings to partner with any domain; and -# to register any-sender (wildcard) rings that can be sent to by any domain. +# to register single-sender (unicast) rings to partner with any domain; +# to register any-sender (wildcard) rings that can be sent to by any domain; +# and send messages to rings. allow domain_type xen_t:argo { register_any_source }; -allow domain_type domain_type:argo { register_single_source }; +allow domain_type domain_type:argo { send register_single_source }; # Allow guest console output to the serial console. This is used by PV Linux # and stub domains for early boot output, so don't audit even when we deny it.
xen/common/argo.c +11 -0
@@ -1976,6 +1976,17 @@ sendv(struct domain *src_d, const xen_argo_addr_t *src_addr,if ( !dst_d )return -ESRCH ; + ret =xsm_argo_send (src_d, dst_d); +if ( ret ) + { +gprintk (XENLOG_ERR ,"argo: XSM REJECTED %i -> %i\n" , + src_d->domain_id, dst_d->domain_id); + +put_domain (dst_d); + +return ret; + } +read_lock (&L1_global_argo_rwlock);if ( !src_d->argo )
xen/include/xsm/dummy.h +6 -0
@@ -732,6 +732,12 @@ static XSM_INLINE int xsm_argo_register_any_source(struct domain *d)return 0 ; } +static XSM_INLINE int xsm_argo_send(const struct domain *d, +const struct domain *t) +{ +return 0 ; +} +#endif /* CONFIG_ARGO */ #include <public/version.h>
xen/include/xsm/xsm.h +6 -0
@@ -185,6 +185,7 @@ struct xsm_operations {int (*argo_register_single_source) (const struct domain *d,const struct domain *t);int (*argo_register_any_source) (const struct domain *d); +int (*argo_send) (const struct domain *d,const struct domain *t);#endif };
@@ -715,6 +716,11 @@ static inline xsm_argo_register_any_source(const struct domain *d)return xsm_ops->argo_register_any_source (d); } +static inline int xsm_argo_send (const struct domain *d,const struct domain *t) +{ +return xsm_ops->argo_send (d, t); +} +#endif /* CONFIG_ARGO */ #endif /* XSM_NO_WRAPPERS */
xen/xsm/dummy.c +1 -0
@@ -155,5 +155,6 @@ void __init xsm_fixup_ops (struct xsm_operations *ops)#ifdef CONFIG_ARGO set_to_dummy_if_null (ops, argo_register_single_source);set_to_dummy_if_null (ops, argo_register_any_source); +set_to_dummy_if_null (ops, argo_send);#endif }
xen/xsm/flask/hooks.c +7 -0
@@ -1732,6 +1732,12 @@ static int flask_argo_register_any_source(const struct domain *d)return avc_has_perm (domain_sid (d),SECINITSID_XEN ,SECCLASS_ARGO , ARGO__REGISTER_ANY_SOURCE, NULL); } + +static int flask_argo_send (const struct domain *d,const struct domain *t) +{ +return domain_has_perm (d, t,SECCLASS_ARGO ,ARGO__SEND ); +} +#endif long do_flask_op (XEN_GUEST_HANDLE_PARAM (xsm_op_t) u_flask_op);
@@ -1871,6 +1877,7 @@ static struct xsm_operations flask_ops = {#ifdef CONFIG_ARGO .argo_register_single_source = flask_argo_register_single_source, .argo_register_any_source = flask_argo_register_any_source, + .argo_send = flask_argo_send,#endif };
xen/xsm/flask/policy/access_vectors +2 -0
@@ -541,4 +541,6 @@ class argo # Domain requesting registration of a communication ring # to receive messages from any other domain. register_any_source + # Domain sending a message to another domain. + send }
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 13/15 xsm, argo: XSM control for any access to argo by a domain
In reply to Christopher ClarkWill inhibit initialization of the domain's argo data structure to prevent receiving any messages or notifications and access to any of the argo hypercall operations.
---
v3 Daniel/Jan: add to the default xsm policy for enable
v3 Add Daniel's Acked-by
v3 #04 Jason/Roger: soft_reset: can assume reinit is ok if d->argo set v2 self: fix xsm use in soft-reset prior to introduction
v1 #5 (#17) feedback Paul: XSM control for any access: use currd v1 #16 feedback Jan: apply const to function signatures
tools/flask/policy/modules/guest_features.te | 4 ++-- xen/common/argo.c | 10 +++++----- xen/include/xsm/dummy.h | 5 +++++ xen/include/xsm/xsm.h | 6 ++++++ xen/xsm/dummy.c | 1 + xen/xsm/flask/hooks.c | 7 +++++++ xen/xsm/flask/policy/access_vectors | 3 +++ 7 files changed, 29 insertions(+), 7 deletions(-)
tools/flask/policy/modules/guest_features.te +2 -2
@@ -5,11 +5,11 @@ allow domain_type xen_t:xen tmem_op; # pmu_ctrl is for) allow domain_type xen_t:xen2 pmu_use; -# Allow all domains: +# Allow all domains to enable the Argo interdomain communication hypercall; # to register single-sender (unicast) rings to partner with any domain; # to register any-sender (wildcard) rings that can be sent to by any domain; # and send messages to rings. -allow domain_type xen_t:argo { register_any_source }; +allow domain_type xen_t:argo { enable register_any_source }; allow domain_type domain_type:argo { send register_single_source }; # Allow guest console output to the serial console. This is used by PV Linux
xen/common/argo.c +5 -5
@@ -2056,7 +2056,7 @@ do_argo_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) arg1,argo_dprintk ("->do_argo_op(%u,%p,%p,%lu,0x%lx)\n" , cmd, (void *)arg1.p, (void *)arg2.p, arg3, arg4); -if (unlikely (!opt_argo) ) +if (unlikely (!opt_argo ||xsm_argo_enable (currd)) )return -EOPNOTSUPP ; switch (cmd)
@@ -2202,7 +2202,7 @@ argo_init(struct domain *d) {struct argo_domain *argo; -if ( !opt_argo ) +if ( !opt_argo ||xsm_argo_enable (d) ) {argo_dprintk ("argo disabled, domid: %u\n" , d->domain_id);return 0 ;
@@ -2259,9 +2259,9 @@ argo_soft_reset(struct domain *d)wildcard_rings_pending_remove (d);/* -* Since opt_argo cannot change at runtime, if d->argo is true then -* opt_argo must be true, and we can assume that init is allowed to -* proceed again here. +* Since neither opt_argo or xsm_argo_enable(d) can change at runtime, +* if d->argo is true then both opt_argo and xsm_argo_enable(d) must be +* true, and we can assume that init is allowed to proceed again here. */ argo_domain_init (d->argo); }
xen/include/xsm/dummy.h +5 -0
@@ -721,6 +721,11 @@ static XSM_INLINE int xsm_dm_op(XSM_DEFAULT_ARG struct domain *d)#endif /* CONFIG_X86 */ #ifdef CONFIG_ARGO +static XSM_INLINE int xsm_argo_enable(struct domain *d) +{ +return 0 ; +} + static XSM_INLINE int xsm_argo_register_single_source(struct domain *d,struct domain *t) {
xen/include/xsm/xsm.h +6 -0
@@ -182,6 +182,7 @@ struct xsm_operations {int (*xen_version) (uint32_t cmd);int (*domain_resource_map) (struct domain *d);#ifdef CONFIG_ARGO +int (*argo_enable) (const struct domain *d);int (*argo_register_single_source) (const struct domain *d,const struct domain *t);int (*argo_register_any_source) (const struct domain *d);
@@ -705,6 +706,11 @@ static inline int xsm_domain_resource_map(xsm_default_t def, struct domain *d) }#ifdef CONFIG_ARGO +static inline xsm_argo_enable(const struct domain *d) +{ +return xsm_ops->argo_enable (d); +} + static inline xsm_argo_register_single_source(const struct domain *d, const struct domain *t) {
xen/xsm/dummy.c +1 -0
@@ -153,6 +153,7 @@ void __init xsm_fixup_ops (struct xsm_operations *ops)set_to_dummy_if_null (ops, xen_version);set_to_dummy_if_null (ops, domain_resource_map);#ifdef CONFIG_ARGO +set_to_dummy_if_null (ops, argo_enable);set_to_dummy_if_null (ops, argo_register_single_source);set_to_dummy_if_null (ops, argo_register_any_source);set_to_dummy_if_null (ops, argo_send);
xen/xsm/flask/hooks.c +7 -0
@@ -1720,6 +1720,12 @@ static int flask_domain_resource_map(struct domain *d) }#ifdef CONFIG_ARGO +static int flask_argo_enable (const struct domain *d) +{ +return avc_has_perm (domain_sid (d),SECINITSID_XEN ,SECCLASS_ARGO , +ARGO__ENABLE ,NULL ); +} +static int flask_argo_register_single_source (const struct domain *d,const struct domain *t) {
@@ -1875,6 +1881,7 @@ static struct xsm_operations flask_ops = { .xen_version = flask_xen_version, .domain_resource_map = flask_domain_resource_map, #ifdef CONFIG_ARGO + .argo_enable = flask_argo_enable, .argo_register_single_source = flask_argo_register_single_source, .argo_register_any_source = flask_argo_register_any_source, .argo_send = flask_argo_send,
xen/xsm/flask/policy/access_vectors +3 -0
@@ -535,6 +535,9 @@ class version # Class argo is used to describe the Argo interdomain communication system. class argo { + # Enable initialization of a domain's argo subsystem and + # permission to access the argo hypercall operations. + enable # Domain requesting registration of a communication ring # to receive messages from a specific other domain. register_single_source
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 14/15 xsm, argo: notify: don't describe rings that cannot be sent to
In reply to Christopher Clark---
v3 #10 Roger: drop out label, use return -EFAULT in fill_ring_data
v3: Add Daniel's Acked-by
xen/common/argo.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
xen/common/argo.c +11 -0
@@ -1352,6 +1352,17 @@ fill_ring_data(const struct domain *currd,if ( !dst_d || !dst_d->argo ) goto out; +/* +* Don't supply information about rings that a guest is not +* allowed to send to. +*/ + ret =xsm_argo_send (currd, dst_d); +if ( ret ) + { +put_domain (dst_d); +return ret; + } +read_lock (&dst_d->argo->rings_L2_rwlock); ring_info = find_ring_info_by_match(dst_d, ent.ring.aport,
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 15/15 MAINTAINERS: add new section for Argo and self as maintainer
In reply to Christopher Clark---
MAINTAINERS | 8 ++++++++ 1 file changed, 8 insertions(+)
MAINTAINERS +8 -0
@@ -158,6 +158,14 @@ S: Supported F: xen/arch/x86/hvm/svm/ F: xen/arch/x86/cpu/vpmu_amd.c +ARGO +M: Christopher Clark <[email protected]> +S: Maintained +F: xen/include/public/argo.h +F: xen/include/xen/argo.h +F: xen/common/argo.c +F: xen/common/compat/argo.c + ARINC653 SCHEDULER M: Josh Whitehead <[email protected]> M: Robert VanVossen <[email protected]>
Signature · 7 lines
2.7.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 15/15] MAINTAINERS: add new section for Argo and self as maintainer
Quoted 12 lines · Christopher Clark
Signed-off-by Christopher Clark <[email protected]>---
MAINTAINERS | 8 ++++++++ 1 file changed, 8 insertions(+)MAINTAINERS +0
-0@@ -158,6 +158,14 @@ S: Supported F: xen/arch/x86/hvm/svm/
MAINTAINERS S: Supported · line 168
Quoted 10 lines
MAINTAINERS +8
-0F: xen/arch/x86/cpu/vpmu_amd.c +ARGO +M: Christopher Clark <[email protected]> +S: Maintained +F: xen/include/public/argo.h +F: xen/include/xen/argo.h +F: xen/common/argo.c +F: xen/common/compat/argo.c +
Use tab for indentation please.
Wei.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v5 15/15] MAINTAINERS: add new section for Argo and self as maintainer
In reply to Christopher ClarkQuoted 4 lines
Quoted 1 line
Quoted 1 line · On 21.01.19 at 10:59, <[email protected]> wrote:
MAINTAINERS S: Supported · line 167
Quoted 10 lines
MAINTAINERS +7
-0@@ -158,6 +158,14 @@ S: Supported F: xen/arch/x86/hvm/svm/ F: xen/arch/x86/cpu/vpmu_amd.c +ARGO +M: Christopher Clark <[email protected]> +S: Maintained +F: xen/include/public/argo.h +F: xen/include/xen/argo.h +F: xen/common/argo.c +F: xen/common/compat/argo.c
Please follow the whitespace model of adjacent entries, the more in light of this recent patch:
https://lists.xenproject.org/archives/html/xen-devel/2019-01/msg00196.html
(I can't really tell why I didn't apply it yet)
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 13 lines · Christopher Clark
Version five of this patch series:
* Changes are primarily addressing feedback from the v4 series reviews. Many points noted on the invididual commit posts.
* Critical sections have been shrunk, with allocations and frees pulled outside where possible, reordering logic within hypercall ops.
* A new ring hash function implemented, derived from the djb2 string hash function.
* Flags returned by the notify op have been simplified.
Quoted 7 lines
* Now uses a single argo boot parameter, taking a list:
- top level boolean to enable/disable Argo
- mac-permissive option to enable/disable wildcard rings
- command line doc edit: no "CONFIG_ARGO" but refers to build config* Switched to use the standard list data structures used by Xen's common code.
AFAIK this was not requested by any reviewer, so I wonder why you made such change. The more that you open coded some of the list_ macros instead of just doing a s/hlist_/list_/ replacement.
I'm fine with using list instead of hlist, but I don't understand why you decided to open code list_for_each and list_for_each_safe instead of using the macros provided by Xen. Is there an issue with such macros?
I've made a couple of minor comments, but I think the current status is good, and fixing those minor comments is going to be trivial.
Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 20 lines · Roger Pau Monné
Quoted 18 lines · Christopher Clark
Version five of this patch series:
* Changes are primarily addressing feedback from the v4 series reviews. Many points noted on the invididual commit posts.
* Critical sections have been shrunk, with allocations and frees pulled outside where possible, reordering logic within hypercall ops.
* A new ring hash function implemented, derived from the djb2 string hash function.
* Flags returned by the notify op have been simplified.
* Now uses a single argo boot parameter, taking a list:
- top level boolean to enable/disable Argo
- mac-permissive option to enable/disable wildcard rings
- command line doc edit: no "CONFIG_ARGO" but refers to build config
Quoted 7 lines
Quoted 2 lines
* Switched to use the standard list data structures used by Xen's common code.
AFAIK this was not requested by any reviewer, so I wonder why you made such change. The more that you open coded some of the
list_macros instead of just doing a s/hlist_/list_/ replacement.
I'm fine with using list instead of hlist,
At your request, v7 replaces open coding with Xen's list macros. The hlist macros were not used by any of the common code in Xen.
Quoted 4 lines
but I don't understand why
you decided to open codelist_for_eachandlist_for_each_safeinstead of using the macros provided by Xen. Is there an issue with such macros?
As discussed offline:
- Using Xen's list macros will expedite Argo's merge for Xen 4.12
- List macros in Xen list.h originated in Linux list.h and have diverged
- OpenXT has use cases for measured launch and nested virtualization, which influence downstream performance and security requirements for Argo and Xen
- OpenXT can temporarily patch Xen 4.12 for downstream use
Quoted 2 lines
I've made a couple of minor comments, but I think the current status is good, and fixing those minor comments is going to be trivial.
Ack, thanks. Hopefully v7 looks good.
Christopher
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 46 lines · Christopher Clark
Quoted 20 lines · Roger Pau Monné
Quoted 18 lines · Christopher Clark
Version five of this patch series:
* Changes are primarily addressing feedback from the v4 series reviews. Many points noted on the invididual commit posts.
* Critical sections have been shrunk, with allocations and frees pulled outside where possible, reordering logic within hypercall ops.
* A new ring hash function implemented, derived from the djb2 string hash function.
* Flags returned by the notify op have been simplified.
* Now uses a single argo boot parameter, taking a list:
- top level boolean to enable/disable Argo
- mac-permissive option to enable/disable wildcard rings
- command line doc edit: no "CONFIG_ARGO" but refers to build configQuoted 7 lines
Quoted 2 lines
* Switched to use the standard list data structures used by Xen's common code.
AFAIK this was not requested by any reviewer, so I wonder why you made such change. The more that you open coded some of the
list_macros instead of just doing a s/hlist_/list_/ replacement.
I'm fine with using list instead of hlist,At your request, v7 replaces open coding with Xen's list macros. The hlist macros were not used by any of the common code in Xen.
Quoted 4 lines
but I don't understand why
you decided to open codelist_for_eachandlist_for_each_safeinstead of using the macros provided by Xen. Is there an issue with such macros?As discussed offline:
- Using Xen's list macros will expedite Argo's merge for Xen 4.12
- List macros in Xenlist.horiginated in Linuxlist.hand have diverged
- OpenXT has use cases for measured launch and nested virtualization, which influence downstream performance and security requirements for Argo and Xen
- OpenXT can temporarily patch Xen 4.12 for downstream use
Quoted 4 lines
Quoted 2 lines
I've made a couple of minor comments, but I think the current status is good, and fixing those minor comments is going to be trivial.
Ack, thanks. Hopefully v7 looks good.
As a note, the common flow of interactions usually involves the contributor replying to the comments made by the reviewer in order to try to reach an agreement before sending a new version.
There are comments from v5 that haven't been fixed in v7 (the mask usage and list_first_entry_or_null for example) and the reply to the reviewer's comment was sent at the same time as v7, leaving no time for further discussion (and for reaching an agreement suitable to both parties) before sending v7.
Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 48 lines · Roger Pau Monné
Quoted 46 lines · Christopher Clark
Quoted 20 lines · Roger Pau Monné
Quoted 18 lines · Christopher Clark
Version five of this patch series:
* Changes are primarily addressing feedback from the v4 series reviews. Many points noted on the invididual commit posts.
* Critical sections have been shrunk, with allocations and frees pulled outside where possible, reordering logic within hypercall ops.
* A new ring hash function implemented, derived from the djb2 string hash function.
* Flags returned by the notify op have been simplified.
* Now uses a single argo boot parameter, taking a list:
- top level boolean to enable/disable Argo
- mac-permissive option to enable/disable wildcard rings
- command line doc edit: no "CONFIG_ARGO" but refers to build configQuoted 7 lines
Quoted 2 lines
* Switched to use the standard list data structures used by Xen's common code.
AFAIK this was not requested by any reviewer, so I wonder why you made such change. The more that you open coded some of the
list_macros instead of just doing a s/hlist_/list_/ replacement.
I'm fine with using list instead of hlist,At your request, v7 replaces open coding with Xen's list macros. The hlist macros were not used by any of the common code in Xen.
Quoted 4 lines
but I don't understand why
you decided to open codelist_for_eachandlist_for_each_safeinstead of using the macros provided by Xen. Is there an issue with such macros?As discussed offline:
- Using Xen's list macros will expedite Argo's merge for Xen 4.12
- List macros in Xenlist.horiginated in Linuxlist.hand have diverged
- OpenXT has use cases for measured launch and nested virtualization, which influence downstream performance and security requirements for Argo and Xen
- OpenXT can temporarily patch Xen 4.12 for downstream use
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
I've made a couple of minor comments, but I think the current status is good, and fixing those minor comments is going to be trivial.
Ack, thanks. Hopefully v7 looks good.
As a note, the common flow of interactions usually involves the contributor replying to the comments made by the reviewer in order to try to reach an agreement before sending a new version.
Yes, v7 was sent to address Jan and Julien's review comments in parallel with our ongoing discussion on v5 macros. v7 also provided a checkpoint for Argo testers to maximize test coverage as the series converges into a Xen 4.12 merge candidate for Juergen. It addressed:
- Jan's v6 review comments
- Julien's v1 review comment
- most of your xen-devel and offline review comments
Quoted 5 lines
There are comments from v5 that haven't been fixed in v7 (the mask usage and
list_first_entry_or_nullfor example) and the reply to the reviewer's comment was sent at the same time as v7, leaving no time for further discussion (and for reaching an agreement suitable to both parties) before sending v7.
Code changes from our ongoing discussion will be addressed in v8. A proposal to address mask usage has been put forward in the parallel thread. Your proposed usage of list_first_entry_or_null will be made in v8, subject to the previous offline discussion about list macros (duplicated here for convenience):
Quoted 8 lines
Quoted 8 lines
As discussed offline:
- Using Xen's list macros will expedite Argo's merge for Xen 4.12
- List macros in Xenlist.horiginated in Linuxlist.hand have diverged
- OpenXT has use cases for measured launch and nested virtualization, which influence downstream performance and security requirements for Argo and Xen
- OpenXT can temporarily patch Xen 4.12 for downstream use
Christopher
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 58 lines · Christopher Clark
Quoted 48 lines · Roger Pau Monné
Quoted 46 lines · Christopher Clark
Quoted 20 lines · Roger Pau Monné
Quoted 18 lines · Christopher Clark
Version five of this patch series:
* Changes are primarily addressing feedback from the v4 series reviews. Many points noted on the invididual commit posts.
* Critical sections have been shrunk, with allocations and frees pulled outside where possible, reordering logic within hypercall ops.
* A new ring hash function implemented, derived from the djb2 string hash function.
* Flags returned by the notify op have been simplified.
* Now uses a single argo boot parameter, taking a list:
- top level boolean to enable/disable Argo
- mac-permissive option to enable/disable wildcard rings
- command line doc edit: no "CONFIG_ARGO" but refers to build configQuoted 7 lines
Quoted 2 lines
* Switched to use the standard list data structures used by Xen's common code.
AFAIK this was not requested by any reviewer, so I wonder why you made such change. The more that you open coded some of the
list_macros instead of just doing a s/hlist_/list_/ replacement.
I'm fine with using list instead of hlist,At your request, v7 replaces open coding with Xen's list macros. The hlist macros were not used by any of the common code in Xen.
Quoted 4 lines
but I don't understand why
you decided to open codelist_for_eachandlist_for_each_safeinstead of using the macros provided by Xen. Is there an issue with such macros?As discussed offline:
- Using Xen's list macros will expedite Argo's merge for Xen 4.12
- List macros in Xenlist.horiginated in Linuxlist.hand have diverged
- OpenXT has use cases for measured launch and nested virtualization, which influence downstream performance and security requirements for Argo and Xen
- OpenXT can temporarily patch Xen 4.12 for downstream useQuoted 8 lines
Quoted 4 lines
Quoted 2 lines
I've made a couple of minor comments, but I think the current status is good, and fixing those minor comments is going to be trivial.
Ack, thanks. Hopefully v7 looks good.
As a note, the common flow of interactions usually involves the contributor replying to the comments made by the reviewer in order to try to reach an agreement before sending a new version.
Quoted 8 lines
Yes, v7 was sent to address Jan and Julien's review comments in parallel with our ongoing discussion on v5 macros. v7 also provided a checkpoint for Argo testers to maximize test coverage as the series converges into a Xen 4.12 merge candidate for Juergen. It addressed:
- Jan's v6 review comments
- Julien's v1 review comment
- most of your xen-devel and offline review comments
I think it will benefit the community to give this review in public, so other reviewers know whats going on. IMO getting this private review makes it harder for me (as a reviewer) to know the motivation of some of the changes between versions, and likely also makes it harder for you since you have to keep track of comments from multiple sources on different channels.
Is there anything that prevents those people from making the review comments publicly on xen-devel?
We should very much try to fix that so everyone can make review comments on the public mailing list.
Quoted 6 lines
Quoted 5 lines
There are comments from v5 that haven't been fixed in v7 (the mask usage and
list_first_entry_or_nullfor example) and the reply to the reviewer's comment was sent at the same time as v7, leaving no time for further discussion (and for reaching an agreement suitable to both parties) before sending v7.
Quoted 13 lines
Code changes from our ongoing discussion will be addressed in v8. A proposal to address mask usage has been put forward in the parallel thread. Your proposed usage of
list_first_entry_or_nullwill be made in v8, subject to the previous offline discussion about list macros (duplicated here for convenience):Quoted 7 lines
Quoted 7 lines
As discussed offline:
- Using Xen's list macros will expedite Argo's merge for Xen 4.12
- List macros in Xenlist.horiginated in Linuxlist.hand have diverged
- OpenXT has use cases for measured launch and nested virtualization, which influence downstream performance and security requirements for Argo and Xen
FWIW, I don't see the connection between nested virtualization or measured launch and the list macros. I think a little bit more context would be helpful here in order to understand the issue.
Quoted 1 line
Quoted 1 line
Quoted 1 line
- OpenXT can temporarily patch Xen 4.12 for downstream use
Patching the macros for OpenXT is perfectly fine, but it would be better to understand and fix the problem upstream if possible.
How are you patching the macros?
What are you trying to achieve by patching them?
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 9 lines · Roger Pau Monné
Quoted 8 lines
Yes, v7 was sent to address Jan and Julien's review comments in parallel with our ongoing discussion on v5 macros. v7 also provided a checkpoint for Argo testers to maximize test coverage as the series converges into a Xen 4.12 merge candidate for Juergen. It addressed:
- Jan's v6 review comments
- Julien's v1 review comment
- most of your xen-devel and offline review comments
Quoted 6 lines
I think it will benefit the community to give this review in public, so other reviewers know whats going on. IMO getting this private review makes it harder for me (as a reviewer) to know the motivation of some of the changes between versions, and likely also makes it harder for you since you have to keep track of comments from multiple sources on different channels.
There is one more reason to require public comments which I have only learned recently: for safety certifications we need to keep a record of all review comments and patches that address them for traceability.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 19 lines · Stefano Stabellini
Quoted 9 lines · Roger Pau Monné
Quoted 8 lines
Yes, v7 was sent to address Jan and Julien's review comments in parallel with our ongoing discussion on v5 macros. v7 also provided a checkpoint for Argo testers to maximize test coverage as the series converges into a Xen 4.12 merge candidate for Juergen. It addressed:
- Jan's v6 review comments
- Julien's v1 review comment
- most of your xen-devel and offline review commentsQuoted 6 lines
I think it will benefit the community to give this review in public, so other reviewers know whats going on. IMO getting this private review makes it harder for me (as a reviewer) to know the motivation of some of the changes between versions, and likely also makes it harder for you since you have to keep track of comments from multiple sources on different channels.
Quoted 3 lines
There is one more reason to require public comments which I have only learned recently: for safety certifications we need to keep a record of all review comments and patches that address them for traceability.
Do you mean:
(A) all _merged_ patches and their review comments
or
(B) all comments and patches (merged or not) that address them
i.e. would the certification process be seeking traceability of safety-impacting patches (code, scenario A) or decisions (including decisions to leave code unchanged, scenario B)?
If you mean (B), would we need an update to the Xen Security Problem Response Process [1]? e.g. public archive of all comments from pre-disclosure discussion, along with content hashes stored immutably?
Rich
[1] https://www.xenproject.org/security-policy.html
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 39 lines · Rich Persaud
On Feb 4, 2019, at 13:22, Stefano Stabellini <[email protected]> wrote:
On Mon, 4 Feb 2019, Roger Pau Monné wrote:
Yes, v7 was sent to address Jan and Julien's review comments in parallelwith our ongoing discussion on v5 macros. v7 also provided a checkpoint
for Argo testers to maximize test coverage as the series converges into
a Xen 4.12 merge candidate for Juergen. It addressed:
- Jan's v6 review comments
- Julien's v1 review comment
- most of your xen-devel and offline review comments
I think it will benefit the community to give this review in public,
so other reviewers know whats going on. IMO getting this private
review makes it harder for me (as a reviewer) to know the motivation
of some of the changes between versions, and likely also makes it
harder for you since you have to keep track of comments from multiple
sources on different channels.
There is one more reason to require public comments which I have only learned recently: for safety certifications we need to keep a record of all review comments and patches that address them for traceability.
Do you mean:
Quoted 8 lines
(A) all _merged_ patches and their review comments
or
(B) all comments and patches (merged or not) that address them
i.e. would the certification process be seeking traceability of safety-impacting patches (code, scenario A) or decisions (including decisions to leave code unchanged, scenario B)?
I meant (A), however, I don't know specifically if anything from (B) is also required.
Quoted 5 lines
If you mean (B), would we need an update to the Xen Security Problem Response Process [1]? e.g. public archive of all comments from pre-disclosure discussion, along with content hashes stored immutably?
Rich
I don't think the archives of the pre-disclosure security discussions need to be public, but they probably need to be available.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Gentle ping on the questions below.
Quoted 26 lines · Roger Pau Monné
Quoted 5 lines · Christopher Clark
Quoted 4 lines · Roger Pau Monné
Quoted 2 lines · Christopher Clark
Quoted 1 line · Roger Pau Monné
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line · Christopher Clark
As discussed offline:
- Using Xen's list macros will expedite Argo's merge for Xen 4.12
- List macros in Xenlist.horiginated in Linuxlist.hand have diverged
- OpenXT has use cases for measured launch and nested virtualization, which influence downstream performance and security requirements for Argo and XenFWIW, I don't see the connection between nested virtualization or measured launch and the list macros. I think a little bit more context would be helpful here in order to understand the issue.
Quoted 1 line
Quoted 1 line
Quoted 1 line
- OpenXT can temporarily patch Xen 4.12 for downstream use
Patching the macros for OpenXT is perfectly fine, but it would be better to understand and fix the problem upstream if possible.
How are you patching the macros?
Quoted 8 lines
What are you trying to achieve by patching them?
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Wrong 'To:' field in the previous email, sorry.
Gentle ping on the questions below.
Quoted 26 lines · Roger Pau Monné
Quoted 5 lines · Christopher Clark
Quoted 4 lines · Roger Pau Monné
Quoted 2 lines · Christopher Clark
Quoted 1 line · Roger Pau Monné
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line · Christopher Clark
As discussed offline:
- Using Xen's list macros will expedite Argo's merge for Xen 4.12
- List macros in Xenlist.horiginated in Linuxlist.hand have diverged
- OpenXT has use cases for measured launch and nested virtualization, which influence downstream performance and security requirements for Argo and XenFWIW, I don't see the connection between nested virtualization or measured launch and the list macros. I think a little bit more context would be helpful here in order to understand the issue.
Quoted 1 line
Quoted 1 line
Quoted 1 line
- OpenXT can temporarily patch Xen 4.12 for downstream use
Patching the macros for OpenXT is perfectly fine, but it would be better to understand and fix the problem upstream if possible.
How are you patching the macros?
Quoted 8 lines
What are you trying to achieve by patching them?
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 78 lines
Quoted 62 lines
Quoted 53 lines
Quoted 53 lines
Quoted 28 lines · Roger Pau Monné
Quoted 26 lines · Christopher Clark
On Thu, Jan 31, 2019 at 5:39 AM Roger Pau Monné <[email protected]> wrote:
On Wed, Jan 30, 2019 at 08:05:30PM -0800, Christopher Clark wrote:
On Tue, Jan 22, 2019 at 6:19 AM Roger Pau Monné <[email protected]> wrote:On Mon, Jan 21, 2019 at 01:59:40AM -0800, Christopher Clark wrote:
Version five of this patch series:* Changes are primarily addressing feedback from the v4 series reviews. Many points noted on the invididual commit posts.
* Critical sections have been shrunk, with allocations and frees pulled outside where possible, reordering logic within hypercall ops.
* A new ring hash function implemented, derived from the djb2 string hash function.
* Flags returned by the notify op have been simplified.
* Now uses a single argo boot parameter, taking a list:
- top level boolean to enable/disable Argo
- mac-permissive option to enable/disable wildcard rings
- command line doc edit: no "CONFIG_ARGO" but refers to build configQuoted 7 lines
Quoted 2 lines
* Switched to use the standard list data structures used by Xen's common code.
AFAIK this was not requested by any reviewer, so I wonder why you made such change. The more that you open coded some of the
list_macros instead of just doing a s/hlist_/list_/ replacement.
I'm fine with using list instead of hlist,At your request, v7 replaces open coding with Xen's list macros. The hlist macros were not used by any of the common code in Xen.
Quoted 4 lines
but I don't understand why
you decided to open codelist_for_eachandlist_for_each_safeinstead of using the macros provided by Xen. Is there an issue with such macros?As discussed offline:
- Using Xen's list macros will expedite Argo's merge for Xen 4.12
- List macros in Xenlist.horiginated in Linuxlist.hand have diverged
- OpenXT has use cases for measured launch and nested virtualization, which influence downstream performance and security requirements for Argo and Xen
- OpenXT can temporarily patch Xen 4.12 for downstream useQuoted 8 lines
Quoted 4 lines
Quoted 2 lines
I've made a couple of minor comments, but I think the current status is good, and fixing those minor comments is going to be trivial.
Ack, thanks. Hopefully v7 looks good.
As a note, the common flow of interactions usually involves the contributor replying to the comments made by the reviewer in order to try to reach an agreement before sending a new version.
Quoted 8 lines
Yes, v7 was sent to address Jan and Julien's review comments in parallel with our ongoing discussion on v5 macros. v7 also provided a checkpoint for Argo testers to maximize test coverage as the series converges into a Xen 4.12 merge candidate for Juergen. It addressed:
- Jan's v6 review comments
- Julien's v1 review comment
- most of your xen-devel and offline review commentsI think it will benefit the community to give this review in public, so other reviewers know whats going on. IMO getting this private review makes it harder for me (as a reviewer) to know the motivation of some of the changes between versions, and likely also makes it harder for you since you have to keep track of comments from multiple sources on different channels.
Quoted 5 lines
Is there anything that prevents those people from making the review comments publicly on xen-devel?
We should very much try to fix that so everyone can make review comments on the public mailing list.
I've advocated for open-source principles in several large organizations. At XenSource and Citrix, we created organizational separation between the OSS Xen dev team and product teams. I don't know if that structure remains today, but it was once helpful in reducing conflict between public OSS and private product roadmaps.
The separation between server and client Xen product teams was less ideal, which eventually lead to OpenXT. Six years after v4v was posted to xen-devel, Xen Argo is the first step to possible reunification, a small chance at reversal, via public open-source, of architectural and resource fragmentation that took place privately.
Like QubesOS, OpenXT (and predecessor Citrix XenClient) development is spread across many open-source projects, including Xen, enabling user workflows that balance hardware-assisted security with usability. Spanning ecosystems, OpenXT is:
- unbundling OSS capabilities, e.g. TrenchBoot and coreboot for launch integrity
- moving code upstream (Argo, stubdom, blktap, Qemu, OpenEmbedded meta-virt)
- refactoring for peer & downstream derivatives, on client devices and beyond
To achieve this cross-community integration, we work with many stakeholders amid competing priorities for limited dev resources. It has taken six years to turn the ship from a Xen separation which began within one organization. This progress was accrued across multiple organizations and policies.
Argo was improved by the Xen upstreaming effort. Future Xen code contributions will benefit from the lessons learned.
Quoted 14 lines
Quoted 14 lines
Quoted 5 lines
There are comments from v5 that haven't been fixed in v7 (the mask usage and
list_first_entry_or_nullfor example) and the reply to the reviewer's comment was sent at the same time as v7, leaving no time for further discussion (and for reaching an agreement suitable to both parties) before sending v7.Code changes from our ongoing discussion will be addressed in v8. A proposal to address mask usage has been put forward in the parallel thread. Your proposed usage of
list_first_entry_or_nullwill be made in v8, subject to the previous offline discussion about list macros (duplicated here for convenience):Quoted 2 lines
Quoted 2 lines
As discussed offline:
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
- Using Xen's list macros will expedite Argo's merge for Xen 4.12
- List macros in Xenlist.horiginated in Linuxlist.hand have diverged
- OpenXT has use cases for measured launch and nested virtualization, which influence downstream performance and security requirements for Argo and XenFWIW, I don't see the connection between nested virtualization or measured launch and the list macros.
The issue most relevant to xen-devel is the divergence of list macros in Xen list.h from their origin in Linux list.h. Since this issue is independent of Argo, I've started a separate thread on "macro supply chains" [1].
Quoted 2 lines
I think a little bit more context
would be helpful here in order to understand the issue.
For more context on nested virtualization, see Ian Pratt's talk [2] on AX, uXen and nested Hyper-V. If a production system is designed to meet performance and security requirements that are delivered by multiple hypervisors, which could be open-source (e.g. Xen or uXen), proprietary (e.g. Hyper-V) or in firmware (e.g. AX on HP laptops), then a measured launch increases the level of assurance that the system is booted with validated hypervisors that can cooperate to meet those requirements.
For more context on measured launch, see the boot integrity talks from PSEC 2018 [3].
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
- OpenXT can temporarily patch Xen 4.12 for downstream use
Patching the macros for OpenXT is perfectly fine, but it would be better to understand and fix the problem upstream if possible.
How are you patching the macros?
What are you trying to achieve by patching them?
This will be determined during an upcoming OpenXT development, testing and certification cycle, when upstream Xen Argo is evaluated in the context of OpenXT and derivative use cases.
Rich
[1] Macro supply chains
https://lists.xen.org/archives/html/xen-devel/2019-02/msg00832.html
[2] "Hypervisor Security — Lessons Learned", Ian Pratt, 2018
https://www.platformsecuritysummit.com/2018/speaker/pratt/
[3] Boot Integrity presentations, 2018
https://www.platformsecuritysummit.com/2018/topic/boot/
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel