Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.4 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 694CEC00A89 for ; Thu, 5 Nov 2020 11:33:37 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D49D52078E for ; Thu, 5 Nov 2020 11:33:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="LYr36KQf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D49D52078E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.19733.45049 (Exim 4.92) (envelope-from ) id 1kadW4-0006rF-CP; Thu, 05 Nov 2020 11:33:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 19733.45049; Thu, 05 Nov 2020 11:33:20 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kadW4-0006r8-8v; Thu, 05 Nov 2020 11:33:20 +0000 Received: by outflank-mailman (input) for mailman id 19733; Thu, 05 Nov 2020 11:33:18 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kadW2-0006r3-2X for xen-devel@lists.xenproject.org; Thu, 05 Nov 2020 11:33:18 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id de39cebb-fcb7-42e9-bcfa-f79d3b787125; Thu, 05 Nov 2020 11:33:16 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 4FBF8ABE3; Thu, 5 Nov 2020 11:33:15 +0000 (UTC) Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kadW2-0006r3-2X for xen-devel@lists.xenproject.org; Thu, 05 Nov 2020 11:33:18 +0000 X-Inumbo-ID: de39cebb-fcb7-42e9-bcfa-f79d3b787125 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id de39cebb-fcb7-42e9-bcfa-f79d3b787125; Thu, 05 Nov 2020 11:33:16 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1604575995; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/FW7ZIc43wsFfXwFpsgIs8hKt6y6twweyPDlhTMzmFQ=; b=LYr36KQfi8F5T+nqzxBXyURwPl/cucLx3hiMle6zlOozjMIXE7Nwnj+nkjmRi9yaWF7tn8 0UCbdHjV6owhcC/KuzVnAdjbFOa3mRChj1X4geBKxXn9sxmtGJjMMqISs3isyquRh4ZVWP ym/azaLMrq2xbnI9bvBQ/PX0u125gk0= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 4FBF8ABE3; Thu, 5 Nov 2020 11:33:15 +0000 (UTC) Subject: Re: [PATCH v4.1 2/2] xen/evtchn: rework per event channel lock To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= Cc: Andrew Cooper , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= , Wei Liu , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , xen-devel@lists.xenproject.org References: <20201104115739.20144-1-jgross@suse.com> From: Jan Beulich Message-ID: Date: Thu, 5 Nov 2020 12:33:14 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit On 04.11.2020 16:53, Jürgen Groß wrote: > On 04.11.20 16:29, Jan Beulich wrote: >>> @@ -738,7 +725,8 @@ int evtchn_send(struct domain *ld, unsigned int lport) >>> >>> lchn = evtchn_from_port(ld, lport); >>> >>> - spin_lock_irqsave(&lchn->lock, flags); >>> + if ( !evtchn_read_trylock(lchn) ) >>> + return 0; >> >> Isn't there a change in behavior here? While sends through >> ECS_UNBOUND ports indeed get silently ignored, ECS_FREE ones ought >> to be getting -EINVAL (as should ECS_UNBOUND ones if they're >> Xen-consumer ones). With the failed trylock you don't know which >> of the two the port is in the process of being transitioned >> to/from. The same would apply for other operations distinguishing >> the two states. Right now both evtchn_status() and >> evtchn_bind_vcpu() only use the domain-wide lock, but the latter >> is getting switched by "evtchn: convert domain event lock to an >> r/w one" (granted there's an RFC remark there whether that >> transformation is worthwhile). Anyway, the main point of my remark >> is that there's another subtlety here which I don't think becomes >> obvious from description or comments - where the two states are >> mentioned, it gets to look as if they can be treated equally. > > Hmm, evtchn_send() seems to be called always with interrupts enabled. > We could just use a standard read_lock() here if you think the different > states should be treated as today. This would avoid the caveat in this specific case, but it would remain elsewhere (at least as an abstract trap to fall into). I suppose evtchn_status() could use a regular read_lock() too, for the same reason (if it was to be switched), and evtchn_bind_vcpu() may need a write lock anyway (which is forbidden in your model, i.e. I'd likely need to drop the switch to the finer grained lock there). >>> --- a/xen/include/xen/event.h >>> +++ b/xen/include/xen/event.h >>> @@ -105,6 +105,39 @@ void notify_via_xen_event_channel(struct domain *ld, int lport); >>> #define bucket_from_port(d, p) \ >>> ((group_from_port(d, p))[((p) % EVTCHNS_PER_GROUP) / EVTCHNS_PER_BUCKET]) >>> >>> +/* >>> + * Lock an event channel exclusively. This is allowed only when the channel is >>> + * free or unbound either when taking or when releasing the lock, as any >>> + * concurrent operation on the event channel using evtchn_read_trylock() will >>> + * just assume the event channel is free or unbound at the moment when the >>> + * evtchn_read_trylock() returns false. >>> + */ >>> +static inline void evtchn_write_lock(struct evtchn *evtchn) >>> +{ >>> + write_lock(&evtchn->lock); >>> + >>> + evtchn->old_state = evtchn->state; >>> +} >>> + >>> +static inline void evtchn_write_unlock(struct evtchn *evtchn) >>> +{ >>> + /* Enforce lock discipline. */ >>> + ASSERT(evtchn->old_state == ECS_FREE || evtchn->old_state == ECS_UNBOUND || >>> + evtchn->state == ECS_FREE || evtchn->state == ECS_UNBOUND); >>> + >>> + write_unlock(&evtchn->lock); >>> +} >> >> These two aren't needed outside of event_channel.c, are they? If >> so, and if they ought to go in a header rather than directly into >> the .c file (where I'd prefer the latter, for the sake of minimal >> exposure), then it should be event_channel.h. > > I wanted to have the locking functions in one place. > > In case you prefer it otherwise (and you seem to do so) I'd rather move > the write lock functions to the .c file. Yes please. Jan