Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.4 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6656AC55179 for ; Wed, 4 Nov 2020 15:53:45 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0665222280 for ; Wed, 4 Nov 2020 15:53:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="hZ0dMu4m" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0665222280 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.19392.44550 (Exim 4.92) (envelope-from ) id 1kaL6J-0004MW-P6; Wed, 04 Nov 2020 15:53:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 19392.44550; Wed, 04 Nov 2020 15:53:31 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kaL6J-0004MP-MA; Wed, 04 Nov 2020 15:53:31 +0000 Received: by outflank-mailman (input) for mailman id 19392; Wed, 04 Nov 2020 15:53:30 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kaL6I-0004MK-SY for xen-devel@lists.xenproject.org; Wed, 04 Nov 2020 15:53:30 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 372b27a0-f417-48be-9aaf-1dfcd7aec012; Wed, 04 Nov 2020 15:53:29 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id C4F80AE21; Wed, 4 Nov 2020 15:53:28 +0000 (UTC) Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kaL6I-0004MK-SY for xen-devel@lists.xenproject.org; Wed, 04 Nov 2020 15:53:30 +0000 X-Inumbo-ID: 372b27a0-f417-48be-9aaf-1dfcd7aec012 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 372b27a0-f417-48be-9aaf-1dfcd7aec012; Wed, 04 Nov 2020 15:53:29 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1604505208; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lJFO2VI+vE9y/ADUibwFoqOaJoZzmYkwl15bMIxXOSo=; b=hZ0dMu4mkggltgqWTE6FmM4bHlY8JvoI6t/2VKyfRXwU5wGMIjYyHldT/thvUQuA4CNbz5 EcA0H+Ce++xDhVKAQ8RnLSn1pOEO05AH4SVSRpcgj/YnxFCa7AQwbK+oLDoeCDQroaSIdr P2TrjLj4TLUOYD+gh7JYwflXBNGEfXg= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id C4F80AE21; Wed, 4 Nov 2020 15:53:28 +0000 (UTC) Subject: Re: [PATCH v4.1 2/2] xen/evtchn: rework per event channel lock To: Jan Beulich Cc: Andrew Cooper , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= , Wei Liu , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , xen-devel@lists.xenproject.org References: <20201104115739.20144-1-jgross@suse.com> From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= Message-ID: Date: Wed, 4 Nov 2020 16:53:28 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit On 04.11.20 16:29, Jan Beulich wrote: >> @@ -738,7 +725,8 @@ int evtchn_send(struct domain *ld, unsigned int lport) >> >> lchn = evtchn_from_port(ld, lport); >> >> - spin_lock_irqsave(&lchn->lock, flags); >> + if ( !evtchn_read_trylock(lchn) ) >> + return 0; > > Isn't there a change in behavior here? While sends through > ECS_UNBOUND ports indeed get silently ignored, ECS_FREE ones ought > to be getting -EINVAL (as should ECS_UNBOUND ones if they're > Xen-consumer ones). With the failed trylock you don't know which > of the two the port is in the process of being transitioned > to/from. The same would apply for other operations distinguishing > the two states. Right now both evtchn_status() and > evtchn_bind_vcpu() only use the domain-wide lock, but the latter > is getting switched by "evtchn: convert domain event lock to an > r/w one" (granted there's an RFC remark there whether that > transformation is worthwhile). Anyway, the main point of my remark > is that there's another subtlety here which I don't think becomes > obvious from description or comments - where the two states are > mentioned, it gets to look as if they can be treated equally. Hmm, evtchn_send() seems to be called always with interrupts enabled. We could just use a standard read_lock() here if you think the different states should be treated as today. > >> --- a/xen/include/xen/event.h >> +++ b/xen/include/xen/event.h >> @@ -105,6 +105,39 @@ void notify_via_xen_event_channel(struct domain *ld, int lport); >> #define bucket_from_port(d, p) \ >> ((group_from_port(d, p))[((p) % EVTCHNS_PER_GROUP) / EVTCHNS_PER_BUCKET]) >> >> +/* >> + * Lock an event channel exclusively. This is allowed only when the channel is >> + * free or unbound either when taking or when releasing the lock, as any >> + * concurrent operation on the event channel using evtchn_read_trylock() will >> + * just assume the event channel is free or unbound at the moment when the >> + * evtchn_read_trylock() returns false. >> + */ >> +static inline void evtchn_write_lock(struct evtchn *evtchn) >> +{ >> + write_lock(&evtchn->lock); >> + >> + evtchn->old_state = evtchn->state; >> +} >> + >> +static inline void evtchn_write_unlock(struct evtchn *evtchn) >> +{ >> + /* Enforce lock discipline. */ >> + ASSERT(evtchn->old_state == ECS_FREE || evtchn->old_state == ECS_UNBOUND || >> + evtchn->state == ECS_FREE || evtchn->state == ECS_UNBOUND); >> + >> + write_unlock(&evtchn->lock); >> +} > > These two aren't needed outside of event_channel.c, are they? If > so, and if they ought to go in a header rather than directly into > the .c file (where I'd prefer the latter, for the sake of minimal > exposure), then it should be event_channel.h. I wanted to have the locking functions in one place. In case you prefer it otherwise (and you seem to do so) I'd rather move the write lock functions to the .c file. > >> @@ -114,6 +114,7 @@ struct evtchn >> u16 virq; /* state == ECS_VIRQ */ >> } u; >> u8 priority; >> + u8 old_state; /* State when taking lock in write mode. */ >> u32 fifo_lastq; /* Data for fifo events identifying last queue. */ >> #ifdef CONFIG_XSM >> union { > > While there's a gap here after the prior patch (which I'm still > not overly happy about), I'm still inclined to ask that the field > be put inside #ifndef NDEBUG, as its only consumer is an > ASSERT(). Fine with me Juergen