Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71BA3C4727C for ; Fri, 25 Sep 2020 14:57:53 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1B9E520715 for ; Fri, 25 Sep 2020 14:57:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="YglL60LC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1B9E520715 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kLpAC-0007gd-4t; Fri, 25 Sep 2020 14:57:32 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kLpAB-0007gY-E2 for xen-devel@lists.xenproject.org; Fri, 25 Sep 2020 14:57:31 +0000 X-Inumbo-ID: 7eca7edf-2f80-4a27-8b58-3ead1932f0a0 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 7eca7edf-2f80-4a27-8b58-3ead1932f0a0; Fri, 25 Sep 2020 14:57:30 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1601045849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xq3i4wvKSwhLGlIBMLf6cQJPpcF+QxhDY1/k2FgR0BI=; b=YglL60LCds7Lokaou17kAFJDlkaDQnCWwuI5gr4P+qlduV3X5aDW3f3FyUtIjm1nGB9z/1 N53Xc0GzlDoTog7R0HTeyZVpuM7iBLma/+YLaEpveTcLPY4n0TL/IQIq2xhO69mdepCJeV j1FpBklzIwkqYWpfeU027l7elZnDj8I= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 92A04B011; Fri, 25 Sep 2020 14:57:29 +0000 (UTC) Subject: Re: [PATCH] evtchn/Flask: pre-allocate node on send path To: Jan Beulich , Julien Grall Cc: "xen-devel@lists.xenproject.org" , Daniel de Graaf , Andrew Cooper , George Dunlap , Ian Jackson , Wei Liu , Stefano Stabellini References: <8237e286-168f-a4e7-be8b-aba5ff781e7c@xen.org> <706b94ae-ca05-2218-6025-e5d62297dda6@suse.com> From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= Message-ID: <4390eb35-768e-1ca1-099e-da33da9f939e@suse.com> Date: Fri, 25 Sep 2020 16:57:28 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <706b94ae-ca05-2218-6025-e5d62297dda6@suse.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" On 25.09.20 14:21, Jan Beulich wrote: > On 25.09.2020 12:34, Julien Grall wrote: >> On 24/09/2020 11:53, Jan Beulich wrote: >>> xmalloc() & Co may not be called with IRQs off, or else check_lock() >>> will have its assertion trigger about locks getting acquired >>> inconsistently. Re-arranging the locking in evtchn_send() doesn't seem >>> very reasonable, especially since the per-channel lock was introduced to >>> avoid acquiring the per-domain event lock on the send paths. Issue a >>> second call to xsm_evtchn_send() instead, before acquiring the lock, to >>> give XSM / Flask a chance to pre-allocate whatever it may need. >> >> This is the sort of fall-out I was expecting when we decide to turn off >> the interrupts for big chunk of code. I couldn't find any at the time >> though... >> >> Can you remind which caller of send_guest{global, vcpu}_virq() will call >> them with interrupts off? > > I don't recall which one of the two it was that I hit; we wanted > both to use the lock anyway. send_guest_pirq() very clearly also > gets called with IRQs off. > >> Would it be possible to consider deferring the call to a softirq >> taslket? If so, this would allow us to turn the interrupts again. > > Of course this is in principle possible; the question is how > involved this is going to get. However, on x86 oprofile's call to > send_guest_vcpu_virq() can't easily be replaced - it's dangerous > enough already that in involves locks in NMI context. I don't > fancy seeing it use more commonly used ones. Is it really so hard to avoid calling send_guest_vcpu_virq() in NMI context? Today it is called only if the NMI happened inside the guest, so the main Xen stack is unused at this time. It should be rather straight forward to mimic a stack frame on the main stack and iret to a special handler from NMI context. This handler would then call send_guest_vcpu_virq() and return to the guest. This would basically be similar to the Linux kernel's __run_on_irqstack(). Juergen