Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 221D8CD6E6B for ; Thu, 4 Jun 2026 11:48:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wV6YT-0006z6-K3; Thu, 04 Jun 2026 07:48:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wV6YI-0006w9-0T for qemu-arm@nongnu.org; Thu, 04 Jun 2026 07:48:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wV6YE-0005pc-JY for qemu-arm@nongnu.org; Thu, 04 Jun 2026 07:47:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780573673; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Bfd+9ky4ls39Il5SYfH5+JPixa/D1xbdBk7i9KyEp0=; b=P/b5S6D5EdmIehlhX+mdNaIZwrViCmeQwhF3ad9Mwosu0y3zDjwItPBRPmbqY+bhmj07sT J8al5pttD9q7qow8gKL0rtAXh3E7PhqPgIPNNKZ1FYEkVSniOBf2LmeKesUy4hJdvRLwNM Pi5TwVEKu57cYpwYXbFBLywsX3y4pZI= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-297-F05QoGgzOWOsv4eoCbR49Q-1; Thu, 04 Jun 2026 07:47:52 -0400 X-MC-Unique: F05QoGgzOWOsv4eoCbR49Q-1 X-Mimecast-MFC-AGG-ID: F05QoGgzOWOsv4eoCbR49Q_1780573671 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-490a7aeaddbso4822055e9.1 for ; Thu, 04 Jun 2026 04:47:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780573671; x=1781178471; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:reply-to:user-agent:mime-version:date :message-id:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=4Bfd+9ky4ls39Il5SYfH5+JPixa/D1xbdBk7i9KyEp0=; b=djPf5vHlYVOV/49TOeFZtlaq9bDuPWKuMFJcYqWv4Xn952JNlMe0GwQ2iQEdnhyWlW /ct1N5HFmwm8rKSYQlPpn7c2xnII4L3Je4k0EJL00pE6tXTh6850QU34Gg5rb9NIkOfl yIAikq0riqPcLqDG0OzlUHR97J6Dt4GUYdBXMBnYF7w4Hp4dhTIz4VyeJI5BpBhziKRZ frsYvrI6eH0ouxRSaO6DubVYbV7lUkRcM3thcTgJeLK28NO4V81x4ENWhbIn/aCUkvtZ KOB/ERdxtymp1EqgDHtm6F/q+Yyr0GVsXnlnaR38z9wY+5AYKkXT6YubOlJtGctuBw/g 0uSg== X-Forwarded-Encrypted: i=1; AFNElJ8YboFP6N2Ew9Dt+4XwSw2wdf2Cns+S44444UIl1Co/5LIbQAwSm0q/rMfgBEZ9BRyR8vawFLbmcA==@nongnu.org X-Gm-Message-State: AOJu0Yx0gWxhPHtF3CdFLl2OjsSUGaV0U3WjY/tBvfLUiIUjVvCGMFA9 uXVmKESFFpizkW7mP/2LS0m8K2Bv6uuod7NhAexAnJ88M3uxY/l1XhWA8q6+K/8tbSJW8Mqvb+h vn6+Rc4KcN4ZEniDlCKdEHNZiOYr1GnjEgxcYcxYRIHFjWZpCPPDSqQ== X-Gm-Gg: Acq92OEAtGsG0kteUie19Sor/j70DVCvtn+C5M62YXx4PO1tAadXop2EJ9Ij/MsS9BN 5vv2Da/cVhKTWnj443UDr+9Q4SCdjDfPFU7vDdC0fJBC7X4hWfQzUQDGa1oSRTieswpAIckkcl7 TlOYdpso8FYCjenX3TCL6y5OS+16ra/n4HhcpABKuauzbbS9TFTX7ioY0ivNDnssfLNsLcsnHNi AsdKISnFU1McwqRJIw3hhlt/bFxyoF2DcBLPcMM04wxaJexTjYbVF+2Ss5n0dLFtBLyTwtyyTub 8pRK0tZeB5IzLZ4I0lxVTaZ8/jTJZ/TA4qnoWhcU0rz07WmKt9jEairh1MN3z9+hmPUUoLAqe68 5rR/vyJcwDfMhfvVj3hzl5rZ7oJmvv43MvhIPYHlrTGylMVh8sTFFtkr7+M753bJUEKhe3aeXDs jbVpXO X-Received: by 2002:a05:600c:5488:b0:489:32b:ac0b with SMTP id 5b1f17b1804b1-490bc4d2697mr55346135e9.6.1780573670849; Thu, 04 Jun 2026 04:47:50 -0700 (PDT) X-Received: by 2002:a05:600c:5488:b0:489:32b:ac0b with SMTP id 5b1f17b1804b1-490bc4d2697mr55345705e9.6.1780573670451; Thu, 04 Jun 2026 04:47:50 -0700 (PDT) Received: from ?IPV6:2a01:e0a:f0e:9070:527b:9dff:feef:3874? ([2a01:e0a:f0e:9070:527b:9dff:feef:3874]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bd670c2esm51993945e9.0.2026.06.04.04.47.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 04 Jun 2026 04:47:49 -0700 (PDT) Message-ID: <39ee654d-1dbb-49f5-b780-bda2c7586e8e@redhat.com> Date: Thu, 4 Jun 2026 13:47:46 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v6 19/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing To: Shameer Kolothum Thodi , "qemu-arm@nongnu.org" , "qemu-devel@nongnu.org" Cc: "peter.maydell@linaro.org" , "clg@redhat.com" , "alex@shazbot.org" , Nicolin Chen , Nathan Chen , Matt Ochs , Jiandi An , Jason Gunthorpe , "zhenzhong.duan@intel.com" , Krishnakant Jaju , "phrdina@redhat.com" References: <20260601114221.457995-1-skolothumtho@nvidia.com> <20260601114221.457995-20-skolothumtho@nvidia.com> From: Eric Auger In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Tp6mXaTyV7W8dAkIW3BACOAib8nI_p3Cenv4hPw5WYU_1780573671 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=170.10.129.124; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.445, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-arm@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: eric.auger@redhat.com Errors-To: qemu-arm-bounces+qemu-arm=archiver.kernel.org@nongnu.org Sender: qemu-arm-bounces+qemu-arm=archiver.kernel.org@nongnu.org On 6/4/26 1:22 PM, Shameer Kolothum Thodi wrote: > >> -----Original Message----- >> From: Eric Auger >> Sent: 04 June 2026 10:56 >> To: Shameer Kolothum Thodi ; qemu- >> arm@nongnu.org; qemu-devel@nongnu.org >> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin >> Chen ; Nathan Chen ; Matt >> Ochs ; Jiandi An ; Jason Gunthorpe >> ; zhenzhong.duan@intel.com; Krishnakant Jaju >> ; phrdina@redhat.com >> Subject: Re: [PATCH v6 19/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF >> page0 as VCMDQ backing >> >> External email: Use caution opening links or attachments >> >> >> Hi Shameer, >> >> On 6/1/26 1:42 PM, Shameer Kolothum wrote: >>> hw/arm/tegra241-cmdqv: Use mmap'd VINTF Page 0 as VCMDQ backing >> the title is double here. > Ah..Ok. > >> By the way may I suggest >> hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 for mapped vcmdqs >> direct view >> >> to emphasize this relates the direct view and only for mapped vcmdqs > It's true this eventually relates only to the direct aperture, but not yet. > VINTF page0 only gets mapped into the guest in patch 21. Until then, > for allocated vCMDQs we use the mmap'd VINTF page0 for both direct > and VINTF aperture accesses. > > May be we can rename to: > Route allocated VCMDQ Page0 accesses to the mmap'd VINTF page0 Indeed. Yes above makes sense. It is about allocated VCMDQs only > >>> Introduce tegra241_cmdqv_vintf_ptr() to route VCMDQ Page 0 register >>> accesses through the mmap'd VINTF Page 0 backing once a hardware >>> queue has been allocated for the VCMDQ. >>> >>> The two QEMU-trapped Page 0 apertures (direct at 0x10000, VINTF at >>> 0x30000) are hardware aliases of the same underlying registers. A >>> subsequent patch installs the VINTF aperture as a RAM-device into >>> guest MMIO; in this patch both remain QEMU-trapped. >>> >>> The direct VCMDQ aperture stays QEMU-trapped (rather than aliased >>> to the VINTF mmap) so that writes to an unallocated VCMDQ remain >>> well-defined. The CMDQV architecture allows software to program a >>> VCMDQ through the direct aperture without first allocating it to a >>> VINTF; aliasing to the VINTF mmap would route those writes into >>> unallocated logical slots where the hardware silently drops them. >>> >>> A VCMDQ Page 0 access is served from one of two sources: >>> >>> - Cache-backed: no hw_queue is allocated for the VCMDQ >>> (HW_QUEUE_ALLOC has not yet succeeded). Both apertures use >>> QEMU's register cache. >>> >>> - HW-backed: HW_QUEUE_ALLOC has succeeded. Both apertures access >>> the registers directly through the mmap'd VINTF Page 0. >>> >>> tegra241_cmdqv_sync_vcmdq() copies any cached writes (CONS_INDX, >>> PROD_INDX, CONFIG, GERRORN) into the mmap'd page on the cache-to-HW >>> transition so the guest's earlier register state survives. >>> >>> Signed-off-by: Shameer Kolothum >>> --- >>> hw/arm/tegra241-cmdqv.c | 73 >> +++++++++++++++++++++++++++++++++++++++++ >>> 1 file changed, 73 insertions(+) >>> >>> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c >>> index 54d95f4e4e..dce7089697 100644 >>> --- a/hw/arm/tegra241-cmdqv.c >>> +++ b/hw/arm/tegra241-cmdqv.c >>> @@ -39,6 +39,47 @@ static void >> tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv) >>> } >>> } >>> >>> +/* >>> + * Return a pointer into the mmap'd VINTF page0 for the VCMDQ Page 0 >>> + * register at @offset0 in VCMDQ slot @index, or NULL when the VCMDQ >>> + * has no hw_queue allocated or the VINTF page0 is not mmap'd. >>> + */ >>> +static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV >> *cmdqv, >>> + int index, hwaddr offset0) >>> +{ >>> + if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) { >>> + return NULL; >>> + } >>> + return (uint32_t *)(cmdqv->vintf_page0 + >>> + (index * CMDQV_VCMDQ_STRIDE) + >>> + (offset0 - CMDQV_VCMDQ_PAGE0_BASE)); >>> +} >>> + >>> +/* >>> + * Flush cached register writes into the mmap'd VINTF page0 after a >>> + * successful HW_QUEUE_ALLOC, so the guest's earlier writes survive >>> + * the cache-to-hardware transition. >>> + */ >>> +static void tegra241_cmdqv_sync_vcmdq(Tegra241CMDQV *cmdqv, int >> index) >>> +{ >>> + uint32_t *ptr; >>> + >>> + ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, >> A_VCMDQ0_CONS_INDX); >>> + if (!ptr) { >>> + return; >>> + } >>> + *ptr = cmdqv->vcmdq_cons_indx[index]; >>> + >>> + ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, >> A_VCMDQ0_PROD_INDX); >>> + *ptr = cmdqv->vcmdq_prod_indx[index]; >>> + >>> + ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, A_VCMDQ0_CONFIG); >>> + *ptr = cmdqv->vcmdq_config[index]; >>> + >>> + ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, A_VCMDQ0_GERRORN); >>> + *ptr = cmdqv->vcmdq_gerrorn[index]; >>> +} >>> + >>> /* >>> * Allocate a host HW VCMDQ from the current cached BASE / size for >> @index. >>> * >>> @@ -95,6 +136,9 @@ static bool >> tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, >>> cmdqv->vcmdq_gerror[index] &= >> ~R_VCMDQ0_GERROR_CMDQ_INIT_ERR_MASK; >>> cmdqv->vcmdq_status[index] |= >> R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK; >>> + /* Pre-alloc cached writes survive the cache-to-hardware transition. */ >> what about the opposite transition? > Yeah. We don’t do anything now. I think better to reset the cache on > opposite transition. OK. I would add this explanation. > >>> + tegra241_cmdqv_sync_vcmdq(cmdqv, index); >>> + >>> return true; >>> } >>> >>> @@ -113,13 +157,22 @@ static void >> tegra241_cmdqv_setup_all_vcmdq(Tegra241CMDQV *cmdqv, >>> * >>> * The caller normalizes the MMIO offset such that @offset0 always refers >>> * to a VCMDQ0_* register, while @index selects the VCMDQ instance. >>> + * >>> + * If the VCMDQ is allocated and VINTF page0 is mmap'd, read directly >>> + * from the VINTF page0 backing. Otherwise, fall back to the cache. >>> */ >>> static uint64_t tegra241_cmdqv_read_vcmdq_page0(Tegra241CMDQV >> *cmdqv, >>> hwaddr offset0, int index, >>> bool direct) >>> { >>> + uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0); >>> uint64_t val = 0; >>> >>> + if (ptr) { >> shouldn't you test "direct" as well? I know that later on this is will >> be backed by another mr but I think this would be clearer. > Is the suggestion here to just limit this to "direct"? We can't do that. > Both apertures alias the same registers, and for an allocated and > mmapped VINTF page0 we access the hw directly. > > The vintf page0 gets mapped to the guest only in patch 21, so > after that only the direct aperture is trapped here anyway. Yes I agree this would be true only once patch 21 is applied. Sorry for the noise. > >>> + val = *ptr; >>> + goto out; >>> + } >>> + >>> switch (offset0) { >>> case A_VCMDQ0_CONS_INDX: >>> val = cmdqv->vcmdq_cons_indx[index]; >>> @@ -144,6 +197,7 @@ static uint64_t >> tegra241_cmdqv_read_vcmdq_page0(Tegra241CMDQV *cmdqv, >>> "%s unhandled read access at 0x%" PRIx64 "\n", >>> __func__, offset0); >>> } >>> +out: >>> trace_tegra241_cmdqv_read_vcmdq_page0(index, direct ? "direct" : "vi", >>> offset0, val); >>> return val; >>> @@ -220,11 +274,29 @@ static uint64_t >> tegra241_cmdqv_config_vintf_read(Tegra241CMDQV *cmdqv, >>> * >>> * Page 0 registers are all 32-bit; this helper is only called for 4-byte >>> * writes. >>> + * >>> + * If the VCMDQ is allocated and VINTF page0 is mmap'd, write directly >>> + * to the VINTF page0 backing. Otherwise, update the cache. >>> */ >>> static void tegra241_cmdqv_write_vcmdq_page0(Tegra241CMDQV >> *cmdqv, >>> hwaddr offset0, int index, >>> uint32_t value, bool direct) >>> { >>> + uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0); >> same here? >>> + >>> + if (ptr) { >>> + switch (offset0) { >>> + case A_VCMDQ0_CONS_INDX: >>> + case A_VCMDQ0_PROD_INDX: >>> + case A_VCMDQ0_CONFIG: >>> + case A_VCMDQ0_GERRORN: >>> + *ptr = value; >>> + goto out; >>> + default: >>> + break; >>> + } >>> + } >>> + >>> switch (offset0) { >>> case A_VCMDQ0_CONS_INDX: >>> cmdqv->vcmdq_cons_indx[index] = value; >>> @@ -255,6 +327,7 @@ static void >> tegra241_cmdqv_write_vcmdq_page0(Tegra241CMDQV *cmdqv, >>> "%s unhandled write access at 0x%" PRIx64 "\n", >>> __func__, offset0); >>> } >>> +out: >>> trace_tegra241_cmdqv_write_vcmdq_page0(index, direct ? "direct" : "vi", >> by the way could be interesting to log in the trace event that mmapped >> was used instead of cache regs. > Could do. > > Thanks, > Shameer Thanks Eric