Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A3A80C001DF for ; Tue, 8 Aug 2023 12:51:33 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id CDB658634D; Tue, 8 Aug 2023 14:51:10 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="kIfQhL8A"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 74FDB847BC; Tue, 8 Aug 2023 10:48:16 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 69C2C8073D for ; Tue, 8 Aug 2023 10:48:13 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=fei2.wu@intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691484493; x=1723020493; h=message-id:date:subject:from:to:references:in-reply-to: content-transfer-encoding:mime-version; bh=HuyUOYgNjSq7DsbtCLOe6+nE/84oYrs9xIrlfU4081Y=; b=kIfQhL8AvH/dFVe2yTsODwwHJr4y2Mdf8sRmbsPj99nbsjyXHhBEDw4E Ots3ie29eAt4xrKZOJMSgdmN9DumOwTpknVijjS50umKzYGVQ4D9mDI+E 60hzt8BIEUczpdnl+kVhM8RXOi8dX1OuklDzspfK4v+XRe1Xq6IwLm8T1 Rj7FzrtbuNHAS24+g281XCktYKAyXlA4zhcsnClSj9vQfgHi1++RJbo58 FiMrRSltwYs5vU3n0oaXN9PCwMxB24cTLnuXgqxN04Ai1+KNGTB/b451w r96L9tkWRLNS6KUtgziMVqQBafBzV9ffCbkk2uQd4qhvKAt4Zuoji32DS w==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="351064219" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="351064219" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 01:48:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="766298156" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="766298156" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by orsmga001.jf.intel.com with ESMTP; 08 Aug 2023 01:48:09 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 8 Aug 2023 01:48:09 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 8 Aug 2023 01:48:08 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27 via Frontend Transport; Tue, 8 Aug 2023 01:48:08 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.172) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.27; Tue, 8 Aug 2023 01:48:08 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=A9SiNBsOpe9fGu27WtYqGEf+DjwUHV7g+TAFMwBn4os3bfSjKWVez5gRJPNB4tANdCjO8y6501xY01442rbV7fVhJp0dp8kFlYL1drAksoFzIAmKbIxTGh9rT5dHW30un3mZI2pHhXePrSqORiy0HaXgOwmyjL3rHszMibAmPilhZrqOcSyRe02htwbydh79WMxOvzwff6WEoCVC+vX2/1GqmZE1mlSA0rTqLFlbKyTr1j+pmV4BU4DV1Qf/tfNfP0jgKg0JstVTZIofP+ah1znh1vawaLWtj3re4hvqjVKonZXUdf4+dT26/QZXhIGadEgjCIXp3Fwg7/w3eu/Mjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oFVTvrv3OLynahfrNSq7QMFxhH455poBS2YoK+mBPKE=; b=hB8ctoQWQSPAQhygKS1cUTSobLGW9l1+0oulPBS1EMWm81JXZEGVtkfvwMDqVrAptGuIcp4bCV8FEKv9vEB+vcU4Ji/D+Wc/AjiS7m6PE154xcDXxxurxRFLibJmFbXF2+7hlurHk53Lajuqs2+WjPAGNt7V+OZdZqk3rgBQbCG0kvVTlcHM3zFCFBIiUjKCJYEd/vnqUCBWGpuBarNK0XgguQNBx2+9BOqJ1ytVgn+UylIbGZFN2WEZI/9Rv3bZskSB9i9VuUfnL/rmj2UZvMC9zz9GHmQNsb/Xo35oxTI1QhcMdCv27siKdj3iNJF7hJom9PYjODofU6KlwobFWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from BY5PR11MB4500.namprd11.prod.outlook.com (2603:10b6:a03:1c3::24) by SN7PR11MB7137.namprd11.prod.outlook.com (2603:10b6:806:2a0::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Tue, 8 Aug 2023 08:48:07 +0000 Received: from BY5PR11MB4500.namprd11.prod.outlook.com ([fe80::f5f4:f0a:d677:8017]) by BY5PR11MB4500.namprd11.prod.outlook.com ([fe80::f5f4:f0a:d677:8017%4]) with mapi id 15.20.6652.026; Tue, 8 Aug 2023 08:48:07 +0000 Message-ID: Date: Tue, 8 Aug 2023 16:47:57 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0 Thunderbird/102.14.0 Subject: Re: memory hole close to 4GB on qemu risc-v Content-Language: en-US From: "Wu, Fei" To: , , , References: <2b94f5bf-a79e-3dd1-c36a-54cd8f8bd478@intel.com> In-Reply-To: <2b94f5bf-a79e-3dd1-c36a-54cd8f8bd478@intel.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-ClientProxiedBy: SG2P153CA0049.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c6::18) To BY5PR11MB4500.namprd11.prod.outlook.com (2603:10b6:a03:1c3::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR11MB4500:EE_|SN7PR11MB7137:EE_ X-MS-Office365-Filtering-Correlation-Id: af1362fc-a27d-4c94-edf4-08db97ec2f58 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Fby80S+F+FE0Coill2fqrmmgO7PGQ6N9GqXEcYEEOR7JgdBpja6AcmF2EX8cUuJsB4UlJz5/YtgzJR1gJJBjk3yvaNgC4pDpMuGB1rEnctNL4WoNLtE4IHyfbT335KdWnfKVztujmVXt84If8Qa9IiQEkO00Q68yOZVJkxdPCm0Ib73GURQvbMz+kYZwBl5TFvHl9Sof4RGxex3/X3Vfhrm+vIgDqyJwALH2+xGL16QD2unJPSp90+srsovju216njUVK7g3H/yiHRE1Nlnt9+4Z0fiCg1TFyP5Mg6trjGkkqsR9FbTVKthNWAvzq3k8WHXX9OeyoT8omZzKabtQaOamwtOINeXEYeLnoGK6pPu1IXANhrYVLKGkzLYMIaUcZfdBGe11FaROk6IMMT2aAsGNXhsgdkvD8BrwfUjCyeVZ7ghU3Dln3BNf8/AAIgvx/aPuaxkyuraYxWFbfbp3uUQIOs8WOqjotNya/IgoEAqFGKM66Lnhkh3ahX16qJQGRPnv9Vsk/nVTxg6dIqyEDvFx9CW9Xd9duv4Tl54pjygvibQU7nz0Kg5Fv4xGPYg4/Vy8QcWtLj8ZireL91/tvpsc6+UXsJcxwA5ukxkEAROpW648eXK8actcHM490CJcBJbsuzSOgSNqpK3MtlJH9w== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BY5PR11MB4500.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(366004)(39860400002)(376002)(346002)(396003)(136003)(186006)(451199021)(1800799003)(83380400001)(53546011)(8676002)(26005)(6506007)(316002)(66556008)(2906002)(66476007)(5660300002)(66946007)(8936002)(6666004)(6486002)(41300700001)(2616005)(478600001)(6512007)(966005)(38100700002)(82960400001)(36756003)(31696002)(86362001)(31686004)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TUt6SndOdm12VnVicm9ZQ0w5aWQ0YVVKdTlDZExINUdTVmpYcmlqOHhGcTlS?= =?utf-8?B?WnMySjRhUDlSTDNqRFBGOG9XNVQvRUR5TUFMaHJ3Yi9Ta1pmdWNFc2tHOFQy?= =?utf-8?B?TVpTQUpvZTV6NlpKKzR2anIrMmNxaFZkNmhxV2J5MlpMdFpsL1pmdFFzODJy?= =?utf-8?B?eW9RQ3JoZGRjMEg3NDlaK3RacVZVUUNtTXVXOU1aNnBPUWgxZTVkRFBNK1Bu?= =?utf-8?B?WlovaXdPMG5OeDE1WlpVcWtwWmJhMFZMT0NRNXQvNnFRbzdpNUdKS29zZXJw?= =?utf-8?B?WXBieDhDbEhzdTNtVGJxSjNHRFFuVXhkRFZ4S3lCMEtkYWlkMDdOeTRxYTRX?= =?utf-8?B?SnB5Q0ZpU0xKSmdwdTBwNlgvRGpEQTdYMllYaUpNcElJNWZoWUlSRm5MSVFZ?= =?utf-8?B?eDFSQXNpTm0wckFxczM4Z2h4VjNnY05icGZFVnkyVkFyS0FBN3FEdkI0alhH?= =?utf-8?B?aFlEUTM2RUYrL3Q4Szk0V0lwM2lUUzV1aW15U0JhOU50ZUtDVW9xSERHZDNU?= =?utf-8?B?bU8wWFA4bktRc0paekZCUWlnd0xpYWRvdGNmOHN3KzNDcWNYT3pJT1p3TG1E?= =?utf-8?B?d3RFZTFRRGFJK0YrNU0yRm1nNXRUM09RZ2Jab0htNC9ZRnR6OWZQOHY4QkFs?= =?utf-8?B?M2thUzFaU1BCTk50QUQ3Y3FyUFVUeHFRK2ZnUXdyMWVWYTM0d0gxSFViL0pr?= =?utf-8?B?WjFUMHE3Qmg2WXgvaWt3Y29QSm02aUpZcUx3eU9TT1ArZCtrZkhLT21OYld0?= =?utf-8?B?Qjh5YkppUEpuRG04bHNGWjBHMEIxMDBGM3ZLcjNFY2lUdXVaaVJ3QS84dnI0?= =?utf-8?B?WGlaN0lRVVpIdUVRZVRaU0J4ckRHOGhTTzhpdkpXK090UE5pU1JiTFNkdEx5?= =?utf-8?B?azhWaXJwb2xIRGdrQnVGUnduUVB6YXo2M0tzdzM0N1dBQ0kveUlscTE0dXhF?= =?utf-8?B?UVBlSHo1RDcvQkszZXp3cThYaFdMeGFrQkc4Q2hlZ3dDNUJncExwQ2oyMWFW?= =?utf-8?B?TzFTQ1lVVEdDYWVNMjNNOHhCSUkyVVl3MlUwdGcwU3Q1NzVOOEdEL3BlM0Yr?= =?utf-8?B?NUZuNVFLdW0wRllLU3ZwWWhtMUtqbktrTUdJS2YvMkJMTjdOTlF3eDhYVHBq?= =?utf-8?B?dDRpZnZVSUV4akw0dXdLYkVPQkJKVTR0andOUzg1eXEwQzJwUmwyK1RYV1NK?= =?utf-8?B?MjZOdnVWQUJGdmtReGlvRjlmWHlBczVlNkJGR1UxZVAzOElxdC9oRHZvTEVT?= =?utf-8?B?bWpPa2FPMXgreDluQkFzQTBRVmhiaXdCMU5UVW5KdjdIL0R0OEVjbm1BUUdo?= =?utf-8?B?Sk1oWXVkWnRaREtzK1BVcW9EV1U3RHhCMm9OQWpCMUVmQ1RDbGlOTVBEVzAw?= =?utf-8?B?bzA1T0dUVnNsTk1qd0VHOUlSMmlKZDhadFZPR2xudWV6LzBVWlZYQVI5anNB?= =?utf-8?B?ZzYwRHRwcHNENm4yN1lTZWVURG5NZ3hmQ2U4T3JHWFhhR0hGa05QVGhwWnhJ?= =?utf-8?B?eFJvaENVNmo2K2NvU1VtNWdIbTBtZUNSN09Tdm9ZVnM1UUxoNE5QMWh6aFdu?= =?utf-8?B?OEliS1VmNEtCbm5xcnJRNUUvakIva3RGeHhmZmdvcFpJMkUrY2tJaHltYUVi?= =?utf-8?B?d3NEZjNyOHJRS0dLcFVIc3Z6MnY4U1d6UjVOVUFxbTYvMG83bG5abHplc1JL?= =?utf-8?B?R2wyWkhHRkVSbFZ5SUkyWGYwa0pZbW5EdFdVUVJZVWMxYnZMaW1UM0lRWTNi?= =?utf-8?B?U2Q3L3MwWTdmY0dqNVU5TUczMTNmSHhMOU1uRHFua3NGZ0xkckZQUzV2MWty?= =?utf-8?B?R2hqWGRtNjZDTFNrVGNKeVN6WFhvODZDbWpWMnhhaW1tWWFsVFA3Z0I0MFR1?= =?utf-8?B?NG5GTU9TQmdiaDVIRjgxSU5sZTFPcHJWVG9vZGRxT0laUEFBczBoL3dkQ3Jn?= =?utf-8?B?Y0lobXUzb0NFTTJESWU5UE5KcEhLQmtLY0p5eDFYZk9kK040L29XdVhSNUIr?= =?utf-8?B?WnI0cktmVzQ2dzRVZUFYTUxyQmJZRTN6VFB1cDVySFlrQW1EQ1lPcllnOFdH?= =?utf-8?B?WUJ3N09LUzNqQmNxMElLRlVtYmpqYWJENXByWVkrMjFva0RTYkJudnVpbDBY?= =?utf-8?Q?DFmPKTSJCOCKiAJHZBb3mnBfe?= X-MS-Exchange-CrossTenant-Network-Message-Id: af1362fc-a27d-4c94-edf4-08db97ec2f58 X-MS-Exchange-CrossTenant-AuthSource: BY5PR11MB4500.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2023 08:48:06.9817 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hE0XHvlXeII5I3K18e7Ef5uLLf1KTW18sWf9+OkW/78BAp6E6F7ua/Zx+RFhS6Ayn19MX+HzLww1lxtvKFMjaA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB7137 X-OriginatorOrg: intel.com X-Mailman-Approved-At: Tue, 08 Aug 2023 14:51:08 +0200 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean On 8/7/2023 3:50 PM, Wu, Fei wrote: > Hi All, > > I am working on enabling PCIe passthrough on qemu risc-v, in order for > the guest to access the host x86 pci resource directly, ram on guest > won't cover this range, so if guest has 4GB ram, two ranges are created: > 1. 2G-3G > 2. 4G-7G > > u-boot is not able to handle this but reports: > > Reserving 869k for U-Boot at: ff725000 > initcall: 0000000080216f94 > Reserving 8320k for malloc() at: fef05000 > initcall: 0000000080216ee8 > Unhandled exception: Store/AMO access fault > > EP: 0000000080201a38 RA: 0000000080216f10 TVAL: 00000000fef04f90 > > It looks like fdtdec_setup_mem_size_base() in u-boot sets ram_base/size > without considering all memory nodes, and board_get_usable_ram_top() > returns 4G-1 that causes gd->relocaddr set to the wrong address which > has no ram backed. More details can be found here: > https://patchew.org/QEMU/20230731015317.1026996-1-fei2.wu@intel.com/ > > > Can we add support for this multi-range ram setting? Any comments? > The test command is: qemu-system-riscv64 -machine virt -nographic -m 4G -smp 4 \ -bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf \ -kernel /home/wufei/nvme/u-boot/u-boot.bin \ -device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \ -drive file=ubuntu-22.04.1-preinstalled-server-riscv64+unmatched.img,if=virtio,format=raw Here is a quick fix, is it good? diff --git a/arch/riscv/cpu/generic/dram.c b/arch/riscv/cpu/generic/dram.c index 44e11bd56c..fb53a57b4e 100644 --- a/arch/riscv/cpu/generic/dram.c +++ b/arch/riscv/cpu/generic/dram.c @@ -13,7 +13,7 @@ DECLARE_GLOBAL_DATA_PTR; int dram_init(void) { - return fdtdec_setup_mem_size_base(); + return fdtdec_setup_mem_size_base_lowest(); } int dram_init_banksize(void) diff --git a/configs/qemu-riscv64_smode_defconfig b/configs/qemu-riscv64_smode_defconfig index 1d0f021ade..de08a49dab 100644 --- a/configs/qemu-riscv64_smode_defconfig +++ b/configs/qemu-riscv64_smode_defconfig @@ -1,6 +1,6 @@ CONFIG_RISCV=y CONFIG_SYS_MALLOC_LEN=0x800000 -CONFIG_NR_DRAM_BANKS=1 +CONFIG_NR_DRAM_BANKS=2 CONFIG_HAS_CUSTOM_SYS_INIT_SP_ADDR=y CONFIG_CUSTOM_SYS_INIT_SP_ADDR=0x80200000 CONFIG_ENV_SIZE=0x20000 Thanks, Fei.