Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3B40AC001E0 for ; Tue, 15 Aug 2023 11:40:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:CC: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=IATLC9zDkmt2FY8CuUiMTsvmTMaLQqO57vtszIAAgUo=; b=2A1AzL/eqc7z689nXjGc4LB7T3 VOVSFKNsCAXDZc9MQUikekJqlLN2OcRaYbMOzfa8BxbCzyzlXz/oahboWwQQiaPniNfeqXTlsuoJK 2CQUYwfkJafhfZet9Gajwh3N0x6lWvbY+9my/6pL5qTZ1zJi7LZlZO3HzlxiQTwyuxEDKeQpnxEgB cz4vDnFwLNjb3Zc3MGzs5OcNc1Zdv/dWjObxZgxBBoGOY4/YAg54PlWnV/tj4i72EysgysqS3dLZb WRgvqzvBQeOcUiGZZd98XHloAU3YTvSNQnANnYuqDt3oR6UEf1uFBay+KS3xOOlgy0j0jYZb+HAHZ kwDJE9fw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qVsPB-001XbZ-1Y; Tue, 15 Aug 2023 11:40:09 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qVsP7-001Xae-2W for linux-arm-kernel@lists.infradead.org; Tue, 15 Aug 2023 11:40:07 +0000 Received: from dggpeml500002.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RQ8Nn48VsztRwL; Tue, 15 Aug 2023 19:36:25 +0800 (CST) Received: from [10.67.103.44] (10.67.103.44) by dggpeml500002.china.huawei.com (7.185.36.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 15 Aug 2023 19:40:01 +0800 Subject: Re: [PATCH 1/2] coresight: trbe: Fix TRBE potential sleep in atomic context To: Suzuki K Poulose , , , , References: <20230814093813.19152-1-hejunhao3@huawei.com> <20230814093813.19152-2-hejunhao3@huawei.com> <37e36bde-fc70-6d2d-8dec-28e572f618cc@arm.com> <4049aee1-7f4d-5a54-ce27-7d802ae9616d@huawei.com> CC: , , , , From: hejunhao Message-ID: <426cd32b-4f11-e720-7463-86e34d7ec817@huawei.com> Date: Tue, 15 Aug 2023 19:40:00 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: X-Originating-IP: [10.67.103.44] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpeml500002.china.huawei.com (7.185.36.158) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230815_044006_158700_E08BAA4B X-CRM114-Status: GOOD ( 23.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Suzuki On 2023/8/15 6:57, Suzuki K Poulose wrote: > On 14/08/2023 14:32, hejunhao wrote: >> Hi Suzuki >> >> >> On 2023/8/14 18:34, Suzuki K Poulose wrote: >>> Hi Junhao >>> >>> On 14/08/2023 10:38, Junhao He wrote: >>>> smp_call_function_single() will allocate an IPI interrupt vector to >>>> the target processor and send a function call request to the interrupt >>>> vector. After the target processor receives the IPI interrupt, it will >>>> execute arm_trbe_remove_coresight_cpu() call request in the interrupt >>>> handler. >>>> >>>> According to the device_unregister() stack information, if other >>>> process >>>> is useing the device, the down_write() may sleep, and trigger >>>> deadlocks >>>> or unexpected errors. >>>> >>>> arm_trbe_remove_coresight_cpu >>>> coresight_unregister >>>> device_unregister >>>> device_del >>>> kobject_del >>>> __kobject_del >>>> sysfs_remove_dir >>>> kernfs_remove >>>> down_write ---------> it may sleep >>>> >>>> Add a helper arm_trbe_disable_cpu() to disable TRBE precpu irq and >>>> reset >>>> per TRBE. >>>> Simply call arm_trbe_remove_coresight_cpu() directly without useing >>>> the >>>> smp_call_function_single(), which is the same as registering the TRBE >>>> coresight device. >>>> >>>> Fixes: 3fbf7f011f24 ("coresight: sink: Add TRBE driver") >>>> Signed-off-by: Junhao He >>>> --- >>>> drivers/hwtracing/coresight/coresight-trbe.c | 35 >>>> +++++++++++--------- >>>> 1 file changed, 20 insertions(+), 15 deletions(-) >>>> >>>> diff --git a/drivers/hwtracing/coresight/coresight-trbe.c >>>> b/drivers/hwtracing/coresight/coresight-trbe.c >>>> index 7720619909d6..ce1e6f537b8d 100644 >>>> --- a/drivers/hwtracing/coresight/coresight-trbe.c >>>> +++ b/drivers/hwtracing/coresight/coresight-trbe.c >>>> @@ -1225,6 +1225,17 @@ static void arm_trbe_enable_cpu(void *info) >>>> enable_percpu_irq(drvdata->irq, IRQ_TYPE_NONE); >>>> } >>>> +static void arm_trbe_disable_cpu(void *info) >>>> +{ >>>> + struct trbe_drvdata *drvdata = info; >>>> + struct trbe_cpudata *cpudata = this_cpu_ptr(drvdata->cpudata); >>>> + >>>> + disable_percpu_irq(drvdata->irq); >>>> + trbe_reset_local(cpudata); >>>> + cpudata->drvdata = NULL; >>>> +} >>>> + >>>> + >>>> static void arm_trbe_register_coresight_cpu(struct trbe_drvdata >>>> *drvdata, int cpu) >>>> { >>>> struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, >>>> cpu); >>>> @@ -1326,18 +1337,12 @@ static void arm_trbe_probe_cpu(void *info) >>>> cpumask_clear_cpu(cpu, &drvdata->supported_cpus); >>>> } >>>> -static void arm_trbe_remove_coresight_cpu(void *info) >>>> +static void arm_trbe_remove_coresight_cpu(struct trbe_drvdata >>>> *drvdata, int cpu) >>>> { >>>> - int cpu = smp_processor_id(); >>>> - struct trbe_drvdata *drvdata = info; >>>> - struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, >>>> cpu); >>>> struct coresight_device *trbe_csdev = >>>> coresight_get_percpu_sink(cpu); >>>> - disable_percpu_irq(drvdata->irq); >>>> - trbe_reset_local(cpudata); >>>> if (trbe_csdev) { >>>> coresight_unregister(trbe_csdev); >>>> - cpudata->drvdata = NULL; >>>> coresight_set_percpu_sink(cpu, NULL); >>> >>> I am a bit concerned about "resetting" the sink from a different CPU. >>> Could we instead, schedule a delayed work to unregister the trbe_csdev? >> >> Yes, I will try to do that. >> Sorry for my following questions. >> As you mean, do we need to take the same care when setting the percpu >> sink >> in the register trbe_csdev ? > > Apologies, having taken another look, we set the percpu_sink for > a cpu outside smp_call_function(). So, I think your patch is fine. > > >> >> Best regards, >> Junhao. >> >>> >>> >>>> } >>>> } >>>> @@ -1366,8 +1371,12 @@ static int arm_trbe_remove_coresight(struct >>>> trbe_drvdata *drvdata) >>>> { >>>> int cpu; >>>> - for_each_cpu(cpu, &drvdata->supported_cpus) >>>> - smp_call_function_single(cpu, >>>> arm_trbe_remove_coresight_cpu, drvdata, 1); >>>> + for_each_cpu(cpu, &drvdata->supported_cpus) { >>>> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >>>> + smp_call_function_single(cpu, arm_trbe_disable_cpu, >>>> drvdata, 1); >>>> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >>>> + arm_trbe_remove_coresight_cpu(drvdata, cpu); > > Do we need to test the cpu here in both places ? We already check that > in the loop entry. The reason why we repeat the check during the probe, > is to skip any CPUs that may have a TRBE not accessible. > > Suzuki > Ok, Will fix in next version. Best regards, Junhao. > >>>> + } >>>> free_percpu(drvdata->cpudata); >>>> return 0; >>>> } >>>> @@ -1406,12 +1415,8 @@ static int arm_trbe_cpu_teardown(unsigned >>>> int cpu, struct hlist_node *node) >>>> { >>>> struct trbe_drvdata *drvdata = hlist_entry_safe(node, struct >>>> trbe_drvdata, hotplug_node); >>>> - if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) { >>>> - struct trbe_cpudata *cpudata = >>>> per_cpu_ptr(drvdata->cpudata, cpu); >>>> - >>>> - disable_percpu_irq(drvdata->irq); >>>> - trbe_reset_local(cpudata); >>>> - } >>>> + if (cpumask_test_cpu(cpu, &drvdata->supported_cpus)) >>>> + arm_trbe_disable_cpu(drvdata); >>>> return 0; >>>> } >>> >>> >>> . >>> >> > > > . > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel