Return-Path: Received: from mx2.suse.de ([195.135.220.15]:53873 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S935662AbdEWLXX (ORCPT ); Tue, 23 May 2017 07:23:23 -0400 Date: Tue, 23 May 2017 13:23:20 +0200 From: Jan Kara To: Sahitya Tummala Cc: linux-fsdevel@vger.kernel.org Subject: Re: __list_lru_walk_one() acquiring spin lock for longer time Message-ID: <20170523112320.GI1230@quack2.suse.cz> References: <26f4cf76-818c-d324-ed01-12207c258d0f@codeaurora.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <26f4cf76-818c-d324-ed01-12207c258d0f@codeaurora.org> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Thu 18-05-17 19:12:58, Sahitya Tummala wrote: > Hi all, > > I am observing "BUG: spinlock lockup suspected" issue in the below path - > > [] spin_bug+0x90 > [] do_raw_spin_lock+0xfc > [] _raw_spin_lock+0x28 > [] list_lru_add+0x28 > [] dput+0x1c8 > [] path_put+0x20 > [] terminate_walk+0x3c > [] path_lookupat+0x100 > [] filename_lookup+0x6c > [] user_path_at_empty+0x54 > [] SyS_faccessat+0xd0 > [] el0_svc_naked+0x24 > > This nlru->lock has been acquired by another CPU in this path - > > [] d_lru_shrink_move+0x34 > [] dentry_lru_isolate_shrink+0x48 > [] __list_lru_walk_one.isra.10+0x94 > [] list_lru_walk_node+0x40 > [] shrink_dcache_sb+0x60 > [] do_remount_sb+0xbc > [] do_emergency_remount+0xb0 > [] process_one_work+0x228 > [] worker_thread+0x2e0 > [] kthread+0xf4 > [] ret_from_fork+0x10 > > At the time of crash I see that __list_lru_walk_one() shows number of > entries isolated as 1774475 with nr_items still pending as 130748. On my > system, I see that for dentries of 100000, it takes around 75ms for > __list_lru_walk_one() to complete. So for a total of 1900000 dentries as in > issue scenario, it will take upto 1425ms, which explains why the spin lockup > condition got hit on the other CPU. > > It looks like __list_lru_walk_one() is expected to take more time if there > are more number of dentries present. Any suggestion on how to optimize > the__list_lru_walk_one() path to avoid the above spin lockup condition? Well, I suppose you can have 'cond_resched_lock(&nlru->lock)' check in __list_lru_walk_one() and goto restart if it rescheduled... Honza -- Jan Kara SUSE Labs, CR