From: Christoph Hellwig Date: Wed, 15 Jan 2020 07:56:14 +0100 Subject: [Cluster-devel] RFC: hold i_rwsem until aio completes In-Reply-To: <20200114192700.GC22037@ziepe.ca> References: <20200114161225.309792-1-hch@lst.de> <20200114192700.GC22037@ziepe.ca> Message-ID: <20200115065614.GC21219@lst.de> List-Id: To: cluster-devel.redhat.com MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Tue, Jan 14, 2020 at 03:27:00PM -0400, Jason Gunthorpe wrote: > I've seen similar locking patterns quite a lot, enough I've thought > about having a dedicated locking primitive to do it. It really wants > to be a rwsem, but as here the rwsem rules don't allow it. > > The common pattern I'm looking at looks something like this: > > 'try begin read'() // aka down_read_trylock() > > /* The lockdep release hackery you describe, > the rwsem remains read locked */ > 'exit reader'() > > .. delegate unlock to work queue, timer, irq, etc .. > > in the new context: > > 're_enter reader'() // Get our lockdep tracking back > > 'end reader'() // aka up_read() > > vs a typical write side: > > 'begin write'() // aka down_write() > > /* There is no reason to unlock it before kfree of the rwsem memory. > Somehow the user prevents any new down_read_trylock()'s */ > 'abandon writer'() // The object will be kfree'd with a locked writer > kfree() > > The typical goal is to provide an object destruction path that can > serialize and fence all readers wherever they may be before proceeding > to some synchronous destruction. > > Usually this gets open coded with some atomic/kref/refcount and a > completion or wait queue. Often implemented wrongly, lacking the write > favoring bias in the rwsem, and lacking any lockdep tracking on the > naked completion. > > Not to discourage your patch, but to ask if we can make the solution > more broadly applicable? Your requirement seems a little different, and in fact in many ways similar to the percpu_ref primitive.