Skip to contentSkip to conversation

fs: Substitute bit-spinlocks for PREEMPT_RT and debugging

PATCH In review Thomas Gleixner 6 participants 27 messages Aug 1, 2019 – Aug 20, 2019

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

v2 cover fs: Substitute bit-spinlocks for PREEMPT_RT and debugging

Bit spinlocks are problematic if PREEMPT_RT is enabled. They disable preemption, which is undesired for latency reasons and breaks when regular spinlocks are taken within the bit_spinlock locked region because regular spinlocks are converted to 'sleeping spinlocks' on RT.

Bit spinlocks are also not covered by lock debugging, e.g. lockdep. With the spinlock substitution in place, they can be exposed via a new config
switch: CONFIG_DEBUG_BIT_SPINLOCKS.

This series handles only buffer head and jbd2, but does not touch the hlist_bl bit-spinlock usage. See V1 for further details:

https://lkml.kernel.org/r/[email protected]

Changes vs. V1:

- Collected reviewed-by tags for the BH_Uptodate part

- Reworked the JBD2 part according to Jan's review:

- Convert state lock to a regular spinlock unconditionally

- Refactor jbd2_journal_put_journal_head() to be RT friendly by restricting the bit-spinlock held section to the minimum required operations.

That part is probably over-engineered, but I'm sure Jan will tell me sooner than later.

Thanks,

tglx

#1 lore raw

PATCH v2 1/7 locking/lockdep: Add Kconfig option for bit spinlocks

Some usage sites of bit spinlocks have a substitution with regular spinlocks which depends on CONFIG_PREEMPT_RT. But this substitution can also be used to expose these locks to the regular lock debugging infrastructure, e.g. lockdep.

As this increases the size of affected data structures significantly this is guarded by a separate Kconfig switch.

Note, that only the bit spinlocks which have a substitution implemented will be covered by this. All other bit spinlocks evade lock debugging as before.

---
 lib/Kconfig.debug |   10 ++++++++++
 1 file changed, 10 insertions(+)
lib/Kconfig.debug +10 -0
@@ -1201,6 +1201,16 @@ config DEBUG_RWSEMS
 	  This debugging feature allows mismatched rw semaphore locks
 	  and unlocks to be detected and reported.
 
+config DEBUG_BIT_SPINLOCKS
+	bool "Bit spinlock debugging"
+	depends on DEBUG_SPINLOCK
+	help
+	  This debugging feature substitutes bit spinlocks in some use
+	  cases, e.g. buffer head, zram, with with regular spinlocks so
+	  these locks are exposed to lock debugging features.
+
+	  Not all bit spinlocks are covered by this.
+
 config DEBUG_LOCK_ALLOC
 	bool "Lock debugging: detect incorrect freeing of live locks"
 	depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT
#2 lore raw

PATCH v2 2/7 fs/buffer: Move BH_Uptodate_Lock locking into wrapper functions

In reply to Thomas Gleixner

Bit spinlocks are problematic if PREEMPT_RT is enabled, because they disable preemption, which is undesired for latency reasons and breaks when regular spinlocks are taken within the bit_spinlock locked region because regular spinlocks are converted to 'sleeping spinlocks' on RT. So RT replaces the bit spinlocks with regular spinlocks to avoid this problem.

To avoid ifdeffery at the source level, wrap all BH_Uptodate_Lock bitlock operations with inline functions, so the spinlock substitution can be done at one place.

Using regular spinlocks can also be enabled for lock debugging purposes so the lock operations become visible to lockdep.

---
V2: Collected Reviewed-by tag

---
 fs/buffer.c                 |   20 ++++++--------------
 fs/ext4/page-io.c           |    6 ++----
 fs/ntfs/aops.c              |   10 +++-------
 include/linux/buffer_head.h |   16 ++++++++++++++++
 4 files changed, 27 insertions(+), 25 deletions(-)
fs/buffer.c +6 -14
@@ -275,8 +275,7 @@ static void end_buffer_async_read(struct
 	 * decide that the page is now completely done.
 	 */
 	first = page_buffers(page);
-	local_irq_save(flags);
-	bit_spin_lock(BH_Uptodate_Lock, &first->b_state);
+	flags = bh_uptodate_lock_irqsave(first);
 	clear_buffer_async_read(bh);
 	unlock_buffer(bh);
 	tmp = bh;
@@ -289,8 +288,7 @@ static void end_buffer_async_read(struct
 		}
 		tmp = tmp->b_this_page;
 	} while (tmp != bh);
-	bit_spin_unlock(BH_Uptodate_Lock, &first->b_state);
-	local_irq_restore(flags);
+	bh_uptodate_unlock_irqrestore(first, flags);
 
 	/*
 	 * If none of the buffers had errors and they are all
@@ -302,9 +300,7 @@ static void end_buffer_async_read(struct
 	return;
 
 still_busy:
-	bit_spin_unlock(BH_Uptodate_Lock, &first->b_state);
-	local_irq_restore(flags);
-	return;
+	bh_uptodate_unlock_irqrestore(first, flags);
 }
 
 /*
@@ -331,8 +327,7 @@ void end_buffer_async_write(struct buffe
 	}
 
 	first = page_buffers(page);
-	local_irq_save(flags);
-	bit_spin_lock(BH_Uptodate_Lock, &first->b_state);
+	flags = bh_uptodate_lock_irqsave(first);
 
 	clear_buffer_async_write(bh);
 	unlock_buffer(bh);
@@ -344,15 +339,12 @@ void end_buffer_async_write(struct buffe
 		}
 		tmp = tmp->b_this_page;
 	}
-	bit_spin_unlock(BH_Uptodate_Lock, &first->b_state);
-	local_irq_restore(flags);
+	bh_uptodate_unlock_irqrestore(first, flags);
 	end_page_writeback(page);
 	return;
 
 still_busy:
-	bit_spin_unlock(BH_Uptodate_Lock, &first->b_state);
-	local_irq_restore(flags);
-	return;
+	bh_uptodate_unlock_irqrestore(first, flags);
 }
 EXPORT_SYMBOL(end_buffer_async_write);
 
fs/ext4/page-io.c +2 -4
@@ -90,8 +90,7 @@ static void ext4_finish_bio(struct bio *
 		 * We check all buffers in the page under BH_Uptodate_Lock
 		 * to avoid races with other end io clearing async_write flags
 		 */
-		local_irq_save(flags);
-		bit_spin_lock(BH_Uptodate_Lock, &head->b_state);
+		flags = bh_uptodate_lock_irqsave(head);
 		do {
 			if (bh_offset(bh) < bio_start ||
 			    bh_offset(bh) + bh->b_size > bio_end) {
@@ -103,8 +102,7 @@ static void ext4_finish_bio(struct bio *
 			if (bio->bi_status)
 				buffer_io_error(bh);
 		} while ((bh = bh->b_this_page) != head);
-		bit_spin_unlock(BH_Uptodate_Lock, &head->b_state);
-		local_irq_restore(flags);
+		bh_uptodate_unlock_irqrestore(head, flags);
 		if (!under_io) {
 			fscrypt_free_bounce_page(bounce_page);
 			end_page_writeback(page);
fs/ntfs/aops.c +3 -7
@@ -92,8 +92,7 @@ static void ntfs_end_buffer_async_read(s
 				"0x%llx.", (unsigned long long)bh->b_blocknr);
 	}
 	first = page_buffers(page);
-	local_irq_save(flags);
-	bit_spin_lock(BH_Uptodate_Lock, &first->b_state);
+	flags = bh_uptodate_lock_irqsave(first);
 	clear_buffer_async_read(bh);
 	unlock_buffer(bh);
 	tmp = bh;
@@ -108,8 +107,7 @@ static void ntfs_end_buffer_async_read(s
 		}
 		tmp = tmp->b_this_page;
 	} while (tmp != bh);
-	bit_spin_unlock(BH_Uptodate_Lock, &first->b_state);
-	local_irq_restore(flags);
+	bh_uptodate_unlock_irqrestore(first, flags);
 	/*
 	 * If none of the buffers had errors then we can set the page uptodate,
 	 * but we first have to perform the post read mst fixups, if the
@@ -142,9 +140,7 @@ static void ntfs_end_buffer_async_read(s
 	unlock_page(page);
 	return;
 still_busy:
-	bit_spin_unlock(BH_Uptodate_Lock, &first->b_state);
-	local_irq_restore(flags);
-	return;
+	bh_uptodate_unlock_irqrestore(first, flags);
 }
 
 /**
include/linux/buffer_head.h +16 -0
@@ -78,6 +78,22 @@ struct buffer_head {
 	atomic_t b_count;		/* users using this buffer_head */
 };
 
+static inline unsigned long bh_uptodate_lock_irqsave(struct buffer_head *bh)
+{
+	unsigned long flags;
+
+	local_irq_save(flags);
+	bit_spin_lock(BH_Uptodate_Lock, &bh->b_state);
+	return flags;
+}
+
+static inline void
+bh_uptodate_unlock_irqrestore(struct buffer_head *bh, unsigned long flags)
+{
+	bit_spin_unlock(BH_Uptodate_Lock, &bh->b_state);
+	local_irq_restore(flags);
+}
+
 /*
  * macro tricks to expand the set_buffer_foo(), clear_buffer_foo()
  * and buffer_foo() functions.
#3 lore raw

PATCH v2 3/7 fs/buffer: Substitute BH_Uptodate_Lock for RT and bit spinlock debugging

In reply to Thomas Gleixner

Bit spinlocks are problematic if PREEMPT_RT is enabled. They disable preemption, which is undesired for latency reasons and breaks when regular spinlocks are taken within the bit_spinlock locked region because regular spinlocks are converted to 'sleeping spinlocks' on RT.

Substitute the BH_Uptodate_Lock bit spinlock with a regular spinlock for PREEMPT_RT enabled kernels.

Bit spinlocks are also not covered by lock debugging, e.g. lockdep. With the spinlock substitution in place, they can be exposed via CONFIG_DEBUG_BIT_SPINLOCKS.

---
V2: Collected Reviewed-by tag

---
 fs/buffer.c                 |    1 +
 include/linux/buffer_head.h |   31 +++++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+)
fs/buffer.c +1 -0
@@ -3360,6 +3360,7 @@ struct buffer_head *alloc_buffer_head(gf
 	struct buffer_head *ret = kmem_cache_zalloc(bh_cachep, gfp_flags);
 	if (ret) {
 		INIT_LIST_HEAD(&ret->b_assoc_buffers);
+		buffer_head_init_locks(ret);
 		preempt_disable();
 		__this_cpu_inc(bh_accounting.nr);
 		recalc_bh_state();
include/linux/buffer_head.h +31 -0
@@ -76,8 +76,35 @@ struct buffer_head {
 	struct address_space *b_assoc_map;	/* mapping this buffer is
 						   associated with */
 	atomic_t b_count;		/* users using this buffer_head */
+
+#if defined(CONFIG_PREEMPT_RT) || defined(CONFIG_DEBUG_BIT_SPINLOCKS)
+	spinlock_t b_uptodate_lock;
+#endif
 };
 
+#if defined(CONFIG_PREEMPT_RT) || defined(CONFIG_DEBUG_BIT_SPINLOCKS)
+
+static inline unsigned long bh_uptodate_lock_irqsave(struct buffer_head *bh)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&bh->b_uptodate_lock, flags);
+	return flags;
+}
+
+static inline void
+bh_uptodate_unlock_irqrestore(struct buffer_head *bh, unsigned long flags)
+{
+	spin_unlock_irqrestore(&bh->b_uptodate_lock, flags);
+}
+
+static inline void buffer_head_init_locks(struct buffer_head *bh)
+{
+	spin_lock_init(&bh->b_uptodate_lock);
+}
+
+#else /* PREEMPT_RT || DEBUG_BIT_SPINLOCKS */
+
 static inline unsigned long bh_uptodate_lock_irqsave(struct buffer_head *bh)
 {
 	unsigned long flags;
@@ -94,6 +121,10 @@ bh_uptodate_unlock_irqrestore(struct buf
 	local_irq_restore(flags);
 }
 
+static inline void buffer_head_init_locks(struct buffer_head *bh) { }
+
+#endif /* !PREEMPT_RT && !DEBUG_BIT_SPINLOCKS */
+
 /*
  * macro tricks to expand the set_buffer_foo(), clear_buffer_foo()
  * and buffer_foo() functions.
#4 lore raw

PATCH v2 4/7 fs/jbd2: Remove jbd_trylock_bh_state()

In reply to Thomas Gleixner

No users.

---
 include/linux/jbd2.h |    5 -----
 1 file changed, 5 deletions(-)
include/linux/jbd2.h +0 -5
@@ -347,11 +347,6 @@ static inline void jbd_lock_bh_state(str
 	bit_spin_lock(BH_State, &bh->b_state);
 }
 
-static inline int jbd_trylock_bh_state(struct buffer_head *bh)
-{
-	return bit_spin_trylock(BH_State, &bh->b_state);
-}
-
 static inline int jbd_is_locked_bh_state(struct buffer_head *bh)
 {
 	return bit_spin_is_locked(BH_State, &bh->b_state);
reviewer Reviewed-by on v2 4/7
#5 lore raw

Re: [patch V2 4/7] fs/jbd2: Remove jbd_trylock_bh_state()

Quoted 6 lines · Thomas Gleixner

No users.

Right.

Honza

Quoted 20 lines
---
 include/linux/jbd2.h |    5 -----
 1 file changed, 5 deletions(-)
include/linux/jbd2.h +0 -5
@@ -347,11 +347,6 @@ static inline void jbd_lock_bh_state(str
 	bit_spin_lock(BH_State, &bh->b_state);
 }
 
-static inline int jbd_trylock_bh_state(struct buffer_head *bh)
-{
-	return bit_spin_trylock(BH_State, &bh->b_state);
-}
-
 static inline int jbd_is_locked_bh_state(struct buffer_head *bh)
 {
 	return bit_spin_is_locked(BH_State, &bh->b_state);
Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#6 lore raw

PATCH v2 5/7 fs/jbd2: Simplify journal_unmap_buffer()

In reply to Thomas Gleixner

journal_unmap_buffer() checks first whether the buffer head is a journal. If so it takes locks and then invokes jbd2_journal_grab_journal_head() followed by another check whether this is journal head buffer.

The double checking is pointless.

Replace the initial check with jbd2_journal_grab_journal_head() which alredy checks whether the buffer head is actually a journal.

Allows also early access to the journal head pointer for the upcoming conversion of state lock to a regular spinlock.

---
V2: New patch

---
 fs/jbd2/transaction.c |    8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)
fs/jbd2/transaction.c +2 -6
@@ -2196,7 +2196,8 @@ static int journal_unmap_buffer(journal_
 	 * holding the page lock. --sct
 	 */
 
-	if (!buffer_jbd(bh))
+	jh = jbd2_journal_grab_journal_head(bh);
+	if (!jh)
 		goto zap_buffer_unlocked;
 
 	/* OK, we have data buffer in journaled mode */
@@ -2204,10 +2205,6 @@ static int journal_unmap_buffer(journal_
 	jbd_lock_bh_state(bh);
 	spin_lock(&journal->j_list_lock);
 
-	jh = jbd2_journal_grab_journal_head(bh);
-	if (!jh)
-		goto zap_buffer_no_jh;
-
 	/*
 	 * We cannot remove the buffer from checkpoint lists until the
 	 * transaction adding inode to orphan list (let's call it T)
@@ -2329,7 +2326,6 @@ static int journal_unmap_buffer(journal_
 	 */
 	jh->b_modified = 0;
 	jbd2_journal_put_journal_head(jh);
-zap_buffer_no_jh:
 	spin_unlock(&journal->j_list_lock);
 	jbd_unlock_bh_state(bh);
 	write_unlock(&journal->j_state_lock);
reviewer Reviewed-by on v2 5/7
#7 lore raw

Re: [patch V2 5/7] fs/jbd2: Simplify journal_unmap_buffer()

Quoted 9 lines · Thomas Gleixner

journal_unmap_buffer() checks first whether the buffer head is a journal. If so it takes locks and then invokes jbd2_journal_grab_journal_head() followed by another check whether this is journal head buffer.

The double checking is pointless.

Replace the initial check with jbd2_journal_grab_journal_head() which alredy checks whether the buffer head is actually a journal.

Quoted 7 lines

Allows also early access to the journal head pointer for the upcoming conversion of state lock to a regular spinlock.

Nice simplification. You can add:

Honza

Quoted 39 lines

---
V2: New patch

---
 fs/jbd2/transaction.c |    8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)
fs/jbd2/transaction.c +2 -6
@@ -2196,7 +2196,8 @@ static int journal_unmap_buffer(journal_
 	 * holding the page lock. --sct
 	 */
 
-	if (!buffer_jbd(bh))
+	jh = jbd2_journal_grab_journal_head(bh);
+	if (!jh)
 		goto zap_buffer_unlocked;
 
 	/* OK, we have data buffer in journaled mode */
@@ -2204,10 +2205,6 @@ static int journal_unmap_buffer(journal_
 	jbd_lock_bh_state(bh);
 	spin_lock(&journal->j_list_lock);
 
-	jh = jbd2_journal_grab_journal_head(bh);
-	if (!jh)
-		goto zap_buffer_no_jh;
-
 	/*
 	 * We cannot remove the buffer from checkpoint lists until the
 	 * transaction adding inode to orphan list (let's call it T)
@@ -2329,7 +2326,6 @@ static int journal_unmap_buffer(journal_
 	 */
 	jh->b_modified = 0;
 	jbd2_journal_put_journal_head(jh);
-zap_buffer_no_jh:
 	spin_unlock(&journal->j_list_lock);
 	jbd_unlock_bh_state(bh);
 	write_unlock(&journal->j_state_lock);
Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#8 lore raw

PATCH v2 6/7 fs/jbd2: Make state lock a spinlock

In reply to Thomas Gleixner

Bit-spinlocks are problematic on PREEMPT_RT if functions which might sleep on RT, e.g. spin_lock(), alloc/free(), are invoked inside the lock held region because bit spinlocks disable preemption even on RT.

A first attempt was to replace state lock with a spinlock placed in struct buffer_head and make the locking conditional on PREEMPT_RT and DEBUG_BIT_SPINLOCKS.

Jan pointed out that there is a 4 byte hole in struct journal_head where a regular spinlock fits in and he would not object to convert the state lock to a spinlock unconditionally.

Aside of solving the RT problem, this also gains lockdep coverage for the journal head state lock (bit-spinlocks are not covered by lockdep as it's hard to fit a lockdep map into a single bit).

The trivial change would have been to convert the jbd_*lock_bh_state() inlines, but that comes with the downside that these functions take a buffer head pointer which needs to be converted to a journal head pointer which adds another level of indirection.

As almost all functions which use this lock have a journal head pointer readily available, it makes more sense to remove the lock helper inlines and write out spin_*lock() at all call sites.

Fixup all locking comments as well.

---
V2: New patch

---
 fs/jbd2/commit.c             |    8 ++--
 fs/jbd2/journal.c            |   10 +++--
 fs/jbd2/transaction.c        |   83 +++++++++++++++++++++----------------------
 fs/ocfs2/suballoc.c          |   19 +++++----
 include/linux/jbd2.h         |   20 +---------
 include/linux/journal-head.h |   19 ++++++---
 6 files changed, 77 insertions(+), 82 deletions(-)
fs/jbd2/commit.c +4 -4
@@ -482,10 +482,10 @@ void jbd2_journal_commit_transaction(jou
 		if (jh->b_committed_data) {
 			struct buffer_head *bh = jh2bh(jh);
 
-			jbd_lock_bh_state(bh);
+			spin_lock(&jh->state_lock);
 			jbd2_free(jh->b_committed_data, bh->b_size);
 			jh->b_committed_data = NULL;
-			jbd_unlock_bh_state(bh);
+			spin_unlock(&jh->state_lock);
 		}
 		jbd2_journal_refile_buffer(journal, jh);
 	}
@@ -927,7 +927,7 @@ void jbd2_journal_commit_transaction(jou
 		 * done with it.
 		 */
 		get_bh(bh);
-		jbd_lock_bh_state(bh);
+		spin_lock(&jh->state_lock);
 		J_ASSERT_JH(jh,	jh->b_transaction == commit_transaction);
 
 		/*
@@ -1023,7 +1023,7 @@ void jbd2_journal_commit_transaction(jou
 		}
 		JBUFFER_TRACE(jh, "refile or unfile buffer");
 		__jbd2_journal_refile_buffer(jh);
-		jbd_unlock_bh_state(bh);
+		spin_unlock(&jh->state_lock);
 		if (try_to_free)
 			release_buffer_page(bh);	/* Drops bh reference */
 		else
fs/jbd2/journal.c +6 -4
@@ -365,7 +365,7 @@ int jbd2_journal_write_metadata_buffer(t
 	/* keep subsequent assertions sane */
 	atomic_set(&new_bh->b_count, 1);
 
-	jbd_lock_bh_state(bh_in);
+	spin_lock(&jh_in->state_lock);
 repeat:
 	/*
 	 * If a new transaction has already done a buffer copy-out, then
@@ -407,13 +407,13 @@ int jbd2_journal_write_metadata_buffer(t
 	if (need_copy_out && !done_copy_out) {
 		char *tmp;
 
-		jbd_unlock_bh_state(bh_in);
+		spin_unlock(&jh_in->state_lock);
 		tmp = jbd2_alloc(bh_in->b_size, GFP_NOFS);
 		if (!tmp) {
 			brelse(new_bh);
 			return -ENOMEM;
 		}
-		jbd_lock_bh_state(bh_in);
+		spin_lock(&jh_in->state_lock);
 		if (jh_in->b_frozen_data) {
 			jbd2_free(tmp, bh_in->b_size);
 			goto repeat;
@@ -466,7 +466,7 @@ int jbd2_journal_write_metadata_buffer(t
 	__jbd2_journal_file_buffer(jh_in, transaction, BJ_Shadow);
 	spin_unlock(&journal->j_list_lock);
 	set_buffer_shadow(bh_in);
-	jbd_unlock_bh_state(bh_in);
+	spin_unlock(&jh_in->state_lock);
 
 	return do_escape | (done_copy_out << 1);
 }
@@ -2412,6 +2412,8 @@ static struct journal_head *journal_allo
 		ret = kmem_cache_zalloc(jbd2_journal_head_cache,
 				GFP_NOFS | __GFP_NOFAIL);
 	}
+	if (ret)
+		spin_lock_init(&ret->state_lock);
 	return ret;
 }
 
fs/jbd2/transaction.c +42 -41

306 lines not on this page · raw message

fs/ocfs2/suballoc.c +11 -8
@@ -1252,6 +1252,7 @@ static int ocfs2_test_bg_bit_allocatable
 					 int nr)
 {
 	struct ocfs2_group_desc *bg = (struct ocfs2_group_desc *) bg_bh->b_data;
+	struct journal_head *jh;
 	int ret;
 
 	if (ocfs2_test_bit(nr, (unsigned long *)bg->bg_bitmap))
@@ -1260,13 +1261,14 @@ static int ocfs2_test_bg_bit_allocatable
 	if (!buffer_jbd(bg_bh))
 		return 1;
 
-	jbd_lock_bh_state(bg_bh);
-	bg = (struct ocfs2_group_desc *) bh2jh(bg_bh)->b_committed_data;
+	jh = bh2jh(bg_bh);
+	spin_lock(&jh->state_lock);
+	bg = (struct ocfs2_group_desc *) jh->b_committed_data;
 	if (bg)
 		ret = !ocfs2_test_bit(nr, (unsigned long *)bg->bg_bitmap);
 	else
 		ret = 1;
-	jbd_unlock_bh_state(bg_bh);
+	spin_unlock(&jh->state_lock);
 
 	return ret;
 }
@@ -2387,6 +2389,7 @@ static int ocfs2_block_group_clear_bits(
 	int status;
 	unsigned int tmp;
 	struct ocfs2_group_desc *undo_bg = NULL;
+	struct journal_head *jh;
 
 	/* The caller got this descriptor from
 	 * ocfs2_read_group_descriptor().  Any corruption is a code bug. */
@@ -2405,10 +2408,10 @@ static int ocfs2_block_group_clear_bits(
 		goto bail;
 	}
 
+	jh = bh2jh(group_bh);
 	if (undo_fn) {
-		jbd_lock_bh_state(group_bh);
-		undo_bg = (struct ocfs2_group_desc *)
-					bh2jh(group_bh)->b_committed_data;
+		spin_lock(&jh->state_lock);
+		undo_bg = (struct ocfs2_group_desc *) jh->b_committed_data;
 		BUG_ON(!undo_bg);
 	}
 
@@ -2423,7 +2426,7 @@ static int ocfs2_block_group_clear_bits(
 	le16_add_cpu(&bg->bg_free_bits_count, num_bits);
 	if (le16_to_cpu(bg->bg_free_bits_count) > le16_to_cpu(bg->bg_bits)) {
 		if (undo_fn)
-			jbd_unlock_bh_state(group_bh);
+			spin_unlock(&jh->state_lock);
 		return ocfs2_error(alloc_inode->i_sb, "Group descriptor # %llu has bit count %u but claims %u are freed. num_bits %d\n",
 				   (unsigned long long)le64_to_cpu(bg->bg_blkno),
 				   le16_to_cpu(bg->bg_bits),
@@ -2432,7 +2435,7 @@ static int ocfs2_block_group_clear_bits(
 	}
 
 	if (undo_fn)
-		jbd_unlock_bh_state(group_bh);
+		spin_unlock(&jh->state_lock);
 
 	ocfs2_journal_dirty(handle, group_bh);
 bail:
include/linux/jbd2.h +2 -18
@@ -313,7 +313,6 @@ enum jbd_state_bits {
 	BH_Revoked,		/* Has been revoked from the log */
 	BH_RevokeValid,		/* Revoked flag is valid */
 	BH_JBDDirty,		/* Is dirty but journaled */
-	BH_State,		/* Pins most journal_head state */
 	BH_JournalHead,		/* Pins bh->b_private and jh->b_bh */
 	BH_Shadow,		/* IO on shadow buffer is running */
 	BH_Verified,		/* Metadata block has been verified ok */
@@ -342,21 +341,6 @@ static inline struct journal_head *bh2jh
 	return bh->b_private;
 }
 
-static inline void jbd_lock_bh_state(struct buffer_head *bh)
-{
-	bit_spin_lock(BH_State, &bh->b_state);
-}
-
-static inline int jbd_is_locked_bh_state(struct buffer_head *bh)
-{
-	return bit_spin_is_locked(BH_State, &bh->b_state);
-}
-
-static inline void jbd_unlock_bh_state(struct buffer_head *bh)
-{
-	bit_spin_unlock(BH_State, &bh->b_state);
-}
-
 static inline void jbd_lock_bh_journal_head(struct buffer_head *bh)
 {
 	bit_spin_lock(BH_JournalHead, &bh->b_state);
@@ -551,9 +535,9 @@ struct transaction_chp_stats_s {
  *      ->jbd_lock_bh_journal_head()	(This is "innermost")
  *
  *    j_state_lock
- *    ->jbd_lock_bh_state()
+ *    ->jh->state_lock
  *
- *    jbd_lock_bh_state()
+ *    jh->state_lock
  *    ->j_list_lock
  *
  *    j_state_lock
include/linux/journal-head.h +12 -7
@@ -24,13 +24,18 @@ struct journal_head {
 	struct buffer_head *b_bh;
 
 	/*
+	 * Protect the buffer head state
+	 */
+	spinlock_t state_lock;
+
+	/*
 	 * Reference count - see description in journal.c
 	 * [jbd_lock_bh_journal_head()]
 	 */
 	int b_jcount;
 
 	/*
-	 * Journalling list for this buffer [jbd_lock_bh_state()]
+	 * Journalling list for this buffer [jh->state_lock]
 	 * NOTE: We *cannot* combine this with b_modified into a bitfield
 	 * as gcc would then (which the C standard allows but which is
 	 * very unuseful) make 64-bit accesses to the bitfield and clobber
@@ -41,20 +46,20 @@ struct journal_head {
 	/*
 	 * This flag signals the buffer has been modified by
 	 * the currently running transaction
-	 * [jbd_lock_bh_state()]
+	 * [jh->state_lock]
 	 */
 	unsigned b_modified;
 
 	/*
 	 * Copy of the buffer data frozen for writing to the log.
-	 * [jbd_lock_bh_state()]
+	 * [jh->state_lock]
 	 */
 	char *b_frozen_data;
 
 	/*
 	 * Pointer to a saved copy of the buffer containing no uncommitted
 	 * deallocation references, so that allocations can avoid overwriting
-	 * uncommitted deletes. [jbd_lock_bh_state()]
+	 * uncommitted deletes. [jh->state_lock]
 	 */
 	char *b_committed_data;
 
@@ -63,7 +68,7 @@ struct journal_head {
 	 * metadata: either the running transaction or the committing
 	 * transaction (if there is one).  Only applies to buffers on a
 	 * transaction's data or metadata journaling list.
-	 * [j_list_lock] [jbd_lock_bh_state()]
+	 * [j_list_lock] [jh->state_lock]
 	 * Either of these locks is enough for reading, both are needed for
 	 * changes.
 	 */
@@ -73,13 +78,13 @@ struct journal_head {
 	 * Pointer to the running compound transaction which is currently
 	 * modifying the buffer's metadata, if there was already a transaction
 	 * committing it when the new transaction touched it.
-	 * [t_list_lock] [jbd_lock_bh_state()]
+	 * [t_list_lock] [jh->state_lock]
 	 */
 	transaction_t *b_next_transaction;
 
 	/*
 	 * Doubly-linked list of buffers on a transaction's data, metadata or
-	 * forget queue. [t_list_lock] [jbd_lock_bh_state()]
+	 * forget queue. [t_list_lock] [jh->state_lock]
 	 */
 	struct journal_head *b_tnext, *b_tprev;
 
#9 lore raw

Re: [patch V2 6/7] fs/jbd2: Make state lock a spinlock

Quoted 21 lines · Thomas Gleixner
fs/jbd2/transaction.c +2 -2
@@ -1931,7 +1932,7 @@ static void __jbd2_journal_temp_unlink_b
 	transaction_t *transaction;
 	struct buffer_head *bh = jh2bh(jh);
 
-	J_ASSERT_JH(jh, jbd_is_locked_bh_state(bh));
+	assert_spin_locked(&jh->state_lock);
 	transaction = jh->b_transaction;
 	if (transaction)
 		assert_spin_locked(&transaction->t_journal->j_list_lock);
@@ -2415,7 +2416,7 @@ void __jbd2_journal_file_buffer(struct j
 	int was_dirty = 0;
 	struct buffer_head *bh = jh2bh(jh);
 
-	J_ASSERT_JH(jh, jbd_is_locked_bh_state(bh));
+	assert_spin_locked(&jh->state_lock);
 	assert_spin_locked(&transaction->t_journal->j_list_lock);
 
 	J_ASSERT_JH(jh, jh->b_jlist < BJ_Types);

fs/jbd2/transaction.c void __jbd2_journal_refile_buffer(struct · line 2507

Quoted 8 lines
fs/jbd2/transaction.c +1 -1
@@ -2500,7 +2501,7 @@ void __jbd2_journal_refile_buffer(struct
 	int was_dirty, jlist;
 	struct buffer_head *bh = jh2bh(jh);
 
-	J_ASSERT_JH(jh, jbd_is_locked_bh_state(bh));
+	assert_spin_locked(&jh->state_lock);
 	if (jh->b_transaction)
 		assert_spin_locked(&jh->b_transaction->t_journal->j_list_lock);
 

Do those want to be:

  lockdep_assert_held(&jh->state_lock);

instead? The difference is of course that lockdep_assert_held() requires the current context to hold the lock, where assert_*_locked() merely checks _someone_ holds it.

reviewer
#10 lore raw

Re: [patch V2 6/7] fs/jbd2: Make state lock a spinlock

Quoted 34 lines · Peter Zijlstra
Quoted 21 lines · Thomas Gleixner
fs/jbd2/transaction.c +2 -2
@@ -1931,7 +1932,7 @@ static void __jbd2_journal_temp_unlink_b
 	transaction_t *transaction;
 	struct buffer_head *bh = jh2bh(jh);
 
-	J_ASSERT_JH(jh, jbd_is_locked_bh_state(bh));
+	assert_spin_locked(&jh->state_lock);
 	transaction = jh->b_transaction;
 	if (transaction)
 		assert_spin_locked(&transaction->t_journal->j_list_lock);
@@ -2415,7 +2416,7 @@ void __jbd2_journal_file_buffer(struct j
 	int was_dirty = 0;
 	struct buffer_head *bh = jh2bh(jh);
 
-	J_ASSERT_JH(jh, jbd_is_locked_bh_state(bh));
+	assert_spin_locked(&jh->state_lock);
 	assert_spin_locked(&transaction->t_journal->j_list_lock);
 
 	J_ASSERT_JH(jh, jh->b_jlist < BJ_Types);

fs/jbd2/transaction.c void __jbd2_journal_refile_buffer(struct · line 2507

Quoted 8 lines
fs/jbd2/transaction.c +1 -1
@@ -2500,7 +2501,7 @@ void __jbd2_journal_refile_buffer(struct
 	int was_dirty, jlist;
 	struct buffer_head *bh = jh2bh(jh);
 
-	J_ASSERT_JH(jh, jbd_is_locked_bh_state(bh));
+	assert_spin_locked(&jh->state_lock);
 	if (jh->b_transaction)
 		assert_spin_locked(&jh->b_transaction->t_journal->j_list_lock);
 

Do those want to be:

Quoted 5 lines
  lockdep_assert_held(&jh->state_lock);

instead? The difference is of course that lockdep_assert_held() requires the current context to hold the lock, where assert_*_locked() merely checks _someone_ holds it.

Yeah, jbd2 doesn't play any weird locking tricks so lockdep_assert_held() is fine. I'll replace those when I'm updating the patch.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
reviewer
#11 lore raw

Re: [patch V2 6/7] fs/jbd2: Make state lock a spinlock

In reply to Thomas Gleixner
Quoted 27 lines · Thomas Gleixner

Bit-spinlocks are problematic on PREEMPT_RT if functions which might sleep on RT, e.g. spin_lock(), alloc/free(), are invoked inside the lock held region because bit spinlocks disable preemption even on RT.

A first attempt was to replace state lock with a spinlock placed in struct buffer_head and make the locking conditional on PREEMPT_RT and DEBUG_BIT_SPINLOCKS.

Jan pointed out that there is a 4 byte hole in struct journal_head where a regular spinlock fits in and he would not object to convert the state lock to a spinlock unconditionally.

Aside of solving the RT problem, this also gains lockdep coverage for the journal head state lock (bit-spinlocks are not covered by lockdep as it's hard to fit a lockdep map into a single bit).

The trivial change would have been to convert the jbd_*lock_bh_state() inlines, but that comes with the downside that these functions take a buffer head pointer which needs to be converted to a journal head pointer which adds another level of indirection.

As almost all functions which use this lock have a journal head pointer readily available, it makes more sense to remove the lock helper inlines and write out spin_*lock() at all call sites.

Fixup all locking comments as well.

Quoted 8 lines

Just a heads up that I didn't miss this patch. Just it has some bugs and I figured that rather than explaining to you subtleties of jh lifetime it is easier to fix up the problems myself since you're probably not keen on becoming jbd2 developer ;)... which was more complex than I thought so I'm not completely done yet. Hopefuly tomorrow.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#12 lore raw

Re: [patch V2 6/7] fs/jbd2: Make state lock a spinlock

Quoted 11 lines · Jan Kara
Quoted 4 lines · Thomas Gleixner

As almost all functions which use this lock have a journal head pointer readily available, it makes more sense to remove the lock helper inlines and write out spin_*lock() at all call sites.

Just a heads up that I didn't miss this patch. Just it has some bugs and I figured that rather than explaining to you subtleties of jh lifetime it is easier to fix up the problems myself since you're probably not keen on becoming jbd2 developer ;)... which was more complex than I thought so I'm not completely done yet. Hopefuly tomorrow.

I'm curious where I was too naive :)

Thanks for having a look!

tglx

reviewer
#13 lore raw

Re: [patch V2 6/7] fs/jbd2: Make state lock a spinlock

Quoted 7 lines · Thomas Gleixner
Quoted 6 lines · Jan Kara
Quoted 4 lines · Thomas Gleixner

As almost all functions which use this lock have a journal head pointer readily available, it makes more sense to remove the lock helper inlines and write out spin_*lock() at all call sites.

Quoted 7 lines
Quoted 5 lines

Just a heads up that I didn't miss this patch. Just it has some bugs and I figured that rather than explaining to you subtleties of jh lifetime it is easier to fix up the problems myself since you're probably not keen on becoming jbd2 developer ;)... which was more complex than I thought so I'm not completely done yet. Hopefuly tomorrow.

I'm curious where I was too naive :)

Well, the most obvious where places where the result ended up being like

	jbd2_journal_put_journal_head(jh);
	spin_unlock(&jh->state_lock);

That's possible use-after-free.

But there were also other more subtle places where jbd2_journal_put_journal_head() was not directly visible as it was hidden inside journal list handling functions such as __jbd2_journal_refile_buffer() or so. And these needed some more work.

Anyway, I'm now done fixing up the patch, doing some xfstests runs to verify things didn't break in any obvious way...

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#14 lore raw

Re: [patch V2 6/7] fs/jbd2: Make state lock a spinlock

Quoted 14 lines · Jan Kara
Quoted 13 lines · Thomas Gleixner
Quoted 11 lines · Jan Kara
Quoted 4 lines · Thomas Gleixner

As almost all functions which use this lock have a journal head pointer readily available, it makes more sense to remove the lock helper inlines and write out spin_*lock() at all call sites.

Just a heads up that I didn't miss this patch. Just it has some bugs and I figured that rather than explaining to you subtleties of jh lifetime it is easier to fix up the problems myself since you're probably not keen on becoming jbd2 developer ;)... which was more complex than I thought so I'm not completely done yet. Hopefuly tomorrow.

Quoted 8 lines
Quoted 1 line

I'm curious where I was too naive :)

Well, the most obvious where places where the result ended up being like

	jbd2_journal_put_journal_head(jh);
	spin_unlock(&jh->state_lock);

That's possible use-after-free.

Duh yes.

Quoted 7 lines

But there were also other more subtle places where jbd2_journal_put_journal_head() was not directly visible as it was hidden inside journal list handling functions such as __jbd2_journal_refile_buffer() or so. And these needed some more work.

Anyway, I'm now done fixing up the patch, doing some xfstests runs to verify things didn't break in any obvious way...

Very appreciated.

Thanks,

tglx

#15 lore raw

PATCH v2 7/7 fs/jbd2: Free journal head outside of locked region

In reply to Thomas Gleixner

On PREEMPT_RT bit-spinlocks have the same semantics as on PREEMPT_RT=n, i.e. they disable preemption. That means functions which are not safe to be called in preempt disabled context on RT trigger a might_sleep() assert.

The journal head bit spinlock is mostly held for short code sequences with trivial RT safe functionality, except for one place:

jbd2_journal_put_journal_head() invokes __journal_remove_journal_head() with the journal head bit spinlock held. __journal_remove_journal_head() invokes kmem_cache_free() which must not be called with preemption disabled on RT.

Jan suggested to rework the removal function so the actual free happens outside the bit-spinlocked region.

Split it into two parts:

- Do the sanity checks and the buffer head detach under the lock

- Do the actual free after dropping the lock

There is error case handling in the free part which needs to dereference the b_size field of the now detached buffer head. Due to paranoia (caused by ignorance) the size is retrieved in the detach function and handed into the free function. Might be over-engineered, but better safe than sorry.

This makes the journal head bit-spinlock usage RT compliant and also avoids nested locking which is not covered by lockdep.

---
V2: New patch

---
 fs/jbd2/journal.c |   28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)
fs/jbd2/journal.c +20 -8
@@ -2521,9 +2521,10 @@ struct journal_head *jbd2_journal_grab_j
 	return jh;
 }
 
-static void __journal_remove_journal_head(struct buffer_head *bh)
+static size_t __journal_remove_journal_head(struct buffer_head *bh)
 {
 	struct journal_head *jh = bh2jh(bh);
+	size_t b_size = READ_ONCE(bh->b_size);
 
 	J_ASSERT_JH(jh, jh->b_jcount >= 0);
 	J_ASSERT_JH(jh, jh->b_transaction == NULL);
@@ -2533,17 +2534,25 @@ static void __journal_remove_journal_hea
 	J_ASSERT_BH(bh, buffer_jbd(bh));
 	J_ASSERT_BH(bh, jh2bh(jh) == bh);
 	BUFFER_TRACE(bh, "remove journal_head");
+
+	/* Unlink before dropping the lock */
+	bh->b_private = NULL;
+	jh->b_bh = NULL;	/* debug, really */
+	clear_buffer_jbd(bh);
+
+	return b_size;
+}
+
+static void journal_release_journal_head(struct journal_head *jh, size_t b_size)
+{
 	if (jh->b_frozen_data) {
 		printk(KERN_WARNING "%s: freeing b_frozen_data\n", __func__);
-		jbd2_free(jh->b_frozen_data, bh->b_size);
+		jbd2_free(jh->b_frozen_data, b_size);
 	}
 	if (jh->b_committed_data) {
 		printk(KERN_WARNING "%s: freeing b_committed_data\n", __func__);
-		jbd2_free(jh->b_committed_data, bh->b_size);
+		jbd2_free(jh->b_committed_data, b_size);
 	}
-	bh->b_private = NULL;
-	jh->b_bh = NULL;	/* debug, really */
-	clear_buffer_jbd(bh);
 	journal_free_journal_head(jh);
 }
 
@@ -2559,11 +2568,14 @@ void jbd2_journal_put_journal_head(struc
 	J_ASSERT_JH(jh, jh->b_jcount > 0);
 	--jh->b_jcount;
 	if (!jh->b_jcount) {
-		__journal_remove_journal_head(bh);
+		size_t b_size = __journal_remove_journal_head(bh);
+
 		jbd_unlock_bh_journal_head(bh);
+		journal_release_journal_head(jh, b_size);
 		__brelse(bh);
-	} else
+	} else {
 		jbd_unlock_bh_journal_head(bh);
+	}
 }
 
 /*
reviewer
#16 lore raw

Re: [patch V2 7/7] fs/jbd2: Free journal head outside of locked region

Quoted 26 lines · Thomas Gleixner

On PREEMPT_RT bit-spinlocks have the same semantics as on PREEMPT_RT=n, i.e. they disable preemption. That means functions which are not safe to be called in preempt disabled context on RT trigger a might_sleep() assert.

The journal head bit spinlock is mostly held for short code sequences with trivial RT safe functionality, except for one place:

jbd2_journal_put_journal_head() invokes __journal_remove_journal_head() with the journal head bit spinlock held. __journal_remove_journal_head() invokes kmem_cache_free() which must not be called with preemption disabled on RT.

Jan suggested to rework the removal function so the actual free happens outside the bit-spinlocked region.

Split it into two parts:

- Do the sanity checks and the buffer head detach under the lock

- Do the actual free after dropping the lock

There is error case handling in the free part which needs to dereference the b_size field of the now detached buffer head. Due to paranoia (caused by ignorance) the size is retrieved in the detach function and handed into the free function. Might be over-engineered, but better safe than sorry.

Quoted 8 lines

This makes the journal head bit-spinlock usage RT compliant and also avoids nested locking which is not covered by lockdep.

Looks mostly good. Just a small suggestion for simplification below:

fs/jbd2/journal.c void jbd2_journal_put_journal_head(struc · line 2575

Quoted 10 lines
fs/jbd2/journal.c +3 -1
@@ -2559,11 +2568,14 @@ void jbd2_journal_put_journal_head(struc
 	J_ASSERT_JH(jh, jh->b_jcount > 0);
 	--jh->b_jcount;
 	if (!jh->b_jcount) {
-		__journal_remove_journal_head(bh);
+		size_t b_size = __journal_remove_journal_head(bh);
+
 		jbd_unlock_bh_journal_head(bh);
+		journal_release_journal_head(jh, b_size);
 		__brelse(bh);

The bh is pinned until you call __brelse(bh) above and bh->b_size doesn't change during the lifetime of the buffer. So there's no need of fetching bh->b_size in __journal_remove_journal_head() and passing it back. You can just:

		journal_release_journal_head(jh, bh->b_size);

fs/jbd2/journal.c void jbd2_journal_put_journal_head(struc · line 2580

Quoted 6 lines
fs/jbd2/journal.c +2 -1
-	} else
+	} else {
 		jbd_unlock_bh_journal_head(bh);
+	}
 }
 

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#17 lore raw

Re: [patch V2 7/7] fs/jbd2: Free journal head outside of locked region

Quoted 22 lines · Jan Kara
Quoted 3 lines · Thomas Gleixner

On PREEMPT_RT bit-spinlocks have the same semantics as on PREEMPT_RT=n, i.e. they disable preemption. That means functions which are not safe to be called in preempt disabled context on RT trigger a might_sleep() assert.

Looks mostly good. Just a small suggestion for simplification below:

fs/jbd2/journal.c void jbd2_journal_put_journal_head(struc · line 2575

Quoted 10 lines
fs/jbd2/journal.c +3 -1
@@ -2559,11 +2568,14 @@ void jbd2_journal_put_journal_head(struc
 	J_ASSERT_JH(jh, jh->b_jcount > 0);
 	--jh->b_jcount;
 	if (!jh->b_jcount) {
-		__journal_remove_journal_head(bh);
+		size_t b_size = __journal_remove_journal_head(bh);
+
 		jbd_unlock_bh_journal_head(bh);
+		journal_release_journal_head(jh, b_size);
 		__brelse(bh);

The bh is pinned until you call __brelse(bh) above and bh->b_size doesn't change during the lifetime of the buffer. So there's no need of fetching bh->b_size in __journal_remove_journal_head() and passing it back. You can just:

Quoted 1 line
		journal_release_journal_head(jh, bh->b_size);

Ah. Nice. I assumed that this would be possible, but then my ignorance induced paranoia won.

Thanks,

tglx

#18 lore raw
In reply to Thomas Gleixner

Hi Thomas,

did you look into killing bіt spinlocks as a public API instead?

The main users seems to be buffer heads, which are so bloated that an extra spinlock doesn't really matter anyway.

The list_bl and rhashtable uses kinda make sense to be, but they are pretty nicely abstracted away anyway. The remaining users look pretty questionable to start with.

#19 lore raw

Christoph,

Quoted 1 line · Christoph Hellwig

did you look into killing bіt spinlocks as a public API instead?

Last time I did, there was resistance :)

But I'm happy to try again.

Quoted 6 lines

The main users seems to be buffer heads, which are so bloated that an extra spinlock doesn't really matter anyway.

The list_bl and rhashtable uses kinda make sense to be, but they are pretty nicely abstracted away anyway. The remaining users look pretty questionable to start with.

What about the page lock?

  mm/slub.c:      bit_spin_lock(PG_locked, &page->flags);

Thanks,

tglx

#20 lore raw
Quoted 1 line · Thomas Gleixner

Last time I did, there was resistance :)

Do you have a pointer? Note that in the buffer head case maybe a hash lock based on the page address is even better, as we only ever use the lock in the first buffer head of a page anyway..

Quoted 3 lines

What about the page lock?

  mm/slub.c:      bit_spin_lock(PG_locked, &page->flags);

One caller ouf of a gazillion that spins on the page lock instead of sleepign on it like everyone else. That should not have passed your smell test to start with :)

#21 lore raw
Quoted 6 lines · Christoph Hellwig
Quoted 1 line · Thomas Gleixner

Last time I did, there was resistance :)

Do you have a pointer? Note that in the buffer head case maybe a hash lock based on the page address is even better, as we only ever use the lock in the first buffer head of a page anyway..

I need to search my archives, but I'm on a spotty and slow connection right now. Will do so when back home.

Quoted 7 lines
Quoted 3 lines

What about the page lock?

  mm/slub.c:      bit_spin_lock(PG_locked, &page->flags);

One caller ouf of a gazillion that spins on the page lock instead of sleepign on it like everyone else. That should not have passed your smell test to start with :)

I surely stared at it, but that cannot sleep. It's in the middle of a preempt and interrupt disabled region and used on architectures which do not support CMPXCHG_DOUBLE and ALIGNED_STRUCT_PAGE ...

Thanks,

tglx

#22 lore raw
Quoted 9 lines · Thomas Gleixner
Quoted 5 lines
Quoted 1 line
  mm/slub.c:      bit_spin_lock(PG_locked, &page->flags);

One caller ouf of a gazillion that spins on the page lock instead of sleepign on it like everyone else. That should not have passed your smell test to start with :)

I surely stared at it, but that cannot sleep. It's in the middle of a preempt and interrupt disabled region and used on architectures which do not support CMPXCHG_DOUBLE and ALIGNED_STRUCT_PAGE ...

I know. But the problem here is that normally PG_locked is used together with wait_on_page_bit_*, but this one instances uses the bit spinlock helpers. This is the equivalent of calling spin_lock on a struct mutex rather than having a mutex_lock_spin helper for this case. Does SLUB work on -rt at all?

#23 lore raw
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · Thomas Gleixner
Quoted 5 lines
Quoted 1 line
  mm/slub.c:      bit_spin_lock(PG_locked, &page->flags);

One caller ouf of a gazillion that spins on the page lock instead of sleepign on it like everyone else. That should not have passed your smell test to start with :)

Quoted 8 lines
Quoted 3 lines

I surely stared at it, but that cannot sleep. It's in the middle of a preempt and interrupt disabled region and used on architectures which do not support CMPXCHG_DOUBLE and ALIGNED_STRUCT_PAGE ...

I know. But the problem here is that normally PG_locked is used together with wait_on_page_bit_*, but this one instances uses the bit spinlock helpers. This is the equivalent of calling spin_lock on a struct mutex rather than having a mutex_lock_spin helper for this case.

Yes, I know :(

Quoted 1 line

Does SLUB work on -rt at all?

It's the only allocator we support with a few tweaks :)

Thanks,

tglx

#24 lore raw
Quoted 6 lines · Thomas Gleixner
Quoted 4 lines

I know. But the problem here is that normally PG_locked is used together with wait_on_page_bit_*, but this one instances uses the bit spinlock helpers. This is the equivalent of calling spin_lock on a struct mutex rather than having a mutex_lock_spin helper for this case.

Yes, I know :(

But this means we should exclude slub from the bit_spin_lock removal. It really should use it's own version of it anyhow insted of pretending that the page lock is a bit spinlock.

Quoted 4 lines
Quoted 2 lines

Does SLUB work on -rt at all?

It's the only allocator we support with a few tweaks :)

What do you do about this particular piece of code there?

#25 lore raw
Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Thomas Gleixner
Quoted 4 lines

I know. But the problem here is that normally PG_locked is used together with wait_on_page_bit_*, but this one instances uses the bit spinlock helpers. This is the equivalent of calling spin_lock on a struct mutex rather than having a mutex_lock_spin helper for this case.

Yes, I know :(

But this means we should exclude slub from the bit_spin_lock removal. It really should use it's own version of it anyhow insted of pretending that the page lock is a bit spinlock.

But PG_locked isn't used as a mutex _when the page is allocated by slab_. Yes, every other user uses PG_locked as a mutex, but I don't see why that should constrain slub's usage of PG_locked.

#26 lore raw
In reply to Christoph Hellwig
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines
Quoted 1 line

Does SLUB work on -rt at all?

It's the only allocator we support with a few tweaks :)

What do you do about this particular piece of code there?

This part remains untouched. This "lock" is acquired within ->list_lock which is a raw_spinlock_t and disables preemption/interrupts on -RT.

Sebastian