Linux 4.9.61-rt52-rc2
Subsystems (8)
cover Linux 4.9.61-rt52-rc2
Dear RT Folks,
This is the RT stable review cycle of patch 4.9.61-rt52-rc2.
Please scream at me if I messed something up. Please test the patches too.
The -rc release will be uploaded to kernel.org and will be deleted when the final release is out. This is just a review release (or release candidate).
The pre-releases will not be pushed to the git repository, only the final release is.
If all goes well, this patch will be converted to the next main release on 11/27/2017.
Note, I realized I only backported from v4.13.10-rt3 and not v4.13.13-rt5. Added a few more patches.
Enjoy,
-- Steve
To build 4.9.61-rt52-rc2 directly, the following patches should be applied:
http://www.kernel.org/pub/linux/kernel/v4.x/linux-4.9.tar.xz
http://www.kernel.org/pub/linux/kernel/v4.x/patch-4.9.61.xz
http://www.kernel.org/pub/linux/kernel/projects/rt/4.9/patch-4.9.61-rt52-rc2.patch.xz
You can also build from 4.9.61-rt51 by applying the incremental patch:
http://www.kernel.org/pub/linux/kernel/projects/rt/4.9/incr/patch-4.9.61-rt51-rt52-rc2.patch.xz
Changes from 4.9.61-rt51:
---
Haris Okanovic (1):
tpm_tis: fix stall after iowrite*()s
Mike Galbraith (1):
drivers/zram: fix zcomp_stream_get() smp_processor_id() use in preemptible code
Mikulas Patocka (1):
locking/rt-mutex: fix deadlock in device mapper / block-IO
Sebastian Andrzej Siewior (3):
fs/dcache: disable preemption on i_dir_seq's write side
fs: convert two more BH_Uptodate_Lock related bitspinlocks md/raid5: do not disable interrupts
Steven Rostedt (VMware) (1):
Linux 4.9.61-rt52-rc2
----
drivers/block/zram/zcomp.c | 3 ++- drivers/char/tpm/tpm_tis.c | 29 +++++++++++++++++++++++++++-- drivers/md/raid5.c | 4 ++-- fs/dcache.c | 12 +++++++----- fs/ext4/page-io.c | 6 ++---- fs/inode.c | 2 +- fs/libfs.c | 6 ++++-- fs/xfs/xfs_aops.c | 6 ++---- include/linux/fs.h | 2 +- kernel/locking/rtmutex.c | 13 +++++++++++++ localversion-rt | 2 +- 11 files changed, 62 insertions(+), 23 deletions(-)
PATCH 1/7 drivers/zram: fix zcomp_stream_get() smp_processor_id() use in preemptible code
4.9.61-rt52-rc2 stable review patch.
If anyone has any objections, please let me know.
------------------
From: Mike Galbraith <[email protected]>
Use get_local_ptr() instead this_cpu_ptr() to avoid a warning regarding smp_processor_id() in preemptible code.raw_cpu_ptr() would be fine, too because the per-CPU data structure is protected with a spin lock so it does not matter much if we take the other one.
--- drivers/block/zram/zcomp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/block/zram/zcomp.c +2 -1
@@ -120,7 +120,7 @@ struct zcomp_strm *zcomp_stream_get(struct zcomp *comp) {struct zcomp_strm *zstrm; - zstrm = *this_cpu_ptr (comp->stream); + zstrm = *get_local_ptr (comp->stream);spin_lock (&zstrm->zcomp_lock);return zstrm; }
@@ -131,6 +131,7 @@ void zcomp_stream_put(struct zcomp *comp) zstrm = *this_cpu_ptr (comp->stream);spin_unlock (&zstrm->zcomp_lock); +put_local_ptr (zstrm); } int zcomp_compress(struct zcomp_strm *zstrm,
Signature · 1 line
2.13.2
PATCH 2/7 fs/dcache: disable preemption on i_dir_seqs write side
In reply to Steven Rostedt4.9.61-rt52-rc2 stable review patch.
If anyone has any objections, please let me know.
------------------
From: Sebastian Andrzej Siewior <[email protected]>
i_dir_seq is an opencoded seqcounter. Based on the code it looks like we could have two writers in parallel despite the fact that the d_lock is held. The problem is that during the write process on RT the preemption is still enabled and if this process is interrupted by a reader with RT priority then we lock up.
To avoid that lock up I am disabling the preemption during the update. The rename of i_dir_seq is here to ensure to catch new write sides in future.
--- fs/dcache.c | 12 +++++++----- fs/inode.c | 2 +- fs/libfs.c | 6 ++++-- include/linux/fs.h | 2 +- 4 files changed, 13 insertions(+), 9 deletions(-)
fs/dcache.c +7 -5
@@ -2409,9 +2409,10 @@ EXPORT_SYMBOL(d_rehash);static inline unsigned start_dir_add (struct inode *dir) { +preempt_disable_rt ();for (;;) { -unsigned n = dir->i_dir_seq; -if (!(n &1 ) &&cmpxchg (&dir->i_dir_seq, n, n +1 ) == n) +unsigned n = dir->__i_dir_seq; +if (!(n &1 ) &&cmpxchg (&dir->__i_dir_seq, n, n +1 ) == n)return n;cpu_relax (); }
@@ -2419,7 +2420,8 @@ static inline unsigned start_dir_add(struct inode *dir)static inline void end_dir_add (struct inode *dir,unsigned n) { -smp_store_release (&dir->i_dir_seq, n +2 ); +smp_store_release (&dir->__i_dir_seq, n +2 ); +preempt_enable_rt (); }static void d_wait_lookup (struct dentry *dentry)
@@ -2455,7 +2457,7 @@ struct dentry *d_alloc_parallel(struct dentry *parent, retry:rcu_read_lock (); - seq =smp_load_acquire (&parent->d_inode->i_dir_seq) & ~1 ; + seq =smp_load_acquire (&parent->d_inode->__i_dir_seq) & ~1 ; r_seq =read_seqbegin (&rename_lock); dentry =__d_lookup_rcu (parent, name, &d_seq);if (unlikely (dentry)) {
@@ -2477,7 +2479,7 @@ struct dentry *d_alloc_parallel(struct dentry *parent, goto retry; }hlist_bl_lock (b); -if (unlikely (parent->d_inode->i_dir_seq != seq)) { +if (unlikely (parent->d_inode->__i_dir_seq != seq)) {hlist_bl_unlock (b);rcu_read_unlock (); goto retry;
fs/inode.c +1 -1
@@ -153,7 +153,7 @@ int inode_init_always(struct super_block *sb, struct inode *inode) inode->i_bdev =NULL ; inode->i_cdev =NULL ; inode->i_link =NULL ; - inode->i_dir_seq =0 ; + inode->__i_dir_seq =0 ; inode->i_rdev =0 ; inode->dirtied_when =0 ;
fs/libfs.c +4 -2
@@ -89,7 +89,7 @@ static struct dentry *next_positive(struct dentry *parent,struct list_head *from, int count) { - unsigned *seq = &parent->d_inode->i_dir_seq, n; + unsigned *seq = &parent->d_inode->__i_dir_seq, n;struct dentry *res;struct list_head *p;bool skipped;
@@ -122,8 +122,9 @@ static struct dentry *next_positive(struct dentry *parent,static void move_cursor (struct dentry *cursor,struct list_head *after) {struct dentry *parent = cursor->d_parent; -unsigned n, *seq = &parent->d_inode->i_dir_seq; +unsigned n, *seq = &parent->d_inode->__i_dir_seq;spin_lock (&parent->d_lock); +preempt_disable_rt ();for (;;) { n = *seq;if (!(n &1 ) &&cmpxchg (seq, n, n +1 ) == n)
@@ -136,6 +137,7 @@ static void move_cursor(struct dentry *cursor, struct list_head *after)else list_add_tail (&cursor->d_child, &parent->d_subdirs);smp_store_release (seq, n +2 ); +preempt_enable_rt ();spin_unlock (&parent->d_lock); }
include/linux/fs.h +1 -1
@@ -688,7 +688,7 @@ struct inode {struct block_device *i_bdev;struct cdev *i_cdev;char *i_link; -unsigned i_dir_seq; +unsigned __i_dir_seq; };__u32 i_generation;
Signature · 1 line
2.13.2
PATCH 3/7 tpm_tis: fix stall after iowrite*()s
In reply to Steven Rostedt4.9.61-rt52-rc2 stable review patch.
If anyone has any objections, please let me know.
------------------
From: Haris Okanovic <[email protected]>
ioread8() operations to TPM MMIO addresses can stall the cpu when immediately following a sequence of iowrite*()'s to the same region.
For example, cyclitest measures ~400us latency spikes when a non-RT usermode application communicates with an SPI-based TPM chip (Intel Atom E3940 system, PREEMPT_RT_FULL kernel). The spikes are caused by a stalling ioread8() operation following a sequence of 30+ iowrite8()s to the same address. I believe this happens because the write sequence is buffered (in cpu or somewhere along the bus), and gets flushed on the first LOAD instruction (ioread*()) that follows.
The enclosed change appears to fix this issue: read the TPM chip's access register (status code) after every iowrite*() operation to amortize the cost of flushing data to chip across multiple instructions.
--- drivers/char/tpm/tpm_tis.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)
drivers/char/tpm/tpm_tis.c +27 -2
@@ -50,6 +50,31 @@ static inline struct tpm_tis_tcg_phy *to_tpm_tis_tcg_phy(struct tpm_tis_data *da return container_of(data, struct tpm_tis_tcg_phy, priv); } +#ifdef CONFIG_PREEMPT_RT_FULL +/* +* Flushes previous write operations to chip so that a subsequent +* ioread*()s won't stall a cpu. +*/ +static inline void tpm_tis_flush (void __iomem *iobase) +{ +ioread8 (iobase +TPM_ACCESS (0 )); +} +#else +#define tpm_tis_flush (iobase) do { } while (0) +#endif + +static inline void tpm_tis_iowrite8 (u8 b,void __iomem *iobase,u32 addr) +{ +iowrite8 (b, iobase + addr); +tpm_tis_flush (iobase); +} + +static inline void tpm_tis_iowrite32 (u32 b,void __iomem *iobase,u32 addr) +{ +iowrite32 (b, iobase + addr); +tpm_tis_flush (iobase); +} +static bool interrupts = true;module_param (interrupts, bool,0444 );MODULE_PARM_DESC (interrupts,"Enable interrupts" );
@@ -103,7 +128,7 @@ static int tpm_tcg_write_bytes(struct tpm_tis_data *data, u32 addr, u16 len,struct tpm_tis_tcg_phy *phy =to_tpm_tis_tcg_phy (data);while (len--) -iowrite8 (*value++, phy->iobase + addr); +tpm_tis_iowrite8 (*value++, phy->iobase, addr);return 0 ; }
@@ -127,7 +152,7 @@ static int tpm_tcg_write32(struct tpm_tis_data *data, u32 addr, u32 value) {struct tpm_tis_tcg_phy *phy =to_tpm_tis_tcg_phy (data); -iowrite32 (value, phy->iobase + addr); +tpm_tis_iowrite32 (value, phy->iobase, addr);return 0 ; }
Signature · 1 line
2.13.2
PATCH 4/7 fs: convert two more BH_Uptodate_Lock related bitspinlocks
In reply to Steven Rostedt4.9.61-rt52-rc2 stable review patch.
If anyone has any objections, please let me know.
------------------
From: Sebastian Andrzej Siewior <[email protected]>
We convert all BH_Uptodate_Lock based bit-spinlocks to use bh_uptodate_lock_irqsave() instead. Those two were introduced after the initial change in -RT and were not noticed before.
--- fs/ext4/page-io.c | 6 ++---- fs/xfs/xfs_aops.c | 6 ++---- 2 files changed, 4 insertions(+), 8 deletions(-)
fs/ext4/page-io.c +2 -4
@@ -95,8 +95,7 @@ static void ext4_finish_bio(struct bio *bio)* We check all buffers in the page under BH_Uptodate_Lock * to avoid races with other end io clearing async_write flags */ -local_irq_save (flags); -bit_spin_lock (BH_Uptodate_Lock, &head->b_state); + flags =bh_uptodate_lock_irqsave (head); do { if (bh_offset(bh) < bio_start || bh_offset(bh) + bh->b_size > bio_end) {
@@ -108,8 +107,7 @@ static void ext4_finish_bio(struct bio *bio)if (bio->bi_error)buffer_io_error (bh); }while ((bh = bh->b_this_page) != head); -bit_spin_unlock (BH_Uptodate_Lock, &head->b_state); -local_irq_restore (flags); +bh_uptodate_unlock_irqrestore (head, flags);if (!under_io) {#ifdef CONFIG_EXT4_FS_ENCRYPTION if (data_page)
fs/xfs/xfs_aops.c +2 -4
@@ -112,8 +112,7 @@ xfs_finish_page_writeback(ASSERT (bvec->bv_offset + bvec->bv_len <=PAGE_SIZE );ASSERT ((bvec->bv_len & (i_blocksize (inode) -1 )) ==0 ); -local_irq_save (flags); -bit_spin_lock (BH_Uptodate_Lock, &head->b_state); + flags =bh_uptodate_lock_irqsave (head); do { if (off >= bvec->bv_offset && off < bvec->bv_offset + bvec->bv_len) {
@@ -136,8 +135,7 @@ xfs_finish_page_writeback( } off += bh->b_size; }while ((bh = bh->b_this_page) != head); -bit_spin_unlock (BH_Uptodate_Lock, &head->b_state); -local_irq_restore (flags); +bh_uptodate_unlock_irqrestore (head, flags);if (!busy)end_page_writeback (bvec->bv_page);
Signature · 1 line
2.13.2
PATCH 5/7 locking/rt-mutex: fix deadlock in device mapper / block-IO
In reply to Steven Rostedt4.9.61-rt52-rc2 stable review patch.
If anyone has any objections, please let me know.
------------------
From: Mikulas Patocka <[email protected]>
When some block device driver creates a bio and submits it to another block device driver, the bio is added to current->bio_list (in order to avoid unbounded recursion).
However, this queuing of bios can cause deadlocks, in order to avoid them, device mapper registers a function flush_current_bio_list. This function is called when device mapper driver blocks. It redirects bios queued on current->bio_list to helper workqueues, so that these bios can proceed even if the driver is blocked.
The problem with CONFIG_PREEMPT_RT_FULL is that when the device mapper driver blocks, it won't call flush_current_bio_list (because tsk_is_pi_blocked returns true in sched_submit_work), so deadlocks in block device stack can happen.
Note that we can't call blk_schedule_flush_plug if tsk_is_pi_blocked returns true - that would cause BUG_ON(rt_mutex_real_waiter(task->pi_blocked_on)) in task_blocks_on_rt_mutex when flush_current_bio_list attempts to take a spinlock.
So the proper fix is to call blk_schedule_flush_plug in rt_mutex_fastlock, when fast acquire failed and when the task is about to block.
CC: [email protected]
[bigeasy: The deadlock is not device-mapper specific, it can also occur
in plain EXT4]
--- kernel/locking/rtmutex.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
kernel/locking/rtmutex.c +13 -0
@@ -22,6 +22,7 @@#include <linux/sched/deadline.h> #include <linux/timer.h> #include <linux/ww_mutex.h> +#include <linux/blkdev.h> #include "rtmutex_common.h"
@@ -1968,6 +1969,15 @@ rt_mutex_fastlock(struct rt_mutex *lock, int state,if (likely (rt_mutex_cmpxchg_acquire (lock,NULL , current)))return 0 ; +/* +* If rt_mutex blocks, the function sched_submit_work will not call +* blk_schedule_flush_plug (because tsk_is_pi_blocked would be true). +* We must call blk_schedule_flush_plug here, if we don't call it, +* a deadlock in device mapper may happen. +*/ +if (unlikely (blk_needs_flush_plug (current))) +blk_schedule_flush_plug (current); +return slowfn (lock, state,NULL ,RT_MUTEX_MIN_CHAINWALK , ww_ctx); }
@@ -1985,6 +1995,9 @@ rt_mutex_timed_fastlock(struct rt_mutex *lock, int state, likely(rt_mutex_cmpxchg_acquire(lock, NULL, current)))return 0 ; +if (unlikely (blk_needs_flush_plug (current))) +blk_schedule_flush_plug (current); +return slowfn (lock, state, timeout, chwalk, ww_ctx); }
Signature · 1 line
2.13.2
PATCH 6/7 md/raid5: do not disable interrupts
In reply to Steven Rostedt4.9.61-rt52-rc2 stable review patch.
If anyone has any objections, please let me know.
------------------
From: Sebastian Andrzej Siewior <[email protected]>
|BUG: sleeping function called from invalid context at kernel/locking/rtmutex.c:974 |in_atomic(): 0, irqs_disabled(): 1, pid: 2992, name: lvm |CPU: 2 PID: 2992 Comm: lvm Not tainted 4.13.10-rt3+ #54 |Call Trace: | dump_stack+0x4f/0x65 | ___might_sleep+0xfc/0x150 | atomic_dec_and_spin_lock+0x3c/0x80 | raid5_release_stripe+0x73/0x110 | grow_one_stripe+0xce/0xf0 | setup_conf+0x841/0xaa0 | raid5_run+0x7e7/0xa40 | md_run+0x515/0xaf0 | raid_ctr+0x147d/0x25e0 | dm_table_add_target+0x155/0x320 | table_load+0x103/0x320 | ctl_ioctl+0x1d9/0x510 | dm_ctl_ioctl+0x9/0x10 | do_vfs_ioctl+0x8e/0x670 | SyS_ioctl+0x3c/0x70 | entry_SYSCALL_64_fastpath+0x17/0x98
The interrupts were disabled because ->device_lock is taken with interrupts disabled.
--- drivers/md/raid5.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/md/raid5.c +2 -2
@@ -429,7 +429,7 @@ void raid5_release_stripe(struct stripe_head *sh)md_wakeup_thread (conf->mddev->thread);return ; slow_path: -local_irq_save (flags); +local_irq_save_nort (flags);/* we are ok here if STRIPE_ON_RELEASE_LIST is set or not */ if (atomic_dec_and_lock (&sh->count, &conf->device_lock)) {INIT_LIST_HEAD (&list);
@@ -438,7 +438,7 @@ void raid5_release_stripe(struct stripe_head *sh)spin_unlock (&conf->device_lock);release_inactive_stripe_list (conf, &list, hash); } -local_irq_restore (flags); +local_irq_restore_nort (flags); }static inline void remove_hash (struct stripe_head *sh)
Signature · 1 line
2.13.2
PATCH 7/7 Linux 4.9.61-rt52-rc2
In reply to Steven Rostedt4.9.61-rt52-rc2 stable review patch.
If anyone has any objections, please let me know.
------------------
From: "Steven Rostedt (VMware)" <[email protected]>
--- localversion-rt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
localversion-rt +1 -1
@@ -1 +1 @@ --rt51 +-rt52-rc2
Signature · 1 line
2.13.2