Skip to contentSkip to conversation

backing-dev: replace private thread pool with workqueue

PATCH In review Tejun Heo 4 participants 15 messages Sep 6, 2010 – Nov 27, 2010

Subsystems (6)
lore
Share & tools
lore
author maintainer
#2 lore raw

PATCH 2/5 backing-dev: kill unused bdi_writeback->nr

In reply to Tejun Heo

bdi_writeback->nr is no longer used. Kill it.

---
 include/linux/backing-dev.h |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)
include/linux/backing-dev.h +0 -1
index 35b0074..97842ab 100644
@@ -47,7 +47,6 @@ enum bdi_stat_item {
 
 struct bdi_writeback {
 	struct backing_dev_info *bdi;	/* our parent bdi */
-	unsigned int nr;
 
 	unsigned long last_old_flush;	/* last old data flush */
 	unsigned long last_active;	/* last time bdi thread was active */
Signature · 1 line
1.7.1

Tejun Heo

author maintainer
#0 lore raw

PATCH 1/1 backing-dev: replace private thread pool with workqueue

Hello,

This patchset converts backing-dev to use workqueue instead of the private thread pool. The conversion is pretty straight forward. The only notable thing is that the writeback work skips usual bdi->work_list processing if it's being executed from the rescuer thread. Being executed from the rescuer thread means that the system is under memory pressure and it just starts writeouts by calling bdi_flush_io() and reschedules itself if necessary. This effectly is the same to what bdi_forker_thread() did when it failed to create a thread for a pending BDI.

This patch contains the following five patches.

0001-workqueue-implement-workqueue_on_rescuer.patch 0002-backing-dev-kill-unused-bdi_writeback-nr.patch 0003-backing-dev-replace-private-thread-pool-with-workque.patch 0004-backing-dev-update-trace-points.patch 0005-backing-dev-replace-sync_supers_tsk-timer-with-a-del.patch

It's currently based on v2.6.36-rc3 + wq#for-linus. Once Linus pulls from wq#for-linus, I'll rebase these, but the patches apply on v2.6.36-rc3 too. These patches are also available in the following git tree,

git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.git wq-wb

and contains the following changes.

 fs/fs-writeback.c                |   91 +----------
 include/linux/backing-dev.h      |   18 --
 include/linux/workqueue.h        |    1 
 include/trace/events/writeback.h |    9 -
 kernel/workqueue.c               |   15 +
 mm/backing-dev.c                 |  302 ++++++++++-----------------------------
 6 files changed, 111 insertions(+), 325 deletions(-)

Thanks.

--
tejun

author maintainer
#5 lore raw

PATCH 4/5 backing-dev: update trace points

* writeback_nothread is replaced with writeback_rescuer, which is triggered when a writeback work is called from the bdi_wq rescuer.

* writeback_wake[_forker]_thread are replaced with writeback_timer.

* writeback_thread_start/stop are replaced with writeback_start/stop which are triggered when a writeback work starts and finishes execution.

---
 include/trace/events/writeback.h |    9 ++++-----
 mm/backing-dev.c                 |    6 ++++++
 2 files changed, 10 insertions(+), 5 deletions(-)
include/trace/events/writeback.h +4 -5
index f345f66..58e1f01 100644
@@ -46,7 +46,7 @@ DECLARE_EVENT_CLASS(writeback_work_class,
 DEFINE_EVENT(writeback_work_class, name, \
 	TP_PROTO(struct backing_dev_info *bdi, struct wb_writeback_work *work), \
 	TP_ARGS(bdi, work))
-DEFINE_WRITEBACK_WORK_EVENT(writeback_nothread);
+DEFINE_WRITEBACK_WORK_EVENT(writeback_rescuer);
 DEFINE_WRITEBACK_WORK_EVENT(writeback_queue);
 DEFINE_WRITEBACK_WORK_EVENT(writeback_exec);
 
@@ -81,12 +81,11 @@ DEFINE_EVENT(writeback_class, name, \
 	TP_ARGS(bdi))
 
 DEFINE_WRITEBACK_EVENT(writeback_nowork);
-DEFINE_WRITEBACK_EVENT(writeback_wake_thread);
-DEFINE_WRITEBACK_EVENT(writeback_wake_forker_thread);
+DEFINE_WRITEBACK_EVENT(writeback_timer);
 DEFINE_WRITEBACK_EVENT(writeback_bdi_register);
 DEFINE_WRITEBACK_EVENT(writeback_bdi_unregister);
-DEFINE_WRITEBACK_EVENT(writeback_thread_start);
-DEFINE_WRITEBACK_EVENT(writeback_thread_stop);
+DEFINE_WRITEBACK_EVENT(writeback_start);
+DEFINE_WRITEBACK_EVENT(writeback_stop);
 
 DECLARE_EVENT_CLASS(wbc_class,
 	TP_PROTO(struct writeback_control *wbc, struct backing_dev_info *bdi),
mm/backing-dev.c +6 -0
index e874916..3b2a657 100644
@@ -314,6 +314,7 @@ static void bdi_timer_fn(unsigned long data)
 {
 	struct backing_dev_info *bdi = (struct backing_dev_info *)data;
 
+	trace_writeback_timer(bdi);
 	queue_work(bdi_wq, &bdi->wb.work);
 }
 
@@ -347,6 +348,8 @@ static void bdi_work_fn(struct work_struct *work)
 	struct backing_dev_info *bdi = wb->bdi;
 	long pages_written;
 
+	trace_writeback_start(bdi);
+
 	WARN(!test_bit(BDI_registered, &bdi->state),
 	     "bdi %p/%s is not registered!\n", bdi, bdi->name);
 
@@ -369,6 +372,7 @@ static void bdi_work_fn(struct work_struct *work)
 			trace_writeback_pages_written(pages_written);
 		} while (!list_empty(&bdi->work_list));
 	} else {
+		trace_writeback_rescuer(bdi, work);
 		bdi_flush_io(bdi);
 		if (!list_empty(&bdi->work_list))
 			queue_work(bdi_wq, work);
@@ -378,6 +382,8 @@ static void bdi_work_fn(struct work_struct *work)
 		bdi_delayed_writeback(bdi);
 
 	current->flags &= ~(PF_FLUSHER | PF_SWAPWRITE);
+
+	trace_writeback_stop(bdi);
 }
 
 /*
Signature · 1 line
1.7.1
author maintainer
#4 lore raw

PATCH 3/5 backing-dev: replace private thread pool with workqueue

In reply to Tejun Heo

bdi writeback has been using a private thread pool. Now that workqueue can provide flexible concurrency, drop the dedicated thread pool and use workqueue instead.

bdi_writeback->task and ->wakeup_timer are replaced with ->work and ->timer. A separate timer is used instead of delayed_work because mixing immediate and delayed queueing doesn't work too well with delayed_work.

bdi_wq is added to execute bdi_writeback->work. It's an unbound freezeable workqueue w/ a rescuer. As the thread pool is now managed by the workqueue code, the special forker thread is no longer necessary and removed along with bdi_forker_thread().

The work function for bdi_writeback->work is bdi_work_fn() and mostly equivalent to a single iteration of bdi_writeback_thread() - IOW, it calls wb_do_writeback() until bdi->work_list is exhausted and reschedules itself if there is dirty data to write out.

One special provision is that the writeback work doesn't run bdi->worklist if it's being executed from the rescuer. Instead, it just calls bdi_flush_io() and reschedules itself if necessary. This basically is the same behavior as bdi_forker_thread()'s when it failed to create a thread for a pending bdi. Writebacks are issued but not waited upon so that other bdi's have a chance to clean and free memory.

Trace points which make sense without change are preserved. The next patch will fix up the rest.

bdi_wakeup_thread_delayed() is renamed to bdi_delayed_writeback().

---
 fs/fs-writeback.c           |   91 ++--------------
 include/linux/backing-dev.h |   17 +--
 mm/backing-dev.c            |  249 ++++++++++---------------------------------
 3 files changed, 72 insertions(+), 285 deletions(-)
fs/fs-writeback.c +8 -83
index 7d9d06b..1e9b807 100644
@@ -78,16 +78,7 @@ static void bdi_queue_work(struct backing_dev_info *bdi,
 
 	spin_lock_bh(&bdi->wb_lock);
 	list_add_tail(&work->list, &bdi->work_list);
-	if (bdi->wb.task) {
-		wake_up_process(bdi->wb.task);
-	} else {
-		/*
-		 * The bdi thread isn't there, wake up the forker thread which
-		 * will create and run it.
-		 */
-		trace_writeback_nothread(bdi, work);
-		wake_up_process(default_backing_dev_info.wb.task);
-	}
+	queue_work(bdi_wq, &bdi->wb.work);
 	spin_unlock_bh(&bdi->wb_lock);
 }
 
@@ -99,14 +90,12 @@ __bdi_start_writeback(struct backing_dev_info *bdi, long nr_pages,
 
 	/*
 	 * This is WB_SYNC_NONE writeback, so if allocation fails just
-	 * wakeup the thread for old dirty data writeback
+	 * queue the work for old dirty data writeback.
 	 */
 	work = kzalloc(sizeof(*work), GFP_ATOMIC);
 	if (!work) {
-		if (bdi->wb.task) {
-			trace_writeback_nowork(bdi);
-			wake_up_process(bdi->wb.task);
-		}
+		trace_writeback_nowork(bdi);
+		queue_work(bdi_wq, &bdi->wb.work);
 		return;
 	}
 
@@ -773,70 +762,6 @@ long wb_do_writeback(struct bdi_writeback *wb, int force_wait)
 }
 
 /*
- * Handle writeback of dirty data for the device backed by this bdi. Also
- * wakes up periodically and does kupdated style flushing.
- */
-int bdi_writeback_thread(void *data)
-{
-	struct bdi_writeback *wb = data;
-	struct backing_dev_info *bdi = wb->bdi;
-	long pages_written;
-
-	current->flags |= PF_FLUSHER | PF_SWAPWRITE;
-	set_freezable();
-	wb->last_active = jiffies;
-
-	/*
-	 * Our parent may run at a different priority, just set us to normal
-	 */
-	set_user_nice(current, 0);
-
-	trace_writeback_thread_start(bdi);
-
-	while (!kthread_should_stop()) {
-		/*
-		 * Remove own delayed wake-up timer, since we are already awake
-		 * and we'll take care of the preriodic write-back.
-		 */
-		del_timer(&wb->wakeup_timer);
-
-		pages_written = wb_do_writeback(wb, 0);
-
-		trace_writeback_pages_written(pages_written);
-
-		if (pages_written)
-			wb->last_active = jiffies;
-
-		set_current_state(TASK_INTERRUPTIBLE);
-		if (!list_empty(&bdi->work_list)) {
-			__set_current_state(TASK_RUNNING);
-			continue;
-		}
-
-		if (wb_has_dirty_io(wb) && dirty_writeback_interval)
-			schedule_timeout(msecs_to_jiffies(dirty_writeback_interval * 10));
-		else {
-			/*
-			 * We have nothing to do, so can go sleep without any
-			 * timeout and save power. When a work is queued or
-			 * something is made dirty - we will be woken up.
-			 */
-			schedule();
-		}
-
-		try_to_freeze();
-	}
-
-	/* Flush any work that raced with us exiting */
-	if (!list_empty(&bdi->work_list))
-		wb_do_writeback(wb, 1);
-
-	trace_writeback_thread_stop(bdi);
-	return 0;
-}
-
-
-/*
  * Start writeback of `nr_pages' pages.  If `nr_pages' is zero, write back
  * the whole world.
  */
@@ -911,7 +836,7 @@ void __mark_inode_dirty(struct inode *inode, int flags)
 {
 	struct super_block *sb = inode->i_sb;
 	struct backing_dev_info *bdi = NULL;
-	bool wakeup_bdi = false;
+	bool delayed_wb = false;
 
 	/*
 	 * Don't do this for I_DIRTY_PAGES - that doesn't actually
@@ -978,7 +903,7 @@ void __mark_inode_dirty(struct inode *inode, int flags)
 				 * write-back happens later.
 				 */
 				if (!wb_has_dirty_io(&bdi->wb))
-					wakeup_bdi = true;
+					delayed_wb = true;
 			}
 
 			inode->dirtied_when = jiffies;
@@ -988,8 +913,8 @@ void __mark_inode_dirty(struct inode *inode, int flags)
 out:
 	spin_unlock(&inode_lock);
 
-	if (wakeup_bdi)
-		bdi_wakeup_thread_delayed(bdi);
+	if (delayed_wb)
+		bdi_delayed_writeback(bdi);
 }
 EXPORT_SYMBOL(__mark_inode_dirty);
 
include/linux/backing-dev.h +5 -12
index 97842ab..ebeed5b 100644
@@ -13,8 +13,8 @@
 #include <linux/proportions.h>
 #include <linux/kernel.h>
 #include <linux/fs.h>
-#include <linux/sched.h>
 #include <linux/timer.h>
+#include <linux/workqueue.h>
 #include <linux/writeback.h>
 #include <asm/atomic.h>
 
@@ -26,7 +26,6 @@ struct dentry;
  * Bits in backing_dev_info.state
  */
 enum bdi_state {
-	BDI_pending,		/* On its way to being activated */
 	BDI_wb_alloc,		/* Default embedded wb allocated */
 	BDI_async_congested,	/* The async (write) queue is getting full */
 	BDI_sync_congested,	/* The sync queue is getting full */
@@ -49,10 +48,9 @@ struct bdi_writeback {
 	struct backing_dev_info *bdi;	/* our parent bdi */
 
 	unsigned long last_old_flush;	/* last old data flush */
-	unsigned long last_active;	/* last time bdi thread was active */
 
-	struct task_struct *task;	/* writeback thread */
-	struct timer_list wakeup_timer; /* used for delayed bdi thread wakeup */
+	struct work_struct work;	/* writeback work */
+	struct timer_list timer;	/* used for delayed bdi execution */
 	struct list_head b_dirty;	/* dirty inodes */
 	struct list_head b_io;		/* parked for writeback */
 	struct list_head b_more_io;	/* parked for more writeback */
@@ -103,13 +101,13 @@ void bdi_unregister(struct backing_dev_info *bdi);
 int bdi_setup_and_register(struct backing_dev_info *, char *, unsigned int);
 void bdi_start_writeback(struct backing_dev_info *bdi, long nr_pages);
 void bdi_start_background_writeback(struct backing_dev_info *bdi);
-int bdi_writeback_thread(void *data);
 int bdi_has_dirty_io(struct backing_dev_info *bdi);
 void bdi_arm_supers_timer(void);
-void bdi_wakeup_thread_delayed(struct backing_dev_info *bdi);
+void bdi_delayed_writeback(struct backing_dev_info *bdi);
 
 extern spinlock_t bdi_lock;
 extern struct list_head bdi_list;
+extern struct workqueue_struct *bdi_wq;
 
 static inline int wb_has_dirty_io(struct bdi_writeback *wb)
 {
@@ -308,11 +306,6 @@ static inline bool bdi_cap_swap_backed(struct backing_dev_info *bdi)
 	return bdi->capabilities & BDI_CAP_SWAP_BACKED;
 }
 
-static inline bool bdi_cap_flush_forker(struct backing_dev_info *bdi)
-{
-	return bdi == &default_backing_dev_info;
-}
-
 static inline bool mapping_cap_writeback_dirty(struct address_space *mapping)
 {
 	return bdi_cap_writeback_dirty(mapping->backing_dev_info);
mm/backing-dev.c +59 -190

327 lines not on this page · raw message

Signature · 1 line
1.7.1
author maintainer
#6 lore raw

PATCH 5/5 backing-dev: replace sync_supers_tsk/timer with a delayed_work

In reply to Tejun Heo

sync_supers_task/timer can trivially replaced with a delayed_work. As it's not on the allocation path, run it on the system_wq.

NOTE: Both before and after this patch, sync_supers is not freezeable.
This might need to be changed.

---
 mm/backing-dev.c |   49 ++++++++++++++-----------------------------------
 1 files changed, 14 insertions(+), 35 deletions(-)
mm/backing-dev.c +14 -35
index 3b2a657..4fb8095 100644
@@ -45,11 +45,8 @@ LIST_HEAD(bdi_list);
 LIST_HEAD(bdi_pending_list);
 struct workqueue_struct *bdi_wq;
 
-static struct task_struct *sync_supers_tsk;
-static struct timer_list sync_supers_timer;
-
-static int bdi_sync_supers(void *);
-static void sync_supers_timer_fn(unsigned long);
+static void bdi_sync_supers_work_fn(struct work_struct *work);
+static DECLARE_DELAYED_WORK(bdi_sync_supers_work, bdi_sync_supers_work_fn);
 
 #ifdef CONFIG_DEBUG_FS
 #include <linux/debugfs.h>
@@ -239,10 +236,6 @@ static int __init default_bdi_init(void)
 				 WQ_UNBOUND | WQ_FREEZEABLE | WQ_RESCUER, 0);
 	BUG_ON(!bdi_wq);
 
-	sync_supers_tsk = kthread_run(bdi_sync_supers, NULL, "sync_supers");
-	BUG_ON(IS_ERR(sync_supers_tsk));
-
-	setup_timer(&sync_supers_timer, sync_supers_timer_fn, 0);
 	bdi_arm_supers_timer();
 
 	err = bdi_init(&default_backing_dev_info);
@@ -271,43 +264,29 @@ static void bdi_flush_io(struct backing_dev_info *bdi)
 }
 
 /*
- * kupdated() used to do this. We cannot do it from the bdi_forker_thread()
- * or we risk deadlocking on ->s_umount. The longer term solution would be
- * to implement sync_supers_bdi() or similar and simply do it from the
- * bdi writeback thread individually.
+ * kupdated() used to do this.  Don't do it from bdi_wq as we might
+ * deadlock on ->s_umount.
  */
-static int bdi_sync_supers(void *unused)
+static void bdi_sync_supers_work_fn(struct work_struct *work)
 {
-	set_user_nice(current, 0);
-
-	while (!kthread_should_stop()) {
-		set_current_state(TASK_INTERRUPTIBLE);
-		schedule();
-
-		/*
-		 * Do this periodically, like kupdated() did before.
-		 */
-		sync_supers();
-	}
+	/* Do this periodically, like kupdated() did before */
+	sync_supers();
 
-	return 0;
+	if (dirty_writeback_interval)
+		schedule_delayed_work(&bdi_sync_supers_work,
+			msecs_to_jiffies(dirty_writeback_interval * 10));
 }
 
 void bdi_arm_supers_timer(void)
 {
-	unsigned long next;
+	/* Interval might have decreased, cancel delayed work first */
+	cancel_delayed_work(&bdi_sync_supers_work);
 
 	if (!dirty_writeback_interval)
 		return;
 
-	next = msecs_to_jiffies(dirty_writeback_interval * 10) + jiffies;
-	mod_timer(&sync_supers_timer, round_jiffies_up(next));
-}
-
-static void sync_supers_timer_fn(unsigned long unused)
-{
-	wake_up_process(sync_supers_tsk);
-	bdi_arm_supers_timer();
+	schedule_delayed_work(&bdi_sync_supers_work,
+			      msecs_to_jiffies(dirty_writeback_interval * 10));
 }
 
 static void bdi_timer_fn(unsigned long data)
Signature · 1 line
1.7.1
author maintainer
#1 lore raw

PATCH 1/5 workqueue: implement workqueue_on_rescuer()

In reply to Tejun Heo

workqueue_on_rescuer() tests whether the caller is running on the rescuer of the specified workqueue.

---
 include/linux/workqueue.h |    1 +
 kernel/workqueue.c        |   15 +++++++++++++++
 2 files changed, 16 insertions(+), 0 deletions(-)
include/linux/workqueue.h +1 -0
index f11100f..c988063 100644
@@ -339,6 +339,7 @@ extern int cancel_work_sync(struct work_struct *work);
 extern void workqueue_set_max_active(struct workqueue_struct *wq,
 				     int max_active);
 extern bool workqueue_congested(unsigned int cpu, struct workqueue_struct *wq);
+extern bool workqueue_on_rescuer(struct workqueue_struct *wq);
 extern unsigned int work_cpu(struct work_struct *work);
 extern unsigned int work_busy(struct work_struct *work);
 
kernel/workqueue.c +15 -0
index 727f24e..a91e1ca 100644
@@ -2941,6 +2941,21 @@ bool workqueue_congested(unsigned int cpu, struct workqueue_struct *wq)
 EXPORT_SYMBOL_GPL(workqueue_congested);
 
 /**
+ * workqueue_on_rescuer - test whether executing on the rescuer
+ * @wq: target workqueue
+ *
+ * Test whether the caller is currently executing on @wq's rescuer.
+ *
+ * RETURNS:
+ * %true if executing on @wq's rescuer, %false otherwise.
+ */
+bool workqueue_on_rescuer(struct workqueue_struct *wq)
+{
+	return wq->rescuer && current == wq->rescuer->task;
+}
+EXPORT_SYMBOL_GPL(workqueue_on_rescuer);
+
+/**
  * work_cpu - return the last known associated cpu for @work
  * @work: the work of interest
  *
Signature · 1 line
1.7.1
#8 lore raw
In reply to Tejun Heo

What does this conversion buy us?

Having explicit per-bdi threads makes the model very simple and safe, while I don't quite trust the now extremly complex work queues yet.

author maintainer
#9 lore raw

Hello,

Quoted 4 lines · Christoph Hellwig

What does this conversion buy us?

Having explicit per-bdi threads makes the model very simple and safe, while I don't quite trust the now extremly complex work queues yet.

Well, the whole idea behind the new workqueue is unifying all those dedicated pools of threads. If the conversion is too early given the new workqueue implementation, we can postpone bdi conversion by one more release cycle, but in the long run there is no point in keeping these private thread pools.

Thanks.

Signature · 1 line
tejun
#10 lore raw
Quoted 5 lines · Tejun Heo

Hello,

Quoted 2 lines · Christoph Hellwig

What does this conversion buy us?

Quoted 8 lines
Quoted 2 lines

Having explicit per-bdi threads makes the model very simple and safe, while I don't quite trust the now extremly complex work queues yet.

Well, the whole idea behind the new workqueue is unifying all those dedicated pools of threads. If the conversion is too early given the new workqueue implementation, we can postpone bdi conversion by one more release cycle, but in the long run there is no point in keeping these private thread pools.

I agree (with both of you). It's definitely too early to convert it over, but if we can in the longer run, it never hurts to get rid of code. The writeback threads aren't a typical threadpool, in that the threads stick around and only go away when idle for too long. If they stick around, you get the same process hammering IO at your device. So converting that over to the generic cwq may or may not be at a performance cost, it'll definitely have to be tested.

Signature · 1 line
Jens Axboe
author maintainer
#11 lore raw

Hello,

Quoted 7 lines · Jens Axboe

I agree (with both of you). It's definitely too early to convert it over, but if we can in the longer run, it never hurts to get rid of code. The writeback threads aren't a typical threadpool, in that the threads stick around and only go away when idle for too long. If they stick around, you get the same process hammering IO at your device. So converting that over to the generic cwq may or may not be at a performance cost, it'll definitely have to be tested.

One thing to try is removing WQ_UNBOUND and see how it affects the performance. I put WQ_UNBOUND there mainly to keep the behavior about the same as the current code but given what it does I think it would probably fare better with workers bound to CPUs.

Thanks.

Signature · 1 line
tejun
#3 lore raw

Re: [PATCH 2/5] backing-dev: kill unused bdi_writeback->nr

In reply to Tejun Heo
Quoted 3 lines · Tejun Heo

bdi_writeback->nr is no longer used. Kill it.

This is obviously an independent clean-up which can be picked straight away?

Signature · 2 lines
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
#7 lore raw

Re: [PATCH 5/5] backing-dev: replace sync_supers_tsk/timer with a delayed_work

In reply to Tejun Heo
Quoted 10 lines · Tejun Heo

sync_supers_task/timer can trivially replaced with a delayed_work. As it's not on the allocation path, run it on the system_wq.

NOTE: Both before and after this patch, sync_supers is not freezeable.
This might need to be changed.

---
 mm/backing-dev.c |   49 ++++++++++++++-----------------------------------
 1 files changed, 14 insertions(+), 35 deletions(-)

Actually what Al Viro wants is to make sync_supers() go away and make every FS to take care of its own superblock. So I guess this change is useful in a sense that it'll be then easier to push this down to FSes.

Signature · 2 lines
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
author maintainer
#12 lore raw
In reply to Tejun Heo
Quoted 11 lines · Tejun Heo

Hello,

Quoted 7 lines · Jens Axboe

I agree (with both of you). It's definitely too early to convert it over, but if we can in the longer run, it never hurts to get rid of code. The writeback threads aren't a typical threadpool, in that the threads stick around and only go away when idle for too long. If they stick around, you get the same process hammering IO at your device. So converting that over to the generic cwq may or may not be at a performance cost, it'll definitely have to be tested.

Quoted 4 lines

One thing to try is removing WQ_UNBOUND and see how it affects the performance. I put WQ_UNBOUND there mainly to keep the behavior about the same as the current code but given what it does I think it would probably fare better with workers bound to CPUs.

cmwq now seems pretty solid. There hasn't been any noticeable failure yet. I think we can move on with this conversion now. Shall I refresh the patchset against the current block tree?

Thanks.

Signature · 1 line
tejun
#13 lore raw
Quoted 12 lines · Tejun Heo
Quoted 11 lines · Tejun Heo

Hello,

Quoted 7 lines · Jens Axboe

I agree (with both of you). It's definitely too early to convert it over, but if we can in the longer run, it never hurts to get rid of code. The writeback threads aren't a typical threadpool, in that the threads stick around and only go away when idle for too long. If they stick around, you get the same process hammering IO at your device. So converting that over to the generic cwq may or may not be at a performance cost, it'll definitely have to be tested.

Quoted 8 lines
Quoted 4 lines

One thing to try is removing WQ_UNBOUND and see how it affects the performance. I put WQ_UNBOUND there mainly to keep the behavior about the same as the current code but given what it does I think it would probably fare better with workers bound to CPUs.

cmwq now seems pretty solid. There hasn't been any noticeable failure yet. I think we can move on with this conversion now. Shall I refresh the patchset against the current block tree?

I'd still prefer to wait a while. The writeback code is still very much a moving target, so I don't think mixing in a different work queue scheme is likely going to do anyone any good at this point in time.

Signature · 1 line
Jens Axboe
author maintainer
#14 lore raw

Hello,

Quoted 7 lines · Jens Axboe
Quoted 3 lines

cmwq now seems pretty solid. There hasn't been any noticeable failure yet. I think we can move on with this conversion now. Shall I refresh the patchset against the current block tree?

I'd still prefer to wait a while. The writeback code is still very much a moving target, so I don't think mixing in a different work queue scheme is likely going to do anyone any good at this point in time.

Alright, will ping again after the next merge window.

Thanks.

Signature · 1 line
tejun