cgroup, blkcg: prevent dirty inodes to pin dying memory cgroups
Subsystems (4)
PATCH v2 1/1 cgroup, blkcg: prevent dirty inodes to pin dying memory cgroups
We've noticed that the number of dying cgroups on our production hosts tends to grow with the uptime. This time it's caused by the writeback code.
An inode which is getting dirty for the first time is associated with the wb structure (look at __inode_attach_wb()). It can later be switched to another wb under some conditions (e.g. some other cgroup is writing a lot of data to the same inode), but generally stays associated up to the end of life of the inode structure.
The problem is that the wb structure holds a reference to the original memory cgroup. So if an inode has been dirty once, it has a good chance to pin down the original memory cgroup.
An example from the real life: some service runs periodically and updates rpm packages. Each time in a new memory cgroup. Installed .so files are heavily used by other cgroups, so corresponding inodes tend to stay alive for a long. So do pinned memory cgroups. In production I've seen many hosts with 1-2 thousands of dying cgroups.
This is not the first problem with the dying memory cgroups. As always, the problem is with their relative size: memory cgroups are large objects, easily 100x-1000x larger that inodes. So keeping a couple of thousands of dying cgroups in memory without a good reason (what we easily do with inodes) is quite costly (and is measured in tens and hundreds of Mb).
To solve this problem let's perform a periodic scan of inodes attached to the dying wbs, and detach those of them, which are clean and don't have an active io operation.
That will eventually release the wb structure and corresponding memory cgroup.
To make this scanning effective, let's keep a list of attached inodes. inode->i_io_list can be reused for this purpose.
The scan is performed from the cgroup offlining path. Dying wbs are placed on the global list. On each cgroup removal we traverse the whole list ignoring wbs with active io operations. That will allow the majority of io operations to be finished after the removal of the cgroup.
Big thanks to Jan Kara and Dennis Zhou for their ideas and contribution to this patch.
--- fs/fs-writeback.c | 52 +++++++++++++++++++++++--- include/linux/backing-dev-defs.h | 2 + include/linux/writeback.h | 1 + mm/backing-dev.c | 63 ++++++++++++++++++++++++++++++-- 4 files changed, 108 insertions(+), 10 deletions(-)
fs/fs-writeback.c +46 -6
@@ -136,16 +136,21 @@ static bool inode_io_list_move_locked(struct inode *inode,* inode_io_list_del_locked - remove an inode from its bdi_writeback IO list * @inode: inode to be removed * @wb: bdi_writeback @inode is being removed from +* @keep_attached: keep the inode on the list of inodes attached to wb * * Remove @inode which may be on one of @wb->b_{dirty|io|more_io} lists and * clear %WB_has_dirty_io if all are empty afterwards. */ static void inode_io_list_del_locked (struct inode *inode, -struct bdi_writeback *wb) +struct bdi_writeback *wb, +bool keep_attached) {assert_spin_locked (&wb->list_lock); -list_del_init (&inode->i_io_list); +if (keep_attached) +list_move (&inode->i_io_list, &wb->b_attached); +else +list_del_init (&inode->i_io_list);wb_io_lists_depopulated (wb); }
@@ -426,7 +431,7 @@ static void inode_switch_wbs_work_fn(struct work_struct *work)if (!list_empty (&inode->i_io_list)) {struct inode *pos; -inode_io_list_del_locked (inode, old_wb); +inode_io_list_del_locked (inode, old_wb, false); inode->i_wb = new_wb;list_for_each_entry (pos, &new_wb->b_dirty, i_io_list) if (time_after_eq(inode->dirtied_when,
@@ -544,6 +549,41 @@ static void inode_switch_wbs(struct inode *inode, int new_wb_id)kfree (isw); } +/** +* cleanup_offline_wb - detach attached clean inodes +* @wb: target wb +* +* Clear the ->i_wb pointer of the attached inodes and drop +* the corresponding wb reference. Skip inodes which are dirty, +* freeing, switching or in the active writeback process. +*/ +void cleanup_offline_wb (struct bdi_writeback *wb) +{ +struct inode *inode, *tmp; +bool ret = true; + +spin_lock (&wb->list_lock); +if (list_empty (&wb->b_attached)) + goto unlock; + +list_for_each_entry_safe (inode, tmp, &wb->b_attached, i_io_list) { +if (!spin_trylock (&inode->i_lock)) +continue ; +xa_lock_irq (&inode->i_mapping->i_pages); +if (!(inode->i_state & + (I_FREEING |I_CLEAR |I_SYNC |I_DIRTY |I_WB_SWITCH ))) { +WARN_ON_ONCE (inode->i_wb != wb); + inode->i_wb =NULL ; +wb_put (wb); +list_del_init (&inode->i_io_list); + } +xa_unlock_irq (&inode->i_mapping->i_pages); +spin_unlock (&inode->i_lock); + } +unlock: +spin_unlock (&wb->list_lock); +} +/** * wbc_attach_and_unlock_inode - associate wbc with target inode and unlock it * @wbc: writeback_control of interest
@@ -1120,7 +1160,7 @@ void inode_io_list_del(struct inode *inode)struct bdi_writeback *wb; wb =inode_to_wb_and_lock_list (inode); -inode_io_list_del_locked (inode, wb); +inode_io_list_del_locked (inode, wb, false);spin_unlock (&wb->list_lock); }
@@ -1425,7 +1465,7 @@ static void requeue_inode(struct inode *inode, struct bdi_writeback *wb,inode_io_list_move_locked (inode, wb, &wb->b_dirty_time); }else {/* The inode is clean. Remove from writeback lists. */ -inode_io_list_del_locked (inode, wb); +inode_io_list_del_locked (inode, wb, true); } }
@@ -1570,7 +1610,7 @@ static int writeback_single_inode(struct inode *inode,* touch it. See comment above for explanation. */ if (!(inode->i_state &I_DIRTY_ALL )) -inode_io_list_del_locked (inode, wb); +inode_io_list_del_locked (inode, wb, true);spin_unlock (&wb->list_lock);inode_sync_complete (inode); out:
include/linux/backing-dev-defs.h +2 -0
@@ -137,6 +137,7 @@ struct bdi_writeback {struct list_head b_io;/* parked for writeback */ struct list_head b_more_io;/* parked for more writeback */ struct list_head b_dirty_time;/* time stamps are dirty */ +struct list_head b_attached;/* attached inodes */ spinlock_t list_lock;/* protects the b_* lists */ struct percpu_counter stat[NR_WB_STAT_ITEMS ];
@@ -177,6 +178,7 @@ struct bdi_writeback {struct cgroup_subsys_state *blkcg_css;/* and blkcg */ struct list_head memcg_node;/* anchored at memcg->cgwb_list */ struct list_head blkcg_node;/* anchored at blkcg->cgwb_list */ +struct list_head offline_node;union {struct work_struct release_work;
include/linux/writeback.h +1 -0
@@ -220,6 +220,7 @@ void wbc_account_cgroup_owner(struct writeback_control *wbc, struct page *page,int cgroup_writeback_by_id (u64 bdi_id,int memcg_id,unsigned long nr_pages,enum wb_reason reason,struct wb_completion *done);void cgroup_writeback_umount (void ); +void cleanup_offline_wb (struct bdi_writeback *wb);/** * inode_attach_wb - associate an inode with its wb
mm/backing-dev.c +59 -4
128 lines not on this page · raw message
Signature · 1 line
2.21.0
Jan Kara
maintainerQuoted 71 lines · Roman Gushchin
We've noticed that the number of dying cgroups on our production hosts tends to grow with the uptime. This time it's caused by the writeback code.
An inode which is getting dirty for the first time is associated with the wb structure (look at
__inode_attach_wb()). It can later be switched to another wb under some conditions (e.g. some other cgroup is writing a lot of data to the same inode), but generally stays associated up to the end of life of the inode structure.The problem is that the wb structure holds a reference to the original memory cgroup. So if an inode has been dirty once, it has a good chance to pin down the original memory cgroup.
An example from the real life: some service runs periodically and updates rpm packages. Each time in a new memory cgroup. Installed .so files are heavily used by other cgroups, so corresponding inodes tend to stay alive for a long. So do pinned memory cgroups. In production I've seen many hosts with 1-2 thousands of dying cgroups.
This is not the first problem with the dying memory cgroups. As always, the problem is with their relative size: memory cgroups are large objects, easily 100x-1000x larger that inodes. So keeping a couple of thousands of dying cgroups in memory without a good reason (what we easily do with inodes) is quite costly (and is measured in tens and hundreds of Mb).
To solve this problem let's perform a periodic scan of inodes attached to the dying wbs, and detach those of them, which are clean and don't have an active io operation.
That will eventually release the wb structure and corresponding memory cgroup.To make this scanning effective, let's keep a list of attached inodes.
inode->i_io_listcan be reused for this purpose.The scan is performed from the cgroup offlining path. Dying wbs are placed on the global list. On each cgroup removal we traverse the whole list ignoring wbs with active io operations. That will allow the majority of io operations to be finished after the removal of the cgroup.
Big thanks to Jan Kara and Dennis Zhou for their ideas and contribution to this patch.
Signed-off-by Roman Gushchin <[email protected]>--- fs/fs-writeback.c | 52 +++++++++++++++++++++++--- include/linux/backing-dev-defs.h | 2 + include/linux/writeback.h | 1 + mm/backing-dev.c | 63 ++++++++++++++++++++++++++++++-- 4 files changed, 108 insertions(+), 10 deletions(-)fs/fs-writeback.c +3
-1@@ -136,16 +136,21 @@ static bool inode_io_list_move_locked(struct inode *inode,* inode_io_list_del_locked - remove an inode from its bdi_writeback IO list * @inode: inode to be removed * @wb: bdi_writeback @inode is being removed from +* @keep_attached: keep the inode on the list of inodes attached to wb * * Remove @inode which may be on one of @wb->b_{dirty|io|more_io} lists and * clear %WB_has_dirty_io if all are empty afterwards. */ static void inode_io_list_del_locked (struct inode *inode, -struct bdi_writeback *wb) +struct bdi_writeback *wb, +bool keep_attached)
fs/fs-writeback.c static bool inode_io_list_move_locked(struct inode *inode, · line 155
Quoted 10 lines
fs/fs-writeback.c +4
-1{assert_spin_locked (&wb->list_lock); -list_del_init (&inode->i_io_list); +if (keep_attached) +list_move (&inode->i_io_list, &wb->b_attached); +else +list_del_init (&inode->i_io_list);wb_io_lists_depopulated (wb); }
Rather than adding this (somewhat ugly) bool argument to inode_io_list_del_locked() I'd teach inode_io_list_move_locked() about the new b_attached list and use that function where needed...
fs/fs-writeback.c static void inode_switch_wbs_work_fn(struct work_struct *work) · line 437
Quoted 9 lines
fs/fs-writeback.c +1
-1@@ -426,7 +431,7 @@ static void inode_switch_wbs_work_fn(struct work_struct *work)if (!list_empty (&inode->i_io_list)) {struct inode *pos; -inode_io_list_del_locked (inode, old_wb); +inode_io_list_del_locked (inode, old_wb, false); inode->i_wb = new_wb;list_for_each_entry (pos, &new_wb->b_dirty, i_io_list) if (time_after_eq(inode->dirtied_when,
This bit looks wrong. Not the change you made as such but the fact that you can now move inode from b_attached list of old wb to the dirty list of new wb.
Quoted 10 lines
fs/fs-writeback.c +6
-0@@ -544,6 +549,41 @@ static void inode_switch_wbs(struct inode *inode, int new_wb_id)kfree (isw); } +/** +* cleanup_offline_wb - detach attached clean inodes +* @wb: target wb +* +* Clear the ->i_wb pointer of the attached inodes and drop +* the corresponding wb reference. Skip inodes which are dirty,
fs/fs-writeback.c static void inode_switch_wbs(struct inode *inode, int new_wb_id) · line 567
Quoted 10 lines
fs/fs-writeback.c +10
-0+* freeing, switching or in the active writeback process. +*/ +void cleanup_offline_wb (struct bdi_writeback *wb) +{ +struct inode *inode, *tmp; +bool ret = true; + +spin_lock (&wb->list_lock); +if (list_empty (&wb->b_attached)) + goto unlock;
What's the point of this check? list_for_each_entry_safe() below will just do the same...
fs/fs-writeback.c static void inode_switch_wbs(struct inode *inode, int new_wb_id) · line 577
Quoted 10 lines
fs/fs-writeback.c +10
-0+ +list_for_each_entry_safe (inode, tmp, &wb->b_attached, i_io_list) { +if (!spin_trylock (&inode->i_lock)) +continue ; +xa_lock_irq (&inode->i_mapping->i_pages); +if (!(inode->i_state & + (I_FREEING |I_CLEAR |I_SYNC |I_DIRTY |I_WB_SWITCH ))) { +WARN_ON_ONCE (inode->i_wb != wb); + inode->i_wb =NULL ; +wb_put (wb);
Hum, currently the code assumes that once i_wb is set, it never becomes NULL again. In particular the inode e.g. in fs/fs-writeback.c:inode_congested() or generally unlocked_inode_to_wb_begin() users could get broken by this. The i_wb switching code is so complex exactly because of these interactions.
Maybe you thought through the interactions and things are actually fine but if nothing else you'd need a big fat comment here explaining why this is fine and update inode_congested() comments etc.
Quoted 9 lines
fs/fs-writeback.c +9
-0+list_del_init (&inode->i_io_list); + } +xa_unlock_irq (&inode->i_mapping->i_pages); +spin_unlock (&inode->i_lock); + } +unlock: +spin_unlock (&wb->list_lock); +} +
...
include/linux/backing-dev-defs.h struct bdi_writeback { · line 140
Quoted 9 lines
include/linux/backing-dev-defs.h +1
-0@@ -137,6 +137,7 @@ struct bdi_writeback {struct list_head b_io;/* parked for writeback */ struct list_head b_more_io;/* parked for more writeback */ struct list_head b_dirty_time;/* time stamps are dirty */ +struct list_head b_attached;/* attached inodes */
Maybe
/* clean inodes pointing to this wb through inode->i_wb */
would be more explanatory?
include/linux/backing-dev-defs.h struct bdi_writeback { · line 143
Quoted 3 lines
include/linux/backing-dev-defs.h +0
-0spinlock_t list_lock;/* protects the b_* lists */ struct percpu_counter stat[NR_WB_STAT_ITEMS ];
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 75 lines · Jan Kara
Quoted 74 lines · Roman Gushchin
We've noticed that the number of dying cgroups on our production hosts tends to grow with the uptime. This time it's caused by the writeback code.
An inode which is getting dirty for the first time is associated with the wb structure (look at
__inode_attach_wb()). It can later be switched to another wb under some conditions (e.g. some other cgroup is writing a lot of data to the same inode), but generally stays associated up to the end of life of the inode structure.The problem is that the wb structure holds a reference to the original memory cgroup. So if an inode has been dirty once, it has a good chance to pin down the original memory cgroup.
An example from the real life: some service runs periodically and updates rpm packages. Each time in a new memory cgroup. Installed .so files are heavily used by other cgroups, so corresponding inodes tend to stay alive for a long. So do pinned memory cgroups. In production I've seen many hosts with 1-2 thousands of dying cgroups.
This is not the first problem with the dying memory cgroups. As always, the problem is with their relative size: memory cgroups are large objects, easily 100x-1000x larger that inodes. So keeping a couple of thousands of dying cgroups in memory without a good reason (what we easily do with inodes) is quite costly (and is measured in tens and hundreds of Mb).
To solve this problem let's perform a periodic scan of inodes attached to the dying wbs, and detach those of them, which are clean and don't have an active io operation.
That will eventually release the wb structure and corresponding memory cgroup.To make this scanning effective, let's keep a list of attached inodes.
inode->i_io_listcan be reused for this purpose.The scan is performed from the cgroup offlining path. Dying wbs are placed on the global list. On each cgroup removal we traverse the whole list ignoring wbs with active io operations. That will allow the majority of io operations to be finished after the removal of the cgroup.
Big thanks to Jan Kara and Dennis Zhou for their ideas and contribution to this patch.
Signed-off-by Roman Gushchin <[email protected]>--- fs/fs-writeback.c | 52 +++++++++++++++++++++++--- include/linux/backing-dev-defs.h | 2 + include/linux/writeback.h | 1 + mm/backing-dev.c | 63 ++++++++++++++++++++++++++++++-- 4 files changed, 108 insertions(+), 10 deletions(-)fs/fs-writeback.c +3
-1@@ -136,16 +136,21 @@ static bool inode_io_list_move_locked(struct inode *inode,* inode_io_list_del_locked - remove an inode from its bdi_writeback IO list * @inode: inode to be removed * @wb: bdi_writeback @inode is being removed from +* @keep_attached: keep the inode on the list of inodes attached to wb * * Remove @inode which may be on one of @wb->b_{dirty|io|more_io} lists and * clear %WB_has_dirty_io if all are empty afterwards. */ static void inode_io_list_del_locked (struct inode *inode, -struct bdi_writeback *wb) +struct bdi_writeback *wb, +bool keep_attached) {assert_spin_locked (&wb->list_lock);
Quoted 11 lines
fs/fs-writeback.c static bool inode_io_list_move_locked(struct inode *inode, · line 155
Quoted 7 lines
fs/fs-writeback.c +4
-1-list_del_init (&inode->i_io_list); +if (keep_attached) +list_move (&inode->i_io_list, &wb->b_attached); +else +list_del_init (&inode->i_io_list);wb_io_lists_depopulated (wb); }Rather than adding this (somewhat ugly) bool argument to
inode_io_list_del_locked()I'd teachinode_io_list_move_locked()about the newb_attachedlist and use that function where needed...
Ok, will do in v3.
Quoted 5 lines
Quoted 5 lines
fs/fs-writeback.c +0
-0@@ -426,7 +431,7 @@ static void inode_switch_wbs_work_fn(struct work_struct *work)if (!list_empty (&inode->i_io_list)) {struct inode *pos;
Quoted 9 lines
fs/fs-writeback.c static void inode_switch_wbs_work_fn(struct work_struct *work) · line 437
Quoted 5 lines
fs/fs-writeback.c +1
-1-inode_io_list_del_locked (inode, old_wb); +inode_io_list_del_locked (inode, old_wb, false); inode->i_wb = new_wb;list_for_each_entry (pos, &new_wb->b_dirty, i_io_list) if (time_after_eq(inode->dirtied_when,This bit looks wrong. Not the change you made as such but the fact that you can now move inode from
b_attachedlist of old wb to the dirty list of new wb.
Hm, can you, please, elaborate a bit more why it's wrong? The reference to the old_wb will be dropped by the switching code.
Quoted 16 lines
Quoted 16 lines
fs/fs-writeback.c +11
-0@@ -544,6 +549,41 @@ static void inode_switch_wbs(struct inode *inode, int new_wb_id)kfree (isw); } +/** +* cleanup_offline_wb - detach attached clean inodes +* @wb: target wb +* +* Clear the ->i_wb pointer of the attached inodes and drop +* the corresponding wb reference. Skip inodes which are dirty, +* freeing, switching or in the active writeback process. +*/ +void cleanup_offline_wb (struct bdi_writeback *wb) +{ +struct inode *inode, *tmp;
Quoted 8 lines
fs/fs-writeback.c static void inode_switch_wbs(struct inode *inode, int new_wb_id) · line 567
Quoted 5 lines
fs/fs-writeback.c +5
-0+bool ret = true; + +spin_lock (&wb->list_lock); +if (list_empty (&wb->b_attached)) + goto unlock;What's the point of this check?
list_for_each_entry_safe()below will just do the same...
Right, will remove.
Quoted 18 lines
fs/fs-writeback.c static void inode_switch_wbs(struct inode *inode, int new_wb_id) · line 577
Quoted 11 lines
fs/fs-writeback.c +10
-0+ +list_for_each_entry_safe (inode, tmp, &wb->b_attached, i_io_list) { +if (!spin_trylock (&inode->i_lock)) +continue ; +xa_lock_irq (&inode->i_mapping->i_pages); +if (!(inode->i_state & + (I_FREEING |I_CLEAR |I_SYNC |I_DIRTY |I_WB_SWITCH ))) { +WARN_ON_ONCE (inode->i_wb != wb); + inode->i_wb =NULL ; +wb_put (wb);Hum, currently the code assumes that once
i_wbis set, it never becomes NULL again. In particular the inode e.g. in fs/fs-writeback.c:inode_congested() or generallyunlocked_inode_to_wb_begin()users could get broken by this. Thei_wbswitching code is so complex exactly because of these interactions.
Quoted 3 lines
Maybe you thought through the interactions and things are actually fine but if nothing else you'd need a big fat comment here explaining why this is fine and update
inode_congested()comments etc.
Yeah, I thought that once inode is clean and not switching it's safe to clear the i_wb pointer, but seems that it's not completely true.
One idea I have is to always release wbs using rcu delayed work, so that it will be save to dereference i_wb pointer under rcu, if only it's not NULL (the check has to be added). I'll try to implement this scheme, but if you know in advance that it's not gonna work, please, let me know.
Quoted 24 lines
Quoted 10 lines
fs/fs-writeback.c +9
-0+list_del_init (&inode->i_io_list); + } +xa_unlock_irq (&inode->i_mapping->i_pages); +spin_unlock (&inode->i_lock); + } +unlock: +spin_unlock (&wb->list_lock); +} +...
include/linux/backing-dev-defs.h struct bdi_writeback { · line 140
Quoted 9 lines
include/linux/backing-dev-defs.h +1
-0@@ -137,6 +137,7 @@ struct bdi_writeback {struct list_head b_io;/* parked for writeback */ struct list_head b_more_io;/* parked for more writeback */ struct list_head b_dirty_time;/* time stamps are dirty */ +struct list_head b_attached;/* attached inodes */ Maybe
Quoted 2 lines
/* clean inodes pointing to this wb through inode->i_wb */would be more explanatory?
Sure, np.
Quoted 9 lines
include/linux/backing-dev-defs.h struct bdi_writeback { · line 143
Quoted 4 lines
include/linux/backing-dev-defs.h +0
-0spinlock_t list_lock;/* protects the b_* lists */ struct percpu_counter stat[NR_WB_STAT_ITEMS ];Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Thank you for looking into it!
Jan Kara
maintainerQuoted 17 lines · Roman Gushchin
Quoted 6 lines · Jan Kara
Quoted 4 lines · Roman Gushchin
fs/fs-writeback.c +0
-0@@ -426,7 +431,7 @@ static void inode_switch_wbs_work_fn(struct work_struct *work)if (!list_empty (&inode->i_io_list)) {struct inode *pos;Quoted 9 lines
fs/fs-writeback.c static void inode_switch_wbs_work_fn(struct work_struct *work) · line 437
Quoted 5 lines
fs/fs-writeback.c +1
-1-inode_io_list_del_locked (inode, old_wb); +inode_io_list_del_locked (inode, old_wb, false); inode->i_wb = new_wb;list_for_each_entry (pos, &new_wb->b_dirty, i_io_list) if (time_after_eq(inode->dirtied_when,This bit looks wrong. Not the change you made as such but the fact that you can now move inode from
b_attachedlist of old wb to the dirty list of new wb.
Quoted 2 lines
Hm, can you, please, elaborate a bit more why it's wrong? The reference to the
old_wbwill be dropped by the switching code.
My point is that the code in full looks like:
if (!list_empty(&inode->i_io_list)) {
struct inode *pos;
inode_io_list_del_locked(inode, old_wb);
inode->i_wb = new_wb;
list_for_each_entry(pos, &new_wb->b_dirty, i_io_list)
if (time_after_eq(inode->dirtied_when,
pos->dirtied_when))
break;
inode_io_list_move_locked(inode, new_wb, pos->i_io_list.prev);
} else {So inode is always moved from some io list in old_wb to b_dirty list of new_wb. This is fine when it could be only on b_dirty, b_io, b_more_io lists of old_wb. But once you add b_attached list to the game, it is not correct anymore. You should not add clean inode to b_dirty list of new_wb.
Quoted 21 lines
Quoted 17 lines
fs/fs-writeback.c static void inode_switch_wbs(struct inode *inode, int new_wb_id) · line 577
Quoted 10 lines
fs/fs-writeback.c +10
-0+ +list_for_each_entry_safe (inode, tmp, &wb->b_attached, i_io_list) { +if (!spin_trylock (&inode->i_lock)) +continue ; +xa_lock_irq (&inode->i_mapping->i_pages); +if (!(inode->i_state & + (I_FREEING |I_CLEAR |I_SYNC |I_DIRTY |I_WB_SWITCH ))) { +WARN_ON_ONCE (inode->i_wb != wb); + inode->i_wb =NULL ; +wb_put (wb);Hum, currently the code assumes that once
i_wbis set, it never becomes NULL again. In particular the inode e.g. in fs/fs-writeback.c:inode_congested() or generallyunlocked_inode_to_wb_begin()users could get broken by this. Thei_wbswitching code is so complex exactly because of these interactions.Quoted 3 lines
Maybe you thought through the interactions and things are actually fine but if nothing else you'd need a big fat comment here explaining why this is fine and update
inode_congested()comments etc.
Quoted 7 lines
Yeah, I thought that once inode is clean and not switching it's safe to clear the
i_wbpointer, but seems that it's not completely true.One idea I have is to always release wbs using rcu delayed work, so that it will be save to dereference
i_wbpointer under rcu, if only it's not NULL (the check has to be added). I'll try to implement this scheme, but if you know in advance that it's not gonna work, please, let me know.
I think I'd just drop inode_to_wb_is_valid() because once i_wb can change to NULL, that function is just pointless in that single callsite. Also we have to count with the fact that unlocked_inode_to_wb_begin() can return NULL and gracefully do as much as possible in that case for all the callers. And I agree that those occurences in mm/page-writeback.c should be blocked by inode being clean and you holding all those locks so you can warn if that happens I guess.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 35 lines · Jan Kara
Quoted 17 lines · Roman Gushchin
Quoted 6 lines · Jan Kara
Quoted 4 lines · Roman Gushchin
fs/fs-writeback.c +0
-0@@ -426,7 +431,7 @@ static void inode_switch_wbs_work_fn(struct work_struct *work)if (!list_empty (&inode->i_io_list)) {struct inode *pos;Quoted 9 lines
fs/fs-writeback.c static void inode_switch_wbs_work_fn(struct work_struct *work) · line 437
Quoted 5 lines
fs/fs-writeback.c +1
-1-inode_io_list_del_locked (inode, old_wb); +inode_io_list_del_locked (inode, old_wb, false); inode->i_wb = new_wb;list_for_each_entry (pos, &new_wb->b_dirty, i_io_list) if (time_after_eq(inode->dirtied_when,This bit looks wrong. Not the change you made as such but the fact that you can now move inode from
b_attachedlist of old wb to the dirty list of new wb.Quoted 2 lines
Hm, can you, please, elaborate a bit more why it's wrong? The reference to the
old_wbwill be dropped by the switching code.My point is that the code in full looks like:
if (!list_empty(&inode->i_io_list)) { struct inode *pos; inode_io_list_del_locked(inode, old_wb); inode->i_wb = new_wb; list_for_each_entry(pos, &new_wb->b_dirty, i_io_list) if (time_after_eq(inode->dirtied_when, pos->dirtied_when)) break; inode_io_list_move_locked(inode, new_wb, pos->i_io_list.prev); } else {
Quoted 4 lines
So inode is always moved from some io list in
old_wbtob_dirtylist ofnew_wb. This is fine when it could be only onb_dirty,b_io,b_more_iolists ofold_wb. But once you addb_attachedlist to the game, it is not correct anymore. You should not add clean inode tob_dirtylist ofnew_wb.
I see...
Hm, will checking of i_state for not containing I_DIRTY_ALL bits be enough here? Alternatively, I can introduce a new bit which will explicitly point at the inode being on the b_attached list, but I'd prefer not to do it.
Quoted 30 lines
Quoted 22 lines
Quoted 18 lines
fs/fs-writeback.c static void inode_switch_wbs(struct inode *inode, int new_wb_id) · line 577
Quoted 11 lines
fs/fs-writeback.c +10
-0+ +list_for_each_entry_safe (inode, tmp, &wb->b_attached, i_io_list) { +if (!spin_trylock (&inode->i_lock)) +continue ; +xa_lock_irq (&inode->i_mapping->i_pages); +if (!(inode->i_state & + (I_FREEING |I_CLEAR |I_SYNC |I_DIRTY |I_WB_SWITCH ))) { +WARN_ON_ONCE (inode->i_wb != wb); + inode->i_wb =NULL ; +wb_put (wb);Hum, currently the code assumes that once
i_wbis set, it never becomes NULL again. In particular the inode e.g. in fs/fs-writeback.c:inode_congested() or generallyunlocked_inode_to_wb_begin()users could get broken by this. Thei_wbswitching code is so complex exactly because of these interactions.Quoted 3 lines
Maybe you thought through the interactions and things are actually fine but if nothing else you'd need a big fat comment here explaining why this is fine and update
inode_congested()comments etc.Quoted 7 lines
Yeah, I thought that once inode is clean and not switching it's safe to clear the
i_wbpointer, but seems that it's not completely true.One idea I have is to always release wbs using rcu delayed work, so that it will be save to dereference
i_wbpointer under rcu, if only it's not NULL (the check has to be added). I'll try to implement this scheme, but if you know in advance that it's not gonna work, please, let me know.
Quoted 7 lines
I think I'd just drop
inode_to_wb_is_valid()because oncei_wbcan change to NULL, that function is just pointless in that single callsite. Also we have to count with the fact thatunlocked_inode_to_wb_begin()can return NULL and gracefully do as much as possible in that case for all the callers. And I agree that those occurences inmm/page-writeback.cshould be blocked by inode being clean and you holding all those locks so you can warn if that happens I guess.
Yeah, it sounds good to me. I actually have a patch, which I'll post after some more extensive testing (want to make sure I do not hit these warns).
Thank you!
Jan Kara
maintainerQuoted 41 lines · Roman Gushchin
Quoted 35 lines · Jan Kara
Quoted 17 lines · Roman Gushchin
Quoted 6 lines · Jan Kara
Quoted 4 lines · Roman Gushchin
fs/fs-writeback.c +0
-0@@ -426,7 +431,7 @@ static void inode_switch_wbs_work_fn(struct work_struct *work)if (!list_empty (&inode->i_io_list)) {struct inode *pos;Quoted 9 lines
fs/fs-writeback.c static void inode_switch_wbs_work_fn(struct work_struct *work) · line 437
Quoted 5 lines
fs/fs-writeback.c +1
-1-inode_io_list_del_locked (inode, old_wb); +inode_io_list_del_locked (inode, old_wb, false); inode->i_wb = new_wb;list_for_each_entry (pos, &new_wb->b_dirty, i_io_list) if (time_after_eq(inode->dirtied_when,This bit looks wrong. Not the change you made as such but the fact that you can now move inode from
b_attachedlist of old wb to the dirty list of new wb.Quoted 2 lines
Hm, can you, please, elaborate a bit more why it's wrong? The reference to the
old_wbwill be dropped by the switching code.My point is that the code in full looks like:
if (!list_empty(&inode->i_io_list)) { struct inode *pos; inode_io_list_del_locked(inode, old_wb); inode->i_wb = new_wb; list_for_each_entry(pos, &new_wb->b_dirty, i_io_list) if (time_after_eq(inode->dirtied_when, pos->dirtied_when)) break; inode_io_list_move_locked(inode, new_wb, pos->i_io_list.prev); } else {Quoted 4 lines
So inode is always moved from some io list in
old_wbtob_dirtylist ofnew_wb. This is fine when it could be only onb_dirty,b_io,b_more_iolists ofold_wb. But once you addb_attachedlist to the game, it is not correct anymore. You should not add clean inode tob_dirtylist ofnew_wb.
Quoted 5 lines
I see...
Hm, will checking of
i_statefor not containingI_DIRTY_ALLbits be enough here? Alternatively, I can introduce a new bit which will explicitly point at the inode being on theb_attachedlist, but I'd prefer not to do it.
Yeah, keying of i_state should work. And while we are at it, we could also correctly handle I_DIRTY_TIME case and move inode only to b_dirty_time list. That seems to be (mostly harmless) preexisting issue.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR