Skip to contentSkip to conversation

rbtree: clarify documentation of rbtree_postorder_for_each_entry_safe()

PATCH In review Cody P Schafer 2 participants 3 messages Oct 4, 2015 – Oct 13, 2015

Subsystems (2)
Memory management MM misc
lore
Share & tools
lore

Cody P Schafer

author
#0 lore raw

PATCH 1/1 rbtree: clarify documentation of rbtree_postorder_for_each_entry_safe()

Noticed that commit a20135ffbc44 ("writeback: don't drain bdi_writeback_congested on bdi destruction") added a usage of rbtree_postorder_for_each_entry_safe() in mm/backing-dev.c which appears to try to rb_erase() elements from an rbtree while iterating over it using rbtree_postorder_for_each_entry_safe().

Doing this will cause random nodes to be missed by the iteration because rb_erase() may rebalance the tree, changing the ordering that we're trying to iterate over.

The previous documentation for rbtree_postorder_for_each_entry_safe() wasn't clear that this wasn't allowed, it was taken from the docs for list_for_each_entry_safe(), where erasing isn't a problem due to list_del() not reordering.

Explicitly warn developers about this potential pit-fall.

Note that I haven't fixed the actual issue that (it appears) the commit referenced above introduced (not familiar enough with that code).

In general (and in this case), the patterns to follow are:
- switch to rb_first() + rb_erase(), don't use rbtree_postorder_for_each_entry_safe().
- keep the postorder iteration and don't rb_erase() at all. Instead just clear the fields of rb_node & cgwb_congested_tree as required by other users of those structures.

CC Tejun Heo <[email protected]>
Signed-off-by Cody P Schafer <[email protected]>
---
 include/linux/rbtree.h | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
include/linux/rbtree.h +10 -2
index 830c499..39de3df 100644
@@ -101,13 +101,21 @@ static inline void rb_link_node_rcu(struct rb_node *node, struct rb_node *parent
 	})
 
 /**
- * rbtree_postorder_for_each_entry_safe - iterate over rb_root in post order of
- * given type safe against removal of rb_node entry
+ * rbtree_postorder_for_each_entry_safe - iterate in post-order over rb_root of
+ * given type allowing the backing memory of @pos to be invalidated
  *
  * @pos:	the 'type *' to use as a loop cursor.
  * @n:		another 'type *' to use as temporary storage
  * @root:	'rb_root *' of the rbtree.
  * @field:	the name of the rb_node field within 'type'.
+ *
+ * This function provides a similar guarantee as list_for_each_entry_safe() and
+ * allows the iteration to continue independent of changes to @pos by the body
+ * of the loop.
+ *
+ * Note, however, that it cannot handle other modifications that re-order the
+ * rbtree it is iterating over. This includes calling rb_erase() on @pos, as
+ * rb_erase() may rebalance the tree, causing us to miss some nodes.
  */
 #define rbtree_postorder_for_each_entry_safe(pos, n, root, field) \
 	for (pos = rb_entry_safe(rb_first_postorder(root), typeof(*pos), field); \
Signature · 1 line
2.4.9
#1 lore raw

Hello,

Quoted 8 lines · Cody P Schafer

Doing this will cause random nodes to be missed by the iteration because rb_erase() may rebalance the tree, changing the ordering that we're trying to iterate over.

The previous documentation for rbtree_postorder_for_each_entry_safe() wasn't clear that this wasn't allowed, it was taken from the docs for list_for_each_entry_safe(), where erasing isn't a problem due to list_del() not reordering.

Ugh... that's a misleading name for an iterator if it doesn't allow removal of elements during iteration.

Quoted 8 lines

Explicitly warn developers about this potential pit-fall.

Note that I haven't fixed the actual issue that (it appears) the commit referenced above introduced (not familiar enough with that code).

In general (and in this case), the patterns to follow are:
- switch to rb_first() + rb_erase(), don't use rbtree_postorder_for_each_entry_safe().

I'll update it to a while loop on rb_first().

Thanks.

Signature · 1 line
tejun
#2 lore raw

PATCH 1/1 writeback: remove broken rbtree_postorder_for_each_entry_safe() usage in cgwb_bdi_destroy()

In reply to Cody P Schafer

a20135ffbc44 ("writeback: don't drain bdi_writeback_congested on bdi destruction") added rbtree_postorder_for_each_entry_safe() which is used to remove all entries; however, according to Cody, the iterator isn't safe against operations which may rebalance the tree. Fix it by switching to repeatedly removing rb_first() until empty.

Signed-off-by Tejun Heo <[email protected]>
Reported-by Cody P Schafer <[email protected]>
Fixes a20135ffbc44 ("writeback: don't drain bdi_writeback_congested on bdi destruction")
Link http://lkml.kernel.org/g/[email protected] open here
---
 mm/backing-dev.c |   10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
mm/backing-dev.c +6 -4
@@ -681,7 +681,7 @@ static int cgwb_bdi_init(struct backing_
 static void cgwb_bdi_destroy(struct backing_dev_info *bdi)
 {
 	struct radix_tree_iter iter;
-	struct bdi_writeback_congested *congested, *congested_n;
+	struct rb_node *rbn;
 	void **slot;
 
 	WARN_ON(test_bit(WB_registered, &bdi->wb.state));

@@ -691,9 +691,11 @@ static void cgwb_bdi_destroy(struct back
 	radix_tree_for_each_slot(slot, &bdi->cgwb_tree, &iter, 0)
 		cgwb_kill(*slot);
 
-	rbtree_postorder_for_each_entry_safe(congested, congested_n,
-					&bdi->cgwb_congested_tree, rb_node) {
-		rb_erase(&congested->rb_node, &bdi->cgwb_congested_tree);
+	while ((rbn = rb_first(&bdi->cgwb_congested_tree))) {
+		struct bdi_writeback_congested *congested =
+			rb_entry(rbn, struct bdi_writeback_congested, rb_node);
+
+		rb_erase(rbn, &bdi->cgwb_congested_tree);
 		congested->bdi = NULL;	/* mark @congested unlinked */
 	}