Skip to contentSkip to conversation

staging/lustre/osc: Revert erroneous list_for_each_entry_safe use

PATCH In review Oleg Drokin 2 participants 5 messages Dec 7, 2016 Staging

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 staging/lustre/osc: Revert erroneous list_for_each_entry_safe use

I have been having a lot of unexplainable crashes in osc_lru_shrink lately that I could not see a good explanation for and then I found this patch that slip under the radar somehow that incorrectly converted while loop for lru list iteration into list_for_each_entry_safe totally ignoring that in the body of the loop we drop spinlocks guarding this list and move list entries around.
Not sure why it was not showing up right away, perhaps some of the more recent LRU changes committed caused some extra pressure on this code that finally highlighted the breakage.

Reverts: 8adddc36b1fc ("staging: lustre: osc: Use list_for_each_entry_safe")

CC Bhaktipriya Shridhar <[email protected]>

---
I also do not see this patch in any of the mailing lists I am subscribed to. I wonder if there's a way to subscribe to those Greg's
"This is a note to let you know that I've just added the patch ...." emails that concern Lustre to get them even if I am not on the CC list in the patch itself?

 drivers/staging/lustre/lustre/osc/osc_page.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/staging/lustre/lustre/osc/osc_page.c +3 -2
index c5129d1..e356e4a 100644
@@ -537,7 +537,6 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli,
 	struct cl_object *clobj = NULL;
 	struct cl_page **pvec;
 	struct osc_page *opg;
-	struct osc_page *temp;
 	int maxscan = 0;
 	long count = 0;
 	int index = 0;
@@ -568,7 +567,7 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli,
 	if (force)
 		cli->cl_lru_reclaim++;
 	maxscan = min(target << 1, atomic_long_read(&cli->cl_lru_in_list));
-	list_for_each_entry_safe(opg, temp, &cli->cl_lru_list, ops_lru) {
+	while (!list_empty(&cli->cl_lru_list)) {
 		struct cl_page *page;
 		bool will_free = false;
 
@@ -578,6 +577,8 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli,
 		if (--maxscan < 0)
 			break;
 
+		opg = list_entry(cli->cl_lru_list.next, struct osc_page,
+				 ops_lru);
 		page = opg->ops_cl.cpl_page;
 		if (lru_page_busy(cli, page)) {
 			list_move_tail(&opg->ops_lru, &cli->cl_lru_list);
Signature · 1 line
2.7.4
#1 lore raw
Quoted 12 lines · Oleg Drokin

I have been having a lot of unexplainable crashes in osc_lru_shrink lately that I could not see a good explanation for and then I found this patch that slip under the radar somehow that incorrectly converted while loop for lru list iteration into list_for_each_entry_safe totally ignoring that in the body of the loop we drop spinlocks guarding this list and move list entries around.
Not sure why it was not showing up right away, perhaps some of the more recent LRU changes committed caused some extra pressure on this code that finally highlighted the breakage.

Reverts: 8adddc36b1fc ("staging: lustre: osc: Use list_for_each_entry_safe")

Quoted 8 lines
CC Bhaktipriya Shridhar <[email protected]>

---
I also do not see this patch in any of the mailing lists I am subscribed to. I wonder if there's a way to subscribe to those Greg's
"This is a note to let you know that I've just added the patch ...." emails that concern Lustre to get them even if I am not on the CC list in the patch itself?

This came in from the Outreacy application process, which now requires that they cc: the maintainers to catch this type of issue. So you should have seen these types of patches this last round, the commit you reference was done before that change happened, sorry.

This change should go to stable kernels, so I'll mark it that way.

thanks,

greg k-h

#2 lore raw
Quoted 22 lines · Greg Kroah-Hartman
Quoted 12 lines · Oleg Drokin

I have been having a lot of unexplainable crashes in osc_lru_shrink lately that I could not see a good explanation for and then I found this patch that slip under the radar somehow that incorrectly converted while loop for lru list iteration into list_for_each_entry_safe totally ignoring that in the body of the loop we drop spinlocks guarding this list and move list entries around.
Not sure why it was not showing up right away, perhaps some of the more recent LRU changes committed caused some extra pressure on this code that finally highlighted the breakage.

Reverts: 8adddc36b1fc ("staging: lustre: osc: Use list_for_each_entry_safe")

Quoted 8 lines
CC Bhaktipriya Shridhar <[email protected]>

---
I also do not see this patch in any of the mailing lists I am subscribed to. I wonder if there's a way to subscribe to those Greg's
"This is a note to let you know that I've just added the patch ...." emails that concern Lustre to get them even if I am not on the CC list in the patch itself?

Quoted 4 lines

This came in from the Outreacy application process, which now requires that they cc: the maintainers to catch this type of issue. So you should have seen these types of patches this last round, the commit you reference was done before that change happened, sorry.

Do you know approximate date range of when these patches ere sneaking in? I'd like to take a look at the rest of it proactively just to see if there are more undiscovered surprises?

Quoted 1 line

This change should go to stable kernels, so I'll mark it that way.

Thanks!

#3 lore raw
Quoted 25 lines · Oleg Drokin
Quoted 22 lines · Greg Kroah-Hartman
Quoted 20 lines · Oleg Drokin

I have been having a lot of unexplainable crashes in osc_lru_shrink lately that I could not see a good explanation for and then I found this patch that slip under the radar somehow that incorrectly converted while loop for lru list iteration into list_for_each_entry_safe totally ignoring that in the body of the loop we drop spinlocks guarding this list and move list entries around.
Not sure why it was not showing up right away, perhaps some of the more recent LRU changes committed caused some extra pressure on this code that finally highlighted the breakage.

Reverts: 8adddc36b1fc ("staging: lustre: osc: Use list_for_each_entry_safe")

CC Bhaktipriya Shridhar <[email protected]>

---
I also do not see this patch in any of the mailing lists I am subscribed to. I wonder if there's a way to subscribe to those Greg's
"This is a note to let you know that I've just added the patch ...." emails that concern Lustre to get them even if I am not on the CC list in the patch itself?

Quoted 6 lines
Quoted 4 lines

This came in from the Outreacy application process, which now requires that they cc: the maintainers to catch this type of issue. So you should have seen these types of patches this last round, the commit you reference was done before that change happened, sorry.

Do you know approximate date range of when these patches ere sneaking in?

Anytime before a few months ago.

Quoted 2 lines

I'd like to take a look at the rest of it proactively just to see if there are more undiscovered surprises?

If your testing isn't finding any problems, all should be good, right? :)

#4 lore raw
Quoted 26 lines · Greg Kroah-Hartman
Quoted 25 lines · Oleg Drokin
Quoted 22 lines · Greg Kroah-Hartman
Quoted 20 lines · Oleg Drokin

I have been having a lot of unexplainable crashes in osc_lru_shrink lately that I could not see a good explanation for and then I found this patch that slip under the radar somehow that incorrectly converted while loop for lru list iteration into list_for_each_entry_safe totally ignoring that in the body of the loop we drop spinlocks guarding this list and move list entries around.
Not sure why it was not showing up right away, perhaps some of the more recent LRU changes committed caused some extra pressure on this code that finally highlighted the breakage.

Reverts: 8adddc36b1fc ("staging: lustre: osc: Use list_for_each_entry_safe")

CC Bhaktipriya Shridhar <[email protected]>

---
I also do not see this patch in any of the mailing lists I am subscribed to. I wonder if there's a way to subscribe to those Greg's
"This is a note to let you know that I've just added the patch ...." emails that concern Lustre to get them even if I am not on the CC list in the patch itself?

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

This came in from the Outreacy application process, which now requires that they cc: the maintainers to catch this type of issue. So you should have seen these types of patches this last round, the commit you reference was done before that change happened, sorry.

Do you know approximate date range of when these patches ere sneaking in?

Anytime before a few months ago.

Ugh, I see.

Quoted 5 lines
Quoted 2 lines

I'd like to take a look at the rest of it proactively just to see if there are more undiscovered surprises?

If your testing isn't finding any problems, all should be good, right? :)

I see processes hanging waiting for RPC response (rarely) that is very suspicious, but I did not get to the root of it yet.
Also my test system is limited in capacity, they don't let me anywhere near those TOP100 systems with the staging client ;)