md: do not use ++ in rcu_dereference() argument
Subsystems (4)
PATCH 1/1 md: do not use ++ in rcu_dereference() argument
From: Vasiliy Kulikov <[email protected]>
rcu_dereference() is macro, so it might use its argument twice. Argument must not has side effects.
It was found by compiler warning:drivers/md/raid1.c: In function ‘read_balance’:drivers/md/raid1.c:445: warning: operation on ‘new_disk’ may be undefined
--- drivers/md/raid1.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
drivers/md/raid1.c +2 -1
@@ -442,7 +442,7 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) r1_bio->bios[new_disk] =IO_BLOCKED || !rdev || !test_bit (In_sync, &rdev->flags) ||test_bit (WriteMostly, &rdev->flags); - rdev = rcu_dereference(conf->mirrors[++new_disk].rdev)) { + rdev = rcu_dereference(conf->mirrors[new_disk].rdev)) {if (rdev &&test_bit (In_sync, &rdev->flags) && r1_bio->bios[new_disk] !=IO_BLOCKED )
@@ -452,6 +452,7 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) new_disk = wonly_disk;break ; } + new_disk++; } goto rb_out; }
Signature · 1 line
1.7.0.4
Quoted 28 lines · Kulikov Vasiliy
From: Vasiliy Kulikov <[email protected]>
rcu_dereference()is macro, so it might use its argument twice. Argument must not has side effects.It was found by compiler warning:
drivers/md/raid1.c: In function ‘read_balance’:drivers/md/raid1.c:445: warning: operation on ‘new_disk’ may be undefinedSigned-off-by Vasiliy Kulikov <[email protected]>--- drivers/md/raid1.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)drivers/md/raid1.c +1
-1@@ -442,7 +442,7 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) r1_bio->bios[new_disk] =IO_BLOCKED || !rdev || !test_bit (In_sync, &rdev->flags) ||test_bit (WriteMostly, &rdev->flags); - rdev = rcu_dereference(conf->mirrors[++new_disk].rdev)) { + rdev = rcu_dereference(conf->mirrors[new_disk].rdev)) {if (rdev &&test_bit (In_sync, &rdev->flags) && r1_bio->bios[new_disk] !=IO_BLOCKED )
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 457
Quoted 6 lines
drivers/md/raid1.c +1
-0@@ -452,6 +452,7 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) new_disk = wonly_disk;break ; } + new_disk++; } goto rb_out;
This change looks wrong.
In the original implementation new_disk is incremented and then we do the array lookup.
With your implementation it looks like we increment it after the array lookup.
Sam
Quoted 10 lines · Sam Ravnborg
Quoted 8 lines · Kulikov Vasiliy
From: Vasiliy Kulikov <[email protected]>
rcu_dereference()is macro, so it might use its argument twice. Argument must not has side effects.It was found by compiler warning:
drivers/md/raid1.c: In function ‘read_balance’:drivers/md/raid1.c:445: warning: operation on ‘new_disk’ may be undefined
Quoted 5 lines
This change looks wrong.
In the original implementationnew_diskis incremented and then we do the array lookup.
With your implementation it looks like we increment it after the array lookup.
No, the original code increments new_disk and then dereferences mirrors.
The full code:
for (rdev = rcu_dereference(conf->mirrors[new_disk].rdev);
r1_bio->bios[new_disk] = IO_BLOCKED ||
!rdev || !test_bit(In_sync, &rdev->flags)
|| test_bit(WriteMostly, &rdev->flags);
rdev = rcu_dereference(conf->mirrors[++new_disk].rdev)) {
if (rdev && test_bit(In_sync, &rdev->flags) &&
r1_bio->bios[new_disk] != IO_BLOCKED)
wonly_disk = new_disk;
if (new_disk = conf->raid_disks - 1) {
new_disk = wonly_disk;
break;
}
}
so,
for (a; b; c = f(++g)) {
...
}
=
a;
while (b) {
...
l_continue:
c = f(++g);
}
=
a;
while (b) {
...
l_continue:
g++;
c = f(g);
}
=
for (a; b; c = f(g)) {
...
g++;
} Or you mean smth more?
Signature · 1 line
Vasiliy
Quoted 34 lines · Kulikov Vasiliy
Quoted 10 lines · Sam Ravnborg
Quoted 8 lines · Kulikov Vasiliy
From: Vasiliy Kulikov <[email protected]>
rcu_dereference()is macro, so it might use its argument twice. Argument must not has side effects.It was found by compiler warning:
drivers/md/raid1.c: In function ‘read_balance’:drivers/md/raid1.c:445: warning: operation on ‘new_disk’ may be undefinedQuoted 5 lines
This change looks wrong.
In the original implementationnew_diskis incremented and then we do the array lookup.
With your implementation it looks like we increment it after the array lookup.No, the original code increments
new_diskand then dereferences mirrors.The full code:
for (rdev = rcu_dereference(conf->mirrors[new_disk].rdev); r1_bio->bios[new_disk] = IO_BLOCKED || !rdev || !test_bit(In_sync, &rdev->flags) || test_bit(WriteMostly, &rdev->flags); rdev = rcu_dereference(conf->mirrors[++new_disk].rdev)) { if (rdev && test_bit(In_sync, &rdev->flags) && r1_bio->bios[new_disk] != IO_BLOCKED) wonly_disk = new_disk; if (new_disk = conf->raid_disks - 1) { new_disk = wonly_disk; break;
Quoted 8 lines
}
}
so,
for (a; b; c = f(++g)) {
...
} Thanks - that explains it.
This code really screams for a helper function but thats another matter.
Sam
Quoted 38 lines · On Sun, 5 Sep 2010 22:39:08 +0200 Sam Ravnborg <[email protected]> wrote:
Quoted 37 lines · Kulikov Vasiliy
Quoted 10 lines · Sam Ravnborg
Quoted 8 lines · Kulikov Vasiliy
From: Vasiliy Kulikov <[email protected]>
rcu_dereference()is macro, so it might use its argument twice. Argument must not has side effects.It was found by compiler warning:
drivers/md/raid1.c: In function ‘read_balance’:drivers/md/raid1.c:445: warning: operation on ‘new_disk’ may be undefinedQuoted 5 lines
This change looks wrong.
In the original implementationnew_diskis incremented and then we do the array lookup.
With your implementation it looks like we increment it after the array lookup.No, the original code increments
new_diskand then dereferences mirrors.The full code:
for (rdev = rcu_dereference(conf->mirrors[new_disk].rdev); r1_bio->bios[new_disk] = IO_BLOCKED || !rdev || !test_bit(In_sync, &rdev->flags) || test_bit(WriteMostly, &rdev->flags); rdev = rcu_dereference(conf->mirrors[++new_disk].rdev)) { if (rdev && test_bit(In_sync, &rdev->flags) && r1_bio->bios[new_disk] != IO_BLOCKED) wonly_disk = new_disk; if (new_disk = conf->raid_disks - 1) { new_disk = wonly_disk; break; } }
Quoted 8 lines
Quoted 5 lines
so, for (a; b; c = f(++g)) { ... }Thanks - that explains it.
This code really screams for a helper function but thats another matter.
Not an uncommon complaint about my code as it happens......
I've taken the opportunity to substantially re-write that code.
Comments?
Thanks,
NeilBrown
commit e4062735c8f7233923df5858ed20f1278f3ee669
Author: NeilBrown <[email protected]> Date: Mon Sep 6 14:10:08 2010 +1000
md: tidy up device searches in read_balance.
We have a pre-increment side-effect in the arg to a macro:
rcu_dereference
This is poor form and triggers a warning. Rather than just fix that, take the opportunity to re-write the code it make it more readable.
drivers/md/raid1.c +38 -57
@@ -420,11 +420,13 @@ static void raid1_end_write_request(struct bio *bio, int error)static int read_balance (conf_t *conf,r1bio_t *r1_bio) {const sector_t this_sector = r1_bio->sector; -int new_disk = conf->last_used, disk = new_disk; -int wonly_disk =-1 ; +int new_disk =-1 ; +int start_disk; +int i;const int sectors = r1_bio->sectors;sector_t new_distance, current_distance;mdk_rdev_t *rdev; +int choose_first;rcu_read_lock ();/*
@@ -435,54 +437,35 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) retry:if (conf->mddev->recovery_cp < MaxSector && (this_sector + sectors >= conf->next_resync)) { -/* Choose the first operational device, for consistancy */ - new_disk =0 ; - -for (rdev =rcu_dereference (conf->mirrors[new_disk].rdev); - r1_bio->bios[new_disk] =IO_BLOCKED || - !rdev || !test_bit (In_sync, &rdev->flags) - ||test_bit (WriteMostly, &rdev->flags); - rdev =rcu_dereference (conf->mirrors[++new_disk].rdev)) { - -if (rdev &&test_bit (In_sync, &rdev->flags) && - r1_bio->bios[new_disk] !=IO_BLOCKED ) - wonly_disk = new_disk; - -if (new_disk = conf->raid_disks -1 ) { - new_disk = wonly_disk; -break ; - } - } - goto rb_out; + choose_first =1 ; + start_disk =0 ; + }else { + choose_first =0 ; + start_disk = conf->last_used; } -/* make sure the disk is operational */ -for (rdev =rcu_dereference (conf->mirrors[new_disk].rdev); - r1_bio->bios[new_disk] =IO_BLOCKED || - !rdev || !test_bit (In_sync, &rdev->flags) || -test_bit (WriteMostly, &rdev->flags); - rdev =rcu_dereference (conf->mirrors[new_disk].rdev)) { - -if (rdev &&test_bit (In_sync, &rdev->flags) && - r1_bio->bios[new_disk] !=IO_BLOCKED ) - wonly_disk = new_disk; - -if (new_disk <=0 ) - new_disk = conf->raid_disks; - new_disk--; -if (new_disk = disk) { - new_disk = wonly_disk; -break ; +for (i =0 ; i < conf->raid_disks ; i++) { +int disk = start_disk + i; +if (disk >= conf->raid_disks) + disk -= conf->raid_disks; + +if (r1_bio->bios[disk] =IO_BLOCKED + || !(rdev =rcu_dereference (conf->mirrors[disk].rdev)) + || !test_bit (In_sync, &rdev->flags)) +continue ; + +if (test_bit (WriteMostly, &rdev->flags)) { + new_disk = disk; +continue ; } + new_disk = disk; +break ; } -if (new_disk <0 ) +if (new_disk <0 || choose_first) goto rb_out; - disk = new_disk; -/* now disk = new_disk = starting point for search */ -/* * Don't change to another disk for sequential reads: */
@@ -491,20 +474,20 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio)if (this_sector = conf->mirrors[new_disk].head_position) goto rb_out; - current_distance =abs (this_sector - conf->mirrors[disk].head_position); + current_distance =abs (this_sector + - conf->mirrors[new_disk].head_position); -/* Find the disk whose head is closest */ +/* look for a better disk - i.e. head is closer */ + start_disk = new_disk; +for (i =1 ; i < conf->raid_disks; i++) { +int disk = start_disk +1 ; +if (disk >= conf->raid_disks) + disk -= conf->raid_disks; - do { - if (disk <= 0) - disk = conf->raid_disks; - disk--; - - rdev = rcu_dereference(conf->mirrors[disk].rdev); - - if (!rdev || r1_bio->bios[disk] = IO_BLOCKED || - !test_bit(In_sync, &rdev->flags) || - test_bit(WriteMostly, &rdev->flags)) +if (r1_bio->bios[disk] =IO_BLOCKED + || !(rdev =rcu_dereference (conf->mirrors[disk].rdev)) + || !test_bit (In_sync, &rdev->flags) + ||test_bit (WriteMostly, &rdev->flags))continue ;if (!atomic_read (&rdev->nr_pending)) {
@@ -516,11 +499,9 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) current_distance = new_distance; new_disk = disk; } - }while (disk != conf->last_used); + } rb_out: - -if (new_disk >=0 ) { rdev =rcu_dereference (conf->mirrors[new_disk].rdev);if (!rdev)
Quoted 111 lines · Neil Brown
Quoted 36 lines · On Sun, 5 Sep 2010 22:39:08 +0200 Sam Ravnborg <[email protected]> wrote:
Quoted 35 lines · Kulikov Vasiliy
Quoted 6 lines · Sam Ravnborg
Quoted 5 lines · Kulikov Vasiliy
From: Vasiliy Kulikov <[email protected]>
rcu_dereference()is macro, so it might use its argument twice. Argument must not has side effects.Quoted 8 lines
Quoted 3 lines
It was found by compiler warning:
drivers/md/raid1.c: In function ‘read_balance’:drivers/md/raid1.c:445: warning: operation on ‘new_disk’ may be undefinedThis change looks wrong.
In the original implementationnew_diskis incremented and then we do the array lookup.
With your implementation it looks like we increment it after the array lookup.No, the original code increments
new_diskand then dereferences mirrors.The full code:
for (rdev = rcu_dereference(conf->mirrors[new_disk].rdev); r1_bio->bios[new_disk] = IO_BLOCKED || !rdev || !test_bit(In_sync, &rdev->flags) || test_bit(WriteMostly, &rdev->flags); rdev = rcu_dereference(conf->mirrors[++new_disk].rdev)) { if (rdev && test_bit(In_sync, &rdev->flags) && r1_bio->bios[new_disk] != IO_BLOCKED) wonly_disk = new_disk; if (new_disk = conf->raid_disks - 1) { new_disk = wonly_disk; break; } }Quoted 7 lines
Quoted 5 lines
so, for (a; b; c = f(++g)) { ... }Thanks - that explains it.
This code really screams for a helper function but thats another matter.Not an uncommon complaint about my code as it happens......
I've taken the opportunity to substantially re-write that code.
Comments?
Thanks,
NeilBrowncommit
e4062735c8f7233923df5858ed20f1278f3ee669Author: NeilBrown <[email protected]> Date: Mon Sep 6 14:10:08 2010 +1000md: tidy up device searches in
read_balance.We have a pre-increment side-effect in the arg to a macro:
rcu_dereferenceThis is poor form and triggers a warning. Rather than just fix that, take the opportunity to re-write the code it make it more readable.
Reported-by Kulikov Vasiliy <[email protected]>Signed-off-by NeilBrown <[email protected]>drivers/md/raid1.c +4
-18@@ -420,11 +420,13 @@ static void raid1_end_write_request(struct bio *bio, int error)static int read_balance (conf_t *conf,r1bio_t *r1_bio) {const sector_t this_sector = r1_bio->sector; -int new_disk = conf->last_used, disk = new_disk; -int wonly_disk =-1 ; +int new_disk =-1 ; +int start_disk; +int i;const int sectors = r1_bio->sectors;sector_t new_distance, current_distance;mdk_rdev_t *rdev; +int choose_first;rcu_read_lock ();/* @@ -435,54 +437,35 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) retry:if (conf->mddev->recovery_cp < MaxSector && (this_sector + sectors >= conf->next_resync)) { -/* Choose the first operational device, for consistancy */ - new_disk =0 ; - -for (rdev =rcu_dereference (conf->mirrors[new_disk].rdev); - r1_bio->bios[new_disk] =IO_BLOCKED || - !rdev || !test_bit (In_sync, &rdev->flags) - ||test_bit (WriteMostly, &rdev->flags); - rdev =rcu_dereference (conf->mirrors[++new_disk].rdev)) { - -if (rdev &&test_bit (In_sync, &rdev->flags) && - r1_bio->bios[new_disk] !=IO_BLOCKED ) - wonly_disk = new_disk; - -if (new_disk = conf->raid_disks -1 ) { - new_disk = wonly_disk; -break ;
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 446
Quoted 10 lines
drivers/md/raid1.c +5
-3- } - } - goto rb_out; + choose_first =1 ; + start_disk =0 ; + }else { + choose_first =0 ; + start_disk = conf->last_used; }
perhaps you can drop the else when you init withchoose_first = 0;start_disk = conf->last_used;
Quoted 17 lines
drivers/md/raid1.c +0
-16-/* make sure the disk is operational */ -for (rdev =rcu_dereference (conf->mirrors[new_disk].rdev); - r1_bio->bios[new_disk] =IO_BLOCKED || - !rdev || !test_bit (In_sync, &rdev->flags) || -test_bit (WriteMostly, &rdev->flags); - rdev =rcu_dereference (conf->mirrors[new_disk].rdev)) { - -if (rdev &&test_bit (In_sync, &rdev->flags) && - r1_bio->bios[new_disk] !=IO_BLOCKED ) - wonly_disk = new_disk; - -if (new_disk <=0 ) - new_disk = conf->raid_disks; - new_disk--; -if (new_disk = disk) { - new_disk = wonly_disk;
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 456
Quoted 10 lines
drivers/md/raid1.c +9
-1-break ; +for (i =0 ; i < conf->raid_disks ; i++) { +int disk = start_disk + i; +if (disk >= conf->raid_disks) + disk -= conf->raid_disks; + +if (r1_bio->bios[disk] =IO_BLOCKED + || !(rdev =rcu_dereference (conf->mirrors[disk].rdev)) + || !test_bit (In_sync, &rdev->flags)) +continue ;
i think it is more readable to use:
rdev = rcu_dereference(conf->mirrors[disk].rdev); if ()
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 464
Quoted 7 lines
drivers/md/raid1.c +5
-0+if (test_bit (WriteMostly, &rdev->flags)) { + new_disk = disk; +continue ; } + new_disk = disk; +break ; }
to improve readability:
new_disk = disk; if ( ! test_bit(WriteMostly, &rdev->flags) ) break;
Quoted 32 lines
drivers/md/raid1.c +9
-12-if (new_disk <0 ) +if (new_disk <0 || choose_first) goto rb_out; - disk = new_disk; -/* now disk = new_disk = starting point for search */ -/* * Don't change to another disk for sequential reads: */ @@ -491,20 +474,20 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio)if (this_sector = conf->mirrors[new_disk].head_position) goto rb_out; - current_distance =abs (this_sector - conf->mirrors[disk].head_position); + current_distance =abs (this_sector + - conf->mirrors[new_disk].head_position); -/* Find the disk whose head is closest */ +/* look for a better disk - i.e. head is closer */ + start_disk = new_disk; +for (i =1 ; i < conf->raid_disks; i++) { +int disk = start_disk +1 ; +if (disk >= conf->raid_disks) + disk -= conf->raid_disks; - do { - if (disk <= 0) - disk = conf->raid_disks; - disk--; - - rdev = rcu_dereference(conf->mirrors[disk].rdev);
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 492
Quoted 10 lines
drivers/md/raid1.c +4
-4- -if (!rdev || r1_bio->bios[disk] =IO_BLOCKED || - !test_bit (In_sync, &rdev->flags) || -test_bit (WriteMostly, &rdev->flags)) +if (r1_bio->bios[disk] =IO_BLOCKED + || !(rdev =rcu_dereference (conf->mirrors[disk].rdev)) + || !test_bit (In_sync, &rdev->flags) + ||test_bit (WriteMostly, &rdev->flags))continue ;
again i would set
rdev=rcu_dereference(conf->mirrors[disk].rdev));
before if() like it was in the original the statement is complex anything that reduces the complexity is good.
Quoted 6 lines
drivers/md/raid1.c +0
-0if (!atomic_read (&rdev->nr_pending)) {@@ -516,11 +499,9 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) current_distance = new_distance; new_disk = disk;
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 507
Quoted 10 lines
drivers/md/raid1.c +1
-3} - }while (disk != conf->last_used); + } rb_out: - -if (new_disk >=0 ) { rdev =rcu_dereference (conf->mirrors[new_disk].rdev);if (!rdev)
just my 2 cents,
re,
wh
Quoted 6 lines
--
To unsubscribe from this list: send the line "unsubscribe kernel-janitors" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 72 lines · On Mon, 06 Sep 2010 09:43:32 +0200 walter harms <[email protected]> wrote:
Quoted 69 lines · Neil Brown
I've taken the opportunity to substantially re-write that code.
Comments?
Thanks,
NeilBrowncommit
e4062735c8f7233923df5858ed20f1278f3ee669Author: NeilBrown <[email protected]> Date: Mon Sep 6 14:10:08 2010 +1000md: tidy up device searches in
read_balance.We have a pre-increment side-effect in the arg to a macro:
rcu_dereferenceThis is poor form and triggers a warning. Rather than just fix that, take the opportunity to re-write the code it make it more readable.
Reported-by Kulikov Vasiliy <[email protected]>Signed-off-by NeilBrown <[email protected]>drivers/md/raid1.c +8
-21@@ -420,11 +420,13 @@ static void raid1_end_write_request(struct bio *bio, int error)static int read_balance (conf_t *conf,r1bio_t *r1_bio) {const sector_t this_sector = r1_bio->sector; -int new_disk = conf->last_used, disk = new_disk; -int wonly_disk =-1 ; +int new_disk =-1 ; +int start_disk; +int i;const int sectors = r1_bio->sectors;sector_t new_distance, current_distance;mdk_rdev_t *rdev; +int choose_first;rcu_read_lock ();/* @@ -435,54 +437,35 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) retry:if (conf->mddev->recovery_cp < MaxSector && (this_sector + sectors >= conf->next_resync)) { -/* Choose the first operational device, for consistancy */ - new_disk =0 ; - -for (rdev =rcu_dereference (conf->mirrors[new_disk].rdev); - r1_bio->bios[new_disk] =IO_BLOCKED || - !rdev || !test_bit (In_sync, &rdev->flags) - ||test_bit (WriteMostly, &rdev->flags); - rdev =rcu_dereference (conf->mirrors[++new_disk].rdev)) { - -if (rdev &&test_bit (In_sync, &rdev->flags) && - r1_bio->bios[new_disk] !=IO_BLOCKED ) - wonly_disk = new_disk; - -if (new_disk = conf->raid_disks -1 ) { - new_disk = wonly_disk; -break ; - } - } - goto rb_out; + choose_first =1 ; + start_disk =0 ; + }else { + choose_first =0 ;
Quoted 8 lines
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 446
Quoted 3 lines
drivers/md/raid1.c +1
-0+ start_disk = conf->last_used; }perhaps you can drop the else when you init with
choose_first= 0;start_disk=conf->last_used;
Perhaps. Though given the 'retry' loop it isn't obvious that would do the right thing.
I think I'll keep this bit as-is. I think it helps see to two cases more clearly.
Quoted 10 lines
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 456
Quoted 4 lines
drivers/md/raid1.c +4
-0+if (r1_bio->bios[disk] =IO_BLOCKED + || !(rdev =rcu_dereference (conf->mirrors[disk].rdev)) + || !test_bit (In_sync, &rdev->flags)) +continue ;i think it is more readable to use:
rdev = rcu_dereference(conf->mirrors[disk].rdev); if ()
I think assignments inside 'if' statements have their place, but it seems that this is far from universal. I've made this change, thanks.
Quoted 12 lines
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 464
Quoted 9 lines
drivers/md/raid1.c +5
-0+if (test_bit (WriteMostly, &rdev->flags)) { + new_disk = disk; +continue ; } + new_disk = disk; +break ; }to improve readability:
Quoted 3 lines
new_disk = disk; if ( ! test_bit(WriteMostly, &rdev->flags) ) break;
Yes, that is a distinct improvement. I've made that change too.
Thanks a lot for your review!!
NeilBrown
Quoted 1 line
Comments?
Looks better but can still use a few improvements. See below.
Sam
Quoted 26 lines
Thanks,
NeilBrowncommit
e4062735c8f7233923df5858ed20f1278f3ee669Author: NeilBrown <[email protected]> Date: Mon Sep 6 14:10:08 2010 +1000md: tidy up device searches in
read_balance.We have a pre-increment side-effect in the arg to a macro:
rcu_dereferenceThis is poor form and triggers a warning. Rather than just fix that, take the opportunity to re-write the code it make it more readable.
Reported-by Kulikov Vasiliy <[email protected]>Signed-off-by NeilBrown <[email protected]>drivers/md/raid1.c +0
-0@@ -420,11 +420,13 @@ static void raid1_end_write_request(struct bio *bio, int error)static int read_balance (conf_t *conf,r1bio_t *r1_bio) {
drivers/md/raid1.c static void raid1_end_write_request(struct bio *bio, int error) · line 429
Quoted 10 lines
drivers/md/raid1.c +4
-2const sector_t this_sector = r1_bio->sector; -int new_disk = conf->last_used, disk = new_disk; -int wonly_disk =-1 ; +int new_disk =-1 ; +int start_disk; +int i;const int sectors = r1_bio->sectors;sector_t new_distance, current_distance;mdk_rdev_t *rdev; +int choose_first;
To increase readability the general recommendation is:
1) Sort variable definitions with the longest first.
2) Do not assing variables when they are defined, do that on a separate line below the variable definitions.
With one empty line after variable definitions.
Quoted 45 lines
drivers/md/raid1.c +5
-30rcu_read_lock ();/* @@ -435,54 +437,35 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) retry:if (conf->mddev->recovery_cp < MaxSector && (this_sector + sectors >= conf->next_resync)) { -/* Choose the first operational device, for consistancy */ - new_disk =0 ; - -for (rdev =rcu_dereference (conf->mirrors[new_disk].rdev); - r1_bio->bios[new_disk] =IO_BLOCKED || - !rdev || !test_bit (In_sync, &rdev->flags) - ||test_bit (WriteMostly, &rdev->flags); - rdev =rcu_dereference (conf->mirrors[++new_disk].rdev)) { - -if (rdev &&test_bit (In_sync, &rdev->flags) && - r1_bio->bios[new_disk] !=IO_BLOCKED ) - wonly_disk = new_disk; - -if (new_disk = conf->raid_disks -1 ) { - new_disk = wonly_disk; -break ; - } - } - goto rb_out; + choose_first =1 ; + start_disk =0 ; + }else { + choose_first =0 ; + start_disk = conf->last_used; } -/* make sure the disk is operational */ -for (rdev =rcu_dereference (conf->mirrors[new_disk].rdev); - r1_bio->bios[new_disk] =IO_BLOCKED || - !rdev || !test_bit (In_sync, &rdev->flags) || -test_bit (WriteMostly, &rdev->flags); - rdev =rcu_dereference (conf->mirrors[new_disk].rdev)) { - -if (rdev &&test_bit (In_sync, &rdev->flags) && - r1_bio->bios[new_disk] !=IO_BLOCKED ) - wonly_disk = new_disk; -
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 451
Quoted 10 lines
drivers/md/raid1.c +4
-6-if (new_disk <=0 ) - new_disk = conf->raid_disks; - new_disk--; -if (new_disk = disk) { - new_disk = wonly_disk; -break ; +for (i =0 ; i < conf->raid_disks ; i++) { +int disk = start_disk + i; +if (disk >= conf->raid_disks) + disk -= conf->raid_disks;
1) Please comment on the purpose of the for loop
2) See comments above aboyt variable definitions
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 455
Quoted 4 lines
drivers/md/raid1.c +4
-0+ +if (r1_bio->bios[disk] =IO_BLOCKED + || !(rdev =rcu_dereference (conf->mirrors[disk].rdev)) + || !test_bit (In_sync, &rdev->flags))
The rather complex expression - which includes a well hidden assignment - is repeated a few lines later.
Please use a helper function and do not use such hidden assignments.
Quoted 15 lines
drivers/md/raid1.c +7
-1+continue ; + +if (test_bit (WriteMostly, &rdev->flags)) { + new_disk = disk; +continue ; } + new_disk = disk; +break ; }@@ -491,20 +474,20 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio)if (this_sector = conf->mirrors[new_disk].head_position) goto rb_out; - current_distance =abs (this_sector - conf->mirrors[disk].head_position);
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 485
Quoted 10 lines
drivers/md/raid1.c +8
-1+ current_distance =abs (this_sector + - conf->mirrors[new_disk].head_position); -/* Find the disk whose head is closest */ +/* look for a better disk - i.e. head is closer */ + start_disk = new_disk; +for (i =1 ; i < conf->raid_disks; i++) { +int disk = start_disk +1 ; +if (disk >= conf->raid_disks) + disk -= conf->raid_disks;
See comments about for loop above.
I also cannot see why we suddenly start with 1 where the other almost identical for loop starts with 0?
If I wonder then someone else will wonder too => comment please.
Quoted 6 lines
drivers/md/raid1.c +0
-5- do { - if (disk <= 0) - disk = conf->raid_disks; - disk--; -
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 491
Quoted 10 lines
drivers/md/raid1.c +4
-5- rdev =rcu_dereference (conf->mirrors[disk].rdev); - -if (!rdev || r1_bio->bios[disk] =IO_BLOCKED || - !test_bit (In_sync, &rdev->flags) || -test_bit (WriteMostly, &rdev->flags)) +if (r1_bio->bios[disk] =IO_BLOCKED + || !(rdev =rcu_dereference (conf->mirrors[disk].rdev)) + || !test_bit (In_sync, &rdev->flags) + ||test_bit (WriteMostly, &rdev->flags))continue ;
Here the complex expression is repeated - at least almost identical.
Quoted 6 lines
drivers/md/raid1.c +0
-0if (!atomic_read (&rdev->nr_pending)) {@@ -516,11 +499,9 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) current_distance = new_distance; new_disk = disk;
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 507
Quoted 10 lines
drivers/md/raid1.c +1
-3} - }while (disk != conf->last_used); + } rb_out: - -if (new_disk >=0 ) { rdev =rcu_dereference (conf->mirrors[new_disk].rdev);if (!rdev)
Sam
Quoted 4 lines · On Mon, 6 Sep 2010 21:21:28 +0200 Sam Ravnborg <[email protected]> wrote:
Quoted 1 line
Comments?
Looks better but can still use a few improvements. See below.
Thanks for your review and comments....
Quoted 36 lines
Sam
Quoted 34 lines
Thanks,
NeilBrowncommit
e4062735c8f7233923df5858ed20f1278f3ee669Author: NeilBrown <[email protected]> Date: Mon Sep 6 14:10:08 2010 +1000md: tidy up device searches in
read_balance.We have a pre-increment side-effect in the arg to a macro:
rcu_dereferenceThis is poor form and triggers a warning. Rather than just fix that, take the opportunity to re-write the code it make it more readable.
Reported-by Kulikov Vasiliy <[email protected]>Signed-off-by NeilBrown <[email protected]>drivers/md/raid1.c +3
-2@@ -420,11 +420,13 @@ static void raid1_end_write_request(struct bio *bio, int error)static int read_balance (conf_t *conf,r1bio_t *r1_bio) {const sector_t this_sector = r1_bio->sector; -int new_disk = conf->last_used, disk = new_disk; -int wonly_disk =-1 ; +int new_disk =-1 ; +int start_disk; +int i;const int sectors = r1_bio->sectors;sector_t new_distance, current_distance;
Quoted 9 lines
drivers/md/raid1.c static void raid1_end_write_request(struct bio *bio, int error) · line 429
Quoted 2 lines
drivers/md/raid1.c +1
-0mdk_rdev_t *rdev; +int choose_first;To increase readability the general recommendation is:
1) Sort variable definitions with the longest first.
2) Do not assing variables when they are defined, do that on a separate line below the variable definitions.
With one empty line after variable definitions.
I'm don't really agree with this. I think declaring related variables together is much more important that sorting them by length. I guess it is a very subjective thing.
And I think initialising at the point of declaration is often a good idea, though not always.
I've moved 'sectors' up near 'this_sector' but nothing else.
Quoted 48 lines
Quoted 48 lines
drivers/md/raid1.c +5
-33rcu_read_lock ();/* @@ -435,54 +437,35 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) retry:if (conf->mddev->recovery_cp < MaxSector && (this_sector + sectors >= conf->next_resync)) { -/* Choose the first operational device, for consistancy */ - new_disk =0 ; - -for (rdev =rcu_dereference (conf->mirrors[new_disk].rdev); - r1_bio->bios[new_disk] =IO_BLOCKED || - !rdev || !test_bit (In_sync, &rdev->flags) - ||test_bit (WriteMostly, &rdev->flags); - rdev =rcu_dereference (conf->mirrors[++new_disk].rdev)) { - -if (rdev &&test_bit (In_sync, &rdev->flags) && - r1_bio->bios[new_disk] !=IO_BLOCKED ) - wonly_disk = new_disk; - -if (new_disk = conf->raid_disks -1 ) { - new_disk = wonly_disk; -break ; - } - } - goto rb_out; + choose_first =1 ; + start_disk =0 ; + }else { + choose_first =0 ; + start_disk = conf->last_used; } -/* make sure the disk is operational */ -for (rdev =rcu_dereference (conf->mirrors[new_disk].rdev); - r1_bio->bios[new_disk] =IO_BLOCKED || - !rdev || !test_bit (In_sync, &rdev->flags) || -test_bit (WriteMostly, &rdev->flags); - rdev =rcu_dereference (conf->mirrors[new_disk].rdev)) { - -if (rdev &&test_bit (In_sync, &rdev->flags) && - r1_bio->bios[new_disk] !=IO_BLOCKED ) - wonly_disk = new_disk; - -if (new_disk <=0 ) - new_disk = conf->raid_disks; - new_disk--;
Quoted 8 lines
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 451
Quoted 7 lines
drivers/md/raid1.c +4
-3-if (new_disk = disk) { - new_disk = wonly_disk; -break ; +for (i =0 ; i < conf->raid_disks ; i++) { +int disk = start_disk + i; +if (disk >= conf->raid_disks) + disk -= conf->raid_disks;1) Please comment on the purpose of the for loop
That would be the comment "make sure the disk is operational" ??
Quoted 1 line
2) See comments above aboyt variable definitions
Still disagree - sorry.
Quoted 8 lines
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 455
Quoted 5 lines
drivers/md/raid1.c +4
-0+ +if (r1_bio->bios[disk] =IO_BLOCKED + || !(rdev =rcu_dereference (conf->mirrors[disk].rdev)) + || !test_bit (In_sync, &rdev->flags))The rather complex expression - which includes a well hidden assignment - is repeated a few lines later.
Please use a helper function and do not use such hidden assignments.
I've moved the assignment out but I don't agree that a helper function is needed.
Once must balance the total complexity of the function (which should be kept low) against the cost of having to go look at a separate piece of code to see what a helper function actually does.
In this case I think that separating this code out would be 'hiding' rather than 'abstraction'.
Quoted 20 lines
Quoted 20 lines
drivers/md/raid1.c +9
-1+continue ; + +if (test_bit (WriteMostly, &rdev->flags)) { + new_disk = disk; +continue ; } + new_disk = disk; +break ; }@@ -491,20 +474,20 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio)if (this_sector = conf->mirrors[new_disk].head_position) goto rb_out; - current_distance =abs (this_sector - conf->mirrors[disk].head_position); + current_distance =abs (this_sector + - conf->mirrors[new_disk].head_position);
Quoted 11 lines
drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 485
Quoted 7 lines
drivers/md/raid1.c +6
-1-/* Find the disk whose head is closest */ +/* look for a better disk - i.e. head is closer */ + start_disk = new_disk; +for (i =1 ; i < conf->raid_disks; i++) { +int disk = start_disk +1 ; +if (disk >= conf->raid_disks) + disk -= conf->raid_disks;See comments about for loop above.
I also cannot see why we suddenly start with 1 where the other almost identical for loop starts with 0?
If I wonder then someone else will wonder too => comment please.
Before we were finding a working disk.
Now were a finding a better disk.
First is an absolute statement that needs to consider every device, second is comparative and only needs to consider every other device (... uhm, that doesn't sound right - I don't mean every second device, I mean every device that isn't this one).
Suggestions on a comment that would make that clearer?
Quoted 18 lines
Quoted 7 lines
drivers/md/raid1.c +0
-5- do { - if (disk <= 0) - disk = conf->raid_disks; - disk--; -drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 491
Quoted 10 lines
drivers/md/raid1.c +4
-5- rdev =rcu_dereference (conf->mirrors[disk].rdev); - -if (!rdev || r1_bio->bios[disk] =IO_BLOCKED || - !test_bit (In_sync, &rdev->flags) || -test_bit (WriteMostly, &rdev->flags)) +if (r1_bio->bios[disk] =IO_BLOCKED + || !(rdev =rcu_dereference (conf->mirrors[disk].rdev)) + || !test_bit (In_sync, &rdev->flags) + ||test_bit (WriteMostly, &rdev->flags))continue ;Here the complex expression is repeated - at least almost identical.
The 'almost' is key.
There are two if conditions that are similar but different. And only two. Factoring parts out would still leave one of them a little complex and would split the task of understanding the condition over two separate parts of program text.
I don't think the benefits of a function out-weigh the costs.
Thanks,
NeilBrown
Quoted 19 lines
Quoted 7 lines
drivers/md/raid1.c +0
-0if (!atomic_read (&rdev->nr_pending)) {@@ -516,11 +499,9 @@ static int read_balance(conf_t *conf, r1bio_t *r1_bio) current_distance = new_distance; new_disk = disk;drivers/md/raid1.c static int read_balance(conf_t *conf, r1bio_t *r1_bio) · line 507
Quoted 10 lines
drivers/md/raid1.c +1
-3} - }while (disk != conf->last_used); + } rb_out: - -if (new_disk >=0 ) { rdev =rcu_dereference (conf->mirrors[new_disk].rdev);if (!rdev)Sam
Quoted 3 lines · Neil Brown
I've taken the opportunity to substantially re-write that code.
It's better to have two patches, one a backportable one liner that fixes the bug, the other, on top, that cleans up the code but has no sematic changes.
This makes it substantially easier to review. When considering the first patch you see the change plainly. When reviewing the second patch you make sure no semantic changes were made at all.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 5 lines · On Thu, 16 Sep 2010 14:54:29 +0200 Avi Kivity <[email protected]> wrote:
Quoted 3 lines · Neil Brown
I've taken the opportunity to substantially re-write that code.
Quoted 8 lines
It's better to have two patches, one a backportable one liner that fixes the bug, the other, on top, that cleans up the code but has no sematic changes.
This makes it substantially easier to review. When considering the first patch you see the change plainly. When reviewing the second patch you make sure no semantic changes were made at all.
Good advice, I agree.
However the conversation seem have drifted towards viewing the new macro definition as the bug, and the pre-increment in an argument as a valid thing to do.
In that case, there is no bug to fix, just a code clean up required. So I'm currently planning on just submitting that cleanup in the next merge window, and leaving the rcu guys to 'fix' the macro.
Thanks,
NeilBrown
Quoted 8 lines · On Sunday 05 September 2010 20:32:18 Kulikov Vasiliy wrote:
From: Vasiliy Kulikov <[email protected]>
rcu_dereference()is macro, so it might use its argument twice. Argument must not has side effects.It was found by compiler warning:
drivers/md/raid1.c: In function ‘read_balance’:drivers/md/raid1.c:445: warning: operation on ‘new_disk’ may be undefined
I think the rcu_dereference macro should really not evaluate its argument twice, and I don't see where it does.
As a general rule, we try to write macros in Linux such that they behave like functions and don't have surprising side-effects.
Which kernel and gcc version do you see the warning with?
Arnd
Quoted 10 lines · Arnd Bergmann
Quoted 8 lines · On Sunday 05 September 2010 20:32:18 Kulikov Vasiliy wrote:
From: Vasiliy Kulikov <[email protected]>
rcu_dereference()is macro, so it might use its argument twice. Argument must not has side effects.It was found by compiler warning:
drivers/md/raid1.c: In function ‘read_balance’:drivers/md/raid1.c:445: warning: operation on ‘new_disk’ may be undefined
Quoted 8 lines
I think the
rcu_dereferencemacro should really not evaluate its argument twice, and I don't see where it does.
As a general rule, we try to write macros in Linux such that they behave like functions and don't have surprising side-effects.Which kernel and gcc version do you see the warning with?
Arnd
gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5), linux-next.
#define __rcu_dereference_check(p, c, space) \
({ \
typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \
^
rcu_lockdep_assert(c); \
(void) (((typeof (*p) space *)p) = p); \
^ ^
smp_read_barrier_depends(); \
((typeof(*p) __force __kernel *)(_________p1)); \
})If I understand this, it is evaluated three times, right?
Signature · 1 line
Vasiliy
Quoted 11 lines · On Tuesday 07 September 2010 21:21:55 Kulikov Vasiliy wrote:
#define __rcu_dereference_check(p, c, space) \
({ \
typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \
^
rcu_lockdep_assert(c); \
(void) (((typeof (*p) space *)p) = p); \
^ ^
smp_read_barrier_depends(); \
((typeof(*p) __force __kernel *)(_________p1)); \
})Quoted 1 line
If I understand this, it is evaluated three times, right?
Yes, that looks like my own fault, I added that :(
This patch seems to fix it, but I need to think about it some more to make sure it still does everything we need.
include/linux/rcupdate.h +3 -3
@@ -313,21 +313,21 @@ extern int rcu_my_thread_group_empty(void);#define __rcu_access_pointer (p, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ - (void) (((typeof (*p) space *)p) = p); \ + (void) (((typeof (*p) space *)NULL) = (typeof(p))NULL); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })#define __rcu_dereference_check (p, c, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ rcu_lockdep_assert(c); \ - (void) (((typeof (*p) space *)p) = p); \ + (void) (((typeof (*p) space *)NULL) = (typeof(p))NULL); \ smp_read_barrier_depends(); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })#define __rcu_dereference_protected (p, c, space) \ ({ \ rcu_lockdep_assert(c); \ - (void) (((typeof (*p) space *)p) = p); \ + (void) (((typeof (*p) space *)NULL) = (typeof(p))NULL); \ ((typeof(*p) __force __kernel *)(p)); \ })
Quoted 12 lines · Arnd Bergmann
Quoted 11 lines · On Tuesday 07 September 2010 21:21:55 Kulikov Vasiliy wrote:
#define __rcu_dereference_check(p, c, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ ^ rcu_lockdep_assert(c); \ (void) (((typeof (*p) space *)p) = p); \ ^ ^ smp_read_barrier_depends(); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })
Quoted 6 lines
Quoted 1 line
If I understand this, it is evaluated three times, right?
Yes, that looks like my own fault, I added that :(
This patch seems to fix it, but I need to think about it some more to make sure it still does everything we need.
Let me know when you are satisfied with it, and then I will pick it up.
Thanx, Paul
Quoted 31 lines
Signed-off-by Arnd Bergmann <[email protected]>include/linux/rcupdate.h +3
-3@@ -313,21 +313,21 @@ extern int rcu_my_thread_group_empty(void);#define __rcu_access_pointer (p, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ - (void) (((typeof (*p) space *)p) = p); \ + (void) (((typeof (*p) space *)NULL) = (typeof(p))NULL); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })#define __rcu_dereference_check (p, c, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ rcu_lockdep_assert(c); \ - (void) (((typeof (*p) space *)p) = p); \ + (void) (((typeof (*p) space *)NULL) = (typeof(p))NULL); \ smp_read_barrier_depends(); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })#define __rcu_dereference_protected (p, c, space) \ ({ \ rcu_lockdep_assert(c); \ - (void) (((typeof (*p) space *)p) = p); \ + (void) (((typeof (*p) space *)NULL) = (typeof(p))NULL); \ ((typeof(*p) __force __kernel *)(p)); \ })
Quoted 13 lines · Paul E. McKenney
Quoted 12 lines · Arnd Bergmann
Quoted 11 lines · On Tuesday 07 September 2010 21:21:55 Kulikov Vasiliy wrote:
#define __rcu_dereference_check(p, c, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ ^ rcu_lockdep_assert(c); \ (void) (((typeof (*p) space *)p) = p); \ ^ ^ smp_read_barrier_depends(); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
If I understand this, it is evaluated three times, right?
Yes, that looks like my own fault, I added that :(
This patch seems to fix it, but I need to think about it some more to make sure it still does everything we need.
Let me know when you are satisfied with it, and then I will pick it up.
I guess it would be good to put it in now. I haven't had the time to try out all cases, but the current code in -next is definitely broken, so please put the fix in now.
Arnd
Quoted 18 lines · Arnd Bergmann
Quoted 17 lines · Paul E. McKenney
Quoted 16 lines · Arnd Bergmann
Quoted 11 lines · On Tuesday 07 September 2010 21:21:55 Kulikov Vasiliy wrote:
#define __rcu_dereference_check(p, c, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ ^ rcu_lockdep_assert(c); \ (void) (((typeof (*p) space *)p) = p); \ ^ ^ smp_read_barrier_depends(); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })Quoted 1 line
If I understand this, it is evaluated three times, right?
Yes, that looks like my own fault, I added that :(
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
This patch seems to fix it, but I need to think about it some more to make sure it still does everything we need.
Let me know when you are satisfied with it, and then I will pick it up.
I guess it would be good to put it in now. I haven't had the time to try out all cases, but the current code in -next is definitely broken, so please put the fix in now.
Hmmm... One approach would be have a secondary macro that was:
#define __rcu_dereference_check_sparse(p, space) \ (void) (((typeof (*p) space *)p) = p);
when running sparse and:
#define __rcu_dereference_check_sparse(p, space)
otherwise.
Would that do the trick?
Thanx, Paul
Quoted 33 lines · Paul E. McKenney
Quoted 18 lines · Arnd Bergmann
Quoted 17 lines · Paul E. McKenney
Quoted 16 lines · Arnd Bergmann
Quoted 11 lines · On Tuesday 07 September 2010 21:21:55 Kulikov Vasiliy wrote:
#define __rcu_dereference_check(p, c, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ ^ rcu_lockdep_assert(c); \ (void) (((typeof (*p) space *)p) = p); \ ^ ^ smp_read_barrier_depends(); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })Quoted 1 line
If I understand this, it is evaluated three times, right?
Yes, that looks like my own fault, I added that :(
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
This patch seems to fix it, but I need to think about it some more to make sure it still does everything we need.
Let me know when you are satisfied with it, and then I will pick it up.
I guess it would be good to put it in now. I haven't had the time to try out all cases, but the current code in -next is definitely broken, so please put the fix in now.
Hmmm... One approach would be have a secondary macro that was:
#define __rcu_dereference_check_sparse(p, space) \ (void) (((typeof (*p) space *)p) = p);
Quoted 7 lines
when running sparse and:
#define __rcu_dereference_check_sparse(p, space)otherwise.
Would that do the trick?
Here is a patch that more fully describes what I had in mind. I have done light compile/sparse testing, but this should be considered to be full of bugs.
Thoughts?
Thanx, Paul
------------------------------------------------------------------------
commit 493a0a1001137f7058da0e01c3d05b0fcb92841d
Author: Paul E. McKenney <[email protected]> Date: Mon Sep 13 17:24:21 2010 -0700
rcu: only one evaluation of arg in rcu_dereference_check() unless sparse
The current version of the __rcu_access_pointer(), __rcu_dereference_check(), and __rcu_dereference_protected() macros evaluate their "p" argument three times, not counting typeof()s. This is bad news if that argument contains a side effect. This commit therefore evaluates this argument only once in normal kernel builds. However, the straightforward approach defeats sparse's RCU-pointer checking, so this commit also adds a KBUILD_CHECKSRC symbol defined when running a checker. Therefore, when this new KBUILD_CHECKSRC symbol is defined, the additional pair of evaluations of the "p" argument are performed in order to permit sparse to detect misuse of RCU-protected pointers.
This commit also fixes some sparse complaints in rcutorture.c.
Makefile +1 -1
@@ -330,7 +330,7 @@ PERL = perl CHECK = sparse CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \ - -Wbitwise -Wno-return-void $(CF) + -Wbitwise -Wno-return-void -DKBUILD_CHECKSRC $(CF) CFLAGS_MODULE AFLAGS_MODULE LDFLAGS_MODULE diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 682bf4c..9fda1e6 100644
include/linux/rcupdate.h +11 -3
@@ -309,24 +309,32 @@ extern int rcu_my_thread_group_empty(void);* (e.g., __rcu_bh, * __rcu_sched, and __srcu), should this make sense in * the future. */ + +#ifdef KBUILD_CHECKSRC +#define rcu_dereference_sparse (p, space) \ + ((void)(((typeof(*p) space *)p) = p)) +#else /* #ifdef KBUILD_CHECKSRC */ +#define rcu_dereference_sparse (p, space) +#endif /* #else #ifdef KBUILD_CHECKSRC */ +#define __rcu_access_pointer (p, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ - (void) (((typeof (*p) space *)p) = p); \ + rcu_dereference_sparse(p, space); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })#define __rcu_dereference_check (p, c, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ rcu_lockdep_assert(c); \ - (void) (((typeof (*p) space *)p) = p); \ + rcu_dereference_sparse(p, space); \ smp_read_barrier_depends(); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })#define __rcu_dereference_protected (p, c, space) \ ({ \ rcu_lockdep_assert(c); \ - (void) (((typeof (*p) space *)p) = p); \ + rcu_dereference_sparse(p, space); \ ((typeof(*p) __force __kernel *)(p)); \ })
kernel/rcutorture.c +6 -4
@@ -131,7 +131,7 @@ struct rcu_torture { };static LIST_HEAD (rcu_torture_freelist); -static struct rcu_torture *rcu_torture_current; +static struct rcu_torture __rcu *rcu_torture_current;static long rcu_torture_current_version;static struct rcu_torture rcu_tortures[10 *RCU_TORTURE_PIPE_LEN ];static DEFINE_SPINLOCK (rcu_torture_lock);
@@ -171,7 +171,7 @@ int rcutorture_runnable = RCUTORTURE_RUNNABLE_INIT;#endif /* #else #ifdef CONFIG_BOOST_RCU */ static unsigned long boost_starttime;/* jiffies of next boost test start. */ -DEFINE_MUTEX (boost_mutex);/* protect setting boost_starttime */ +static DEFINE_MUTEX (boost_mutex);/* protect setting boost_starttime */ /* and boost task create/destroy. */ /* Mediate rmmod and system shutdown. Concurrent rmmod & shutdown illegal! */
@@ -180,7 +180,8 @@ DEFINE_MUTEX(boost_mutex); /* protect setting boost_starttime */#define FULLSTOP_SHUTDOWN 1/* System shutdown with rcutorture running. */ #define FULLSTOP_RMMOD 2/* Normal rmmod of rcutorture. */ static int fullstop =FULLSTOP_RMMOD ; -DEFINE_MUTEX (fullstop_mutex);/* Protect fullstop transitions and spawning */ +static DEFINE_MUTEX (fullstop_mutex); +/* Protect fullstop transitions and spawning */ /* of kthreads. */ /*
@@ -873,7 +874,8 @@ rcu_torture_writer(void *arg)continue ; rp->rtort_pipe_count =0 ;udelay (rcu_random (&rand) &0x3ff ); - old_rp = rcu_torture_current; + old_rp =rcu_dereference_check (rcu_torture_current, + current = writer_task); rp->rtort_mbtest =1 ;rcu_assign_pointer (rcu_torture_current, rp);smp_wmb ();/* Mods to old_rp must follow rcu_assign_pointer() */
Quoted 10 lines · Paul E. McKenney
The current version of the
__rcu_access_pointer(),__rcu_dereference_check(), and__rcu_dereference_protected()macros evaluate their "p" argument three times, not counting typeof()s. This is bad news if that argument contains a side effect. This commit therefore evaluates this argument only once in normal kernel builds. However, the straightforward approach defeats sparse's RCU-pointer checking, so this commit also adds aKBUILD_CHECKSRCsymbol defined when running a checker. Therefore, when this newKBUILD_CHECKSRCsymbol is defined, the additional pair of evaluations of the "p" argument are performed in order to permit sparse to detect misuse of RCU-protected pointers.
In general, I don't like the idea much because that means we're passing semantically different code into sparse and gcc. Of course if my other patch doesn't work, we might need to do it after all.
Makefile PERL = perl · line 334
Quoted 11 lines
Makefile +1
-1@@ -330,7 +330,7 @@ PERL = perl CHECK = sparse CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \ - -Wbitwise -Wno-return-void $(CF) + -Wbitwise -Wno-return-void -DKBUILD_CHECKSRC $(CF) CFLAGS_MODULE > AFLAGS_MODULE > LDFLAGS_MODULE
sparse already define __CHECKER__ itself, no need to define another symbol.
Makefile PERL = perl
Quoted 6 lines
Makefile +6
-0+#ifdef KBUILD_CHECKSRC +#define rcu_dereference_sparse(p, space) \ + ((void)(((typeof(*p) space *)p) = p)) +#else /* #ifdef KBUILD_CHECKSRC */ +#define rcu_dereference_sparse(p, space) +#endif /* #else #ifdef KBUILD_CHECKSRC */
Did you see a problem with my macro?
#define rcu_dereference_sparse(p, space) \
((void)(((typeof(*p) space *)NULL) = ((typeof(p))NULL)))I think this should warn in all the cases we want it to, but have no side-effects.
Quoted 14 lines
#define __rcu_access_pointer(p, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \Makefile +2
-2- (void) (((typeof (*p) space *)p) = p); \ + rcu_dereference_sparse(p, space); \ ((typeof(*p) __force __kernel *)(_________p1)); \ }) #define __rcu_dereference_check(p, c, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ rcu_lockdep_assert(c); \ - (void) (((typeof (*p) space *)p) = p); \ + rcu_dereference_sparse(p, space); \
Makefile
Quoted 10 lines
Makefile +1
-1smp_read_barrier_depends(); \ ((typeof(*p) __force __kernel *)(_________p1)); \ }) #define __rcu_dereference_protected(p, c, space) \ ({ \ rcu_lockdep_assert(c); \ - (void) (((typeof (*p) space *)p) = p); \ + rcu_dereference_sparse(p, space); \ ((typeof(*p) __force __kernel *)(p)); \ })
This part might be useful in any case, to better document what the cast and compare does, and to prevent the three users from diverging.
kernel/rcutorture.c
Quoted 4 lines
kernel/rcutorture.c +0
-0
This didn't seem to belong here.
Arnd
Quoted 12 lines · Arnd Bergmann
Quoted 10 lines · Paul E. McKenney
The current version of the
__rcu_access_pointer(),__rcu_dereference_check(), and__rcu_dereference_protected()macros evaluate their "p" argument three times, not counting typeof()s. This is bad news if that argument contains a side effect. This commit therefore evaluates this argument only once in normal kernel builds. However, the straightforward approach defeats sparse's RCU-pointer checking, so this commit also adds aKBUILD_CHECKSRCsymbol defined when running a checker. Therefore, when this newKBUILD_CHECKSRCsymbol is defined, the additional pair of evaluations of the "p" argument are performed in order to permit sparse to detect misuse of RCU-protected pointers.
Quoted 3 lines
In general, I don't like the idea much because that means we're passing semantically different code into sparse and gcc. Of course if my other patch doesn't work, we might need to do it after all.
Agreed in principle, but please see below.
Quoted 7 lines
Quoted 7 lines
Makefile +0
-0@@ -330,7 +330,7 @@ PERL = perl CHECK = sparse
Quoted 5 lines
Makefile PERL = perl
Quoted 4 lines
Makefile +1
-1CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \ - -Wbitwise -Wno-return-void $(CF) + -Wbitwise -Wno-return-void -DKBUILD_CHECKSRC $(CF) CFLAGS_MODULE > > AFLAGS_MODULE > > LDFLAGS_MODULE >sparse already define
__CHECKER__itself, no need to define another symbol.
Good point, will fix if we are in fact sticking with this solution.
Quoted 11 lines
Makefile PERL = perl
Quoted 6 lines
Makefile +6
-0+#ifdef KBUILD_CHECKSRC +#define rcu_dereference_sparse(p, space) \ + ((void)(((typeof(*p) space *)p) = p)) +#else /* #ifdef KBUILD_CHECKSRC */ +#define rcu_dereference_sparse(p, space) +#endif /* #else #ifdef KBUILD_CHECKSRC */Did you see a problem with my macro?
#define rcu_dereference_sparse(p, space) \ ((void)(((typeof(*p) space *)NULL) = ((typeof(p))NULL)))
I don't see a specific problem with it. However, I am not sure that it really does what we want, and you indicated some doubts when you posted it. So I opted for something that very obviously will work. If you can assure me that sparse will interpret the typeof()s and space casts properly, I have no problem going with your version.
Quoted 1 line
I think this should warn in all the cases we want it to, but have no side-effects.
I still note a tone of uncertainty in the above sentence. ;-)
Quoted 25 lines
Quoted 14 lines
#define __rcu_access_pointer(p, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \Makefile +2
-2- (void) (((typeof (*p) space *)p) = p); \ + rcu_dereference_sparse(p, space); \ ((typeof(*p) __force __kernel *)(_________p1)); \ }) #define __rcu_dereference_check(p, c, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ rcu_lockdep_assert(c); \ - (void) (((typeof (*p) space *)p) = p); \ + rcu_dereference_sparse(p, space); \Makefile
Quoted 10 lines
Makefile +1
-1smp_read_barrier_depends(); \ ((typeof(*p) __force __kernel *)(_________p1)); \ }) #define __rcu_dereference_protected(p, c, space) \ ({ \ rcu_lockdep_assert(c); \ - (void) (((typeof (*p) space *)p) = p); \ + rcu_dereference_sparse(p, space); \ ((typeof(*p) __force __kernel *)(p)); \ })
Quoted 2 lines
This part might be useful in any case, to better document what the cast and compare does, and to prevent the three users from diverging.
And it would probably make sense to pull the rcu_dereference_sparse() into the macro, for that matter.
Quoted 6 lines
kernel/rcutorture.c
Quoted 4 lines
kernel/rcutorture.c +0
-0This didn't seem to belong here.
Yep, I really should put this in a separate commit.
Thanx, Paul