Skip to contentSkip to conversation

atomic_fetch_andnot() in nohz_idle_balance()

Question Andrea Parri 3 participants 6 messages Nov 21, 2018 – Nov 27, 2018

lore
Share & tools
lore
#0 lore raw

Hi,

The comment for the atomic_fetch_andnot() in nohz_idle_balance() says:

"barrier, pairs with nohz_balance_enter_idle(), ensures ..."

which, well, does sound a note of warning... ;-)

I see that nohz_balance_enter_idle() has an smp_mb__after_atomic() but the comment for the latter suggests that this barrier is pairing with the smp_mb() in _nohz_idle_balance().

So, what is the intended pairing barrier for the atomic_fetch_andnot()? what (which memory accesses) do you want "to order" here?

Thanks,
Andrea

#1 lore raw
Quoted 6 lines · Andrea Parri

Hi,

The comment for the atomic_fetch_andnot() in nohz_idle_balance() says:

"barrier, pairs with nohz_balance_enter_idle(), ensures ..."

Quoted 8 lines

which, well, does sound a note of warning... ;-)

I see that nohz_balance_enter_idle() has an smp_mb__after_atomic() but the comment for the latter suggests that this barrier is pairing with the smp_mb() in _nohz_idle_balance().

So, what is the intended pairing barrier for the atomic_fetch_andnot()? what (which memory accesses) do you want "to order" here?

I can't seem to make sense of that comment either; the best I can come up with is that it would order the prior NOHZ_KICK_MASK load vs us then changing it.

But that would order against kick_ilb(), not enter_idle.

Vincent?

#2 lore raw
Quoted 17 lines · Peter Zijlstra
Quoted 6 lines · Andrea Parri

Hi,

The comment for the atomic_fetch_andnot() in nohz_idle_balance() says:

"barrier, pairs with nohz_balance_enter_idle(), ensures ..."

Quoted 8 lines

which, well, does sound a note of warning... ;-)

I see that nohz_balance_enter_idle() has an smp_mb__after_atomic() but the comment for the latter suggests that this barrier is pairing with the smp_mb() in _nohz_idle_balance().

So, what is the intended pairing barrier for the atomic_fetch_andnot()? what (which memory accesses) do you want "to order" here?

Quoted 7 lines

I can't seem to make sense of that comment either; the best I can come up with is that it would order the prior NOHZ_KICK_MASK load vs us then changing it.

But that would order against kick_ilb(), not enter_idle.

Vincent?

I can't come with a good explanation.
After looking into my email archive, the only explanation that i have is that the comments remains from a previous iteration of the feature that was based on a nohz.stats_state mechanism

#3 lore raw
Quoted 22 lines · Vincent Guittot
Quoted 21 lines · Peter Zijlstra
Quoted 6 lines · Andrea Parri

Hi,

The comment for the atomic_fetch_andnot() in nohz_idle_balance() says:

"barrier, pairs with nohz_balance_enter_idle(), ensures ..."

Quoted 8 lines

which, well, does sound a note of warning... ;-)

I see that nohz_balance_enter_idle() has an smp_mb__after_atomic() but the comment for the latter suggests that this barrier is pairing with the smp_mb() in _nohz_idle_balance().

So, what is the intended pairing barrier for the atomic_fetch_andnot()? what (which memory accesses) do you want "to order" here?

I can't seem to make sense of that comment either; the best I can come up with is that it would order the prior NOHZ_KICK_MASK load vs us then changing it.

Quoted 8 lines
Quoted 3 lines

But that would order against kick_ilb(), not enter_idle.

Vincent?

I can't come with a good explanation.
After looking into my email archive, the only explanation that i have is that the comments remains from a previous iteration of the feature that was based on a nohz.stats_state mechanism

I'm afraid I still can't help your comment... put in other terms, would you feel "unconfortable" with _relax()ing the andnot()? (and if so ...)

Andrea

#4 lore raw
Quoted 28 lines · Andrea Parri
Quoted 26 lines · Vincent Guittot
Quoted 24 lines · Peter Zijlstra
Quoted 6 lines · Andrea Parri

Hi,

The comment for the atomic_fetch_andnot() in nohz_idle_balance() says:

"barrier, pairs with nohz_balance_enter_idle(), ensures ..."

Quoted 8 lines

which, well, does sound a note of warning... ;-)

I see that nohz_balance_enter_idle() has an smp_mb__after_atomic() but the comment for the latter suggests that this barrier is pairing with the smp_mb() in _nohz_idle_balance().

So, what is the intended pairing barrier for the atomic_fetch_andnot()? what (which memory accesses) do you want "to order" here?

I can't seem to make sense of that comment either; the best I can come up with is that it would order the prior NOHZ_KICK_MASK load vs us then changing it.

But that would order against kick_ilb(), not enter_idle.

Vincent?

Quoted 7 lines
Quoted 4 lines

I can't come with a good explanation.
After looking into my email archive, the only explanation that i have is that the comments remains from a previous iteration of the feature that was based on a nohz.stats_state mechanism

I'm afraid I still can't help your comment... put in other terms, would you feel "unconfortable" with _relax()ing the andnot()? (and if so ...)

so I think that the comment is useless and can be removed because it was referring to a line code above the comment that was present in a previous iteration of the patchset. This line disappeared in final version but the comment has stayed.

If your question is: can we use atomic_fetch_andnot_relaxed() instead of atomic_fetch_andnot() in nohz_idle_balance() ?
I think that it's possible

Quoted 2 lines

Andrea

#5 lore raw
Quoted 38 lines · Vincent Guittot
Quoted 28 lines · Andrea Parri
Quoted 26 lines · Vincent Guittot
Quoted 24 lines · Peter Zijlstra
Quoted 6 lines · Andrea Parri

Hi,

The comment for the atomic_fetch_andnot() in nohz_idle_balance() says:

"barrier, pairs with nohz_balance_enter_idle(), ensures ..."

Quoted 8 lines

which, well, does sound a note of warning... ;-)

I see that nohz_balance_enter_idle() has an smp_mb__after_atomic() but the comment for the latter suggests that this barrier is pairing with the smp_mb() in _nohz_idle_balance().

So, what is the intended pairing barrier for the atomic_fetch_andnot()? what (which memory accesses) do you want "to order" here?

I can't seem to make sense of that comment either; the best I can come up with is that it would order the prior NOHZ_KICK_MASK load vs us then changing it.

But that would order against kick_ilb(), not enter_idle.

Vincent?

Quoted 7 lines
Quoted 4 lines

I can't come with a good explanation.
After looking into my email archive, the only explanation that i have is that the comments remains from a previous iteration of the feature that was based on a nohz.stats_state mechanism

I'm afraid I still can't help your comment... put in other terms, would you feel "unconfortable" with _relax()ing the andnot()? (and if so ...)

Quoted 8 lines

so I think that the comment is useless and can be removed because it was referring to a line code above the comment that was present in a previous iteration of the patchset. This line disappeared in final version but the comment has stayed.

If your question is: can we use atomic_fetch_andnot_relaxed() instead of atomic_fetch_andnot() in nohz_idle_balance() ?
I think that it's possible

Ah!, thank you for the clarification. Just sent a clean-up patch for the comment (but deferring for the _relaxed() change...).

Andrea

Quoted 3 lines
Quoted 3 lines

Andrea