Skip to contentSkip to conversation

use unlikely() for current_kernel_time() loop

PATCH In review Andreas Dilger 4 participants 14 messages Jun 7, 2006 – Jun 8, 2006

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 use unlikely() for current_kernel_time() loop

Hello,
I just noticed this minor optimization. current_kernel_time() is called from current_fs_time() so it is used fairly often but it doesn't use unlikely(read_seqretry(&xtime_lock, seq)) as other users of xtime_lock do. Also removes extra whitespace on the empty line above.

kernel/time.c renamed from kernel/time.c.orig +2 -2
@@ -424,9 +424,9 @@ inline struct timespec current_kernel_time
 
 	do {
 		seq = read_seqbegin(&xtime_lock);
-		
+
 		now = xtime;
-	} while (read_seqretry(&xtime_lock, seq));
+	} while (unlikely(read_seqretry(&xtime_lock, seq)));
 
 	return now; 
 }

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

#1 lore raw
Quoted 5 lines · Andreas Dilger

Hello,
I just noticed this minor optimization. current_kernel_time() is called from current_fs_time() so it is used fairly often but it doesn't use unlikely(read_seqretry(&xtime_lock, seq)) as other users of xtime_lock do. Also removes extra whitespace on the empty line above.

It would be better to put the unlikely into the read_seqretry I guess.

-Andi

#2 lore raw
Quoted 10 lines · On 08 Jun 2006 04:28:12 +0200 Andi Kleen <[email protected]> wrote:
Quoted 5 lines · Andreas Dilger

Hello,
I just noticed this minor optimization. current_kernel_time() is called from current_fs_time() so it is used fairly often but it doesn't use unlikely(read_seqretry(&xtime_lock, seq)) as other users of xtime_lock do. Also removes extra whitespace on the empty line above.

It would be better to put the unlikely into the read_seqretry I guess.

yup. But it'd be good to check that this actually causes the compiler to do the right thing, rather than simply ignoring it.

I'm not sure how one would do that though. I guess compare before-and-after assembly code, work out if "after" is better.

#3 lore raw
Quoted 11 lines · Andrew Morton
Quoted 8 lines · On 08 Jun 2006 04:28:12 +0200 Andi Kleen <[email protected]> wrote:
Quoted 5 lines · Andreas Dilger

Hello,
I just noticed this minor optimization. current_kernel_time() is called from current_fs_time() so it is used fairly often but it doesn't use unlikely(read_seqretry(&xtime_lock, seq)) as other users of xtime_lock do. Also removes extra whitespace on the empty line above.

Quoted 5 lines
Quoted 2 lines

It would be better to put the unlikely into the read_seqretry I guess.

yup. But it'd be good to check that this actually causes the compiler to do the right thing, rather than simply ignoring it.

If it was put into a macro wrapper it should be safe enough.

Quoted 3 lines

I'm not sure how one would do that though. I guess compare before-and-after assembly code, work out if "after" is better.

Nothing on x86-64 at least - it uses -fno-reorder-blocks by default.

-Andi

#4 lore raw

Andi,

Quoted 3 lines · Andi Kleen

Nothing on x86-64 at least - it uses -fno-reorder-blocks by default.

Why is that?

--brian

#5 lore raw
Quoted 8 lines · Brian F. G. Bidulock

Andi,

Quoted 3 lines · Andi Kleen

Nothing on x86-64 at least - it uses -fno-reorder-blocks by default.

Why is that?

Originally because it made assembly too unreadable. Later it was discovered it produces smaller code too.

-Andi

#6 lore raw

Andi,

Quoted 4 lines · Andi Kleen

Originally because it made assembly too unreadable. Later it was discovered it produces smaller code too.

Thank you for the explanation. But, this brings to mind two other questions:

Does the option not also make assembly less readable on other architectures?

If one is interested in smaller code, why not use -Os?

Also, does -fno-reorder-blocks actually defeat __builtin_expect()? (GCC documentation doesn't really say that.)

#7 lore raw
Quoted 7 lines · Brian F. G. Bidulock
Quoted 3 lines

Originally because it made assembly too unreadable. Later it was discovered it produces smaller code too.

Thank you for the explanation. But, this brings to mind two other questions:

Does the option not also make assembly less readable on other architectures?

Yes it does. Maybe the people who spend a lot of time debugging these don't feel the pain anymore.

Quoted 2 lines

If one is interested in smaller code, why not use -Os?

We already use -Os if you set the right config (but this change was actually long before that)

It's possible that -Os includes -fno-block-reordering though.

Quoted 2 lines

Also, does -fno-reorder-blocks actually defeat __builtin_expect()? (GCC documentation doesn't really say that.)

AFAIK gcc mostly uses the probability information for block reordering to make the fast path fall through without jumps.

There are some more uses, but they don't impact the code very much.

-Andi

#8 lore raw

Andi,

Quoted 3 lines · Andi Kleen

AFAIK gcc mostly uses the probability information for block reordering to make the fast path fall through without jumps.

After a quick try on RH EL4 gcc 3.4.4-2 it appears that -fno-reorder-blocks indeed defeats __builtin_expect() as you say. (Which is rather bizarre as __builtin_expect() no longer does what one expects.) I think that I'm going to strip it out for my externally compiled modules. Otherwise, the source code rearrangements necessary to get the same effect will make the source code unreadable and generate larger code, which I think is worse than those effects on the assembler code.

Thanks again.

--brian

#9 lore raw
In reply to Andi Kleen

Andi,

Quoted 4 lines · Andi Kleen
Quoted 4 lines
Quoted 4 lines

Originally because it made assembly too unreadable. Later it was discovered it produces smaller code too.

Another quick check on Intel 630. w/o -fno-reorder-blocks: code increased 2% in size; performance increased 2% per hyperthread; code still as readable with objdump -S -d. Rather marginal one way or the other.

#10 lore raw
Quoted 2 lines

performance increased 2% per hyperthread;

That would surprise me. Most likely you made some measurement error.

I guess it can be a difference if it's used in very hot loops, but then i would expect a fallthrough path to win. Kernel code very rarely has hot loops like this.

-Andi

#11 lore raw

Andi,

Quoted 5 lines · Andi Kleen
Quoted 2 lines

performance increased 2% per hyperthread;

That would surprise me. Most likely you made some measurement error.

Don't think there was an error. Tests performed on STREAMS external modules optimized with profiling data.

#12 lore raw
Quoted 11 lines · Brian F. G. Bidulock

Andi,

Quoted 5 lines · Andi Kleen
Quoted 2 lines

performance increased 2% per hyperthread;

That would surprise me. Most likely you made some measurement error.

Don't think there was an error. Tests performed on STREAMS external modules optimized with profiling data.

You mean with compiler profile feedback?

If that is slower then it would sound like a compiler bug.

-Andi

#13 lore raw

Andi,

Quoted 5 lines · Andi Kleen
Quoted 3 lines

Don't think there was an error. Tests performed on STREAMS external modules optimized with profiling data.

You mean with compiler profile feedback?

No, hand optimized from oprofile runs.

Quoted 1 line

If that is slower then it would sound like a compiler bug.

What slower? It was 2% faster. And that was only the external
kernel modules compiled without the -fno-redorder-blocks flag.

--brian