Skip to contentSkip to conversation

remove __read_mostly?

DISCUSSION Andrew Morton 5 participants 10 messages Jun 25, 2006 – Jun 26, 2006

lore
Share & tools
lore
#0 lore raw

I'm thinking we should remove __read_mostly.

Because if we use this everywhere where it's supposed to be used, we end up with .bss and .data 100% populated with write-often variables, packed closely together. The cachelines will really flying around.

IOW: __read_mostly optimises read-mostly variables and pessimises write-often variables.

We want something which optimises both read-mostly and write-often storage.
We do that by marking the write-often variables with
__cacheline_aligned_in_smp.

OK?

[Problem is, I don't think any of the make-foo-__read_mostly patches actually identified _which_ write-often variables were affecting `foo', so we'll be back to square one.]

[Actually, we should do

	#define __write_often __cacheline_aligned_in_smp

and use __write_often

a) for documentation and

b) so the optimisation can be centrally turned off, for space optimisation or for performance validation.]

#1 lore raw
Quoted 7 lines · Andrew Morton

I'm thinking we should remove __read_mostly.

Because if we use this everywhere where it's supposed to be used, we end up with .bss and .data 100% populated with write-often variables, packed closely together. The cachelines will really flying around.

Quoted 6 lines

IOW: __read_mostly optimises read-mostly variables and pessimises write-often variables.

We want something which optimises both read-mostly and write-often storage.
We do that by marking the write-often variables with
__cacheline_aligned_in_smp.

We already mark write often structures with __cacheline_aligned_in_smp.

The idea behind __read_mostly is to separate variables like cpu maps, bootcpuinfo etc which are written to very very rarely -- during initialization/hot-plugging, but read quite often something like ~100 % read ratio. They might not be sharing the cacheline with a
variable which is not as frequently written to, to mark __cacheline_aligned_in_smp, but still, these often read variables would be invalidated in cache every time there is a write on these other variables. Considering that there are quite a few structures which are read often, (like cpu maps, cpu info, node to cpu maps etc) it made sense to place them in a separate section.

When we mark variables __read_mostly, it doesn't mean all that is left in .bss and .data is write mostly. Surely the rest of .data and .bss which do not qualify for ~100% read would not be ~100% write/RMW. We would have variables which experience varying ratio of reads and writes.

So as I see it the current scheme of thing is
1. If a variable is written quite often -- mark __cacheline_aligned_in_smp.
2. If a variable is read quite often .. something like 99:1 read -- mark them __read_mostly

Quoted 6 lines

OK?

[Problem is, I don't think any of the make-foo-__read_mostly patches actually identified _which_ write-often variables were affecting `foo', so we'll be back to square one.]

Most of the variables we marked as __read_mostly were ones we were seeing inter-node transfers for read, when we shouldn't be seeing any -- because they are just written to once during bootup. And it did not make sense to mark the other writers on the inter-node cacheline __cacheline_aligned_in_smp, since it would mean a bloat of 128-4096 bytes depending on the arch, and those variables were not written often enough to warrant padding. A few not so-often-written variables on the same cacheline would mean these read mostly variables are always being invalidated.

Quoted 12 lines

[Actually, we should do

	#define __write_often __cacheline_aligned_in_smp

and use __write_often

a) for documentation and

b) so the optimisation can be centrally turned off, for space optimisation or for performance validation.]

Sure, we can change users of __cacheline_aligned_in_smp to __write_often and change the section to .data.write_mostly from .data.cacheline_aligned for readability sake. But, IMHO we do need the __read_mostly section for variables which experience near 100% read access. I cannot see any negative aspects of having a separate read mostly section as there is no padding or bloating involved. Granted, we might not see performance difference with some builds due to the compiler/linker placement / code changes, but this might resurface later with a newer build/code changes.

Thanks,
Kiran

#2 lore raw
Quoted 4 lines · Ravikiran

The idea behind __read_mostly is to separate variables like cpu maps, bootcpuinfo etc which are written to very very rarely -- during initialization/hot-plugging, but read quite often something like ~100 % read ratio.

So these variables are __read_hot_write_cold?

In other words, the name __read_mostly is a little misleading, in my book. That name only suggests read much more than written. In your words:

something like 99:1 read

It doesn't state that the variable is so "read hot", it is worth keeping off "write hot" cache lines.

Let's say for example we have a variable is accessed only once per hour, and that this access is always a read except once a week (once every 168 hours) when it is a write.

I would not mark that variable __read_mostly, even though it passed your 99:1 test. That variable is read_cold_write_evencolder. It's an ideal candidate for the canon fodder that we use to fill up the rest of a cache line that has a hot variable.

If Andrew's suggestion to remove __read_mostly doesn't fly, then I'd vote for a name change:

__read_mostly ==> __read_hot_write_cold

I think we want to identify the hottest memory words, keeping them on separate cache lines, except that __read_hot_write_cold words can share cache lines with each other.

A given cache line would have at most one hot write word, or one or more read hot, write cold words.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
#3 lore raw
Quoted 5 lines · Paul Jackson

In other words, the name __read_mostly is a little misleading, in my book. That name only suggests read much more than written. In your words:

something like 99:1 read

99:1 may be too small a ratio.

A read_mostly marked variable should be changed rarely (meaning is is extremely unlikely that his is going to change) but read frequently.

F.e. configuration data for timer operations, number of possible processors and stuff like that.

If we would make the operation to write to the read_mostly section more expensive (by f.e. replicating the data per node) then this would hold off the uses that are changing too frequently.

#4 lore raw
Quoted 1 line · Christoph

99:1 may be too small a ratio.

Could well be. I was just quoting Ravikiran's number. I suspect he was using the number loosely, not as a precise value.

Quoted 2 lines

A read_mostly marked variable should be changed rarely (meaning is is extremely unlikely that his is going to change) but read frequently.

Yes - well said.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
#5 lore raw
Quoted 5 lines · Paul Jackson
Quoted 1 line · Christoph

99:1 may be too small a ratio.

Could well be. I was just quoting Ravikiran's number. I suspect he was using the number loosely, not as a precise value.

Yes I was using it loosely. I also mentioned ~100% read which is probably more accurate :). It is indeed "read hot write cold". Writes on these variables are typically during bootup/subsystem initialization/hot plug events.

Thanks,
Kiran

#6 lore raw
In reply to Andrew Morton
Quoted 5 lines · Andrew Morton

I'm thinking we should remove __read_mostly.

Because if we use this everywhere where it's supposed to be used, we end up with .bss and .data 100% populated with write-often variables, packed closely together. The cachelines will really flying around.

one thing we could/should do is have a "written during boot only" section; which we then can mark read only as well :)

#7 lore raw
Quoted 10 lines · Arjan van de Ven
Quoted 5 lines · Andrew Morton

I'm thinking we should remove __read_mostly.

Because if we use this everywhere where it's supposed to be used, we end up with .bss and .data 100% populated with write-often variables, packed closely together. The cachelines will really flying around.

one thing we could/should do is have a "written during boot only" section; which we then can mark read only as well :)

To replicate an IRIX idea:

We could have section with variables that can only be modified by special command. F.e.

enable_write_to_most_read_section()
<change variable>
disable_write_to_most_read_section()

This section could be a per cpu section that would be replicated to all nodes on the system on disable_write_to_most_read_section(). That would mean that critical read only data would be node local.

#8 lore raw
In reply to Andrew Morton
Quoted 4 lines

Because if we use this everywhere where it's supposed to be used, we end up with .bss and .data 100% populated with write-often variables, packed closely together. The cachelines will really flying around.

this argument is true if you have unrelated data together; however if related data is together than suddenly you improve and increase cache density....

#9 lore raw
In reply to Andrew Morton
Quoted 5 lines · Andrew Morton

I'm thinking we should remove __read_mostly.

Because if we use this everywhere where it's supposed to be used, we end up with .bss and .data 100% populated with write-often variables, packed closely together. The cachelines will really flying around.

What we really want is a write-often variable in a cacheline combined with infrequently read and write data. However, data that is frequently read (that is __read_mostly) would still need to be in a separate section.

Quoted 8 lines

IOW: __read_mostly optimises read-mostly variables and pessimises write-often variables.

We want something which optimises both read-mostly and write-often storage.
We do that by marking the write-often variables with
__cacheline_aligned_in_smp.

OK?

I think we would want to group write-often variables with infrequently used variable. But how does one convince the linker to doing that?

I agree that there is a problem with shift frequently written variables together which may in itself cause ill effects. So __read_mostly should only be used when we have identified real cacheline sharing problems and real cache hot variables may have to be put in a separate cacheline for itself. I think we already do that and that is at least the way I have handled it. Too many __read_mostly kill the whole point of the exercise.