Skip to contentSkip to conversation

CONFIG_MIPS_UNCACHED

DISCUSSION Quinn Jensen 5 participants 7 messages Jan 23, 2001 – Jan 26, 2001 MIPS

lore
Share & tools
lore
#0 lore raw

Ralf,

On some machines with weird firmware (e.g. IDT 334 board) the processor comes up with the cache already enabled for kseg0. In this case, the set_cp0_config() call in mips32.c to turn off the cache (gated by CONFIG_MIPS_UNCACHED) should probably come after the first call to flush_cache_all(), which is safer but still not totally safe, I suppose. Or am I totally hosed trying to turn the kseg0 cache off after it was once on?

Quinn Jensen

#1 lore raw
Quoted 11 lines · Quinn Jensen

Ralf,

On some machines with weird firmware (e.g. IDT 334 board) the processor comes up with the cache already enabled for kseg0. In this case, the set_cp0_config() call in mips32.c to turn off the cache (gated by CONFIG_MIPS_UNCACHED) should probably come after the first call to flush_cache_all(), which is safer but still not totally safe, I suppose. Or am I totally hosed trying to turn the kseg0 cache off after it was once on?

That's an issue not only when you're "turning off" the cache, but whenever you muck with the kseg0 cache coherency attribute. The Galileo EV96100, running Galileo's pmon, comes up with kseg0 set to 3, which is the default linux kseg0 cache coherency attribute. However, calling set_cp0_config() without first flushing the cache destroys some data, eventhough the same exact kseg0 attribute is set.

I think a complete cache flush is in order before calling set_cp0_config() and that's a change we should make for all cpus.

Pete

#2 lore raw
Quoted 14 lines · Pete Popov
Quoted 11 lines · Quinn Jensen

Ralf,

On some machines with weird firmware (e.g. IDT 334 board) the processor comes up with the cache already enabled for kseg0. In this case, the set_cp0_config() call in mips32.c to turn off the cache (gated by CONFIG_MIPS_UNCACHED) should probably come after the first call to flush_cache_all(), which is safer but still not totally safe, I suppose. Or am I totally hosed trying to turn the kseg0 cache off after it was once on?

Quoted 7 lines

That's an issue not only when you're "turning off" the cache, but whenever you muck with the kseg0 cache coherency attribute. The Galileo EV96100, running Galileo's pmon, comes up with kseg0 set to 3, which is the default linux kseg0 cache coherency attribute. However, calling set_cp0_config() without first flushing the cache destroys some data, eventhough the same exact kseg0 attribute is set.

It is really surprising to know this. It sounds like a CPU bug to me. Can some MIPS "gods" clarify if such a behaviour is a bug or allowed?

BTW, the CPU in EV96100 is QED RM7000, I believe.

Jun

#3 lore raw
Quoted 4 lines · Jun Sun

It is really surprising to know this. It sounds like a CPU bug to me. Can some MIPS "gods" clarify if such a behaviour is a bug or allowed?

BTW, the CPU in EV96100 is QED RM7000, I believe.

If you want to be strictly correct you have to execute the code that disables caching of KSEG0 in uncached space like KSEG1, then flush the caches before you can resume execution in KSEG0. Otherwise you might end up with dirty d-caches which when flushed will overwrite more uptodate data in memory. The window is very small but yet exists if things are just right.

Ralf

#4 lore raw
Quoted 6 lines · Ralf Baechle
Quoted 3 lines · Jun Sun

It is really surprising to know this. It sounds like a CPU bug to me. Can some MIPS "gods" clarify if such a behaviour is a bug or allowed?

Quoted 8 lines
Quoted 1 line

BTW, the CPU in EV96100 is QED RM7000, I believe.

If you want to be strictly correct you have to execute the code that disables caching of KSEG0 in uncached space like KSEG1, then flush the caches before you can resume execution in KSEG0. Otherwise you might end up with dirty d-caches which when flushed will overwrite more uptodate data in memory. The window is very small but yet exists if things are just right.

The EV96100 running Galileo's pmon exhibits exactly this symptom. Pmon apparently sets up kseg0 to cache coherency 3; but eventhough the kernel also sets it to 3, if I don't flush the caches first I end up with overwritten data. A different version of pmon that I have sets kseg0 to 1 (writethrough). Changing that to 3 isn't a problem -- or at least it doesn't seem to cause any problems.

Pete

#5 lore raw
Quoted 17 lines · Pete Popov
Quoted 6 lines · Ralf Baechle
Quoted 3 lines · Jun Sun

It is really surprising to know this. It sounds like a CPU bug to me. Can some MIPS "gods" clarify if such a behaviour is a bug or allowed?

Quoted 8 lines
Quoted 1 line

BTW, the CPU in EV96100 is QED RM7000, I believe.

If you want to be strictly correct you have to execute the code that disables caching of KSEG0 in uncached space like KSEG1, then flush the caches before you can resume execution in KSEG0. Otherwise you might end up with dirty d-caches which when flushed will overwrite more uptodate data in memory. The window is very small but yet exists if things are just right.

Quoted 7 lines

The EV96100 running Galileo's pmon exhibits exactly this symptom. Pmon apparently sets up kseg0 to cache coherency 3; but eventhough the kernel also sets it to 3, if I don't flush the caches first I end up with overwritten data. A different version of pmon that I have sets kseg0 to 1 (writethrough). Changing that to 3 isn't a problem -- or at least it doesn't seem to cause any problems.

I don't think it is the same problem.

Here is the simplified view of the process, if I understand Pete correctly:

1. pmon sets kseg0 to 3 (cache enabled)
2. kernel starts in KSEG0
3. kernel sets kseg0 to 3 again (essentially keeps the same config value)
4. kernel flushes cache
===> Q: data corruption or not?

I think the data should be consistent. Otherwise it looks like a CPU bug to me.

What ralf described is something like the following:

1. pmon sets kseg0 to 3 (KSEG0 cache enabled)
2. kernel starts in KSEG0
3. kernel sets kseg0 to 2 (disable kseg0 cache)
4. kernel flushes cache
===> Q: data corruption or not? YES, data can be corrupted!

Jun

#6 lore raw
In reply to Ralf Baechle
Quoted 2 lines
Quoted 2 lines

It is really surprising to know this. It sounds like a CPU bug to me.

Can

Quoted 10 lines
Quoted 3 lines

some MIPS "gods" clarify if such a behaviour is a bug or allowed?

BTW, the CPU in EV96100 is QED RM7000, I believe.

If you want to be strictly correct you have to execute the code that disables caching of KSEG0 in uncached space like KSEG1, then flush the caches before you can resume execution in KSEG0. Otherwise you might end up with dirty d-caches which when flushed will overwrite more uptodate data in memory. The window is very small but yet exists if things are just right.

That's one possible failure mode. The other is that there
are dirty lines (those that were brought into the cache and modified by stores, but never written back to memory) that are rendered invisible by forcing KSEG0 to go uncached. This will cause failure of any code that is expecting to see the results of those stores if said code executes before some other event causes the write-back - and if the machine is running with cacheing completely disabled, as it would be in the case under discussion, such an event may never occur!

Regards,

Kevin K.