RISC-V: Probe for misaligned access speed
Subsystems (2)
v4 cover RISC-V: Probe for misaligned access speed
The current setting for the hwprobe bit indicating misaligned access speed is controlled by a vendor-specific feature probe function. This is essentially a per-SoC table we have to maintain on behalf of each vendor going forward. Let's convert that instead to something we detect at runtime.
We have two assembly routines at the heart of our probe: one that does a bunch of word-sized accesses (without aligning its input buffer), and the other that does byte accesses. If we can move a larger number of bytes using misaligned word accesses than we can with the same amount of time doing byte accesses, then we can declare misaligned accesses as "fast".
The tradeoff of reducing this maintenance burden is boot time. We spend 4-6 jiffies per core doing this measurement (0-2 on jiffie edge alignment, and 4 on measurement). The timing loop was based on raid6_choose_gen(), which uses (16+1)*N jiffies (where N is the number of algorithms). By taking only the fastest iteration out of all attempts for use in the comparison, variance between runs is very low. On my THead C906, it looks like this:
[ 0.047563] cpu0: Ratio of byte access time to unaligned word access is 4.34, unaligned accesses are fast
Several others have chimed in with results on slow machines with the older algorithm, which took all runs into account, including noise like interrupts. Even with this variation, results indicate that in all cases (fast, slow, and emulated) the measured numbers are nowhere near each other (always multiple factors away).
Changes in v4:
- Avoid the bare 64-bit divide which fails to link on 32-bit systems, use div_u64() (Palmer, buildrobot)
Changes in v3:
- Fix documentation indentation (Conor)
- Rename __copy_..._unaligned() to __riscv_copy_..._unaligned() (Conor)
- Renamed c0,c1 to start_cycles, end_cycles (Conor)
- Renamed j0,j1 to start_jiffies, now
- Renamed check_unaligned_access0() to check_unaligned_access_boot_cpu() (Conor)
Changes in v2:
- Explain more in the commit message (Conor)
- Use a new algorithm that looks for the fastest run (David)
- Clarify documentatin further (David and Conor)
- Unify around a single word, "unaligned" (Conor)
- Align asm operands, and other misc whitespace changes (Conor)
Evan Green (2):
RISC-V: Probe for unaligned access speed
RISC-V: alternative: Remove feature_probe_func
Documentation/riscv/hwprobe.rst | 11 ++- arch/riscv/errata/thead/errata.c | 8 --- arch/riscv/include/asm/alternative.h | 5 -- arch/riscv/include/asm/cpufeature.h | 2 + arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/alternative.c | 19 ----- arch/riscv/kernel/copy-unaligned.S | 71 ++++++++++++++++++ arch/riscv/kernel/copy-unaligned.h | 13 ++++ arch/riscv/kernel/cpufeature.c | 104 +++++++++++++++++++++++++++ arch/riscv/kernel/smpboot.c | 3 +- 10 files changed, 198 insertions(+), 39 deletions(-) create mode 100644 arch/riscv/kernel/copy-unaligned.S create mode 100644 arch/riscv/kernel/copy-unaligned.h
Signature · 7 lines
2.34.1 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
PATCH v4 1/2 RISC-V: Probe for unaligned access speed
Rather than deferring unaligned access speed determinations to a vendor function, let's probe them and find out how fast they are. If we determine that an unaligned word access is faster than N byte accesses, mark the hardware's unaligned access as "fast". Otherwise, we mark accesses as slow.
The algorithm itself runs for a fixed amount of jiffies. Within each iteration it attempts to time a single loop, and then keeps only the best (fastest) loop it saw. This algorithm was found to have lower variance from run to run than my first attempt, which counted the total number of iterations that could be done in that fixed amount of jiffies. By taking only the best iteration in the loop, assuming at least one loop wasn't perturbed by an interrupt, we eliminate the effects of interrupts and other "warm up" factors like branch prediction. The only downside is it depends on having an rdtime granular and accurate enough to measure a single copy. If we ever manage to complete a loop in 0 rdtime ticks, we leave the unaligned setting at UNKNOWN.
There is a slight change in user-visible behavior here. Previously, all boards except the THead C906 reported misaligned access speed of UNKNOWN. C906 reported FAST. With this change, since we're now measuring misaligned access speed on each hart, all RISC-V systems will have this key set as either FAST or SLOW.
Currently, we don't have a way to confidently measure the difference between SLOW and EMULATED, so we label anything not fast as SLOW. This will mislabel some systems that are actually EMULATED as SLOW. When we get support for delegating misaligned access traps to the kernel (as opposed to the firmware quietly handling it), we can explicitly test in Linux to see if unaligned accesses trap. Those systems will start to report EMULATED, though older (today's) systems without that new SBI mechanism will continue to report SLOW.
I've updated the documentation for those hwprobe values to reflect this, specifically: SLOW may or may not be emulated by software, and FAST represents means being faster than equivalent byte accesses. The change in documentation is accurate with respect to both the former and current behavior.
---
Changes in v4:
- Avoid the bare 64-bit divide which fails to link on 32-bit systems, use div_u64() (Palmer, buildrobot)
Changes in v3:
- Fix documentation indentation (Conor)
- Rename __copy_..._unaligned() to __riscv_copy_..._unaligned() (Conor)
- Renamed c0,c1 to start_cycles, end_cycles (Conor)
- Renamed j0,j1 to start_jiffies, now
- Renamed check_unaligned_access0() to check_unaligned_access_boot_cpu() (Conor)
Changes in v2:
- Explain more in the commit message (Conor)
- Use a new algorithm that looks for the fastest run (David)
- Clarify documentatin further (David and Conor)
- Unify around a single word, "unaligned" (Conor)
- Align asm operands, and other misc whitespace changes (Conor)
Documentation/riscv/hwprobe.rst | 11 ++- arch/riscv/include/asm/cpufeature.h | 2 + arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/copy-unaligned.S | 71 +++++++++++++++++++ arch/riscv/kernel/copy-unaligned.h | 13 ++++ arch/riscv/kernel/cpufeature.c | 104 ++++++++++++++++++++++++++++ arch/riscv/kernel/smpboot.c | 2 + 7 files changed, 198 insertions(+), 6 deletions(-) create mode 100644 arch/riscv/kernel/copy-unaligned.S create mode 100644 arch/riscv/kernel/copy-unaligned.h
Documentation/riscv/hwprobe.rst +5 -6
@@ -87,13 +87,12 @@ The following keys are defined: emulated via software, either in or below the kernel. These accesses are always extremely slow. - * :c:macro:`RISCV_HWPROBE_MISALIGNED_SLOW`: Misaligned accesses are supported - in hardware, but are slower than the cooresponding aligned accesses - sequences. + * :c:macro:`RISCV_HWPROBE_MISALIGNED_SLOW`: Misaligned accesses are slower + than equivalent byte accesses. Misaligned accesses may be supported + directly in hardware, or trapped and emulated by software. - * :c:macro:`RISCV_HWPROBE_MISALIGNED_FAST`: Misaligned accesses are supported - in hardware and are faster than the cooresponding aligned accesses - sequences. + * :c:macro:`RISCV_HWPROBE_MISALIGNED_FAST`: Misaligned accesses are faster + than equivalent byte accesses. * :c:macro:`RISCV_HWPROBE_MISALIGNED_UNSUPPORTED`: Misaligned accesses are not supported at all and will generate a misaligned address fault.
arch/riscv/include/asm/cpufeature.h +2 -0
@@ -30,4 +30,6 @@ DECLARE_PER_CPU(long, misaligned_access_speed);/* Per-cpu ISA extensions. */ extern struct riscv_isainfo hart_isa[NR_CPUS ]; +void check_unaligned_access (int cpu); +#endif
arch/riscv/kernel/Makefile +1 -0
@@ -38,6 +38,7 @@ extra-y += vmlinux.lds obj-y += head.o obj-y += soc.o obj-$(CONFIG_RISCV_ALTERNATIVE) += alternative.o +obj-y += copy-unaligned.o obj-y += cpu.o obj-y += cpufeature.o obj-y += entry.o
arch/riscv/kernel/copy-unaligned.S new file +71 -0
71 lines not on this page · raw message
arch/riscv/kernel/copy-unaligned.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (C) 2023 Rivos, Inc. +*/ +#ifndef __RISCV_KERNEL_COPY_UNALIGNED_H +#define __RISCV_KERNEL_COPY_UNALIGNED_H + +#include <linux/types.h> + +void __riscv_copy_words_unaligned (void *dst,const void *src,size_t size); +void __riscv_copy_bytes_unaligned (void *dst,const void *src,size_t size); + +#endif /* __RISCV_KERNEL_COPY_UNALIGNED_H */
arch/riscv/kernel/cpufeature.c +104 -0
122 lines not on this page · raw message
arch/riscv/kernel/smpboot.c +2 -0
@@ -26,6 +26,7 @@#include <linux/sched/task_stack.h> #include <linux/sched/mm.h> #include <asm/cpu_ops.h> +#include <asm/cpufeature.h> #include <asm/irq.h> #include <asm/mmu_context.h> #include <asm/numa.h>
@@ -245,6 +246,7 @@ asmlinkage __visible void smp_callin(void)numa_add_cpu (curr_cpuid);set_cpu_online (curr_cpuid,1 ); +check_unaligned_access (curr_cpuid);probe_vendor_features (curr_cpuid);if (has_vector ()) {
Signature · 7 lines
2.34.1 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
Re: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
Hi Evan,
Quoted 33 lines · Evan Green
Rather than deferring unaligned access speed determinations to a vendor function, let's probe them and find out how fast they are. If we determine that an unaligned word access is faster than N byte accesses, mark the hardware's unaligned access as "fast". Otherwise, we mark accesses as slow.
The algorithm itself runs for a fixed amount of jiffies. Within each iteration it attempts to time a single loop, and then keeps only the best (fastest) loop it saw. This algorithm was found to have lower variance from run to run than my first attempt, which counted the total number of iterations that could be done in that fixed amount of jiffies. By taking only the best iteration in the loop, assuming at least one loop wasn't perturbed by an interrupt, we eliminate the effects of interrupts and other "warm up" factors like branch prediction. The only downside is it depends on having an rdtime granular and accurate enough to measure a single copy. If we ever manage to complete a loop in 0 rdtime ticks, we leave the unaligned setting at UNKNOWN.
There is a slight change in user-visible behavior here. Previously, all boards except the THead C906 reported misaligned access speed of UNKNOWN. C906 reported FAST. With this change, since we're now measuring misaligned access speed on each hart, all RISC-V systems will have this key set as either FAST or SLOW.
Currently, we don't have a way to confidently measure the difference between SLOW and EMULATED, so we label anything not fast as SLOW. This will mislabel some systems that are actually EMULATED as SLOW. When we get support for delegating misaligned access traps to the kernel (as opposed to the firmware quietly handling it), we can explicitly test in Linux to see if unaligned accesses trap. Those systems will start to report EMULATED, though older (today's) systems without that new SBI mechanism will continue to report SLOW.
Quoted 8 lines
I've updated the documentation for those hwprobe values to reflect this, specifically: SLOW may or may not be emulated by software, and FAST represents means being faster than equivalent byte accesses. The change in documentation is accurate with respect to both the former and current behavior.
Signed-off-by Evan Green <[email protected]>Acked-by Conor Dooley <[email protected]>
Thanks for your patch, which is now commit 584ea6564bcaead2 ("RISC-V:
Probe for unaligned access speed") in v6.6-rc1.
On the boards I have, I get:
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fast
icicle:
cpu1: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
cpu2: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
cpu3: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
k210:
cpu1: Ratio of byte access time to unaligned word access is
0.02, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is
0.02, unaligned accesses are slow
starlight:
cpu1: Ratio of byte access time to unaligned word access is
0.01, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is
0.02, unaligned accesses are slow
vexriscv/orangecrab:
cpu0: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
I am a bit surprised by the near-zero values. Are these expected? Thanks!
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Re: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
Quoted 46 lines · Geert Uytterhoeven
Hi Evan,
Quoted 33 lines · Evan Green
Rather than deferring unaligned access speed determinations to a vendor function, let's probe them and find out how fast they are. If we determine that an unaligned word access is faster than N byte accesses, mark the hardware's unaligned access as "fast". Otherwise, we mark accesses as slow.
The algorithm itself runs for a fixed amount of jiffies. Within each iteration it attempts to time a single loop, and then keeps only the best (fastest) loop it saw. This algorithm was found to have lower variance from run to run than my first attempt, which counted the total number of iterations that could be done in that fixed amount of jiffies. By taking only the best iteration in the loop, assuming at least one loop wasn't perturbed by an interrupt, we eliminate the effects of interrupts and other "warm up" factors like branch prediction. The only downside is it depends on having an rdtime granular and accurate enough to measure a single copy. If we ever manage to complete a loop in 0 rdtime ticks, we leave the unaligned setting at UNKNOWN.
There is a slight change in user-visible behavior here. Previously, all boards except the THead C906 reported misaligned access speed of UNKNOWN. C906 reported FAST. With this change, since we're now measuring misaligned access speed on each hart, all RISC-V systems will have this key set as either FAST or SLOW.
Currently, we don't have a way to confidently measure the difference between SLOW and EMULATED, so we label anything not fast as SLOW. This will mislabel some systems that are actually EMULATED as SLOW. When we get support for delegating misaligned access traps to the kernel (as opposed to the firmware quietly handling it), we can explicitly test in Linux to see if unaligned accesses trap. Those systems will start to report EMULATED, though older (today's) systems without that new SBI mechanism will continue to report SLOW.
Quoted 8 lines
I've updated the documentation for those hwprobe values to reflect this, specifically: SLOW may or may not be emulated by software, and FAST represents means being faster than equivalent byte accesses. The change in documentation is accurate with respect to both the former and current behavior.
Signed-off-by Evan Green <[email protected]>Acked-by Conor Dooley <[email protected]>
Quoted 8 lines
Thanks for your patch, which is now commit
584ea6564bcaead2("RISC-V:
Probe for unaligned access speed") in v6.6-rc1.On the boards I have, I get:
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fast
Hrm, I'm a little surprised to be seeing this number come out so close to 1. If you reboot a few times, what kind of variance do you get on this?
Quoted 27 lines
icicle:
cpu1: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
cpu2: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
cpu3: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slowcpu0: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slowk210:
cpu1: Ratio of byte access time to unaligned word access is
0.02, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is
0.02, unaligned accesses are slowstarlight:
cpu1: Ratio of byte access time to unaligned word access is
0.01, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is
0.02, unaligned accesses are slow
Quoted 7 lines
vexriscv/orangecrab:
cpu0: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slowI am a bit surprised by the near-zero values. Are these expected? Thanks!
This could be expected, if firmware is trapping the unaligned accesses and coming out >100x slower than a native access. If you're interested in getting a little more resolution, you could try to print a few more decimal places with something like (sorry gmail mangles the whitespace on this):
arch/riscv/kernel/cpufeature.c +0 -2
@@ -632,11 +632,11 @@ void check_unaligned_access(int cpu)if (word_cycles < byte_cycles) speed =RISCV_HWPROBE_MISALIGNED_FAST ; - ratio =div_u64 ((byte_cycles *100 ), word_cycles); - pr_info("cpu%d: Ratio of byte access time to unaligned word
access is %d.%02d, unaligned accesses are %s\n",
+ ratio = div_u64((byte_cycles * 100000), word_cycles);
+ pr_info("cpu%d: Ratio of byte access time to unaligned word
access is %d.%05d, unaligned accesses are %s\n",
cpu,
- ratio / 100,
- ratio % 100,
+ ratio / 100000,
+ ratio % 100000,
(speed == RISCV_HWPROBE_MISALIGNED_FAST) ? "fast" : "slow");
per_cpu(misaligned_access_speed, cpu) = speed;If you did, I'd be interested to see the results.
-Evan
Re: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
Hi Evan,
Quoted 49 lines · Evan Green
Quoted 48 lines · Geert Uytterhoeven
Quoted 33 lines · Evan Green
Rather than deferring unaligned access speed determinations to a vendor function, let's probe them and find out how fast they are. If we determine that an unaligned word access is faster than N byte accesses, mark the hardware's unaligned access as "fast". Otherwise, we mark accesses as slow.
The algorithm itself runs for a fixed amount of jiffies. Within each iteration it attempts to time a single loop, and then keeps only the best (fastest) loop it saw. This algorithm was found to have lower variance from run to run than my first attempt, which counted the total number of iterations that could be done in that fixed amount of jiffies. By taking only the best iteration in the loop, assuming at least one loop wasn't perturbed by an interrupt, we eliminate the effects of interrupts and other "warm up" factors like branch prediction. The only downside is it depends on having an rdtime granular and accurate enough to measure a single copy. If we ever manage to complete a loop in 0 rdtime ticks, we leave the unaligned setting at UNKNOWN.
There is a slight change in user-visible behavior here. Previously, all boards except the THead C906 reported misaligned access speed of UNKNOWN. C906 reported FAST. With this change, since we're now measuring misaligned access speed on each hart, all RISC-V systems will have this key set as either FAST or SLOW.
Currently, we don't have a way to confidently measure the difference between SLOW and EMULATED, so we label anything not fast as SLOW. This will mislabel some systems that are actually EMULATED as SLOW. When we get support for delegating misaligned access traps to the kernel (as opposed to the firmware quietly handling it), we can explicitly test in Linux to see if unaligned accesses trap. Those systems will start to report EMULATED, though older (today's) systems without that new SBI mechanism will continue to report SLOW.
Quoted 8 lines
I've updated the documentation for those hwprobe values to reflect this, specifically: SLOW may or may not be emulated by software, and FAST represents means being faster than equivalent byte accesses. The change in documentation is accurate with respect to both the former and current behavior.
Signed-off-by Evan Green <[email protected]>Acked-by Conor Dooley <[email protected]>Thanks for your patch, which is now commit
584ea6564bcaead2("RISC-V:
Probe for unaligned access speed") in v6.6-rc1.On the boards I have, I get:
Quoted 7 lines
Quoted 3 lines
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fastHrm, I'm a little surprised to be seeing this number come out so close to 1. If you reboot a few times, what kind of variance do you get on this?
Rock-solid at 1.05 (even with increased resolution: 1.05853 on 3 tries)
Quoted 11 lines
Quoted 11 lines
icicle:
cpu1: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
cpu2: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
cpu3: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slowcpu0: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
cpu1: Ratio of byte access time to unaligned word access is 0.00344, unaligned accesses are slow
cpu2: Ratio of byte access time to unaligned word access is 0.00343, unaligned accesses are slow
cpu3: Ratio of byte access time to unaligned word access is 0.00343, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is 0.00340, unaligned accesses are slow
Quoted 6 lines
Quoted 6 lines
k210:
cpu1: Ratio of byte access time to unaligned word access is
0.02, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is
0.02, unaligned accesses are slow
cpu1: Ratio of byte access time to unaligned word access is 0.02392, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is 0.02084, unaligned accesses are slow
Quoted 6 lines
Quoted 6 lines
starlight:
cpu1: Ratio of byte access time to unaligned word access is
0.01, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is
0.02, unaligned accesses are slow
cpu1: Ratio of byte access time to unaligned word access is 0.01872, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is 0.01930, unaligned accesses are slow
Quoted 4 lines
Quoted 4 lines
vexriscv/orangecrab:
cpu0: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slow
cpu0: Ratio of byte access time to unaligned word access is 0.00417, unaligned accesses are slow
Quoted 7 lines
Quoted 1 line
I am a bit surprised by the near-zero values. Are these expected?
This could be expected, if firmware is trapping the unaligned accesses and coming out >100x slower than a native access. If you're interested in getting a little more resolution, you could try to print a few more decimal places with something like (sorry gmail mangles the whitespace on this):
Looks like you need to add one digit to get anything useful on half of the systems.
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
RE: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
From: Geert Uytterhoeven
Quoted 1 line
Sent: 14 September 2023 08:33
...
Quoted 9 lines
Quoted 7 lines
Quoted 3 lines
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fastHrm, I'm a little surprised to be seeing this number come out so close to 1. If you reboot a few times, what kind of variance do you get on this?
Rock-solid at 1.05 (even with increased resolution: 1.05853 on 3 tries)
Would that match zero overhead unless the access crosses a cache line boundary?
(I can't remember whether the test is using increasing addresses.)
...
Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
vexriscv/orangecrab:
cpu0: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slowcpu0: Ratio of byte access time to unaligned word access is 0.00417, unaligned accesses are slow
Quoted 7 lines
Quoted 1 line
I am a bit surprised by the near-zero values. Are these expected?
This could be expected, if firmware is trapping the unaligned accesses and coming out >100x slower than a native access. If you're interested in getting a little more resolution, you could try to print a few more decimal places with something like (sorry gmail mangles the whitespace on this):
I'd expect one of three possible values:
- 1.0x: Basically zero cost except for cache line/page boundaries.
- ~2: Hardware does two reads and merges the values.
- >100: Trap fixed up in software.
I'd think the '2' case could be considered fast. You only need to time one access to see if it was a fault.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Re: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
Quoted 12 lines · David Laight
From: Geert Uytterhoeven
Quoted 1 line
Sent: 14 September 2023 08:33
...
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fastHrm, I'm a little surprised to be seeing this number come out so close to 1. If you reboot a few times, what kind of variance do you get on this?
Quoted 5 lines
Quoted 1 line
Rock-solid at 1.05 (even with increased resolution: 1.05853 on 3 tries)
Would that match zero overhead unless the access crosses a cache line boundary?
(I can't remember whether the test is using increasing addresses.)
Yes, the test does use increasing addresses, it copies across 4 pages. We start with a warmup, so caching effects beyond L1 are largely not taken into account.
Quoted 18 lines
...
Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
vexriscv/orangecrab:
cpu0: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slowcpu0: Ratio of byte access time to unaligned word access is 0.00417, unaligned accesses are slow
Quoted 7 lines
Quoted 1 line
I am a bit surprised by the near-zero values. Are these expected?
This could be expected, if firmware is trapping the unaligned accesses and coming out >100x slower than a native access. If you're interested in getting a little more resolution, you could try to print a few more decimal places with something like (sorry gmail mangles the whitespace on this):
Quoted 7 lines
I'd expect one of three possible values:
- 1.0x: Basically zero cost except for cache line/page boundaries.
- ~2: Hardware does two reads and merges the values.
- >100: Trap fixed up in software.I'd think the '2' case could be considered fast. You only need to time one access to see if it was a fault.
We're comparing misaligned word accesses with byte accesses of the same total size. So 1.0 means a misaligned load is basically no different from 8 byte loads. The goal was to help people that are forced to do odd loads and stores decide whether they are better off moving by bytes or by misaligned words. (In contrast, the answer to "should I do a misaligned word load or an aligned word load" is generally always "do the aligned one if you can", so comparing those two things didn't seem as useful).
We opted for 1.0 as a cutoff, since even at 1.05, you get a boost from doing misaligned word loads over byte copies. I asked about the variance because I don't want to see machines that change their mind from boot to boot. I originally considered trying to create a "gray zone" where the answer goes back to UNKNOWN, but in the end that just moves the fiddly point rather than really eliminating it.
You're right that in theory we just need one perfect access to test, but testing only once makes it susceptible to hiccups. We went with doing it many times in a fixed period and taking the minimum to hopefully remove noise like NMI-like things, branch prediction misses, or cache eviction.
Geert,
Thanks for providing the numbers. Yes, we could add another digit to the print. Though if you already know you're at least 100x slower, maybe knowing exactly how much slower isn't super meaningful, just very much avoid unaligned accesses on these systems :). Hopefully over time the number of systems like this will dwindle.
-Evan
RE: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
From: Evan Green
Quoted 17 lines
Sent: 14 September 2023 16:01
Quoted 14 lines · David Laight
From: Geert Uytterhoeven
Quoted 1 line
Sent: 14 September 2023 08:33
...
Quoted 9 lines
Quoted 7 lines
Quoted 3 lines
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fastHrm, I'm a little surprised to be seeing this number come out so close to 1. If you reboot a few times, what kind of variance do you get on this?
Rock-solid at 1.05 (even with increased resolution: 1.05853 on 3 tries)
Quoted 7 lines
Quoted 3 lines
Would that match zero overhead unless the access crosses a cache line boundary?
(I can't remember whether the test is using increasing addresses.)Yes, the test does use increasing addresses, it copies across 4 pages. We start with a warmup, so caching effects beyond L1 are largely not taken into account.
That seems entirely excessive.
If you want to avoid data cache issues (which probably do) then just repeating a single access would almost certainly suffice.
Repeatedly using a short buffer (say 256 bytes) won't add much loop overhead.
Although you may want to do a test that avoids transfers that cross cache line and especially page boundaries. Either of those could easily be much slower than a read that is entirely within a cache line.
...
Quoted 24 lines
Quoted 16 lines
Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
vexriscv/orangecrab:
cpu0: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slowcpu0: Ratio of byte access time to unaligned word access is 0.00417, unaligned accesses are slow
Quoted 7 lines
Quoted 1 line
I am a bit surprised by the near-zero values. Are these expected?
This could be expected, if firmware is trapping the unaligned accesses and coming out >100x slower than a native access. If you're interested in getting a little more resolution, you could try to print a few more decimal places with something like (sorry gmail mangles the whitespace on this):
Quoted 7 lines
I'd expect one of three possible values:
- 1.0x: Basically zero cost except for cache line/page boundaries.
- ~2: Hardware does two reads and merges the values.
- >100: Trap fixed up in software.I'd think the '2' case could be considered fast. You only need to time one access to see if it was a fault.
Quoted 8 lines
We're comparing misaligned word accesses with byte accesses of the same total size. So 1.0 means a misaligned load is basically no different from 8 byte loads. The goal was to help people that are forced to do odd loads and stores decide whether they are better off moving by bytes or by misaligned words. (In contrast, the answer to "should I do a misaligned word load or an aligned word load" is generally always "do the aligned one if you can", so comparing those two things didn't seem as useful).
Ah, I'd have compared the cost of aligned accesses with misaligned ones. That would tell you whether you really need to avoid them. The cost of byte and aligned word accesses should be much the same (for each access that is) - if not you've got a real bottleneck.
If a misaligned access is 8 times slower than an aligned one it is still 'quite slow'.
I'd definitely call that 8 not 1 - even if you treat it as 'fast'.
For comparison you (well I) can write x64-64 asm for the ip-checksum loop that will execute 1 memory read every clock (8 bytes/clock). It is very slightly slower for misaligned buffers, but by less than 1 clock per cache line.
That's what I'd call 1.0 :-)
I'd expect even simple hardware to do misaligned reads as two reads and then merge the data - so should really be no slower than two separate aligned reads.
Since you'd expect a cpu to do an L1 data cache read every clock (probably pipelined) the misaligned read should just add 1 clock.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Re: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
Quoted 27 lines · David Laight
From: Evan Green
Quoted 17 lines
Sent: 14 September 2023 16:01
Quoted 14 lines · David Laight
From: Geert Uytterhoeven
Quoted 1 line
Sent: 14 September 2023 08:33
...
Quoted 9 lines
Quoted 7 lines
Quoted 3 lines
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fastHrm, I'm a little surprised to be seeing this number come out so close to 1. If you reboot a few times, what kind of variance do you get on this?
Rock-solid at 1.05 (even with increased resolution: 1.05853 on 3 tries)
Quoted 7 lines
Quoted 3 lines
Would that match zero overhead unless the access crosses a cache line boundary?
(I can't remember whether the test is using increasing addresses.)Yes, the test does use increasing addresses, it copies across 4 pages. We start with a warmup, so caching effects beyond L1 are largely not taken into account.
Quoted 10 lines
That seems entirely excessive.
If you want to avoid data cache issues (which probably do) then just repeating a single access would almost certainly suffice.
Repeatedly using a short buffer (say 256 bytes) won't add much loop overhead.
Although you may want to do a test that avoids transfers that cross cache line and especially page boundaries. Either of those could easily be much slower than a read that is entirely within a cache line.
We won't be faulting on any of these pages, and they should remain in the TLB, so I don't expect many page boundary specific effects. If there is a steep penalty for misaligned loads across a cache line, such that it's worse than doing byte accesses, I want the test results to be dinged for that.
Quoted 35 lines
...
Quoted 24 lines
Quoted 16 lines
Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
vexriscv/orangecrab:
cpu0: Ratio of byte access time to unaligned word access is
0.00, unaligned accesses are slowcpu0: Ratio of byte access time to unaligned word access is 0.00417, unaligned accesses are slow
Quoted 7 lines
Quoted 1 line
I am a bit surprised by the near-zero values. Are these expected?
This could be expected, if firmware is trapping the unaligned accesses and coming out >100x slower than a native access. If you're interested in getting a little more resolution, you could try to print a few more decimal places with something like (sorry gmail mangles the whitespace on this):
Quoted 7 lines
I'd expect one of three possible values:
- 1.0x: Basically zero cost except for cache line/page boundaries.
- ~2: Hardware does two reads and merges the values.
- >100: Trap fixed up in software.I'd think the '2' case could be considered fast. You only need to time one access to see if it was a fault.
Quoted 8 lines
We're comparing misaligned word accesses with byte accesses of the same total size. So 1.0 means a misaligned load is basically no different from 8 byte loads. The goal was to help people that are forced to do odd loads and stores decide whether they are better off moving by bytes or by misaligned words. (In contrast, the answer to "should I do a misaligned word load or an aligned word load" is generally always "do the aligned one if you can", so comparing those two things didn't seem as useful).
Quoted 8 lines
Ah, I'd have compared the cost of aligned accesses with misaligned ones. That would tell you whether you really need to avoid them. The cost of byte and aligned word accesses should be much the same (for each access that is) - if not you've got a real bottleneck.
If a misaligned access is 8 times slower than an aligned one it is still 'quite slow'.
I'd definitely call that 8 not 1 - even if you treat it as 'fast'.
The number itself isn't exported or saved anywhere, it's just printed as diagnostic explanation into the final fast/slow designation.
Misaligned word loads are never going to be faster than aligned ones, and aren't really going to be equal either. It's also generally not something that causes software a lot of angst: we align most of our buffers and structures with help from the compiler, and generally do an aligned access whenever possible. It's the times when we're forced to do odd sizes or accesses we know are already misaligned that this hwprobe bit was designed to help. In those cases, users are forced to decide if they should do a misaligned word access or byte accesses, so we aim to provide that result.
If there's a use case for knowing "misaligned accesses are exactly as fast as aligned ones", we could detect this threshold in the same test, and add another hwprobe bit for it.
-Evan
Quoted 18 lines
For comparison you (well I) can write x64-64 asm for the ip-checksum loop that will execute 1 memory read every clock (8 bytes/clock). It is very slightly slower for misaligned buffers, but by less than 1 clock per cache line.
That's what I'd call 1.0 :-)I'd expect even simple hardware to do misaligned reads as two reads and then merge the data - so should really be no slower than two separate aligned reads.
Since you'd expect a cpu to do an L1 data cache read every clock (probably pipelined) the misaligned read should just add 1 clock.David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
RE: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
From: Evan Green
Quoted 41 lines
Sent: 14 September 2023 17:37
Quoted 27 lines · David Laight
From: Evan Green
Quoted 17 lines
Sent: 14 September 2023 16:01
Quoted 14 lines · David Laight
From: Geert Uytterhoeven
Quoted 1 line
Sent: 14 September 2023 08:33
...
Quoted 9 lines
Quoted 7 lines
Quoted 3 lines
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fastHrm, I'm a little surprised to be seeing this number come out so close to 1. If you reboot a few times, what kind of variance do you get on this?
Rock-solid at 1.05 (even with increased resolution: 1.05853 on 3 tries)
Quoted 7 lines
Quoted 3 lines
Would that match zero overhead unless the access crosses a cache line boundary?
(I can't remember whether the test is using increasing addresses.)Yes, the test does use increasing addresses, it copies across 4 pages. We start with a warmup, so caching effects beyond L1 are largely not taken into account.
Quoted 10 lines
That seems entirely excessive.
If you want to avoid data cache issues (which probably do) then just repeating a single access would almost certainly suffice.
Repeatedly using a short buffer (say 256 bytes) won't add much loop overhead.
Although you may want to do a test that avoids transfers that cross cache line and especially page boundaries. Either of those could easily be much slower than a read that is entirely within a cache line.
Quoted 5 lines
We won't be faulting on any of these pages, and they should remain in the TLB, so I don't expect many page boundary specific effects. If there is a steep penalty for misaligned loads across a cache line, such that it's worse than doing byte accesses, I want the test results to be dinged for that.
That is an entirely different issue.
Are you absolutely certain that the reason 8 byte loads take as long as a 64-bit mis-aligned load isn't because the entire test is limited by L1 cache fills?
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Re: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
Quoted 49 lines · David Laight
From: Evan Green
Quoted 41 lines
Sent: 14 September 2023 17:37
Quoted 27 lines · David Laight
From: Evan Green
Quoted 17 lines
Sent: 14 September 2023 16:01
Quoted 14 lines · David Laight
From: Geert Uytterhoeven
Quoted 1 line
Sent: 14 September 2023 08:33
...
Quoted 9 lines
Quoted 7 lines
Quoted 3 lines
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fastHrm, I'm a little surprised to be seeing this number come out so close to 1. If you reboot a few times, what kind of variance do you get on this?
Rock-solid at 1.05 (even with increased resolution: 1.05853 on 3 tries)
Quoted 7 lines
Quoted 3 lines
Would that match zero overhead unless the access crosses a cache line boundary?
(I can't remember whether the test is using increasing addresses.)Yes, the test does use increasing addresses, it copies across 4 pages. We start with a warmup, so caching effects beyond L1 are largely not taken into account.
Quoted 10 lines
That seems entirely excessive.
If you want to avoid data cache issues (which probably do) then just repeating a single access would almost certainly suffice.
Repeatedly using a short buffer (say 256 bytes) won't add much loop overhead.
Although you may want to do a test that avoids transfers that cross cache line and especially page boundaries. Either of those could easily be much slower than a read that is entirely within a cache line.Quoted 5 lines
We won't be faulting on any of these pages, and they should remain in the TLB, so I don't expect many page boundary specific effects. If there is a steep penalty for misaligned loads across a cache line, such that it's worse than doing byte accesses, I want the test results to be dinged for that.
Quoted 5 lines
That is an entirely different issue.
Are you absolutely certain that the reason 8 byte loads take as long as a 64-bit mis-aligned load isn't because the entire test is limited by L1 cache fills?
Fair question. I hacked up a little code [1] to retry the test at several different sizes, as well as printing out the best and worst times. I only have one piece of real hardware, the THead C906, which has a 32KB L1 D-cache.
Here are the results at various sizes, starting with the original:
[ 0.047556] cpu0: Ratio of byte access time to unaligned word access is 4.35, unaligned accesses are fast [ 0.047578] EVAN size 0x1f80 word cycles best 69 worst 29e, byte cycles best 1c9 worst 3b7 [ 0.071549] cpu0: Ratio of byte access time to unaligned word access is 4.29, unaligned accesses are fast [ 0.071566] EVAN size 0x1000 word cycles best 36 worst 210, byte cycles best e8 worst 2b2 [ 0.095540] cpu0: Ratio of byte access time to unaligned word access is 4.14, unaligned accesses are fast [ 0.095556] EVAN size 0x200 word cycles best 7 worst 1d9, byte cycles best 1d worst 1d5 [ 0.119539] cpu0: Ratio of byte access time to unaligned word access is 5.00, unaligned accesses are fast [ 0.119555] EVAN size 0x100 word cycles best 3 worst 1a8, byte cycles best f worst 1b5 [ 0.143538] cpu0: Ratio of byte access time to unaligned word access is 3.50, unaligned accesses are fast [ 0.143556] EVAN size 0x80 word cycles best 2 worst 1a5, byte
cycles best 7 worst 1aa
[1] https://pastebin.com/uwwU2CVn
I don't see any cliffs as the numbers get smaller, so it seems to me there are no working set issues. Geert, it might be interesting to see these same results on the rzfive. The thing that made me uncomfortable with the smaller buffer sizes is it starts to bump up against the resolution of the timer. Another option would have been to time several iterations, but I went with the larger buffer instead as I'd hoped it would minimize other overhead like the function calls, branch prediction, C loop management, etc.
-Evan
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Re: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
In reply to Geert UytterhoevenHi Prabahkar,
Quoted 54 lines · Geert Uytterhoeven
Quoted 53 lines · Evan Green
Quoted 51 lines · Geert Uytterhoeven
Quoted 33 lines · Evan Green
Rather than deferring unaligned access speed determinations to a vendor function, let's probe them and find out how fast they are. If we determine that an unaligned word access is faster than N byte accesses, mark the hardware's unaligned access as "fast". Otherwise, we mark accesses as slow.
The algorithm itself runs for a fixed amount of jiffies. Within each iteration it attempts to time a single loop, and then keeps only the best (fastest) loop it saw. This algorithm was found to have lower variance from run to run than my first attempt, which counted the total number of iterations that could be done in that fixed amount of jiffies. By taking only the best iteration in the loop, assuming at least one loop wasn't perturbed by an interrupt, we eliminate the effects of interrupts and other "warm up" factors like branch prediction. The only downside is it depends on having an rdtime granular and accurate enough to measure a single copy. If we ever manage to complete a loop in 0 rdtime ticks, we leave the unaligned setting at UNKNOWN.
There is a slight change in user-visible behavior here. Previously, all boards except the THead C906 reported misaligned access speed of UNKNOWN. C906 reported FAST. With this change, since we're now measuring misaligned access speed on each hart, all RISC-V systems will have this key set as either FAST or SLOW.
Currently, we don't have a way to confidently measure the difference between SLOW and EMULATED, so we label anything not fast as SLOW. This will mislabel some systems that are actually EMULATED as SLOW. When we get support for delegating misaligned access traps to the kernel (as opposed to the firmware quietly handling it), we can explicitly test in Linux to see if unaligned accesses trap. Those systems will start to report EMULATED, though older (today's) systems without that new SBI mechanism will continue to report SLOW.
Quoted 8 lines
I've updated the documentation for those hwprobe values to reflect this, specifically: SLOW may or may not be emulated by software, and FAST represents means being faster than equivalent byte accesses. The change in documentation is accurate with respect to both the former and current behavior.
Signed-off-by Evan Green <[email protected]>Acked-by Conor Dooley <[email protected]>Thanks for your patch, which is now commit
584ea6564bcaead2("RISC-V:
Probe for unaligned access speed") in v6.6-rc1.On the boards I have, I get:
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fast
Quoted 5 lines
Quoted 3 lines
Hrm, I'm a little surprised to be seeing this number come out so close to 1. If you reboot a few times, what kind of variance do you get on this?
Rock-solid at 1.05 (even with increased resolution: 1.05853 on 3 tries)
After upgrading the firmware from [1] to [2], this changed to "0.00, unaligned accesses are slow".
[1] RZ-Five-ETH
U-Boot 2020.10-g611c657e43 (Aug 26 2022 - 11:29:06 +0100)
[2] OpenSBI v1.3-75-g3cf0ea4
U-Boot 2023.01-00209-g1804c8ab17 (Oct 04 2023 - 13:18:01 +0100)
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Re: [PATCH v4 1/2] RISC-V: Probe for unaligned access speed
Hi Geert,
Quoted 64 lines · Geert Uytterhoeven
Hi Prabahkar,
Quoted 54 lines · Geert Uytterhoeven
Quoted 53 lines · Evan Green
Quoted 51 lines · Geert Uytterhoeven
Quoted 33 lines · Evan Green
Rather than deferring unaligned access speed determinations to a vendor function, let's probe them and find out how fast they are. If we determine that an unaligned word access is faster than N byte accesses, mark the hardware's unaligned access as "fast". Otherwise, we mark accesses as slow.
The algorithm itself runs for a fixed amount of jiffies. Within each iteration it attempts to time a single loop, and then keeps only the best (fastest) loop it saw. This algorithm was found to have lower variance from run to run than my first attempt, which counted the total number of iterations that could be done in that fixed amount of jiffies. By taking only the best iteration in the loop, assuming at least one loop wasn't perturbed by an interrupt, we eliminate the effects of interrupts and other "warm up" factors like branch prediction. The only downside is it depends on having an rdtime granular and accurate enough to measure a single copy. If we ever manage to complete a loop in 0 rdtime ticks, we leave the unaligned setting at UNKNOWN.
There is a slight change in user-visible behavior here. Previously, all boards except the THead C906 reported misaligned access speed of UNKNOWN. C906 reported FAST. With this change, since we're now measuring misaligned access speed on each hart, all RISC-V systems will have this key set as either FAST or SLOW.
Currently, we don't have a way to confidently measure the difference between SLOW and EMULATED, so we label anything not fast as SLOW. This will mislabel some systems that are actually EMULATED as SLOW. When we get support for delegating misaligned access traps to the kernel (as opposed to the firmware quietly handling it), we can explicitly test in Linux to see if unaligned accesses trap. Those systems will start to report EMULATED, though older (today's) systems without that new SBI mechanism will continue to report SLOW.
Quoted 8 lines
I've updated the documentation for those hwprobe values to reflect this, specifically: SLOW may or may not be emulated by software, and FAST represents means being faster than equivalent byte accesses. The change in documentation is accurate with respect to both the former and current behavior.
Signed-off-by Evan Green <[email protected]>Acked-by Conor Dooley <[email protected]>Thanks for your patch, which is now commit
584ea6564bcaead2("RISC-V:
Probe for unaligned access speed") in v6.6-rc1.On the boards I have, I get:
rzfive:
cpu0: Ratio of byte access time to unaligned word access is
1.05, unaligned accesses are fastQuoted 5 lines
Quoted 3 lines
Hrm, I'm a little surprised to be seeing this number come out so close to 1. If you reboot a few times, what kind of variance do you get on this?
Rock-solid at 1.05 (even with increased resolution: 1.05853 on 3 tries)
Quoted 9 lines
After upgrading the firmware from [1] to [2], this changed to "0.00, unaligned accesses are slow".
[1] RZ-Five-ETH
U-Boot 2020.10-g611c657e43 (Aug 26 2022 - 11:29:06 +0100)[2] OpenSBI v1.3-75-g3cf0ea4
U-Boot 2023.01-00209-g1804c8ab17 (Oct 04 2023 - 13:18:01 +0100)
Thanks, let me go through the changes.
Cheers,
Prabhakar
PATCH v4 2/2 RISC-V: alternative: Remove feature_probe_func
In reply to Evan GreenNow that we're testing unaligned memory copy and making that determination generically, there are no more users of the vendor feature_probe_func(). While I think it's probably going to need to come back, there are no users right now, so let's remove it until it's needed.
---
(no changes since v1)
arch/riscv/errata/thead/errata.c | 8 -------- arch/riscv/include/asm/alternative.h | 5 ----- arch/riscv/kernel/alternative.c | 19 ------------------- arch/riscv/kernel/smpboot.c | 1 - 4 files changed, 33 deletions(-)
arch/riscv/errata/thead/errata.c +0 -8
@@ -120,11 +120,3 @@ void thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,if (stage ==RISCV_ALTERNATIVES_EARLY_BOOT )local_flush_icache_all (); } - -void thead_feature_probe_func (unsigned int cpu, -unsigned long archid, -unsigned long impid) -{ -if ((archid ==0 ) && (impid ==0 )) -per_cpu (misaligned_access_speed, cpu) =RISCV_HWPROBE_MISALIGNED_FAST ; -}
arch/riscv/include/asm/alternative.h +0 -5
@@ -30,7 +30,6 @@#define ALT_OLD_PTR (a) __ALT_PTR(a, old_offset)#define ALT_ALT_PTR (a) __ALT_PTR(a, alt_offset) -void probe_vendor_features (unsigned int cpu);void __init apply_boot_alternatives (void );void __init apply_early_boot_alternatives (void );void apply_module_alternatives (void *start,size_t length);
@@ -53,15 +52,11 @@ void thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, unsigned long archid, unsigned long impid, unsigned int stage); -void thead_feature_probe_func (unsigned int cpu,unsigned long archid, -unsigned long impid); -void riscv_cpufeature_patch_func (struct alt_entry *begin,struct alt_entry *end,unsigned int stage);#else /* CONFIG_RISCV_ALTERNATIVE */ -static inline void probe_vendor_features (unsigned int cpu) { }static inline void apply_boot_alternatives (void ) { }static inline void apply_early_boot_alternatives (void ) { }static inline void apply_module_alternatives (void *start,size_t length) { }
arch/riscv/kernel/alternative.c +0 -19
@@ -27,8 +27,6 @@ struct cpu_manufacturer_info_t {void (*patch_func)(struct alt_entry *begin,struct alt_entry *end,unsigned long archid,unsigned long impid,unsigned int stage); -void (*feature_probe_func)(unsigned int cpu,unsigned long archid, -unsigned long impid); };static void riscv_fill_cpu_mfr_info (struct cpu_manufacturer_info_t *cpu_mfr_info)
@@ -43,7 +41,6 @@ static void riscv_fill_cpu_mfr_info(struct cpu_manufacturer_info_t *cpu_mfr_info cpu_mfr_info->imp_id =sbi_get_mimpid ();#endif - cpu_mfr_info->feature_probe_func =NULL ;switch (cpu_mfr_info->vendor_id) {#ifdef CONFIG_ERRATA_SIFIVE case SIFIVE_VENDOR_ID :
@@ -53,7 +50,6 @@ static void riscv_fill_cpu_mfr_info(struct cpu_manufacturer_info_t *cpu_mfr_info#ifdef CONFIG_ERRATA_THEAD case THEAD_VENDOR_ID : cpu_mfr_info->patch_func = thead_errata_patch_func; - cpu_mfr_info->feature_probe_func = thead_feature_probe_func;break ;#endif default :
@@ -143,20 +139,6 @@ void riscv_alternative_fix_offsets(void *alt_ptr, unsigned int len, } } -/* Called on each CPU as it starts */ -void probe_vendor_features (unsigned int cpu) -{ -struct cpu_manufacturer_info_t cpu_mfr_info; - -riscv_fill_cpu_mfr_info (&cpu_mfr_info); -if (!cpu_mfr_info.feature_probe_func) -return ; - - cpu_mfr_info.feature_probe_func (cpu, - cpu_mfr_info.arch_id, - cpu_mfr_info.imp_id); -} -/* * This is called very early in the boot process (directly after we run * a feature detect on the boot CPU). No need to worry about other CPUs
@@ -211,7 +193,6 @@ void __init apply_boot_alternatives(void)/* If called on non-boot cpu things could go wrong */ WARN_ON (smp_processor_id () !=0 ); -probe_vendor_features (0 );_apply_alternatives ((struct alt_entry *)__alt_start, (struct alt_entry *)__alt_end,RISCV_ALTERNATIVES_BOOT );
arch/riscv/kernel/smpboot.c +0 -1
@@ -247,7 +247,6 @@ asmlinkage __visible void smp_callin(void)numa_add_cpu (curr_cpuid);set_cpu_online (curr_cpuid,1 );check_unaligned_access (curr_cpuid); -probe_vendor_features (curr_cpuid);if (has_vector ()) {if (riscv_v_setup_vsize ())
Signature · 7 lines
2.34.1 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt <[email protected]>:
Hello:
This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt <[email protected]>:
Quoted 6 lines · On Fri, 18 Aug 2023 12:41:34 -0700 you wrote:
The current setting for the hwprobe bit indicating misaligned access speed is controlled by a vendor-specific feature probe function. This is essentially a per-SoC table we have to maintain on behalf of each vendor going forward. Let's convert that instead to something we detect at runtime.
Quoted 8 lines
We have two assembly routines at the heart of our probe: one that does a bunch of word-sized accesses (without aligning its input buffer), and the other that does byte accesses. If we can move a larger number of bytes using misaligned word accesses than we can with the same amount of time doing byte accesses, then we can declare misaligned accesses as "fast".
[...]
Here is the summary with links:
- [v4,1/2] RISC-V: Probe for unaligned access speed
https://git.kernel.org/riscv/c/b98673c5b037 - [v4,2/2] RISC-V: alternative: Remove feature_probe_func https://git.kernel.org/riscv/c/b6e3f6e009a1
You are awesome, thank you!
Signature · 9 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv