simple framebuffer slower by factor of 20, on socfpga (arm) platform

I have an socfpga board, which uses has simple framebuffer implemented in the FPGA. On 3.15, framebuffer is fast:

root@wagabuibui:~# time cat /dev/fb0 > /dev/null
real               0m 0.00s
user               0m 0.00s
sys                0m 0.00s

on 3.18, this takes 220msec. Similar slowdown exists for writes. Simple framebuffer did not change at all between 3.15 and 3.18; resource flags of the framebuffer are still same (0x200).

If I enable caching on 3.18, it speeds up a bit, to 70msec or so... Which means problem is not only in caching.

Any ideas?

Hi Pavel,

Quoted 15 lines · Pavel Machek

I have an socfpga board, which uses has simple framebuffer implemented in the FPGA. On 3.15, framebuffer is fast:

root@wagabuibui:~# time cat /dev/fb0 > /dev/null
real               0m 0.00s
user               0m 0.00s
sys                0m 0.00s

on 3.18, this takes 220msec. Similar slowdown exists for writes. Simple framebuffer did not change at all between 3.15 and 3.18; resource flags of the framebuffer are still same (0x200).

If I enable caching on 3.18, it speeds up a bit, to 70msec or so... Which means problem is not only in caching.

Quoted 1 line

Any ideas?

My first guess was commit 67dc0d4758e5 ("vt_buffer: drop console buffer copying optimisations"), but this was introduced only in v4.0-rc1.

Just in case you encounter another performance regression after upgrading to a more modern kernel ;-)

Hi Pavel,

Quoted 15 lines · Pavel Machek

I have an socfpga board, which uses has simple framebuffer implemented in the FPGA. On 3.15, framebuffer is fast:

root@wagabuibui:~# time cat /dev/fb0 > /dev/null
real               0m 0.00s
user               0m 0.00s
sys                0m 0.00s

on 3.18, this takes 220msec. Similar slowdown exists for writes. Simple framebuffer did not change at all between 3.15 and 3.18; resource flags of the framebuffer are still same (0x200).

If I enable caching on 3.18, it speeds up a bit, to 70msec or so... Which means problem is not only in caching.

Quoted 1 line

Any ideas?

My first guess was commit 67dc0d4758e5 ("vt_buffer: drop console buffer copying optimisations"), but this was introduced only in v4.0-rc1.

Just in case you encounter another performance regression after upgrading to a more modern kernel ;-)

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Quoted 20 lines · Geert Uytterhoeven

Hi Pavel,

Quoted 16 lines · Pavel Machek

I have an socfpga board, which uses has simple framebuffer implemented in the FPGA. On 3.15, framebuffer is fast:

root@wagabuibui:~# time cat /dev/fb0 > /dev/null
real               0m 0.00s
user               0m 0.00s
sys                0m 0.00s

on 3.18, this takes 220msec. Similar slowdown exists for writes. Simple framebuffer did not change at all between 3.15 and 3.18; resource flags of the framebuffer are still same (0x200).

If I enable caching on 3.18, it speeds up a bit, to 70msec or so... Which means problem is not only in caching.

Any ideas?

Quoted 5 lines

My first guess was commit 67dc0d4758e5 ("vt_buffer: drop console buffer copying optimisations"), but this was introduced only in v4.0-rc1.

Just in case you encounter another performance regression after upgrading to a more modern kernel ;-)

:-). I did a git bisect, and it pointed to this. And reverting it indeed fixes the problem in 3.18. Problem is still there in 4.0.

Archit do you know what is going on there? Should the revert be filled for 4.0?

Pavel

commit 981409b25e2a99409b26daa67293ca1cfd5ea0a0

Author: Archit Taneja <[email protected]>
Date:   Fri Nov 16 14:46:04 2012 +0530

fbdev: arm has __raw I/O accessors, use them in fb.h

This removes the sparse warnings on arm platforms:

warning: cast removes address space of expression

Signed-off-by Archit Taneja <[email protected]>

Signed-off-by Tomi Valkeinen <[email protected]>

Cc H Hartley Sweeten <hsweeten at visionengravers.com>

Cc Alexander Shiyan <[email protected]>

Cc Russell King <[email protected]>

Quoted 21 lines · Pavel Machek

Quoted 20 lines · Geert Uytterhoeven

Hi Pavel,

Quoted 16 lines · Pavel Machek

I have an socfpga board, which uses has simple framebuffer implemented in the FPGA. On 3.15, framebuffer is fast:

root@wagabuibui:~# time cat /dev/fb0 > /dev/null
real               0m 0.00s
user               0m 0.00s
sys                0m 0.00s

on 3.18, this takes 220msec. Similar slowdown exists for writes. Simple framebuffer did not change at all between 3.15 and 3.18; resource flags of the framebuffer are still same (0x200).

If I enable caching on 3.18, it speeds up a bit, to 70msec or so... Which means problem is not only in caching.

Any ideas?

Quoted 8 lines

Quoted 5 lines

My first guess was commit 67dc0d4758e5 ("vt_buffer: drop console buffer copying optimisations"), but this was introduced only in v4.0-rc1.

Just in case you encounter another performance regression after upgrading to a more modern kernel ;-)

:-). I did a git bisect, and it pointed to this. And reverting it indeed fixes the problem in 3.18. Problem is still there in 4.0.

Quoted 22 lines · Tomi Valkeinen

Quoted 21 lines · Pavel Machek

Quoted 20 lines · Geert Uytterhoeven

Hi Pavel,

Quoted 16 lines · Pavel Machek

I have an socfpga board, which uses has simple framebuffer implemented in the FPGA. On 3.15, framebuffer is fast:

root@wagabuibui:~# time cat /dev/fb0 > /dev/null
real               0m 0.00s
user               0m 0.00s
sys                0m 0.00s

on 3.18, this takes 220msec. Similar slowdown exists for writes. Simple framebuffer did not change at all between 3.15 and 3.18; resource flags of the framebuffer are still same (0x200).

If I enable caching on 3.18, it speeds up a bit, to 70msec or so... Which means problem is not only in caching.

Any ideas?

Quoted 8 lines

Quoted 8 lines

Quoted 5 lines

My first guess was commit 67dc0d4758e5 ("vt_buffer: drop console buffer copying optimisations"), but this was introduced only in v4.0-rc1.

Just in case you encounter another performance regression after upgrading to a more modern kernel

:-). I did a git bisect, and it pointed to this. And reverting it indeed fixes the problem in 3.18. Problem is still there in 4.0.

The difference is probably caused by memcpy() vs memcpy_fromio(). The comment above memcpy_fromio() says "This needs to be optimized". I think generally speaking memcpy_fromio() is correct for a framebuffer.

That said, if the fb is in RAM, and is only written by the CPU, I think a normal memcpy() for fb_memcpy_fromfb() should be fine...

Quoted 7 lines

The difference is probably caused by memcpy() vs memcpy_fromio(). The comment above memcpy_fromio() says "This needs to be optimized". I think generally speaking memcpy_fromio() is correct for a framebuffer.

That said, if the fb is in RAM, and is only written by the CPU, I think a normal memcpy() for fb_memcpy_fromfb() should be fine...

Could memcpy() cause alignment traps here if the fb pointer is unaligned and uncached?

Quoted 22 lines · Tomi Valkeinen

Quoted 21 lines · Pavel Machek

Quoted 20 lines · Geert Uytterhoeven

Hi Pavel,

Quoted 16 lines · Pavel Machek

I have an socfpga board, which uses has simple framebuffer implemented in the FPGA. On 3.15, framebuffer is fast:

root@wagabuibui:~# time cat /dev/fb0 > /dev/null
real               0m 0.00s
user               0m 0.00s
sys                0m 0.00s

on 3.18, this takes 220msec. Similar slowdown exists for writes. Simple framebuffer did not change at all between 3.15 and 3.18; resource flags of the framebuffer are still same (0x200).

If I enable caching on 3.18, it speeds up a bit, to 70msec or so... Which means problem is not only in caching.

Any ideas?

Quoted 8 lines

Quoted 8 lines

Quoted 5 lines

My first guess was commit 67dc0d4758e5 ("vt_buffer: drop console buffer copying optimisations"), but this was introduced only in v4.0-rc1.

Just in case you encounter another performance regression after upgrading to a more modern kernel ;-)

:-). I did a git bisect, and it pointed to this. And reverting it indeed fixes the problem in 3.18. Problem is still there in 4.0.

The difference is probably caused by memcpy() vs memcpy_fromio(). The comment above memcpy_fromio() says "This needs to be optimized". I think generally speaking memcpy_fromio() is correct for a framebuffer.

That said, if the fb is in RAM, and is only written by the CPU, I think a normal memcpy() for fb_memcpy_fromfb() should be fine...

Quoted 2 lines

That said, if the fb is in RAM, and is only written by the CPU, I think a normal memcpy() for fb_memcpy_fromfb() should be fine...

I didn't test for performance regressions when I posted this patch.

A look at _memcpy_fromio in arch/arm/kernel/io.c shows that readb() is used all the time, even when the source and destination addresses are aligned for larger reads to be possible. Other archs seem to use readl() or readq() when they can. Maybe that makes memcpy_fromio slower than the implementation of memcpy on arm?

Hi!

Quoted 5 lines · Archit Taneja

Quoted 2 lines

That said, if the fb is in RAM, and is only written by the CPU, I think a normal memcpy() for fb_memcpy_fromfb() should be fine...

I didn't test for performance regressions when I posted this patch.

Quoted 5 lines

A look at _memcpy_fromio in arch/arm/kernel/io.c shows that readb() is used all the time, even when the source and destination addresses are aligned for larger reads to be possible. Other archs seem to use readl() or readq() when they can. Maybe that makes memcpy_fromio slower than the implementation of memcpy on arm?

Ok, can you prepare a patch for me to try? Or should we just revert the original commit?

Quoted 12 lines · Pavel Machek

Quoted 10 lines · Archit Taneja

Quoted 2 lines

That said, if the fb is in RAM, and is only written by the CPU, I think a normal memcpy() for fb_memcpy_fromfb() should be fine...

I didn't test for performance regressions when I posted this patch.

A look at _memcpy_fromio in arch/arm/kernel/io.c shows that readb() is used all the time, even when the source and destination addresses are aligned for larger reads to be possible. Other archs seem to use readl() or readq() when they can. Maybe that makes memcpy_fromio slower than the implementation of memcpy on arm?

Quoted 2 lines

Ok, can you prepare a patch for me to try? Or should we just revert the original commit?

The old way worked fine, afaik, so maybe we can revert. But still, isn't it more correct to use memcpy_fromio? It's (possibly) io memory we have here.

Quoted 16 lines · Tomi Valkeinen

Quoted 14 lines · Pavel Machek

Quoted 10 lines · Archit Taneja

Quoted 2 lines

That said, if the fb is in RAM, and is only written by the CPU, I think a normal memcpy() for fb_memcpy_fromfb() should be fine...

I didn't test for performance regressions when I posted this patch.

A look at _memcpy_fromio in arch/arm/kernel/io.c shows that readb() is used all the time, even when the source and destination addresses are aligned for larger reads to be possible. Other archs seem to use readl() or readq() when they can. Maybe that makes memcpy_fromio slower than the implementation of memcpy on arm?

Ok, can you prepare a patch for me to try? Or should we just revert the original commit?

Quoted 3 lines

The old way worked fine, afaik, so maybe we can revert. But still, isn't it more correct to use memcpy_fromio? It's (possibly) io memory we have here.

Yes it is.

So please optimize ARM's _memcpy_fromio(), _memcpy_toio(), and _memset_io(). That will benefit other drivers on ARM, too.

Quoted 16 lines · Tomi Valkeinen

Quoted 14 lines · Pavel Machek

Quoted 10 lines · Archit Taneja

Quoted 2 lines

That said, if the fb is in RAM, and is only written by the CPU, I think a normal memcpy() for fb_memcpy_fromfb() should be fine...

I didn't test for performance regressions when I posted this patch.

A look at _memcpy_fromio in arch/arm/kernel/io.c shows that readb() is used all the time, even when the source and destination addresses are aligned for larger reads to be possible. Other archs seem to use readl() or readq() when they can. Maybe that makes memcpy_fromio slower than the implementation of memcpy on arm?

Ok, can you prepare a patch for me to try? Or should we just revert the original commit?

Quoted 3 lines

The old way worked fine, afaik, so maybe we can revert. But still, isn't it more correct to use memcpy_fromio? It's (possibly) io memory we have here.

Yes it is.

So please optimize ARM's _memcpy_fromio(), _memcpy_toio(), and _memset_io(). That will benefit other drivers on ARM, too.

Quoted 2 lines · Geert Uytterhoeven

So please optimize ARM's _memcpy_fromio(), _memcpy_toio(), and _memset_io(). That will benefit other drivers on ARM, too.

That's not going to happen.

I've had a patch which does that, but people are concerned that it changes the behaviour of the functions by changing the access size, which could cause regressions. It seems people are far too worried about that to even consider trying. :(

Quoted 4 lines · Russell King - ARM Linux

Quoted 2 lines · Geert Uytterhoeven

So please optimize ARM's _memcpy_fromio(), _memcpy_toio(), and _memset_io(). That will benefit other drivers on ARM, too.

Quoted 6 lines

That's not going to happen.

I've had a patch which does that, but people are concerned that it changes the behaviour of the functions by changing the access size, which could cause regressions. It seems people are far too worried about that to even consider trying. :(

What about making the optimized implementation available via kconfig?

Quoted 6 lines · Nicolas Pitre

Quoted 4 lines · Russell King - ARM Linux

Quoted 2 lines · Geert Uytterhoeven

So please optimize ARM's _memcpy_fromio(), _memcpy_toio(), and _memset_io(). That will benefit other drivers on ARM, too.

Quoted 8 lines

Quoted 6 lines

That's not going to happen.

I've had a patch which does that, but people are concerned that it changes the behaviour of the functions by changing the access size, which could cause regressions. It seems people are far too worried about that to even consider trying. :(

What about making the optimized implementation available via kconfig?

I'd prefer not to. My personal feeling is to put the patch in and just be done with it - these functions are supposed to be used on IO areas which don't care about access size (in other words, are memory-like rather than being register-like.) Here's the rather old patch:

From: Russell King <[email protected]>
Subject: [PATCH] ARM: optimize memset_io()/memcpy_fromio()/memcpy_toio()

If we are building for a LE platform, and we haven't overriden the MMIO ops, then we can optimize the mem*io operations using the standard string functions.

Signed-off-by Russell King <[email protected]>

---
 arch/arm/include/asm/io.h | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

arch/arm/include/asm/io.h +25 -0

index d070741b2b37..358c8206419b 100644
@@ -23,6 +23,7 @@
 
 #ifdef __KERNEL__
 
+#include <linux/string.h>
 #include <linux/types.h>
 #include <asm/byteorder.h>
 #include <asm/memory.h>

@@ -312,9 +313,33 @@ extern void _memset_io(volatile void __iomem *, int, size_t);
 #define writesw(p,d,l)		__raw_writesw(p,d,l)
 #define writesl(p,d,l)		__raw_writesl(p,d,l)
 
+#ifndef __ARMBE__
+static inline void memset_io(volatile void __iomem *dst, unsigned c,
+	size_t count)
+{
+	memset((void __force *)dst, c, count);
+}
+#define memset_io(dst,c,count) memset_io(dst,c,count)
+
+static inline void memcpy_fromio(void *to, const volatile void __iomem *from,
+	size_t count)
+{
+	memcpy(to, (const void __force *)from, count);
+}
+#define memcpy_fromio(to,from,count) memcpy_fromio(to,from,count)
+
+static inline void memcpy_toio(volatile void __iomem *to, const void *from,
+	size_t count)
+{
+	memcpy((void __force *)to, from, count);
+}
+#define memcpy_toio(to,from,count) memcpy_toio(to,from,count)
+
+#else
 #define memset_io(c,v,l)	_memset_io(c,(v),(l))
 #define memcpy_fromio(a,c,l)	_memcpy_fromio((a),c,(l))
 #define memcpy_toio(c,a,l)	_memcpy_toio(c,(a),(l))
+#endif
 
 #endif	/* readl */
 

Signature · 7 lines

1.8.3.1



-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

Quoted 6 lines · Nicolas Pitre

Quoted 4 lines · Russell King - ARM Linux

Quoted 2 lines · Geert Uytterhoeven

So please optimize ARM's _memcpy_fromio(), _memcpy_toio(), and _memset_io(). That will benefit other drivers on ARM, too.

Quoted 8 lines

Quoted 6 lines

That's not going to happen.

I've had a patch which does that, but people are concerned that it changes the behaviour of the functions by changing the access size, which could cause regressions. It seems people are far too worried about that to even consider trying. :(

What about making the optimized implementation available via kconfig?

I'd prefer not to. My personal feeling is to put the patch in and just be done with it - these functions are supposed to be used on IO areas which don't care about access size (in other words, are memory-like rather than being register-like.) Here's the rather old patch:

From: Russell King <[email protected]>
Subject: [PATCH] ARM: optimize memset_io()/memcpy_fromio()/memcpy_toio()

If we are building for a LE platform, and we haven't overriden the MMIO ops, then we can optimize the mem*io operations using the standard string functions.

Signed-off-by Russell King <[email protected]>