Skip to contentSkip to conversation

x86: simplefb: avoid overflow

PATCH Superseded Tom Gundersen 4 participants 7 messages Sep 6, 2013 x86

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86: simplefb: avoid overflow

lfb_size can easily be say 4M, which would make the bitshit overflow and the test fail.

Cc David Herrmann <[email protected]>
Cc H. Peter Anvin <[email protected]>
---
 arch/x86/kernel/sysfb_simplefb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/sysfb_simplefb.c +1 -1
index 22513e9..fff44a5 100644
@@ -72,7 +72,7 @@ __init int create_simplefb(const struct screen_info *si,
 	 * the part that is occupied by the framebuffer */
 	len = mode->height * mode->stride;
 	len = PAGE_ALIGN(len);
-	if (len > si->lfb_size << 16) {
+	if (len > ((unsigned long) si->lfb_size) << 16) {
 		printk(KERN_WARNING "sysfb: VRAM smaller than advertised\n");
 		return -EINVAL;
 	}
Signature · 1 line
1.8.4
Reviewed-by on v1 1/1
#1 lore raw

Hi

Quoted 15 lines · Tom Gundersen

lfb_size can easily be say 4M, which would make the bitshit overflow and the test fail.

Cc David Herrmann <[email protected]>
Cc H. Peter Anvin <[email protected]>
---
 arch/x86/kernel/sysfb_simplefb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/sysfb_simplefb.c +0 -0
index 22513e9..fff44a5 100644

arch/x86/kernel/sysfb_simplefb.c __init int create_simplefb(const struct screen_info *si, · line 75

Quoted 5 lines
arch/x86/kernel/sysfb_simplefb.c +1 -1
@@ -72,7 +72,7 @@ __init int create_simplefb(const struct screen_info *si,
         * the part that is occupied by the framebuffer */
        len = mode->height * mode->stride;
        len = PAGE_ALIGN(len);
-       if (len > si->lfb_size << 16) {
+       if (len > ((unsigned long) si->lfb_size) << 16) {

Nice catch. vesafb uses "lfb_size * 65535" which causes an implicit cast. I thought <<16 looks nicer but that doesn't do any implicit cast..

Nothing crucial, as it only causes the simple-fb conversion to fail. But would still be good to see in -rc2:

Thanks
David

Quoted 6 lines
arch/x86/kernel/sysfb_simplefb.c +0 -1
                printk(KERN_WARNING "sysfb: VRAM smaller than advertised\n");
                return -EINVAL;
        }
--

1.8.4

#2 lore raw
Quoted 11 lines · David Herrmann
Quoted 10 lines · Tom Gundersen

lfb_size can easily be say 4M, which would make the bitshit overflow and the test fail.

Cc David Herrmann <[email protected]>
Cc H. Peter Anvin <[email protected]>
---
 arch/x86/kernel/sysfb_simplefb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 10 lines

arch/x86/kernel/sysfb_simplefb.c __init int create_simplefb(const struct screen_info *si, · line 75

Quoted 10 lines
arch/x86/kernel/sysfb_simplefb.c +1 -1
index 22513e9..fff44a5 100644
@@ -72,7 +72,7 @@ __init int create_simplefb(const struct screen_info *si,
         * the part that is occupied by the framebuffer */
        len = mode->height * mode->stride;
        len = PAGE_ALIGN(len);
-       if (len > si->lfb_size << 16) {
+       if (len > ((unsigned long) si->lfb_size) << 16) {

On 32-bit, "unsigned long" is the same size as __u32, so this doesn't make any difference.

Quoted 3 lines

Nice catch. vesafb uses "lfb_size * 65535" which causes an implicit cast. I thought <<16 looks nicer but that doesn't do any implicit cast..

"lfb_size * 65535" is the same. "lfb_size" is __u32, "65535" is int. So there's no implicit cast. Or am I missing something?

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#3 lore raw
Quoted 26 lines · Geert Uytterhoeven
Quoted 11 lines · David Herrmann
Quoted 10 lines · Tom Gundersen

lfb_size can easily be say 4M, which would make the bitshit overflow and the test fail.

Cc David Herrmann <[email protected]>
Cc H. Peter Anvin <[email protected]>
---
  arch/x86/kernel/sysfb_simplefb.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 10 lines

arch/x86/kernel/sysfb_simplefb.c __init int create_simplefb(const struct screen_info *si, · line 75

Quoted 10 lines
arch/x86/kernel/sysfb_simplefb.c +1 -1
index 22513e9..fff44a5 100644
@@ -72,7 +72,7 @@ __init int create_simplefb(const struct screen_info *si,
          * the part that is occupied by the framebuffer */
         len = mode->height * mode->stride;
         len = PAGE_ALIGN(len);
-       if (len > si->lfb_size << 16) {
+       if (len > ((unsigned long) si->lfb_size) << 16) {

On 32-bit, "unsigned long" is the same size as __u32, so this doesn't make any difference.

Quoted 7 lines
Quoted 3 lines

Nice catch. vesafb uses "lfb_size * 65535" which causes an implicit cast. I thought <<16 looks nicer but that doesn't do any implicit cast..

"lfb_size * 65535" is the same. "lfb_size" is __u32, "65535" is int. So there's no implicit cast. Or am I missing something?

<< 16 is * 65536 not 65535...

-hpa

#4 lore raw
Quoted 36 lines · H. Peter Anvin
Quoted 35 lines · Geert Uytterhoeven
Quoted 11 lines · David Herrmann
Quoted 10 lines · Tom Gundersen

lfb_size can easily be say 4M, which would make the bitshit overflow and the test fail.

Cc David Herrmann <[email protected]>
Cc H. Peter Anvin <[email protected]>
---
  arch/x86/kernel/sysfb_simplefb.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 12 lines

arch/x86/kernel/sysfb_simplefb.c __init int create_simplefb(const struct screen_info · line 76

Quoted 12 lines
arch/x86/kernel/sysfb_simplefb.c +1 -1
index 22513e9..fff44a5 100644
@@ -72,7 +72,7 @@ __init int create_simplefb(const struct screen_info
 *si,
          * the part that is occupied by the framebuffer */
         len = mode->height * mode->stride;
         len = PAGE_ALIGN(len);
-       if (len > si->lfb_size << 16) {
+       if (len > ((unsigned long) si->lfb_size) << 16) {

On 32-bit, "unsigned long" is the same size as __u32, so this doesn't make any difference.

Quoted 3 lines

Nice catch. vesafb uses "lfb_size * 65535" which causes an implicit cast. I thought <<16 looks nicer but that doesn't do any implicit cast..

Quoted 5 lines
Quoted 3 lines

"lfb_size * 65535" is the same. "lfb_size" is __u32, "65535" is int. So there's no implicit cast. Or am I missing something?

<< 16 is * 65536 not 65535...

Oops, you're right. So that overflows a little bit later.
Still, that doesn't change any reasoning w.r.t implicit casts and overflows.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#5 lore raw
In reply to Geert Uytterhoeven

Hi

Quoted 12 lines · Geert Uytterhoeven
Quoted 11 lines · David Herrmann
Quoted 10 lines · Tom Gundersen

lfb_size can easily be say 4M, which would make the bitshit overflow and the test fail.

Cc David Herrmann <[email protected]>
Cc H. Peter Anvin <[email protected]>
---
 arch/x86/kernel/sysfb_simplefb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 13 lines
Quoted 10 lines

arch/x86/kernel/sysfb_simplefb.c __init int create_simplefb(const struct screen_info *si, · line 75

Quoted 10 lines
arch/x86/kernel/sysfb_simplefb.c +1 -1
index 22513e9..fff44a5 100644
@@ -72,7 +72,7 @@ __init int create_simplefb(const struct screen_info *si,
         * the part that is occupied by the framebuffer */
        len = mode->height * mode->stride;
        len = PAGE_ALIGN(len);
-       if (len > si->lfb_size << 16) {
+       if (len > ((unsigned long) si->lfb_size) << 16) {

On 32-bit, "unsigned long" is the same size as __u32, so this doesn't make any difference.

lfb_size cannot be 4M on 32bit machines. Well, if it is, the firmware passed bogus information as you cannot have a 262G-region on 32bit. But on 64bit it can pass 4M just fine.
So we don't care for the 32bit case here, only 64bit. The (__u64) cast would be more obvious. Don't know.. It's just a sanity check, anyways, so I'm fine with it.

Quoted 6 lines
Quoted 3 lines

Nice catch. vesafb uses "lfb_size * 65535" which causes an implicit cast. I thought <<16 looks nicer but that doesn't do any implicit cast..

"lfb_size * 65535" is the same. "lfb_size" is __u32, "65535" is int. So there's no implicit cast. Or am I missing something?

Yepp, indeed. My bad, so vesafb doesn't do any better here.

I wonder, though, which firmware passes such values. It means the lfb_size area is reserved >4G. We do have huge VMEM these days.. Tom, on what hardware did you hit that?

Thanks
David

#6 lore raw
Quoted 30 lines · David Herrmann

Hi

Quoted 12 lines · Geert Uytterhoeven
Quoted 11 lines · David Herrmann
Quoted 10 lines · Tom Gundersen

lfb_size can easily be say 4M, which would make the bitshit overflow and the test fail.

Cc David Herrmann <[email protected]>
Cc H. Peter Anvin <[email protected]>
---
 arch/x86/kernel/sysfb_simplefb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 13 lines
Quoted 10 lines

arch/x86/kernel/sysfb_simplefb.c __init int create_simplefb(const struct screen_info *si, · line 75

Quoted 10 lines
arch/x86/kernel/sysfb_simplefb.c +1 -1
index 22513e9..fff44a5 100644
@@ -72,7 +72,7 @@ __init int create_simplefb(const struct screen_info *si,
         * the part that is occupied by the framebuffer */
        len = mode->height * mode->stride;
        len = PAGE_ALIGN(len);
-       if (len > si->lfb_size << 16) {
+       if (len > ((unsigned long) si->lfb_size) << 16) {

On 32-bit, "unsigned long" is the same size as __u32, so this doesn't make any difference.

Quoted 6 lines

lfb_size cannot be 4M on 32bit machines. Well, if it is, the firmware passed bogus information as you cannot have a 262G-region on 32bit. But on 64bit it can pass 4M just fine.
So we don't care for the 32bit case here, only 64bit. The (__u64) cast would be more obvious. Don't know.. It's just a sanity check, anyways, so I'm fine with it.

I used unsigned long simply to match the type of len...

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Nice catch. vesafb uses "lfb_size * 65535" which causes an implicit cast. I thought <<16 looks nicer but that doesn't do any implicit cast..

Quoted 8 lines
Quoted 2 lines

"lfb_size * 65535" is the same. "lfb_size" is __u32, "65535" is int. So there's no implicit cast. Or am I missing something?

Yepp, indeed. My bad, so vesafb doesn't do any better here.

I wonder, though, which firmware passes such values. It means the lfb_size area is reserved >4G. We do have huge VMEM these days.. Tom, on what hardware did you hit that?

As always when weird things happen:

DMI: Apple Inc. MacBookAir5,1/Mac-66F35F19FE2A0D05, BIOS MBA51.88Z.00EF.B01.1207271122 07/27/2012

Cheers,

Tom