omap v4l2: fix wrong resizer calculation

PATCH Superseded Vladimir Pantelic 2 participants 3 messages Apr 26, 2011 – Jun 15, 2011 Media (V4L/DVB)

lore

Share & tools

lore

Vladimir Pantelic

author

2011-04-26 07:36 UTC #0 lore raw

PATCH 1/1 omap v4l2: fix wrong resizer calculation

Quoted 1 line

From 47473f26bda2ef2e3a5e8f0203fde0c74c412c36 Mon Sep 17 00:00:00 2001

From: Vladimir Pantelic <[email protected]>
Date: Tue, 26 Apr 2011 09:28:11 +0200
Subject: [PATCH] omap v4l2: fix wrong resizer calculation

in omap_vout_new_crop(), "crop" is the output parameter and therefore new_crop/try_crop should be used to calculate the resizer values

Signed-off-by Vladimir Pantelic <[email protected]>

---
 drivers/media/video/omap/omap_voutlib.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

drivers/media/video/omap/omap_voutlib.c +3 -3

index 2aa6a76..8ae7481 100644
@@ -193,7 +193,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 		return -EINVAL;
 
 	if (cpu_is_omap24xx()) {
-		if (crop->height != win->w.height) {
+		if (try_crop.height != win->w.height) {
 			/* If we're resizing vertically, we can't support a
 			 * crop width wider than 768 pixels.
 			 */

@@ -202,7 +202,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 		}
 	}
 	/* vertical resizing */
-	vresize = (1024 * crop->height) / win->w.height;
+	vresize = (1024 * try_crop.height) / win->w.height;
 	if (cpu_is_omap24xx() && (vresize > 2048))
 		vresize = 2048;
 	else if (cpu_is_omap34xx() && (vresize > 4096))

@@ -221,7 +221,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 			try_crop.height = 2;
 	}
 	/* horizontal resizing */
-	hresize = (1024 * crop->width) / win->w.width;
+	hresize = (1024 * try_crop.width) / win->w.width;
 	if (cpu_is_omap24xx() && (hresize > 2048))
 		hresize = 2048;
 	else if (cpu_is_omap34xx() && (hresize > 4096))

Signature · 1 line

1.6.0.2

Vladimir Pantelic

author

2011-06-15 10:16 UTC #1 lore raw

ping

Quoted 1 line

From 47473f26bda2ef2e3a5e8f0203fde0c74c412c36 Mon Sep 17 00:00:00 2001

From: Vladimir Pantelic <[email protected]>
Date: Tue, 26 Apr 2011 09:28:11 +0200
Subject: [PATCH] omap v4l2: fix wrong resizer calculation

in omap_vout_new_crop(), "crop" is the output parameter and therefore new_crop/try_crop should be used to calculate the resizer values

Signed-off-by Vladimir Pantelic <[email protected]>

---
 drivers/media/video/omap/omap_voutlib.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

drivers/media/video/omap/omap_voutlib.c +3 -3

index 2aa6a76..8ae7481 100644
@@ -193,7 +193,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 		return -EINVAL;
 
 	if (cpu_is_omap24xx()) {
-		if (crop->height != win->w.height) {
+		if (try_crop.height != win->w.height) {
 			/* If we're resizing vertically, we can't support a
 			 * crop width wider than 768 pixels.
 			 */

@@ -202,7 +202,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 		}
 	}
 	/* vertical resizing */
-	vresize = (1024 * crop->height) / win->w.height;
+	vresize = (1024 * try_crop.height) / win->w.height;
 	if (cpu_is_omap24xx() && (vresize > 2048))
 		vresize = 2048;
 	else if (cpu_is_omap34xx() && (vresize > 4096))

@@ -221,7 +221,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 			try_crop.height = 2;
 	}
 	/* horizontal resizing */
-	hresize = (1024 * crop->width) / win->w.width;
+	hresize = (1024 * try_crop.width) / win->w.width;
 	if (cpu_is_omap24xx() && (hresize > 2048))
 		hresize = 2048;
 	else if (cpu_is_omap34xx() && (hresize > 4096))

Signature · 1 line

1.6.0.2

Hiremath, Vaibhav

2011-06-15 14:36 UTC #2 lore raw

Quoted 8 lines

-----Original Message-----
From: Vladimir Pantelic [mailto:[email protected]]
Sent: Wednesday, June 15, 2011 3:46 PM
To: Kernel development list
Cc: [email protected]; Hiremath, Vaibhav
Subject: Re: [PATCH] omap v4l2: fix wrong resizer calculation
ping

[Hiremath, Vaibhav] Sorry for delayed response, somehow I completely missed this patch.

Thanks for finding this bug, the patch looks ok to me. Only one suggestion I have is to update the description of patch commit, request you to change like following -

----
The omap_vout_new_crop() function has possible bug, uses uninitialized variable "crop.width/height" which is actually output of the function. Instead we should be using "try_crop.width/height" to calculate the resizer value.
----

Rest everything looks ok, while submitting patch again you can add my ack; I will queue to my Arago repo.

Also I have tested this patch on OMAP3EVM platform, and it seems to be working fine.

Thanks,
Vaibhav