Skip to contentSkip to conversation

lib: libmtd: fix sign extension of ioctl request in mtd_xlock

PATCH Applied Bastian Germann 2 participants 2 messages Apr 27 – Sep 7

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 lib: libmtd: fix sign extension of ioctl request in mtd_xlock

The req parameter of mtd_xlock() was declared as int. On 64-bit big-endian architectures such as ppc64el, _IOW()-derived ioctl numbers have bit 31 set (because _IOC_WRITE=4 is placed at bit 29). Storing such a value in a signed int and then passing it to ioctl(), whose second argument is unsigned long, causes implicit sign extension:
0x80084d05 becomes 0xffffffff80084d05, which does not match the expected constant and fails the cmocka check_expected() assertion in the unit tests for mtd_lock and mtd_unlock.

Fix by declaring req as unsigned long, matching the type used by the ioctl() syscall interface.

Fixes test_mtd_lock / test_mtd_unlock failures on ppc64el
---
 lib/libmtd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/libmtd.c +1 -1
index f588e09..96a2167 100644
@@ -902,7 +902,7 @@ static int mtd_valid_erase_block(const struct mtd_dev_info *mtd, int eb)
 }
 
 static int mtd_xlock(const struct mtd_dev_info *mtd, int fd, int eb,
-		     int blocks, int req, const char *sreq)
+		     int blocks, unsigned long req, const char *sreq)
 {
 	int ret;
 	struct erase_info_user ei;

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/