Skip to contentSkip to conversation

vsprintf and short-wchar for EFI_LOADER

PATCH New Rob Clark 4 participants 14 messages Aug 9, 2017 – Sep 6, 2017

Subsystems (3)
lore
Share & tools
lore

Rob Clark

author
#0 lore raw

cover vsprintf and short-wchar for EFI_LOADER

As requested, I've split this out of the larger EFI_LOADER patchset.

This adds two things that the later EFI_LOADER patchset depends on:

1) UUID/GUID support
2) %ls UTF string support, in particular UTF-16. In the UEFI API, all strings are UTF-16

I had started converting this over to using c11 + u"string" literals, instead of using -fshort-wchar + L"string" literals and %ls, but I ran into two problems:

1) we lose out on the printf (and friends) va_arg type checking if we roll our own custom printf fmt modifier for UTF-16 strings
2) and worse than that, we have to disable -Wformat warnings

So given that we have a significant downside for not just using -fshort-wchar, and I don't think any really strong argument against, I am back to thinking that we should just go with -fshort-wchar.

The current patchset has a Kconfig option to opt-in to -fshort-wchar, which EFI_LOADER selects. If the consensus is to enable -fshort-wchar for everything, I'll drop the kconfig part and make it unconditional.

The current version of this patchset and efi-loader patchset are at

https://github.com/robclark/u-boot.git vsprintf

and

https://github.com/robclark/u-boot.git enough-uefi-for-shim-2

I'll resend the enought-uefi-for-shim-2 patchset after I have a chance to figure out fs-test.sh and add tests for fs_readdir().

Rob Clark (5):
Kconfig: add option to build with -fshort-wchar
lib: add some utf16 handling helpers
vsprintf.c: add UTF-16 string (%ls) support
vsprintf.c: add GUID printing
examples: add fallback memcpy

 Kconfig                      |  8 +++++
 Makefile                     |  4 +++
 examples/api/Makefile        |  2 ++
 examples/api/glue.c          | 12 +++++++
 include/charset.h            | 55 ++++++++++++++++++++++++++++++
 include/config_fallbacks.h   |  1 +
 lib/Makefile                 |  1 +
 lib/charset.c                | 81 ++++++++++++++++++++++++++++++++++++++++++++
 lib/efi_loader/Kconfig       |  1 +
 lib/efi_loader/efi_console.c | 17 ++--------
 lib/vsprintf.c               | 77 ++++++++++++++++++++++++++++++++++++++---
 11 files changed, 241 insertions(+), 18 deletions(-)
 create mode 100644 include/charset.h
 create mode 100644 lib/charset.c
Signature · 1 line
2.13.0
author
#1 lore raw

PATCH 1/5 Kconfig: add option to build with -fshort-wchar

UEFI expects strings to be UTF-16. So add an option so that when EFI_LOADER is enabled, we can use the expected unicode string size.

---
 Kconfig                | 8 ++++++++
 Makefile               | 4 ++++
 lib/efi_loader/Kconfig | 1 +
 3 files changed, 13 insertions(+)
Kconfig +8 -0
index c1451bceda..7319f1fa94 100644
@@ -53,6 +53,14 @@ config CC_OPTIMIZE_FOR_SIZE
 
 	  This option is enabled by default for U-Boot.
 
+config CC_SHORT_WCHAR
+	bool "Use 16b wchar"
+	default n
+	help
+	  Enabling this option will pass "-fshort-wchar" to gcc, for
+	  16bit unicode strings.  This is used by EFI_LOADER, as the
+	  UEFI spec defines strings to be UTF-16.
+
 config DISTRO_DEFAULTS
 	bool "Select defaults suitable for booting general purpose Linux distributions"
 	default y if ARCH_SUNXI || TEGRA
Makefile +4 -0
index 50a002e72f..91b11f5a7b 100644
@@ -590,6 +590,10 @@ else
 KBUILD_CFLAGS	+= -O2
 endif
 
+ifdef CONFIG_CC_SHORT_WCHAR
+KBUILD_CFLAGS	+= -fshort-wchar
+endif
+
 KBUILD_CFLAGS += $(call cc-option,-fno-stack-protector)
 KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks)
 
lib/efi_loader/Kconfig +1 -0
index d2b6327119..e28ef51ad4 100644
@@ -1,6 +1,7 @@
 config EFI_LOADER
 	bool "Support running EFI Applications in U-Boot"
 	depends on (ARM || X86) && OF_LIBFDT
+	select CC_SHORT_WCHAR
 	default y
 	help
 	  Select this option if you want to run EFI applications (like grub2)
Signature · 1 line
2.13.0
#2 lore raw

PATCH 1/5 Kconfig: add option to build with -fshort-wchar

Quoted 4 lines · Rob Clark

UEFI expects strings to be UTF-16. So add an option so that when EFI_LOADER is enabled, we can use the expected unicode string size.

So, I kludged this to just globally pass -fshort-wchar and I see no size changes in a world build
(https://gist.github.com/trini/60c4e58c662553ca902b4e878a9a8dc5). I feel we should explain why, and enable this along with the stdc-2011 change.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170809/61191905/attachment.sig>
author
#3 lore raw

PATCH 2/5 lib: add some utf16 handling helpers

In reply to Rob Clark

We'll eventually want these in a few places in efi_loader, and also vsprintf.

---
 include/charset.h            | 55 ++++++++++++++++++++++++++++++
 lib/Makefile                 |  1 +
 lib/charset.c                | 81 ++++++++++++++++++++++++++++++++++++++++++++
 lib/efi_loader/efi_console.c | 17 ++--------
 4 files changed, 140 insertions(+), 14 deletions(-)
 create mode 100644 include/charset.h
 create mode 100644 lib/charset.c
include/charset.h new file +55 -0
new file mode 100644
index 0000000000..47ff6c7af1
@@ -0,0 +1,55 @@
+/*
+ *  charset conversion utils
+ *
+ *  Copyright (c) 2017 Rob Clark
+ *
+ *  SPDX-License-Identifier:     GPL-2.0+
+ */
+
+#ifndef __CHARSET_H_
+#define __CHARSET_H_
+
+#define MAX_UTF8_PER_UTF16 4
+
+/**
+ * utf16_strlen() - Get the length of an utf16 string
+ *
+ * Returns the number of 16 bit characters in an utf16 string, not
+ * including the terminating NULL character.
+ *
+ * @in     the string to measure
+ * @return the string length
+ */
+size_t utf16_strlen(uint16_t *in);
+
+/**
+ * utf16_strnlen() - Get the length of a fixed-size utf16 string.
+ *
+ * Returns the number of 16 bit characters in an utf16 string,
+ * not including the terminating NULL character, but at most
+ * 'count' number of characters.  In doing this, utf16_strnlen()
+ * looks at only the first 'count' characters.
+ *
+ * @in     the string to measure
+ * @count  the maximum number of characters to count
+ * @return the string length, up to a maximum of 'count'
+ */
+size_t utf16_strnlen(const uint16_t *in, size_t count);
+
+/**
+ * utf16_to_utf8() - Convert an utf16 string to utf8
+ *
+ * Converts 'size' characters of the utf16 string 'src' to utf8
+ * written to the 'dest' buffer.
+ *
+ * NOTE that a single utf16 character can generate up to 4 utf8
+ * characters.  See MAX_UTF8_PER_UTF16.
+ *
+ * @dest   the destination buffer to write the utf8 characters
+ * @src    the source utf16 string
+ * @size   the number of utf16 characters to convert
+ * @return the pointer to the first unwritten byte in 'dest'
+ */
+uint8_t *utf16_to_utf8(uint8_t *dest, const uint16_t *src, size_t size);
+
+#endif /* __CHARSET_H_ */
lib/Makefile +1 -0
index eacc7d6485..b88b6ebd53 100644
@@ -19,6 +19,7 @@ obj-$(CONFIG_OF_LIVE) += of_live.o
 obj-$(CONFIG_CMD_DHRYSTONE) += dhry/
 
 obj-$(CONFIG_AES) += aes.o
+obj-y += charset.o
 obj-$(CONFIG_USB_TTY) += circbuf.o
 obj-y += crc7.o
 obj-y += crc8.o
lib/charset.c new file +81 -0

81 lines not on this page · raw message

lib/efi_loader/efi_console.c +3 -14
index 5ebce4b544..3fc82b8726 100644
@@ -7,6 +7,7 @@
  */
 
 #include <common.h>
+#include <charset.h>
 #include <efi_loader.h>
 
 static bool console_size_queried;
@@ -138,20 +139,8 @@ static efi_status_t EFIAPI efi_cout_reset(
 
 static void print_unicode_in_utf8(u16 c)
 {
-	char utf8[4] = { 0 };
-	char *b = utf8;
-
-	if (c < 0x80) {
-		*(b++) = c;
-	} else if (c < 0x800) {
-		*(b++) = 192 + c / 64;
-		*(b++) = 128 + c % 64;
-	} else {
-		*(b++) = 224 + c / 4096;
-		*(b++) = 128 + c / 64 % 64;
-		*(b++) = 128 + c % 64;
-	}
-
+	char utf8[MAX_UTF8_PER_UTF16] = { 0 };
+	utf16_to_utf8((u8 *)utf8, &c, 1);
 	puts(utf8);
 }
 
Signature · 1 line
2.13.0
#4 lore raw

PATCH 2/5 lib: add some utf16 handling helpers

Hi Rob,

Quoted 4 lines · Rob Clark

We'll eventually want these in a few places in efi_loader, and also vsprintf.

Quoted 8 lines
---
 include/charset.h            | 55 ++++++++++++++++++++++++++++++
 lib/Makefile                 |  1 +
 lib/charset.c                | 81 ++++++++++++++++++++++++++++++++++++++++++++
 lib/efi_loader/efi_console.c | 17 ++--------
 4 files changed, 140 insertions(+), 14 deletions(-)
 create mode 100644 include/charset.h
 create mode 100644 lib/charset.c

Looks good.

Please can you add some tests for these functions?

Regards,
Simon

author
#5 lore raw

PATCH 3/5 vsprintf.c: add UTF-16 string (%ls) support

In reply to Rob Clark

This is convenient for efi_loader which deals a lot with UTF-16. Only enabled with CC_SHORT_WCHAR, leaving room to add a UTF-32 version when CC_SHORT_WCHAR is not enabled.

---
 examples/api/Makefile |  1 +
 lib/vsprintf.c        | 31 +++++++++++++++++++++++++++++--
 2 files changed, 30 insertions(+), 2 deletions(-)
examples/api/Makefile +1 -0
index dab6398bab..87c15d0f68 100644
@@ -34,6 +34,7 @@ EXT_COBJ-y += lib/div64.o
 EXT_COBJ-y += lib/string.o
 EXT_COBJ-y += lib/time.o
 EXT_COBJ-y += lib/vsprintf.o
+EXT_COBJ-y += lib/charset.o
 EXT_SOBJ-$(CONFIG_PPC) += arch/powerpc/lib/ppcstring.o
 ifeq ($(ARCH),arm)
 EXT_SOBJ-$(CONFIG_USE_ARCH_MEMSET) += arch/arm/lib/memset.o
lib/vsprintf.c +29 -2
index 874a2951f7..0678b49b01 100644
@@ -17,6 +17,7 @@
 #include <linux/ctype.h>
 
 #include <common.h>
+#include <charset.h>
 
 #include <div64.h>
 #define noinline __attribute__((noinline))
@@ -270,6 +271,26 @@ static char *string(char *buf, char *end, char *s, int field_width,
 	return buf;
 }
 
+static char *string16(char *buf, char *end, u16 *s, int field_width,
+		int precision, int flags)
+{
+	u16 *str = s ? s : (u16[]){'<','N','U','L','L','>','\0'};
+	int utf16_len = utf16_strnlen(str, precision);
+	u8 utf8[utf16_len * MAX_UTF8_PER_UTF16];
+	int utf8_len, i;
+
+	utf8_len = utf16_to_utf8(utf8, str, utf16_len) - utf8;
+
+	if (!(flags & LEFT))
+		while (utf8_len < field_width--)
+			ADDCH(buf, ' ');
+	for (i = 0; i < utf8_len; ++i)
+		ADDCH(buf, utf8[i]);
+	while (utf8_len < field_width--)
+		ADDCH(buf, ' ');
+	return buf;
+}
+
 #ifdef CONFIG_CMD_NET
 static const char hex_asc[] = "0123456789abcdef";
 #define hex_asc_lo(x)	hex_asc[((x) & 0x0f)]
@@ -528,8 +549,14 @@ repeat:
 			continue;
 
 		case 's':
-			str = string(str, end, va_arg(args, char *),
-				     field_width, precision, flags);
+			if (CONFIG_IS_ENABLED(CC_SHORT_WCHAR) &&
+			    qualifier == 'l') {
+				str = string16(str, end, va_arg(args, u16 *),
+					       field_width, precision, flags);
+			} else {
+				str = string(str, end, va_arg(args, char *),
+					     field_width, precision, flags);
+			}
 			continue;
 
 		case 'p':
Signature · 1 line
2.13.0
#6 lore raw

PATCH 3/5 vsprintf.c: add UTF-16 string (%ls) support

Quoted 10 lines · Rob Clark

This is convenient for efi_loader which deals a lot with UTF-16. Only enabled with CC_SHORT_WCHAR, leaving room to add a UTF-32 version when CC_SHORT_WCHAR is not enabled.

---
 examples/api/Makefile |  1 +
 lib/vsprintf.c        | 31 +++++++++++++++++++++++++++++--
 2 files changed, 30 insertions(+), 2 deletions(-)
author
#7 lore raw

PATCH 4/5 vsprintf.c: add GUID printing

In reply to Rob Clark

This works (roughly) the same way as linux's, but we currently always print lower-case (ie. we just keep %pUB and %pUL for compat with linux), mostly just because that is what uuid_bin_to_str() supports.

%pUb: 01020304-0506-0708-090a-0b0c0d0e0f10
%pUl: 04030201-0605-0807-090a-0b0c0d0e0f10

It will be used by a later efi_loader paths for efi variables and for device-path-to-text protocol, and also quite useful for debug prints of protocol GUIDs.

---
 examples/api/Makefile      |  1 +
 include/config_fallbacks.h |  1 +
 lib/vsprintf.c             | 46 ++++++++++++++++++++++++++++++++++++++++++++--
 3 files changed, 46 insertions(+), 2 deletions(-)
examples/api/Makefile +1 -0
index 87c15d0f68..899527267d 100644
@@ -35,6 +35,7 @@ EXT_COBJ-y += lib/string.o
 EXT_COBJ-y += lib/time.o
 EXT_COBJ-y += lib/vsprintf.o
 EXT_COBJ-y += lib/charset.o
+EXT_COBJ-$(CONFIG_LIB_UUID) += lib/uuid.o
 EXT_SOBJ-$(CONFIG_PPC) += arch/powerpc/lib/ppcstring.o
 ifeq ($(ARCH),arm)
 EXT_SOBJ-$(CONFIG_USE_ARCH_MEMSET) += arch/arm/lib/memset.o
include/config_fallbacks.h +1 -0
index 961a83d758..56b9de09f2 100644
@@ -57,6 +57,7 @@
 
 #if (CONFIG_IS_ENABLED(PARTITION_UUIDS) || \
 	CONFIG_IS_ENABLED(EFI_PARTITION) || \
+	CONFIG_IS_ENABLED(EFI_LOADER) || \
 	defined(CONFIG_RANDOM_UUID) || \
 	defined(CONFIG_CMD_UUID) || \
 	defined(CONFIG_BOOTP_PXE)) && \
lib/vsprintf.c +44 -2

72 lines not on this page · raw message

Signature · 1 line
2.13.0
#8 lore raw

PATCH 4/5 vsprintf.c: add GUID printing

Quoted 113 lines · Rob Clark

This works (roughly) the same way as linux's, but we currently always print lower-case (ie. we just keep %pUB and %pUL for compat with linux), mostly just because that is what uuid_bin_to_str() supports.

%pUb: 01020304-0506-0708-090a-0b0c0d0e0f10
%pUl: 04030201-0605-0807-090a-0b0c0d0e0f10

It will be used by a later efi_loader paths for efi variables and for device-path-to-text protocol, and also quite useful for debug prints of protocol GUIDs.

---
 examples/api/Makefile      |  1 +
 include/config_fallbacks.h |  1 +
 lib/vsprintf.c             | 46 ++++++++++++++++++++++++++++++++++++++++++++--
 3 files changed, 46 insertions(+), 2 deletions(-)
examples/api/Makefile +1 -0
index 87c15d0f68..899527267d 100644
@@ -35,6 +35,7 @@ EXT_COBJ-y += lib/string.o
 EXT_COBJ-y += lib/time.o
 EXT_COBJ-y += lib/vsprintf.o
 EXT_COBJ-y += lib/charset.o
+EXT_COBJ-$(CONFIG_LIB_UUID) += lib/uuid.o
 EXT_SOBJ-$(CONFIG_PPC) += arch/powerpc/lib/ppcstring.o
 ifeq ($(ARCH),arm)
 EXT_SOBJ-$(CONFIG_USE_ARCH_MEMSET) += arch/arm/lib/memset.o
include/config_fallbacks.h +1 -0
index 961a83d758..56b9de09f2 100644
@@ -57,6 +57,7 @@
 
 #if (CONFIG_IS_ENABLED(PARTITION_UUIDS) || \
 	CONFIG_IS_ENABLED(EFI_PARTITION) || \
+	CONFIG_IS_ENABLED(EFI_LOADER) || \
 	defined(CONFIG_RANDOM_UUID) || \
 	defined(CONFIG_CMD_UUID) || \
 	defined(CONFIG_BOOTP_PXE)) && \
lib/vsprintf.c +37 -2
index 0678b49b01..71a995dee0 100644
@@ -18,6 +18,7 @@
 
 #include <common.h>
 #include <charset.h>
+#include <uuid.h>
 
 #include <div64.h>
 #define noinline __attribute__((noinline))
@@ -366,6 +367,40 @@ static char *ip4_addr_string(char *buf, char *end, u8 *addr, int field_width,
 }
 #endif
 
+#ifdef CONFIG_LIB_UUID
+/*
+ * This works (roughly) the same way as linux's, but we currently always
+ * print lower-case (ie. we just keep %pUB and %pUL for compat with linux),
+ * mostly just because that is what uuid_bin_to_str() supports.
+ *
+ *   %pUb:   01020304-0506-0708-090a-0b0c0d0e0f10
+ *   %pUl:   04030201-0605-0807-090a-0b0c0d0e0f10
+ */
+static char *uuid_string(char *buf, char *end, u8 *addr, int field_width,
+			 int precision, int flags, const char *fmt)
+{
+	char uuid[UUID_STR_LEN + 1];
+	int str_format = UUID_STR_FORMAT_STD;
+
+	switch (*(++fmt)) {
+	case 'L':
+	case 'l':
+		str_format = UUID_STR_FORMAT_GUID;
+		break;
+	case 'B':
+	case 'b':
+		/* this is the default */
+		break;
+	default:
+		break;
+	}
+
+	uuid_bin_to_str(addr, uuid, str_format);
+
+	return string(buf, end, uuid, field_width, precision, flags);
+}
+#endif
+
 /*
  * Show a '%p' thing.  A kernel extension is that the '%p' is followed
  * by an extra set of alphanumeric characters that are extended format
@@ -399,8 +434,8 @@ static char *pointer(const char *fmt, char *buf, char *end, void *ptr,
 			      flags);
 #endif
 
-#ifdef CONFIG_CMD_NET
 	switch (*fmt) {
+#ifdef CONFIG_CMD_NET
 	case 'a':
 		flags |= SPECIAL | ZEROPAD;
 
@@ -430,8 +465,15 @@ static char *pointer(const char *fmt, char *buf, char *end, void *ptr,
 					       precision, flags);
 		flags &= ~SPECIAL;
 		break;
-	}
 #endif
+#ifdef CONFIG_LIB_UUID

lib/vsprintf.c static char *pointer(const char *fmt, char *buf, char *end, void *ptr, · line 479

Quoted 10 lines
lib/vsprintf.c +7 -0
+	case 'U':
+		return uuid_string(buf, end, ptr, field_width, precision,
+				   flags, fmt);
+#endif
+	default:
+		break;
+	}
 	flags |= SMALL;
 	if (field_width == -1) {
 		field_width = 2*sizeof(void *);

Successfully tested printing using %pUl with a patch under development on arm64.

author
#9 lore raw

PATCH 5/5 examples: add fallback memcpy

In reply to Rob Clark

Solves build issue:

Building current source for 134 boards (12 threads, 1 job per thread)
arm: + lsxhl
+examples/api/vsprintf.o: In function `string16':
+lib/vsprintf.c:278: undefined reference to memcpy' +examples/api/uuid.o: In function uuid_bin_to_str':
+lib/uuid.c:197: undefined reference to memcpy' +lib/uuid.c:199: undefined reference to memcpy'
+make[3]: *** [examples/api/demo] Error 1
+make[2]: *** [examples/api] Error 2
+make[1]: *** [examples] Error 2
+make: *** [sub-make] Error 2

    133    0    1 /134    sheevaplug
---
 examples/api/glue.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
examples/api/glue.c +12 -0
index 8aabf32c89..575c1e55f3 100644
@@ -416,3 +416,15 @@ void ub_display_clear(void)
 {
 	syscall(API_DISPLAY_CLEAR, NULL);
 }
+
+__weak void *memcpy(void *dest, const void *src, size_t size)
+{
+	unsigned char *dptr = dest;
+	const unsigned char *ptr = src;
+	const unsigned char *end = src + size;
+
+	while (ptr < end)
+		*dptr++ = *ptr++;
+
+	return dest;
+}
Signature · 1 line
2.13.0
#10 lore raw

cover vsprintf and short-wchar for EFI_LOADER

In reply to Rob Clark
Quoted 42 lines · Rob Clark

As requested, I've split this out of the larger EFI_LOADER patchset.

This adds two things that the later EFI_LOADER patchset depends on:

1) UUID/GUID support
2) %ls UTF string support, in particular UTF-16. In the UEFI API, all strings are UTF-16

I had started converting this over to using c11 + u"string" literals, instead of using -fshort-wchar + L"string" literals and %ls, but I ran into two problems:

1) we lose out on the printf (and friends) va_arg type checking if we roll our own custom printf fmt modifier for UTF-16 strings
2) and worse than that, we have to disable -Wformat warnings

So given that we have a significant downside for not just using -fshort-wchar, and I don't think any really strong argument against, I am back to thinking that we should just go with -fshort-wchar.

The current patchset has a Kconfig option to opt-in to -fshort-wchar, which EFI_LOADER selects. If the consensus is to enable -fshort-wchar for everything, I'll drop the kconfig part and make it unconditional.

The current version of this patchset and efi-loader patchset are at

https://github.com/robclark/u-boot.git vsprintf

and

https://github.com/robclark/u-boot.git enough-uefi-for-shim-2

I'll resend the enought-uefi-for-shim-2 patchset after I have a chance to figure out fs-test.sh and add tests for fs_readdir().

Rob Clark (5):
Kconfig: add option to build with -fshort-wchar
lib: add some utf16 handling helpers
vsprintf.c: add UTF-16 string (%ls) support
vsprintf.c: add GUID printing
examples: add fallback memcpy

Quoted 15 lines
 Kconfig                      |  8 +++++
 Makefile                     |  4 +++
 examples/api/Makefile        |  2 ++
 examples/api/glue.c          | 12 +++++++
 include/charset.h            | 55 ++++++++++++++++++++++++++++++
 include/config_fallbacks.h   |  1 +
 lib/Makefile                 |  1 +
 lib/charset.c                | 81 ++++++++++++++++++++++++++++++++++++++++++++
 lib/efi_loader/Kconfig       |  1 +
 lib/efi_loader/efi_console.c | 17 ++--------
 lib/vsprintf.c               | 77 ++++++++++++++++++++++++++++++++++++++---
 11 files changed, 241 insertions(+), 18 deletions(-)
 create mode 100644 include/charset.h
 create mode 100644 lib/charset.c

Hello Tom,

what is happening with this patch series?

[U-Boot,5/5] examples: add fallback memcpy
https://patchwork.ozlabs.org/patch/800030/
[U-Boot,4/5] vsprintf.c: add GUID printing
https://patchwork.ozlabs.org/patch/800029/
[U-Boot,3/5] vsprintf.c: add UTF-16 string (%ls) support
https://patchwork.ozlabs.org/patch/800027/
[U-Boot,2/5] lib: add some utf16 handling helpers
https://patchwork.ozlabs.org/patch/800028/
[U-Boot,1/5] Kconfig: add option to build with -fshort-wchar
https://patchwork.ozlabs.org/patch/800026/

Was it too late for 2017.09 and will be merged with 2017.11?

Best regards

Heinrich

#11 lore raw

cover vsprintf and short-wchar for EFI_LOADER

Quoted 74 lines · Heinrich Schuchardt
Quoted 42 lines · Rob Clark

As requested, I've split this out of the larger EFI_LOADER patchset.

This adds two things that the later EFI_LOADER patchset depends on:

1) UUID/GUID support
2) %ls UTF string support, in particular UTF-16. In the UEFI API, all strings are UTF-16

I had started converting this over to using c11 + u"string" literals, instead of using -fshort-wchar + L"string" literals and %ls, but I ran into two problems:

1) we lose out on the printf (and friends) va_arg type checking if we roll our own custom printf fmt modifier for UTF-16 strings
2) and worse than that, we have to disable -Wformat warnings

So given that we have a significant downside for not just using -fshort-wchar, and I don't think any really strong argument against, I am back to thinking that we should just go with -fshort-wchar.

The current patchset has a Kconfig option to opt-in to -fshort-wchar, which EFI_LOADER selects. If the consensus is to enable -fshort-wchar for everything, I'll drop the kconfig part and make it unconditional.

The current version of this patchset and efi-loader patchset are at

https://github.com/robclark/u-boot.git vsprintf

and

https://github.com/robclark/u-boot.git enough-uefi-for-shim-2

I'll resend the enought-uefi-for-shim-2 patchset after I have a chance to figure out fs-test.sh and add tests for fs_readdir().

Rob Clark (5):
Kconfig: add option to build with -fshort-wchar
lib: add some utf16 handling helpers
vsprintf.c: add UTF-16 string (%ls) support
vsprintf.c: add GUID printing
examples: add fallback memcpy

Quoted 15 lines
 Kconfig                      |  8 +++++
 Makefile                     |  4 +++
 examples/api/Makefile        |  2 ++
 examples/api/glue.c          | 12 +++++++
 include/charset.h            | 55 ++++++++++++++++++++++++++++++
 include/config_fallbacks.h   |  1 +
 lib/Makefile                 |  1 +
 lib/charset.c                | 81 ++++++++++++++++++++++++++++++++++++++++++++
 lib/efi_loader/Kconfig       |  1 +
 lib/efi_loader/efi_console.c | 17 ++--------
 lib/vsprintf.c               | 77 ++++++++++++++++++++++++++++++++++++++---
 11 files changed, 241 insertions(+), 18 deletions(-)
 create mode 100644 include/charset.h
 create mode 100644 lib/charset.c

Hello Tom,

what is happening with this patch series?

[U-Boot,5/5] examples: add fallback memcpy
https://patchwork.ozlabs.org/patch/800030/
[U-Boot,4/5] vsprintf.c: add GUID printing
https://patchwork.ozlabs.org/patch/800029/
[U-Boot,3/5] vsprintf.c: add UTF-16 string (%ls) support
https://patchwork.ozlabs.org/patch/800027/
[U-Boot,2/5] lib: add some utf16 handling helpers
https://patchwork.ozlabs.org/patch/800028/
[U-Boot,1/5] Kconfig: add option to build with -fshort-wchar
https://patchwork.ozlabs.org/patch/800026/

Quoted 1 line

Was it too late for 2017.09 and will be merged with 2017.11?

I replied to the first patch saying that we should always -fshort-wchar and enforce stdc-2011 as the default. That would be good to open 2017.11 with as well. Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170906/b7067d6b/attachment.sig>
author
#12 lore raw

cover vsprintf and short-wchar for EFI_LOADER

Quoted 75 lines · Tom Rini
Quoted 74 lines · Heinrich Schuchardt
Quoted 42 lines · Rob Clark

As requested, I've split this out of the larger EFI_LOADER patchset.

This adds two things that the later EFI_LOADER patchset depends on:

1) UUID/GUID support
2) %ls UTF string support, in particular UTF-16. In the UEFI API, all strings are UTF-16

I had started converting this over to using c11 + u"string" literals, instead of using -fshort-wchar + L"string" literals and %ls, but I ran into two problems:

1) we lose out on the printf (and friends) va_arg type checking if we roll our own custom printf fmt modifier for UTF-16 strings
2) and worse than that, we have to disable -Wformat warnings

So given that we have a significant downside for not just using -fshort-wchar, and I don't think any really strong argument against, I am back to thinking that we should just go with -fshort-wchar.

The current patchset has a Kconfig option to opt-in to -fshort-wchar, which EFI_LOADER selects. If the consensus is to enable -fshort-wchar for everything, I'll drop the kconfig part and make it unconditional.

The current version of this patchset and efi-loader patchset are at

https://github.com/robclark/u-boot.git vsprintf

and

https://github.com/robclark/u-boot.git enough-uefi-for-shim-2

I'll resend the enought-uefi-for-shim-2 patchset after I have a chance to figure out fs-test.sh and add tests for fs_readdir().

Rob Clark (5):
Kconfig: add option to build with -fshort-wchar
lib: add some utf16 handling helpers
vsprintf.c: add UTF-16 string (%ls) support
vsprintf.c: add GUID printing
examples: add fallback memcpy

Quoted 15 lines
 Kconfig                      |  8 +++++
 Makefile                     |  4 +++
 examples/api/Makefile        |  2 ++
 examples/api/glue.c          | 12 +++++++
 include/charset.h            | 55 ++++++++++++++++++++++++++++++
 include/config_fallbacks.h   |  1 +
 lib/Makefile                 |  1 +
 lib/charset.c                | 81 ++++++++++++++++++++++++++++++++++++++++++++
 lib/efi_loader/Kconfig       |  1 +
 lib/efi_loader/efi_console.c | 17 ++--------
 lib/vsprintf.c               | 77 ++++++++++++++++++++++++++++++++++++++---
 11 files changed, 241 insertions(+), 18 deletions(-)
 create mode 100644 include/charset.h
 create mode 100644 lib/charset.c

Hello Tom,

what is happening with this patch series?

[U-Boot,5/5] examples: add fallback memcpy
https://patchwork.ozlabs.org/patch/800030/
[U-Boot,4/5] vsprintf.c: add GUID printing
https://patchwork.ozlabs.org/patch/800029/
[U-Boot,3/5] vsprintf.c: add UTF-16 string (%ls) support
https://patchwork.ozlabs.org/patch/800027/
[U-Boot,2/5] lib: add some utf16 handling helpers
https://patchwork.ozlabs.org/patch/800028/
[U-Boot,1/5] Kconfig: add option to build with -fshort-wchar
https://patchwork.ozlabs.org/patch/800026/

Quoted 6 lines
Quoted 1 line

Was it too late for 2017.09 and will be merged with 2017.11?

I replied to the first patch saying that we should always -fshort-wchar and enforce stdc-2011 as the default. That would be good to open 2017.11 with as well. Thanks!

Oh, I guess I should resend that. Probably c11 should be a different patch (although I am in favour of switching)..

When does the merge window open for 2017.11? It would be good to land this and the fs/fat patches early so we could start merging the pile of efi_loader patches that depend on these.

BR,
-R

#13 lore raw

cover vsprintf and short-wchar for EFI_LOADER

Quoted 83 lines · Rob Clark
Quoted 75 lines · Tom Rini
Quoted 74 lines · Heinrich Schuchardt
Quoted 42 lines · Rob Clark

As requested, I've split this out of the larger EFI_LOADER patchset.

This adds two things that the later EFI_LOADER patchset depends on:

1) UUID/GUID support
2) %ls UTF string support, in particular UTF-16. In the UEFI API, all strings are UTF-16

I had started converting this over to using c11 + u"string" literals, instead of using -fshort-wchar + L"string" literals and %ls, but I ran into two problems:

1) we lose out on the printf (and friends) va_arg type checking if we roll our own custom printf fmt modifier for UTF-16 strings
2) and worse than that, we have to disable -Wformat warnings

So given that we have a significant downside for not just using -fshort-wchar, and I don't think any really strong argument against, I am back to thinking that we should just go with -fshort-wchar.

The current patchset has a Kconfig option to opt-in to -fshort-wchar, which EFI_LOADER selects. If the consensus is to enable -fshort-wchar for everything, I'll drop the kconfig part and make it unconditional.

The current version of this patchset and efi-loader patchset are at

https://github.com/robclark/u-boot.git vsprintf

and

https://github.com/robclark/u-boot.git enough-uefi-for-shim-2

I'll resend the enought-uefi-for-shim-2 patchset after I have a chance to figure out fs-test.sh and add tests for fs_readdir().

Rob Clark (5):
Kconfig: add option to build with -fshort-wchar
lib: add some utf16 handling helpers
vsprintf.c: add UTF-16 string (%ls) support
vsprintf.c: add GUID printing
examples: add fallback memcpy

Quoted 15 lines
 Kconfig                      |  8 +++++
 Makefile                     |  4 +++
 examples/api/Makefile        |  2 ++
 examples/api/glue.c          | 12 +++++++
 include/charset.h            | 55 ++++++++++++++++++++++++++++++
 include/config_fallbacks.h   |  1 +
 lib/Makefile                 |  1 +
 lib/charset.c                | 81 ++++++++++++++++++++++++++++++++++++++++++++
 lib/efi_loader/Kconfig       |  1 +
 lib/efi_loader/efi_console.c | 17 ++--------
 lib/vsprintf.c               | 77 ++++++++++++++++++++++++++++++++++++++---
 11 files changed, 241 insertions(+), 18 deletions(-)
 create mode 100644 include/charset.h
 create mode 100644 lib/charset.c

Hello Tom,

what is happening with this patch series?

[U-Boot,5/5] examples: add fallback memcpy
https://patchwork.ozlabs.org/patch/800030/
[U-Boot,4/5] vsprintf.c: add GUID printing
https://patchwork.ozlabs.org/patch/800029/
[U-Boot,3/5] vsprintf.c: add UTF-16 string (%ls) support
https://patchwork.ozlabs.org/patch/800027/
[U-Boot,2/5] lib: add some utf16 handling helpers
https://patchwork.ozlabs.org/patch/800028/
[U-Boot,1/5] Kconfig: add option to build with -fshort-wchar
https://patchwork.ozlabs.org/patch/800026/

Quoted 6 lines
Quoted 1 line

Was it too late for 2017.09 and will be merged with 2017.11?

I replied to the first patch saying that we should always -fshort-wchar and enforce stdc-2011 as the default. That would be good to open 2017.11 with as well. Thanks!

Quoted 6 lines

Oh, I guess I should resend that. Probably c11 should be a different patch (although I am in favour of switching)..

When does the merge window open for 2017.11? It would be good to land this and the fs/fat patches early so we could start merging the pile of efi_loader patches that depend on these.

v2017.09 will be released on the 11th, if things are going well, sometime more morning than not, eastern time. Then I'll start grabbing stuff and testing. If you would like to suggest orders of things to pull in, pointers to bundles in patchwork would be great. Otherwise I'll be assembling them myself as always. Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170906/09ec51b4/attachment.sig>