ignore UTF-8 byte order mark
PATCH 1/1 ignore UTF-8 byte order mark
Currently openvpn exits when reading configuration from a file that has an UTF-8 byte order mark (EF BB BF) at its start.
While it is useless to put a BOM into UTF-8 encoded files, the Unicode standard [1] permits it (on page 36):
"Use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature."
Notepad.exe, the default text editor on Windows, always puts the BOM into UTF-8 encoded files when saving them. Others may do as well.
Just ignoring the UTF-8 BOM will make config files with UTF-8 BOM readable.
[1] http://www.unicode.org/versions/Unicode5.0.0/ch02.pdf
--- src/openvpn/options.c | 3 +++ 1 file changed, 3 insertions(+)
src/openvpn/options.c +3 -0
@@ -3773,6 +3773,9 @@ read_config_file (struct options *options, {CLEAR (p); ++line_num; +/* Ignore UTF-8 BOM at start of stream */ +if (line_num ==1 &&strncmp (line,"\xEF\xBB\xBF" ,3 ) ==0 ) +memmove (line, line +3 ,strlen (line) -2 );if (parse_line (line, p,SIZE (p), file, line_num, msglevel, &options->gc)) {bypass_doubledash (&p[0 ]);
Signature · 1 line
tg: (77fb72e..) t/0005/ignore_utf8_bom (depends on: t/0004/utf8_compatibility)
Quoted 30 lines · Heiko Hund
Currently openvpn exits when reading configuration from a file that has an UTF-8 byte order mark (EF BB BF) at its start.
While it is useless to put a BOM into UTF-8 encoded files, the Unicode standard [1] permits it (on page 36):
"Use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature."
Notepad.exe, the default text editor on Windows, always puts the BOM into UTF-8 encoded files when saving them. Others may do as well.
Just ignoring the UTF-8 BOM will make config files with UTF-8 BOM readable.
[1] http://www.unicode.org/versions/Unicode5.0.0/ch02.pdf
Signed-off-by Heiko Hund <[email protected]...>--- src/openvpn/options.c | 3 +++ 1 file changed, 3 insertions(+)src/openvpn/options.c +0
-0
src/openvpn/options.c read_config_file (struct options *options, · line 3779
Quoted 7 lines
src/openvpn/options.c +3
-0@@ -3773,6 +3773,9 @@ read_config_file (struct options *options, {CLEAR (p); ++line_num; +/* Ignore UTF-8 BOM at start of stream */ +if (line_num ==1 &&strncmp (line,"\xEF\xBB\xBF" ,3 ) ==0 ) +memmove (line, line +3 ,strlen (line) -2 );if (parse_line (line, p,SIZE (p), file, line_num, msglevel, &options->gc))
Wouldn't this approach solve it in an easier way?
int offset = 0;
if (line_num == 1 && strncmp (line, "\xEF\xBB\xBF", 3) == 0)
offset = 3
if (parse_line (line+offset, p, SIZE (p), file, line_num ....
Should be safer as you don't need care about proper NULL termination of the buffer.
Signature · 3 lines
kind regards, David Sommerseth
Quoted 10 lines · On Saturday 02 February 2013 17:47:40 David Sommerseth wrote:
Wouldn't this approach solve it in an easier way?
int offset = 0;
if (line_num== 1 && strncmp (line, "\xEF\xBB\xBF", 3) == 0)
offset = 3if (
parse_line(line+offset, p, SIZE (p), file,line_num....Should be safer as you don't need care about proper NULL termination of the buffer.
Although fgets(3) is supposed to always \0 terminate the buffer I like your approach better, as it's more lightweight. Will send an updated version.
Heiko
PATCH v2 1/1 ignore UTF-8 byte order mark
In reply to Heiko HundCurrently openvpn exits when reading configuration from a file that has an UTF-8 byte order mark (EF BB BF) at its start.
While it is useless to put a BOM into UTF-8 encoded files, the Unicode standard [1] permits it (on page 36):
"Use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature."
Notepad.exe, the default text editor on Windows, always puts the BOM into UTF-8 encoded files when saving them. Others may do as well.
Just ignoring the UTF-8 BOM will make config files with UTF-8 BOM readable.
[1] http://www.unicode.org/versions/Unicode5.0.0/ch02.pdf
--- src/openvpn/options.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
src/openvpn/options.c +5 -1
@@ -3773,7 +3773,11 @@ read_config_file (struct options *options, {CLEAR (p); ++line_num; -if (parse_line (line, p,SIZE (p), file, line_num, msglevel, &options->gc)) +/* Ignore UTF-8 BOM at start of stream */ +int offset =0 ; +if (line_num ==1 &&strncmp (line,"\xEF\xBB\xBF" ,3 ) ==0 ) + offset =3 ; +if (parse_line (line + offset, p,SIZE (p), file, line_num, msglevel, &options->gc)) {bypass_doubledash (&p[0 ]);check_inline_file_via_fp (fp, p, &options->gc);
Signature · 1 line
1.7.9.5
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 25 lines · Heiko Hund
Currently openvpn exits when reading configuration from a file that has an UTF-8 byte order mark (EF BB BF) at its start.
While it is useless to put a BOM into UTF-8 encoded files, the Unicode standard [1] permits it (on page 36):
"Use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature."
Notepad.exe, the default text editor on Windows, always puts the BOM into UTF-8 encoded files when saving them. Others may do as well.
Just ignoring the UTF-8 BOM will make config files with UTF-8 BOM readable.
[1] http://www.unicode.org/versions/Unicode5.0.0/ch02.pdf
Signed-off-by Heiko Hund <[email protected]...> ---
src/openvpn/options.c| 6 +++++- 1 file changed, 5
insertions(+), 1 deletion(-)src/openvpn/options.c index renamed from src/openvpn/options.c +0
-0
Quoted 6 lines
8ca41a3..c52b699 100644 ---
a/src/openvpn/options.c+++b/src/openvpn/options.c@@ -3773,7 +3773,11 @@read_config_file
(struct options *options, { CLEAR (p); ++line_num; - if
(parse_line(line, p, SIZE (p), file,line_num, msglevel,&options->gc)) + /* Ignore UTF-8 BOM at start of
stream */ + int offset = 0;
Some compilers will complain with int being declared here. Moving it up to before the CLEAR (p); line is probably going to work better ...
- --
kind regards,
David Sommerseth
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAlEOd/EACgkQDC186MBRfrr9+wCghuv++EJkTlCAb3p0YoLzSkGv b1YAn3PKYmvfaIZzaYsu3NAXl6m2Dutz
=Fyps
-----END PGP SIGNATURE-----
Quoted 29 lines · Heiko Hund
Currently openvpn exits when reading configuration from a file that has an UTF-8 byte order mark (EF BB BF) at its start.
While it is useless to put a BOM into UTF-8 encoded files, the Unicode standard [1] permits it (on page 36):
"Use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature."
Notepad.exe, the default text editor on Windows, always puts the BOM into UTF-8 encoded files when saving them. Others may do as well.
Just ignoring the UTF-8 BOM will make config files with UTF-8 BOM readable.
[1] http://www.unicode.org/versions/Unicode5.0.0/ch02.pdf
Signed-off-by Heiko Hund <[email protected]...>--- src/openvpn/options.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)src/openvpn/options.c +0
-0
src/openvpn/options.c read_config_file (struct options *options, · line 3779
Quoted 8 lines
src/openvpn/options.c +4
-1@@ -3773,7 +3773,11 @@ read_config_file (struct options *options, {CLEAR (p); ++line_num; -if (parse_line (line, p,SIZE (p), file, line_num, msglevel, &options->gc)) +/* Ignore UTF-8 BOM at start of stream */ +int offset =0 ; +if (line_num ==1 &&strncmp (line,"\xEF\xBB\xBF" ,3 ) ==0 ) + offset =3 ;
Lets try again without word-wrap ....
Some compilers will complain with int offset being declared "down" here. Moving it up to before the CLEAR (p); line is probably going to work better ...
Signature · 3 lines
kind regards, David Sommerseth
PATCH v3 1/1 ignore UTF-8 byte order mark
In reply to Heiko HundCurrently openvpn exits when reading configuration from a file that has an UTF-8 byte order mark (EF BB BF) at its start.
While it is useless to put a BOM into UTF-8 encoded files, the Unicode standard [1] permits it (on page 36):
"Use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature."
Notepad.exe, the default text editor on Windows, always puts the BOM into UTF-8 encoded files when saving them. Others may do as well.
Just ignoring the UTF-8 BOM will make config files with UTF-8 BOM readable.
[1] http://www.unicode.org/versions/Unicode5.0.0/ch02.pdf
--- src/openvpn/options.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
src/openvpn/options.c +5 -1
@@ -3771,9 +3771,13 @@ read_config_file (struct options *options, line_num =0 ;while (fgets (line,sizeof (line), fp)) { +int offset =0 ;CLEAR (p); ++line_num; -if (parse_line (line, p,SIZE (p), file, line_num, msglevel, &options->gc)) +/* Ignore UTF-8 BOM at start of stream */ +if (line_num ==1 &&strncmp (line,"\xEF\xBB\xBF" ,3 ) ==0 ) + offset =3 ; +if (parse_line (line + offset, p,SIZE (p), file, line_num, msglevel, &options->gc)) {bypass_doubledash (&p[0 ]);check_inline_file_via_fp (fp, p, &options->gc);
Signature · 1 line
1.7.10.4
Your patch has been applied to the master branch.
commit 6e6f55f4ba5deda5649679a13e4e323e07b3e661
Author: Heiko Hund Date: Mon Feb 4 11:39:25 2013 +0000
Ignore UTF-8 byte order mark
Signed-off-by: Heiko Hund <[email protected]...>
Acked-by: David Sommerseth <[email protected]...>
Message-Id: [email protected]...
URL: http://article.gmane.org/gmane.network.openvpn.devel/7342
Signed-off-by: David Sommerseth <[email protected]...>--
kind regards,
David Sommerseth
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Forgot to add this was also applied to release/2.3 commit a8fe779b04cc5bf698a1bf5bed4b0189126c3a38
Quoted 5 lines · David Sommerseth
Your patch has been applied to the master branch.
commit
6e6f55f4ba5deda5649679a13e4e323e07b3e661Author: Heiko Hund
Date: Mon Feb 4 11:39:25 2013 +0000
Quoted 8 lines
Ignore UTF-8 byte order mark
Signed-off-by Heiko Hund <[email protected]...> Acked-by: DavidSommerseth <[email protected]...> Message-Id:
[email protected]... URL:
http://article.gmane.org/gmane.network.openvpn.devel/7342Signed-off-by David Sommerseth <[email protected]...>
- --
kind regards,
David Sommerseth
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAlESqzAACgkQDC186MBRfrrKygCfTqg7qDJDo/f0rlKln8Qi4bbJ 2yIAoJeuU9Wt2GSrKGDHgbi0FFIZ2TZQ
=PUer
-----END PGP SIGNATURE-----