NLS refactor and UTF-8 normalization
Subsystems (11)
cover NLS refactor and UTF-8 normalization
The goal of this patchset is to adapt the NLS subsystem to support a full UTF-8 normalization and casefold operations for specific versions of unicode. There are many use cases of this feature and, while my specific interest is on case-insensitiveness for local filesystems, I am aware this might be in the wishlist of people working on SMB, NFS and others.
The first part of the patchset refactors the NLS subsytem to allow multiple tables of the same encoding (differentiated by the version) in an inexpensive way. It also create hooks for some higher-level operations, like comparisons and normalization. A new interface is exported to request a specific version of the charset.
The second part of the patchset introduces the utf8 normalization code as a new NLS charset. It is implemented as a separated charset, called utf8n to preserve curent behavior of the nls_utf8 charset. The normalization core is provided by the SGI patches from 2014, which I refactored, adapted and updated to unicode 10.0.0.
The last patch is a test module, which does some sanity check on the normalization code when it is loaded.
As usual, the unicode source files are not part of the patch because they are too big and would bounce the emails. If you are interested in fetching this with minimal effort, you can clone a branch from:
https://gitlab.collabora.com/krisman/linux.git -b nls-merge-int
Gabriel Krisman Bertazi (11):
nls: Wrap uni2char/char2uni callers
nls: Wrap charset field access
nls: Wrap charset hooks in ops structure
nls: Split default charset from NLS core
nls: Split struct nls_charset from struct nls_table
nls: Add support for multiple versions of an encoding
nls: Add new interface for string comparisons
nls: Let charsets define the behavior of tolower/toupper
nls: Add optional normalization and casefold hooks
nls: utf8norm: Integrate utf8norm code with NLS subsystem
nls: utf8norm: Introduce test module for utf8norm implementation
Olaf Weber (4):
nls: utf8norm: Add unicode character database files
scripts: add trie generator for UTF-8
nls: utf8norm: Introduce code for UTF-8 normalization
nls: utf8norm: reduce the size of utf8data[]
drivers/staging/ncpfs/ioctl.c | 13 +- drivers/staging/ncpfs/ncplib_kernel.c | 8 +- fs/befs/linuxvfs.c | 8 +- fs/cifs/cifs_unicode.c | 15 +- fs/cifs/cifsfs.c | 2 +- fs/cifs/connect.c | 2 +- fs/cifs/dir.c | 7 +- fs/fat/dir.c | 13 +- fs/fat/inode.c | 6 +- fs/fat/namei_vfat.c | 6 +- fs/hfs/super.c | 6 +- fs/hfs/trans.c | 9 +- fs/hfsplus/options.c | 2 +- fs/hfsplus/unicode.c | 6 +- fs/isofs/inode.c | 5 +- fs/isofs/joliet.c | 3 +- fs/jfs/jfs_unicode.c | 9 +- fs/jfs/super.c | 3 +- fs/nls/Kconfig | 13 + fs/nls/Makefile | 20 +- fs/nls/mac-celtic.c | 34 +- fs/nls/mac-centeuro.c | 34 +- fs/nls/mac-croatian.c | 34 +- fs/nls/mac-cyrillic.c | 34 +- fs/nls/mac-gaelic.c | 34 +- fs/nls/mac-greek.c | 34 +- fs/nls/mac-iceland.c | 34 +- fs/nls/mac-inuit.c | 34 +- fs/nls/mac-roman.c | 34 +- fs/nls/mac-romanian.c | 34 +- fs/nls/mac-turkish.c | 34 +- fs/nls/nls_ascii.c | 34 +- fs/nls/nls_base.c | 505 +--- fs/nls/nls_cp1250.c | 34 +- fs/nls/nls_cp1251.c | 34 +- fs/nls/nls_cp1255.c | 36 +- fs/nls/nls_cp437.c | 34 +- fs/nls/nls_cp737.c | 34 +- fs/nls/nls_cp775.c | 34 +- fs/nls/nls_cp850.c | 34 +- fs/nls/nls_cp852.c | 34 +- fs/nls/nls_cp855.c | 34 +- fs/nls/nls_cp857.c | 34 +- fs/nls/nls_cp860.c | 34 +- fs/nls/nls_cp861.c | 34 +- fs/nls/nls_cp862.c | 34 +- fs/nls/nls_cp863.c | 34 +- fs/nls/nls_cp864.c | 34 +- fs/nls/nls_cp865.c | 34 +- fs/nls/nls_cp866.c | 34 +- fs/nls/nls_cp869.c | 34 +- fs/nls/nls_cp874.c | 36 +- fs/nls/nls_cp932.c | 36 +- fs/nls/nls_cp936.c | 36 +- fs/nls/nls_cp949.c | 36 +- fs/nls/nls_cp950.c | 36 +- fs/nls/nls_default.c | 488 ++++ fs/nls/nls_euc-jp.c | 29 +- fs/nls/nls_iso8859-1.c | 34 +- fs/nls/nls_iso8859-13.c | 34 +- fs/nls/nls_iso8859-14.c | 34 +- fs/nls/nls_iso8859-15.c | 34 +- fs/nls/nls_iso8859-2.c | 34 +- fs/nls/nls_iso8859-3.c | 34 +- fs/nls/nls_iso8859-4.c | 34 +- fs/nls/nls_iso8859-5.c | 34 +- fs/nls/nls_iso8859-6.c | 34 +- fs/nls/nls_iso8859-7.c | 34 +- fs/nls/nls_iso8859-9.c | 34 +- fs/nls/nls_koi8-r.c | 34 +- fs/nls/nls_koi8-ru.c | 30 +- fs/nls/nls_koi8-u.c | 34 +- fs/nls/nls_utf8.c | 34 +- fs/nls/nls_utf8n-core.c | 291 +++ fs/nls/nls_utf8n-norm.c | 797 ++++++ fs/nls/nls_utf8n-selftest.c | 307 +++ fs/nls/ucd/README | 33 + fs/nls/utf8n.h | 117 + fs/ntfs/inode.c | 2 +- fs/ntfs/super.c | 6 +- fs/ntfs/unistr.c | 13 +- fs/udf/super.c | 3 +- fs/udf/unicode.c | 4 +- include/linux/nls.h | 126 +- scripts/Makefile | 1 + scripts/mkutf8data.c | 3464 +++++++++++++++++++++++++ 86 files changed, 7138 insertions(+), 912 deletions(-) create mode 100644 fs/nls/nls_default.c create mode 100644 fs/nls/nls_utf8n-core.c create mode 100644 fs/nls/nls_utf8n-norm.c create mode 100644 fs/nls/nls_utf8n-selftest.c create mode 100644 fs/nls/ucd/README create mode 100644 fs/nls/utf8n.h create mode 100644 scripts/mkutf8data.c
Signature · 1 line
2.17.0
PATCH 1/15 nls: Wrap uni2char/char2uni callers
Generated with the following coccinele script:
<smpl>
@@
expression A, B, C, D;
@@
(
- A->uni2char(B, C, D)
+ nls_uni2char(A, B, C, D)
|
- A->char2uni(B, C, D)
+ nls_char2uni(A, B, C, D)
)
</smpl>
--- drivers/staging/ncpfs/ncplib_kernel.c | 8 ++++---- fs/befs/linuxvfs.c | 4 ++-- fs/cifs/cifs_unicode.c | 9 +++++---- fs/cifs/dir.c | 7 ++++--- fs/fat/dir.c | 8 ++++---- fs/fat/namei_vfat.c | 6 +++--- fs/hfs/trans.c | 9 +++++---- fs/hfsplus/unicode.c | 6 +++--- fs/isofs/joliet.c | 3 ++- fs/jfs/jfs_unicode.c | 7 +++---- fs/nls/nls_euc-jp.c | 4 ++-- fs/nls/nls_koi8-ru.c | 6 +++--- fs/ntfs/unistr.c | 8 ++++---- include/linux/nls.h | 13 +++++++++++++ 14 files changed, 57 insertions(+), 41 deletions(-)
drivers/staging/ncpfs/ncplib_kernel.c +4 -4
@@ -1153,7 +1153,7 @@ ncp__io2vol(struct ncp_server *server, unsigned char *vname, unsigned int *vlen, iname +=5 ; }else { nospec:; -if ( (chl = in->char2uni (iname, iname_end - iname, &ec)) <0 ) +if ( (chl =nls_char2uni (in, iname, iname_end - iname, &ec)) <0 )return chl; iname += chl; }
@@ -1161,7 +1161,7 @@ nospec:;/* unitoupper should be here! */ - chl = out->uni2char (ec, vname, vname_end - vname); + chl =nls_uni2char (out, ec, vname, vname_end - vname);if (chl <0 )return chl;
@@ -1215,7 +1215,7 @@ ncp__vol2io(struct ncp_server *server, unsigned char *iname, unsigned int *ilen,wchar_t ec;int chl; -if ( (chl = in->char2uni (vname, vname_end - vname, &ec)) <0 ) { +if ( (chl =nls_char2uni (in, vname, vname_end - vname, &ec)) <0 ) { err = chl; goto quit; }
@@ -1233,7 +1233,7 @@ ncp__vol2io(struct ncp_server *server, unsigned char *iname, unsigned int *ilen, } iname += k; }else { -if ( (chl = out->uni2char (ec, iname, iname_end - iname)) >=0 ) { +if ( (chl =nls_uni2char (out, ec, iname, iname_end - iname)) >=0 ) { iname += chl; }else {int k;
fs/befs/linuxvfs.c +2 -2
@@ -549,7 +549,7 @@ befs_utf2nls(struct super_block *sb, const char *in,/* convert from Unicode to nls */ if (uni >MAX_WCHAR_T ) goto conv_err; - unilen = nls->uni2char (uni, &result[o], in_len - o); + unilen =nls_uni2char (nls, uni, &result[o], in_len - o);if (unilen <0 ) goto conv_err; }
@@ -623,7 +623,7 @@ befs_nls2utf(struct super_block *sb, const char *in,for (i = o =0 ; i < in_len; i += unilen, o += utflen) {/* convert from nls to unicode */ - unilen = nls->char2uni (&in[i], in_len - i, &uni); + unilen =nls_char2uni (nls, &in[i], in_len - i, &uni);if (unilen <0 ) goto conv_err;
fs/cifs/cifs_unicode.c +5 -4
@@ -148,7 +148,7 @@ cifs_mapchar(char *target, const __u16 *from, const struct nls_table *cp,return len;/* if character not one of seven in special remap set */ - len = cp->uni2char (src_char, target,NLS_MAX_CHARSET_SIZE ); + len =nls_uni2char (cp, src_char, target,NLS_MAX_CHARSET_SIZE );if (len <=0 ) goto surrogate_pair;
@@ -292,7 +292,7 @@ cifs_strtoUTF16(__le16 *to, const char *from, int len, }for (i =0 ; len && *from; i++, from += charlen, len -= charlen) { - charlen = codepage->char2uni (from, len, &wchar_to); + charlen =nls_char2uni (codepage, from, len, &wchar_to);if (charlen <1 ) {cifs_dbg (VFS ,"strtoUTF16: char2uni of 0x%x returned %d\n" , *from, charlen);
@@ -518,7 +518,8 @@ cifsConvertToUTF16(__le16 *target, const char *source, int srclen,* as they use backslash as separator. */ if (dst_char ==0 ) { - charlen = cp->char2uni (source + i, srclen - i, &tmp); + charlen =nls_char2uni (cp, source + i, srclen - i, + &tmp); dst_char =cpu_to_le16 (tmp);/*
@@ -608,7 +609,7 @@ cifs_local_to_utf16_bytes(const char *from, int len,wchar_t wchar_to;for (i =0 ; len && *from; i++, from += charlen, len -= charlen) { - charlen = codepage->char2uni (from, len, &wchar_to); + charlen =nls_char2uni (codepage, from, len, &wchar_to);/* Failed conversion defaults to a question mark */ if (charlen <1 ) charlen =1 ;
fs/cifs/dir.c +4 -3
@@ -910,7 +910,7 @@ static int cifs_ci_hash(const struct dentry *dentry, struct qstr *q) hash =init_name_hash (dentry);for (i =0 ; i < q->len; i += charlen) { - charlen = codepage->char2uni (&q->name[i], q->len - i, &c); + charlen =nls_char2uni (codepage, &q->name[i], q->len - i, &c);/* error out if we can't convert the character */ if (unlikely (charlen <0 ))return charlen;
@@ -939,8 +939,9 @@ static int cifs_ci_compare(const struct dentry *dentry,for (i =0 ; i < len; i += l1) {/* Convert characters in both strings to UTF-16. */ - l1 = codepage->char2uni (&str[i], len - i, &c1); - l2 = codepage->char2uni (&name->name[i], name->len - i, &c2); + l1 =nls_char2uni (codepage, &str[i], len - i, &c1); + l2 =nls_char2uni (codepage, &name->name[i], name->len - i, + &c2);/* * If we can't convert either character, just declare it to
fs/fat/dir.c +4 -4
@@ -153,7 +153,7 @@ static int uni16_to_x8(struct super_block *sb, unsigned char *ascii,while (*ip && ((len -NLS_MAX_CHARSET_SIZE ) >0 )) { ec = *ip++; - charlen = nls->uni2char (ec, op,NLS_MAX_CHARSET_SIZE ); + charlen =nls_uni2char (nls, ec, op,NLS_MAX_CHARSET_SIZE );if (charlen >0 ) { op += charlen; len -= charlen;
@@ -195,7 +195,7 @@ fat_short2uni(struct nls_table *t, unsigned char *c, int clen, wchar_t *uni) {int charlen; - charlen = t->char2uni (c, clen, uni); + charlen =nls_char2uni (t, c, clen, uni);if (charlen <0 ) { *uni =0x003f ;/* a question mark */ charlen =1 ;
@@ -210,7 +210,7 @@ fat_short2lower_uni(struct nls_table *t, unsigned char *c,int charlen;wchar_t wc; - charlen = t->char2uni (c, clen, &wc); + charlen =nls_char2uni (t, c, clen, &wc);if (charlen <0 ) { *uni =0x003f ;/* a question mark */ charlen =1 ;
@@ -220,7 +220,7 @@ fat_short2lower_uni(struct nls_table *t, unsigned char *c,if (!nc) nc = *c; - charlen = t->char2uni (&nc,1 , uni); + charlen =nls_char2uni (t, &nc,1 , uni);if (charlen <0 ) { *uni =0x003f ;/* a question mark */ charlen =1 ;
fs/fat/namei_vfat.c +3 -3
@@ -289,7 +289,7 @@ static inline int to_shortname_char(struct nls_table *nls,return 1 ; } - len = nls->uni2char (*src, buf, buf_size); + len =nls_uni2char (nls, *src, buf, buf_size);if (len <=0 ) { info->valid =0 ; buf[0 ] ='_' ;
@@ -544,8 +544,8 @@ xlate_to_uni(const unsigned char *name, int len, unsigned char *outname, ip +=5 ; i +=5 ; }else { - charlen = nls->char2uni (ip, len - i, - (wchar_t *)op); + charlen =nls_char2uni (nls, ip, len - i, + (wchar_t *)op);if (charlen <0 )return -EINVAL ; ip += charlen;
fs/hfs/trans.c +5 -4
@@ -49,7 +49,8 @@ int hfs_mac2asc(struct super_block *sb, char *out, const struct hfs_name *in)while (srclen >0 ) {if (nls_disk) { - size = nls_disk->char2uni (src, srclen, &ch); + size =nls_char2uni (nls_disk, src, srclen, + &ch);if (size <=0 ) { ch ='?' ; size =1 ;
@@ -62,7 +63,7 @@ int hfs_mac2asc(struct super_block *sb, char *out, const struct hfs_name *in) }if (ch =='/' ) ch =':' ; - size = nls_io->uni2char (ch, dst, dstlen); + size =nls_uni2char (nls_io, ch, dst, dstlen);if (size <0 ) {if (size == -ENAMETOOLONG ) goto out;
@@ -110,7 +111,7 @@ void hfs_asc2mac(struct super_block *sb, struct hfs_name *out, const struct qstrwchar_t ch;while (srclen >0 ) { - size = nls_io->char2uni (src, srclen, &ch); + size =nls_char2uni (nls_io, src, srclen, &ch);if (size <0 ) { ch ='?' ; size =1 ;
@@ -120,7 +121,7 @@ void hfs_asc2mac(struct super_block *sb, struct hfs_name *out, const struct qstrif (ch ==':' ) ch ='/' ;if (nls_disk) { - size = nls_disk->uni2char (ch, dst, dstlen); + size =nls_uni2char (nls_disk, ch, dst, dstlen);if (size <0 ) {if (size == -ENAMETOOLONG ) goto out;
fs/hfsplus/unicode.c +3 -3
@@ -190,7 +190,7 @@ int hfsplus_uni2asc(struct super_block *sb, c0 =':' ;break ; } - res = nls->uni2char (c0, op, len); + res =nls_uni2char (nls, c0, op, len);if (res <0 ) {if (res == -ENAMETOOLONG ) goto out;
@@ -233,7 +233,7 @@ int hfsplus_uni2asc(struct super_block *sb, cc = c0; } done: - res = nls->uni2char (cc, op, len); + res =nls_uni2char (nls, cc, op, len);if (res <0 ) {if (res == -ENAMETOOLONG ) goto out;
@@ -256,7 +256,7 @@ int hfsplus_uni2asc(struct super_block *sb,static inline int asc2unichar (struct super_block *sb,const char *astr,int len,wchar_t *uc) { -int size =HFSPLUS_SB (sb)->nls->char2uni (astr, len, uc); +int size =nls_char2uni (HFSPLUS_SB (sb)->nls, astr, len, uc);if (size <=0 ) { *uc ='?' ; size =1 ;
fs/isofs/joliet.c +2 -1
@@ -25,7 +25,8 @@ uni16_to_x8(unsigned char *ascii, __be16 *uni, int len, struct nls_table *nls)while ((ch =get_unaligned (ip)) && len) {int llen; - llen = nls->uni2char (be16_to_cpu (ch), op,NLS_MAX_CHARSET_SIZE ); + llen =nls_uni2char (nls,be16_to_cpu (ch), op, +NLS_MAX_CHARSET_SIZE );if (llen >0 ) op += llen;else
fs/jfs/jfs_unicode.c +3 -4
@@ -41,9 +41,8 @@ int jfs_strfromUCS_le(char *to, const __le16 * from,for (i =0 ; (i < len) && from[i]; i++) {int charlen; charlen = - codepage->uni2char (le16_to_cpu (from[i]), - &to[outlen], -NLS_MAX_CHARSET_SIZE ); +nls_uni2char (codepage,le16_to_cpu (from[i]), + &to[outlen],NLS_MAX_CHARSET_SIZE );if (charlen >0 ) outlen += charlen;else
@@ -88,7 +87,7 @@ static int jfs_strtoUCS(wchar_t * to, const unsigned char *from, int len,if (codepage) {for (i =0 ; len && *from; i++, from += charlen, len -= charlen) { - charlen = codepage->char2uni (from, len, &to[i]); + charlen =nls_char2uni (codepage, from, len, &to[i]);if (charlen <1 ) {jfs_err ("jfs_strtoUCS: char2uni returned %d." , charlen);
fs/nls/nls_euc-jp.c +2 -2
@@ -413,7 +413,7 @@ static int uni2char(const wchar_t uni,if (!p_nls)return -EINVAL ; -if ((n = p_nls->uni2char (uni, out, boundlen)) <0 ) +if ((n =nls_uni2char (p_nls, uni, out, boundlen)) <0 )return n;/* translate SJIS into EUC-JP */
@@ -543,7 +543,7 @@ static int char2uni(const unsigned char *rawstring, int boundlen, sjis_temp[1 ] =0x00 ; } -if ( (n = p_nls->char2uni (sjis_temp,sizeof (sjis_temp), uni)) <0 ) +if ( (n =nls_char2uni (p_nls, sjis_temp,sizeof (sjis_temp), uni)) <0 )return n;return euc_offset;
fs/nls/nls_koi8-ru.c +3 -3
@@ -28,12 +28,12 @@ static int uni2char(const wchar_t uni,else if (uni ==0x255d || uni ==0x256c )return 0 ;else -return p_nls->uni2char (uni, out, boundlen); +return nls_uni2char (p_nls, uni, out, boundlen);return 1 ; }else /* fast path */ -return p_nls->uni2char (uni, out, boundlen); +return nls_uni2char (p_nls, uni, out, boundlen); } static int char2uni(const unsigned char *rawstring, int boundlen,
@@ -47,7 +47,7 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return 1 ; } - n = p_nls->char2uni (rawstring, boundlen, uni); + n =nls_char2uni (p_nls, rawstring, boundlen, uni);return n; }
fs/ntfs/unistr.c +4 -4
@@ -269,8 +269,8 @@ int ntfs_nlstoucs(const ntfs_volume *vol, const char *ins, ucs =kmem_cache_alloc (ntfs_name_cache,GFP_NOFS );if (likely (ucs)) {for (i = o =0 ; i < ins_len; i += wc_len) { - wc_len = nls->char2uni (ins + i, ins_len - i, - &wc); + wc_len =nls_char2uni (nls, ins + i, + ins_len - i, &wc);if (likely (wc_len >=0 && o <NTFS_MAX_NAME_LEN )) {if (likely (wc)) {
@@ -355,8 +355,8 @@ int ntfs_ucstonls(const ntfs_volume *vol, const ntfschar *ins, goto mem_err_out; }for (i = o =0 ; i < ins_len; i++) { -retry: wc = nls->uni2char (le16_to_cpu (ins[i]), ns + o, - ns_len - o); +retry: wc =nls_uni2char (nls,le16_to_cpu (ins[i]), + ns + o, ns_len - o);if (wc >0 ) { o += wc;continue ;
include/linux/nls.h +13 -0
@@ -59,6 +59,19 @@ extern int utf8s_to_utf16s(const u8 *s, int len,extern int utf16s_to_utf8s (const wchar_t *pwcs,int len,enum utf16_endian endian,u8 *s,int maxlen); +static inline int nls_uni2char (const struct nls_table *table,wchar_t uni, +unsigned char *out,int boundlen) +{ +return table->uni2char (uni, out, boundlen); +} + +static inline int nls_char2uni (const struct nls_table *table, +const unsigned char *rawstring, +int boundlen,wchar_t *uni) +{ +return table->char2uni (rawstring, boundlen, uni); +} +static inline unsigned char nls_tolower (struct nls_table *t,unsigned char c) {unsigned char nc = t->charset2lower[c];
Signature · 1 line
2.17.0
PATCH 2/15 nls: Wrap charset field access
In reply to Gabriel Krisman BertaziWhich will simplify further patches. No behavior changes intended.
<smpl>
@@
struct nls_table *c;
@@
- c->charset
+ nls_charset_name(c)
</smpl>
--- drivers/staging/ncpfs/ioctl.c | 13 +++++++------ fs/befs/linuxvfs.c | 4 ++-- fs/cifs/cifs_unicode.c | 6 +++--- fs/cifs/cifsfs.c | 2 +- fs/cifs/connect.c | 2 +- fs/fat/inode.c | 6 ++++-- fs/hfs/super.c | 6 ++++-- fs/hfsplus/options.c | 2 +- fs/isofs/inode.c | 5 +++-- fs/jfs/jfs_unicode.c | 2 +- fs/jfs/super.c | 3 ++- fs/nls/nls_base.c | 2 +- fs/ntfs/inode.c | 2 +- fs/ntfs/super.c | 6 +++--- fs/ntfs/unistr.c | 5 +++-- fs/udf/super.c | 3 ++- include/linux/nls.h | 5 +++++ 17 files changed, 44 insertions(+), 30 deletions(-)
drivers/staging/ncpfs/ioctl.c +7 -6
@@ -231,21 +231,22 @@ ncp_get_charsets(struct ncp_server* server, struct ncp_nls_ioctl __user *arg)memset (&user,0 ,sizeof (user));mutex_lock (&server->root_setup_lock); -if (server->nls_vol && server->nls_vol->charset) { - len =strlen (server->nls_vol->charset); +if (server->nls_vol &&nls_charset_name (server->nls_vol)) { + len =strlen (nls_charset_name (server->nls_vol));if (len >NCP_IOCSNAME_LEN ) len =NCP_IOCSNAME_LEN ; -strncpy (user.codepage, server->nls_vol->charset, len); +strncpy (user.codepage,nls_charset_name (server->nls_vol), len); user.codepage[len] =0 ; }if (NCP_IS_FLAG (server,NCP_FLAG_UTF8 ))strcpy (user.iocharset,"utf8" ); -else if (server->nls_io && server->nls_io->charset) { - len =strlen (server->nls_io->charset); +else if (server->nls_io &&nls_charset_name (server->nls_io)) { + len =strlen (nls_charset_name (server->nls_io));if (len >NCP_IOCSNAME_LEN ) len =NCP_IOCSNAME_LEN ; -strncpy (user.iocharset, server->nls_io->charset, len); +strncpy (user.iocharset, +nls_charset_name (server->nls_io), len); user.iocharset[len] =0 ; }mutex_unlock (&server->root_setup_lock);
fs/befs/linuxvfs.c +2 -2
@@ -562,7 +562,7 @@ befs_utf2nls(struct super_block *sb, const char *in, conv_err:befs_error (sb,"Name using character set %s contains a character that " -"cannot be converted to unicode." , nls->charset); +"cannot be converted to unicode." ,nls_charset_name (nls));befs_debug (sb,"<--- %s" , __func__);kfree (result);return -EILSEQ ;
@@ -642,7 +642,7 @@ befs_nls2utf(struct super_block *sb, const char *in, conv_err:befs_error (sb,"Name using character set %s contains a character that " -"cannot be converted to unicode." , nls->charset); +"cannot be converted to unicode." ,nls_charset_name (nls));befs_debug (sb,"<--- %s" , __func__);kfree (result);return -EILSEQ ;
fs/cifs/cifs_unicode.c +3 -3
@@ -156,7 +156,7 @@ cifs_mapchar(char *target, const __u16 *from, const struct nls_table *cp, surrogate_pair:/* convert SURROGATE_PAIR and IVS */ -if (strcmp (cp->charset,"utf8" )) +if (strcmp (nls_charset_name (cp),"utf8" )) goto unknown; len =utf16s_to_utf8s (from,3 ,UTF16_LITTLE_ENDIAN , target,6 );if (len <=0 )
@@ -271,7 +271,7 @@ cifs_strtoUTF16(__le16 *to, const char *from, int len,wchar_t wchar_to;/* needed to quiet sparse */ /* special case for utf8 to handle no plane0 chars */ -if (!strcmp (codepage->charset,"utf8" )) { +if (!strcmp (nls_charset_name (codepage),"utf8" )) {/* * convert utf8 -> utf16, we assume we have enough space * as caller should have assumed conversion does not overflow
@@ -530,7 +530,7 @@ cifsConvertToUTF16(__le16 *target, const char *source, int srclen, goto ctoUTF16;/* convert SURROGATE_PAIR */ -if (strcmp (cp->charset,"utf8" ) || !wchar_to) +if (strcmp (nls_charset_name (cp),"utf8" ) || !wchar_to) goto unknown;if (*(source + i) &0x80 ) { charlen =utf8_to_utf32 (source + i,6 , &u);
fs/cifs/cifsfs.c +1 -1
@@ -389,7 +389,7 @@ cifs_show_nls(struct seq_file *s, struct nls_table *cur)/* Display iocharset= option if it's not default charset */ def =load_nls_default ();if (def != cur) -seq_printf (s,",iocharset=%s" , cur->charset); +seq_printf (s,",iocharset=%s" ,nls_charset_name (cur));unload_nls (def); }
fs/cifs/connect.c +1 -1
@@ -3092,7 +3092,7 @@ compare_mount_options(struct super_block *sb, struct cifs_mnt_data *mnt_data) old->mnt_dir_mode != new->mnt_dir_mode)return 0 ; -if (strcmp (old->local_nls->charset, new->local_nls->charset)) +if (strcmp (nls_charset_name (old->local_nls),nls_charset_name (new->local_nls)))return 0 ;if (old->actimeo != new->actimeo)
fs/fat/inode.c +4 -2
@@ -934,10 +934,12 @@ static int fat_show_options(struct seq_file *m, struct dentry *root)seq_printf (m,",allow_utime=%04o" , opts->allow_utime);if (sbi->nls_disk)/* strip "cp" prefix from displayed option */ -seq_printf (m,",codepage=%s" , &sbi->nls_disk->charset[2 ]); +seq_printf (m,",codepage=%s" , + &nls_charset_name (sbi->nls_disk)[2 ]);if (isvfat) {if (sbi->nls_io) -seq_printf (m,",iocharset=%s" , sbi->nls_io->charset); +seq_printf (m,",iocharset=%s" , +nls_charset_name (sbi->nls_io));switch (opts->shortname) {case VFAT_SFN_DISPLAY_WIN95 |VFAT_SFN_CREATE_WIN95 :
fs/hfs/super.c +4 -2
@@ -151,9 +151,11 @@ static int hfs_show_options(struct seq_file *seq, struct dentry *root)if (sbi->session >=0 )seq_printf (seq,",session=%u" , sbi->session);if (sbi->nls_disk) -seq_printf (seq,",codepage=%s" , sbi->nls_disk->charset); +seq_printf (seq,",codepage=%s" , +nls_charset_name (sbi->nls_disk));if (sbi->nls_io) -seq_printf (seq,",iocharset=%s" , sbi->nls_io->charset); +seq_printf (seq,",iocharset=%s" , +nls_charset_name (sbi->nls_io));if (sbi->s_quiet)seq_printf (seq,",quiet" );return 0 ;
fs/hfsplus/options.c +1 -1
@@ -230,7 +230,7 @@ int hfsplus_show_options(struct seq_file *seq, struct dentry *root)if (sbi->session >=0 )seq_printf (seq,",session=%u" , sbi->session);if (sbi->nls) -seq_printf (seq,",nls=%s" , sbi->nls->charset); +seq_printf (seq,",nls=%s" ,nls_charset_name (sbi->nls));if (test_bit (HFSPLUS_SB_NODECOMPOSE , &sbi->flags))seq_puts (seq,",nodecompose" );if (test_bit (HFSPLUS_SB_NOBARRIER , &sbi->flags))
fs/isofs/inode.c +3 -2
@@ -516,8 +516,9 @@ static int isofs_show_options(struct seq_file *m, struct dentry *root)#ifdef CONFIG_JOLIET if (sbi->s_nls_iocharset && -strcmp (sbi->s_nls_iocharset->charset,CONFIG_NLS_DEFAULT ) !=0 ) -seq_printf (m,",iocharset=%s" , sbi->s_nls_iocharset->charset); +strcmp (nls_charset_name (sbi->s_nls_iocharset),CONFIG_NLS_DEFAULT ) !=0 ) +seq_printf (m,",iocharset=%s" , +nls_charset_name (sbi->s_nls_iocharset));#endif return 0 ; }
fs/jfs/jfs_unicode.c +1 -1
@@ -92,7 +92,7 @@ static int jfs_strtoUCS(wchar_t * to, const unsigned char *from, int len,jfs_err ("jfs_strtoUCS: char2uni returned %d." , charlen);jfs_err ("charset = %s, char = 0x%x" , - codepage->charset, *from); +nls_charset_name (codepage), *from);return charlen; } }
fs/jfs/super.c +2 -1
@@ -736,7 +736,8 @@ static int jfs_show_options(struct seq_file *seq, struct dentry *root)if (sbi->flag &JFS_DISCARD )seq_printf (seq,",discard=%u" , sbi->minblks_trim);if (sbi->nls_tab) -seq_printf (seq,",iocharset=%s" , sbi->nls_tab->charset); +seq_printf (seq,",iocharset=%s" , +nls_charset_name (sbi->nls_tab));if (sbi->flag &JFS_ERR_CONTINUE )seq_printf (seq,",errors=continue" );if (sbi->flag &JFS_ERR_PANIC )
fs/nls/nls_base.c +1 -1
@@ -277,7 +277,7 @@ static struct nls_table *find_nls(char *charset)struct nls_table *nls;spin_lock (&nls_lock);for (nls = tables; nls; nls = nls->next) { -if (!strcmp (nls->charset, charset)) +if (!strcmp (nls_charset_name (nls), charset))break ;if (nls->alias && !strcmp (nls->alias, charset))break ;
fs/ntfs/inode.c +1 -1
@@ -2313,7 +2313,7 @@ int ntfs_show_options(struct seq_file *sf, struct dentry *root)seq_printf (sf,",fmask=0%o" , vol->fmask);seq_printf (sf,",dmask=0%o" , vol->dmask); } -seq_printf (sf,",nls=%s" , vol->nls_map->charset); +seq_printf (sf,",nls=%s" ,nls_charset_name (vol->nls_map));if (NVolCaseSensitive (vol))seq_printf (sf,",case_sensitive" );if (NVolShowSystemFiles (vol))
fs/ntfs/super.c +3 -3
@@ -224,7 +224,7 @@ static bool parse_options(ntfs_volume *vol, char *opt) }ntfs_error (vol->sb,"NLS character set %s not " "found. Using previous one %s." , - v, old_nls->charset); + v,nls_charset_name (old_nls)); nls_map = old_nls; }else /* nls_map */ {unload_nls (old_nls);
@@ -274,7 +274,7 @@ static bool parse_options(ntfs_volume *vol, char *opt) "on remount.");return false; }/* else (!vol->nls_map) */ -ntfs_debug ("Using NLS character set %s." , nls_map->charset); +ntfs_debug ("Using NLS character set %s." ,nls_charset_name (nls_map)); vol->nls_map = nls_map; }else /* (!nls_map) */ {if (!vol->nls_map) {
@@ -285,7 +285,7 @@ static bool parse_options(ntfs_volume *vol, char *opt)return false; }ntfs_debug ("Using default NLS character set (%s)." , - vol->nls_map->charset); +nls_charset_name (vol->nls_map)); } }if (mft_zone_multiplier !=-1 ) {
fs/ntfs/unistr.c +3 -2
@@ -297,7 +297,7 @@ int ntfs_nlstoucs(const ntfs_volume *vol, const char *ins,if (wc_len <0 ) {ntfs_error (vol->sb,"Name using character set %s contains " "characters that cannot be converted to " -"Unicode." , nls->charset); +"Unicode." ,nls_charset_name (nls)); i = -EILSEQ ; }else /* if (o >= NTFS_MAX_NAME_LEN) */ { ntfs_error(vol->sb, "Name is too long (maximum length for a "
@@ -386,7 +386,8 @@ retry: wc = nls_uni2char(nls, le16_to_cpu(ins[i]), conversion_err:ntfs_error (vol->sb,"Unicode name contains characters that cannot be " "converted to character set %s. You might want to " -"try to use the mount option nls=utf8." , nls->charset); +"try to use the mount option nls=utf8." , +nls_charset_name (nls));if (ns != *outs)kfree (ns);if (wc != -ENAMETOOLONG )
fs/udf/super.c +2 -1
@@ -378,7 +378,8 @@ static int udf_show_options(struct seq_file *seq, struct dentry *root)if (UDF_QUERY_FLAG (sb,UDF_FLAG_UTF8 ))seq_puts (seq,",utf8" );if (UDF_QUERY_FLAG (sb,UDF_FLAG_NLS_MAP ) && sbi->s_nls_map) -seq_printf (seq,",iocharset=%s" , sbi->s_nls_map->charset); +seq_printf (seq,",iocharset=%s" , +nls_charset_name (sbi->s_nls_map));return 0 ; }
include/linux/nls.h +5 -0
@@ -72,6 +72,11 @@ static inline int nls_char2uni(const struct nls_table *table,return table->char2uni (rawstring, boundlen, uni); } +static inline const char *nls_charset_name (const struct nls_table *table) +{ +return table->charset; +} +static inline unsigned char nls_tolower (struct nls_table *t,unsigned char c) {unsigned char nc = t->charset2lower[c];
Signature · 1 line
2.17.0
PATCH 3/15 nls: Wrap charset hooks in ops structure
In reply to Gabriel Krisman BertaziWith the exception of the struct declaration, this patch was generated by the following Coccinelle script:
<smpl>
@nlstable@
identifier p;
expression uni2char_fn;
expression char2uni_fn;
@@
static struct nls_table p = {
- .char2uni = char2uni_fn, - .uni2char = uni2char_fn, + .ops = &charset_ops,
};
@createops@
identifier nlstable.p;
expression nlstable.uni2char_fn;
expression nlstable.char2uni_fn;
@@
+static const struct nls_ops charset_ops = {
+ .uni2char = uni2char_fn,
+ .char2uni = char2uni_fn,
+};
+
static struct nls_table p = {};
@@
struct nls_table *c;
@@
(
- c->uni2char
+ c->ops->uni2char
|
- c->char2uni
+ c->ops->char2uni
)
</smpl>
--- fs/nls/mac-celtic.c | 8 ++++++-- fs/nls/mac-centeuro.c | 8 ++++++-- fs/nls/mac-croatian.c | 8 ++++++-- fs/nls/mac-cyrillic.c | 8 ++++++-- fs/nls/mac-gaelic.c | 8 ++++++-- fs/nls/mac-greek.c | 8 ++++++-- fs/nls/mac-iceland.c | 8 ++++++-- fs/nls/mac-inuit.c | 8 ++++++-- fs/nls/mac-roman.c | 8 ++++++-- fs/nls/mac-romanian.c | 8 ++++++-- fs/nls/mac-turkish.c | 8 ++++++-- fs/nls/nls_ascii.c | 8 ++++++-- fs/nls/nls_base.c | 8 ++++++-- fs/nls/nls_cp1250.c | 8 ++++++-- fs/nls/nls_cp1251.c | 8 ++++++-- fs/nls/nls_cp1255.c | 8 ++++++-- fs/nls/nls_cp437.c | 8 ++++++-- fs/nls/nls_cp737.c | 8 ++++++-- fs/nls/nls_cp775.c | 8 ++++++-- fs/nls/nls_cp850.c | 8 ++++++-- fs/nls/nls_cp852.c | 8 ++++++-- fs/nls/nls_cp855.c | 8 ++++++-- fs/nls/nls_cp857.c | 8 ++++++-- fs/nls/nls_cp860.c | 8 ++++++-- fs/nls/nls_cp861.c | 8 ++++++-- fs/nls/nls_cp862.c | 8 ++++++-- fs/nls/nls_cp863.c | 8 ++++++-- fs/nls/nls_cp864.c | 8 ++++++-- fs/nls/nls_cp865.c | 8 ++++++-- fs/nls/nls_cp866.c | 8 ++++++-- fs/nls/nls_cp869.c | 8 ++++++-- fs/nls/nls_cp874.c | 8 ++++++-- fs/nls/nls_cp932.c | 8 ++++++-- fs/nls/nls_cp936.c | 8 ++++++-- fs/nls/nls_cp949.c | 8 ++++++-- fs/nls/nls_cp950.c | 8 ++++++-- fs/nls/nls_euc-jp.c | 8 ++++++-- fs/nls/nls_iso8859-1.c | 8 ++++++-- fs/nls/nls_iso8859-13.c | 8 ++++++-- fs/nls/nls_iso8859-14.c | 8 ++++++-- fs/nls/nls_iso8859-15.c | 8 ++++++-- fs/nls/nls_iso8859-2.c | 8 ++++++-- fs/nls/nls_iso8859-3.c | 8 ++++++-- fs/nls/nls_iso8859-4.c | 8 ++++++-- fs/nls/nls_iso8859-5.c | 8 ++++++-- fs/nls/nls_iso8859-6.c | 8 ++++++-- fs/nls/nls_iso8859-7.c | 8 ++++++-- fs/nls/nls_iso8859-9.c | 8 ++++++-- fs/nls/nls_koi8-r.c | 8 ++++++-- fs/nls/nls_koi8-ru.c | 8 ++++++-- fs/nls/nls_koi8-u.c | 8 ++++++-- fs/nls/nls_utf8.c | 8 ++++++-- fs/udf/unicode.c | 4 ++-- include/linux/nls.h | 16 ++++++++++------ 54 files changed, 324 insertions(+), 112 deletions(-)
fs/nls/mac-celtic.c +6 -2
@@ -577,10 +577,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="macceltic" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/mac-centeuro.c +6 -2
@@ -507,10 +507,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="maccenteuro" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/mac-croatian.c +6 -2
@@ -577,10 +577,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="maccroatian" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/mac-cyrillic.c +6 -2
@@ -472,10 +472,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="maccyrillic" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/mac-gaelic.c +6 -2
@@ -542,10 +542,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="macgaelic" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/mac-greek.c +6 -2
@@ -472,10 +472,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="macgreek" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/mac-iceland.c +6 -2
@@ -577,10 +577,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="maciceland" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/mac-inuit.c +6 -2
@@ -507,10 +507,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="macinuit" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/mac-roman.c +6 -2
@@ -612,10 +612,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="macroman" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/mac-romanian.c +6 -2
@@ -577,10 +577,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="macromanian" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/mac-turkish.c +6 -2
@@ -577,10 +577,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="macturkish" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_ascii.c +6 -2
@@ -142,10 +142,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="ascii" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_base.c +6 -2
@@ -520,10 +520,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table default_table = { .charset ="default" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp1250.c +6 -2
@@ -323,10 +323,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp1250" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp1251.c +6 -2
@@ -277,10 +277,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp1251" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp1255.c +6 -2
@@ -358,11 +358,15 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp1255" , .alias ="iso8859-8" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp437.c +6 -2
@@ -363,10 +363,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp437" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp737.c +6 -2
@@ -326,10 +326,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp737" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp775.c +6 -2
@@ -295,10 +295,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp775" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp850.c +6 -2
@@ -291,10 +291,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp850" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp852.c +6 -2
@@ -313,10 +313,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp852" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp855.c +6 -2
@@ -275,10 +275,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp855" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp857.c +6 -2
@@ -277,10 +277,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp857" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp860.c +6 -2
@@ -340,10 +340,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp860" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp861.c +6 -2
@@ -363,10 +363,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp861" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp862.c +6 -2
@@ -397,10 +397,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp862" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp863.c +6 -2
@@ -357,10 +357,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp863" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp864.c +6 -2
@@ -383,10 +383,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp864" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp865.c +6 -2
@@ -363,10 +363,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp865" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp866.c +6 -2
@@ -281,10 +281,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp866" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp869.c +6 -2
@@ -291,10 +291,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp869" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp874.c +6 -2
@@ -249,11 +249,15 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp874" , .alias ="tis-620" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp932.c +6 -2
@@ -7907,11 +7907,15 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return -EINVAL ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp932" , .alias ="sjis" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp936.c +6 -2
@@ -11085,11 +11085,15 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return n; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp936" , .alias ="gb2312" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp949.c +6 -2
@@ -13920,11 +13920,15 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return n; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp949" , .alias ="euc-kr" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_cp950.c +6 -2
@@ -9456,11 +9456,15 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return n; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="cp950" , .alias ="big5" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_euc-jp.c +6 -2
@@ -549,10 +549,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return euc_offset; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="euc-jp" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, };static int __init init_nls_euc_jp (void )
fs/nls/nls_iso8859-1.c +6 -2
@@ -233,10 +233,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-1" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_iso8859-13.c +6 -2
@@ -261,10 +261,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-13" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_iso8859-14.c +6 -2
@@ -317,10 +317,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-14" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_iso8859-15.c +6 -2
@@ -283,10 +283,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-15" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_iso8859-2.c +6 -2
@@ -284,10 +284,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-2" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_iso8859-3.c +6 -2
@@ -284,10 +284,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-3" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_iso8859-4.c +6 -2
@@ -284,10 +284,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-4" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_iso8859-5.c +6 -2
@@ -248,10 +248,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-5" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_iso8859-6.c +6 -2
@@ -239,10 +239,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-6" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_iso8859-7.c +6 -2
@@ -293,10 +293,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-7" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_iso8859-9.c +6 -2
@@ -248,10 +248,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="iso8859-9" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_koi8-r.c +6 -2
@@ -299,10 +299,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="koi8-r" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_koi8-ru.c +6 -2
@@ -51,10 +51,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return n; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="koi8-ru" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, };static int __init init_nls_koi8_ru (void )
fs/nls/nls_koi8-u.c +6 -2
@@ -306,10 +306,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="koi8-u" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, };
fs/nls/nls_utf8.c +6 -2
@@ -40,10 +40,14 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return n; } +static const struct nls_ops charset_ops = { + .uni2char = uni2char, + .char2uni = char2uni, +}; +static struct nls_table table = { .charset ="utf8" , - .uni2char = uni2char, - .char2uni = char2uni, + .ops = &charset_ops, .charset2lower = identity,/* no conversion */ .charset2upper = identity, };
fs/udf/unicode.c +2 -2
@@ -368,7 +368,7 @@ int udf_get_filename(struct super_block *sb, const uint8_t *sname, int slen,if (UDF_QUERY_FLAG (sb,UDF_FLAG_UTF8 )) { conv_f = udf_uni2char_utf8; }else if (UDF_QUERY_FLAG (sb,UDF_FLAG_NLS_MAP )) { - conv_f =UDF_SB (sb)->s_nls_map->uni2char; + conv_f =UDF_SB (sb)->s_nls_map->ops->uni2char; }else BUG ();
@@ -387,7 +387,7 @@ int udf_put_filename(struct super_block *sb, const uint8_t *sname, int slen,if (UDF_QUERY_FLAG (sb,UDF_FLAG_UTF8 )) { conv_f = udf_char2uni_utf8; }else if (UDF_QUERY_FLAG (sb,UDF_FLAG_NLS_MAP )) { - conv_f =UDF_SB (sb)->s_nls_map->char2uni; + conv_f =UDF_SB (sb)->s_nls_map->ops->char2uni; }else BUG ();
include/linux/nls.h +10 -6
@@ -22,12 +22,16 @@ typedef u16 wchar_t;/* Arbitrary Unicode character */ typedef u32 unicode_t ; -struct nls_table { -const char *charset; -const char *alias; +struct nls_ops {int (*uni2char) (wchar_t uni,unsigned char *out,int boundlen);int (*char2uni) (const unsigned char *rawstring,int boundlen,wchar_t *uni); +}; + +struct nls_table { +const char *charset; +const char *alias; +const struct nls_ops *ops;const unsigned char *charset2lower;const unsigned char *charset2upper;struct module *owner;
@@ -62,14 +66,14 @@ extern int utf16s_to_utf8s(const wchar_t *pwcs, int len,static inline int nls_uni2char (const struct nls_table *table,wchar_t uni,unsigned char *out,int boundlen) { -return table->uni2char (uni, out, boundlen); +return table->ops->uni2char (uni, out, boundlen); }static inline int nls_char2uni (const struct nls_table *table,const unsigned char *rawstring,int boundlen,wchar_t *uni) { -return table->char2uni (rawstring, boundlen, uni); +return table->ops->char2uni (rawstring, boundlen, uni); }static inline const char *nls_charset_name (const struct nls_table *table)
@@ -116,7 +120,7 @@ nls_nullsize(const struct nls_table *codepage)int charlen;char tmp[NLS_MAX_CHARSET_SIZE ]; - charlen = codepage->uni2char (0 , tmp,NLS_MAX_CHARSET_SIZE ); + charlen = codepage->ops->uni2char (0 , tmp,NLS_MAX_CHARSET_SIZE );return charlen >0 ? charlen :1 ; }
Signature · 1 line
2.17.0
PATCH 4/15 nls: Split default charset from NLS core
In reply to Gabriel Krisman Bertazi--- fs/nls/Makefile | 2 +- fs/nls/nls_base.c | 458 ----------------------------------------- fs/nls/nls_default.c | 473 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 474 insertions(+), 459 deletions(-) create mode 100644 fs/nls/nls_default.c
fs/nls/Makefile +1 -1
@@ -3,7 +3,7 @@ # Makefile for native language support # -obj-$(CONFIG_NLS) += nls_base.o +obj-$(CONFIG_NLS) += nls_base.o nls_default.o obj-$(CONFIG_NLS_CODEPAGE_437) += nls_cp437.o obj-$(CONFIG_NLS_CODEPAGE_737) += nls_cp737.o
fs/nls/nls_base.c +0 -458
477 lines not on this page · raw message
fs/nls/nls_default.c new file +473 -0
473 lines not on this page · raw message
Signature · 1 line
2.17.0
kbuild test robot
build botRe: [PATCH 04/15] nls: Split default charset from NLS core
Thank you for the patch! Perhaps something to improve:
Hi Gabriel,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v4.17-rc4]
[cannot apply to next-20180509]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Gabriel-Krisman-Bertazi/NLS-refactor-and-UTF-8-normalization/20180509-190115
reproduce:
# apt-get install sparse
make ARCH=x86_64 allmodconfig
make C=1 CF=-D__CHECK_ENDIAN__sparse warnings: (new ones prefixed by >>)
>> fs/nls/nls_default.c:118:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short [unsigned] [short] [usertype] <noident> @@ got unsigned] [short] [usertype] <noident> @@ fs/nls/nls_default.c:118:20: expected unsigned short [unsigned] [short] [usertype] <noident> fs/nls/nls_default.c:118:20: got restricted __le16 [usertype] <noident> fs/nls/nls_default.c:121:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short [unsigned] [short] [usertype] <noident> @@ got unsigned] [short] [usertype] <noident> @@ fs/nls/nls_default.c:121:20: expected unsigned short [unsigned] [short] [usertype] <noident> fs/nls/nls_default.c:121:20: got restricted __be16 [usertype] <noident> >> fs/nls/nls_default.c:118:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short [unsigned] [short] [usertype] <noident> @@ got unsigned] [short] [usertype] <noident> @@ fs/nls/nls_default.c:118:20: expected unsigned short [unsigned] [short] [usertype] <noident> fs/nls/nls_default.c:118:20: got restricted __le16 [usertype] <noident> fs/nls/nls_default.c:121:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short [unsigned] [short] [usertype] <noident> @@ got unsigned] [short] [usertype] <noident> @@ fs/nls/nls_default.c:121:20: expected unsigned short [unsigned] [short] [usertype] <noident> fs/nls/nls_default.c:121:20: got restricted __be16 [usertype] <noident> >> fs/nls/nls_default.c:118:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short [unsigned] [short] [usertype] <noident> @@ got unsigned] [short] [usertype] <noident> @@ fs/nls/nls_default.c:118:20: expected unsigned short [unsigned] [short] [usertype] <noident> fs/nls/nls_default.c:118:20: got restricted __le16 [usertype] <noident> fs/nls/nls_default.c:121:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short [unsigned] [short] [usertype] <noident> @@ got unsigned] [short] [usertype] <noident> @@ fs/nls/nls_default.c:121:20: expected unsigned short [unsigned] [short] [usertype] <noident> fs/nls/nls_default.c:121:20: got restricted __be16 [usertype] <noident> >> fs/nls/nls_default.c:118:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short [unsigned] [short] [usertype] <noident> @@ got unsigned] [short] [usertype] <noident> @@ fs/nls/nls_default.c:118:20: expected unsigned short [unsigned] [short] [usertype] <noident> fs/nls/nls_default.c:118:20: got restricted __le16 [usertype] <noident> fs/nls/nls_default.c:121:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short [unsigned] [short] [usertype] <noident> @@ got unsigned] [short] [usertype] <noident> @@ fs/nls/nls_default.c:121:20: expected unsigned short [unsigned] [short] [usertype] <noident> fs/nls/nls_default.c:121:20: got restricted __be16 [usertype] <noident> >> fs/nls/nls_default.c:174:24: sparse: cast to restricted __le16 >> fs/nls/nls_default.c:176:24: sparse: cast to restricted __be16 >> fs/nls/nls_default.c:176:24: sparse: cast to restricted __be16 >> fs/nls/nls_default.c:176:24: sparse: cast to restricted __be16 >> fs/nls/nls_default.c:176:24: sparse: cast to restricted __be16 >> fs/nls/nls_default.c:174:24: sparse: cast to restricted __le16 >> fs/nls/nls_default.c:176:24: sparse: cast to restricted __be16 >> fs/nls/nls_default.c:176:24: sparse: cast to restricted __be16 >> fs/nls/nls_default.c:176:24: sparse: cast to restricted __be16 >> fs/nls/nls_default.c:176:24: sparse: cast to restricted __be16
vim +118 fs/nls/nls_default.c
110
111 static inline void put_utf16(wchar_t *s, unsigned c, enum utf16_endian endian)
112 {
113 switch (endian) {
114 default:
115 *s = (wchar_t) c;
116 break;
117 case UTF16_LITTLE_ENDIAN:
> 118 *s = __cpu_to_le16(c);
119 break;
120 case UTF16_BIG_ENDIAN:
> 121 *s = __cpu_to_be16(c);
122 break;
123 }
124 }
125
126 int utf8s_to_utf16s(const u8 *s, int inlen, enum utf16_endian endian,
127 wchar_t *pwcs, int maxout)
128 {
129 u16 *op;
130 int size;
131 unicode_t u;
132
133 op = pwcs;
134 while (inlen > 0 && maxout > 0 && *s) {
135 if (*s & 0x80) {
136 size = utf8_to_utf32(s, inlen, &u);
137 if (size < 0)
138 return -EINVAL;
139 s += size;
140 inlen -= size;
141
142 if (u >= PLANE_SIZE) {
143 if (maxout < 2)
144 break;
145 u -= PLANE_SIZE;
146 put_utf16(op++, SURROGATE_PAIR |
147 ((u >> 10) & SURROGATE_BITS),
148 endian);
149 put_utf16(op++, SURROGATE_PAIR |
150 SURROGATE_LOW |
151 (u & SURROGATE_BITS),
152 endian);
153 maxout -= 2;
154 } else {
155 put_utf16(op++, u, endian);
156 maxout--;
157 }
158 } else {
159 put_utf16(op++, *s++, endian);
160 inlen--;
161 maxout--;
162 }
163 }
164 return op - pwcs;
165 }
166 EXPORT_SYMBOL(utf8s_to_utf16s);
167
168 static inline unsigned long get_utf16(unsigned c, enum utf16_endian endian)
169 {
170 switch (endian) {
171 default:
172 return c;
173 case UTF16_LITTLE_ENDIAN:
> 174 return __le16_to_cpu(c);
175 case UTF16_BIG_ENDIAN:
> 176 return __be16_to_cpu(c); 177 }
178 }
179
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
Re: [PATCH 04/15] nls: Split default charset from NLS core
Quoted 5 lines · kbuild test robot
sparse warnings: (new ones prefixed by >>)
Quoted 3 lines
Quoted 3 lines
fs/nls/nls_default.c:118:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short [unsigned] [short] [usertype] <noident> @@ got unsigned] [short] [usertype] <noident> @@
Quoted 7 lines
fs/nls/nls_default.c:118:20: expected unsigned short [unsigned] [short] [usertype] <noident>
fs/nls/nls_default.c:118:20: got restricted__le16[usertype] <noident>
fs/nls/nls_default.c:121:20: sparse: incorrect type in assignment
(different base types) @@ expected unsigned short [unsigned] [short] [usertype] <noident> @@ got unsigned] [short] [usertype] <noident> @@
fs/nls/nls_default.c:121:20: expected unsigned short [unsigned] [short] [usertype] <noident>
fs/nls/nls_default.c:121:20: got restricted__be16[usertype] <noident>
[snip.]
This is not caused by my patch, but it is identified because I am moving code around. I don't plan to fix it in my patch set at this moment.
Signature · 1 line
Gabriel Krisman Bertazi
PATCH 5/15 nls: Split struct nls_charset from struct nls_table
In reply to Gabriel Krisman Bertazistruct nls_charset carries only the information required to register the charset with the NLS core and generate the nls_table on demand. nls_table is what is given to users to manipulate the encoding.
With the exception of the following files (files declaration and default charset), which were edited by hand, the other files were generated with the following Coccinelle patch:
Files edited by hand:
- fs/nls/nls_base.c
- include/linux/nls.h
- fs/nls/nls_default.c
<smpl>
@nlstable@
identifier p;
expression charset_str;
@@
static struct nls_table p = {
- .charset = charset_str,
};
@createops@
identifier nlstable.p;
expression nlstable.charset_str;
@@
+static struct nls_charset nls_charset;
static struct nls_table p = {
+ .charset = &nls_charset,
};
+
+static struct nls_charset nls_charset = {
+ .charset = charset_str,
+ .tables = &p,
+};
@@
expression A;
@@
? return
- register_nls(A);
+ register_nls(&nls_charset);
@@
expression A;
@@
- unregister_nls(A);
+ unregister_nls(&nls_charset);
@mvalias@
identifier p;
expression alias_str;
@@
static struct nls_table p = {
- .alias = alias_str,
};
@@
expression mvalias.alias_str;
@@
static struct nls_charset nls_charset = {
+ .alias = alias_str,
};
</smpl>
--- fs/nls/mac-celtic.c | 12 +++++++++--- fs/nls/mac-centeuro.c | 12 +++++++++--- fs/nls/mac-croatian.c | 12 +++++++++--- fs/nls/mac-cyrillic.c | 12 +++++++++--- fs/nls/mac-gaelic.c | 12 +++++++++--- fs/nls/mac-greek.c | 12 +++++++++--- fs/nls/mac-iceland.c | 12 +++++++++--- fs/nls/mac-inuit.c | 12 +++++++++--- fs/nls/mac-roman.c | 12 +++++++++--- fs/nls/mac-romanian.c | 12 +++++++++--- fs/nls/mac-turkish.c | 12 +++++++++--- fs/nls/nls_ascii.c | 12 +++++++++--- fs/nls/nls_base.c | 40 ++++++++++++++++++++++++++-------------- fs/nls/nls_cp1250.c | 12 +++++++++--- fs/nls/nls_cp1251.c | 12 +++++++++--- fs/nls/nls_cp1255.c | 14 ++++++++++---- fs/nls/nls_cp437.c | 12 +++++++++--- fs/nls/nls_cp737.c | 12 +++++++++--- fs/nls/nls_cp775.c | 12 +++++++++--- fs/nls/nls_cp850.c | 12 +++++++++--- fs/nls/nls_cp852.c | 12 +++++++++--- fs/nls/nls_cp855.c | 12 +++++++++--- fs/nls/nls_cp857.c | 12 +++++++++--- fs/nls/nls_cp860.c | 12 +++++++++--- fs/nls/nls_cp861.c | 12 +++++++++--- fs/nls/nls_cp862.c | 12 +++++++++--- fs/nls/nls_cp863.c | 12 +++++++++--- fs/nls/nls_cp864.c | 12 +++++++++--- fs/nls/nls_cp865.c | 12 +++++++++--- fs/nls/nls_cp866.c | 12 +++++++++--- fs/nls/nls_cp869.c | 12 +++++++++--- fs/nls/nls_cp874.c | 14 ++++++++++---- fs/nls/nls_cp932.c | 14 ++++++++++---- fs/nls/nls_cp936.c | 14 ++++++++++---- fs/nls/nls_cp949.c | 14 ++++++++++---- fs/nls/nls_cp950.c | 14 ++++++++++---- fs/nls/nls_default.c | 9 +++++++-- fs/nls/nls_euc-jp.c | 12 +++++++++--- fs/nls/nls_iso8859-1.c | 12 +++++++++--- fs/nls/nls_iso8859-13.c | 12 +++++++++--- fs/nls/nls_iso8859-14.c | 12 +++++++++--- fs/nls/nls_iso8859-15.c | 12 +++++++++--- fs/nls/nls_iso8859-2.c | 12 +++++++++--- fs/nls/nls_iso8859-3.c | 12 +++++++++--- fs/nls/nls_iso8859-4.c | 12 +++++++++--- fs/nls/nls_iso8859-5.c | 12 +++++++++--- fs/nls/nls_iso8859-6.c | 12 +++++++++--- fs/nls/nls_iso8859-7.c | 12 +++++++++--- fs/nls/nls_iso8859-9.c | 12 +++++++++--- fs/nls/nls_koi8-r.c | 12 +++++++++--- fs/nls/nls_koi8-ru.c | 12 +++++++++--- fs/nls/nls_koi8-u.c | 12 +++++++++--- fs/nls/nls_utf8.c | 12 +++++++++--- include/linux/nls.h | 18 ++++++++++++------ 54 files changed, 510 insertions(+), 181 deletions(-)
fs/nls/mac-celtic.c +9 -3
@@ -582,21 +582,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="macceltic" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="macceltic" , + .tables = &table, +}; +static int __init init_nls_macceltic (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_macceltic(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_macceltic)
fs/nls/mac-centeuro.c +9 -3
@@ -512,21 +512,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="maccenteuro" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="maccenteuro" , + .tables = &table, +}; +static int __init init_nls_maccenteuro (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_maccenteuro(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_maccenteuro)
fs/nls/mac-croatian.c +9 -3
@@ -582,21 +582,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="maccroatian" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="maccroatian" , + .tables = &table, +}; +static int __init init_nls_maccroatian (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_maccroatian(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_maccroatian)
fs/nls/mac-cyrillic.c +9 -3
@@ -477,21 +477,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="maccyrillic" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="maccyrillic" , + .tables = &table, +}; +static int __init init_nls_maccyrillic (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_maccyrillic(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_maccyrillic)
fs/nls/mac-gaelic.c +9 -3
@@ -547,21 +547,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="macgaelic" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="macgaelic" , + .tables = &table, +}; +static int __init init_nls_macgaelic (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_macgaelic(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_macgaelic)
fs/nls/mac-greek.c +9 -3
@@ -477,21 +477,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="macgreek" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="macgreek" , + .tables = &table, +}; +static int __init init_nls_macgreek (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_macgreek(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_macgreek)
fs/nls/mac-iceland.c +9 -3
@@ -582,21 +582,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="maciceland" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="maciceland" , + .tables = &table, +}; +static int __init init_nls_maciceland (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_maciceland(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_maciceland)
fs/nls/mac-inuit.c +9 -3
@@ -512,21 +512,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="macinuit" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="macinuit" , + .tables = &table, +}; +static int __init init_nls_macinuit (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_macinuit(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_macinuit)
fs/nls/mac-roman.c +9 -3
@@ -617,21 +617,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="macroman" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="macroman" , + .tables = &table, +}; +static int __init init_nls_macroman (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_macroman(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_macroman)
fs/nls/mac-romanian.c +9 -3
@@ -582,21 +582,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="macromanian" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="macromanian" , + .tables = &table, +}; +static int __init init_nls_macromanian (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_macromanian(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_macromanian)
fs/nls/mac-turkish.c +9 -3
@@ -582,21 +582,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="macturkish" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="macturkish" , + .tables = &table, +}; +static int __init init_nls_macturkish (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_macturkish(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_macturkish)
fs/nls/nls_ascii.c +9 -3
@@ -147,21 +147,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="ascii" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="ascii" , + .tables = &table, +}; +static int __init init_nls_ascii (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_ascii(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_ascii)
fs/nls/nls_base.c +26 -14
@@ -16,13 +16,18 @@#include <linux/kmod.h> #include <linux/spinlock.h> -static struct nls_table default_table; -static struct nls_table *tables = &default_table; +extern struct nls_charset default_charset; +static struct nls_charset *charsets = &default_charset;static DEFINE_SPINLOCK (nls_lock); +static struct nls_table *nls_load_table (struct nls_charset *charset) +{ +/* For now, return the default table, which is the first one found. */ +return charset->tables; +} -int __register_nls (struct nls_table *nls,struct module *owner) +int __register_nls (struct nls_charset *nls,struct module *owner) { -struct nls_table ** tmp = &tables; +struct nls_charset **tmp = &charsets;if (nls->next)return -EBUSY ;
@@ -36,16 +41,16 @@ int __register_nls(struct nls_table *nls, struct module *owner) } tmp = &(*tmp)->next; } - nls->next = tables; - tables = nls; + nls->next = charsets; + charsets = nls;spin_unlock (&nls_lock);return 0 ; }EXPORT_SYMBOL (__register_nls); -int unregister_nls (struct nls_table * nls) +int unregister_nls (struct nls_charset * nls) { -struct nls_table ** tmp = &tables; +struct nls_charset **tmp = &charsets;spin_lock (&nls_lock);while (*tmp) {
@@ -60,12 +65,12 @@ int unregister_nls(struct nls_table * nls)return -EINVAL ; } -static struct nls_table *find_nls (char *charset) +static struct nls_charset *find_nls (char *charset) { -struct nls_table *nls; +struct nls_charset *nls;spin_lock (&nls_lock); -for (nls = tables; nls; nls = nls->next) { -if (!strcmp (nls_charset_name (nls), charset)) +for (nls = charsets; nls; nls = nls->next) { +if (!strcmp (nls->charset, charset))break ;if (nls->alias && !strcmp (nls->alias, charset))break ;
@@ -78,13 +83,20 @@ static struct nls_table *find_nls(char *charset)struct nls_table *load_nls (char *charset) { -return try_then_request_module (find_nls (charset),"nls_%s" , charset); +struct nls_charset *nls_charset; + + nls_charset =try_then_request_module (find_nls (charset), +"nls_%s" , charset); +if (!nls_charset) +return NULL ; + +return nls_load_table (nls_charset); }void unload_nls (struct nls_table *nls) {if (nls) -module_put (nls->owner); +module_put (nls->charset->owner); }EXPORT_SYMBOL (unregister_nls);
fs/nls/nls_cp1250.c +9 -3
@@ -328,20 +328,26 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp1250" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp1250" , + .tables = &table, +}; +static int __init init_nls_cp1250 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp1250(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp1250)
fs/nls/nls_cp1251.c +9 -3
@@ -282,21 +282,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp1251" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp1251" , + .tables = &table, +}; +static int __init init_nls_cp1251 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp1251(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp1251)
fs/nls/nls_cp1255.c +10 -4
@@ -363,22 +363,28 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp1255" , - .alias ="iso8859-8" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .alias ="iso8859-8" , + .charset ="cp1255" , + .tables = &table, +}; +static int __init init_nls_cp1255 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp1255(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp1255)
fs/nls/nls_cp437.c +9 -3
@@ -368,21 +368,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp437" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp437" , + .tables = &table, +}; +static int __init init_nls_cp437 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp437(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp437)
fs/nls/nls_cp737.c +9 -3
@@ -331,21 +331,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp737" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp737" , + .tables = &table, +}; +static int __init init_nls_cp737 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp737(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp737)
fs/nls/nls_cp775.c +9 -3
@@ -300,21 +300,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp775" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp775" , + .tables = &table, +}; +static int __init init_nls_cp775 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp775(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp775)
fs/nls/nls_cp850.c +9 -3
@@ -296,21 +296,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp850" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp850" , + .tables = &table, +}; +static int __init init_nls_cp850 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp850(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp850)
fs/nls/nls_cp852.c +9 -3
@@ -318,21 +318,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp852" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp852" , + .tables = &table, +}; +static int __init init_nls_cp852 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp852(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp852)
fs/nls/nls_cp855.c +9 -3
@@ -280,21 +280,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp855" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp855" , + .tables = &table, +}; +static int __init init_nls_cp855 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp855(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp855)
fs/nls/nls_cp857.c +9 -3
@@ -282,21 +282,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp857" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp857" , + .tables = &table, +}; +static int __init init_nls_cp857 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp857(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp857)
fs/nls/nls_cp860.c +9 -3
@@ -345,21 +345,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp860" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp860" , + .tables = &table, +}; +static int __init init_nls_cp860 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp860(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp860)
fs/nls/nls_cp861.c +9 -3
@@ -368,21 +368,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp861" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp861" , + .tables = &table, +}; +static int __init init_nls_cp861 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp861(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp861)
fs/nls/nls_cp862.c +9 -3
@@ -402,21 +402,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp862" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp862" , + .tables = &table, +}; +static int __init init_nls_cp862 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp862(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp862)
fs/nls/nls_cp863.c +9 -3
@@ -362,21 +362,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp863" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp863" , + .tables = &table, +}; +static int __init init_nls_cp863 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp863(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp863)
fs/nls/nls_cp864.c +9 -3
@@ -388,21 +388,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp864" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp864" , + .tables = &table, +}; +static int __init init_nls_cp864 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp864(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp864)
fs/nls/nls_cp865.c +9 -3
@@ -368,21 +368,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp865" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp865" , + .tables = &table, +}; +static int __init init_nls_cp865 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp865(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp865)
fs/nls/nls_cp866.c +9 -3
@@ -286,21 +286,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp866" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp866" , + .tables = &table, +}; +static int __init init_nls_cp866 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp866(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp866)
fs/nls/nls_cp869.c +9 -3
@@ -296,21 +296,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp869" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="cp869" , + .tables = &table, +}; +static int __init init_nls_cp869 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp869(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp869)
fs/nls/nls_cp874.c +10 -4
@@ -254,22 +254,28 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp874" , - .alias ="tis-620" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .alias ="tis-620" , + .charset ="cp874" , + .tables = &table, +}; +static int __init init_nls_cp874 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp874(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp874)
fs/nls/nls_cp932.c +10 -4
@@ -7912,22 +7912,28 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp932" , - .alias ="sjis" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .alias ="sjis" , + .charset ="cp932" , + .tables = &table, +}; +static int __init init_nls_cp932 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp932(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp932)
fs/nls/nls_cp936.c +10 -4
@@ -11090,22 +11090,28 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp936" , - .alias ="gb2312" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .alias ="gb2312" , + .charset ="cp936" , + .tables = &table, +}; +static int __init init_nls_cp936 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp936(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp936)
fs/nls/nls_cp949.c +10 -4
@@ -13925,22 +13925,28 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp949" , - .alias ="euc-kr" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .alias ="euc-kr" , + .charset ="cp949" , + .tables = &table, +}; +static int __init init_nls_cp949 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp949(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp949)
fs/nls/nls_cp950.c +10 -4
@@ -9461,22 +9461,28 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="cp950" , - .alias ="big5" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .alias ="big5" , + .charset ="cp950" , + .tables = &table, +}; +static int __init init_nls_cp950 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_cp950(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_cp950)
fs/nls/nls_default.c +7 -2
@@ -17,7 +17,7 @@#include <asm/byteorder.h> #include <linux/nls.h> -static struct nls_table default_table; +struct nls_charset default_charset;struct utf8_table {int cmask;
@@ -453,12 +453,17 @@ static const struct nls_ops charset_ops = { };static struct nls_table default_table = { - .charset ="default" , + .charset = &default_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +struct nls_charset default_charset = { + .charset ="default" , + .tables = &default_table, +}; +/* Returns a simple default translation table */ struct nls_table *load_nls_default (void ) {
fs/nls/nls_euc-jp.c +9 -3
@@ -554,11 +554,17 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="euc-jp" , + .charset = &nls_charset, .ops = &charset_ops, }; +static struct nls_charset nls_charset = { + .charset ="euc-jp" , + .tables = &table, +}; +static int __init init_nls_euc_jp (void ) { p_nls =load_nls ("cp932" );
@@ -566,7 +572,7 @@ static int __init init_nls_euc_jp(void)if (p_nls) { table.charset2upper = p_nls->charset2upper; table.charset2lower = p_nls->charset2lower; -return register_nls (&table); +return register_nls (&nls_charset); }return -EINVAL ;
@@ -574,7 +580,7 @@ static int __init init_nls_euc_jp(void) static void __exit exit_nls_euc_jp(void) { -unregister_nls (&table); +unregister_nls (&nls_charset);unload_nls (p_nls); }
fs/nls/nls_iso8859-1.c +9 -3
@@ -238,21 +238,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-1" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-1" , + .tables = &table, +}; +static int __init init_nls_iso8859_1 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_1(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_1)
fs/nls/nls_iso8859-13.c +9 -3
@@ -266,21 +266,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-13" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-13" , + .tables = &table, +}; +static int __init init_nls_iso8859_13 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_13(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_13)
fs/nls/nls_iso8859-14.c +9 -3
@@ -322,21 +322,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-14" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-14" , + .tables = &table, +}; +static int __init init_nls_iso8859_14 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_14(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_14)
fs/nls/nls_iso8859-15.c +9 -3
@@ -288,21 +288,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-15" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-15" , + .tables = &table, +}; +static int __init init_nls_iso8859_15 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_15(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_15)
fs/nls/nls_iso8859-2.c +9 -3
@@ -289,21 +289,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-2" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-2" , + .tables = &table, +}; +static int __init init_nls_iso8859_2 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_2(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_2)
fs/nls/nls_iso8859-3.c +9 -3
@@ -289,21 +289,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-3" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-3" , + .tables = &table, +}; +static int __init init_nls_iso8859_3 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_3(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_3)
fs/nls/nls_iso8859-4.c +9 -3
@@ -289,21 +289,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-4" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-4" , + .tables = &table, +}; +static int __init init_nls_iso8859_4 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_4(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_4)
fs/nls/nls_iso8859-5.c +9 -3
@@ -253,21 +253,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-5" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-5" , + .tables = &table, +}; +static int __init init_nls_iso8859_5 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_5(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_5)
fs/nls/nls_iso8859-6.c +9 -3
@@ -244,21 +244,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-6" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-6" , + .tables = &table, +}; +static int __init init_nls_iso8859_6 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_6(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_6)
fs/nls/nls_iso8859-7.c +9 -3
@@ -298,21 +298,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-7" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-7" , + .tables = &table, +}; +static int __init init_nls_iso8859_7 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_7(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_7)
fs/nls/nls_iso8859-9.c +9 -3
@@ -253,21 +253,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="iso8859-9" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="iso8859-9" , + .tables = &table, +}; +static int __init init_nls_iso8859_9 (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_iso8859_9(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_iso8859_9)
fs/nls/nls_koi8-r.c +9 -3
@@ -304,21 +304,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="koi8-r" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="koi8-r" , + .tables = &table, +}; +static int __init init_nls_koi8_r (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_koi8_r(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_koi8_r)
fs/nls/nls_koi8-ru.c +9 -3
@@ -56,11 +56,17 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="koi8-ru" , + .charset = &nls_charset, .ops = &charset_ops, }; +static struct nls_charset nls_charset = { + .charset ="koi8-ru" , + .tables = &table, +}; +static int __init init_nls_koi8_ru (void ) { p_nls =load_nls ("koi8-u" );
@@ -68,7 +74,7 @@ static int __init init_nls_koi8_ru(void)if (p_nls) { table.charset2upper = p_nls->charset2upper; table.charset2lower = p_nls->charset2lower; -return register_nls (&table); +return register_nls (&nls_charset); }return -EINVAL ;
@@ -76,7 +82,7 @@ static int __init init_nls_koi8_ru(void) static void __exit exit_nls_koi8_ru(void) { -unregister_nls (&table); +unregister_nls (&nls_charset);unload_nls (p_nls); }
fs/nls/nls_koi8-u.c +9 -3
@@ -311,21 +311,27 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="koi8-u" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = charset2lower, .charset2upper = charset2upper, }; +static struct nls_charset nls_charset = { + .charset ="koi8-u" , + .tables = &table, +}; +static int __init init_nls_koi8_u (void ) { -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_koi8_u(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_koi8_u)
fs/nls/nls_utf8.c +9 -3
@@ -45,25 +45,31 @@ static const struct nls_ops charset_ops = { .char2uni = char2uni, }; +static struct nls_charset nls_charset;static struct nls_table table = { - .charset ="utf8" , + .charset = &nls_charset, .ops = &charset_ops, .charset2lower = identity,/* no conversion */ .charset2upper = identity, }; +static struct nls_charset nls_charset = { + .charset ="utf8" , + .tables = &table, +}; +static int __init init_nls_utf8 (void ) {int i;for (i=0 ; i<256 ; i++) identity[i] = i; -return register_nls (&table); +return register_nls (&nls_charset); } static void __exit exit_nls_utf8(void) { -unregister_nls (&table); +unregister_nls (&nls_charset); }module_init (init_nls_utf8)
include/linux/nls.h +12 -6
@@ -29,15 +29,21 @@ struct nls_ops { };struct nls_table { -const char *charset; -const char *alias; +const struct nls_charset *charset;const struct nls_ops *ops;const unsigned char *charset2lower;const unsigned char *charset2upper; -struct module *owner;struct nls_table *next; }; +struct nls_charset { +const char *charset; +const char *alias; +struct module *owner; +struct nls_table *tables; +struct nls_charset *next; +}; +/* this value hold the maximum octet of charset */ #define NLS_MAX_CHARSET_SIZE 6/* for UTF-8 */
@@ -49,8 +55,8 @@ enum utf16_endian { };/* nls_base.c */ -extern int __register_nls (struct nls_table *,struct module *); -extern int unregister_nls (struct nls_table *); +extern int __register_nls (struct nls_charset *,struct module *); +extern int unregister_nls (struct nls_charset *);extern struct nls_table *load_nls (char *);extern void unload_nls (struct nls_table *);extern struct nls_table *load_nls_default (void );
@@ -78,7 +84,7 @@ static inline int nls_char2uni(const struct nls_table *table,static inline const char *nls_charset_name (const struct nls_table *table) { -return table->charset; +return table->charset->charset; }static inline unsigned char nls_tolower (struct nls_table *t,unsigned char c)
Signature · 1 line
2.17.0
kbuild test robot
build botRe: [PATCH 05/15] nls: Split struct nls_charset from struct nls_table
Thank you for the patch! Yet something to improve:
Hi Gabriel,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v4.17-rc4]
[cannot apply to next-20180508]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Gabriel-Krisman-Bertazi/NLS-refactor-and-UTF-8-normalization/20180509-190115
config: mips-malta_kvm_defconfig (attached as .config)
compiler: mipsel-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=mips
All errors (new ones prefixed by >>):
WARNING: modpost: missing MODULE_LICENSE() in fs/nls/nls_default.o see include/linux/module.h for more information
>> ERROR: "default_charset" [fs/nls/nls_base.ko] undefined!
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
Re: [PATCH 05/15] nls: Split struct nls_charset from struct nls_table
Quoted 18 lines · kbuild test robot
Hi Gabriel,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v4.17-rc4]
[cannot apply to next-20180508]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]url: https://github.com/0day-ci/linux/commits/Gabriel-Krisman-Bertazi/NLS-refactor-and-UTF-8-normalization/20180509-190115
config: mips-malta_kvm_defconfig (attached as .config)
compiler: mipsel-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=mips
Quoted 3 lines
All errors (new ones prefixed by >>):
WARNING: modpost: missing
MODULE_LICENSE()infs/nls/nls_default.o
This is fixed on the v2, I will be publishing no later than this week (unless there is more feedback). It is fixed in the Makefile by assuring the core and default code end up in the base module, which my original goal with this patch.
Signature · 1 line
Gabriel Krisman Bertazi
PATCH 6/15 nls: Add support for multiple versions of an encoding
In reply to Gabriel Krisman BertaziNLS charsets wanting to use this feature can implement the load_create() hook, which creates a nls_table for that specific encoding version. The charset code is responsible for freeing the table memory when the module is removed.
--- fs/nls/nls_base.c | 43 +++++++++++++++++++++++++++++++++++++------ include/linux/nls.h | 4 ++++ 2 files changed, 41 insertions(+), 6 deletions(-)
fs/nls/nls_base.c +37 -6
@@ -19,10 +19,27 @@extern struct nls_charset default_charset;static struct nls_charset *charsets = &default_charset;static DEFINE_SPINLOCK (nls_lock); -static struct nls_table *nls_load_table (struct nls_charset *charset) + +static struct nls_table *nls_load_table (struct nls_charset *charset, +const char *version) { -/* For now, return the default table, which is the first one found. */ -return charset->tables; +struct nls_table *tbl; + +if (!charset->load_table) { +/* If there is no table_create, only 1 table is +* supported and it must have been loaded +* statically. +*/ +return charset->tables; + } + + tbl = charset->load_table (version); +if (!tbl) { +/* Invalid version */ +return ERR_PTR (-EINVAL ); + } + +return tbl; }int __register_nls (struct nls_charset *nls,struct module *owner)
@@ -76,21 +93,35 @@ static struct nls_charset *find_nls(char *charset)break ; }if (nls && !try_module_get (nls->owner)) - nls =NULL ; + nls =ERR_PTR (-EBUSY );spin_unlock (&nls_lock);return nls; } -struct nls_table *load_nls (char *charset) +struct nls_table *load_nls_version (char *charset,char *version) {struct nls_charset *nls_charset; nls_charset =try_then_request_module (find_nls (charset),"nls_%s" , charset);if (!nls_charset) +return ERR_PTR (-EINVAL ); + +return nls_load_table (nls_charset, version); +} +EXPORT_SYMBOL (load_nls_version); + +struct nls_table *load_nls (char *charset) +{ +struct nls_table *table =load_nls_version (charset,NULL ); + +/* Pre-versioned load_nls() didn't return error pointers. Let's +* keep the abi for now to prevent breakage. +*/ +if (IS_ERR (table))return NULL ; -return nls_load_table (nls_charset); +return table; }void unload_nls (struct nls_table *nls)
include/linux/nls.h +4 -0
@@ -30,6 +30,8 @@ struct nls_ops {struct nls_table {const struct nls_charset *charset; +unsigned int version; +const struct nls_ops *ops;const unsigned char *charset2lower;const unsigned char *charset2upper;
@@ -42,6 +44,7 @@ struct nls_charset {struct module *owner;struct nls_table *tables;struct nls_charset *next; +struct nls_table *(*load_table)(const char *version); };/* this value hold the maximum octet of charset */
@@ -58,6 +61,7 @@ enum utf16_endian {extern int __register_nls (struct nls_charset *,struct module *);extern int unregister_nls (struct nls_charset *);extern struct nls_table *load_nls (char *); +extern struct nls_table *load_nls_version (char *charset,char *version);extern void unload_nls (struct nls_table *);extern struct nls_table *load_nls_default (void );#define register_nls (nls) __register_nls((nls), THIS_MODULE)
Signature · 1 line
2.17.0
PATCH 7/15 nls: Add new interface for string comparisons
In reply to Gabriel Krisman BertaziThe existing stricmp() interface is limited by not accepting separated length parameters for each string being compared. This is a problem for charsets doing normalization or full casefold comparison, since different sized strings can still be matched. To resolve this problem, this patch implements a new interface, allowing charsets to do the comparison, if needed.
The original stricmp is left in the code, while all callers are not converted, but was rewritten the new interface.
--- include/linux/nls.h | 42 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 3 deletions(-)
include/linux/nls.h +39 -3
@@ -3,6 +3,7 @@#define _LINUX_NLS_H#include <linux/init.h> +#include <linux/string.h> /* Unicode has changed over the years. Unicode code points no longer * fit into 16 bits; as of Unicode 5 valid code points range from 0
@@ -21,11 +22,18 @@ typedef u16 wchar_t;/* Arbitrary Unicode character */ typedef u32 unicode_t ; +struct nls_table ;struct nls_ops {int (*uni2char) (wchar_t uni,unsigned char *out,int boundlen);int (*char2uni) (const unsigned char *rawstring,int boundlen,wchar_t *uni); +int (*strncmp)(const struct nls_table *charset, +const unsigned char *str1,size_t len1, +const unsigned char *str2,size_t len2); +int (*strncasecmp)(const struct nls_table *charset, +const unsigned char *str1,size_t len1, +const unsigned char *str2,size_t len2); };struct nls_table {
@@ -105,10 +113,17 @@ static inline unsigned char nls_toupper(struct nls_table *t, unsigned char c)return nc ? nc : c; } -static inline int nls_strnicmp (struct nls_table *t,const unsigned char *s1, -const unsigned char *s2,int len) +static inline int nls_strncasecmp (struct nls_table *t, +const unsigned char *s1,size_t len1, +const unsigned char *s2,size_t len2) { -while (len--) { +if (t->ops->strncasecmp) +return t->ops->strncasecmp (t, s1, len1, s2, len2); + +if (len1 != len2) +return 1 ; + +while (len1--) {if (nls_tolower (t, *s1++) !=nls_tolower (t, *s2++))return 1 ; }
@@ -116,6 +131,27 @@ static inline int nls_strnicmp(struct nls_table *t, const unsigned char *s1,return 0 ; } +static inline int nls_strncmp (struct nls_table *t, +const unsigned char *s1,size_t len1, +const unsigned char *s2,size_t len2) +{ +if (t->ops->strncmp) +return t->ops->strncmp (t, s1, len1, s2, len2); + +if (len1 != len2) +return 1 ; + +/* strnicmp did not return negative values. So let's keep the +* abi for now */ +return !!memcmp (s1, s2, len1); +} + +static inline int nls_strnicmp (struct nls_table *t,const unsigned char *s1, +const unsigned char *s2,int len) +{ +return nls_strncasecmp (t, s1, len, s2, len); +} +/* * nls_nullsize - return length of null character for codepage * @codepage - codepage for which to return length of NULL terminator
Signature · 1 line
2.17.0
PATCH 8/15 nls: Let charsets define the behavior of tolower/toupper
In reply to Gabriel Krisman BertaziInstead of always reading from a table, give the charset a chance to implement tolower and toupper it algorithmically.
This patch was created using the semantic patch below, with the exception of the header files (hook definitions) and a fix to files that didn't have the tables statically allocated (koi8-u and cp932).
<smpl>
@tbl@
identifier p;
expression lower_tbl;
expression upper_tbl;
@@
static struct nls_table p = {
- .charset2lower = lower_tbl,
- .charset2upper = upper_tbl,
};
@@
identifier charset_ops;
expression tbl.lower_tbl;
expression tbl.upper_tbl;
@@
+ static unsigned char charset_tolower(const struct nls_table *table, unsigned int c)
+ {
+ return lower_tbl[c];
+ }
+
+ static unsigned char charset_toupper(const struct nls_table *table, unsigned int c)
+ {
+ return upper_tbl[c];
+ }
static const struct nls_ops charset_ops = {
+ .lowercase = charset_toupper,
+ .uppercase = charset_tolower,
};
@@
struct nls_table *t;
expression A;
expression nc;
@@
(
- nc = t->charset2lower[A]
+ nc = nls_tolower(t, A)
|
- nc = t->charset2upper[A]
+ nc = nls_toupper(t, A)
)
<...
- if(!nc)
- nc = A;
...>
</smpl>
--- fs/fat/dir.c | 5 +---- fs/nls/mac-celtic.c | 14 ++++++++++++-- fs/nls/mac-centeuro.c | 14 ++++++++++++-- fs/nls/mac-croatian.c | 14 ++++++++++++-- fs/nls/mac-cyrillic.c | 14 ++++++++++++-- fs/nls/mac-gaelic.c | 14 ++++++++++++-- fs/nls/mac-greek.c | 14 ++++++++++++-- fs/nls/mac-iceland.c | 14 ++++++++++++-- fs/nls/mac-inuit.c | 14 ++++++++++++-- fs/nls/mac-roman.c | 14 ++++++++++++-- fs/nls/mac-romanian.c | 14 ++++++++++++-- fs/nls/mac-turkish.c | 14 ++++++++++++-- fs/nls/nls_ascii.c | 14 ++++++++++++-- fs/nls/nls_cp1250.c | 14 ++++++++++++-- fs/nls/nls_cp1251.c | 14 ++++++++++++-- fs/nls/nls_cp1255.c | 14 ++++++++++++-- fs/nls/nls_cp437.c | 14 ++++++++++++-- fs/nls/nls_cp737.c | 14 ++++++++++++-- fs/nls/nls_cp775.c | 14 ++++++++++++-- fs/nls/nls_cp850.c | 14 ++++++++++++-- fs/nls/nls_cp852.c | 14 ++++++++++++-- fs/nls/nls_cp855.c | 14 ++++++++++++-- fs/nls/nls_cp857.c | 14 ++++++++++++-- fs/nls/nls_cp860.c | 14 ++++++++++++-- fs/nls/nls_cp861.c | 14 ++++++++++++-- fs/nls/nls_cp862.c | 14 ++++++++++++-- fs/nls/nls_cp863.c | 14 ++++++++++++-- fs/nls/nls_cp864.c | 14 ++++++++++++-- fs/nls/nls_cp865.c | 14 ++++++++++++-- fs/nls/nls_cp866.c | 14 ++++++++++++-- fs/nls/nls_cp869.c | 14 ++++++++++++-- fs/nls/nls_cp874.c | 14 ++++++++++++-- fs/nls/nls_cp932.c | 14 ++++++++++++-- fs/nls/nls_cp936.c | 14 ++++++++++++-- fs/nls/nls_cp949.c | 14 ++++++++++++-- fs/nls/nls_cp950.c | 14 ++++++++++++-- fs/nls/nls_default.c | 14 ++++++++++++-- fs/nls/nls_euc-jp.c | 7 ++++--- fs/nls/nls_iso8859-1.c | 14 ++++++++++++-- fs/nls/nls_iso8859-13.c | 14 ++++++++++++-- fs/nls/nls_iso8859-14.c | 14 ++++++++++++-- fs/nls/nls_iso8859-15.c | 14 ++++++++++++-- fs/nls/nls_iso8859-2.c | 14 ++++++++++++-- fs/nls/nls_iso8859-3.c | 14 ++++++++++++-- fs/nls/nls_iso8859-4.c | 14 ++++++++++++-- fs/nls/nls_iso8859-5.c | 14 ++++++++++++-- fs/nls/nls_iso8859-6.c | 14 ++++++++++++-- fs/nls/nls_iso8859-7.c | 14 ++++++++++++-- fs/nls/nls_iso8859-9.c | 14 ++++++++++++-- fs/nls/nls_koi8-r.c | 14 ++++++++++++-- fs/nls/nls_koi8-ru.c | 6 +++--- fs/nls/nls_koi8-u.c | 14 ++++++++++++-- fs/nls/nls_utf8.c | 14 ++++++++++++-- include/linux/nls.h | 12 ++++++++---- 54 files changed, 616 insertions(+), 114 deletions(-)
fs/fat/dir.c +1 -4
@@ -215,10 +215,7 @@ fat_short2lower_uni(struct nls_table *t, unsigned char *c, *uni =0x003f ;/* a question mark */ charlen =1 ; }else if (charlen <=1 ) { -unsigned char nc = t->charset2lower[*c]; - -if (!nc) - nc = *c; +unsigned char nc =nls_tolower (t, *c); charlen =nls_char2uni (t, &nc,1 , uni);if (charlen <0 ) {
fs/nls/mac-celtic.c +12 -2
@@ -577,7 +577,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -586,8 +598,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/mac-centeuro.c +12 -2
@@ -507,7 +507,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -516,8 +528,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/mac-croatian.c +12 -2
@@ -577,7 +577,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -586,8 +598,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/mac-cyrillic.c +12 -2
@@ -472,7 +472,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -481,8 +493,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/mac-gaelic.c +12 -2
@@ -542,7 +542,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -551,8 +563,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/mac-greek.c +12 -2
@@ -472,7 +472,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -481,8 +493,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/mac-iceland.c +12 -2
@@ -577,7 +577,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -586,8 +598,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/mac-inuit.c +12 -2
@@ -507,7 +507,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -516,8 +528,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/mac-roman.c +12 -2
@@ -612,7 +612,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -621,8 +633,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/mac-romanian.c +12 -2
@@ -577,7 +577,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -586,8 +598,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/mac-turkish.c +12 -2
@@ -577,7 +577,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -586,8 +598,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_ascii.c +12 -2
@@ -142,7 +142,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -151,8 +163,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp1250.c +12 -2
@@ -323,7 +323,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -332,8 +344,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp1251.c +12 -2
@@ -277,7 +277,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -286,8 +298,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp1255.c +12 -2
@@ -358,7 +358,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -367,8 +379,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp437.c +12 -2
@@ -363,7 +363,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -372,8 +384,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp737.c +12 -2
@@ -326,7 +326,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -335,8 +347,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp775.c +12 -2
@@ -295,7 +295,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -304,8 +316,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp850.c +12 -2
@@ -291,7 +291,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -300,8 +312,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp852.c +12 -2
@@ -313,7 +313,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -322,8 +334,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp855.c +12 -2
@@ -275,7 +275,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -284,8 +296,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp857.c +12 -2
@@ -277,7 +277,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -286,8 +298,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp860.c +12 -2
@@ -340,7 +340,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -349,8 +361,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp861.c +12 -2
@@ -363,7 +363,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -372,8 +384,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp862.c +12 -2
@@ -397,7 +397,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -406,8 +418,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp863.c +12 -2
@@ -357,7 +357,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -366,8 +378,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp864.c +12 -2
@@ -383,7 +383,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -392,8 +404,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp865.c +12 -2
@@ -363,7 +363,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -372,8 +384,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp866.c +12 -2
@@ -281,7 +281,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -290,8 +302,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp869.c +12 -2
@@ -291,7 +291,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -300,8 +312,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp874.c +12 -2
@@ -249,7 +249,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -258,8 +270,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp932.c +12 -2
@@ -7907,7 +7907,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return -EINVAL ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -7916,8 +7928,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp936.c +12 -2
@@ -11085,7 +11085,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return n; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -11094,8 +11106,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp949.c +12 -2
@@ -13920,7 +13920,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return n; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -13929,8 +13941,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_cp950.c +12 -2
@@ -9456,7 +9456,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return n; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -9465,8 +9477,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_default.c +12 -2
@@ -447,7 +447,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -455,8 +467,6 @@ static const struct nls_ops charset_ops = {static struct nls_table default_table = { .charset = &default_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };struct nls_charset default_charset = {
fs/nls/nls_euc-jp.c +4 -3
@@ -549,7 +549,7 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return euc_offset; } -static const struct nls_ops charset_ops = { +static struct nls_ops charset_ops = { .uni2char = uni2char, .char2uni = char2uni, };
@@ -570,8 +570,9 @@ static int __init init_nls_euc_jp(void) p_nls =load_nls ("cp932" );if (p_nls) { - table.charset2upper = p_nls->charset2upper; - table.charset2lower = p_nls->charset2lower; + + charset_ops.uppercase = p_nls->ops->uppercase; + charset_ops.lowercase = p_nls->ops->lowercase;return register_nls (&nls_charset); }
fs/nls/nls_iso8859-1.c +12 -2
@@ -233,7 +233,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -242,8 +254,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_iso8859-13.c +12 -2
@@ -261,7 +261,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -270,8 +282,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_iso8859-14.c +12 -2
@@ -317,7 +317,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -326,8 +338,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_iso8859-15.c +12 -2
@@ -283,7 +283,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -292,8 +304,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_iso8859-2.c +12 -2
@@ -284,7 +284,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -293,8 +305,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_iso8859-3.c +12 -2
@@ -284,7 +284,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -293,8 +305,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_iso8859-4.c +12 -2
@@ -284,7 +284,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -293,8 +305,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_iso8859-5.c +12 -2
@@ -248,7 +248,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -257,8 +269,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_iso8859-6.c +12 -2
@@ -239,7 +239,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -248,8 +260,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_iso8859-7.c +12 -2
@@ -293,7 +293,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -302,8 +314,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_iso8859-9.c +12 -2
@@ -248,7 +248,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -257,8 +269,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_koi8-r.c +12 -2
@@ -299,7 +299,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -308,8 +320,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_koi8-ru.c +3 -3
@@ -51,7 +51,7 @@ static int char2uni(const unsigned char *rawstring, int boundlen,return n; } -static const struct nls_ops charset_ops = { +static struct nls_ops charset_ops = { .uni2char = uni2char, .char2uni = char2uni, };
@@ -72,8 +72,8 @@ static int __init init_nls_koi8_ru(void) p_nls =load_nls ("koi8-u" );if (p_nls) { - table.charset2upper = p_nls->charset2upper; - table.charset2lower = p_nls->charset2lower; + charset_ops.uppercase = p_nls->ops->uppercase; + charset_ops.lowercase = p_nls->ops->lowercase;return register_nls (&nls_charset); }
fs/nls/nls_koi8-u.c +12 -2
@@ -306,7 +306,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return 1 ; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return charset2lower[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return charset2upper[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -315,8 +327,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = charset2lower, - .charset2upper = charset2upper, };static struct nls_charset nls_charset = {
fs/nls/nls_utf8.c +12 -2
@@ -40,7 +40,19 @@ static int char2uni(const unsigned char *rawstring, int boundlen, wchar_t *uni)return n; } +static unsigned char charset_tolower (const struct nls_table *table, +unsigned int c){ +return identity[c]; +} + +static unsigned char charset_toupper (const struct nls_table *table, +unsigned int c) { +return identity[c]; +} +static const struct nls_ops charset_ops = { + .lowercase = charset_toupper, + .uppercase = charset_tolower, .uni2char = uni2char, .char2uni = char2uni, };
@@ -49,8 +61,6 @@ static struct nls_charset nls_charset;static struct nls_table table = { .charset = &nls_charset, .ops = &charset_ops, - .charset2lower = identity,/* no conversion */ - .charset2upper = identity, };static struct nls_charset nls_charset = {
include/linux/nls.h +8 -4
@@ -34,6 +34,11 @@ struct nls_ops {int (*strncasecmp)(const struct nls_table *charset,const unsigned char *str1,size_t len1,const unsigned char *str2,size_t len2); +unsigned char (*lowercase)(const struct nls_table *charset, +unsigned int c); +unsigned char (*uppercase)(const struct nls_table *charset, +unsigned int c); + };struct nls_table {
@@ -41,9 +46,8 @@ struct nls_table {unsigned int version;const struct nls_ops *ops; -const unsigned char *charset2lower; -const unsigned char *charset2upper;struct nls_table *next; + };struct nls_charset {
@@ -101,14 +105,14 @@ static inline const char *nls_charset_name(const struct nls_table *table)static inline unsigned char nls_tolower (struct nls_table *t,unsigned char c) { -unsigned char nc = t->charset2lower[c]; +unsigned char nc = t->ops->lowercase (t, c);return nc ? nc : c; }static inline unsigned char nls_toupper (struct nls_table *t,unsigned char c) { -unsigned char nc = t->charset2upper[c]; +unsigned char nc = t->ops->uppercase (t, c);return nc ? nc : c; }
Signature · 1 line
2.17.0
PATCH 9/15 nls: Add optional normalization and casefold hooks
In reply to Gabriel Krisman Bertazi--- include/linux/nls.h | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-)
include/linux/nls.h +27 -1
@@ -4,6 +4,7 @@#include <linux/init.h> #include <linux/string.h> +#include <linux/errno.h> /* Unicode has changed over the years. Unicode code points no longer * fit into 16 bits; as of Unicode 5 valid code points range from 0
@@ -38,7 +39,12 @@ struct nls_ops { unsigned int c);unsigned char (*uppercase)(const struct nls_table *charset,unsigned int c); - +int (*casefold)(const struct nls_table *charset, +const unsigned char *str,size_t len, +unsigned char **folded); +int (*normalize)(const struct nls_table *charset, +const unsigned char *str,size_t len, +unsigned char **normalized); };struct nls_table {
@@ -156,6 +162,26 @@ static inline int nls_strnicmp(struct nls_table *t, const unsigned char *s1,return nls_strncasecmp (t, s1, len, s2, len); } +static inline int nls_casefold (const struct nls_table *charset, +const unsigned char *str,size_t len, +unsigned char **folded) +{ +if (charset->ops->casefold) +return charset->ops->casefold (charset, str, len, folded); + +return -ENOTSUPP ; +} + +static inline int nls_normalize (const struct nls_table *charset, +const unsigned char *str,size_t len, +unsigned char **normalized) +{ +if (charset->ops->normalize) +return charset->ops->normalize (charset, str, len, normalized); + +return -ENOTSUPP ; +} +/* * nls_nullsize - return length of null character for codepage * @codepage - codepage for which to return length of NULL terminator
Signature · 1 line
2.17.0
PATCH 10/15 nls: utf8norm: Add unicode character database files
In reply to Gabriel Krisman BertaziFrom: Olaf Weber <[email protected]>
Add files from the Unicode Character Database, version 10.0.0, to the source. A helper program that generates a trie used for normalization from these files is part of a separate commit.
- Notes on the update from 8.0.0 and 10.0.0:
The structure of ucd files and special cases have not experienced any changes between versions 8.0.0 and 10.0.0. 8.0.0 saw the addition of Cherokee LC characters, which is an interesting case for case-folding. The update is accompanied by new tests on the test_ucd module to catch specific cases. No changes to mkutf8data script was required for the update.
The actual files are not part of the commit submitted to the list because they are to big and would bounce. Still, they can be obtained by the following script:
FILES="CaseFolding.txt DerivedAge.txt extracted/DerivedCombiningClass.txt
DerivedCoreProperties.txt NormalizationCorrections.txt NormalizationTest.txt UnicodeData.txt"
VERSION=10.0.0 BASE=http://www.unicode.org/Public/${VERSION}/ucd
for i in ${FILES} ; do
wget "${BASE}/$i" -O fs/nls/ucd/$(basename ${i} .txt)-${VERSION}.txt
done
--- fs/nls/ucd/README | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 fs/nls/ucd/README
fs/nls/ucd/README new file +33 -0
@@ -0,0 +1,33 @@ +The files in this directory are part of the Unicode Character Database +for version 10.0.0 of the Unicode standard. + +The full set of files can be found here: + + http://www.unicode.org/Public/10.0.0/ucd/ + +The latest released version of the UCD can be found here: + + http://www.unicode.org/Public/UCD/latest/ + +The files in this directory are identical, except that they have been +renamed with a suffix indicating the unicode version. + +Individual source links: + + http://www.unicode.org/Public/10.0.0/ucd/CaseFolding.txt + http://www.unicode.org/Public/10.0.0/ucd/DerivedAge.txt + http://www.unicode.org/Public/10.0.0/ucd/extracted/DerivedCombiningClass.txt + http://www.unicode.org/Public/10.0.0/ucd/DerivedCoreProperties.txt + http://www.unicode.org/Public/10.0.0/ucd/NormalizationCorrections.txt + http://www.unicode.org/Public/10.0.0/ucd/NormalizationTest.txt + http://www.unicode.org/Public/10.0.0/ucd/UnicodeData.txt + +md5sums + + 7893b6e005c5a521319a0d12062ae122 CaseFolding-10.0.0.txt + a602e4b44de3350087e40f2eb2184898 DerivedAge-10.0.0.txt + 5abdeb21af4edcc5d1e4c0b5802fc7a7 DerivedCombiningClass-10.0.0.txt + eda11c2c2e3c308d9d3b90e2b3282024 DerivedCoreProperties-10.0.0.txt + 425ece5ffbecd0140d98c13ce05724aa NormalizationCorrections-10.0.0.txt + 7296fe7aa07d7d288e65d559af2ad49b NormalizationTest-10.0.0.txt + 2a52f30695dcc821f0f224650552beaf UnicodeData-10.0.0.txt
Signature · 1 line
2.17.0
PATCH 11/15 scripts: add trie generator for UTF-8
From: Olaf Weber <[email protected]>
mkutf8data.c is the source for a program that generates utf8data.h, which contains the trie that utf8norm.c uses. The trie is generated from the Unicode 10.0.0 data files. The format of the utf8data[] table is described in utf8norm.c, which is added in the next patch.
--- fs/nls/Kconfig | 8 + fs/nls/Makefile | 13 + scripts/Makefile | 1 + scripts/mkutf8data.c | 3239 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 3261 insertions(+) create mode 100644 scripts/mkutf8data.c
fs/nls/Kconfig +8 -0
@@ -616,4 +616,12 @@ config NLS_UTF8 input/output character sets. Say Y here for the UTF-8 encoding of the Unicode/ISO9646 universal character set. +# +# utf8 normalization module +# +config NLS_UTF8_NORMALIZATION + tristate "UTF-8 normalization support" + help + Say Y here to enable utf8 normalization support. + endif # NLS
fs/nls/Makefile +13 -0
@@ -54,3 +54,16 @@ obj-$(CONFIG_NLS_MAC_INUIT) += mac-inuit.o obj-$(CONFIG_NLS_MAC_ROMANIAN) += mac-romanian.o obj-$(CONFIG_NLS_MAC_ROMAN) += mac-roman.o obj-$(CONFIG_NLS_MAC_TURKISH) += mac-turkish.o + +$(obj)/utf8data.h: $(srctree)/$(src)/ucd/*.txt $(objtree)/scripts/mkutf8data FORCE + $(call cmd,mkutf8data) +quiet_cmd_mkutf8data = MKUTF8DATA $@ + cmd_mkutf8data = $(objtree)/scripts/mkutf8data \ + -a $(srctree)/$(src)/ucd/DerivedAge-10.0.0.txt \ + -c $(srctree)/$(src)/ucd/DerivedCombiningClass-10.0.0.txt \ + -p $(srctree)/$(src)/ucd/DerivedCoreProperties-10.0.0.txt \ + -d $(srctree)/$(src)/ucd/UnicodeData-10.0.0.txt \ + -f $(srctree)/$(src)/ucd/CaseFolding-10.0.0.txt \ + -n $(srctree)/$(src)/ucd/NormalizationCorrections-10.0.0.txt \ + -t $(srctree)/$(src)/ucd/NormalizationTest-10.0.0.txt \ + -o $@
scripts/Makefile +1 -0
@@ -19,6 +19,7 @@ hostprogs-$(CONFIG_ASN1) += asn1_compiler hostprogs-$(CONFIG_MODULE_SIG) += sign-file hostprogs-$(CONFIG_SYSTEM_TRUSTED_KEYRING) += extract-cert hostprogs-$(CONFIG_SYSTEM_EXTRA_CERTIFICATE) += insert-sys-cert +hostprogs-$(CONFIG_NLS_UTF8_NORMALIZATION) += mkutf8data HOSTCFLAGS_sortextable.o = -I$(srctree)/tools/include HOSTCFLAGS_asn1_compiler.o = -I$(srctree)/include
scripts/mkutf8data.c new file +3239 -0
2,965 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.17.0
Part of the diff is not shown here. View the raw message.
PATCH 12/15 nls: utf8norm: Introduce code for UTF-8 normalization
In reply to Gabriel Krisman BertaziFrom: Olaf Weber <[email protected]>
Supporting functions for UTF-8 normalization are in utf8norm.c with the header utf8norm.h. Two normalization forms are supported: nfkdi and nfkdicf.
nfkdi:
- Apply unicode normalization form NFKD.
- Remove any Default_Ignorable_Code_Point.
nfkdicf:
- Apply unicode normalization form NFKD.
- Remove any Default_Ignorable_Code_Point.
- Apply a full casefold (C + F).
For the purposes of the code, a string is valid UTF-8 if:
- The values encoded are 0x1..0x10FFFF.
- The surrogate codepoints 0xD800..0xDFFFF are not encoded.
- The shortest possible encoding is used for all values.
The supporting functions work on null-terminated strings (utf8 prefix) and on length-limited strings (utf8n prefix).
Quoted 1 line
From the original SGI patch and for conformity with coding standards,
the utf8data_t typedef was dropped, since it was just masking the struct keyword. On other occasions, namely utf8leaf_t and utf8trie_t, I decided to keep it, since they are simple pointers to memory buffers, and using uchars here wouldn't provide any more meaningful information.
Changes since RFC v2:
- Merge to NLS system
Changes since RFC v1:
- utf8_version_is_supported receives maj, min and rev as separate arguments. (Olaf Weber)
--- fs/nls/Makefile | 4 + fs/nls/nls_utf8n-norm.c | 640 ++++++++++++++++++++++++++++++++++++++++ fs/nls/utf8n.h | 112 +++++++ 3 files changed, 756 insertions(+) create mode 100644 fs/nls/nls_utf8n-norm.c create mode 100644 fs/nls/utf8n.h
fs/nls/Makefile +4 -0
@@ -55,6 +55,10 @@ obj-$(CONFIG_NLS_MAC_ROMANIAN) += mac-romanian.o obj-$(CONFIG_NLS_MAC_ROMAN) += mac-roman.o obj-$(CONFIG_NLS_MAC_TURKISH) += mac-turkish.o +nls_utf8n-y += nls_utf8n-norm.o +obj-$(CONFIG_NLS_UTF8_NORMALIZATION) += nls_utf8n.o + +$(obj)/nls_utf8n-norm.o: $(obj)/utf8data.h $(obj)/utf8data.h: $(srctree)/$(src)/ucd/*.txt $(objtree)/scripts/mkutf8data FORCE $(call cmd,mkutf8data) quiet_cmd_mkutf8data = MKUTF8DATA $@
fs/nls/nls_utf8n-norm.c new file +640 -0
640 lines not on this page · raw message
fs/nls/utf8n.h new file +112 -0
112 lines not on this page · raw message
Signature · 1 line
2.17.0
kbuild test robot
build botRe: [PATCH 12/15] nls: utf8norm: Introduce code for UTF-8 normalization
Thank you for the patch! Yet something to improve:
Hi Olaf,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v4.17-rc4]
[cannot apply to next-20180509]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Gabriel-Krisman-Bertazi/NLS-refactor-and-UTF-8-normalization/20180509-190115
config: i386-allmodconfig (attached as .config)
compiler: gcc-7 (Debian 7.3.0-16) 7.3.0 reproduce:
# save the attached .config to linux build tree
make ARCH=i386 All errors (new ones prefixed by >>):
>> make[3]: *** No rule to make target 'fs/nls/ucd/*.txt', needed by 'fs/nls/utf8data.h'.
make[3]: Target '__build' not remade because of errors.
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
Re: [PATCH 12/15] nls: utf8norm: Introduce code for UTF-8 normalization
Quoted 13 lines · kbuild test robot
Hi Olaf,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v4.17-rc4]
[cannot apply to next-20180509]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]url: https://github.com/0day-ci/linux/commits/Gabriel-Krisman-Bertazi/NLS-refactor-and-UTF-8-normalization/20180509-190115
config: i386-allmodconfig (attached as .config)
compiler: gcc-7 (Debian 7.3.0-16) 7.3.0 reproduce:
Quoted 7 lines
# save the attached .config to linux build tree make ARCH=i386All errors (new ones prefixed by >>):
Quoted 1 line
Quoted 1 line
make[3]: *** No rule to make target '
fs/nls/ucd/*.txt', needed by 'fs/nls/utf8data.h'.make[3]: Target '
__build' not remade because of errors.
Well, buildbot is expected to fail here since, as I mentioned, the ucd files were not added to the patch to prevent bounces in the ML. People should disregard this automatic build error issue during review. Once it gets commited, the ucd/*.txt files will be there.
Signature · 1 line
Gabriel Krisman Bertazi
PATCH 13/15 nls: utf8norm: reduce the size of utf8data[]
In reply to Gabriel Krisman BertaziFrom: Olaf Weber <[email protected]>
Remove the Hangul decompositions from the utf8data trie, and do algorithmic decomposition to calculate them on the fly. To store the decomposition the caller of utf8lookup()/utf8nlookup() must provide a 12-byte buffer, which is used to synthesize a leaf with the decomposition. Trie size is reduced from 245kB to 90kB.
[Extract robustness fixes and merge back to original mkutf8data.c patch]
--- fs/nls/nls_utf8n-norm.c | 191 +++++++++++++++++++++++--- fs/nls/utf8n.h | 4 + scripts/mkutf8data.c | 295 +++++++++++++++++++++++++++++++++++----- 3 files changed, 435 insertions(+), 55 deletions(-)
fs/nls/nls_utf8n-norm.c +171 -20
@@ -98,6 +98,38 @@ static inline int utf8clen(const char *s)return 1 + (c >=0xC0 ) + (c >=0xE0 ) + (c >=0xF0 ); } +/* +* Decode a 3-byte UTF-8 sequence. +*/ +static unsigned int +utf8decode3 (const char *str) +{ +unsigned int uc; + + uc = *str++ &0x0F ; + uc <<=6 ; + uc |= *str++ &0x3F ; + uc <<=6 ; + uc |= *str++ &0x3F ; + +return uc; +} + +/* +* Encode a 3-byte UTF-8 sequence. +*/ +static int +utf8encode3 (char *str,unsigned int val) +{ + str[2 ] = (val &0x3F ) |0x80 ; + val >>=6 ; + str[1 ] = (val &0x3F ) |0x80 ; + val >>=6 ; + str[0 ] = val |0xE0 ; + +return 3 ; +} +/* * utf8trie_t *
@@ -159,7 +191,8 @@ typedef const unsigned char utf8trie_t;* characters with the Default_Ignorable_Code_Point property. * These do affect normalization, as they all have CCC 0. * -* The decompositions in the trie have been fully expanded. +* The decompositions in the trie have been fully expanded, with the +* exception of Hangul syllables, which are decomposed algorithmically. * * Casefolding, if applicable, is also done using decompositions. *
@@ -179,6 +212,105 @@ typedef const unsigned char utf8leaf_t;#define STOPPER (0)#define DECOMPOSE (255) +/* Marker for hangul syllable decomposition. */ +#define HANGUL ((char)(255)) +/* Size of the synthesized leaf used for Hangul syllable decomposition. */ +#define UTF8HANGULLEAF (12) + +/* +* Hangul decomposition (algorithm from Section 3.12 of Unicode 6.3.0) +* +* AC00;<Hangul Syllable, First>;Lo;0;L;;;;;N;;;;; +* D7A3;<Hangul Syllable, Last>;Lo;0;L;;;;;N;;;;; +* +* SBase = 0xAC00 +* LBase = 0x1100 +* VBase = 0x1161 +* TBase = 0x11A7 +* LCount = 19 +* VCount = 21 +* TCount = 28 +* NCount = 588 (VCount * TCount) +* SCount = 11172 (LCount * NCount) +* +* Decomposition: +* SIndex = s - SBase +* +* LV (Canonical/Full) +* LIndex = SIndex / NCount +* VIndex = (Sindex % NCount) / TCount +* LPart = LBase + LIndex +* VPart = VBase + VIndex +* +* LVT (Canonical) +* LVIndex = (SIndex / TCount) * TCount +* TIndex = (Sindex % TCount) +* LVPart = SBase + LVIndex +* TPart = TBase + TIndex +* +* LVT (Full) +* LIndex = SIndex / NCount +* VIndex = (Sindex % NCount) / TCount +* TIndex = (Sindex % TCount) +* LPart = LBase + LIndex +* VPart = VBase + VIndex +* if (TIndex == 0) { +* d = <LPart, VPart> +* } else { +* TPart = TBase + TIndex +* d = <LPart, TPart, VPart> +* } +*/ + +/* Constants */ +#define SB (0xAC00) +#define LB (0x1100) +#define VB (0x1161) +#define TB (0x11A7) +#define LC (19) +#define VC (21) +#define TC (28) +#define NC (VC * TC) +#define SC (LC * NC) + +/* Algorithmic decomposition of hangul syllable. */ +static utf8leaf_t * +utf8hangul (const char *str,unsigned char *hangul) +{ +unsigned int si; +unsigned int li; +unsigned int vi; +unsigned int ti; +unsigned char *h; + +/* Calculate the SI, LI, VI, and TI values. */ + si =utf8decode3 (str) -SB ; + li = si /NC ; + vi = (si %NC ) /TC ; + ti = si %TC ; + +/* Fill in base of leaf. */ + h = hangul; +LEAF_GEN (h) =2 ; +LEAF_CCC (h) =DECOMPOSE ; + h +=2 ; + +/* Add LPart, a 3-byte UTF-8 sequence. */ + h +=utf8encode3 ((char *)h, li +LB ); + +/* Add VPart, a 3-byte UTF-8 sequence. */ + h +=utf8encode3 ((char *)h, vi +VB ); + +/* Add TPart if required, also a 3-byte UTF-8 sequence. */ +if (ti) + h +=utf8encode3 ((char *)h, ti +TB ); + +/* Terminate string. */ + h[0 ] ='\0' ; + +return hangul; +} +/* * Use trie to scan s, touching at most len bytes. * Returns the leaf if one exists, NULL otherwise.
@@ -187,8 +319,8 @@ typedef const unsigned char utf8leaf_t;* is well-formed and corresponds to a known unicode code point. The * shorthand for this will be "is valid UTF-8 unicode". */ -static utf8leaf_t *utf8nlookup (const struct utf8data *data,const char *s, -size_t len) +static utf8leaf_t *utf8nlookup (const struct utf8data *data, +unsigned char *hangul,const char *s,size_t len) {utf8trie_t *trie = utf8data + data->offset;int offlen;
@@ -226,8 +358,7 @@ static utf8leaf_t *utf8nlookup(const struct utf8data *data, const char *s, trie++; }else {/* No right node. */ - node =0 ; - trie =NULL ; +return NULL ; } }else {/* Left leg */
@@ -237,8 +368,7 @@ static utf8leaf_t *utf8nlookup(const struct utf8data *data, const char *s, trie += offlen +1 ; }else if (*trie &RIGHTPATH ) {/* No left node. */ - node =0 ; - trie =NULL ; +return NULL ; }else {/* Left node after this node */ node = (*trie &TRIENODE );
@@ -246,6 +376,14 @@ static utf8leaf_t *utf8nlookup(const struct utf8data *data, const char *s, } } } +/* +* Hangul decomposition is done algorithmically. These are the +* codepoints >= 0xAC00 and <= 0xD7A3. Their UTF-8 encoding is +* always 3 bytes long, so s has been advanced twice, and the +* start of the sequence is at s-2. +*/ +if (LEAF_CCC (trie) ==DECOMPOSE &&LEAF_STR (trie)[0 ] ==HANGUL ) + trie =utf8hangul (s -2 , hangul);return trie; }
@@ -255,9 +393,10 @@ static utf8leaf_t *utf8nlookup(const struct utf8data *data, const char *s,* * Forwards to utf8nlookup(). */ -static utf8leaf_t *utf8lookup (const struct utf8data *data,const char *s) +static utf8leaf_t *utf8lookup (const struct utf8data *data, +unsigned char *hangul,const char *s) { -return utf8nlookup (data, s, (size_t )-1 ); +return utf8nlookup (data, hangul, s, (size_t )-1 ); }/*
@@ -270,11 +409,13 @@ int utf8agemax(const struct utf8data *data, const char *s)utf8leaf_t *leaf;int age =0 ;int leaf_age; +unsigned char hangul[UTF8HANGULLEAF ];if (!data)return -1 ; +while (*s) { - leaf =utf8lookup (data, s); + leaf =utf8lookup (data, hangul, s);if (!leaf)return -1 ;
@@ -297,12 +438,13 @@ int utf8agemin(const struct utf8data *data, const char *s)utf8leaf_t *leaf;int age;int leaf_age; +unsigned char hangul[UTF8HANGULLEAF ];if (!data)return -1 ; age = data->maxage;while (*s) { - leaf =utf8lookup (data, s); + leaf =utf8lookup (data, hangul, s);if (!leaf)return -1 ; leaf_age = utf8agetab[LEAF_GEN (leaf)];
@@ -323,11 +465,13 @@ int utf8nagemax(const struct utf8data *data, const char *s, size_t len)utf8leaf_t *leaf;int age =0 ;int leaf_age; +unsigned char hangul[UTF8HANGULLEAF ];if (!data)return -1 ; +while (len && *s) { - leaf =utf8nlookup (data, s, len); + leaf =utf8nlookup (data, hangul, s, len);if (!leaf)return -1 ; leaf_age = utf8agetab[LEAF_GEN (leaf)];
@@ -349,12 +493,13 @@ int utf8nagemin(const struct utf8data *data, const char *s, size_t len)utf8leaf_t *leaf;int leaf_age;int age; +unsigned char hangul[UTF8HANGULLEAF ];if (!data)return -1 ; age = data->maxage;while (len && *s) { - leaf =utf8nlookup (data, s, len); + leaf =utf8nlookup (data, hangul, s, len);if (!leaf)return -1 ; leaf_age = utf8agetab[LEAF_GEN (leaf)];
@@ -377,11 +522,12 @@ ssize_t utf8len(const struct utf8data *data, const char *s) {utf8leaf_t *leaf;size_t ret =0 ; +unsigned char hangul[UTF8HANGULLEAF ];if (!data)return -1 ;while (*s) { - leaf =utf8lookup (data, s); + leaf =utf8lookup (data, hangul, s);if (!leaf)return -1 ;if (utf8agetab[LEAF_GEN (leaf)] > data->maxage)
@@ -404,11 +550,12 @@ ssize_t utf8nlen(const struct utf8data *data, const char *s, size_t len) {utf8leaf_t *leaf;size_t ret =0 ; +unsigned char hangul[UTF8HANGULLEAF ];if (!data)return -1 ;while (len && *s) { - leaf =utf8nlookup (data, s, len); + leaf =utf8nlookup (data, hangul, s, len);if (!leaf)return -1 ;if (utf8agetab[LEAF_GEN (leaf)] > data->maxage)
@@ -531,10 +678,12 @@ int utf8byte(struct utf8cursor *u8c) }/* Look up the data for the current character. */ -if (u8c->p) - leaf =utf8lookup (u8c->data, u8c->s); -else - leaf =utf8nlookup (u8c->data, u8c->s, u8c->len); +if (u8c->p) { + leaf =utf8lookup (u8c->data, u8c->hangul, u8c->s); + }else { + leaf =utf8nlookup (u8c->data, u8c->hangul, + u8c->s, u8c->len); + }/* No leaf found implies that the input is a binary blob. */ if (!leaf)
@@ -555,7 +704,9 @@ int utf8byte(struct utf8cursor *u8c) ccc =STOPPER ; goto ccc_mismatch; } - leaf =utf8lookup (u8c->data, u8c->s); + + leaf =utf8lookup (u8c->data, u8c->hangul, u8c->s); + ccc =LEAF_CCC (leaf); }/*
fs/nls/utf8n.h +4 -0
@@ -76,6 +76,9 @@ extern int utf8nagemin(const struct utf8data *data, const char *s, size_t len);extern ssize_t utf8len (const struct utf8data *data,const char *s);extern ssize_t utf8nlen (const struct utf8data *data,const char *s,size_t len); +/* Needed in struct utf8cursor below. */ +#define UTF8HANGULLEAF (12) +/* * Cursor structure used by the normalizer. */
@@ -89,6 +92,7 @@ struct utf8cursor {unsigned int slen;short int ccc;short int nccc; +unsigned char hangul[UTF8HANGULLEAF ]; };/*
scripts/mkutf8data.c +260 -35
592 lines not on this page · raw message
Signature · 1 line
2.17.0
PATCH 14/15 nls: utf8norm: Integrate utf8norm code with NLS subsystem
In reply to Gabriel Krisman BertaziChanges since RFC v2:
- Integrate with NLS
Changes since RFC v1:
- Change error return code from EIO to EINVAL. (Olaf Weber)
- Fix issues with strncmp/strcmp. (Olaf Weber)
- Remove stack buffer in normalization/casefold. (Olaf Weber)
- Include length parameter for second string on comparison functions.
- Change length type to size_t.
--- fs/nls/Makefile | 2 +- fs/nls/nls_utf8n-core.c | 291 ++++++++++++++++++++++++++++++++++++++++ fs/nls/nls_utf8n-norm.c | 6 + fs/nls/utf8n.h | 1 + 4 files changed, 299 insertions(+), 1 deletion(-) create mode 100644 fs/nls/nls_utf8n-core.c
fs/nls/Makefile +1 -1
@@ -55,7 +55,7 @@ obj-$(CONFIG_NLS_MAC_ROMANIAN) += mac-romanian.o obj-$(CONFIG_NLS_MAC_ROMAN) += mac-roman.o obj-$(CONFIG_NLS_MAC_TURKISH) += mac-turkish.o -nls_utf8n-y += nls_utf8n-norm.o +nls_utf8n-y += nls_utf8n-norm.o nls_utf8n-core.o obj-$(CONFIG_NLS_UTF8_NORMALIZATION) += nls_utf8n.o $(obj)/nls_utf8n-norm.o: $(obj)/utf8data.h
fs/nls/nls_utf8n-core.c new file +291 -0
291 lines not on this page · raw message
fs/nls/nls_utf8n-norm.c +6 -0
@@ -38,6 +38,12 @@ int utf8version_is_supported(u8 maj, u8 min, u8 rev) }EXPORT_SYMBOL (utf8version_is_supported); +int utf8version_latest () +{ +return utf8vers; +} +EXPORT_SYMBOL (utf8version_latest); +/* * UTF-8 valid ranges. *
fs/nls/utf8n.h +1 -0
@@ -32,6 +32,7 @@/* Highest unicode version supported by the data tables. */ extern int utf8version_is_supported (u8 maj,u8 min,u8 rev); +extern int utf8version_latest (void );/* * Look for the correct const struct utf8data for a unicode version.
Signature · 1 line
2.17.0
PATCH 15/15 nls: utf8norm: Introduce test module for utf8norm implementation
In reply to Gabriel Krisman BertaziChanges since RFC v2:
- Merge with NLS
Changes since RFC v1:
- Include comparison tests for matching strings with different lengths.
- Include tests for characters included in unicode 8.0.0, 9.0.0 and 10.0.0.
--- fs/nls/Kconfig | 5 + fs/nls/Makefile | 1 + fs/nls/nls_utf8n-selftest.c | 307 ++++++++++++++++++++++++++++++++++++ 3 files changed, 313 insertions(+) create mode 100644 fs/nls/nls_utf8n-selftest.c
fs/nls/Kconfig +5 -0
@@ -624,4 +624,9 @@ config NLS_UTF8_NORMALIZATION help Say Y here to enable utf8 normalization support. +config NLS_UTF8_NORMALIZATION_SELFTEST + tristate "Test UTF-8 normalization support" + default n + depends on NLS_UTF8_NORMALIZATION + endif # NLS
fs/nls/Makefile +1 -0
@@ -57,6 +57,7 @@ obj-$(CONFIG_NLS_MAC_TURKISH) += mac-turkish.o nls_utf8n-y += nls_utf8n-norm.o nls_utf8n-core.o obj-$(CONFIG_NLS_UTF8_NORMALIZATION) += nls_utf8n.o +obj-$(CONFIG_NLS_UTF8_NORMALIZATION_SELFTEST) += nls_utf8n-selftest.o $(obj)/nls_utf8n-norm.o: $(obj)/utf8data.h $(obj)/utf8data.h: $(srctree)/$(src)/ucd/*.txt $(objtree)/scripts/mkutf8data FORCE
fs/nls/nls_utf8n-selftest.c new file +307 -0
307 lines not on this page · raw message
Signature · 1 line
2.17.0