Thread-safety of iconv()

DISCUSSION Marko Myllynen 4 participants 4 messages Sep 15, 2014 – Sep 19, 2014 3 messages not in the archive Man-pages: manual pages for Linux -- Sections 2, 3, 4, 5, and 7

lore

Share & tools

lore

Marko Myllynen

author

2014-09-15 09:32 UTC #0 lore raw

Hi Michael, Haitao,

iconv(3) states:

ATTRIBUTES
Multithreading (see pthreads(7))
The iconv() function is thread-safe.

But iconv_open(3) says:

A conversion descriptor contains a conversion state. After creation
using iconv_open(), the state is in the initial state. Using iconv(3) modifies the descriptor's conversion state. (This implies that a con- version descriptor can not be used in multiple threads simultaneously.) To bring the state back to the initial state, use iconv(3) with NULL as inbuf argument.

This looks a bit unclear, do you think you could clarify iconv(3) and/or iconv_open(3) a bit in thread-safety regard?

Thanks,

Signature · 5 lines

Marko Myllynen
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

Peng Haitao

2014-09-18 06:50 UTC #2 lore raw

Quoted 17 lines · Marko Myllynen

Hi Michael, Haitao,
iconv(3) states:
ATTRIBUTES
Multithreading (see pthreads(7))
The iconv() function is thread-safe.
But iconv_open(3) says:
A conversion descriptor contains a conversion state. After creation
using iconv_open(), the state is in the initial state. Using iconv(3) modifies the descriptor's conversion state. (This implies that a con- version descriptor can not be used in multiple threads simultaneously.) To bring the state back to the initial state, use iconv(3) with NULL as inbuf argument.

Quoted 3 lines

This looks a bit unclear, do you think you could clarify iconv(3) and/or iconv_open(3) a bit in thread-safety regard?

Yes, I will clarify it.

When calling iconv() as iconv(cd, inbuf, ...) and iconv(cd, NULL, ...) in multiple threads simultaneously, Segmentation fault may occur.

So, I think the glibc's level "MT-Safe race:cd" is good.
http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iconv-1

Thanks.

Signature · 6 lines

Best Regards,
Peng
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

Carlos O'Donell

2014-09-18 13:14 UTC #4 lore raw

Quoted 18 lines · Peng Haitao

Quoted 17 lines · Marko Myllynen
Hi Michael, Haitao,
iconv(3) states:
ATTRIBUTES
Multithreading (see pthreads(7))
The iconv() function is thread-safe.
But iconv_open(3) says:
A conversion descriptor contains a conversion state. After creation
using iconv_open(), the state is in the initial state. Using iconv(3) modifies the descriptor's conversion state. (This implies that a con- version descriptor can not be used in multiple threads simultaneously.) To bring the state back to the initial state, use iconv(3) with NULL as inbuf argument.

Quoted 2 lines

Quoted 2 lines
This looks a bit unclear, do you think you could clarify iconv(3) and/or iconv_open(3) a bit in thread-safety regard?

It is conditionally thread safe :-)

Quoted 9 lines

Quoted 1 line
Yes, I will clarify it.
When calling iconv() as iconv(cd, inbuf, ...) and iconv(cd, NULL, ...) in multiple threads simultaneously, Segmentation fault may occur.
So, I think the glibc's level "MT-Safe race:cd" is good.
http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iconv-1

The cd' object is opaque to the user, and therefore in the glibc manual Alex and I felt it was appropriate to call out that it *is* thread safe to call this function but that the iconv_t-type object cd has itself no locking and can be corrupted by multiple calls with the same object, hence race:cd` e.g. there is a data-race calling this function with the same cd object.

Alex, please correct me if I'm wrong, but I think that sums it up.

Cheers,
Carlos.

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

Alexandre Oliva

2014-09-19 02:57 UTC #6 lore raw

Quoted 6 lines · Carlos O'Donell

Quoted 3 lines · Peng Haitao
Quoted 2 lines · Marko Myllynen
This looks a bit unclear, do you think you could clarify iconv(3) and/or iconv_open(3) a bit in thread-safety regard?
It is conditionally thread safe :-)

It is MT-Safe, but it assumes callers arrange for mutual exclusion on the cd object passed as argument.

It is no different from e.g. memset being MT-Safe, in spite of the race when two threads call memset concurrently, writing to the same buffer.

It's caller's responsibility to avoid such races in both cases.

Quoted 1 line

Alex, please correct me if I'm wrong, but I think that sums it up.

Yup. A recent blog post covers this issue (among others) in some detail:

http://developerblog.redhat.com/2014/09/10/multi-thread-async-signal-and-async-cancel-safety-docs-in-gnu-libc/

Signature · 8 lines

Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist|Red Hat Brasil GNU Toolchain Engineer
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html