string_copying.7: don't grant strl{cpy,cat} magic

A function can't check whether a pointer points to the start of a string. What it certainly can do is to keep reading until you either find a null byte or read the secret key that lies adjacent in memory and post it to your favorite mailing list.

strlcpy and strlcat behave the exact same way any other function accepting a string behaves: If you don't pass a string, the behavior is undefined. And that, I believe, does not deserve a special mention here, seeing as all the other string functions don't get such a mention either.

Signed-off-by Lennart Jablonka <[email protected]>

---
Hey Alex!

I don't dislike string_copying(7) overall. This is one of the parts of the content that I dislike---it is false, after all. Besides that:

The "definitions" at the top don't make it clear enough that they aren't supposed to be precise definitions used in your usual C jargon; that while string and a string's length and an object's size are defined by C, and while you understand and sometimes use most of these terms, there is no norm that says "When you talk about a pointer to one past a buffer's last byte, you call it 'end'!" That there is no norm that says "When you say 'copy,' you write to the beginning, not to elsewhere!"

Furthermore, the terminology around "character sequences" confused me while reading the page. When do you have a buffer, neither null-terminated nor null-padded, that is defined not to contain null bytes?

And how do functions behave that want a character sequence if
that does contain a null byte? Do they take the null byte to signal the character sequence's end? Need they accept the null byte as part of the character sequence? Is the behavior undefined?

And lastly, the man page doesn't list the functions' standards or who invented them.

 man7/string_copying.7 | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

man7/string_copying.7 +0 -0

index 04426ef77..308cada36 100644

man7/string_copying.7 +0 -0

 .IP
 .BR stpecpy (3)
 is a simpler alternative to these functions.

Quoted 10 lines

A function can't check whether a pointer points to the start of a string. What it certainly can do is to keep reading until you either find a null byte or read the secret key that lies adjacent in memory and post it to your favorite mailing list.

strlcpy and strlcat behave the exact same way any other function accepting a string behaves: If you don't pass a string, the behavior is undefined. And that, I believe, does not deserve a special mention here, seeing as all the other string functions don't get such a mention either.

Hmm, you're right.

What I intended to mean is that while most other functions --e.g., strcpy(3)-- overwrite after the buffer, the design of strlcpy(3) is a bit more clever and makes it so that when the caller invokes UB, it tries to exploit that UB in a way that the input string is entirely read before starting to write, which makes it more likely to crash in a read, rather than writing to random memory (which might still happen if the read is not enough to crash, though).

Quoted 8 lines

The "definitions" at the top don't make it clear enough that they aren't supposed to be precise definitions used in your usual C jargon; that while string and a string's length and an object's size are defined by C, and while you understand and sometimes use most of these terms, there is no norm that says "When you talk about a pointer to one past a buffer's last byte, you call it 'end'!" That there is no norm that says "When you say 'copy,' you write to the beginning, not to elsewhere!"

True. My intention was to settle the jargon and pseudo-standardize these terms (of course in decades, not tomorrow). Every other project uses a different term, and I'd like to unify.

Quoted 4 lines

Furthermore, the terminology around "character sequences" confused me while reading the page. When do you have a buffer, neither null-terminated nor null-padded, that is defined not to contain null bytes?

NGINX uses these internally:

$ grepc ngx_str_t
./src/core/ngx_string.h:16:
typedef struct {
   size_t      len;
   u_char     *data;
} ngx_str_t;

Basically it's a non-zero buffer plus its length. They have interesting properties; for example, you can take a substring (or should I call it sub-sequence) just by taking a pointer to somewhere in the middle, and the length of the substring, without really copying the string.

Quoted 4 lines

And how do functions behave that want a character sequence if
that does contain a null byte? Do they take the null byte to signal the character sequence's end? Need they accept the null byte as part of the character sequence? Is the behavior undefined?

NGINX handles these strings by the length stored in the buffer. Any null byte in the middle of a string would be treated as any other character, although they would be problematic when interfacing libc; in general, care is taken to not have null bytes in those strings. NGINX uses mempcpy(3) (or rather, ngx_cpymem(), which is the same thing) to copy these things, or other more sophisticated functions and macros based on mempcpy(3).

$ grepc ngx_cpymem
./src/core/ngx_string.h:97:
#define ngx_cpymem(dst, src, n)   (((u_char *) ngx_memcpy(dst, src, n)) + (n))

./src/core/ngx_string.h:107:
#define ngx_cpymem(dst, src, n) (((u_char *) memcpy(dst, src, n)) + (n))

Quoted 2 lines

And lastly, the man page doesn't list the functions' standards or who invented them.

That was deliberate. The specific pages of each of those functions already documents that. Since the intention was to differentiate the use cases of each of the functions, I believe mentioning the standards would just deviate from that main point, and so I omitted that info. The point is that choosing one of these functions shouldn't depend on what standards are available to the programmer. Instead, the programmer should use the appropriate function, and then if it's not available, it should be written within the project (probably as a wrapper around other functions) to be able to use it. That's why I provided some naive implementations of some of them.

Quoted 13 lines

Quoted 10 lines

A function can't check whether a pointer points to the start of a string. What it certainly can do is to keep reading until you either find a null byte or read the secret key that lies adjacent in memory and post it to your favorite mailing list.

strlcpy and strlcat behave the exact same way any other function accepting a string behaves: If you don't pass a string, the behavior is undefined. And that, I believe, does not deserve a special mention here, seeing as all the other string functions don't get such a mention either.

Hmm, you're right.

Quoted 7 lines

What I intended to mean is that while most other functions --e.g., strcpy(3)-- overwrite after the buffer, the design of strlcpy(3) is a bit more clever and makes it so that when the caller invokes UB, it tries to exploit that UB in a way that the input string is entirely read before starting to write, which makes it more likely to crash in a read, rather than writing to random memory (which might still happen if the read is not enough to crash, though).

I’m interested in where you got that from. This is strlcpy:

	size_t
	strlcpy(char *dst, const char *src, size_t dsize)
	{
		const char *osrc = src;
		size_t nleft = dsize;

		/* Copy as many bytes as will fit. */
		if (nleft != 0) {
			/*
			 * <humm> This is where reading and
			 * writing take place.  src doesn’t get
			 * read entirely before writing begins.
			 */
			while (--nleft != 0) {
				if ((*dst++ = *src++) == '\0')
					break;
			}
		}

		/* Not enough room in dst, add NUL and traverse rest of src. */
		if (nleft == 0) {
			if (dsize != 0)
				*dst = '\0';		/* NUL-terminate dst */
			while (*src++)
				;
		}

		return(src - osrc - 1);	/* count does not include NUL */
	}

I don’t see what you mean in there.

Quoted 10 lines

Quoted 9 lines

The "definitions" at the top don't make it clear enough that they aren't supposed to be precise definitions used in your usual C jargon; that while string and a string's length and an object's size are defined by C, and while you understand and sometimes use most of these terms, there is no norm that says "When you talk about a pointer to one past a buffer's last byte, you call it 'end'!" That there is no norm that says "When you say 'copy,' you write to the beginning, not to elsewhere!"

Quoted 3 lines

True. My intention was to settle the jargon and pseudo-standardize these terms (of course in decades, not tomorrow). Every other project uses a different term, and I'd like to unify.

I’m not quite sure yet what to think of that, but I do think you should mention that in the man page. “Here are some terms; C defines ‘string,’ but you are invited to re-use all these terms—it’d be great if we were in unity, after all.”

Quoted 16 lines

Quoted 5 lines

Furthermore, the terminology around "character sequences" confused me while reading the page. When do you have a buffer, neither null-terminated nor null-padded, that is defined not to contain null bytes?

NGINX uses these internally:

$ grepc ngx_str_t
./src/core/ngx_string.h:16:
typedef struct {
   size_t      len;
   u_char     *data;
} ngx_str_t;

Quoted 4 lines

Basically it's a non-zero buffer plus its length. They have interesting properties; for example, you can take a substring (or should I call it sub-sequence) just by taking a pointer to somewhere in the middle, and the length of the substring, without really copying the string.

same with a buffer without restriction of null bytes

Quoted 14 lines

Quoted 5 lines

And how do functions behave that want a character sequence if
that does contain a null byte? Do they take the null byte to signal the character sequence's end? Need they accept the null byte as part of the character sequence? Is the behavior undefined?

NGINX handles these strings by the length stored in the buffer. Any null byte in the middle of a string would be treated as any other character, although they would be problematic when interfacing libc; in general, care is taken to not have null bytes in those strings. NGINX uses mempcpy(3) (or rather, ngx_cpymem(), which is the same thing) to copy these things, or other more sophisticated functions and macros based on mempcpy(3).

Quoted 7 lines

$ grepc ngx_cpymem
./src/core/ngx_string.h:97:
#define ngx_cpymem(dst, src, n)   (((u_char *) ngx_memcpy(dst, src, n)) + (n))

./src/core/ngx_string.h:107:
#define ngx_cpymem(dst, src, n) (((u_char *) memcpy(dst, src, n)) + (n))

I think if you want to pseudo-standardize terminology, you should mention how functions are supposed to behave when seeing a null byte in a character sequence. (I plead for undefined behavior.)

Now, do you think character sequences are more common than simple buffers, than not caring at all about the specific bytes? That’s what I do; that’s what I think should be usually done. And do you suppose character sequences are more valuable than simple buffers? Sure, iff the sequences are null-terminated, you can use them as strings, too, but that doesn’t seem like much of a benefit.

Quoted 4 lines

Quoted 3 lines

And lastly, the man page doesn't list the functions' standards or who invented them.

Quoted 10 lines

That was deliberate. The specific pages of each of those functions already documents that. Since the intention was to differentiate the use cases of each of the functions, I believe mentioning the standards would just deviate from that main point, and so I omitted that info. The point is that choosing one of these functions shouldn't depend on what standards are available to the programmer. Instead, the programmer should use the appropriate function, and then if it's not available, it should be written within the project (probably as a wrapper around other functions) to be able to use it. That's why I provided some naive implementations of some of them.

That’s not a bad idea, but for some functions, this is the man page. I wondered where stpecpy comes from, I tried to open stpecpy(3), I got string_copying(7).

I lied. I should have said that it writes what is safe to write, and then uses a somewhat "safer" version of undefined behavior (compared to other string copying functions). The standard differentiates "bounded UB", which doesn't perform out-of-bounds stores, from "critical UB", which performs them. In usual jargon, UB is UB, and there's no mild form of UB; however, the standard prescribes a bounded form of UB. However, I'm not sure compilers --and specifically GCC-- follow such a prescription of bounded UB, so it's better to consider all UB to be critical UB, just to fall on the safe side.

3.4.3
undefined behavior

behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements

Note 1 to entry: Possible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message).

EXAMPLE An example of undefined behavior is the behavior on integer overflow.

undefined

Describes the nature of a value or behavior not defined by POSIX.1-2017 which results from use of an invalid program construct or invalid data input.

The value or behavior may vary among implementations that conform to POSIX.1-2017. An application should not rely on the existence or validity of the value or behavior. An application that relies on any particular value or behavior cannot be assured to be portable across conforming implementations.

Quoted 9 lines

I lied. I should have said that it writes what is safe to write, and then uses a somewhat "safer" version of undefined behavior (compared to other string copying functions). The standard differentiates "bounded UB", which doesn't perform out-of-bounds stores, from "critical UB", which performs them. In usual jargon, UB is UB, and there's no mild form of UB; however, the standard prescribes a bounded form of UB. However, I'm not sure compilers --and specifically GCC-- follow such a prescription of bounded UB, so it's better to consider all UB to be critical UB, just to fall on the safe side.

Do you have a source for this? As far as I am aware, the standards have always followed the "UB is UB" philosophy, which is why standards-oriented people keep trying to reiterate it. I've never heard of anything like "bounded UB" vs. "critical UB". C17 draft N2176 provides no such distinction in its definition:

References:

<https://port70.net/~nsz/c/c11/n1570.html#L.2>

Quoth Matthew House:

Quoted 11 lines · Alejandro Colomar

Quoted 9 lines

I lied. I should have said that it writes what is safe to write, and then uses a somewhat "safer" version of undefined behavior (compared to other string copying functions). The standard differentiates "bounded UB", which doesn't perform out-of-bounds stores, from "critical UB", which performs them. In usual jargon, UB is UB, and there's no mild form of UB; however, the standard prescribes a bounded form of UB. However, I'm not sure compilers --and specifically GCC-- follow such a prescription of bounded UB, so it's better to consider all UB to be critical UB, just to fall on the safe side.

Quoted 5 lines

Do you have a source for this? As far as I am aware, the standards have always followed the "UB is UB" philosophy, which is why standards-oriented people keep trying to reiterate it. I've never heard of anything like "bounded UB" vs. "critical UB". C17 draft N2176 provides no such distinction in its definition:

Quoth Alejandro Colomar:

Quoted 3 lines

References:

<https://port70.net/~nsz/c/c11/n1570.html#L.2>

Looks like a reference to me.

Yes, UB is UB. The optional Annex L on Analyzability does define bounded and unbounded UB. No, you don’t care about them. Yes, that is standard terminology. No, your implementation doesn’t define __STDC_ANALYZABLE__. Yes, that terminology can be useful.

Quoted 10 lines · Alejandro Colomar

I lied. I should have said that it writes what is safe to write, and then uses a somewhat "safer" version of undefined behavior (compared to other string copying functions). The standard differentiates "bounded UB", which doesn't perform out-of-bounds stores, from "critical UB", which performs them. In usual jargon, UB is UB, and there's no mild form of UB; however, the standard prescribes a bounded form of UB. However, I'm not sure compilers --and specifically GCC-- follow such a prescription of bounded UB, so it's better to consider all UB to be critical UB, just to fall on the safe side.

Do you have a source for this? As far as I am aware, the standards have always followed the "UB is UB" philosophy, which is why standards-oriented people keep trying to reiterate it. I've never heard of anything like "bounded UB" vs. "critical UB".

Hi Matthew,

Quoted 12 lines · Matthew House

Quoted 10 lines · Alejandro Colomar

I lied. I should have said that it writes what is safe to write, and then uses a somewhat "safer" version of undefined behavior (compared to other string copying functions). The standard differentiates "bounded UB", which doesn't perform out-of-bounds stores, from "critical UB", which performs them. In usual jargon, UB is UB, and there's no mild form of UB; however, the standard prescribes a bounded form of UB. However, I'm not sure compilers --and specifically GCC-- follow such a prescription of bounded UB, so it's better to consider all UB to be critical UB, just to fall on the safe side.

Quoted 4 lines

Do you have a source for this? As far as I am aware, the standards have always followed the "UB is UB" philosophy, which is why standards-oriented people keep trying to reiterate it. I've never heard of anything like "bounded UB" vs. "critical UB".

The Ada language standard distinguishes "bounded errors" from "erroneous execution".

http://www.ada-auth.org/standards/12rm/html/RM-1-1-5.html

I've been after Alex for a while to read more about Ada. Maybe he has, and its (usually excellent) approach to attacking problems is seeping into his consciousness. ;-)

Nevertheless I would agree that if WG14 refuses to apply such categories to the C language definition, it's not going to help most users to do so in man pages. I suppose the best route for such a distinction to get into the language is via the GCC and Clang compilers.

Regards,
Branden