Various pages: SYNOPSIS: Use VLA syntax in function parameters

-.BI "char *getcwd(char *" buf ", size_t " size ); +.BI "char *getcwd(char " buf [ size "], size_t " size );

Hi Alejandro,

Quoted 2 lines

-.BI "char *getcwd(char *" buf ", size_t " size ); +.BI "char *getcwd(char " buf [ size "], size_t " size );

I dislike this.

Manual pages should show function prototypes as they really are in the header file, or if the header file contains useless fluff like "restrict", a shortened form showing the essence that actually matters for using the API.

They should certainly not show something imaginary
that does not match reality, and even less so using invalid syntax.

Yours,
Ingo

Quoted 11 lines

Quoted 2 lines · Alejandro Colomar

-.BI "char *getcwd(char *" buf ", size_t " size ); +.BI "char *getcwd(char " buf [ size "], size_t " size );

I dislike this.

Manual pages should show function prototypes as they really are in the header file, or if the header file contains useless fluff like "restrict", a shortened form showing the essence that actually matters for using the API.

Regarding restrict, it is essential to differentiate memcpy(3) and memmove(3), which are otherwise identical:

     void *memmove(void *dest, const void *src, size_t n);

     void *memcpy(void *restrict dest, const void *restrict src,
                  size_t n);

Do you regard the (abused) VLA syntax as something much worse than the use of restrict? Or are they more or less equivalent to you?

Quoted 2 lines

They should certainly not show something imaginary
that does not match reality, and even less so using invalid syntax.

Well, not that I haven't had those thoughts, but we already use ilegal syntax in some cases for good reasons. See for example open(2):

        int open(const char *pathname, int flags);
        int open(const char *pathname, int flags, mode_t mode);

Of course, you can't declare two conflicting prototypes like that.

Quoted 4 lines

      void *memmove(void *dest, const void *src, size_t n);

      void *memcpy(void *restrict dest, const void *restrict src,
                   size_t n);

Actually, the syntax of both is identical, only the semantics differ.

That said, you are right that using memcpy(3) when memmove(3) is required is a famous and widespread bug. I doubt putting "restrict" into the SYNOPSIS will discourage careless programmers from making that mistake though.

To me, "restrict" feels like a specialized tool for people writing compiler optimizers, not like something important enough to clutter API documenation.

Quoted 2 lines

Do you regard the (abused) VLA syntax as something much worse than the use of restrict? Or are they more or less equivalent to you?

If your implementation really contains "restrict" in the header file and it's standardized, putting it into the SYNOPSIS seems acceptable to me. Not necessary though and maybe somewhat noisy and distracting.

Putting something that is not in the implementation and/or not in the standard into the SYNOPSIS seems much worse to me.

And invalid syntax in the SYNOPSIS is even worse than that. For example, people may attempt to use SYNOPSIS as an example when designing their own, private function for a similar but not identical purpose and end up writing non-portable code, or even code that does not compile anywhere.

They may be wrong if they blame you for that, but i doubt they will thank you.

Quoted 11 lines

Quoted 3 lines

They should certainly not show something imaginary
that does not match reality, and even less so using invalid syntax.

Well, not that I haven't had those thoughts, but we already use ilegal syntax in some cases for good reasons. See for example open(2):

         int open(const char *pathname, int flags);
         int open(const char *pathname, int flags, mode_t mode);

Of course, you can't declare two conflicting prototypes like that.

This does not seem quite as horrifying as

   char *getcwd(char buf[size], size_t size);

because at least each of the prototypes is valid.

My main concern about it would be that it is likely to make some people think (and C++ programmers in particular :-/) that there is type checking for the third and subsequent arguments, in which case they will be unpleasantly surprised when accidentally writing something like

   open(pathname, flags, &some_var, mode);

and finding out later that it compiled and ran just fine, but the resulting file wasn't quite as confidential as they hoped.

Explicitly displaying the ... to indicate the variable number of arguments, by contrast, makes it very clear that an API is almost certainly unusually dangerous and needs to be used with especial diligence.

Either way, certainly not quite as bad as invalid syntax inside a prototype...

Yours,
Ingo

    void *memcpy(void dest[restrict n], const void src[restrict n], size_t n);

    void *memcpy(void *restrict dest, const void *restrict src, size_t n);

    void *memcpy(void *dest, const void *src, size_t n);

    void *memcpy(void *dest, void *src, size_t n);

    void *memcpy(void *, void *, size_t);

Subject:     Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters Date:     Fri, 2 Sep 2022 16:56:00 -0400
From:     JeanHeyd Meneide <[email protected]>
To:     Alejandro Colomar <[email protected]>
CC:     Ingo Schwarze <[email protected]>, [email protected]

Hi Alejandro and Ingo,

      Just chiming in from a Standards perspective, here. We discussed,
briefly, a way to allow Variable-Length function parameter declarations like the ones shown in this thread (e.g., char *getcwd(char buf[size], size_t size );).

      In GCC, there is a GNU extension that allows explicitly
forward-declaring the prototype. Using the above example, it would look like so:

char *getcwd(size_t size; char buf[size], size_t size);

(Live Example [1])

(Note the ; after the first "size" declaration). This was brought before the Committee to vote on for C23 in the form of N2780 [2], around the January 2022 timeframe. The paper did not pass, and it was seen as a "failed extension". After the vote on that failed, we talked about other ways of allowing places whether there was some appetite to allow "forward parsing" for this sort of case. That is, could we simply allow:

char *getcwd(char buf[size], size_t size);

to work as expected. The vote for this did not gain full consensus either, but there were a lot of abstentions [3]. While I personally voted in favor of allowing such for C, there was distinct worry that this would produce issues for weaker C implementations that did not want to commit to delayed parsing or forward parsing of the entirety of the argument list before resolving types. There are enough abstentions during voting that a working implementation with a writeup of complexity would sway the Committee one way or the other.

This is not to dissuade Alejandro's position, or to bolster Ingo's point; I'm mostly just reporting the Committee's response here. This is an unsolved problem for the Committee, and also a larger holdover from the removal of K&R declarations from C23, which COULD solve this problem:

// decl
char *getcwd();

// impl
char* getcwd(buf, size)
char buf[size];
      size_t size;
{
      /* impl here */
}

      There is room for innovation here, or perhaps bolstering of the
GCC original extension. As it stands right now, compilers only very recently started taking Variably-Modified Type parameters and Static Extent parameters seriously after carefully separating them out of Variable-Length Arrays, warning where they can when static or other array parameters do not match buffer lengths and so-on.

      Not just to the folks in this thread, but to the broader
community for anyone who is paying attention: WG14 would actively like to solve this problem. If someone can:
- prove out a way to do delayed parsing that is not implementation-costly,
- revive the considered-dead GCC extension, or
- provide a 3rd or 4th way to support the goals,

I am certain WG14 would look favorably upon such a thing eventually, brought before the Committee in inclusion for C2y/C3a.

      Whether or not you feel like the manpages are the best place to
start that, I'll leave up to you!

Quoted 3 lines

Whether or not you feel like the manpages are the best place to
start that, I'll leave up to you!

I'll try to defend the reasons to start this in the man-pages.

This feature is mostly for documentation purposes, not being meaningful for code at all (for some meaning of meaningful), since it won't change the function definition in any way, nor the calls to it. At least not by itself; static analysis may get some benefits, though.

Also, new code can be designed from the beginning so that sizes go before their corresponding arrays, so that new code won't typically be affected by the lack of this feature in the language.

This leaves us with legacy code, especially libc, which just works, and doesn't have any urgent needs to change their prototypes in this regard (they could, to improve static analysis, but not what we'd call urgent).

And since most people don't go around reading libc headers searching for function declarations (especially since there are manual pages that show them nicely), it's not like the documentation of the code depends on how the function is _actually_ declared in code (that's why I also defended documenting restrict even if glibc wouldn't have cared to declare it), but it depends basically on what the manual pages say about the function. If the manual pages say a function gets 'restrict' params, it means it gets 'restrict' params, no matter what the code says, and if it doesn't, the function accepts overlapping pointers, at least for most of the public (modulo manual page bugs, that is).

So this extension could very well be added by the manual pages, as a form of documentation, and then maybe picked up by compilers that have enough resources to implement it.

Considering that this feature is mostly about documentation (and a bit of static analysis too), the documentation should be something appealing to the reader.

Let's take an example:

int getnameinfo(const struct sockaddr *restrict addr,
socklen_t addrlen,
char *restrict host, socklen_t hostlen, char *restrict serv, socklen_t servlen, int flags);

and some transformations:

int getnameinfo(const struct sockaddr *restrict addr,
socklen_t addrlen,
char host[restrict hostlen], socklen_t hostlen, char serv[restrict servlen], socklen_t servlen, int flags);

int getnameinfo(socklen_t hostlen;
socklen_t servlen;
const struct sockaddr *restrict addr, socklen_t addrlen,
char host[restrict hostlen], socklen_t hostlen, char serv[restrict servlen], socklen_t servlen, int flags);

(I'm not sure if I used correct GNU syntax, since I never used that extension myself.)

The first transformation above is non-ambiguous, as concise as possible, and its only issue is that it might complicate the implementation a bit too much. I don't think forward-using a parameter's size would be too much of a parsing problem for human readers.

The second one is unnecessarily long and verbose, and semicolons are not very distinguishable from commas, for human readers, which may be very confusing.

        int foo(int a; int b[a], int a);
        int foo(int a, int b[a], int o);

Those two are very different to the compiler, and yet very similar to the human eye. I don't like it. The fact that it allows for simpler compilers isn't enough to overcome the readability issues.

I think I'd prefer having the forward-using syntax as a non-standard extension --or a standard but optional language feature-- to avoid forcing small compilers to implement it, rather than having the GNU extension standardized in all compilers.

the only point i strongly care about is this one:

Manual pages should not use
* non-standard syntax
* non-portable syntax
* ambiguous syntax (i.e. syntax that might have different meanings with different compilers or in different contexts)
* syntax that might be invalid or dangerous with some widely used compiler collections like GCC or LLVM

Regarding the discussions about standardization and extensions, all proposals i have seen look seriously ugly and awkward to me, and i'm not yet convinced such ugliness is sufficiently offset by the relatively minor benefit that is apparent to me right now.

GCC will warn if the bound is specified inconsistently between declarations and also emit warnings if it can see that a buffer which is passed is too small:

https://godbolt.org/z/PsjPG1nv7

BTW: If you declare pointers to arrays (not first elements) you can get run-time bounds checking with UBSan:

https://godbolt.org/z/TvMo89WfP

Quoted 7 lines

Also, new code can be designed from the beginning so that sizes go before their corresponding arrays, so that new code won't typically be affected by the lack of this feature in the language.

Quoted 3 lines

This leaves us with legacy code, especially libc, which just works, and doesn't have any urgent needs to change their prototypes in this regard (they could, to improve static analysis, but not what we'd call urgent).

It would be useful step to find out-of-bounds problem in applications using libc.

Quoted 27 lines

Let's take an example:

int getnameinfo(const struct sockaddr *restrict addr,
socklen_t addrlen,
char *restrict host, socklen_t hostlen, char *restrict serv, socklen_t servlen, int flags);

and some transformations:

int getnameinfo(const struct sockaddr *restrict addr,
socklen_t addrlen,
char host[restrict hostlen], socklen_t hostlen, char serv[restrict servlen], socklen_t servlen, int flags);

int getnameinfo(socklen_t hostlen;
socklen_t servlen;
const struct sockaddr *restrict addr, socklen_t addrlen,
char host[restrict hostlen], socklen_t hostlen, char serv[restrict servlen], socklen_t servlen, int flags);

Quoted 7 lines

(I'm not sure if I used correct GNU syntax, since I never used that extension myself.)

The first transformation above is non-ambiguous, as concise as possible, and its only issue is that it might complicate the implementation a bit too much. I don't think forward-using a parameter's size would be too much of a parsing problem for human readers.

I personally find the second form not terrible. Being able to read code left-to-right, top-down is helpful in more complicated examples.

Quoted 10 lines

The second one is unnecessarily long and verbose, and semicolons are not very distinguishable from commas, for human readers, which may be very confusing.

         int foo(int a; int b[a], int a);
         int foo(int a, int b[a], int o);

Those two are very different to the compiler, and yet very similar to the human eye. I don't like it. The fact that it allows for simpler compilers isn't enough to overcome the readability issues.

This is true, I would probably use it with a comma and/or syntax highlighting.

Quoted 4 lines

I think I'd prefer having the forward-using syntax as a non-standard extension --or a standard but optional language feature-- to avoid forcing small compilers to implement it, rather than having the GNU extension standardized in all compilers.

The problems with the second form are:

- it is not 100% backwards compatible (which maybe ok though) as the semantics of the following code changes:

int n;
int foo(int a[n], int n); // refers to different n!

Code written for new compilers could then be misunderstood by old compilers when a variable with 'n' is in scope.

- it would generally be fundamentally new to C to have backwards references and parser might need to be changes to allow this

- a compiler or tool then has to deal also with ugly corner cases such as mutual references:

int foo(int (*a)[sizeof(*b)], int (*b)[sizeof(*a)]);

We could consider new syntax such as

int foo(char buf[.n], int n);

Personally, I would prefer the conceptual simplicity of forward declarations and the fact that these exist already in GCC over any alternative. I would also not mind new syntax, but then one has to define the rules more precisely to avoid the aforementioned problems.

Hi Martin,

On 9/3/22 14:47, Martin Uecker wrote:
[...]

Quoted 5 lines

GCC will warn if the bound is specified inconsistently between declarations and also emit warnings if it can see that a buffer which is passed is too small:

https://godbolt.org/z/PsjPG1nv7

That's very good news!

BTW, it's nice to see that GCC doesn't need 'static' for array parameters. I never understood what the static keyword adds there. There's no way one can specify an array size an mean anything other than requiring that, for a non-null pointer, the array should have at least that size.

Quoted 5 lines

BTW: If you declare pointers to arrays (not first elements) you can get run-time bounds checking with UBSan:

https://godbolt.org/z/TvMo89WfP

Couldn't that be caught at compile time? n is certainly out of bounds always for such an array, since the last element is n-1.

Quoted 11 lines

Quoted 8 lines

Also, new code can be designed from the beginning so that sizes go before their corresponding arrays, so that new code won't typically be affected by the lack of this feature in the language.

This leaves us with legacy code, especially libc, which just works, and doesn't have any urgent needs to change their prototypes in this regard (they could, to improve static analysis, but not what we'd call urgent).

It would be useful step to find out-of-bounds problem in applications using libc.

Yep, it would be very useful for that. Not urgent, but yes, very useful.

Quoted 63 lines

Quoted 27 lines

Let's take an example:

int getnameinfo(const struct sockaddr *restrict addr,
socklen_t addrlen,
char *restrict host, socklen_t hostlen, char *restrict serv, socklen_t servlen, int flags);

and some transformations:

int getnameinfo(const struct sockaddr *restrict addr,
socklen_t addrlen,
char host[restrict hostlen], socklen_t hostlen, char serv[restrict servlen], socklen_t servlen, int flags);

int getnameinfo(socklen_t hostlen;
socklen_t servlen;
const struct sockaddr *restrict addr, socklen_t addrlen,
char host[restrict hostlen], socklen_t hostlen, char serv[restrict servlen], socklen_t servlen, int flags);

Quoted 7 lines

(I'm not sure if I used correct GNU syntax, since I never used that extension myself.)

The first transformation above is non-ambiguous, as concise as possible, and its only issue is that it might complicate the implementation a bit too much. I don't think forward-using a parameter's size would be too much of a parsing problem for human readers.

I personally find the second form not terrible. Being able to read code left-to-right, top-down is helpful in more complicated examples.

Quoted 10 lines

The second one is unnecessarily long and verbose, and semicolons are not very distinguishable from commas, for human readers, which may be very confusing.

         int foo(int a; int b[a], int a);
         int foo(int a, int b[a], int o);

Those two are very different to the compiler, and yet very similar to the human eye. I don't like it. The fact that it allows for simpler compilers isn't enough to overcome the readability issues.

This is true, I would probably use it with a comma and/or syntax highlighting.

Quoted 4 lines

I think I'd prefer having the forward-using syntax as a non-standard extension --or a standard but optional language feature-- to avoid forcing small compilers to implement it, rather than having the GNU extension standardized in all compilers.

The problems with the second form are:

Quoted 10 lines

- it is not 100% backwards compatible (which maybe ok though) as the semantics of the following code changes:

int n;
int foo(int a[n], int n); // refers to different n!

Code written for new compilers could then be misunderstood by old compilers when a variable with 'n' is in scope.

Hmmm, this one is serious. I can't seem to solve it with that syntax.

Quoted 14 lines

- it would generally be fundamentally new to C to have backwards references and parser might need to be changes to allow this

- a compiler or tool then has to deal also with ugly corner cases such as mutual references:

int foo(int (*a)[sizeof(*b)], int (*b)[sizeof(*a)]);

We could consider new syntax such as

Quoted 8 lines

int foo(char buf[.n], int n);

Personally, I would prefer the conceptual simplicity of forward declarations and the fact that these exist already in GCC over any alternative. I would also not mind new syntax, but then one has to define the rules more precisely to avoid the aforementioned problems.

What about taking something from K&R functions for this?:

int foo(q; w; int a[q], int q, int s[w], int w);

By not specifying the types, the syntax is again short.
This is left-to-right, so no problems with global variables, and no need for complex parsers.
Also, by not specifying types, now it's more obvious to the naked eye that there's a difference:

           int foo(a; int b[a], int a);
           int foo(int a, int b[a], int o);

What do you think about this syntax?

Am Samstag, den 03.09.2022, 15:41 +0200 schrieb Alejandro Colomar:

Quoted 11 lines

Hi Martin,

On 9/3/22 14:47, Martin Uecker wrote:
[...]

Quoted 5 lines

GCC will warn if the bound is specified inconsistently between declarations and also emit warnings if it can see that a buffer which is passed is too small:

https://godbolt.org/z/PsjPG1nv7

Quoted 7 lines

That's very good news!

BTW, it's nice to see that GCC doesn't need 'static' for array parameters. I never understood what the static keyword adds there. There's no way one can specify an array size an mean anything other than requiring that, for a non-null pointer, the array should have at least that size.

From the C standard's point of view,

void foo(int n, char buf[n]);

is semantically equivalent to

void foo(int, char *buf);

and without 'static' the 'n' has no further meaning (this is different for pointers to arrays).

The static keyword implies that the pointer is be valid and non-zero and that there must be at least 'n' elements accessible, so in some sense it is stronger (it implies alid non-zero pointers), but at the same time it does not imply a bound.

But I agree that 'n' without 'static' should simply imply a bound and I think we should use it this way even when the standard currently does not attach a meaning to it.

Quoted 9 lines

What about taking something from K&R functions for this?:

int foo(q; w; int a[q], int q, int s[w], int w);

By not specifying the types, the syntax is again short.
This is left-to-right, so no problems with global variables, and no need for complex parsers.
Also, by not specifying types, now it's more obvious to the naked eye that there's a difference:

I am ok with the syntax, but I am not sure how this would work. If the type is determined only later you would still have to change parsers (some C compilers do type checking and folding during parsing, so need the types to be known during parsing) and you also still have the problem with the mutual dependencies.

We thought about using this syntax

int foo(char buf[.n], int n);

because it is new syntax which means we can restrict the size to be the name of a parameter instead of allowing arbitrary expressions, which then makes forward references less problematic. It is also consistent with designators in initializers and could also be extend to annotate flexible array members or for storing pointers to arrays in structures:

struct {
   int n;
   char buf[.n];
};

struct {
   int n;
   char (*buf)[.n];
};

Hi Martin,

Quoted 24 lines · Martin Uecker

Am Samstag, den 03.09.2022, 15:41 +0200 schrieb Alejandro Colomar:

Quoted 11 lines

Hi Martin,

On 9/3/22 14:47, Martin Uecker wrote:
[...]

Quoted 5 lines

GCC will warn if the bound is specified inconsistently between declarations and also emit warnings if it can see that a buffer which is passed is too small:

https://godbolt.org/z/PsjPG1nv7

Quoted 7 lines

That's very good news!

BTW, it's nice to see that GCC doesn't need 'static' for array parameters. I never understood what the static keyword adds there. There's no way one can specify an array size an mean anything other than requiring that, for a non-null pointer, the array should have at least that size.

From the C standard's point of view,

void foo(int n, char buf[n]);

Quoted 6 lines

is semantically equivalent to

void foo(int, char *buf);

and without 'static' the 'n' has no further meaning (this is different for pointers to arrays).

I know. I just don't understand the rationale for that decission. :/

Quoted 5 lines

The static keyword implies that the pointer is be valid and non-zero and that there must be at least 'n' elements accessible, so in some sense it is stronger (it implies alid non-zero pointers), but at the same time it does not imply a bound.

That stronger meaning, I think is a mistake by the standard. Basically, [static n] means the same as [n] combined with [[gnu::nonnull]]. What the standard should have done would be to keep those two things separate, since one may want to declare non-null non-array pointers, or possibly-null array ones. So the standard should have standardized some form of nonnull for that.

But the recent discussion about presenting
nonnull pointers as [static 1] is horrible. But let's wait till the future hopefully fixes this.

Quoted 3 lines

But I agree that 'n' without 'static' should simply imply a bound and I think we should use it this way even when the standard currently does not attach a meaning to it.

Yep.

[...]

Quoted 10 lines

Quoted 9 lines

What about taking something from K&R functions for this?:

int foo(q; w; int a[q], int q, int s[w], int w);

By not specifying the types, the syntax is again short.
This is left-to-right, so no problems with global variables, and no need for complex parsers.
Also, by not specifying types, now it's more obvious to the naked eye that there's a difference:

Quoted 6 lines

I am ok with the syntax, but I am not sure how this would work. If the type is determined only later you would still have to change parsers (some C compilers do type checking and folding during parsing, so need the types to be known during parsing) and you also still have the problem with the mutual dependencies.

This syntax resembles a lot K&R syntax. Any C compiler that supports them (and I guess most compilers out there do) should be easily convertible to support this syntax (at least more easily than other alternatives). But this is just a guess.

Quoted 11 lines

We thought about using this syntax

int foo(char buf[.n], int n);

because it is new syntax which means we can restrict the size to be the name of a parameter instead of allowing arbitrary expressions, which then makes forward references less problematic. It is also consistent with designators in initializers and could also be extend to annotate flexible array members or for storing pointers to arrays in structures:

It's not crazy. I don't have much to argue against it.

Quoted 5 lines

struct {
   int n;
   char buf[.n];
};

Quoted 4 lines

struct {
   int n;
   char (*buf)[.n];
};

Perhaps some doubts about how this would work for nested structures, but not unreasonable.

Cheers,

Alex

Signature · 2 lines

Alejandro Colomar
<http://www.alejandro-colomar.es/>

Quoted 3 lines

But the recent discussion about presenting
nonnull pointers as [static 1] is horrible. But let's wait till the future hopefully fixes this.

yes, [static 1] is problematic because then the number can not be used as a bound anymore.

My experience is that if one wants to see something fixed, one has to push for it. Standardization is meant to standardize existing practice, so if we want to see this improved, we can not wait for this.

In K&R syntax this worked for definition:

void foo(y, n)
int n;
int y[n];
{ ...

But this worked because you could reorder the
declarations so that later declarations could
refer to previous ones.

So one could do

int foo(int n, char buf[n]; buf, n);

where the second part defines the order of
the parameter or

int foo(buf, n; int n, char buf[n]);

where the first part defins the order,
but the declarations need to have the size
first. But then you need to specify each
parameter twice...

Quoted 26 lines

Quoted 6 lines

We thought about using this syntax

int foo(char buf[.n], int n);

Quoted 7 lines

because it is new syntax which means we can restrict the size to be the name of a parameter instead of allowing arbitrary expressions, which then makes forward references less problematic. It is also consistent with designators in initializers and could also be extend to annotate flexible array members or for storing pointers to arrays in structures:

It's not crazy. I don't have much to argue against it.

Quoted 9 lines

struct {
    int n;
    char buf[.n];
};

struct {
    int n;
    char (*buf)[.n];
};

Quoted 2 lines

Perhaps some doubts about how this would work for nested structures, but not unreasonable.

It is not implemented though...

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

We thought about using this syntax

int foo(char buf[.n], int n);

My experience is that if one wants to see something fixed, one has to push for it. Standardization is meant to standardize existing practice, so if we want to see this improved, we can not wait for this.

Hi Martin,

Quoted 4 lines · Martin Uecker

My experience is that if one wants to see something fixed, one has to push for it.  Standardization is meant to standardize existing practice, so if we want to see this improved, we can not wait for this.

I fully agree with you.  I've been ruminating these patches for some time, for having some more time to think about them.  Now, I like them enough to push. So, after a few minor cosmetic issues detected by some linters, I've pushed the changes to document all of man2 and man3 with hypothetical VLA syntax.

Now, I've released man-pages-6.01 very recently (just a few weeks ago), and I don't plan to release again in a year or two, so there's time to do the implementation in GCC.  From my side, please consider this an ACK or even somewhat of a push to get things done in the compiler side of things :)

I'll show here an excerpt of what kind of syntax has been pushed.  Of course, there's room for improving/fixing, since it's not seen an official release, but for now, this is what's up there:

       int strncmp(const char s1[.n], const char s2[.n], size_t n);

       long mbind(void addr[.len], unsigned long len, int mode, const unsigned long nodemask[(.maxnode + ULONG_WIDTH ‐ 1) / ULONG_WIDTH], unsigned long maxnode, unsigned int flags);

       int cacheflush(void addr[.nbytes], int nbytes, int cache);

I've shown the three kinds of prototypes that have been changed:

-  Normal VLA; nothing fancy except for the '.'.
-  Complex size expressions.
-  'void *' VLAs (assuming GNU conventions: sizeof(void *)==1).

Cheers,

Alex

Review

Quoted 5 lines · Alejandro Colomar via Gcc

I've shown the three kinds of prototypes that have been changed:

(quoted diff) +0 -3

-  Normal VLA; nothing fancy except for the '.'.
-  Complex size expressions.
-  'void *' VLAs (assuming GNU conventions: sizeof(void *)==1).

That doesn't cover any of the tricky issues with such proposals, such as the choice of which entity is referred to by the parameter name when there are multiple nested parameter lists that use the same parameter name, or when the identifier is visible from an outer scope (including in particular the case where it's declared as a typedef name in an outer scope).

Quoted 9 lines · Alejandro Colomar via Gcc

Review

Quoted 6 lines

I've shown the three kinds of prototypes that have been changed:

(quoted diff) +0 -3

-  Normal VLA; nothing fancy except for the '.'.
-  Complex size expressions.
-  'void *' VLAs (assuming GNU conventions: sizeof(void *)==1).

Quoted 6 lines

That doesn't cover any of the tricky issues with such proposals, such as the choice of which entity is referred to by the parameter name when there are multiple nested parameter lists that use the same parameter name, or when the identifier is visible from an outer scope (including in particular the case where it's declared as a typedef name in an outer scope).

In fact I can't tell from these examples whether you mean for a '.' token after '[' to have special semantics, or whether you mean to have a special '. identifier' form of expression valid in certain context (each of which introduces its own complications; for the former, typedef names from outer scopes are problematic; for the latter, it's designated initializers where you get complications, for example). Designing new syntax that doesn't cause ambiguity is generally tricky, and this sort of language extension is the kind of thing where you'd expect to so through at least five iterations of a WG14 paper before you have something like a sound specification.

Am Donnerstag, den 10.11.2022, 01:39 +0000 schrieb Joseph Myers:

Quoted 17 lines · Joseph Myers

Quoted 9 lines · Alejandro Colomar via Gcc

Review

Quoted 6 lines

I've shown the three kinds of prototypes that have been changed:

(quoted diff) +0 -3

-  Normal VLA; nothing fancy except for the '.'.
-  Complex size expressions.
-  'void *' VLAs (assuming GNU conventions: sizeof(void *)==1).

Quoted 6 lines

That doesn't cover any of the tricky issues with such proposals, such as the choice of which entity is referred to by the parameter name when there are multiple nested parameter lists that use the same parameter name, or when the identifier is visible from an outer scope (including in particular the case where it's declared as a typedef name in an outer scope).

Quoted 10 lines

In fact I can't tell from these examples whether you mean for a '.' token after '[' to have special semantics, or whether you mean to have a special '. identifier' form of expression valid in certain context (each of which introduces its own complications; for the former, typedef names from outer scopes are problematic; for the latter, it's designated initializers where you get complications, for example). Designing new syntax that doesn't cause ambiguity is generally tricky, and this sort of language extension is the kind of thing where you'd expect to so through at least five iterations of a WG14 paper before you have something like a sound specification.

I am not sure what Alejandro has in mind exactly, but my idea of using a new notation [.identifier] would be to limit it to accessing other parameter names in the same parameter list only, so that there is

1) no ambiguity what is referred to and
2) one can access parameters which come later

If we want to specify something like this, I think we should also restrict what kind of expressions one allows, e.g. it has to be side-effect free.

But maybe we want to make this even more
restrictive (at least initially).

One problem with WG14 papers is that people put in too much, because the overhead is so high and the standard is not updated very often. It would be better to build such feature more incrementally, which could be done more easily with a compiler extension. One could start supporting just [.x] but not more complicated expressions.

Later WG14 can still accept or reject or modify this proposal based on the experience we get.

(I would also be happy with using GNU forward declarations, and I am not sure why people dislike them so much.)

Martin

One problem with WG14 papers is that people put in too much, because the overhead is so high and the standard is not updated very often. It would be better to build such feature more incrementally, which could be done more easily with a compiler extension. One could start supporting just [.x] but not more complicated expressions.

Quoted 6 lines · Martin Uecker via Gcc

One problem with WG14 papers is that people put in too much, because the overhead is so high and the standard is not updated very often. It would be better to build such feature more incrementally, which could be done more easily with a compiler extension. One could start supporting just [.x] but not more complicated expressions.

Even a compiler extension requires the level of detail of specification that you get with a WG14 paper (and the level of work on finding bugs in that specification), to avoid the problem we've had before with too many features added in GCC 2.x days where a poorly defined feature is "whatever the compiler accepts".

If you use .x as the notation but don't limit it to [.x], you have a completely new ambiguity between ordinary identifiers and member names

struct s { int a; };
void f(int a, int b[((struct s) { .a = 1 }).a]);

where it's newly ambiguous whether ".a = 1" is an assignment to the expression ".a" or a use of a designated initializer.

(I think that if you add any syntax for this, GNU VLA forward declarations are clearly to be preferred to inventing something new like [.x] which introduces its own problems.)

Quoted 7 lines

One problem with WG14 papers is that people put in too much, because the overhead is so high and the standard is not updated very often. It would be better to build such feature more incrementally, which could be done more easily with a compiler extension. One could start supporting just [.x] but not more complicated expressions.

Even a compiler extension requires the level of detail of specification that you get with a WG14 paper (and the level of work on finding bugs in that specification), to avoid the problem we've had before with too many features added in GCC 2.x days where a poorly defined feature is "whatever the compiler accepts".

If you use .x as the notation but don't limit it to [.x], you have a completely new ambiguity between ordinary identifiers and member names

struct s { int a; };
void f(int a, int b[((struct s) { .a = 1 }).a]);

where it's newly ambiguous whether ".a = 1" is an assignment to the expression ".a" or a use of a designated initializer.

(I think that if you add any syntax for this, GNU VLA forward declarations are clearly to be preferred to inventing something new like [.x] which introduces its own problems.)

Quoted 5 lines

Even a compiler extension requires the level of detail of specification that you get with a WG14 paper (and the level of work on finding bugs in that specification), to avoid the problem we've had before with too many features added in GCC 2.x days where a poorly defined feature is "whatever the compiler accepts".

I think the effort needed to specify the feature correctly can be minimized by making the first version of the feature as simple as possible.

Quoted 10 lines

Quoted 6 lines

Even a compiler extension requires the level of detail of specification that you get with a WG14 paper (and the level of work on finding bugs in that specification), to avoid the problem we've had before with too many features added in GCC 2.x days where a poorly defined feature is "whatever the compiler accepts".

I think the effort needed to specify the feature correctly can be minimized by making the first version of the feature as simple as possible.

The version of constexpr in the current C2x working draft is more or less as simple as possible. It also went through lots of revisions to get there. I'm currently testing an implementation of C2x constexpr for GCC 13, and there are still several issues with the specification I found in the implementation process, beyond those raised in WG14 discussions, for which I'll need to raise NB comments to clarify things.

I think that illustrates that you need the several iterations on the specification process, *and* making it as simple as possible, *and* getting implementation experience, *and* the implementation experience being with a close eye to what it implies for all the details in the specification rather than just getting something vaguely functional but not clearly specified.

Quoted 6 lines · Martin Uecker via Gcc

One problem with WG14 papers is that people put in too much, because the overhead is so high and the standard is not updated very often. It would be better to build such feature more incrementally, which could be done more easily with a compiler extension. One could start supporting just [.x] but not more complicated expressions.

Even a compiler extension requires the level of detail of specification that you get with a WG14 paper (and the level of work on finding bugs in that specification), to avoid the problem we've had before with too many features added in GCC 2.x days where a poorly defined feature is "whatever the compiler accepts".

If you use .x as the notation but don't limit it to [.x], you have a completely new ambiguity between ordinary identifiers and member names

struct s { int a; };
void f(int a, int b[((struct s) { .a = 1 }).a]);

where it's newly ambiguous whether ".a = 1" is an assignment to the expression ".a" or a use of a designated initializer.

(I think that if you add any syntax for this, GNU VLA forward declarations are clearly to be preferred to inventing something new like [.x] which introduces its own problems.)

struct s {
    int a;
};

struct t {
    struct s s;
    int a;
};

void f(void)
{
    struct t x = {
        .a = 1,
        .s = {
            .a = ((struct s) {.a = 1}).a,
        },
    };
}

Quoted 2 lines

struct s { int a; };
void f(int a, int b[((struct s) { .a = 1 }).a]);

Is it really ambiguous? Let's show some currently-valid code:

Quoted 4 lines · Alejandro Colomar via Gcc

Quoted 2 lines

struct s { int a; };
void f(int a, int b[((struct s) { .a = 1 }).a]);

Is it really ambiguous? Let's show some currently-valid code:

Well, I still don't know what the syntax addition you propose is. Is it

postfix-expression : . identifier

(with a special rule about how the identifier is interpreted, different from the normal scope rules)? If so, then ".a = 1" could either match assignment-expression directly (assigning to the postfix-expression ".a").

Or it could match designation[opt] initializer, where ".a" is a designator. And as I've noted many times in discussions of C2x proposals on the WG14 reflector, if some sequence of tokens can match the syntax in more than one way, there always needs to be explicit normative text to disambiguate the intended parse - it's not enough that one parse might lead later to a violation of some other constraint (not that either parse leads to a constraint violation in this case).

Or is the syntax

array-declarator : direct-declarator [ . assignment-expression ]

(with appropriate variants with static and type-qualifier-list and for array-abstract-declarator as well, and with different identifier interpretation rules inside the assignment-expression)? If so, then there are big problems parsing [ . ( a ) + ( b ) ], where 'a' is a typedef name in an outer scope, because the appropriate parse would depend on whether 'a' is shadowed by a parameter - unless of course you add appropriate wording like that present in some places about not being able to use this syntax to shadow a typedef name.

Or is it just

array-declarator : direct-declarator [ . identifier ]

which might avoid some of these problems at the expense of being less expressive?

If you're proposing a C syntax addition, you always need to be clear about exactly what the new cases in the syntax would be, and how you resolve ambiguities with any other existing part of the syntax, how you interact with rules on scopes, namespaces and linkage of identifiers, etc.

Hi Joseph,

Quoted 10 lines · Joseph Myers

Quoted 4 lines · Alejandro Colomar via Gcc

Quoted 2 lines

struct s { int a; };
void f(int a, int b[((struct s) { .a = 1 }).a]);

Is it really ambiguous?  Let's show some currently-valid code:

Well, I still don't know what the syntax addition you propose is.  Is it

postfix-expression : . identifier

I'll try to explain it in standardeese, but I'm not sure if I'll get it right, so I'll accompany it with plain English.

Maybe Martin can help.

Since it's to be used as an rvalue, not as a lvalue, I guess a postfix-expression wouldn't be the right one.

Quoted 4 lines

(with a special rule about how the identifier is interpreted, different from the normal scope rules)?  If so, then ".a = 1" could either match assignment-expression directly (assigning to the postfix-expression ".a").

No, assigning to a function parameter from within another parameter declaration wouldn't make sense.  They should be readonly.  Side effects should be forbidden, I think.

Quoted 11 lines

Or it could match designation[opt] initializer, where ".a" is a designator.  And as I've noted many times in discussions of C2x proposals on the WG14 reflector, if some sequence of tokens can match the syntax in more than one way, there always needs to be explicit normative text to disambiguate the intended parse - it's not enough that one parse might lead later to a violation of some other constraint (not that either parse leads to a constraint violation in this case).

Or is the syntax

array-declarator : direct-declarator [ . assignment-expression ]

Not good either.  The '.' should prefix the identifier, not the expression.  So, for example, you would have:

       void *bsearch(const void key[.size], const void base[.size * .nmemb], size_t nmemb, size_t size,
                     int (*compar)(const void [.size], const void [.size]));

That's taken from the current manual page from git HEAD.  See 'base', which gets its size from the multiplication of 'size' and 'nmemb'.

Quoted 10 lines

(with appropriate variants with static and type-qualifier-list and for array-abstract-declarator as well, and with different identifier interpretation rules inside the assignment-expression)?  If so, then there are big problems parsing [ . ( a ) + ( b ) ], where 'a' is a typedef name in an outer scope, because the appropriate parse would depend on whether 'a' is shadowed by a parameter - unless of course you add appropriate wording like that present in some places about not being able to use this syntax to shadow a typedef name.

Quoted 3 lines

Or is it just

array-declarator : direct-declarator [ . identifier ]

For the initial implementation, it would be, I think.

Quoted 3 lines

which might avoid some of these problems at the expense of being less expressive?

Yes.

Quoted 5 lines

If you're proposing a C syntax addition, you always need to be clear about exactly what the new cases in the syntax would be, and how you resolve ambiguities with any other existing part of the syntax, how you interact with rules on scopes, namespaces and linkage of identifiers, etc.

Yeah, I'll try.

I think that the complete feature would allow 'designator' to be used within unary-expression:

unary-expression: designator

Since sizeof(foo) is a unary-expression and you can't assign to it, I'm guessing that similar rules could be used for '.size'.

That would have the effect of allowing both features suggested by Martin: being able to used designators in both structures (as demonstrated in my last email) and function prototypes (as in the thing we're discussing).

I hope I got it right.  I'm not used to lexical grammar so much.

Cheers,

Alex

Since it's to be used as an rvalue, not as a lvalue, I guess a postfix-expression wouldn't be the right one.

Quoted 3 lines

(with a special rule about how the identifier is interpreted, different from the normal scope rules)? If so, then ".a = 1" could either match assignment-expression directly (assigning to the postfix-expression ".a").

No, assigning to a function parameter from within another parameter declaration wouldn't make sense. They should be readonly. Side effects should be forbidden, I think.

Quoted 2 lines · Alejandro Colomar via Gcc

Since it's to be used as an rvalue, not as a lvalue, I guess a postfix-expression wouldn't be the right one.

Several forms of postfix-expression are only rvalues.

Quoted 4 lines

Quoted 3 lines

(with a special rule about how the identifier is interpreted, different from the normal scope rules)? If so, then ".a = 1" could either match assignment-expression directly (assigning to the postfix-expression ".a").

Quoted 3 lines

No, assigning to a function parameter from within another parameter declaration wouldn't make sense. They should be readonly. Side effects should be forbidden, I think.

Such assignments are already allowed. In a function definition, the side effects (including in size expressions for array parameters adjusted to pointers) take place before entry to the function body.

And, in any case, if you did have a constraint disallowing such assignments, it wouldn't suffice for syntactic disambiguation (see the previous point I made about that; I have some rough notes towards a WG14 paper on syntactic disambiguation, but haven't converted them into a coherent paper).

Quoted 7 lines

Quoted 3 lines

No, assigning to a function parameter from within another parameter declaration wouldn't make sense. They should be readonly. Side effects should be forbidden, I think.

Such assignments are already allowed. In a function definition, the side effects (including in size expressions for array parameters adjusted to pointers) take place before entry to the function body.

Then, I'm guessing that rules need to change in a way that .initializer cannot appear as the left operand of an assignment-expression.

Quoted 8 lines · Alejandro Colomar via Gcc

Quoted 7 lines

Quoted 3 lines

No, assigning to a function parameter from within another parameter declaration wouldn't make sense. They should be readonly. Side effects should be forbidden, I think.

Such assignments are already allowed. In a function definition, the side effects (including in size expressions for array parameters adjusted to pointers) take place before entry to the function body.

Quoted 2 lines

Then, I'm guessing that rules need to change in a way that .initializer cannot appear as the left operand of an assignment-expression.

I think needing such a very special case rule tends to indicate that some alternative syntax, not needing such a rule, would be better.

Quoted 3 lines

Since it's to be used as an rvalue, not as a lvalue, I guess a postfix-expression wouldn't be the right one.

Several forms of postfix-expression are only rvalues.

Quoted 7 lines

Quoted 3 lines

(with a special rule about how the identifier is interpreted, different from the normal scope rules)? If so, then ".a = 1" could either match assignment-expression directly (assigning to the postfix-expression ".a").

No, assigning to a function parameter from within another parameter declaration wouldn't make sense. They should be readonly. Side effects should be forbidden, I think.

Such assignments are already allowed. In a function definition, the side effects (including in size expressions for array parameters adjusted to pointers) take place before entry to the function body.

And, in any case, if you did have a constraint disallowing such assignments, it wouldn't suffice for syntactic disambiguation (see the previous point I made about that; I have some rough notes towards a WG14 paper on syntactic disambiguation, but haven't converted them into a coherent paper).

Am Samstag, den 12.11.2022, 14:54 +0000 schrieb Joseph Myers:

Quoted 19 lines · Alejandro Colomar via Gcc

Quoted 3 lines

Since it's to be used as an rvalue, not as a lvalue, I guess a postfix-expression wouldn't be the right one.

Several forms of postfix-expression are only rvalues.

Quoted 7 lines

Quoted 3 lines

(with a special rule about how the identifier is interpreted, different from the normal scope rules)? If so, then ".a = 1" could either match assignment-expression directly (assigning to the postfix-expression ".a").

No, assigning to a function parameter from within another parameter declaration wouldn't make sense. They should be readonly. Side effects should be forbidden, I think.

Such assignments are already allowed. In a function definition, the side effects (including in size expressions for array parameters adjusted to pointers) take place before entry to the function body.

Quoted 5 lines

And, in any case, if you did have a constraint disallowing such assignments, it wouldn't suffice for syntactic disambiguation (see the previous point I made about that; I have some rough notes towards a WG14 paper on syntactic disambiguation, but haven't converted them into a coherent paper).

My idea was to only allow

array-declarator : direct-declarator [ . identifier ]

and only for parameter (not nested inside structs declared in parameter list) as a first step because it seems this would exclude all difficult cases.

But if we need to allow more complicated expressions, then it starts getting more complicated.

One could could allow more generic expressions, and specify that the .identifier refers to a parameter in
the nearest lexically enclosing parameter list or struct/union.

Then

void foo(struct bar { int x; char c[.x] } a, int x);

would not be allowed (which is good because then we could later use the syntax also inside structs). If we apply scoping rules, the following would work:

struct bar { int y; };
void foo(char p[((struct bar){ .y = .x }).y], int x);

But not:

struct bar { int y; };
void foo(char p[((struct bar){ .y = .y }).y], int y);

But there are not only syntactical problems, because also the type of the parameter might become relevant and then you can get circular dependencies:

void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

I am not sure what would the best way to fix it. One could specifiy that parameters referred to by the .identifer syntax must of some integer type and that the sub-expression .identifer is always converted to a 'size_t'.

Maybe one should also add a constraint that all new type length expressions, i.e. using the syntax, can not have side effects. Or even that they follow all the rules of integer constant expressions with the fictitious assumption that all . identifer sub-expressions are integer constant expressions. The rationale being that this would facilitate compile time reasoning about length expressions.

Martin

Quoted 5 lines

But there are not only syntactical problems, because also the type of the parameter might become relevant and then you can get circular dependencies:

void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

This seems to be a difficult stone in the road.

Quoted 6 lines

I am not sure what would the best way to fix it. One could specifiy that parameters referred to by the .identifer syntax must of some integer type and that the sub-expression .identifer is always converted to a 'size_t'.

That makes sense, but then overnight some quite useful thing came to my mind that would not be possible with this limitation:

<https://software.codidact.com/posts/285946>

char *
stpecpy(char dst[.end - .dst], char *src, char end[1]) {
    for (/* void */; dst <= end; dst++) { *dst = *src++;
        if (*dst == '\0')
            return dst;
    }
    /* Truncation detected */
    *end = '\0';

#if !defined(NDEBUG)
    /* Consume the rest of the input string. */
    while (*src++) {};
#endif

    return end + 1;
}

Hi Martin,

Quoted 22 lines · Alejandro Colomar

Quoted 5 lines

But there are not only syntactical problems, because also the type of the parameter might become relevant and then you can get circular dependencies:

void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

This seems to be a difficult stone in the road.

Quoted 6 lines

I am not sure what would the best way to fix it. One could specifiy that parameters referred to by the .identifer syntax must of some integer type and that the sub-expression .identifer is always converted to a 'size_t'.

That makes sense, but then overnight some quite useful thing came to my mind that would not be possible with this limitation:

<https://software.codidact.com/posts/285946>

char *
stpecpy(char dst[.end - .dst], char *src, char end[1])

Quoted 9 lines

{
     for (/* void */; dst <= end; dst++) {
         *dst = *src++;
         if (*dst == '\0')
             return dst;
     }
     /* Truncation detected */
     *end = '\0';

Quoted 7 lines

#if !defined(NDEBUG)
     /* Consume the rest of the input string. */
     while (*src++) {};
#endif

     return end + 1;
}

And I forgot to say it:  Default promotions rank high (probably the highest) in my list of most hated features^Wbugs in C.  I wouldn't convert it to size_t, but rather follow normal promotion rules.

Since you can use anything between INTMAX_MIN and UINTMAX_MAX for accessing an array (which took me some time to understand), I'd also allow the same here. So, the type of the expression between [] could perfectly be signed or unsigned.

So, you could use size_t for very high indices, or e.g. ptrdiff_t if you want to allow negative numbers.  In the function above, since dst can be a pointer to one-past-the-end (it represents a previous truncation; that's why the test dst<=end), forcing a size_t conversion would disallow that syntax.

Cheers,

Alex

Quoted 10 lines

Hi Martin,

Quoted 7 lines · Alejandro Colomar

Quoted 5 lines

But there are not only syntactical problems, because also the type of the parameter might become relevant and then you can get circular dependencies:

void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

This seems to be a difficult stone in the road.

Quoted 13 lines

Quoted 13 lines

Quoted 6 lines

I am not sure what would the best way to fix it. One could specifiy that parameters referred to by the .identifer syntax must of some integer type and that the sub-expression .identifer is always converted to a 'size_t'.

That makes sense, but then overnight some quite useful thing came to my mind that would not be possible with this limitation:

<https://software.codidact.com/posts/285946>

Quoted 2 lines

Quoted 2 lines

char *
stpecpy(char dst[.end - .dst], char *src, char end[1])

Heh, I got an off-by-one error. It should be dst[.end - .dst + 1], of course, and then the result of the whole expression would be 0, which is fine as size_t.

So, never mind.

Quoted 14 lines

Quoted 14 lines

{
     for (/* void */; dst <= end; dst++) {
         *dst = *src++;
         if (*dst == '\0')
             return dst;
     }
     /* Truncation detected */
     *end = '\0';

#if !defined(NDEBUG)
     /* Consume the rest of the input string. */
     while (*src++) {};

#endif

Quoted 4 lines

Quoted 2 lines

     return end + 1;
}

And I forgot to say it: Default promotions rank high (probably the highest) in my list of most hated features^Wbugs in C.

Quoted 2 lines

I wouldn't convert it to size_t, but
rather follow normal promotion rules.

Quoted 8 lines

Since you can use anything between INTMAX_MIN and UINTMAX_MAX for accessing an array (which took me some time to understand), I'd also allow the same here. So, the type of the expression between [] could perfectly be signed or unsigned.

So, you could use size_t for very high indices, or e.g. ptrdiff_t if you want to allow negative numbers. In the function above, since dst can be a pointer to one-past-the-end (it represents a previous truncation; that's why the test dst<=end), forcing a size_t conversion would disallow that syntax.

Quoted 4 lines

Cheers,

Alex

Signature · 1 line

<http://www.alejandro-colomar.es/>

Am Sonntag, den 13.11.2022, 15:02 +0100 schrieb Alejandro Colomar:

Quoted 8 lines · Alejandro Colomar

Quoted 6 lines · Alejandro Colomar

Quoted 4 lines

Quoted 4 lines

But there are not only syntactical problems, because also the type of the parameter might become relevant and then you can get circular dependencies:

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 1 line

void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

This seems to be a difficult stone in the road.

But note that GNU forward declarations solve this nicely.

Quoted 14 lines

Quoted 14 lines

Quoted 14 lines

Quoted 7 lines

I am not sure what would the best way to fix it. One could specifiy that parameters referred to by the .identifer syntax must of some integer type and that the sub-expression .identifer is always converted to a 'size_t'.

That makes sense, but then overnight some quite useful thing came to my mind that would not be possible with this limitation:

<https://software.codidact.com/posts/285946>

Quoted 7 lines

Quoted 2 lines

Quoted 2 lines

char *
stpecpy(char dst[.end - .dst], char *src, char end[1])

Heh, I got an off-by-one error. It should be dst[.end - .dst + 1], of course, and then the result of the whole expression would be 0, which is fine as size_t.

So, never mind.

.end and .dst would have pointer size though.

Quoted 14 lines

Quoted 14 lines

Quoted 14 lines

{
      for (/* void */; dst <= end; dst++) {
          *dst = *src++;
          if (*dst == '\0')
              return dst;
      }
      /* Truncation detected */
      *end = '\0';

#if !defined(NDEBUG)
      /* Consume the rest of the input string. */
      while (*src++) {};

#endif

Quoted 4 lines

Quoted 4 lines

Quoted 2 lines

      return end + 1;
}

And I forgot to say it: Default promotions rank high (probably the highest) in my list of most hated features^Wbugs in C.

If you replaced them with explicit conversion you then have to add by hand all the time, I am pretty sure most people would hate this more. (and it could also hide bugs)

Quoted 3 lines

Quoted 3 lines

I wouldn't convert it to size_t, but
rather follow normal promotion rules.

The point of making it size_t is that you then
do need to know the type of the parameter to make sense of the expression. If the type matters, then you get mutual dependencies as in the example above.

Hi Martin,

Quoted 7 lines · Martin Uecker

Am Sonntag, den 13.11.2022, 15:02 +0100 schrieb Alejandro Colomar:

Quoted 6 lines · Alejandro Colomar

Quoted 5 lines · Alejandro Colomar

Quoted 4 lines

Quoted 4 lines

But there are not only syntactical problems, because also the type of the parameter might become relevant and then you can get circular dependencies:

Quoted 5 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 1 line

void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

This seems to be a difficult stone in the road.

But note that GNU forward declarations solve this nicely.

How would that above be solved with GNU fwd decl? I'm guessing that it can't. How do you forward declare incomplete VMTs?.

Quoted 14 lines

Quoted 14 lines

Quoted 14 lines

Quoted 14 lines

{
      for (/* void */; dst <= end; dst++) {
          *dst = *src++;
          if (*dst == '\0')
              return dst;
      }
      /* Truncation detected */
      *end = '\0';

#if !defined(NDEBUG)
      /* Consume the rest of the input string. */
      while (*src++) {};

#endif

Quoted 8 lines

Quoted 4 lines

Quoted 4 lines

Quoted 2 lines

      return end + 1;
}

And I forgot to say it: Default promotions rank high (probably the highest) in my list of most hated features^Wbugs in C.

If you replaced them with explicit conversion you then have to add by hand all the time, I am pretty sure most people would hate this more. (and it could also hide bugs)

Yeah, casts are also in my top 3 list of things to avoid (although in this case there's no bug); maybe a bit over default promotions :)

I didn't mean that all promotions are bad. Just the gratuitous ones, like promoting everything to int before even needing it. That makes uint16_t a theoretical type, because whenever you try to use it, you end up with a signed 32-bit type; fun heh? :P _BitInt() solves that for me.

But sure, in (1u + 1l), promotions are fine to get a common type.

Quoted 3 lines

Quoted 2 lines

Quoted 2 lines

I wouldn't convert it to size_t, but
rather follow normal promotion rules.

Quoted 4 lines

The point of making it size_t is that you then
do need to know the type of the parameter to make sense of the expression. If the type matters, then you get mutual dependencies as in the example above.

Except if you treat incomplete types as... incomplete types (for which sizeof() is disallowed by the standard). And the issue we're having is that the types are not yet complete at the time we're using them, aren't they?

Kind of like the initialization order fiasco, but since we're in a limited scope, we can detect it.

Am Sonntag, den 13.11.2022, 16:15 +0100 schrieb Alejandro Colomar:

Quoted 10 lines

Hi Martin,

Quoted 7 lines · Martin Uecker

Am Sonntag, den 13.11.2022, 15:02 +0100 schrieb Alejandro Colomar:

Quoted 6 lines · Alejandro Colomar

Quoted 5 lines · Alejandro Colomar

Quoted 4 lines

Quoted 4 lines

But there are not only syntactical problems, because also the type of the parameter might become relevant and then you can get circular dependencies:

Quoted 8 lines

Quoted 5 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 1 line

void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

This seems to be a difficult stone in the road.

But note that GNU forward declarations solve this nicely.

How would that above be solved with GNU fwd decl? I'm guessing that it can't. How do you forward declare incomplete VMTs?.

You can't express it. This was my point: it is impossible to create circular dependencies.

...

Quoted 26 lines

Quoted 14 lines

Quoted 14 lines

Quoted 14 lines

Quoted 14 lines

{
       for (/* void */; dst <= end; dst++) {
           *dst = *src++;
           if (*dst == '\0')
               return dst;
       }
       /* Truncation detected */
       *end = '\0';

#if !defined(NDEBUG)
       /* Consume the rest of the input string. */
       while (*src++) {};

#endif

Quoted 8 lines

Quoted 4 lines

Quoted 4 lines

Quoted 2 lines

       return end + 1;
}

And I forgot to say it: Default promotions rank high (probably the highest) in my list of most hated features^Wbugs in C.

If you replaced them with explicit conversion you then have to add by hand all the time, I am pretty sure most people would hate this more. (and it could also hide bugs)

Yeah, casts are also in my top 3 list of things to avoid (although in this case there's no bug); maybe a bit over default promotions :)

Quoted 4 lines

I didn't mean that all promotions are bad. Just the gratuitous ones, like promoting everything to int before even needing it. That makes uint16_t a theoretical type, because whenever you try to use it, you end up with a signed 32-bit type; fun heh? :P _BitInt() solves that for me.

uint16_t is for storing data. My expectation is that people will find _BitInt() difficult and error-prone to use for small sizes. But maybe I am wrong...

Quoted 11 lines

But sure, in (1u + 1l), promotions are fine to get a common type.

Quoted 7 lines

Quoted 2 lines

Quoted 2 lines

I wouldn't convert it to size_t, but
rather follow normal promotion rules.

The point of making it size_t is that you then
do need to know the type of the parameter to make sense of the expression. If the type matters, then you get mutual dependencies as in the example above.

Quoted 3 lines

Except if you treat incomplete types as... incomplete types (for which sizeof() is disallowed by the standard). And the issue we're having is that the types are not yet complete at the time we're using them, aren't they?

It is not an incomplete type. When doing parsing and do not have a declaration we know nothing about it (not just not the size). If we assume we know the type (by looking ahead) we get mutual dependencies.

Also the capability to parse, fold, and do type checking in one go is something worth preserving in my opinion.

SYNOPSIS:

unary-operator:  . identifier

DESCRIPTION:

-  It is not an lvalue.

   -  This means sizeof() and _Lengthof() cannot be applied to them.

   -  This prevents ambiguity with a designator in an initializer-list within a
nested braced-initializer.

-  The type of a .identifier is always an incomplete type.

   -  This prevents circular dependencies involving sizeof() or _Lengthof().

-  Shadowing rules apply.

   -  This prevents ambiguity.

EXAMPLES:

-  Valid examples (libc):

       int
       strncmp(const char s1[.n],
               const char s2[.n],
               size_t n);

       int
       cacheflush(void addr[.nbytes],
                  int nbytes,
                  int cache);

       long
       mbind(void addr[.len],
             unsigned long len,
             int mode,
             const unsigned long nodemask[(.maxnode + ULONG_WIDTH ‐ 1) / ULONG_WIDTH], unsigned long maxnode, unsigned int flags);

       void *
       bsearch(const void key[.size],
               const void base[.size * .nmemb],
               size_t nmemb,
               size_t size,
               int (*compar)(const void [.size], const void [.size]));

-  Valid examples (my own):

       void
       ustr2str(char dst[restrict .len + 1],
                const char src[restrict .len],
                size_t len);

       char *
       stpecpy(char dst[.end - .dst + 1],
               char *restrict src,
               char end[1]);

-  Valid examples (from this thread):

   -
       struct s { int a; };
       void f(int a, int b[((struct s) { .a = 1 }).a]);

       Explanation:
       -  Because of shadowing rules, .a=1 refers to the struct member. -  Also, if .a referred to the parameter, it would be an rvalue, so
it wouldn't be valid to assign to it.
       -  (...).a refers to the struct member too, since otherwise an rvalue is
not expected there.

   -
       void foo(struct bar { int x; char c[.x] } a, int x);

       Explanation:
       -  Because of shadowing rules, [.x] refers to the struct member.

   -
       struct bar { int y; };
       void foo(char p[((struct bar){ .y = .x }).y], int x);

       Explanation:
       -  .x unambiguously refers to the parameter.

-  Undefined behavior:

   -
       struct bar { int y; };
       void foo(char p[((struct bar){ .y = .y }).y], int y);

       Explanation:
       -  Because of shadowing rules, =.y refers to the struct member. -  .y=.y means initialize the member with itself (uninitialized use). -  (...).y refers to the struct member, since otherwise an rvalue is not
expected there.

-  Constraint violations:

   -
       void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

       Explanation:
       -  sizeof(*.b): Cannot get size of incomplete type. -  sizeof(*.a): Cannot get size of incomplete type.

   -
       void f(size_t s, int a[sizeof(1) = 1]);

       Explanation:
       -  Cannot assign to rvalue.

   -
       void f(size_t s, int a[.s = 1]);

       Explanation:
       -  Cannot assign to rvalue.

   -
       void f(size_t s, int a[sizeof(.s)]);

       Explanation:
       -  sizeof(.s): Cannot get size of incomplete type.

Does this idea make sense to you?

Cheers,
Alex

Quoted 7 lines · Alejandro Colomar

SYNOPSIS:

unary-operator:  . identifier

DESCRIPTION:

Quoted 3 lines

-  It is not an lvalue.

    -  This means sizeof() and _Lengthof() cannot be applied to them.

Sorry, the above is a thinko.

I wanted to say that, like sizeof() and _Lengthof(), you can't assign to it.

Quoted 3 lines

    -  This prevents ambiguity with a designator in an initializer-list within a nested braced-initializer.

-  The type of a .identifier is always an incomplete type.

Quoted 118 lines

    -  This prevents circular dependencies involving sizeof() or _Lengthof().

-  Shadowing rules apply.

    -  This prevents ambiguity.

EXAMPLES:

-  Valid examples (libc):

        int
        strncmp(const char s1[.n],
                const char s2[.n],
                size_t n);

        int
        cacheflush(void addr[.nbytes],
                   int nbytes,
                   int cache);

        long
        mbind(void addr[.len],
              unsigned long len,
              int mode,
              const unsigned long nodemask[(.maxnode + ULONG_WIDTH ‐ 1) / ULONG_WIDTH], unsigned long maxnode, unsigned int flags);

        void *
        bsearch(const void key[.size],
                const void base[.size * .nmemb],
                size_t nmemb,
                size_t size,
                int (*compar)(const void [.size], const void [.size]));

-  Valid examples (my own):

        void
        ustr2str(char dst[restrict .len + 1],
                 const char src[restrict .len],
                 size_t len);

        char *
        stpecpy(char dst[.end - .dst + 1],
                char *restrict src,
                char end[1]);

-  Valid examples (from this thread):

    -
        struct s { int a; };
        void f(int a, int b[((struct s) { .a = 1 }).a]);

        Explanation:
        -  Because of shadowing rules, .a=1 refers to the struct member.
           -  Also, if .a referred to the parameter, it would be an rvalue, so it wouldn't be valid to assign to it. -  (...).a refers to the struct member too, since otherwise an rvalue is not expected there.

    -
        void foo(struct bar { int x; char c[.x] } a, int x);

        Explanation:
        -  Because of shadowing rules, [.x] refers to the struct member.

    -
        struct bar { int y; };
        void foo(char p[((struct bar){ .y = .x }).y], int x);

        Explanation:
        -  .x unambiguously refers to the parameter.

-  Undefined behavior:

    -
        struct bar { int y; };
        void foo(char p[((struct bar){ .y = .y }).y], int y);

        Explanation:
        -  Because of shadowing rules, =.y refers to the struct member.
        -  .y=.y means initialize the member with itself (uninitialized use).
        -  (...).y refers to the struct member, since otherwise an rvalue is not expected there.

-  Constraint violations:

    -
        void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

        Explanation:
        -  sizeof(*.b): Cannot get size of incomplete type. -  sizeof(*.a): Cannot get size of incomplete type.

    -
        void f(size_t s, int a[sizeof(1) = 1]);

        Explanation:
        -  Cannot assign to rvalue.

    -
        void f(size_t s, int a[.s = 1]);

        Explanation:
        -  Cannot assign to rvalue.

    -
        void f(size_t s, int a[sizeof(.s)]);

        Explanation:
        -  sizeof(.s): Cannot get size of incomplete type.

Does this idea make sense to you?

Cheers,
Alex

Quoted 16 lines · Alejandro Colomar

Quoted 10 lines · Alejandro Colomar

SYNOPSIS:

unary-operator:  . identifier

DESCRIPTION:

-  It is not an lvalue.

    -  This means sizeof() and _Lengthof() cannot be applied to them.

Sorry, the above is a thinko.

Quoted 5 lines

I wanted to say that, like sizeof() and _Lengthof(), you can't assign to it.

Quoted 3 lines

    -  This prevents ambiguity with a designator in an initializer-list within a nested braced-initializer.

-  The type of a .identifier is always an incomplete type.

Or rather, more easily prohibit explicitly using typeof(), sizeof(), and _Lengthof() to it.

Quoted 120 lines

Quoted 119 lines

    -  This prevents circular dependencies involving sizeof() or _Lengthof().

-  Shadowing rules apply.

    -  This prevents ambiguity.

EXAMPLES:

-  Valid examples (libc):

        int
        strncmp(const char s1[.n],
                const char s2[.n],
                size_t n);

        int
        cacheflush(void addr[.nbytes],
                   int nbytes,
                   int cache);

        long
        mbind(void addr[.len],
              unsigned long len,
              int mode,
              const unsigned long nodemask[(.maxnode + ULONG_WIDTH ‐ 1) / ULONG_WIDTH], unsigned long maxnode, unsigned int flags);

        void *
        bsearch(const void key[.size],
                const void base[.size * .nmemb],
                size_t nmemb,
                size_t size,
                int (*compar)(const void [.size], const void [.size]));

-  Valid examples (my own):

        void
        ustr2str(char dst[restrict .len + 1],
                 const char src[restrict .len],
                 size_t len);

        char *
        stpecpy(char dst[.end - .dst + 1],
                char *restrict src,
                char end[1]);

-  Valid examples (from this thread):

    -
        struct s { int a; };
        void f(int a, int b[((struct s) { .a = 1 }).a]);

        Explanation:
        -  Because of shadowing rules, .a=1 refers to the struct member.
           -  Also, if .a referred to the parameter, it would be an rvalue, so it wouldn't be valid to assign to it. -  (...).a refers to the struct member too, since otherwise an rvalue is not expected there.

    -
        void foo(struct bar { int x; char c[.x] } a, int x);

        Explanation:
        -  Because of shadowing rules, [.x] refers to the struct member.

    -
        struct bar { int y; };
        void foo(char p[((struct bar){ .y = .x }).y], int x);

        Explanation:
        -  .x unambiguously refers to the parameter.

-  Undefined behavior:

    -
        struct bar { int y; };
        void foo(char p[((struct bar){ .y = .y }).y], int y);

        Explanation:
        -  Because of shadowing rules, =.y refers to the struct member.
        -  .y=.y means initialize the member with itself (uninitialized use).
        -  (...).y refers to the struct member, since otherwise an rvalue is not expected there.

-  Constraint violations:

    -
        void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

        Explanation:
        -  sizeof(*.b): Cannot get size of incomplete type. -  sizeof(*.a): Cannot get size of incomplete type.

    -
        void f(size_t s, int a[sizeof(1) = 1]);

        Explanation:
        -  Cannot assign to rvalue.

    -
        void f(size_t s, int a[.s = 1]);

        Explanation:
        -  Cannot assign to rvalue.

    -
        void f(size_t s, int a[sizeof(.s)]);

        Explanation:
        -  sizeof(.s): Cannot get size of incomplete type.

Does this idea make sense to you?

Cheers,
Alex

Quoted 17 lines

Quoted 17 lines

Quoted 10 lines

SYNOPSIS:

unary-operator:  . identifier

DESCRIPTION:

-  It is not an lvalue.

    -  This means sizeof() and _Lengthof() cannot be applied to them.

Sorry, the above is a thinko.

I wanted to say that, like sizeof() and _Lengthof(), you can't assign to it.

Quoted 2 lines

    -  This prevents ambiguity with a designator in an initializer-list within a nested braced-initializer.

Quoted 3 lines

Quoted 1 line

Quoted 1 line

-  The type of a .identifier is always an incomplete type.

Or rather, more easily prohibit explicitly using typeof(), sizeof(), and _Lengthof() to it.

Hmm, this is not enough.  Pointer arithmetics are interesting, and for that, you need to implicitly know the sizeof(*.p).

How about allowing only integral types or pointers to integral types?

Quoted 102 lines

Quoted 102 lines

Quoted 102 lines

    -  This prevents circular dependencies involving sizeof() or _Lengthof().

-  Shadowing rules apply.

    -  This prevents ambiguity.

EXAMPLES:

-  Valid examples (libc):

        int
        strncmp(const char s1[.n],
                const char s2[.n],
                size_t n);

        int
        cacheflush(void addr[.nbytes],
                   int nbytes,
                   int cache);

        long
        mbind(void addr[.len],
              unsigned long len,
              int mode,
              const unsigned long nodemask[(.maxnode + ULONG_WIDTH ‐ 1) / ULONG_WIDTH], unsigned long maxnode, unsigned int flags);

        void *
        bsearch(const void key[.size],
                const void base[.size * .nmemb],
                size_t nmemb,
                size_t size,
                int (*compar)(const void [.size], const void [.size]));

-  Valid examples (my own):

        void
        ustr2str(char dst[restrict .len + 1],
                 const char src[restrict .len],
                 size_t len);

        char *
        stpecpy(char dst[.end - .dst + 1],
                char *restrict src,
                char end[1]);

-  Valid examples (from this thread):

    -
        struct s { int a; };
        void f(int a, int b[((struct s) { .a = 1 }).a]);

        Explanation:
        -  Because of shadowing rules, .a=1 refers to the struct member.
           -  Also, if .a referred to the parameter, it would be an rvalue, so it wouldn't be valid to assign to it. -  (...).a refers to the struct member too, since otherwise an rvalue is not expected there.

    -
        void foo(struct bar { int x; char c[.x] } a, int x);

        Explanation:
        -  Because of shadowing rules, [.x] refers to the struct member.

    -
        struct bar { int y; };
        void foo(char p[((struct bar){ .y = .x }).y], int x);

        Explanation:
        -  .x unambiguously refers to the parameter.

-  Undefined behavior:

    -
        struct bar { int y; };
        void foo(char p[((struct bar){ .y = .y }).y], int y);

        Explanation:
        -  Because of shadowing rules, =.y refers to the struct member.
        -  .y=.y means initialize the member with itself (uninitialized use).
        -  (...).y refers to the struct member, since otherwise an rvalue is not expected there.

-  Constraint violations:

    -
        void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

        Explanation:
        -  sizeof(*.b): Cannot get size of incomplete type. -  sizeof(*.a): Cannot get size of incomplete type.

    -
        void f(size_t s, int a[sizeof(1) = 1]);

        Explanation:
        -  Cannot assign to rvalue.

Quoted 8 lines

Quoted 8 lines

Quoted 8 lines

    -
        void f(size_t s, int a[.s = 1]);

        Explanation:
        -  Cannot assign to rvalue.

    -
        void f(size_t s, int a[sizeof(.s)]);

Quoted 12 lines

Quoted 11 lines

Quoted 10 lines

        Explanation:
        -  sizeof(.s): Cannot get size of incomplete type.

Does this idea make sense to you?

Cheers,
Alex

SYNOPSIS:

unary-operator: . identifier

- It is not an lvalue.

- This means sizeof() and _Lengthof() cannot be applied to them.

- This prevents ambiguity with a designator in an initializer-list within
a nested braced-initializer.

- The type of a .identifier is always an incomplete type.

- This prevents circular dependencies involving sizeof() or _Lengthof().

- Shadowing rules apply.

- This prevents ambiguity.

   -
       void foo(struct bar { int x; char c[.x] } a, int x);

       Explanation:
       -  Because of shadowing rules, [.x] refers to the struct member.

Quoted 3 lines · Alejandro Colomar via Gcc

SYNOPSIS:

unary-operator: . identifier

That's not what you mean. See the standard syntax.

unary-expression:
[other alternatives]
unary-operator cast-expression

unary-operator: one of
& * + - ~ !

Quoted 3 lines

- It is not an lvalue.

- This means sizeof() and _Lengthof() cannot be applied to them.

sizeof can be applied to non-lvalues.

Quoted 3 lines

- This prevents ambiguity with a designator in an initializer-list within
a nested braced-initializer.

No, it doesn't. See my previous points about syntactic disambiguation being a separate matter from "one parse would result in a constraint violation, so choose another parse that doesn't" (necessarily, because the constraint violation that results could in general be at an arbitrary distance from the point where a choice of parse has to be made). Or see e.g. the disambiguation rule about enum type specifiers: there is an explicit rule "If an enum type specifier is present, then the longest possible sequence of tokens that can be interpreted as a specifier qualifier list is interpreted as part of the enum type specifier." that ensures that "enum e : long int;" interprets "long int" as the enum type specifier, rather than "long" as the enum type specifier and "int" as another type specifier in the sequence of declaration specifiers, even though the latter parse would result in a constraint violation later.

Also, requiring unbounded lookahead to determine what kind of construct is being parsed may be considered questionable for C. (If you have an initializer starting .a.b.c.d.e, possibly with array element access as well, those could all be designators or .a might be a reference to a parameter of struct or union type and .b.c.d.e a sequence of references to members within it and disambiguation under your rule would depend on whether an '=' follows such an unbounded sequence.)

Quoted 4 lines

- The type of a .identifier is always an incomplete type.

- This prevents circular dependencies involving sizeof() or _Lengthof().

We have typeof as well, which can be applied to expressions with incomplete type.

Quoted 4 lines

- Shadowing rules apply.

- This prevents ambiguity.

"Shadowing rules apply" isn't much of a specification. You need detailed wording that would be added to 6.2.1 Scopes of identifiers (or equivalent elsewhere) to make it clear exactly what scopes apply for identifiers looked up using this construct.

Quoted 6 lines

    -
        void foo(struct bar { int x; char c[.x] } a, int x);

        Explanation:
        -  Because of shadowing rules, [.x] refers to the struct member.

I really don't think standardizing VLAs-in-structures would be a good idea. Certainly it would be a massive pain to specify meaningful semantics for them and this outline doesn't even attempt to work through the consequences of removing the rule that "If an identifier is declared as having a variably modified type, it shall be an ordinary identifier (as defined in 6.2.3), have no linkage, and have either block scope or function prototype scope.".

The idea that .x as an expression might refer to either a member or a parameter is also a massive change to the namespace rules, where at present those are in completely different namespaces and so in any given context a name only needs looking up as one or the other.

Again, proposals should be *minimal*.

And even when they are, many issues
may well arise in practice (see the long list of constexpr issues in my commit message for that C2x feature, for example, which I expect to turn into multiple NB comments and at least two accompanying documents).

Am Sonntag, den 13.11.2022, 15:02 +0100 schrieb Alejandro Colomar:

Quoted 9 lines · Alejandro Colomar

Quoted 7 lines

Hi Martin,

Quoted 4 lines · Alejandro Colomar

Quoted 4 lines

But there are not only syntactical problems, because also the type of the parameter might become relevant and then you can get circular dependencies:

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 1 line

void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

This seems to be a difficult stone in the road.

But note that GNU forward declarations solve this nicely.

Quoted 14 lines

Quoted 14 lines

Quoted 14 lines

Quoted 7 lines

I am not sure what would the best way to fix it. One could specifiy that parameters referred to by the .identifer syntax must of some integer type and that the sub-expression .identifer is always converted to a 'size_t'.

That makes sense, but then overnight some quite useful thing came to my mind that would not be possible with this limitation:

<https://software.codidact.com/posts/285946>

Quoted 7 lines

Quoted 2 lines

Quoted 2 lines

char *
stpecpy(char dst[.end - .dst], char *src, char end[1])

Heh, I got an off-by-one error. It should be dst[.end - .dst + 1], of course, and then the result of the whole expression would be 0, which is fine as size_t.

So, never mind.

.end and .dst would have pointer size though.

Quoted 14 lines

Quoted 14 lines

Quoted 14 lines

{
      for (/* void */; dst <= end; dst++) {
          *dst = *src++;
          if (*dst == '\0')
              return dst;
      }
      /* Truncation detected */
      *end = '\0';

#if !defined(NDEBUG)
      /* Consume the rest of the input string. */
      while (*src++) {};

#endif

Quoted 4 lines

Quoted 4 lines

Quoted 2 lines

      return end + 1;
}

And I forgot to say it: Default promotions rank high (probably the highest) in my list of most hated features^Wbugs in C.

If you replaced them with explicit conversion you then have to add by hand all the time, I am pretty sure most people would hate this more. (and it could also hide bugs)

Quoted 2 lines

Quoted 2 lines

I wouldn't convert it to size_t, but
rather follow normal promotion rules.

The point of making it size_t is that you then
do need to know the type of the parameter to make sense of the expression. If the type matters, then you get mutual dependencies as in the example above.

Quoted 4 lines

Quoted 4 lines

Since you can use anything between INTMAX_MIN and UINTMAX_MAX for accessing an array (which took me some time to understand), I'd also allow the same here. So, the type of the expression between [] could perfectly be signed or unsigned.

Quoted 4 lines

Quoted 4 lines

So, you could use size_t for very high indices, or e.g. ptrdiff_t if you want to allow negative numbers. In the function above, since dst can be a pointer to one-past-the-end (it represents a previous truncation; that's why the test dst<=end), forcing a size_t conversion would disallow that syntax.

Yes, this then does not work.

I guess asking the compiler to do two passes on the param list isn't as bad as asking to do unbound lookahead. In this case it's bound: look ahead till the end of the param list; get as much info as possible, and then do it again to complete. Anything not yet clear after two passes is not valid.

How about the compiler parsing the parameter list twice?

Hey,

Quoted 1 line · Alex Colomar via Gcc

How about the compiler parsing the parameter list twice?

This _is_ unbounded look-ahead.  You could avoid this by using "." for
your new syntax.  Use something unambiguous that can't be confused with
other syntactic elements, e.g. with a different punctuator like '@' or the
like.  But I'm generally doubtful of this whole feature within C itself.
It serves a purpose in documentation, so in man-pages it seems fine enough
(but then still could use a different puncuator to not be confusable with
C syntax).

But within C it still can only serve a documentation purpose as no checking could be performed without also changes in how e.g. arrays are
represented (they always would need to come with a size).

It seems
doubtful to introduce completely new and ambiguous syntax with all the
problems Joseph lists just in order to be able to write documentation when
there's a perfectly fine method to do so: comments.

It does not require any changes on how arrays are represented.

As part of VM-types the size becomes part of the type and this can be used for static or dynamic analysis, e.g. you can
- today - get a run-time bounds violation with the sanitizer:

void foo(int n, char (*buf)[n])
{
(*buf)[n] = 1;
}

For

void foo(int n, char buf[n]);

it semantically has no meaning according to the C standard, but a compiler could still warn.

It could also warn for

void foo(int n, char buf[n]);

int main()

{
    char buf[9];
    foo(buf);
}

But in general: This feature is useful not only for documentation but also for analysis.

Hey,

Quoted 6 lines · Martin

It does not require any changes on how arrays are represented.

As part of VM-types the size becomes part of the type and this can be used for static or dynamic analysis, e.g. you can
- today - get a run-time bounds violation with the sanitizer:

Quoted 4 lines

void foo(int n, char (*buf)[n])
{
  (*buf)[n] = 1;
}

This can already statically analyzed as being wrong, no need for dynamic checking.

What I mean is the checking of the claimed contract. Above you assure for the function body that buf has n elements.

This is also a pre-condition for calling this function and _that_ can't be checked in all cases because:

  void foo (int n, char (*buf)[n]) { (*buf)[n-1] = 1; } void callfoo(char * buf) { foo(10, buf); }

buf doesn't have a known size.

And a pre-condition that can't be checked
is no pre-condition at all, as only then it can become a guarantee for the body.

The compiler has no choice than to trust the user that the pre- condition for calling foo is fulfilled.  I can see how being able to just check half of the contract might be useful, but if it doesn't give full checking then
any proposal for syntax should be even more obviously orthogonal than the current one.

Quoted 6 lines

For

void foo(int n, char buf[n]);

it semantically has no meaning according to the C standard, but a compiler could still warn.

Hmm?  Warn about what in this decl?

Quoted 9 lines

It could also warn for

void foo(int n, char buf[n]);

int main()
{
    char buf[9];
    foo(buf);
}

You mean if you write 'foo(10,buf)' (the above, as is, is simply a syntax error for non-matching number of args).  Or was it a mispaste and you mean the one from the godbolt link, i.e.:

void foo(char buf[10]){ buf[9] = 1; }
int main()
{
    char buf[9];
    foo(buf);
}

?  If so, yeah, we warn already.  I don't think this is an argument for (or against) introducing new syntax.
...

Quoted 2 lines

But in general: This feature is useful not only for documentation but also for analysis.

Which feature we're talking about now?  The ones you used all work today,
as you demonstrated.  I thought we would be talking about that ".whatever"
syntax to refer to arbitrary parameters, even following ones?  I think a
disrupting syntax change like that should have a higher bar than "in some
cases, depending on circumstance, we might even be able to warn".

Hi,

Am Dienstag, dem 29.11.2022 um 15:44 +0000 schrieb Michael Matz:

Quoted 10 lines

Hey,

Quoted 6 lines · Martin

It does not require any changes on how arrays are represented.

As part of VM-types the size becomes part of the type and this can be used for static or dynamic analysis, e.g. you can
- today - get a run-time bounds violation with the sanitizer:

Quoted 7 lines

Quoted 4 lines

void foo(int n, char (*buf)[n])
{
  (*buf)[n] = 1;
}

This can already statically analyzed as being wrong, no need for dynamic checking.

In this toy example, but in general in can be checked only at run-time by using the information about the dynamic bound.

Quoted 2 lines

What I mean is the checking of the claimed contract. Above you assure for the function body that buf has n elements.

Yes.

Quoted 7 lines

This is also a pre-condition for calling this function and _that_ can't be checked in all cases because:

  void foo (int n, char (*buf)[n]) { (*buf)[n-1] = 1; } void callfoo(char * buf) { foo(10, buf); }

buf doesn't have a known size.

This does not type check.

Quoted 3 lines

And a pre-condition that can't be checked
is no pre-condition at all, as only then it can become a guarantee for the body.

The example above should look like:

void foo(int n, char (*buf)[n]);

void callfoo(char (*buf)[12]) { foo(10, buf); }

This could be checked by an UB sanitizer as calling the function with an argument of incompatible type is UB (but we currently do not do this)

If you think about

void foo(int n, char buf[n]);

void callfoo(char *buf) { foo(10, buf); }

Then you are right that this can not be checked at this time. But this does not mean it is useless because we still can detect inconsistencies in other cases:

void callfoo(int n, char buf[n - 1]) { foo(n, buf); }

We could also - in the future - have a warning about all situations where bound information is lost, making sure that preconditions are always checked for people who consistently use these annotations.

Quoted 6 lines

The compiler has no choice than to trust the user that the pre- condition for calling foo is fulfilled.  I can see how being able to just check half of the contract might be useful, but if it doesn't give full checking then
any proposal for syntax should be even more obviously orthogonal than the current one.

Your argument is not clear to me.

Quoted 8 lines

Quoted 6 lines

For

void foo(int n, char buf[n]);

it semantically has no meaning according to the C standard, but a compiler could still warn.

Hmm?  Warn about what in this decl?

I meant, we could warn about something like this because it is likely an error:

void foo(int n, char buf[n])

{
   buf[n] = 1;
}

Quoted 10 lines

Quoted 9 lines

It could also warn for

void foo(int n, char buf[n]);

int main()
{
    char buf[9];
    foo(buf);
}

Quoted 3 lines

You mean if you write 'foo(10,buf)' (the above, as is, is simply a syntax error for non-matching number of args).  Or was it a mispaste and you mean the one from the godbolt link, i.e.:

I meant:

char buf[9];
foo(10, buf);

In fact, it turns out we warn already:

https://godbolt.org/z/qcvsv87Ev

Quoted 10 lines

void foo(char buf[10]){ buf[9] = 1; }
int main()
{
    char buf[9];
    foo(buf);
}

?  If so, yeah, we warn already.  I don't think this is an argument for (or against) introducing new syntax.
...

It is argument for having this syntax, because we could extend such warning (those we already have and those we could still add) to more common cases such as

void foo(char buf[.n], size_t n);

In my opinion, this would a huge step forward for safety of C programs as we already have a lot of infrastructure for checking bounds.

Of course, the existing GNU extension would achieve the same thing:

void foo(size_t n; char buf[n], size_t n);

Quoted 12 lines

Quoted 2 lines

But in general: This feature is useful not only for documentation but also for analysis.

Which feature we're talking about now?  The ones you used all work today,
as you demonstrated.  I thought we would be talking about that ".whatever"
syntax to refer to arbitrary parameters, even following ones?  I think a
disrupting syntax change like that should have a higher bar than "in some
cases, depending on circumstance, we might even be able to warn".

We can use our existing features and then apply them to cases where the bound is specified after the pointer, which is more common in practice.

Martin

like. But I'm generally doubtful of this whole feature within C itself. It serves a purpose in documentation, so in man-pages it seems fine enough (but then still could use a different puncuator to not be confusable with C syntax).

Quoted 4 lines · Michael Matz via Gcc

like. But I'm generally doubtful of this whole feature within C itself. It serves a purpose in documentation, so in man-pages it seems fine enough (but then still could use a different puncuator to not be confusable with C syntax).

In man-pages you don't need to invent syntax at all. You can write

int f(char buf[n], int n);

and in the context of a man page it will be clear to readers what is meant,

Quoted 13 lines · Michael Matz via Gcc

Quoted 8 lines

like.  But I'm generally doubtful of this whole feature within C itself.
It serves a purpose in documentation, so in man-pages it seems fine enough
(but then still could use a different puncuator to not be confusable with
C syntax).

In man-pages you don't need to invent syntax at all.  You can write

Quoted 5 lines

int f(char buf[n], int n);

and in the context of a man page it will be clear to readers what is
meant,

Considerably more clear than new invented syntax IMHO.

Am Dienstag, dem 29.11.2022 um 16:53 +0000 schrieb Jonathan Wakely:

Quoted 20 lines · Joseph Myers

Quoted 18 lines · Michael Matz via Gcc

Quoted 8 lines

like.  But I'm generally doubtful of this whole feature within C itself.
It serves a purpose in documentation, so in man-pages it seems fine enough
(but then still could use a different puncuator to not be confusable with
C syntax).

In man-pages you don't need to invent syntax at all.  You can write

int f(char buf[n], int n);

and in the context of a man page it will be clear to readers what is
meant,

Quoted 1 line

Considerably more clear than new invented syntax IMHO.

True, but I think it would be a mistake to use code in man pages which then does not work as expected (or even is subtle wrong) in actual code.

Martin

Hi Martin, Joseph,

Quoted 21 lines · Martin Uecker

Am Dienstag, dem 29.11.2022 um 16:53 +0000 schrieb Jonathan Wakely:

Quoted 20 lines · Joseph Myers

Quoted 18 lines · Michael Matz via Gcc

Quoted 8 lines

like.  But I'm generally doubtful of this whole feature within C itself.
It serves a purpose in documentation, so in man-pages it seems fine enough
(but then still could use a different puncuator to not be confusable with
C syntax).

In man-pages you don't need to invent syntax at all.  You can write

int f(char buf[n], int n);

and in the context of a man page it will be clear to readers what is
meant,

Quoted 5 lines

Quoted 1 line

Considerably more clear than new invented syntax IMHO.

True, but I think it would be a mistake to use code in man pages which then does not work as expected (or even is subtle wrong) in actual code.

Exactly.  Using your

proposed syntax (which was my first draft) would have probably been the source of hidden bugs, since it might work (read compile) in some cases, but with wrong results.

I prefer this hypothetical syntax, which at most will cause compile errors.

Cheers,

Alex

Quoted 5 lines

Martin

Hi!

I'll probably have to release again before the Debian freeze of Bookworm. That's something I didn't want to do, but there's some important bug that affects downstream projects (translation pages), and I need to release. It's a bit weird that the bug has been reported now, because it has always been there (it's not a regression), but still, I want to address it before the next Debian.

And I don't want to start with stable releases, so I won't be releasing man-pages-6.01.1. That means that all changes that I have in the project that I didn't plan to release until 2024 will be released in a few weeks, notably including the VLA syntax.

This means that while this syntax is still an invent, not something real that can be used, I need to be careful about the future if I plan to make it public so soon.

Since we've seen that using a '.' prefix seems to be problematic because of lookahead, and recently Michael Matz proposed using a different punctuator (he proposed '@') for differentiating parameters from struct members, I think going in that direction may be a good idea.

How about '$'?

It's been used for function parameters since... forever? in sh(1). And it's being added to the source character set in C23, so it seems to be a good choice.
It should also be intuitive what it means.

What do you think about it? I'm not asking for your opinion about adding it to GCC, but rather for replacing the current '.' in the man-pages before I release later this month. Do you think I should apply that change?

Cheers,

Alex

--
<http://www.alejandro-colomar.es/>

Since we've seen that using a '.' prefix seems to be problematic because of lookahead, and recently Michael Matz proposed using a different punctuator (he proposed '@') for differentiating parameters from struct members, I think going  in that direction may be a good idea.

How about '$'?

Am Samstag, dem 03.12.2022 um 22:03 +0100 schrieb Alejandro Colomar:
...

Quoted 6 lines

Since we've seen that using a '.' prefix seems to be problematic because of lookahead, and recently Michael Matz proposed using a different punctuator (he proposed '@') for differentiating parameters from struct members, I think going  in that direction may be a good idea.

Quoted 1 line

How about '$'?

I don't see how the lookahead issue has anything to do with the choice of the symbol.

Here, also with the context would fully disambiguate
between other uses so I do not think there is any issue with using this syntax.  '$' is much more problematic as people use it in identifiers, '@' may cause confusion with objective C.

$ is a GNU extension for identifiers already.
Seehttps://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/Dollar-Signs.html#Dollar-Signs

Thanks,
Andrew

Martin

What do you think about it? I'm not asking for your opinion about adding it to GCC, but rather for replacing the current '.' in the man-pages before I release later this month. Do you think I should apply that change?

Maybe allowing integral types and pointers would be enough. However, foreseeing that the _Lengthof() proposal (BTW, which paper was it?) will succeed, and combining it with this one, _Lengthof(pointer) would ideally give the length of the array, so allowing pointers would conflict.

Quoted 4 lines · Alejandro Colomar via Gcc

Maybe allowing integral types and pointers would be enough. However, foreseeing that the _Lengthof() proposal (BTW, which paper was it?) will succeed, and combining it with this one, _Lengthof(pointer) would ideally give the length of the array, so allowing pointers would conflict.

Do you mean N2529 Romero, New pointer-proof keyword to determine array length?

To quote the convenor in WG14 reflector message 18575 (17 Nov 2020) when I asked about its status, "The author asked me not to put those on the agenda. He will supply updated versions later.".

Quoted 3 lines

To quote the convenor in WG14 reflector message 18575 (17 Nov 2020) when I asked about its status, "The author asked me not to put those on the agenda. He will supply updated versions later.".

Since his email is not in the paper, would you mind forwarding him this suggestion of mine of renaming it to avoid confusion with string lengths? Or maybe point him to the mailing list discussion[1]?

[1]:
<https://lore.kernel.org/linux-man/20221110222540.as3jrjdzxsnot3zm@illithid/T/#m794ad2a3173a19099625ee1dec7ea11ab754513d>

Quoted 8 lines · Alejandro Colomar via Gcc

Quoted 3 lines

To quote the convenor in WG14 reflector message 18575 (17 Nov 2020) when I asked about its status, "The author asked me not to put those on the agenda. He will supply updated versions later.".

Since his email is not in the paper, would you mind forwarding him this suggestion of mine of renaming it to avoid confusion with string lengths? Or maybe point him to the mailing list discussion[1]?

Quoted 2 lines

[1]:
<https://lore.kernel.org/linux-man/20221110222540.as3jrjdzxsnot3zm@illithid/T/#m794ad2a3173a19099625ee1dec7ea11ab754513d>

I don't have his email address (I don't see any emails from him on the reflector since I joined it in 2001).

Now, I've released man-pages-6.01 very recently (just a few weeks ago), and I don't plan to release again in a year or two, so there's time to do the implementation in GCC. From my side, please consider this an ACK or even somewhat of a push to get things done in the compiler side of things :)

Hi Alex,

Quoted 5 lines · Alejandro Colomar

Now, I've released man-pages-6.01 very recently (just a few weeks ago), and I don't plan to release again in a year or two, so there's time to do the implementation in GCC. From my side, please consider this an ACK or even somewhat of a push to get things done in the compiler side of things :)

Do you mean you _don't_ plan to release again for a year or two?

You know what Moltke said about plans and contact with the enemy. For one thing, I think the Linux kernel will move too fast to permit such a leisurely cadence.

Also, as soon as Bertrand and I can get groff 1.23 out[1], I am hoping you will, shortly thereafter, migrate to the new MR macro.

<tents fingers, laughs villainously>

Regards,
Branden

[1] Only 6 RC bugs left!

     https://savannah.gnu.org/bugs/index.php?go_report=Apply&group=groff&set=custom&report_id=225&status_id=1&plan_release_id=103

Quoted 2 lines

Also, as soon as Bertrand and I can get groff 1.23 out[1], I am hoping you will, shortly thereafter, migrate to the new MR macro.

Not as soon as it gets released, because I expect (at least a decent amount of) contributors to be able to read the pages to which they contribute to, but as soon as it makes it into Debian stable, yes, that's in my plans.  So, if you make it before the freeze, that means around a couple of months from now.

Quoted 2 lines

Also, as soon as Bertrand and I can get groff 1.23 out[1], I am hoping you will, shortly thereafter, migrate to the new MR macro.

Not as soon as it gets released, because I expect (at least a decent amount of) contributors to be able to read the pages to which they contribute to, but as soon as it makes it into Debian stable, yes, that's in my plans. So, if you make it before the freeze, that means around a couple of months from now.

Hi Branden,

Another interesting thing is what to do here:

$ sed -n 319,320p man2/timerfd_create.2
.TP
.BR poll "(2), " select "(2) (and similar)"

Can I have multiple input lines as the tag for a TP?  How to put 2 MR references in there?

Cheers,

Alex

Of course I forgot to rename the title, and to agg groff@.  Nice.

-------- Forwarded Message --------
Subject: Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters
Date: Thu, 10 Nov 2022 18:47:38 +0100
From: Alejandro Colomar <[email protected]>
To: G. Branden Robinson <[email protected]>
CC: Ingo Schwarze <[email protected]>, [email protected]

[removed gcc@ and other uninterested people; added groff@]

Hi Branden!

Quoted 7 lines · Alejandro Colomar

Quoted 2 lines

Also, as soon as Bertrand and I can get groff 1.23 out[1], I am hoping you will, shortly thereafter, migrate to the new MR macro.

Not as soon as it gets released, because I expect (at least a decent amount of) contributors to be able to read the pages to which they contribute to, but as soon as it makes it into Debian stable, yes, that's in my plans.  So, if you make it before the freeze, that means around a couple of months from now.

I won't be applying the patch now, to avoid contributors seeing people suddenly not seeing man page references while preparing patches.  But I'll start preparing the patch, to see where are the most difficult parts.  And maybe report some issues with the usability.

My first thing was to run:

$ grep -rn '^\.BR .* ([1-9]\w*)'

I'm surprised for good that it seems that there are no false positives.  I didn't expect that.  But since things like exit(1) are code, they are probably either not highlighted at all, or maybe are italicized (as code is).  So that's a good thing.

It showed a few lines that might be problematic, but that's actually bad code, which I need to fix:

man7/credentials.7:270:.BR setuid "(2) (" setgid (2)) man7/credentials.7:274:.BR seteuid "(2) (" setegid (2)) man7/credentials.7:277:.BR setfsuid "(2) (" setfsgid (2)) man7/credentials.7:280:.BR setreuid "(2) (" setregid (2)) man7/credentials.7:284:.BR setresuid "(2) (" setresgid (2))

Those are asking for a 2-line thing, where the second line is RB instead of BR. Which reminds me to check RB:

$ grep -rn '^\.RB .* ([1-9]\w*)'

There are much less cases, and also seem to be fine to script, with a few minor ffixes too.

The big issue is that your MR doesn't support leading text:

        .MR page‐title manual‐section [trailing‐text]

I remember we had this discussion about what to do with it.  A 4th argument? There's also conflict with a hypothetical link that we might want to add later.

My opinion is that the 4th argument should be the leading text.  Asking to use the escape (was it \c?) sequence to workaround that limitation is not very nice.
  Especially for scripting the change.

If you want a 5th argument for a URI, you can specify the leading text as "", which is not much of an issue.  And you keep the trailing text and the leading one together.

What are your thoughts?  What should we do?

Hi Branden,

Quoted 48 lines · Alejandro Colomar

Of course I forgot to rename the title, and to agg groff@.  Nice.

-------- Forwarded Message --------
Subject: Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters
Date: Thu, 10 Nov 2022 18:47:38 +0100
From: Alejandro Colomar <[email protected]>
To: G. Branden Robinson <[email protected]>
CC: Ingo Schwarze <[email protected]>, [email protected]

[removed gcc@ and other uninterested people; added groff@]

Hi Branden!

On 11/10/22 11:59, Alejandro Colomar wrote:
 >> Also, as soon as Bertrand and I can get groff 1.23 out[1], I am hoping >> you will, shortly thereafter, migrate to the new MR macro.
 >
 > Not as soon as it gets released, because I expect (at least a decent amount of) > contributors to be able to read the pages to which they contribute to, but as > soon as it makes it into Debian stable, yes, that's in my plans.  So, if you > make it before the freeze, that means around a couple of months from now.

I won't be applying the patch now, to avoid contributors seeing people suddenly not seeing man page references while preparing patches.  But I'll start preparing the patch, to see where are the most difficult parts.  And maybe report some issues with the usability.

My first thing was to run:

$ grep -rn '^\.BR .* ([1-9]\w*)'

I'm surprised for good that it seems that there are no false positives.  I didn't expect that.  But since things like exit(1) are code, they are probably either not highlighted at all, or maybe are italicized (as code is).  So that's a good thing.

It showed a few lines that might be problematic, but that's actually bad code, which I need to fix:

man7/credentials.7:270:.BR setuid "(2) (" setgid (2)) man7/credentials.7:274:.BR seteuid "(2) (" setegid (2)) man7/credentials.7:277:.BR setfsuid "(2) (" setfsgid (2)) man7/credentials.7:280:.BR setreuid "(2) (" setregid (2)) man7/credentials.7:284:.BR setresuid "(2) (" setresgid (2))

Those are asking for a 2-line thing, where the second line is RB instead of BR. Which reminds me to check RB:

$ grep -rn '^\.RB .* ([1-9]\w*)'

There are much less cases, and also seem to be fine to script, with a few minor ffixes too.

The big issue is that your MR doesn't support leading text:

         .MR page‐title manual‐section [trailing‐text]

Quoted 7 lines

I remember we had this discussion about what to do with it.  A 4th argument? There's also conflict with a hypothetical link that we might want to add later.

My opinion is that the 4th argument should be the leading text.  Asking to use the escape (was it \c?) sequence to workaround that limitation is not very nice.    Especially for scripting the change.

If you want a 5th argument for a URI, you can specify the leading text as "", which is not much of an issue.  And you keep the trailing text and the leading one together.

What are your thoughts?  What should we do?

To document and discuss the way I'm migrating, I'll share here the scripts:

The simplest case: a single man page reference with no other stuff around it:

$ find man* -type f \
   | xargs sed -i 's/^\.BR \([^ ]*\) (\([1-9]\w*\))$/.MR \1 \2/'

Second simplest case: a single man page reference with only trailing stuff:

$ find man* -type f \
   | xargs sed -i 's/^\.BR \([^ ]*\) (\([1-9]\w*\))/.MR \1 \2 /'

And here I continue with hypothetical syntax not yet allowed by groff.

A single man page reference with only leading stuff:

$ find man* -type f \
   | xargs sed -i 's/^\.RB \([^ ]*\) \([^ ]*\) (\([1-9]\w*\))$/.MR \2 \3 "" \1/'

A single man page reference with both leading and trailing stuff (thank $DEITY for not having comments in any of those, so I can just run the script):

$ find man* -type f \
   | xargs sed -i 's/^\.RB \([^ ]*\) \([^ ]*\) (\([1-9]\w*\))\(.*\)/.MR \2 \3 \4 
\1/'

Of course I forgot to rename the title, and to agg groff@. Nice.

I won't be applying the patch now, to avoid contributors seeing people suddenly not seeing man page references while preparing patches. But I'll start preparing the patch, to see where are the most difficult parts. And maybe report some issues with the usability.

My first thing was to run:

$ grep -rn '^\.BR .* ([1-9]\w*)'

I'm surprised for good that it seems that there are no false positives. I didn't expect that. But since things like exit(1) are code, they are probably either not highlighted at all, or maybe are italicized (as code is). So that's a good thing.

It showed a few lines that might be problematic, but that's actually bad code, which I need to fix:

man7/credentials.7:270:.BR setuid "(2) (" setgid (2)) man7/credentials.7:274:.BR seteuid "(2) (" setegid (2)) man7/credentials.7:277:.BR setfsuid "(2) (" setfsgid (2)) man7/credentials.7:280:.BR setreuid "(2) (" setregid (2)) man7/credentials.7:284:.BR setresuid "(2) (" setresgid (2))

Those are asking for a 2-line thing, where the second line is RB instead of BR. Which reminds me to check RB:

$ grep -rn '^\.RB .* ([1-9]\w*)'

There are much less cases, and also seem to be fine to script, with a few minor ffixes too.

The big issue is that your MR doesn't support leading text:

.MR page‐title manual‐section [trailing‐text]

I remember we had this discussion about what to do with it. A 4th argument? There's also conflict with a hypothetical link that we might want to add later.

My opinion is that the 4th argument should be the leading text. Asking to use the escape (was it \c?) sequence to workaround that
limitation is not very nice. Especially for scripting the change.

If you want a 5th argument for a URI, you can specify the leading text as "", which is not much of an issue. And you keep the trailing text and the leading one together.

What are your thoughts? What should we do?

Hi Alex,

Quoted 1 line · Alejandro Colomar

Of course I forgot to rename the title, and to agg groff@. Nice.

It gave me time to reply to this one. :)

Quoted 11 lines

The big issue is that your MR doesn't support leading text:

.MR page‐title manual‐section [trailing‐text]

I remember we had this discussion about what to do with it. A 4th argument? There's also conflict with a hypothetical link that we might want to add later.

My opinion is that the 4th argument should be the leading text. Asking to use the escape (was it \c?) sequence to workaround that
limitation is not very nice. Especially for scripting the change.

Here's what I did for groff.

commit 2ab0dacb95863a2e347d06cf970676c74c784ce2

Author: G. Branden Robinson <[email protected]>
Date:   Fri Oct 8 00:46:41 2021 +1100

[man pages]: Migrate man(7) cross refs to MR.

# Handle simplest case: ".IR foo (1)".
s/^.[BI]R \(\\%\)*\([@_[:alnum:]\\-]\+\) (\(@MAN[157]EXT@\))$/.MR \2 \3/ s/^.[BI]R \(\\%\)*\([@_[:alnum:]\\-]\+\) (\([1-8a-z]\+\))$/.MR \2 \3/
# Handle case: trailing puncutation, e.g., ".IR foo (1),".
s/^.[BI]R \(\\%\)*\([@_[:alnum:]\\-]\+\) (\(@MAN[157]EXT@\))\([^[:space:]]\+\)/.MR \2 \3 \4/ s/^.[BI]R \(\\%\)*\([@_[:alnum:]\\-]\+\) (\([1-8a-z]\+\))\([^[:space:]]\+\)/.MR \2 \3 \4/
# Handle case: 3rd+ arguments or trailing comments. This case is rare
# and will require manual fixup if there are 4+ arguments to MR. Use
# groff -man -rCHECKSTYLE=1 to have them automatically reported.
s/^.[BI]R \(\\%\)*\([@_[:alnum:]\\-]\+\) (\(@MAN[157]EXT@\))\( .*\)/.MR \2 \3\4/ s/^.[BI]R \(\\%\)*\([@_[:alnum:]\\-]\+\) (\([1-8a-z]\+\))\( .*\)/.MR \2 \3\4/

You can ignore the 'MAN[157]EXT' lines; they are relevant only to within-groff pages (because all of our man pages undergo sed-processing to be prepared for installation).

Quoted 6 lines

If you want a 5th argument for a URI, you can specify the leading text as "", which is not much of an issue. And you keep the trailing text and the leading one together.

What are your thoughts? What should we do?

I am reluctant to extend the interface of MR at this point because as it is it has two nice properties: it aligns with mdoc(7)'s Xr macro and with Plan 9 from User Space troff's MR, which did it first.

(Admittedly, P9US troff's MR macro doesn't supply the parentheses. I don't know if they intend to change that. I'm willing to supply a patch to change their implementation and their man pages to align with what I did in groff. As shown above, I believe my sed-fu is in order.)

I think man page authors should learn when the \c escape sequence is appropriate and use it when warranted, and recast their sentences otherwise. That is why I provided an explicit example in the groff_man_style(7) page.

.MR page-title manual-section [trailing-text]
(since groff 1.23) Set a man page cross reference as "page- title(manual-section)". If trailing-text (typically punctuation) is specified, it follows the closing parenthesis without intervening space. Hyphenation is disabled while the cross reference is set. page-title is set in the font specified by the MF string. The cross reference hyperlinks to a URI of the form "man:page-title(manual-section)".

The output driver
.MR grops 1
produces PostScript from
.I troff
output.
.
The Ghostscript program (\c
.MR gs 1 )
interprets PostScript and PDF.

\c solves problems that are complicated to solve any other way. As far as I have seen, you don't ever need it in mdoc(7) pages, for example...but you pay a price. You must learn which of mdoc's several dozen macros are "parsed" versus "callable" (and what the heck the package even _means_ by those words); you must learn that Pf and Ns exist and when to use them; you must learn that certain two-letter words will not behave as you expect; and if you thought using mdoc(7) meant you wouldn't have to type any groff escape sequences, think again--you'll be putting \& all over the place.

People can use mdoc(7) if they want to (and now that I'm learning it better, I will consult as I am able), but its reputation in some circles as a superior solution to man(7) on all fronts that should have kicked its predecessor into the grave long ago is due solely to irresponsible hype from its exponents.

If you need help automating a change to adapt some Linux man-pages documents to use \c before an MR call on the next line (where you were using RB before, for instance), just let know. I am nearly certain that a sed script utilizing its hold space feature can get the job done. (I've used the hold space profitably before, but occasions for it come up seldom enough that I have to review my past solutions before the knowledge comes back. Or maybe it's creeping senescence.)

Regards,
Branden

Hmm, will need to parse that. Anyway, I think now that I have the MR with 4 arguments, moving the 4th to the previous line with sed and N should not be that difficult.

Well, being a compatible extension to the others is not that bad. How does mdoc(7) solve it with Xr?

One of the biggest issues with this is that it breaks what would otherwise represent a single entity, into two lines, so it hurts readability. See as an extreme example the following change I did with my scripts (from posix_spawn(3), if you're curious):

(quoted diff) +1 -1

@@ -129,7 +129,7 @@ .SH DESCRIPTION
 Below, the functions are described in terms of a three-step process: the
 .MR fork 3
 step, the
-.RB pre- exec ()
+.MR exec 3 "" pre-
 step (executed in the child),
 and the
 .MR exec 3

Having 'pre-' as the last part of some random line, separates it from the other part of the word. The \c alternative would be:

step, the pre-
.MR exec 3
step ...

Not terrible, but I'm not in love with it.

I hope I can come up with something, but yes, if not, I'll call you ;)

BTW, so far I've never found a case where I had to use the hold space. I wonder if I may meet a case where I need it in my life. This week I came up with some script for inserting an element into a JSON array at a specified position, but N is all that was needed:
<http://www.alejandro-colomar.es/src/alx/nginx/unitcli.git/tree/bin/setup-unit#n969>.

I've met a few more-complex cases, but not really that much. I always come up with some combination of filters that allows me to avoid the hold space. Sometimes, two scripts run consecutively also helps keep it simple :)

Quoted 3 lines

You know what Moltke said about plans and contact with the enemy. For one thing, I think the Linux kernel will move too fast to permit such a leisurely cadence.

Heh, at this point, I burnt my ships, by using enhanced VLA syntax. If I release that before GCC, I'm expecting to see an avalanche of reports about it (and I also expect that GCC and forums will receive a similar ammount). So yes, I expect to wait some longish time.

Quoted 2 lines

Also, as soon as Bertrand and I can get groff 1.23 out[1], I am hoping you will, shortly thereafter, migrate to the new MR macro.

Not as soon as it gets released, because I expect (at least a decent amount of) contributors to be able to read the pages to which they contribute to,

but as soon as it makes it into Debian stable, yes, that's in my plans. So, if you make it before the freeze, that means around a couple of months from now.

Quoted 1 line

<tents fingers, laughs villainously>

<also tents fingers, laughs villainously>