Rendered at 11:23:24 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
teo_zero 1 days ago [-]
There are a couple of constructs that you end up re-creating as soon as you need non-trivial preprocessing.
One of those is the two-stage CAT, another one is the NARG the article talks about.
__COUNTER__ is going to be added to C29, the next version of the standard, maybe there's hope for __VA_COUNT__ and __CAT__ too!
0c3ca83 1 days ago [-]
Doing non-trivial preprocessing is a mistake. You get slow compilation, hard to debug code, opaque and fragile code, and incredible complexity. If you want metaprogramming, you're better off with C++ it you need C header compatibility, or zig if you don't.
Otherwise, it's probably a good idea to pretend that the only preprocessing directive is #include. With constexpr, even #define is obsolescent.
teo_zero 22 hours ago [-]
Doing non-trivial anything you slow down compilation, make debugging harder and increase fragility and complexity. Sometimes that's a mistake, sometimes not.
Getting rid of something because someone sometimes abuses it, is short-sighted.
0c3ca83 21 hours ago [-]
You don't need to get rid of it to leave it unused. Macro wizardry leads to poor code.
jkercher 20 hours ago [-]
Yes and no. If you want to make something like a generic dynamic array, you can do it with type erasure (`void*`), but then your code is no longer typesafe. I would find that more fragile. The preprocessor allows you to add constructs that make it behave like any other modern language.
As an aside, clangd can dereference a macro in place to make debugging easy.
uecker 19 hours ago [-]
And for generic data structures one can also a very thin layer of macros around type-erased implementations.
0c3ca83 19 hours ago [-]
You're usually better off just using realloc, or an abort-on-oom wraper of it, in my experience.
mrpippy 20 hours ago [-]
> slow compilation, hard to debug code, opaque and fragile code, and incredible complexity
Most people describe C++ in those terms.
0c3ca83 20 hours ago [-]
Exactly. So if you're willing to accept that downside, C++ does a better job of letting you express that kind of cleverness.
uecker 19 hours ago [-]
I am not sure. I agree one should minimize macros in C, but for some things a small macro layer still beats the complexity of C++ and is not inferior in my opinion.
0c3ca83 18 hours ago [-]
C++ templates are far cleaner, easier to debug, and produce more comprehensible errors than C macros. And C++ templates are rightly famous for being terrible at all of the above.
uecker 17 hours ago [-]
Hey, I agree about your second sentence but I do not really see the first. Alone that I can directly inspect the result of the preprocessor makes macros easier to debug than templates in my experience.
Joker_vD 15 hours ago [-]
> I can directly inspect the result of the preprocessor makes macros easier to debug
Tangential note but: why does this require a separate re-invocation of the compiler with the '-E' flag? Say I have this silly example with a comma instead of the semicolon inside W() macro:
#define W(a, b) b, int a
#define X(a, b) W(b, a)
int kekw(int x) {
int z = X(23, k) = x;
return z + k;
}
When I shove it into GCC without -E, it says
<source>: In function 'int kekw(int)':
<source>:1:21: error: expected unqualified-id before 'int'
1 | #define W(a, b) b, int a
| ^~~
<source>:2:18: note: in expansion of macro 'W'
2 | #define X(a, b) W(b, a)
| ^
<source>:5:13: note: in expansion of macro 'X'
5 | int z = X(23, k) = x;
| ^
<source>:6:16: error: 'k' was not declared in this scope
6 | return z + k;
|
instead of, you know, showing me the expanded text so that I could see what's actually wrong with the generated variable declarations? Something like
<source>: In function 'int kekw(int)':
<source>:1:21: error: expected unqualified-id before 'int'
1 | #define W(a, b) b, int a
| ^~~
<source>:2:18: note: in expansion of macro 'W'
2 | #define X(a, b) W(b, a)
| ^
<source>:5:13: note: in expansion of macro 'X'
5 | int z = X(23, k) = x;
| ^
<source>:5:-: note: the expanded source text is
| int z = 23, int k = x;
| ^~~
<source>:6:16: error: 'k' was not declared in this scope
6 | return z + k;
|
perhaps? I understand that tracking the lines/columns inside the expanded text is highly non-trivial (although collapsing them all onto the position of the top-level macro is already just as non-trivial), but still. Show me what the source that you actually can't compile, not the three-steps removed precursor to it — I sure as hell can't expand macros in my head.
Also, what is this first error message about in the first place? Why the hell does it expect unqualified-id before "int"? If you put "int" after the unqualified-id, it still would be illegal; and an unqualified-id is not the only thing that can follow the comma in this context anyway.
david2ndaccount 14 hours ago [-]
During translation of source text to a C program, the source text goes through several phases. It turns raw bytes into preprocessor tokens, then preprocessor tokens get expanded into more preprocessor tokens and then preprocessor tokens get translated into C tokens, which are fed into the C parser. There is no “expanded text”, only expanded tokens. Generally this is implemented as a state machine that produces tokens on demand, so there is no expanded text. When GCC shows you that error message, it uses the recorded source location and looks it up in the original source file. This is trivial to do. Expanding just one line of the source file would either require re-invoking the preprocessor from the beginning or storing already consumed preprocessor tokens.
Obviously this is possible, but no one has gotten around to implementing this feature and it’s not clear you would want either solution: buffering preprocessor tokens in case you need to expand a line for an error message greatly increases memory usage and the vast majority of compilations are successful ones with no errors, slowing down the normal case. And re-invoking the preprocessor to show you one line on every error message makes error reporting slow.
Joker_vD 14 hours ago [-]
> There is no “expanded text”, only expanded tokens.
If the compiler supports "-E", there absolutely is some equivalent of the expanded text, with (somewhat limited) whitespace info available: if you insert whitespace into between "b" and the comma in "b,", then "-E" invocation will dutifully show it. Yes, multiple spaces are collapsed into one, but their presence/absence is tracked already.
> Expanding just one line of the source file would either require re-invoking the preprocessor from the beginning or storing already consumed preprocessor tokens.
Yeah, keep the whole expansion of the latest top-level (not nested) macro. I believe some bytecode-like compression scheme (a replay of that state machine you mentioned) à la LZ would work fairly well, with minor additional memory necessary. Hell, scratch that: since #define/#undef are impossible inside a macro, you don't even need to remember the expansion of the macro, you simply need to keep the current macro definitions and remember what was the last macro you tried to expand — if your error happened inside its expansion, just re-expand again. Zero additional memory needed, since this information is already stored for the current way of error reporting — you can see in the current output that it remembers what macros it expanded and where, the whole path down to the offending token: at 1:21 "in expansion of macro 'W'" which was at 2:18 "in expansion of macro 'X'" which was at 5:13.
> And re-invoking the preprocessor to show you one line on every error message makes error reporting slow.
Eh, it's on the whole faster than making the programmer re-run with "-E" option manually.
david2ndaccount 13 hours ago [-]
> If the compiler supports "-E", there absolutely is some equivalent of the expanded text, with (somewhat limited) whitespace info available: if you insert whitespace into between "b" and the comma in "b,", then "-E" invocation will dutifully show it. Yes, multiple spaces are collapsed into one, but their presence/absence is tracked already.
Whitespace is significant in the preprocessor so of course it tracks whitespace. Whitespace isn’t dropped until phase 7 when pp-tokens are converted to C tokens.
As I said it’s not impossible, write the patch and see if GCC accepts it.
uecker 6 hours ago [-]
Also the position in the original source code is tracked.
In fact, this enhancement should be doable. Please file a bug. But the answer why something is not done in a C compiler is usually: Nobody did substantial investments on it in the last two decades. So if you want to see improvements, usually one has to submit a patch.
fuhsnn 17 hours ago [-]
There was a "__VA_TAIL__" WG14 proposal[1] that made it pretty simple to build custom map-like expansions, the OP's map example would be like:
You can play with it in the proposal author's ellipsis[2] and my slimcc[3], it's simple enough to implement (only about 30 lines of C) so I kept it in despite not seeing signs of it getting accepted.
I use something like this to turn __VA_ARGS__ into count/pointer:
// Create a stack-allocated argv from N args gcc, clang, tcc, mingw, msvc, etc
// argc is always correct, for portability argv has a dummy element when argc==0
// This avoids problematic 0-length compound literal arrays.
#define _ARG_CV(type, ...) _ARG_CV1(((type []){0, __VA_ARGS__}))
#define _ARG_CV1(array) _ALEN(array) - 1, array + (_ALEN(array) > 1)
#define _ALEN(array) sizeof array / sizeof *array
Then it's just:
#define sum(...) sum(_ARG_CV(int, __VA_ARGS__))
david2ndaccount 4 hours ago [-]
Right, you can work around it with things like this, but it means you’re forcing the C compiler to parse a compound literal just to get its length, 5 times! when the preprocessor already knows the length of `__VA_ARGS__`. Such contortions when you want a little bit of info that the compiler already has are what cause your compiler times to blow up if you try to do anything sophisticated with the cpp.
Has anyone tried to extend the C Preprocessor to be a fully turing complete language (perhaps something like Snobol) to allow for Terra like meta-programming?
(I am not talking about metalang99 or Cello like stuff, which exploits the Turing tarpit of the currently specified preprocessor but rather tasteful additions to the preprocessor itself).
0c3ca83 1 days ago [-]
C++ started as a preprocessor.
cowlevel 23 hours ago [-]
but not the C preprocessor, but a completely different preprocessor.
tambourine_man 1 days ago [-]
So did Objective-C
camel-cdr 24 hours ago [-]
The C preprocessor is practically turing complete, in the same sense that C it self or your computer is practially turing complete, but not strictly.
david2ndaccount 12 hours ago [-]
We’ll get there in this series.
andai 18 hours ago [-]
It's always bothered me that programming languages aren't programmable.
Well, I keep hearing that Lisp is, but I haven't put much effort into it. (I'd need a Lisp that runs in the browser, for starters!) Maybe I should take another look...
Or maybe I wish the VM was programmable. "Warn me when this type of operation takes place", or "Print the name of every function before it is invoked", etc. I end up just copy-pasting my own temporary statements all over the program.
FuckButtons 16 hours ago [-]
>(I'd need a Lisp that runs in the browser, for starters!)
We would be in a very different world if Netscape’s exec team had let Brendan Eich just use scheme as he had planned to.
kazinator 16 hours ago [-]
Block scoped let would have been there from the start, for one thing, rather than nearly two decades of function scoped var.
The article shows how to implement map, but it should be a builtin
fuhsnn 16 hours ago [-]
A builtin is definitely better for the efficiency alone. Vanilla chibicc needs 5GB to preprocess the example.c from map-macro, in comparison to compiling an amalgamation of itself only takes 50 MB. It of course wasn't optimized for memory use but it's a peak into how much token expansions are involved in the wizardry.
Neywiny 22 hours ago [-]
But that repo says it's effortful for the preprocessor and you can see in the code it's a recursion limited but what's in the file. If you had, maybe, a few thousand register definitions in an embedded system, that won't do it.
kazinator 15 hours ago [-]
I would add #defrec which defines macros that allow recursion.
rwmj 1 days ago [-]
String comparison in #if would be nice ...
AtalantaDB 1 days ago [-]
Wow) I like the idea of making __map a builtin instead of forcing users to simulate recursion through macro expansion. One thing Im curious about is how you handle expansion semantics. Consider:
#define X 42
#define STR(x) #x
__map(STR, X)
Should this produce "X" (like STR(X) does), or "42" (if __map expands its arguments first)?
david2ndaccount 4 hours ago [-]
__map() expands its arguments, like other function-like macros.
One of those is the two-stage CAT, another one is the NARG the article talks about.
__COUNTER__ is going to be added to C29, the next version of the standard, maybe there's hope for __VA_COUNT__ and __CAT__ too!
Otherwise, it's probably a good idea to pretend that the only preprocessing directive is #include. With constexpr, even #define is obsolescent.
Getting rid of something because someone sometimes abuses it, is short-sighted.
As an aside, clangd can dereference a macro in place to make debugging easy.
Most people describe C++ in those terms.
Tangential note but: why does this require a separate re-invocation of the compiler with the '-E' flag? Say I have this silly example with a comma instead of the semicolon inside W() macro:
When I shove it into GCC without -E, it says instead of, you know, showing me the expanded text so that I could see what's actually wrong with the generated variable declarations? Something like perhaps? I understand that tracking the lines/columns inside the expanded text is highly non-trivial (although collapsing them all onto the position of the top-level macro is already just as non-trivial), but still. Show me what the source that you actually can't compile, not the three-steps removed precursor to it — I sure as hell can't expand macros in my head.Also, what is this first error message about in the first place? Why the hell does it expect unqualified-id before "int"? If you put "int" after the unqualified-id, it still would be illegal; and an unqualified-id is not the only thing that can follow the comma in this context anyway.
Obviously this is possible, but no one has gotten around to implementing this feature and it’s not clear you would want either solution: buffering preprocessor tokens in case you need to expand a line for an error message greatly increases memory usage and the vast majority of compilations are successful ones with no errors, slowing down the normal case. And re-invoking the preprocessor to show you one line on every error message makes error reporting slow.
If the compiler supports "-E", there absolutely is some equivalent of the expanded text, with (somewhat limited) whitespace info available: if you insert whitespace into between "b" and the comma in "b,", then "-E" invocation will dutifully show it. Yes, multiple spaces are collapsed into one, but their presence/absence is tracked already.
> Expanding just one line of the source file would either require re-invoking the preprocessor from the beginning or storing already consumed preprocessor tokens.
Yeah, keep the whole expansion of the latest top-level (not nested) macro. I believe some bytecode-like compression scheme (a replay of that state machine you mentioned) à la LZ would work fairly well, with minor additional memory necessary. Hell, scratch that: since #define/#undef are impossible inside a macro, you don't even need to remember the expansion of the macro, you simply need to keep the current macro definitions and remember what was the last macro you tried to expand — if your error happened inside its expansion, just re-expand again. Zero additional memory needed, since this information is already stored for the current way of error reporting — you can see in the current output that it remembers what macros it expanded and where, the whole path down to the offending token: at 1:21 "in expansion of macro 'W'" which was at 2:18 "in expansion of macro 'X'" which was at 5:13.
> And re-invoking the preprocessor to show you one line on every error message makes error reporting slow.
Eh, it's on the whole faster than making the programmer re-run with "-E" option manually.
Whitespace is significant in the preprocessor so of course it tracks whitespace. Whitespace isn’t dropped until phase 7 when pp-tokens are converted to C tokens.
As I said it’s not impossible, write the patch and see if GCC accepts it.
In fact, this enhancement should be doable. Please file a bug. But the answer why something is not done in a C compiler is usually: Nobody did substantial investments on it in the last two decades. So if you want to see improvements, usually one has to submit a patch.
> #define T(x, ...) printf(#x " = %d\n", x); __VA_TAIL__()
> T(1, 2, 3, 4, x, y, z)
You can play with it in the proposal author's ellipsis[2] and my slimcc[3], it's simple enough to implement (only about 30 lines of C) so I kept it in despite not seeing signs of it getting accepted.
[1] https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3307.htm
[2] https://codeberg.org/gustedt/ellipsis
[3] https://github.com/fuhsnn/slimcc
Your version expands `sum(1, 2, 3, 4, 5, 6)` to:
whereas with __VA_COUNT__ it is just:(I am not talking about metalang99 or Cello like stuff, which exploits the Turing tarpit of the currently specified preprocessor but rather tasteful additions to the preprocessor itself).
Well, I keep hearing that Lisp is, but I haven't put much effort into it. (I'd need a Lisp that runs in the browser, for starters!) Maybe I should take another look...
Or maybe I wish the VM was programmable. "Warn me when this type of operation takes place", or "Print the name of every function before it is invoked", etc. I end up just copy-pasting my own temporary statements all over the program.
We would be in a very different world if Netscape’s exec team had let Brendan Eich just use scheme as he had planned to.
__map(STR, X)
Should this produce "X" (like STR(X) does), or "42" (if __map expands its arguments first)?