For everyone with a “why don’t they just…?”-type suggestion, it’s worth carefully rereading TFA and considering what the macros do now and the problem they are trying to solve.
The macros now do type-safe comparisons that work correctly with combinations of different argument types where this is possible, work in a constant context (eg defining array bounds), work correctly in the face of implicit type coercion and include a 3-way min and max that have these same desirable properties (same as the two way min and max).
The problem(s) are it’s a bit slow to compile and the preprocessor expansion (although not necessarily the final generated assembly - didn’t see anyone saying that) is a bit bloated.
So when you make a “why don’t they just…. ?”-suggestion, make sure your suggestion is at least as good as what they have now in terms of the desirable functionality and correctness and then tackles the actual problems in some way. I’m not sure all of the suggestions I have seen here and in the lwn comments succeed at meeting those two criteria.
I have a "why don’t they just" question: if they're going to rely on compiler intrinsics anyway, why don't they just implement the whole darn thing in the compiler and do whatever they want there? If you're going to marry your code to the compiler, at least take full advantage of it?
Assuming good faith, they don't do that because they are kernel developers not compiler developers, and they want it to build on the compilers that real systems have at hand.
If somebody proposed a new extension and got llvm and gcc to both implement it today, they would still need something to work on old compilers. The oldest GCC supported by Linux is 10 years old nearly.
But for older compilers they already have something that compiles correctly for valid inputs, no? They don't need e.g. the type-safety-check nonsense for that. The newer compilers will still catch mistakes in the usage sites via the intrinsics.
For today's compilers they have something that compiles very slowly and they want the type safety checks because those are the ones the kernel developers use. That answers the original question doesn't it?
If a new compiler extension was proposed and implemented and released in both llvm and gcc, some years after that they could drop support. But there would be no real imperative to drop support early since they will already have developed some code that works okay on those toolchains.
> For today's compilers they have something that compiles very slowly and they want the type safety checks because those are the ones the kernel developers use. That answers the original question doesn't it
No, the point was for today's compilers they could have something that compiles quickly and does everything they want.
They couldn't implement it in today's compilers unless they had a time machine. Today's compilers are already built and distributed and are what kernel developers use to compile their code today. I can't understand what you're finding so difficult about this.
> They couldn't implement it in today's compilers unless they had a time machine. Today's compilers are already built and distributed and are what kernel developers use to compile their code today. I can't understand what you're finding so difficult about this.
I am saying they could have either implemented this in the past (and thus had it land by now) or implement it now (and have it land in e.g. GCC in the near future). Neither appears to be the case. I was not suggesting they have a time machine. I can't understand what you're finding so difficult about this.
You're asking why they don't do it and I gave you an answer. If something different had occurred prior to today then the situation today could be different. But it's not, hence the situation we find ourselves in which you asked about, which is what I was helping you to understand.
You realize that implementing something is not the same as using it, right? Just like how cooking something is not the same as eating it? If they implement it today, they can use it tomorrow. I was asking "why don't they just implement the whole darn thing in the compiler" and your response was... because they don't have a time machine?
> No, my response to that is because they are kernel developers, not compiler developers. For the nth time. This really isn't rocket science. They need something that works in the kernel on today's compilers. The end.
I have no idea how you make this stuff up, but Linux and GCC have thousands of developers working on them from all across the world, and Linux heavily takes advantage of GCC built-ins. There even exist active maintainers who understand and contribute to both... not that that is in any shape or form a requirement for putting out a request from one project to the other. The idea that they would've been unable to find a single person with sufficient interest & understanding of compilers to implement min/max in it is downright absurd. If they wanted to do this someone would have implemented it a long time ago. And yes, the end.
The kernel developers a writing code for the kernel, that has to be compiled with the compilers they have. Glad I was able to clear that up for you, you're welcome.
That series of macros is a nice demonstration of the incredible effort it takes to attempt the most basic generic programming in C. Perhaps it would be more productive to just accept the limitations of the language and define a version of `min` and `max` for each type.
Genericity is not really the issue as I see it. The basic K&R macro using ?: is fully generic. The problem is that C has implicit conversion between essentially any numeric types. The main point of the kernel macro was to prevent such conversions.
I think implicit type conversion is a mistake, perhaps one of the few true design flaws in C. Languages like haskell and rust went with explicit conversions which is probably a better idea overall, even if it does increase the code verbosity a bit. C++ instead doubled down and added many more ways for implicit conversions to happen.
The reason the current macro is so complex is because it supports mixed types while avoiding (failing on) integer promotion bugs. A version supporting arguments of all the same type would be just as trivial as in C++ (albeit relying on GCC extensions like statement expressions).
This is addressed if you chase down the original kernel mailing list thread (the "flamewar" linked in the article). It was important to Linus that the macro was actually min/max and not min_slong max_uint etc, so that people couldn't "accidentally" use the untyped version. In other words he was trying quite hard to "force" people to use these macros.
If you would then say "why doesn't min/max just implement a switch on each primitive type", I think on some level it does just do that.
That will make `typedef` much less useful, though. C is the real problem; not the attempt to do generic programming in C. (And `_Generic` won't solve this problem anyway. Only a GCC extension of statement expressions will.)
A compiler for a language with generic types will definitely do less work to implement generic min/max than the C preprocessor has to do in order to perform all the text substitutions generated by these macros.
Or a demonstration about the poor state of programming languages in 2024.
Of course there are others than C. But not many suitable for writing a kernel. And no obvious choice for what Linux could do today. Of course they are starting with Rust, but nobody can predict when that will make the last C macro unnecessary. Unless your prediction is never...
Most kernels not written in low-level languages (C, stripped-down unsafe C++, or assembly) were written for special hardware. Such hardware was built to handle a lot of the low-level details so that the software didn't have to. This particular approach can't be translated by software alone to modern general-purpose microprocessor-based platforms. Even if we were able to transition to such a do-more-work-in-hardware platform somehow, then we would just be trading kernel bugs for hardware bugs, and the latter aren't always fixable (even with firmware and microcode).
The x86 family didn't come out of Bell Labs. It's not a question of where the programming languages came from but of the boundary line between hardware and software.
Doesn't Rust's `no_std` make it easy to throw out idioms you can't use (or don't like) and just keep the syntax and most of the tooling? I recall that one of Linus's preconditions for inclusion was that Rust's default of panic!ing on allocation failures had to be overrideable.
Whereas C++ tends to make you eat the whole enchilada.
C++ compilers have similar options, e.g. VC++ has /kernel, and there is the freestanding standard as well.
Also see Embedded C++ as used by macOS IO Kit, which thankfully there isn't a Linus at Apple.
Regarding Rust, no_std, doesn't change the npm like dependencies, macro festival (with two ways of doing all kinds of stuff), abstraction party with type driven development, the compile times.
The C++ "freestanding" even when the current round of work is completed, is nowhere close to Rust's core.
Partly this is because Rust thought about this earlier, and partly it's because of the relationship between C++ and the allocator, having operators for a feature which might not even exist on your target, awkward.
I don't see how picking C++ counts as "more sensible". Linus chose C 30+ years ago because it was the only practical option. Bjarne (and some other C++ proponents) worked hard to present C++ as somehow a natural upgrade to C and that worked on Microsoft and at some of the defunct proprietary Unix vendors but it didn't work on Linus.
My strong impression is that both Apple and Microsoft are moving away from C++ in their core systems.
It's not just Linus. I use C++ daily for a variety of things, but I understand the hesitance to use it in real-world kernels and embedded code. For instance: The switches to disable features like exceptions are generally considered "not supported, use at own risk"; removing them is not an official part of the language. C++ templates are slow to compile and have many obvious shortcomings — but if you avoid them, you're back to using C macros. The C++ abstract machine puts a lot of weird complications in the way, adding new UB landmines on top of C.
This doesn't mean using C++ for kernels and such is impossible, but anyone with a conservative, risk-averse mindset (like Linus) isn't going to want to touch it.
Rust, on the other hand, was deliberately designed for such uses, starting from a fresh sheet of paper. Giving systems programmers everything they want without any hidden "gotchas" is its whole raison d'être.
> C++, otoh, is run by committee and far removed from the community.
It is hard to be close to a community with the size and divergence of the C++ one. It is (relatively) easy to be close to a community the size of the Rust one.
Ehhh... not much more so than how much `npm install` hides. These macros are part of a gigantic tower of absurdity that costs many, many times more than they need to, but we use them because dev time is more valuable than CPU time.
This reminds me of how ~25 years ago I wrote little C macro library to perform safe (non-overflowing, and/or saturating) integer arithmetics and comparisons, however the small application I wanted to use it in had the compiler crash after 2-3 hours due to insufficient RAM+swap when compiling a single source file using those macros. It turned out the macro expansions made the translation unit grow to GB size, which must have been 4-5 orders of magnitude over its original size.
The macros automatically derived the signedness and the minimum and maximum value of the integral types involved, in a way that didn’t made platform-specific assumptions like two’s complement or no padding bits. Cases like comparing signed long to unsigned long also needed extra logic, due to there being no larger type that encompasses both. I don’t remember the details, but it did have a significant number of nested macro invocations.
My question is, why isn't there a `min() and `max()` in the standard library? Even accepting C's philosophy of a minimalist stdlib, these feel like uncontroversial functionality to me. TFA shows there's enough complexity involved in doing it correctly that it makes sense to provide an implementation rather than have everyone write the same often-subtly-incorrect macros. They could use a `__builtin_cmp(type, a, b)` which can use the correct type-casts and prevent double-evaluation without needing any macro weirdness.
C codebases can often be updated with minor changes to compile with a C++ compiler, after which C++ features can be gradually introduced. Is the Linux kernel different in this regard?
At this point, C and C++ are not compatible. Please stop with this "Oh, just compile C with a C++ compiler." That ship has sailed over twenty years ago with C99. One major difference is designated initializers, which works differently between C and C++ [1]. And C also allows structure literals in function calls:
Compound literals are especially pernicious[1] because GCC and (I think) clang support them in C++ mode, but they have expression lifetime like C++ temporary objects, not block lifetime as in C. That's a guaranteed recipe for dangling pointers and stack smashing/snooping.
[1] From a C++ is just a C superset perspective. In C compound literals are awesome.
Yes, with minor changes. A few variables need to be renamed, a few cast need to be added. Some churn for sure on such big codebase, but doable nevertheless.
GCC did it, other projects did it, I don't see why the kernel can't.
I think the obvious one is that the Linux kernel is no small little project, and any tree-wide change is highly non-trivial. The compressed current source release is 138 MB. A 2021/5.11 lines of code figure I found was 30.3 million.
It's not impossible, but the Linux kernel is quite a complex project.
The specific macro discussed in the article is using various GCC builtins to safely support mixed integer types; i.e. failing on unsafe type promotions or coercion, but otherwise working automagically. C++ std:min, by contrast, requires all the values to be the same type. AFAIU, to accomplish the same semantics in C++ would require either template metaprogramming, or doing something similar to what the current version is doing with macros and GCC builtins. A C++ solution might ultimately be cleaner, but I don't think there's anything in the standard C++ library that is a drop-in replacement.
Nor would it perform well during compilation, which is very well discussed in the thread. C++ compilation is slow, like the macro expansion of the C versions of min and max.
It’s all in the thread, I don’t know how GP missed it.
From a moderate skim, I'm not seeing much in there that really addresses this beyond FUD and over-simplification.
Which, I mean... it's the Internet. That's kinda expected. But if there's something specific you're seeing, could you link to it? I'm curious as well what the "stick to C" crowd's reasons are. "C plus this one feature of C++" seems rather defensible at a glance (ignoring the social difficulty in choosing which feature), but I'm sure it's much more complicated than that in practice.
Even if a project can choose some specific feature(s) to switch to C++ for, it'd severely reduce the barrier for adding reliance on more features; why did feature X get a pass, but not this other one? And C++ has a lot of such potential features that are harmless and easy to justify at their best, but can become headaches when used more broadly, requiring everyone and everything to deal with them.
The social aspects are very large and it wouldn't surprise me at all if that was by far the main reason... but that's much less of an issue in a kernel-like context where there are already oodles of rules beyond "write valid C code". They can and do impose significant limitations on the languages they use, successfully, for decades. Seems like they'd be able to do that with C++ too.
There is a comment purporting to show a difference between compilation speeds in C and C++ which uses iostream for the C++ example, which completely misses the point. Sure C++ has a lot of warts, but in terms of compilation speed it should be completely reasonable to use a few simple template functions.
Yeah, that's the main thing I saw and it's just plain completely wrong. The fact that C++ can more easily bloat into large build times, and people frequently make larger-build-time projects in it, implies absolutely nothing about its behavior in replacing simple macros like max/min.
There's ample evidence that it'll build just a fast as C there, so it's not an issue in this context, and that both can build quickly with care. That's kinda the point of C++: you can write plain C code plus [this one thing] and you basically don't pay for the rest, and it generally achieves that.
It would be far easier to add as builtins the features that are missing to GCC than change the language, that would involve rewriting a ton of code (even switching from C to C++ they are not 100% compatible, also, they may introduce bugs difficult to spot cause their incompatibility). The Linux kernel already uses a ton of GCC extensions (even the min/max macros suggested in the article) that is not compatible with other compilers anyway (and I don't see a reason to be, since GCC is the compiler of the GNU project anyway, unlikely to compile the Linux kernel with MSVC, or even I don't see much reasons to use clang anyway).
C doesn't have any type-generic function declaration. So any viable solution had to be at least partially powered by a macro and resulting (one-directional) type inference.
Even with _Generic you can't declare generic functions. You'd still need a macro call that uses _Generic to dispatch different implementations depending on the parameter types.
The concrete problem that the kernel is facing is an exponential growth of macro expansion. Any current solution, safe or not in terms of re-evaluation, needs at least two copies of both arguments to be expanded [1], so expressions like `min(min(a, b), c)` will expand some arguments four times. The growth factor would be much larger than two if the definition wasn't carefully designed to avoid such cases.
[1] `_Generic` also requires at least two because you need one copy to select the generic implementation and another to call it. C23 `typeof` (or the equivalent GNU extension) allows for a compact type tuple matching:
inline static int max_int(int x, int y) { return x > y ? x : y; }
inline static unsigned max_uint(unsigned x, unsigned y) { return x > y ? x : y; }
/* ... */
#define MAX(a, b) (_Generic( \
(void (*)(typeof(a), typeof(b))) NULL, \
void (*)(int, int): max_int, \
void (*)(unsigned, unsigned): max_uint, \
/* ... */ \
default: max_type_error \
) (a, b))
The problem is C lacks an equivalent to C++’s constexpr (or preceding template insanity), so you can’t use functions in constant contexts, eg
struct S {
int foo[max(10, 15)];
}
Isn’t possible in C, macros are the only option.
So even if you were to try to use _Generic in a macro to handle type correct dispatch you would not be able to use that macro in many of the contexts it is needed.
Honestly constexpr is something that would really help C, and does not need to bring in any other c++ features. Although I guess in this case the lack of the full template and such features set would mean matching the kernel requirements would still require a macro+_Generic to adopt.
> some of the changes to the macros made some developers (including Bergmann) nervous
These macros are now so complex that they're reluctant to touch them. Seems like there is a clear need for some thorough tests here? This is exactly the sort of thing that is eminently testable.
Maybe. Most C code of non-trivial length is UB or at least implementation defined, you really don't want to "tickle" something in a low-level highly used macro.
You're exactly right, sorry. I meant that even with tests it's probably too subtle to do the standard test driven design method of programming, you have to be way more careful.
Could not the main compilers get involved and add builtins so an ifdef makes the common path on the main compilers like gcc use builtins and the slowdown only hurts those using the less mainstream compilers? If it takes off the other compilers would quickly add the feature.
What you're suggesting is basically a worse version of adding built-in min() and max() to the C spec. Which, to be fair, would be quite nice, but I guess the working group didn't want it in the standard.
The effects of preprocessor in C are quite significant. Old DOS Turbo C had shown amount of lines compiled and the speed in the compilation progress window in the IDE. IIRC straight
Every time when such thing happens, then it becomes a language suggestion opportunity. But for those who suggest another language, are you going to replace a 50M line code base because of you want have a fancy min / max? I don’t think this is a responsible suggestion.
How would hygienic macros fix this? It seems like the macros were working fine, but the amount of generated code increased compile time significantly. Wouldn’t a more sophisticated macro system still generate a bunch of extra code and result in slow compile times? It even seems like the solution was to fall back to “dumb” macros when feasible for compile-time performance reasons.
> Wouldn’t a more sophisticated macro system still generate a bunch of extra code and result in slow compile times?
Not necessarily.
The preprocessor code here picks up the original source, and blows up the initial code (which is about "min3(long_a, long_b, long_c)") to 47 MB of code. no fancy stuff, just 47MB of C-Code on the disk. That's a lot of code the compiler then has to parse and handle.
If hygienic macros are a first-class citizen in the compiler, the compiler parses the original macro code once and then just modifies it in-memory. There is no reason to write 47MB of code somewhere and read it back, this would just happen as an AST modification in memory.
But that is also a much smaller reason. First-class macros allow the compiler to reason based off of the types and structure of the macro inputs. You don't have to guess if something is constant, bounded, unbounded and such. Strong types can enforce this safely and macros and optimization can use these strong guarantees. And sufficiently strong type information can open doors for far, far more powerful optimizations overall.
Just for the record - I'm fully aware why the kernel is where it is, and why it will stay there, but there is far improved compiler and language theory from there.
There are plenty of projects smaller than the Linux kernel that have developed and employed DSLs (to varying degrees of success, I'll grant). I wonder, are there any languages out there designed specifically for kernel programming?
Given the number of preprocessor hacks used in the kernel, and the amount of GCC-specific behavior that the codebase depends on, it seems like they are already halfway there.
Honestly, they'd probably be better off if they ditched all the sed/awk/macro BS and just went back to bash scripts (or perl/TCL, if you don't like weird syntax issues) that spat out C code. Rust saw the writing on the wall and implemented proc macros.
This also shows how programmers sometimes use far too fancy stuff for what they do. If you are doing some intense computation, split it, or do it and later slap sanity check inside. It will be even more readable. If you just play with bunch of vars, sure.. min/max macros can be easier to read.
The most pleasant C codebases that I have read essentially banned macros outside of includes, conditional compilation, and named constants. Please just stop trying to use macros to metaprogram in C.
The macros now do type-safe comparisons that work correctly with combinations of different argument types where this is possible, work in a constant context (eg defining array bounds), work correctly in the face of implicit type coercion and include a 3-way min and max that have these same desirable properties (same as the two way min and max).
The problem(s) are it’s a bit slow to compile and the preprocessor expansion (although not necessarily the final generated assembly - didn’t see anyone saying that) is a bit bloated.
So when you make a “why don’t they just…. ?”-suggestion, make sure your suggestion is at least as good as what they have now in terms of the desirable functionality and correctness and then tackles the actual problems in some way. I’m not sure all of the suggestions I have seen here and in the lwn comments succeed at meeting those two criteria.