I haven't done this for a while, but once upon a graduate program I wrote a compiler from a made-up-language (MUP) to C. MUP had some strange control structures, and if C did not have "goto", it would have been a lot more difficult to implement those structures. Since then, I have always thought languages should have a "goto" statement that human-written code is not allowed to use. :)
It often simplifies a lot of the compiler implementation if you only have reducible control flow. LLVM is fine with irreducible control flow, because it has to handle C, but rustc uses a CFG for the borrow checker, which is flow-sensitive. You can convert irreducible control flow to reducible control flow, but it can explode the size of the graph in pathological cases.
(I don't recall whether the borrow checker actually depends on the control flow being reducible, but some of the possible future improvements we've talked about with "non-lexical lifetimes" definitely do. There are also nasty interactions between RAII and irreducible control flow…)
It does have the effect of picking a single dominating entry point for a loop with multiple entry points, but if you want a single-entry region that's necessarily true. Multiple-entry regions are probably possible, but they could be counterintuitive and produce even stranger error messages than the current region system.
The borrow checker is based on dataflow analyses that should work on arbitrary CFGs, assuming an appropriately generalized notion of region.
I think the point of parent comment is "it's ok to don't have goto in Rust, but it's not the reason to be proud of just not having goto". Goto is important enough operator and doesn't deserve blind hate.
Agreed... I honestly tend to break code into a lot of discrete functions, and combine for workflows, but sometimes you just need to easily jump back a few places, and where goto is available it's not always a bad option. Just one that should be used sparingly... once in a complex workflow is fine.. more than twenty times in a few thousand line method, not so much.
Don Knuth, "Structured Programming with go to Statements":
"Just recently, however, Hoare has shown that there is, in fact, a rather simple way to give an axiomatic definition of go to statements; indeed, he wishes quite frankly that it hadn't been quite so simple....
"Informally, α(L) represents the desired state of affairs at label L; this definition says essentially that a program is correct if α(L) holds at L and before all "go to L" statements, and that control never "falls through" a go to statement to the following text. Stating the assertions α(L) is analogous to formulating loop invariants. Thus, it is not difficult to deal formally with tortuous program structure if it turns out to be necessary; all we need to know is the "meaning" of each label."
Yeah. Even basic things like building SSA can be done more simply on "structured" CFGs.
The benefits are enough that data structure and algorithm designs in the JVM compiler world often take advantage of assuming reducible control flow even though Java bytecode can express irreducible CFGs. Instead, such programs are left to the interpreter.
Rust's CFGs are reducible with unbounded treewidth, not structured, because Rust has named exits from loops that nest arbitrarily.
Most problems on reducible graphs are not easier than for general graphs, because you can always compute a loop forest (for generalized loops, not natural loops with a single entry point) and just consider a derived acyclic graph.
The structured SSA building algorithm I had in mind has a fairly simple extension that covers break, continue, and early return. It's more complicated, but still way simpler than iterated dominance frontiers.
Technically speaking, break / continue / early return are unstructured by the classical definition. The algorithm you are likely thinking of can easily be extended to handle arbitrary control-flow:
> You can convert irreducible control flow to reducible control flow, but it can explode the size of the graph in pathological cases.
True, but only in node-splitting approaches. If you use a label threading variable (as emscripten's relooper does), there is a guaranteed reasonable limit on code size increase (at the cost of performance).
I've always defended goto and gotten a lot of flak for it. As soon as I say, "I wish I had X-language had gotos", I see jaws drop. Response: "Wow, haven't you heard the news?! GOTOs are considered harmful!"
I think goto should be in almost every language. It's one of the most primitive instructions, why shouldn't it be available when needed? Yes, it can be misused, just like any other feature in the language, but it can also be used to great benefit. State machines, for example, can make good use of gotos.
Hell, look at any unix library your machine uses every day. The ones you don't even think about. Start with libncurses: I promise there's a few gotos where needed.
Rust is not a language that looks favorably on things that are easy to misuse and hard to use correctly. You can make the same argument about, say, untagged / C-style unions, which are much more common in C, but Rust doesn't have those either.
I don't plan to stop using ncurses in particular, but one of the reasons I am supportive of Rust is that I would like to stop using all C software sometime in my lifetime.
If you really need them, you can implement them in a library by declaring a suitably sized byte array and having methods that return casted pointers to it. This would be illegal in C due to strict aliasing, but IIRC Rust does not have a strict aliasing rule (because almost all pointers being the equivalent of 'restrict' drastically reduces the benefit).
There are issues with size_of not being a constant expression and such (at least in stable), but those are definitely going to be fixed.
The most common use case would be interop with C libraries that use untagged unions. This is a significant hassle when writing C bindings in Rust today.
I can think of many, mostly revolving around cases where the tag is not stored inline with the structure in question. Where space efficiency is important this is quite common. C interoperability is obviously another important use case (the current solution of passing around [u8] arrays is pretty atrocious).
> It's one of the most primitive instructions, why shouldn't it be available when needed?
I see what you're saying, though I don't find this a compelling argument. By this logic, why not allow direct register access in all languages?
Just because something is a primitive operation doesn't mean you want to include it, especially not if it's more difficult to enforce guarantees your language would like to make about valid programs.
The kernel has plenty of goto, mostly (as far as I know) to handle errors and safe resource deallocation. There is no reason to keep using GOTO if you have better mechanisms to handle these things.
The target of a goto statement is a specific place in code. It is not code that refers to the task you are trying to perform.
The main advantage of using higher-level languages is that you can talk about manipulations of data. Goto doesn't manipulate data, it manipulates the machine. And if you want to _really_ manipulate the machine, you're going to want more than just goto.
>Raise your hand if you plan to stop using ncurses because of how opposed to how "harmful" goto statements are.
Goto statements are not harmful to computers or to code. Ncurses isn't what is harmed by goto, nor does "using ncurses" imply any interaction with goto at all.
Muhammad Ali was a good boxer. He was Muslim. Am I to conclude that one must be Muslim to be a good boxer, now?
Yes, ncurses is a good program. Yes it uses goto. But that doesn't imply that goto is necessary to it being a good program. Or that it is even a mildy efficient way of doing well. (Especially not when there is a whole universe of alternatives.)
If you're going to defend goto (and there are reasons to do this) you should probably do so without employing such blatant logical fallacies. It's irresponsible and detracts from your point.
C's goto has a bad rep it only partially deserves, however I can't agree that it should be in every language. I use it fairly frequently, but virtually all the uses are about cleanup after error. I like that it requires unique label for identification, as it allows for language-enforced documentation. But my gotos always go "down" the source code, never "up. Through goto I can basically setup an error-condition code flow for every function, with multiple conditional entry points into that flow and if there was another language feature that allowed just this particular usage in cleanup, without multiple nested scopes and with clear labelling, then I would see no reason for keeping goto. Throughout the years we developed a way to neatly use an unsafe feature, but it's always better to have such things enforced by language, not by tradition.
If you're not implementing fast FSMs, fast bytecode interpreters (and dynamic dispatchers in general), and you're not using metaprogrming to the full power, not implementing multiple embedded DSLs - then you can live without a goto. Otherwise it is essential.
> Metaprogramming to the full power sounds like Lisp.
It does, yes, although many Lisps got a pretty limited backend for this sort of things. My preferred metaprogramming environment must have a fallthrough mode for allowing generating low-level code where Lisp semantics is not sufficient. Rust seems like a very good target platform in this sense, so the lack of goto really hurts.
> It doesn't have goto.
Of course they do (tagbody in Common Lisp, for example). And when they don't, it's often relatively easy to add one.
> Of course they do (tagbody in Common Lisp, for example). And when they don't, it's often relatively easy to add one.
I was wrong, however it is much more limited than the goto everywhere from C, as where the labels are located is clearly defined in the tagbody and not in every possible statement.
> Raise your hand if you plan to stop using ncurses because of how opposed to how "harmful" goto statements are.
What if I just avoid contributing to the ncurses codebase? I've used plenty of useful tools with absolutely horrific codebases that I'd never want to touch in a million years. Not sure if ncurses is one of them.
The whole "it gets used, ergo it must be a good idea" argument doesn't hold much traction with me - even if I think using it when in C, to enforce single exit style, to work around the lack of RAII constructs, goto is the lesser evil.
I started with GOTO in BASIC. I used it a lot. It structured my initial reasoning about control flow. Despite this, in the past few years, I've used a naked goto maybe once or twice, and in all cases later rewrote it without the goto, which in my opinion increased it's readability. (I generally always have the option of C++ over C, and choose it, rendering single exit style 'useless'.)
Most of those are single exit style gotos. Those that aren't, do cause some concern, despite being "one of the most beautifully written C programs", even to their original author from the looks of it:
if (errno == ENOMEM)
goto retry; /* possible infinite loop? */
For what it's worth, it seems unlikely to be an infinite loop, short of encountering a bug in sysctl, or another process/thread constantly adding data. I had to google the header path to find an appropriate manpage (i.e. not _sysctl, not sysctl the program) to figure this out...
I've encountered worse edge cases before, however, and I'd really prefer my programs crash properly, instead of hanging when they do.
> State machines, for example, can make good use of gotos.
The performance complaints about an additional branch misprediction when using the "for(;;) switch(...)" style without gotos, is one of the few arguments that moves me, if only slightly. That seems like a case your standard optimizing compiler really should be able to handle, however. I'll assume they don't, as I'm too lazy to test if this is merely hearsay...
That said, I'll even use "goto case" in C# on occasion where I'd use case fallthrough in C++, if I'm feeling particularly lazy and don't want to turn things into proper method calls that can simply call each other just yet. I usually clean it up before I start to confuse myself.
It's not something I'd miss if it were gone, however. It's something I use only rarely, and only as a crutch to stave off cleanup. Not exactly a ringing endorsement.
EDIT: Code formatting, proper insertion of subject...
I wish C/C++ had a labeled break construct, like JavaScript, Java, Rust, and other languages have. It's surprisingly powerful, while still remaining structured. I personally have enjoyed learning about it, and about just how rarely an actual goto is really needed.
I've never had to use 'goto' in C++ except to break from a nested loop. In C++ labeled breaks would make 'goto' completely obsolete.
In C it would still have use in the implementation of orderly error handling -- the pattern where you hand-implement exception handling in C by putting an on_error: label at the end of the function that is goto'd on error. The addition of some orderly construct for this in C would eliminate that case, leaving no real role for goto there either.
A bitecode interpreter is another place where it's nice to have gotos.
Here's the base code without gotos:
typedef enum { ADD, MUL, ..., END } opcode;
void run() {
opcode ins;
while (1) {
ins = fetch_next_inst();
switch (ins) {
case ADD:
perform_addition();
break;
case MUL:
perform_multiplication();
break;
...
case END:
wrap_up();
return;
}
}
}
You have 3 jumps on each loop. From the break to the end of the loop, then from the end to the top, and one from the switch to the right case. The first one might be optimized away, but let's remove it explicitly.
typedef enum { ADD, MUL, ..., END } opcode;
void run() {
opcode ins;
start:
ins = fetch_next_inst();
switch (ins) {
case ADD:
perform_addition();
goto start;
case MUL:
perform_multiplication();
goto start;
...
case END:
wrap_up();
return;
}
}
Assuming a non lousy compiler, we haven't improved anything yet. But now the fun starts. We can go down to one jump for each iteration.
Voila! a single jump every time around. Now, depending on what kind of architecture you're running on, the size of the cache, etc, this may or may not be faster.
Granted, this is not the kind of code you write everyday. But sometimes speed matters, and good luck writing this without gotos.
I really dislike 'clever' code like this, even when speed is paramount you will find that by obscuring the flow you make it harder, not easier to really optimize the code.
Over time it tends to evolve into ever messier and harder to understand versions of the initial run after which a future maintainer will end up losing sleep and or hair chasing some production bug.
Consider this (slower!) much clearer alternative, and consider too that if you need to eliminate one goto for speed reasons that you're most likely doing something wrong:
typedef enum { ADD, MUL, ..., END } opcode;
int process_instruction(opcode ins) {
switch (ins) {
case ADD:
perform_addition();
break;
case MUL:
perform_multiplication();
break;
...
case END:
return FALSE;
}
return TRUE;
}
void run() {
do {
} while (process_instruction(fetch_next_instruction());
wrap_up();
}
That's a whole function call overhead (but you can eliminate that with an 'inline'), it's easier to test and much easier to follow what it does.
It would be interesting to see what the actual difference is in speed when comparing those two versions, I suspect that the contents of 'perform_addition' and 'perform_multiplication' are going to be the key here, not whether or not the loop uses a short-cut or an instruction (or two) less. Oh, and you could have eliminated that 'ins' variable.
I imagine that if you have more instructions (a few hundred) the overhead of the switch starts to become significant. At 256 opcodes you need 8 jumps, and hence 8 CPU cycles. That's non-trivial, especially if most of the opcodes can be implemented more quickly than this. For example, a quick Google search says that for an Intel CPU, addition is 1 cycle and multiplication is 3 cycles.
Switches over enums are usually compiled to jump tables. What you see in source and what you get after the optimizer is done with it are sometimes very far apart. Looking at the output of gcc -S can be very enlightening at various levels of optimization.
In summary, the assembly of the goto version is much shorter and there's a 10% speedup over SPEC JVM98. The optimization works out because: a) instruction bodies are often short; b) instruction dispatch is the hottest code path of any interpreter; c) it's orthogonal to all or almost all other optimizations.
That's about what I expected, contrary to the 2-4x times speedup mentioned above. Still, I can imagine in isolated cases you really would get a better number than that 10%. So, for code where performance is super important and where the effort of the optimization and lack of transparency is outweighed by the performance boost this makes sense.
Yes, that 10% is an average over seven benchmarks. The numerical one gets a 25% speedup, the one with a bunch of exceptions doesn't benefit at all. The funny thing is, if you're interested in performance, you're probably going to be writing a JIT, so this optimization is really just a stop-gap measure. In fact, the inline threading I mentioned is a kind of cheap JIT: it copies the in-memory executable code from the instructions at these goto labels to form basic blocks in an executable CFG. That gets you 1.6x speedup on those benchmarks, but it's a lot more work.
> The funny thing is, if you're interested in performance, you're probably going to be writing a JIT, so this optimization is really just a stop-gap measure.
Exactly. And stop-gap measures are fine if and when they're followed up by a proper solution. Unfortunately stop-gap measures tend to be a lot more permanent than originally intended in practice.
Edit: Seeing it written like this now, it's clear you could save yet another jump by defining a macro for what's in the while loop and putting it at the end of every function call.
That's an additional pointer-chase per loop. And more function prefix / suffix work. In actuality, I suspect that it'd be optimized out - but you cannot say "you cannot do X because it relies on compiler optimizations" and replace that with Y that relies on compiler optimizations.
A function call is nothing more than putting your return address and parameters on the stack and jumping to the address of the function. By referring directly to the function's address, there's no "additional pointer chase", since calling the function already does exactly that.
If you were to inline all of the perform functions in the GOTO version vs putting a macro at the end of each function, you're right that there's some function overhead, but I think it's as small as a single instruction to put the return address on the stack. Maybe that would be optimized away, maybe not.
To your point: My argument isn't that you can't do the GOTO version. With optimizations, it's essentially identical. My point is, that is a lot more hard-to-grok code to maintain for something that can be achieved in a simpler way.
You're forgetting the overhead of popping / pushing registers. Both in terms of direct instruction overhead, and indirectly through code size and working set bloat. Which, especially for smaller functions, can be significant. It's one of the problems with a register-oriented architecture.
Sometimes this can be optimized away, but not always.
~20 years of using various combinations of C and C++, and i never knew you could take the address of a label! reminds me of computed GOSUBs from my basic days :)
Before you use such constructs make sure your team lead or boss is ok with it, it's incompatible with many compilers and ugly to boot. This is the kind of code that gives C its bad reputation.
This is the kind of code which makes OCaml the fastest bytecode interpreter out there, and selection between an ad hoc switch and a comuted goto is done transparently with a single ifdef.
Yes, there may be exceptions when this kind of code is preferable. But it's definitely not the rule and the speed difference between the one and the other is so small that only a profiler can guide you to optimizations like these.
> Yes, there may be exceptions when this kind of code is preferable.
In my practice these exceptions are ubiquitous. Multiple tiny interpreted DSLs (which cannot be compiled more efficiently for the latency reasons), efficient protocols, all that stuff. System programming, in other words, and it's exactly the stuff Rust was supposed to be designed for.
> But it's definitely not the rule and the speed difference between the one and the other is so small
2-4 times difference is not "small" in system-level programming.
2-4 times is a spectacular speed-up, of course the inner loop of an interpreter is important but you're implying the rest of the code-path is only 3-4 instructions long (otherwise you can't get that kind of speed-up) which I find hard to believe. Even the inner loop of an interpreter usually calls routines that are longer than the inner loop itself. Is there a particular benchmark you have in mind for that 2-4 number?
Take a look at the OCaml bytecode interpreter: case bodies are all very small there, so the instruction dispatch time really counts. I got this 2x times difference when comparing this bytecode interpreter built with and without computed goto, but I don't remember any other details at the moment. In my own bytecode interpreters the difference was up to 4x, again, because each bytecode instruction implementation was tiny and trivial (like, move something from one register to another, or perform an arithmetic operation).
> Even the inner loop of an interpreter usually calls routines
These are too high level interpreters for the too dynamic languages, they're beyond any hope in terms of performance, by design. I'm talking about some simpler and better designed things (like OCaml, for example).
That's pretty nifty, but it seems like something a really good compiler could achieve automatically. Of course I'm not sure if any compilers actually are that good.
This is not exactly how fast bytecode interpreters are implemented. There is one next step - pre-caching the jump table straight into the bytecode. And there is absolutely no way compiler can transform your ad hoc switch-based interpreter into a threaded code interpreter automatically - it is not allowed to rewrite your data willy-nilly.
Hadn't thought of that, but it sounds interesting. I guess this means that opcodes in your bytecode need to be large enough to store a pointer, this could involve some tradeoffs in terms of minimizing instructions vs fitting in the cache, but I suppose it could fall on the right side of things often enough.
An offset from the first label, or some nearby alignment point? Makes sense.
Do you have a pointer to the relevant part of OCaml's source? The whole thing is probably worth a read, but don't think I can set aside enough time for that soon, and it could take me a while to find the right part.
This is a fairly mechanical conversion, so suppose a sufficiently smart compiler could do that.
But assuming a sufficiently smart compiler when you depend on your code being fast tends to cause issues. Especially if you expect it to run on a bunch of platforms with compilers of varying qualities. Last time I saw code last this, we certainly couldn't rely on compilers being particularly smart, and we did care about speed.
>In C++ labeled breaks would make 'goto' completely obsolete.
In 20 years of using C/C++, I've found one use for "goto" that's hard to substitute: simulating coroutines that yield to an outer context. (Similar to C# yield return).
A "break label" gets you out of a loop. I needed to use "goto" to jump back into the middle of a loop to resume where the "coroutine" previously left off. The keywords "break/setjmp/longjmp" wouldn't have been substitutes for this particular use case.
I've used a switch statement essentially as a jump table for this purpose. I'm curious whether that would have worked, or if you were doing something different enough that you needed a goto.
This construct gives you most of the power of C++ RAII without the overhead. Except that you can't use it to cleanup resources after exiting an anonymous block—it strictly defers to function exit.
This construct gives you most of the power of C++ RAII without the overhead.
But it leaves out the most useful part: in C++ releasing the resources is completely implicit:
{
std::ifstream in(fn);
// Do something with 'in'
} // Released
while with Go defer, you have to call explicitly for the method that you want to call. If you forget this, you still have a resource leak.
Sure, it's an improvement over languages where you have to cover every possible scope exit, but it definitely doesn't give you 'most of the power' of C++ RAII.
That's not true. The only case where the runtime overhead is unavoidable is calling defer in a loop, because it might require allocating the defer chain in the heap; in all other cases, it's just a metter of writing the correct optimization passes in the compiler.
Yes, that's what I meant by "strictly slower". At best, it can be optimized to something similar to what RAII can give you. RAII never has the overhead of the bad case.
I don't want to be excessively picky, but you said that is has "unavoidable runtime overhead", and that's true only in its rarest form (defer within a loop), which is a feature which you can't implement in RAII. IOW, defer is a superset of RAII.
In all other cases (which is almost all usages), it is semantically equivalent to RAII, so the language doesn't force any runtime overhead. The only difference is that the compiler is less mature than an average C++ compiler, but this is an implementation problem, not a design problem.
RAII has no overhead because it is a pattern designed within the context of a zero-overhead language. defer allows you to implement a superset of cases that RAII handles, including those with runtime overhead.
defer isn't a superset of RAII. defer is function-scoped. RAII is block-scoped. If you want to run code at the end of a block, you can do that with RAII and you can't do that with defer—and note that this is what causes all the codegen issues with defer.
To clarify, I meant the "overhead" of defining and instantiating a container class for RAII purposes.
I hadn't considered the type of overhead you're talking about, which is admittedly more important for the kind of software that tends to get written in C.
There was such a thing (std::finally), but it was removed due to lack of use and maintenance. When your standard library is completely built on RAII, you rarely need it.
I don't think I've written a goto since the 1970s, and I've written a lot of code since then. If you need to bail out of something in the middle, make it a function.
As mentioned above, in C goto is often useful for error handling (where you need to free some resources before exiting the function). It's way more convenient and readable to write freeing code once and jump to it instead of having multiple return exits and duplicate the code before every single one of them.
What the parent said still applies in your case. You just make a wrapper function that does the cleanup. I have used this pattern many times:
static int foo_impl(int arg, int **resource1, int **resource2) {
*resource1 = (int *)malloc(sizeof(**resource1));
if (*resource1 == NULL) return EXIT_FAILURE;
/* Do something with resource1 (omitted)... */
*resource2 = (int *)malloc(sizeof(**resource2));
if (*resource2 == NULL) return EXIT_FAILURE;
/* Do something with resource1 and resource2 (e.g.,
stick them in a global hash table as key/value pairs,
or whatever, omitted)... */
return EXIT_SUCCESS;
}
int foo(int arg) {
int *resource1;
int *resource2;
int ret;
resource1 = resource2 = NULL;
ret = foo_impl(arg, &resource1, &resource2);
if (ret != EXIT_SUCCESS) {
free(resource2);
free(resource1);
}
return ret;
}
The advantages are that it makes it very explicit which resources need to be cleaned up (making it easier to review to be sure you haven't forgotten one, or forgotten to initialize one, because they're in a nice list), and it's harder to screw up the control flow. You can't get out of the function without passing by the clean-up code, and you can't accidentally fall into the clean-up code without explicitly returning an error.
I know it is just an example to illustrate the point but this seems to have a bug in it: resource2 won't be initialized if the function fail to allocate resource1 so resource2 will be null when the code enters the deallocation condition.
A way to fix it would be to return different failures for every allocation and to use a switch without a break to clean up, something like:
switch (ret) {
case EXIT_SUCCESS:
free(resource2);
case EXIT_FAILURE2:
free(resource1);
case EXIT_FAILURE1:
break;
}
EDIT: added potential fix to the code. Sorry if there is any bug in the fix, my C is a little bit ... rusty.
EDIT2: fixed a bug in the fix. C is hard, let's go shopping.
EDIT3: the explanation for the bug was also wrong. It is not a memory leak for but freeing a null pointer. Fixed too.
I find myself often having the issue where the code digging through what can and often is, garbage. At some point you find it's pointless to continue. You are done. Fin. Goto error; in that case makes absolutely perfect sense.
Two things about goto and jmp instructions. Most people have no idea how badly they were abused back in old days. For instance to jump into the middle of a subroutine. Yay just saved 9!!! words of core memory!!! And most people forget that old computer scientists were obsessed with creating grammars that you could write formal proofs for.
I've used goto in very specific cases which basically boils down to breaking from inner loops to the outside of the outer loop. Thankfully Rust actually has labeled break and continue statements, so this case of using goto is taken care of.
> Since then, I have always thought languages should have a "goto" statement that human-written code is not allowed to use.
It's an interesting idea.
The problem I see is that users can get around it by wrapping goto in the simplest possible macro. Then the idea has backfired: we now have goto under as many names as there are goto-using programmers. :)
> I'm sorry to be saying all this, and it is with a heavy heart, but we
tried and did not find a way to make the tradeoffs associated with them
sum up to an argument for inclusion in rust.
Note that a lot has changed since April of 2013. Rust does have TCO, we just can't guarantee it. And, LLVM has come a long way, so we probably can support guaranteed TCO now, and have 'become' as a reserved keyword for this purpose.
I praise the idea of making TCO explicit and guaranteed, but wonder if whole new keyword is necessary? It could be avoided by modifying `return` with another keyword which is currently impossible at that place, for example
return as foobar(x, n-1)
return in foobar(x, n-1)
override return foobar(x, n-1)
final return foobar(x, n-1)
It's a little surprising to see entirely new construct invented for something that is just a different implementation of `return`, after all, the end result of computation with tail-return is the same as with normal return, only performance differs.
I really wish Rust/stdlib had been built around a good async io story. Right now it just feels neglected "because the crates ecosystem can deal with it".
I might be wrong here but it always seemed to me that Rust was built to replace C/C++ in critical infrastructure like Firefox, Nginx, Redis, etc. Basically critical network dependent infrastructure.
With respect (because I understand the difficulties that come with time constraints/resourcing/building an efficient async API), currently I'm not sure how Rust expects itself to be a viable replacement (let alone the best replacement) language for any of those types of applications.
Well, and again, it doesn't implement the entire web platform, but Servo is already showing significant speed gains, even without AIO. It's really more useful for servers than clients.
It's important to remember that IO is truly a library concern in Rust. 1.0 means the language is stable, but there's still tons of libraries to build on top of that language. Holding back the language itself for a library that, while important, is only needed for certain applications wouldn't make a whole lot of sense.
In this particular context I believe using Servo as an argument is misleading. And actually its not just about "doesn't implement the entire web platform". Servo gains all of its performance wins from parallel rendering. A very effective but entirely orthogonal optimization. Just because one optimization far outweighs the performance wins of another optimization doesn't mean the other should be neglected entirely. Also, Servo's benchmarks run a relatively light workloads, i.e. one page at a time.
The argument might be similar to using benchmarks for a parallel web rendering engine (a Servo-like) in Go. It would probably have similar median latencies to Servo. And Go may even scale out better if tabs were sandboxed in goroutines instead of processes. However, ofcourse unlike Servo it would have a higher variance/p90/p99 latencies because of the GC. P.S. I'm not a big fan of Go I was just using it to try and bolster my argument that Servo's current benchmarks shouldn't be used as an argument against async io prioritization. Actually, side note, I would love to hear more flaws about building a Servo-like in Go.
The absence of async IO in the standard library in Rust 1.0 is probably an artefact of Rust's original intention to use M:N green threads and blocking IO based on libuv:
A requirement to stick #version 1.0 at the top of code files so we know which version they were intended to be compiled with, such that in future we can introduce breaking changes into version 2.0 and still have support for compiling legacy 1.0 applications.
Seriously, nearly all data storage formats we use today have some kind of version number in them - why are we treating code as dumb text rather than interesting data?
The ability to provide a default implementation of a function in a trait that can be inherited and used by "classes" that extend that trait.
In C++, this would look like:
#include <iostream>
class B { public: int get_id() { return id; }; int id; };
class C : public B { };
int main() { C c; std::cout << c.get_id() << "\n"; return 0;}
Maybe I'm ignorant, and there's an easy way to do this. I was screwing around with Rust a few months before 1.0 and didn't see any mention of it in the docs, though. (Everything I saw required me to provide an impl for functions defined in an interface for every class that implemented that interface.)
> The ability to provide a default implementation of a function in a trait that can be inherited and used by "classes" that extend that trait.
Rust provides this. E.g., the "talk" method in the "Animal" trait below: [0]
trait Animal {
// Static method signature; `Self` refers to the implementor type
fn new(name: &'static str) -> Self;
// Instance methods, only signatures
fn name(&self) -> &'static str;
fn noise(&self) -> &'static str;
// A trait can provide default method definitions
fn talk(&self) {
// These definitions can access other methods declared in the same
// trait
println!("{} says {}", self.name(), self.noise());
}
}
Man. The documentation did not make that clear at all. Was this added in 1.0?
A question (In C++ syntax, as I'm not a Rust programmer): How would one call Animal::talk inside Dog::talk? The obvious thing (commenting out the println and adding Animal::talk(self) ) causes infinite recursion. The other vaguely obvious thing ( Animal.talk(self); ) is a syntax error, which makes sense.
Edit: I'm referring to the code at the linked Rust By Example page.
`Animal::talk(self)` is shorthand for `<Self as Animal>::talk(self)` - i.e. it calls the overridden method. If you want to reuse the supertrait method, you have to implement it as a generic method outside of the trait, and call it from the default implementation:
There is no such thing as `Animal::talk`. It does not exist. `Animal::talk` is purely shorthand for `<_ as Animal>::talk`, meaning that it is a specific type’s implementation of the `talk` method. In the case of `Animal::talk(dog)` where `dog` is of type `&Dog`, the `_` can be inferred to be `Dog`, and so `Animal::talk(dog)` is equivalent to `Dog::talk(dog)` and `<Dog as Animal>::talk(dog)`.
The default implementation, if overridden, does not exist for the given type.
Incremental compilation and a non-braindead module system. Breaking changes will be required to fix both, which is why Rust was released too early if you ask me.
I haven't looked at Rust since I came to these conclusions a few months ago, so add a grain of salt.
I don't recall why incremental compilation requires a breaking change, maybe it doesn't. I shouldn't have looped it into that statement without being certain.
The module system _is_ braindead, though. Last time I brought this up, I made a proof of concept that compiled the exact same program two ways - one by using the module system, and one by running the code through the C preprocessor and literally #include-ing other rust files. The purpose of this PoC was to point out that Rust modules are functionally identical to #include-ing C files would be in C, which is a well known antipattern.
Rust is very close to C when you consider the guts of the toolchain. C has solved several problems with respect to linking and I feel like Rust could have taken several more hints from C. This is probably a consequence of the Rust devs inherently disliking C and wanting to distance themselves from it.
> The purpose of this PoC was to point out that Rust modules are functionally identical to #include-ing C files would be in C, which is a well known antipattern.
But they trivially aren't.
lib.rs:
mod foo;
mod bar;
foo.rs:
fn f() {}
bar.rs:
fn f() {}
No name conflict. foo::f and bar::f happily coexist.
> Rust is very close to C when you consider the guts of the toolchain. C has solved several problems with respect to linking and I feel like Rust could have taken several more hints from C. This is probably a consequence of the Rust devs inherently disliking C and wanting to distance themselves from it.
No, it's that header files are are a big problem in C (DRY violation, hostile to code inlining, slow compilation) and it was felt that a real module system would be an improvement.
Which is barely enough to say that Rust is hugely different than just #include-ing C files.
I was talking more about linking objects incrementally and the consequences of that design, rather than singing the praises of headers (though I do rather like headers). I understand C from the compiler's perspective as well, having written my own linker and assembler from scratch myself, and I really appreciate the elegance of the design.
>slow compilation
That's objectively untrue. It's much faster to compile with something like headers.
As far as inlining and DRY are concerned, back before Rust shipped I spoke with many Rust maintainers about solutions to all of these problems, but it was dismissed because "we're trying to ship". Maybe you shouldn't sail a boat when you need to replace the hull later?
>
Which is barely enough to say that Rust is hugely different than just #include-ing C files.
Well, sure, if you want to get fancy enough you can make a module system out of #include. (Your code snippet isn't enough because it doesn't replicate privacy or imports.) But replicating something approximating Rust's module system with "#include plus other stuff" doesn't show that Rust's module system is "just #include".
> That's objectively untrue. It's much faster to compile with something like headers.
I don't think that's true once you have the proper incremental build setup. With headers, you have to parse large source files over and over again. With a proper module system, the compiler can use a more efficient binary database format (with an index).
For example, consider math.h. If your program is using one function (say, sin) from math.h, you have to parse all the prototypes in math.h. (And if you have inlined functions or templates in the header files, you have to parse those too!) But with a module system, the compiler can serialize an index of all the signatures of functions inside libmath.so, so the compiler can do a direct, O(1) hash table lookup for "sin".
We aren't there today, of course, since we don't have incremental compilation, and C is certainly simpler, but I think doing it right from the start will pay dividends down the road.
> As far as inlining and DRY are concerned, back before Rust shipped I spoke with many Rust maintainers about solutions to all of these problems, but it was dismissed because "we're trying to ship". Maybe you shouldn't sail a boat when you need to replace the hull later?
I don't see anything backwards-incompatible about incremental compilation, and I believe where we're going will end up better than header files when all is said and done.
> For example, consider math.h. If your program is using one function (say, sin) from math.h, you have to parse all the prototypes in math.h. (And if you have inlined functions or templates in the header files, you have to parse those too!)
Isn't this pretty much a solved problem with precompiled / pre tokenized headers? PTH are language / arch / compiler agnostic.
Precompiled headers inch C and C++ closer to a module system, by discarding the traditional notion of what a header is: it becomes a sort of binary metadata instead of a textually included file. But why not just start with the right thing in the first place?
> That said, from a user’s perspective, I don’t think this is any reason to
> despair: the feature is still likely to become available to users in the
> 2017 timeframe, even if it’s in the form of a TS.
This is Rust's approach to splitting a crate between files - not for splitting to different isolated separately-compiled modules. I must say it works great for that purpose, being much better than dealing with build tools.
This is true. I guess you could say it shipped and is still shipping with many unstable APIs because they so cautious about shipping things they can't take back.
I don't see that as a bad thing though, it's not great if you are trying to write Rust programs/libraries -right- now as sometimes you will have to jump through some hoops to only use stable Rust APIs but it will be worth it in the long term.
Or to continue the train of thought, what about things that it did ship with that could have been excluded? :)
Personally I'm starting to dislike index operations since they can panic (and are the shortest way to access), and I rather use the explicit Option based APIs. Though I'm not too worried about those, since a lint disallowing them shouldn't be too hard.
M:N threads didn't work for Rust, and they don't have that many advantages anyway even in languages where they do work. There has been a lot of discussion on this over the years and this has been the conclusion everyone came to.
> they don't have that many advantages anyway even in languages where they do work.
Are you sure about that? One of the main advantages of coroutines/greenlets IMO is writing simple and straightforward blocking code (e.g. an echo server); without them, you either need to use threads (which are slower and much more heavyweight) orcallbacks or related constructs (async/await, futures, ...).
Threads aren't that slow on Linux. The main advantage of M:N threading as implemented in Go over 1:1 is that spawning is fast and doesn't use much memory, because you can avoid the syscall and only a small (initial) stack is required. Rust can't do the latter because it's not GC'd.
Even if it could, many real-world servers actually do non-trivial work in their threads, so the cost of spawning a thread is dwarfed by the actual work the thread ends up doing. There are serious drawbacks to M:N: complexity, fairness, problems in interoperability with the 1:1 world (including essentially unavoidable performance problems with the FFI), etc.
The path forward is likely going to involve adding a way to build cheap state machines (call them generators or async/await) with a clean syntax and giving mio hundreds of thousands of reusable instances.
I don't understand, and the link seems unclear. Perhaps a more direct question: I get a request X, and I need to consult a backend service to answer the request. Do I write synchronous code calling that backend? Or do I have some callback mechanism?
> ... generators or async/await
Ah. This perhaps answers my question. Both of these are essentially compiler-written callbacks.
If this is going to be like C#, then I presume there will be a thread-pool where user code will execute. It seems like a non-ideal story for concurrency. Users will have to take inordinate care not to call any blocking code; otherwise they will prevent one of the threads in the pool from doing useful work.
> It seems like a non-ideal story for concurrency. Users will have to take inordinate care not to call any blocking code; otherwise they will prevent one of the threads in the pool from doing useful work.
The downsides of going M:N are worse. The cgo-like FFI performance problems, for example, are killer for Rust's use case.
Most applications right now should do thread-per-request. Thread spawning is very optimized in both Rust and the Linux kernel, and you can adjust stack sizes if you need to. If you're hitting limits caused by this, you can use mio.
Mac OS X is rarely used for servers, so I'm not particularly concerned about it. On Windows you can use user-mode scheduling—I would like to see a library for this—which is effectively 1:1.
Funny, co-routines have been recently added to C++ and they work wonders. D has fibers which are used extensively and once again, it's a highly desired feature. Go is kind of the poster boy for coroutines and I doubt anyone claims that it doesn't provide many advantages.
To be honest it seems to me like your explanation is an attempt to downplay just how nice fibers/coroutines are rather than acknowledge their utility in many existing languages.
No, his explanation is the few-line summary of years of failed experiments in userspace M:N threading.
Userspace is not equipped to make reasonable scheduling decisions that provide any significant performance advantage, and library/language runtime control of M thread register/stack contexts on top of N kernel threads plays absolute havoc with most operating system's standard libraries.
Go works around this by explicitly not calling into libc et al -- all system calls are issued directly. One big problem with that: directly invoking syscalls is supported on Linux, but NOT supported on OS X.
End result is that Go literally must rely on undefined behavior on any system that does not support direct issuing of syscalls.
From my brief review just now, what MS appears to be proposing for C++17 isn't coroutines in the traditional M:N threading sense, but rather, an explicit mechanism (with syntactical sugar) for capturing reachable variables in a lambda (without preserving the stack), and issuing a call to that magicked-up lambda later via promises.
This is interesting if you love the idea imperative mutable promise-based concurrency, but it's not likely to win you any performance gains, and it's useless in the extreme if imperative mutable promises aren't your cup of tea.
You are correct the current proposed C++ coroutines are vastly different then M:N userspace threading. The allocation differences are drastic, and unlike Go they play nicely with system libraries.
M:N threading may make sense for a higher-level language implementation like Erlang or GHC where you have a runtime in charge of things anyway and can reap benefits with initial stack sizes substantially smaller than a single VM page, allowing creation of millions of threads.
But since Rust is intended for runtime-less lower-level programming it doesn't make any sense here.
Non-lexical scope won't help with that. Remember that Rust does not allow mutable aliasing, so this code can never be allowed:
let mut arr = [1, 2];
let a = &mut arr[0];
let b = &mut arr[0];
This code could be allowed:
let mut arr = [1, 2];
let a = &mut arr[0];
let b = &mut arr[1];
And it gets Hard once variable indexes are involved. And since the whole point of using arrays is to get variable indexing, the Rust developers choose to only implement reborrowing for structs and tuples.
Currently, lifetimes are based entirely on lexical scope. So for example, this doesn't work:
fn main() {
let mut x = 10;
let y = &mut x;
*y = 11;
println!("{}", x);
}
This will complain
error: cannot borrow `x` as immutable because it is also borrowed as mutable
This is because an `&mut` borrow is exclusive: while `y` is alive, we cannot use `x`. We can fix this by making a new scope for `y`:
fn main() {
let mut x = 10;
{
let y = &mut x;
*y = 11;
}
println!("{}", x);
}
This works, and will print `11`.
Non-lexical lifetimes would allow the compiler to demonstrate that these two things are the same, and allow the first one to compile with the behavior of the second.
It's an interesting tradeoff, because right now, the rules are very simple and conservative. Scope is fairly easy to reason about. Non-lexical lifetimes would make certain things easier, but also a bit harder to reason about, because the rules are more complex.
Could you elaborate on or link to a more detailed tradeoff?
I want memory safety with as few hazzle as possible and that Rust doesn't understand the safety of the first example means hazzle. I expect the compiler to try to understand even if it's ’hard’. It doesn't need to understand everything but the mentioned situtation should be doable.
> Could you elaborate on or link to a more detailed tradeoff?
I'm not sure what you mean by a 'more detailed tradeoff.' You mean a more complicated example?
> I expect the compiler to try to understand even if it's ’hard’.
It's not a matter of difficulty, exactly, it's a matter of how easy it is to understand what the compiler is doing. Figuring out non-lexical scopes means that my mental model of what the compiler is doing is more difficult than it is right now, which may or may not be the right tradeoff. I would say that most people want non-lexical lifetimes/SEME regions to be implemented, though.
"
Next time you're in a conversation about language design and someone sighs, shakes their head and tells you that sad legacy design choices are just the burden of the past and we're helpless to avoid repeating them, try to remember that this is not so."
I think rust is neat, but this is somewhat arrogant.
Rust is amazingly young. 10-20 years from now, when rust hopefully has bajillions of users, if this is still true, then you can say it.
I mean, do you really believe that Rust won't have things that turn out to be warts from 1.0 it can't remove 10-20 years from now?
I think you're misreading this. I read it as saying that these things are sad design choices in many similar languages, but that Rust has avoided them. Nothing follows about them not having included other sad design choices that they're unaware of.
The point is that you can at least try to avoid the things you view as sad design choices--you're not necessarily stuck with them.
I used Common Lisp's goto (tagbody with go) to implement portable tail recursion: a "tlet" construct that looks like the "labels" syntax for defining a local function, but is just a stackless goto thunk with parameters.
This also provides some facilities for doing cross-module tail recursion among top-level functions. Here, continuation to the next function is provided by wrapping the function call in a closure, performing a non-local exit which abandons stack frames up to a dispatch loop, which then invokes the closure.
I'd like Rust to be shipped without counterintuitive standard library function names and that book with all its style 'recommendations'. And I'd like the Rust compiler to be shipped without that non-snake case warning enabled by default.
Language creators won't endear themselves to me by ranting. The problem I have with Rust is _not_ the language itself.
This is the first I've heard complaints that Rust is too strict with formatting. If anything, the popularity of gofmt (which is far more opinionated than rustc's default warnings are!) is a testament to the fact that people want languages to be ruthlessly opinionated regarding style these days.
It's so much better to let the computer worry about formatting such that the programmer can worry about the logic.
fmt-tools offer the amazing possibility that one programmer writes and reads the same code with different formatting than another programmer. I'd love to be able to set my formatting in my editor so that I see it how it's best for me but on saving or sharing code the formatting is reverted to the standard.
So, I think there should be a default style for rustfmt, but also support for other styles.
I'm not working on rustfmt but my understanding is that its output is going to be configurable.
Every language carries with it a culture. The culture of Go is one which allows for gofmt to define one true style and refuse to deviate, just as it allows the language designers to refrain from adding generics.
The culture of Rust is not like that, for several reasons. For one, the Rust community loves a good bikeshed. For two, the syntax of Rust is more complicated than Go, and includes situations (match statements & where clauses come to mind) in which people are just going to want to different things.
I know the advantages of one true style - everyone's heard the arguments - and there's a sane, median default as the official style guide, which will be rustfmt's default output. That seems like a good compromise.
It seems you are misunderstanding the situation because you are talking about not making “it” an option and you are then reiterating what I also said.
Even if I repeat myself: I think there should be one default formatting that is standardized and there should be the option to emit in other formats such that everyone can read in the individually preferred format.
With fmt you don't need to establish formatting rules on a project basis, anymore. Everybody can just configure their editor to format the code how they want it to look. That is why I think rustfmt should be compilable as a library, too.
That's absolutely OK and I do understand that people want that. I actually like snake case and only slightly prefer camel case. I'd write snake case in large open source projects, if it's actually preferred. But it's very awful to type snake case code on my keyboard and I'm not ready yet to switch to a US keyboard. The worst thing about that warning is that I get some kind of a bad conscience by disabling it.
> If anything, the popularity of gofmt (which is far more opinionated than rustc's default warnings are!) is a testament to the fact that people want languages to be ruthlessly opinionated regarding style these days.
Its testament to the fact that there are some people that want that; that's very different from that being what people in general want.
The idea that "[u]sing a return as the last line of a function works, but is considered poor style" irks me so much. A lot of what I find appealing about Rust is that it makes so much explicit through its type system, so I don't understand the philosophy behind preferring implicit returns, especially since you could have a scenario where someone hasn't finished writing a function but it still compiles without error.
Implicit returns encourage functional style; foo.map(|x| x + 1) is so much nicer than foo.map(|x| { return x + 1; }). Once you have implicit returns in closures, you might as well have them everywhere for consistency.
That's true, but they're so syntactically different that I can't imagine a scenario where someone is asking whether it's closures or functions that you're supposed to use return statements in.
A foolish consistency is the hobgoblin of little minds. Sometimes a variation in rules allows for more clarity -- one could set their editor to make return a different color, for example, making it easier to glance at program flow. Then again, I'd argue that closures should allow returns too, like they do in C#. I'm an adamant supporter of the explicit camp -- when you have to debug things at 3 in the morning, sometimes that extra little bit of context can make things blindingly obvious.
I find the way it works in scala very nice - a block is just an expression that evaluates to the last statement in the block, and idiomatic code never uses "return" anywhere. Even e.g. a function definition, you can replace the block with a single expression if it's more convenient. I agree with being explicit but I don't think return actually makes things any more explicit (if you're talking about an editor highlighting, the editor knows which line is the value of the block, with or without a "return") - rather it's just syntactic ceremony. It's surprising how much difference having very low-overhead closures makes - you can make so much of your program simpler, because it's not a problem if a caller needs to make a slight modification to something.
> foo.map(|x| x + 1) is so much nicer than foo.map(|x| { return x + 1; })
"Nicer" is a subjective thing.
BTW in the trivial case above one may judge this or that to be nicer, but in a large method, 'return func(a,b,c)' is obvious, whereas looking at 'func(a,b,c)' it is super non-obvious that the value is being returned.
It's not at all non-obvious if you learn the language. It's also not non-obvious if you learn any of the myriad other languages with the same semantics (including popular web languages like Ruby and CoffeeScript).
compromise. Make return very short, like a unary colon ":".
Then "foo.map(|x| :x+1)" is nice. If that doesn't stand out enough for some people on a line of its own, make your editor render the unary colon line in a very bold color for you.
The most compelling argument I've seen is that you don't explicitly `break` at the final run of a loop, either. An explicit `break` or `return` in the middle of a body can be thought as "abnormal" control flow. Otherwise, normal control flow will take place. As for a function, it would be returning the last expression.
That's what I mean. Why should I even care about stuff like this being "considered poor style" by the creators of Rust? "Poor style" is what doesn't work well for my team and me.
> Why should I even care about stuff like this being "considered poor style" by the creators of Rust?
Maybe you are writing code for the Rust standard library, where following the core projects style recommendations would be important for consistency.
Maybe you don't want to start from scratch coming up with your own style, and want a decent starting point from which you can vary as your team figures out what does/doesn't work for them.
Lots of reasons to have a language project also provide default style recommendations.
Style recommendations are great and if I was writing code for the standard library, I would certainly follow them. But they don't simply recommend their style. "We recommend doing this like that" is very different from "Doing this that way is considered poor style". That sounds like their recommendations were objective facts and you shouldn't do anything else, even if that worked better for you and the majority of other programmers.
> That sounds like their recommendations were objective facts
On the one hand, it is an objective fact that it is considered, by the authors, poor style.
On the other hand, that it is "considered" anything is an explicit (not merely implicit) statement that it is subjective (and "poor style" -- or good style, for that matter is inherently subjective in any case.) So, characterizing that language as making it sound "like their recommendations are objective facts" seems quite bizarre.
You can say that it can be an objective fact, though, that the majority of 'good developers' considers all that stuff poor style. Which is what I get from reading that book. If they don't intend to imply that, they should write it in a different way. The authors of various other books on various other programming languages are able to do that.
Yeah, that's one thing that's really putting me off looking at Rust - the possibility of it enforcing style in the future.
Hanging braces style for C/C++ is rarely allowed in the coding standards I've had to use in the past for embedded and real-time stuff in the defence industry, because it can be a source of errors.
Aligned opening and closing braces are much more common (in line with ADA's style).
Come on, even that flag's name is ridiculous. They basically say, for example, that writing camel case, which is the preferred style in most of the projects I've seen, is bad style. Do they like being unnecessarily antipathetic? People think about their personal preferred style for years, and they have very good reasons for choosing them. And in _many_ cases, it's exactly the style that's called bad by Rust's creators.
How about making it the opposite, '#![allow(default_style)]'? Or at least '#![allow(non_default_style)]'?
In some cases skirting style can be harmful (e.g. in match statements there's an ambiguity with enum variants and variable bindings that is not an issue because of the style lints).
But I get your point. I think people would be open to that naming change if you file an issue.
The compiler will not enforce more style in the future, that will be done by Rustfmt, a separate and optional tool. In fact, it is likely that the compiler lints which enforce style will move out of the compiler and in to Rustfmt at some point in the future.
I like the warnings about style and naming conventions. I kinda wish there were more of them. These warnings can help teams avoid arguments about things that don't really matter very much.
Given the frequency of manipulating large bodies of text in contemporary programming I do not agree it is best left to a library implementation.
Without ropes as core std average rust developers will do what they did in java, c++, objC and simply use and abuse std::strings in all cases including those where it will perform poorly.
Can we please get a quote on this one. If reinterpreting memory via a union is valid in C99, including data vs function pointers, then so would reinterpreting that memory via a cast, which would seem to violate one of the most elementary aspects of the standard (e.g. such a rule would be difficult or impossible to implement on a Harvard architecture machine, which the standard previously made plenty of allowance for)
Page 83, section §6.5.2.3 of the C11 standard, footnote 95) says
> If the member used to read the contents of a union object is not the same as the member last used to
store a value in the object, the appropriate part of the object representation of the value is reinterpreted
as an object representation in the new type as described in 6.2.6 (a process sometimes called ‘‘type
punning’’). This might be a trap representation
The C standard does not allow casting directly between function pointers and other pointers; they might not even be the same size. Using a union or memcpy will allow you to standards-compatibly reinterpret the bit pattern of a function pointer as a data pointer or vice versa (modulo size differences), but creating the resulting pointer, even without dereferencing it, might cause a crash if the bit pattern is a "trap representation", and in any case dereferencing it isn't guaranteed to do anything useful. ...Not that a portable program has any business trying to read instruction opcodes in the first place.
Apparently that is correct and I was wrong above (though not in C++) [1]. However, according to the standard, the actual value is unspecified, because float bit representation is undefined.
Going through a union is the correct way of viewing one type as another. Casting pointers to accomplish this is undefined behaviour (except for char *) and compilers do TBBA (type-based alias analysis) based on the assumption you don't do it. Famously, the Linux kernel does this a lot and needs to be compiled with -fno-strict-aliasing.
That hasn't been an optimization for 15 years now. Fast approximate inverse square root is a builtin operation on most FPUs-- on x86, it's RSQRTSS, which tends to be the same cost as a floating point multiply. (for Haswell, it has a 5 cycle latency, and 1 cycle reciprocal throughput)
Interesting, though not useful unless there's a good way to access it from portable C/C++. An __asm__ section or equivalent would work, but would only be useful on a particular CPU -- and different compilers use different syntax for embedding assembly, so that's not ideal either.
But where I'd more likely use that optimization at this point is on Arm or ATmega processors. ARM doesn't seem to have an approximate inverse square root, based on a quick check, and ATmega are frequently still stuck with software floating point, so I'd hardly say that the optimization is dead.
The first is bad because of UTF-8, basically. It's unclear what an index should even return: bytes, codepoints, grapheme clusters? Furthermore, because it's a variable-length encoding, it's not O(1) access, which is what [] implies, and since Rust tries to surface the cost of operations, it would be inappropriate for Rust.
(Note that Rust _does_ have a ranged syntax here, which returns bytes, and is O(1))
Pre vs post increment just leads to a lot of confusion, and doesn't give you much over `x += 1`, in my opinion. Not sure what Graydon's reasoning is here.
Just as additional context, since this seems to come up a lot - that does not mean that UTF-8 is bad. The main alternative is UTF-32, which provides random access to codepoints rather than bytes, but since Unicode graphemes consist of multiple codepoints, UTF-32 is effectively a variable width encoding as well for most purposes, just one that uses much more memory than UTF-8 most of the time and never uses less.
Ok, thanks for the details, I suspected as much.
My intuition says string indices should be accessible as grapheme clusters (and, additionally, codepoints), bytes are meaningless anyway. Whenever I want to cut and slice a string, I usually want it to be at the boundary of a letter. Yet most languages don't seem to do it this way, strangely enough.
If you want to be able to slice at grapheme boundary n, you have to iterate through the string counting up to n to find out where it is. Might as well just provide an iterator over graphemes in the first place.
You can optimize slicing on codepoint boundaries by storing strings internally in a fixed-width representation, like Python does. But now you have to convert encodings all the time just to use UTF-8 for your I/O. And that still only gets you codepoints.
It's not a great tradeoff to do it that way. It wastes memory and CPU time, and randomly accessing characters of strings is just not something you actually need to do often enough to optimize for it.
And UTF-16 random-access strings (like in Java and many other things designed in the 90s) make the problem even worse, because with UTF-8 you will notice the bugs as soon as the first non-English-speaker uses your system, but with UTF-16 the affected languages are a lot more obscure.
Go has increment operators, but they have to be in their own statements, you can't intermingle them in an expression. The reason is, putting them in the same line as other stuff makes for a higher chance of bugs. For example, Tarsnap's CTR bug happened because of this. The removal of a nonce increment would have been very obvious if all increments had to be on a separate line, either by language fiat or properly risk-averse C coding practices.
If you're making a new language, that example could go a couple of ways:
1. It could be a compiler error. The existence of certain operators doesn't mean they can be combined arbitrarily.
2. Sequence points could be defined differently, allowing for such a convoluted line. This would likely hurt performance, as the compiler would have fewer operations to reorder in each sequence point.
Typically, increment and decrement are just syntactic sugar. I don't mind if a language lacks them, but I can see why people like them. A convoluted example of UB in C++ is not a good argument against having them.
Also: In C++, overloaded operators are function calls, and function calls are sequence points. So if i is an object, the behavior is defined. That said, it would still be bad code. :)
I am not sure about if 2 actually would hurt performance in the real world. Remember the whole "must appear to be in order" thing. If the compiler knows that it doesn't change things if they are out of order, it's free to rearrange them.
Note: things like this are part of the potential advantages of static linking.
This could actually work fine in Rust, because it has deterministic evaluation ordering for everything (or almost everything, though I can't think of anything that isn't deterministic).
Knowing that `++i` is `{ i += 1; i }`, we have `i += { i += 1; i };` (which should compile already).
The nested assignment can be hoisted to obtain `i += 1; i += i;`.
That means Rust would compile `i += ++i;` as `i = (i + 1) * 2;`.
Well, it did, but these are things that are good _not_ to have, not things that would be nice to have.
If those are things you do want to have, well, we didn't have a design for AIO that we were happy yet, as even the most advanced Rust library, mio, is still a work in progress. https://github.com/rust-lang/rfcs/issues/1081
Rust also shipped without reflection, which was available in the beginning. I was a bit disappointed as it was something I would have made use of for serialization.
Serialization is usually done better with macros, for performance reasons. We now have Erick Tryzelaar's fantastic serde library for high-quality serialization, competitive with rapidjson.
Oh, serde is interesting. I was specifically thinking about compile-time reflection but apparently rust's macros are more powerful than I realized. I'm inspired to look into this more, thanks!
Serde (https://github.com/serde-rs/serde) is my baby, so let me know if you need any help with it. In a sense serde is a compile time reflection library. It's designed to recursively walk through a structure while calling a function on each component. While the primary purpose is to do serialization, it could be used to do other reflection-y things, like build a UI widget for arbitrary structures. To really support full reflection though serde would have to be modified to support in-place modification. It wouldn't trivial, but it's certainly not impossible.
Wouldn't it be better to create serialization functionality at compile-time?
In the Java space some libraries are moving to compile-time code generation instead of relying on reflection. It is a huge win, since a lot more can be checked beforehand.
Dagger 2 is a good example how it can be beneficial. It provides dependency injection at compile-time, which will in turn check whether all dependencies are satisfied. I haven't seen this being done at compile-time before, but it is definitely a step up from reflection-based DI.
I'm not sure whether macros of Rust can provide such functionality, but the developers seem conservative when it comes to adding functionality. That seems like a good thing.
I was referencing at compile time, yes. How does Dagger generate the code? Does it run an executable first?
Yes, in C++ there is a similar library called ROOT which generates c++ files called "dictionaries" storing class information by running an executable over the files. I don't see (or understand) the downsides in providing the functionality for performing those steps at compile time though. The developers of ROOT are currently pushing for it to be included in C++17 (or beyond).
Sometime I wish no runtime reflection was available to the programmer at runtime in Java, this would prevent people from being tempted to use brittle runtime magic. Of course, you would need a solid macro system instead.
And thanks for the pointer to dagger, this looks like an interesting alternative.
It's not possible, but there's also no good reason to do it. You might as well just make an e.g. Serializable trait that requires you to manually implement the serialize function. You're not saving a whole lot of work by looping through the object's fields and saving them. Most objects only have a few fields to serialize anyway.
Well, you generally don't, because Duff's Device isn't generally regarded as a good idea anymore, in my understanding:
> It turns out that with branch predictions and the relative speed of CPU
> vs. memory changing over the past decade, loop unrolling is pretty much
> pointless.
https://github.com/pythonesque/fallthrough can emulate the fallthrough style if you really, really want to have it. It seems like it hasn't been updated in a while, though.
It's interesting, but a loop makes for smaller code, by a couple dozen bytes or so:
int remaining = len % 4;
if (remaining)
{
do
{
remaining--;
h ^= (data[(length & ~3) + remaining] & 0xff) << (remaining * 8);
}
while(remaining);
h *= m;
}
Fall-through's interesting, but at the same time, as architectures have changed, has become less useful. Self-modifying code at one time was near vital, but has fallen by the wayside, fall-through is doing much the same.
Usually I would just factor out the guts of the expression into a little closure in that case. (let f = |x| h ^= ... x ...) LLVM should inline it just fine, and it'll save you typing.
If loops can have specialized control-flow keywords (break; continue), why not let match have one (fallthrough)?
match x {
0 => { /* do some stuff */; fall_through; },
1 => true,
_ => false
}
However, I don't know yet how useful it would be. I can't remember ever really needing it, so it would probably need a few practical examples before it became a reality but its an idea.
If we really need it I'd rather have C#-style goto-label instead of explicit fall-through, which is strictly less general. (But I'd almost rather it be a tail-duplicating macro to begin with, since the feature is so rarely needed.)
Generally doing anything crafty like this in real-world or production code is just going to cause the next guy who shows up to be very, very confused. Loop unrolling optimizations and the like should really be left to the compiler.
If you're truly sure you're better than the compiler, I'd imagine an assembly language implementation would be easier to understand than nested switch/while fall through madness.
That said. There's an implementation of Duff's device in the Wikipedia entry which (IMO is much easier to understand and maintain) that doesn't use fall-through that should be just fine to write in Rust.
Lack of `goto` is a huge disadvantage for a language with macros. Pity they did not want to provide something like `unsafe` keyword which would enable all the dirty, dangerous, high-performance stuff.
One of the things Servo (the web browser engine written in Rust) has not decided to write from scratch is a (high performance) js engine. If they did, they might have more feedback on this kind of things.
It's an encoding that isn't good at anything: it's neither ASCII-compatible (like UTF-8), nor fixed-length (like UTF-32), but because most characters require only 2 bytes, developers frequently assume that none require more, leading to bugs when a character eventually is represented by 4 bytes.
Utf-32 is only fixed length if you don't care about diacritics, variation selectors, RTL languages, and others. Unicode is not one code point or one char/wchar/uint32 per glyph.
You've changed topic from code points to grapheme clusters. Rust's character/string support is strictly for code points (the documentation is fairly clear about the distinction).
Few string libraries actually deal with grapheme clusters as the native underlying representation (Swift being a notable exception).
The broader point I'm making is that unicode is hard and attempts to simplify it by choosing a different encoding (i.e. switching to utf-32 to save yourself from all problems) are a bit misguided.
UTF-32 is not good for anything either, easy access to codepoints is just as useless as access to UTF-8 bytes. Any meaningful operation on text (even counting number of characters) requires parsing grapheme clusters, which have variable length regardless of what encoding is used.
I don't know much about Rust and Rust library, so I have a question: what if I what to develop Windows only software in Rust, will I need to convert back and forth between UTF-16 and UTF-8 (or whatever Rust uses in other parts of the library)?
The Rust std library had to pick a string encoding, and it picked UTF-8 (which is really the best Unicode encoding). The String type is platform neutral and always UTF-8.
However, it does provide an OsString type, which on windows is UTF-16. Maybe there is a library - and if not, one could be written - targeting Windows only, and implementing stronger UTF-16 string processing on the OsString type.
EDIT: To be clear, Rust's trait system makes this very easy to do. You just define all the methods you want OsString to have in a trait WindowsString, and implement it for OsString, even though OsString is a std library type. One of the great things about Rust is that its trivial to use the std library as shared "pivot" which various third party libraries extend according to your use case.
But there is actually prior art here - Java's contribution to perverse Unicode encodings is called "Modified UTF-8" and encodes every UTF-16 surrogate code unit separately.
Suppose I'm on Linux, but I want to interact with Windows stuff. (CIFS protocol, NTFS on-disk format, disassembler for Windows executables, Wine-like program, cross-compiler, etc.)
I'll be wanting UTF-16 support. Going the other way matters too; if I'm on Windows I may need UCS-32 support.
Sure. That's not a problem. You can write any kind of string type you want, as a library, and convert between them. One of the nice things about Rust is that it's low-level enough that almost everything is a library anyway, so the language won't get in your way if you need SomeNicheString.
Since the full bullet point was “UTF-16 or UCS-2 support anywhere outside windows API compatibility routines” I'm assuming you'd get UTF-8 out of any high-level interface.
Rust is the first language that fixes all of these things, can be used as a replacement for all of C's use cases (including ABI-stable libraries, kernels, etc.), and has a sufficient community / mindshare that you can expect libraries to exist in the language for functionality that you generally expect to find in libraries.
Maybe we should be embarrassed as an industry that it took us 20 years to get there, yes.
Many of the Rust fixes, aren't an issue in Ada, Modula-2, Modula-3, Turbo Pascal, Delphi, Oberon, Oberon-2, Component Pascal, Eiffel, Dylan...
However in those 20 years we saw the rise of UNIX in the industry and with it C and C++, to the point those languages lost mindshare and are only known to those that experienced the IT world before they got widespread.
So new generations have to re-learn system programming without C and C++ way of doing things is possible.
If Rust misses 'goto', it is not suitable for translating SSA into it, which most compilers natually produce (also for LLVM, obviously). I.e., it is not suitable for being used in a compiler backend. Here, it can never fully replace C.
'goto' should not be daemonized, we know better today.
> I don't follow, why would SSA need to be translated into an input language?
Rust is a meta-language with very powerful macros in it.
What is a macro? Macro, essentially, is a compiler. You can have some petty macros implementing tiny syntax sugar on top of your language, that's totally fine, but that's not their purpose.
The real metaprogramming kicks in when you implement very high-level eDSLs on top of your macros. And for this sort of things, having a proper compilation pipeline is a must. Many eDSLs may end up being represented as an SSA. E.g., I'm doing this with a Packrat eDSL - it pays to represent its intermediate language as an SSA for doing some high level optimisations before spitting out the host language code.
> should be the same compiler as that for the rest of the language
Sorry, I do not understand what are you talking about.
Of course it is the same host compiler. But, every macro itself is a small compiler. Sometimes, when your DSL is an elaborate, complex thing, the macro itself is a complicated, big compiler.
With all the bells and whistles of a big compiler - multiple stages, multiple intermediate representations, all that stuff. And SSA is one of the most efficient intermediate representations ever, suitable for a huge number of semantic classes of DSLs. So, making it harder to generate your same host language code out of an intermediate, in-macro representation does not serve any reason, it's plainly conuterproductive.
Any marginally non-trivial macro is a compiler, by definition.
> The macro gets expanded at compile-time into 'normal' source code, by the compiler.
Macro expands a DSL inside it into the underlying host meta-language code. In other words, it compiles this DSL into the host language.
For example, a macro which defines a parser. Inside it is a BNF (or PEG), a very high level language. Macro must compile this language into Rust, and, since it is a very high level language, there are tons of optimisation opportunities that would be totally missed by the underlying Rust and LLVM because they lack this domain-specific knowledge.
Your macro will compile this source DSL in multiple stages (well, because this is the only sane approach to compilation anyway, read about the Nanopass framework for more details). First it will operate on an AST level, do some analysis, error reporting, inlining, may annotate the detected left recursive nodes and binary expression nodes. Then you'd lower it down into the trivial Packrat building blocks - still a tree. But now you notice that there is a lot of redundant reads from the input stream, and if you flatten this tree into an SSA you can optimise them all away.
Alas, after such an optimisation you'd have to promote it back to tree in order to generate Rust, because there is no `goto`.
And this is only one trivial example. In my practice there were dozens such DSLs. For example, same story is with an optimising WAM-based embedded Prolog DSL, multiple querying DSLs, tree walking DSLs (which are essential for implementing macros efficiently).
Given that the reference Rust implementation currently outputs LLVM IR, and LLVM IR is well-suited as a thing to translate compiler output to, I sort of think that you could solve that by just having new compilers use LLVM as the backend, directly. The way to make Rust support a new platform is to make LLVM support it, after all.
(But you're right, "replacement for all of C's use cases" was strictly incorrect. I'd argue that this is a bad use of C, unlike e.g. kernels or bootloaders, but that's a matter of opinion.)
I haven't done this for a while, but once upon a graduate program I wrote a compiler from a made-up-language (MUP) to C. MUP had some strange control structures, and if C did not have "goto", it would have been a lot more difficult to implement those structures. Since then, I have always thought languages should have a "goto" statement that human-written code is not allowed to use. :)