I still don't quite understand why Cloudflare is going their own way rather than joining our Rust implementation (https://github.com/djc/quinn). So far the primary maintainer told me "I quite like how quiche has turned out, and we already started integrating into internal stuff", which I haven't found very satisfying.
Perhaps they just want to strategically own their own stack, but it seems to me aligning with the broader Rust ecosystem and getting features for free from the community would also bring some value.
Initially I thought it was mainly about integrating with BoringSSL rather than ring and rustls, but even the promise of pluggable crypto didn't seem to affect their decision.
Would be nice to have more clarity on this.
(Previously they were happy enough to use Quinn for their Noise experiment, although without crediting our project explicitly in the resulting paper.)
Fundamentally, we wanted an interface to QUIC that we could integrate with our existing server systems. Our servers, naturally, have their own methods of handling a wide range of things like: socket addressing; events state; memory management; TLS handshake processing; request and response processing; and we needed our QUIC implementation to be as independent of these notions as possible. Looking at a library like quinn and some other QUIC implementations, we see that these APIs are tied to many of these external notions, and specific implementations of these notions for that matter, one example being the core protocol objects in quinn that directly link to specific socket and address and even event loop reactor objects - and these objects strictly in Rust to boot.
It was obvious from our experience with our first QUIC proof of concept, and from our evaluation of options for our continued QUIC offerings, that existing solutions available right now would not be able to offer the level of flexibility and abstraction we need, nor would they likely be able to add the interfaces we need in a reasonable timescale, nor would they likely tolerate our requests to do so.
I'm one of the engineers working on QUIC projects at Cloudflare.
This response from jgrahamc just came to my attention. His comments are based on a discussion I had with him about our first QUIC project and then additionally what I knew of quinn. I described how quinn was arranged and that it provided an interface that was oriented towards the tokio IO library. What I seem to have failed to communicate was the IO specific interfaces were not the only interfaces provided by quinn, and that more abstract interfaces do exist. This mistaken impression seems to have been bundled with the multitude of reasons that we did not choose various other QUIC implementations.
I'm sorry that this this mischaracterisation was made against your work, and hope anyone reading this thread does not take away the same impression.
Thank you for openly discussing this. Ralith has mostly responded in terms of the API our quinn-proto offers; it seems like you and/or ghedo may have missed that?
I just also wanted to respond to this:
> nor would they likely be able to add the interfaces we need in a reasonable timescale, nor would they likely tolerate our requests to do so.
I have a pretty different perspective on this: I've offered opportunities for collaboration from the moment ghedo started quiche in his free time, and explicitly offered to make changes to quinn{,-proto} to be able to do the kind of things that you might need when it was clear Cloudflare would be adopting quiche as well.
> the core protocol objects in quinn that directly link to specific socket and address and even event loop reactor objects - and these objects strictly in Rust to boot.
This isn't accurate. The core protocol objects in Quinn are those defined in "quinn-proto", which has absolutely no awareness of how I/O is performed or what event loop is in use: the interface is structurally very similar to Quiche's, with the caller providing data/timeouts by any means and consuming events that result. See https://docs.rs/quinn-proto/0.2.0/quinn_proto/struct.Endpoin... for details.
The high level "quinn" crate interface, which incorporates awareness of UDP sockets and the tokio event loop, is strictly an optional binding; that's why it's a separate crate.
> existing solutions available right now would not be able to offer the level of flexibility and abstraction we need, nor would they likely be able to add the interfaces we need in a reasonable timescale, nor would they likely tolerate our requests to do so.
Support for this type of flexibility, up to and including allowances for a C API suitable for integration with existing software, has actually been an express design goal of Quinn from the start, and I regret that that wasn't clear.
* They may not want to be too dependent on outside maintainers
* It may be a poor fit for the rest of their infrastructure
* They may just not like you
> Would be nice to have more clarity on this
They almost certainly do not want to get into a possible public argument and hounding them for answers will have nothing but the opposite effect. Large companies have their own way of doing things and what you or I think is irrelevant.
We are more than happy to get into why we made quiche but the main developer is away and I'd like him to respond. So, we'll have to punt until he's available.
Disagree. This happened with tokio, the most used crate that provides an execution runtime for futures for async programming, and it's fucking awful to use. But, the community rallied behind it and is now stuck with it for better or for worse. There's been barely any effort to provide any documentation, even recently the main developer pawned it off to be crowdsourced and it really shows.
I used to think that as well, but sometimes different projects have different priorities or philosophies. In that case, you can either be the jerk who complains about free open source code you didn't pay for, try to bend the project to your will and probably fail, or roll your own.
Really we should learn to expect that "I don't support that" is the default unless somebody's _actual_ business is supporting you doing whatever it is. Which it rarely is these days. This obviously matters most for security, but it's true everywhere.
Another security example is that a tremendous amount of stuff ultimately decides whether to trust a certificate based on Mozilla's root trust programme.
That seems fine right? Well, kinda. It's definitely better than "I found this file named CA_TRUST.Z on an old Solaris machine in 1997 and so that'll have to do" as a policy.
But it's different from two things you or end users might expect you're achieving.
1. This isn't "You know, the same Certificate Authorities trusted everywhere". There is no such thing. The big root trust programmes have a fair amount of commonality, which is fine if you're out to obtain a certificate for a web site (pick anyone in the common set) but they aren't even close to mirrors. Mozilla's list is significantly different from Apple's which is likewise different from Microsoft's.
2. This also isn't "Like in Firefox". This problem is more subtle, after all the Firefox build process consumes the same file, ultimately, as your code does. But, Firefox has a long list of extra rules beyond whether or not something is in Mozilla's published trust store and your application would need lots of extra work to implement all those rules and track updates to them. Did you do that work? No you did not.
This probably won't bite you, and likewise relying on bits of BoringSSL that Google doesn't care about probably won't cause massive security problems in your code. But only probably.
We do have a lot of Go but we are not religious about languages as that would be silly. Rust has some interesting properties and compiles readily to WebAssembly. We'll continue to use Go and Rust for projects. We also have lots of Lua, PHP and C++ in production. Plus lots of Python and other languages.
I hope rust becomes more mainstream in companies. At the moment it seems like a lot of smaller components are implemented in rust, to allow employees to get familiar with it.
Rust is mainstream at Cloudflare. It's used for many new projects. New projects naturally start small.
There are millions of lines of existing production code in other languages, but they won't get rewritten. Even if they would be better in Rust, there's a high opportunity cost of rewriting existing code that works, instead of spending the effort on adding value elsewhere.
Perhaps they just want to strategically own their own stack, but it seems to me aligning with the broader Rust ecosystem and getting features for free from the community would also bring some value.
Initially I thought it was mainly about integrating with BoringSSL rather than ring and rustls, but even the promise of pluggable crypto didn't seem to affect their decision.
Would be nice to have more clarity on this.
(Previously they were happy enough to use Quinn for their Noise experiment, although without crediting our project explicitly in the resulting paper.)