Which I'd quite bad. I don't understand why/who lobbied to implement it that way in Firefox. Can someone disclose the decision-making behind this? It would help devs a lot if HTTP/2 works without TLS too.
All the main browsers want to encourage HTTPS everywhere. They're taking a number of initiatives to encourage it, some are carrots, some are sticks.
Mostly it's ending up as the stick, eg "You can only get these major performance advantages over HTTPS". Soon there is an intent to specifically call out non-HTTPS content very prominently in the browser, even more so that HTTPS content was ever called out back when that was introduced.
Yes, exactly. But why? What's the agenda? Why now? It's not like Amazon.com was broken between 1996 and 2016. There is a push to HTTPS at all costs, even if things (ad networks, devs, software) is not ready and is there even a need for 100% TLS? That an open source browser is engaged in such initiative is not good. With HTTP you can be anonymous, and most website visits aren't mission critical things. With HTTPS your traffic is quite unique and the players with big money have the resources anyway, there TLS is no big thing as we learned in the last few years. So which Think-tank or what ever is behind this initiative?
>there TLS is no big thing as we learned in the last few years
Citation? We've learned what has always thought to have been true: Encryption works very well and intel collectors need to resort to attacking other aspects. TLS has had its share of flaws but it's still very much a "big thing" to defeat.
The problem is, basically, middleboxes. It's unfortunately still common to intercept HTTP connections and redirect them to some sort of caching proxy, and some of these proxies will misbehave when seeing HTTP/2 on the wire, causing hard-to-diagnose connection issues. It's even worse when firewalls are involved, since they can get confused easily and will drop the connection (manifesting as a hang in the browser) in that case.
With TLS, not only is it less common for the connection to be intercepted, but also a MITM proxy which doesn't understand HTTP/2 won't negotiate it with either end, transparently falling back to HTTP/1.1. And firewalls will only see the outer TLS protocol.
In addition to browsers wanting to push https adoption, there was concern over how middleboxes would deal with it. Http2 is very different on the wire than http1 and middleboxes are known to be extraordinarily horrible at handling new protocols gracefully.
Just some random places to start to see why the major web browsers are not going to spend any time on non-encrypted protocols. A lot of devs need to help themselves and move to encrypted to connections before they are forced at browser-point.
I'd like to go further: HTTP/2 is the present not just for content serving, but could be for APIs.
Right now, everyone who's supported both mobile and web APIs (or cared about bandwidth constraint) can tell you how frustrating it is that well factored restful (or rest-ish) APIs are often the casualty of accepting the realities in mobile client support. You end up increasingly collapsing requests into one Special Bigger Request because One Special Bigger Request almost always has better performance on bandwidth-constrained clients than the same performance elsewhere.
And this is especially frustrating because often times the client can't tell us exactly what they need in the initial request. The best we often can do is stash an etags header in the request and try and get very fancy there. We end up with smarter endpoints which require tons more programmer maintenance, and move us away from the world of autogenerated and automatically audited endpoints.
HTTP/2 w/ Push offers us a way to split the difference. While individual endpoints can be served very quickly, we can have our APIs recognize (via etags, probabilistic inference, previous negotiation, etc) that more data is about to be requested and push it immediately.
This means we can keep our well-factored, simple API designs that we developed 10 years ago without sacrificing performance to the bandwidth-constrained. It means that we can further lean on the abstraction Service Worker gives us in web clients to just naively poll in client code and trust in clientside caching to make this efficient (with the fallback being the same network call we would have always made).
It's why I'm absolutely infuriated with so many languages for balking at supporting HTTP/2 as part of their stack (more often than not, on the grounds that SSL + ALPN seems tedious to implement).
One thing though, I think "One Special Bigger Request" still makes sense in a lot of situation where you have some kind of "transaction" going on. I often encountered code that calls multiple HTTP endpoints and a failure in one of those requests would actually lead to inconsistent data in the backend. This code is pretty common because many developers assume that requests never fail.
> I'd like to go further: HTTP/2 is the present not just for content serving, but could be for APIs.
Very true. The talk around around server push has suffered a bit from tunnel-vision, only taking browser-media delivery into account, making it look like a browser feature only.
HTTP2 + server push can actually make GraphQL less relevant, same way that there hasn't been talk around supporting websockets in it (it just doesn't look necessary anymore).
But for all this to happen, server push needs to become cacheable. And I really hope that the cache digests rfc takes off and gets more implementations.
I think that within a datacenter, a bunch of microservices make sense. When communicating with a customer, you want to craft just the data they need and minimize roundtrips.
GraphQL seems to be a great candidate for bridging this gap. I recommend using graphql to allow clients can use to make "One Special Bigger Request" which then gets split into multiple requests on the backend.
This just made me add the 'http2' directive to my Nginx configs, and it seems that I'm getting a 15% decrease of loading time for my websites. Should have done this long ago...
Just to be sure, use a site like ssllabs.com to make sure you are actually using an ALPN. For example on Ubuntu 14, setting http2 does nothing, as it requires Openssl 1.0.2. If you are using Ubuntu 16, or a distro that provides the newer Openssl it should work correctly.
Personally, I've found it hard to find much information about pre-HTTP/1.0. To my understanding, "HTTP/0.9" is a retroactive version number. The original HTTP protocol was incredibly simple. An HTTP Request was simply "GET /", then the response was just the content immediately followed by a closed connection. There were no headers.
"HTTP/0.9" is used to describe the implementations of "HTTP/1.0" before it was -- for lack of a better word -- 'ratified'.
If you, or anyone, has more information or analysis of pre-HTTP/1.0 protocol implementations (client or server), I would be very interested in reading it.
Also, if there is a public, pre-HTTP/1.0 HTTP server available on the internet, I would very much like to check it out.
Thanks.
Edit: Alternatively, if it's possible to run an pre-HTTP/1.0 server (or client), that would be phenomenal. It only occurred to me after making the comment, however I highly doubt that it would be a simple process to get one running. I suspect that I wouldn't be able to spin a Docker container with some ancient version of Apache (I assume due to changes in the Linux environment, shared libraries, changes to gcc, etc.), but possibly, a VM running an equally-ancient version of Red Hat (pre-EL) would do the trick.
Wow, that's a disaster. Hopefully it was a wake-up call to the Shoutcast administrators to not depend on 10+ year old software that doesn't follow actual standards.
You mean, you expect tenthousands of broadcasters, with millions of users, to switch their systems over, because some people at Google decided that they should control the web? It's very questionable that such a move can even be made without a discussion in one of the web standards groups, and can be made entirely by Google employees.
In the long term, changing this is unavoidable, but without warning, and without discussion, deprecating a widely used standard is not exactly awesome. You'd expect such a change to require discussion in standards groups, months of warnings before it happens, etc
You mean, a 10 year old, unpatched protocol. You don't know the actual software running behind it, and how they might have modified it.
And just deprecating an entire category of protocols still in use without Google even sending an email as warning to the servers that still use it, well, that’s insane.
While I look forward to a faster desktop and mobile browsing experience with HTTP/2 I do worry about the complexity. I hope that the simpler protocols remain supported for a long time to enable implementations on low resource embedded systems.
The last thing we need is for all the closed-source, internet-connected, black boxes in our lives to poorly implement a complicated web standard protocol. There are so many places where we have already seen vulnerabilities with implementations of simple web severs and clients.
The "simple" text based protocols are NOT simple to implement correctly. Go try out line-wrapping and play with using \r\n vs \r vs \n as line endings and tell me what the compatibility ends up like.
And this does create real-world security problems. Some VoIP companies allow you to make free calls by screwing with their SIP proxies because popular software handles line endings differently allowing you to make their edge software interpret packets differently than their core.
Even parsing is easier in a nice binary format (and much, much, faster, too).
It's not got much to do with the serialization protocol itself as such - most server software uses a library which handles it properly and then doesn't have to worry about it. It's more that web developers have to get bits of their stack (the dynamic and static serving aspects) to cooperate in ways they never had to before (in fact, the general wisdom was to separate them as much as possible as they have totally different requirements when it comes to performant serving).
This is not even to get into persistent connections & server push etc., which is quite a big subject for developers used to handling independent fire-and-forget requests.
I guess I'm saying the massively complexity comes in trying to use these new features.
For those interested in HTTP/2 Push, check out Eric Bidelman's Push Manifest format[1], which is a JSON format describing the dependencies to push for any request URL. You can generate the manifest by hand or with tools, and then there are servers that can read the manifest and push the right resources.[2]
In Polymer we're most of the way done with support for automatically generating push manifests from a dependency analysis of your project[3] and have push manifest support in our dev server and soon a small production server, so you can `polymer deploy` and get a push-enabled site.
Doesn't this lead to the most highly used static assets being pushed for every subsequent request? Isn't that wasteful?
I use varnish cache extensively and am considering writing a probabilistic server push module. It could keep track of subsequent requests and push the next set of assets. This is more of a server module approach rather than client side.
Can someone recommend an up-to-date book/guide on HTTP(2)? I'm thinking this may do: https://http2.github.io/, but I'm not sure if that's the best guide.
Much of the jargon in the comments is unknown to me. The article in the OP is a good start.
> AFAIK, the Assets Pipeline in Rails is not designed for HTTP2...
Not true. You can define as many manifests as you want, and design your own heuristics around resources in rails. It doesn't need to be an HTTP feature.
You have to patch rails to fill up the link header with the assets however. If you do it, proper reverse proxies like h2o can serve assets using server push.
To be clear, Server Push as described in the article is not actually provided by nginx yet and thus by the server config in the article. I have no doubt that the awesome nginx folks will implement in due time though.
Its implementation is already present in nginx plus since 2015. https://www.nginx.com/blog/http2-r7/
Not sure if they are going to provide server pushes in the community version soon.
Does not invalidate his main point: the developers have a conflict of interest into releasing features for the open source version that competes with the paid version.
My reply wasn't intended to. I was pointing out that even the Plus version doesn't have Server Push. I don't expect it will ever be present in the open source version, barring a fork.
I'm not sure they will ever implement it actually. Nginx does not support HTTP/2 server push as of March 2017 (v1.11).
Owen Garrett at nginx has summarized the reasons as follows:
- it is a non-essential and optional part of http/2
- if the client already has the resource cached, then by pushing it to them you might be unnecessarily wasting bandwidth.
- server push spec might change in the future.
- Link headers as hints is useful, but usage has been low from web developers.
- server push has been available as part of SPDY and was not utilized by many web developers.
Read his original comments here[1]. This table[2] accurately describes the pro's and con's of server hints vs server push.
Personally this disappoints me because I think this would be a valuable feature for web developers willing to invest the time to optimize page speeds. I also know that some large CDN's like Cloudflare have implemented their own version within nginx to optimize page downloads.
As I've said previously [1][2][3], you shouldn't PUSH with `Link re="preload"` for performance gains. You will likely run into TCP HOL Blocking or browser request race conditions. Instead send PUSH_PROMISES in the whitespace while the client waits for the TTFB.
Is the common pattern to support http2 at the highest level?
If a request goes through CDN/edge network -> Load Balancer -> nginx -> Python app, should http2 be enabled on all components or is enabling just on the CDN enough? Any pros/cons?
With HTTPS, usually enabling on the CDN is good enough for most cases.
That's not correct - while any HTTP/2 capable servers along the path may implement server push, the most important one is the closest server to the user (i.e. the CDN). The only requirement is that the intermediate servers must not strip out the server push headers/tags, so that the edge server knows what to actually push.
It's quite possible to have your own server provide HTTP/1.1 and still take advantage of server push, and this is a usecase supported by Akamai[1], Cloudflare[2], and I suspect other CDNs (can't find details) - you may get better performance using HTTP/2 on that intermediate link, but that's less likely to be because of server push.
It's here, but current server-side implementations are super weak. rel="preload" is a hack. Only the few languages that have a native HTTP 2 push api are really here.
btw. I don't talk about the forced TLS stuff.
besides I'm working in a company who sells boxes for intranet's (where deploying TLS for domains is not always easy, lack of proper DNS or other things, etc...)
The spec has just a lot of edge cases especially when handling state/what exactly is a specification violation.
Especially handling clients who opened a connection and now are in closing state, can be really really tricky to handle.
On the implementation side the Statefulness adds a lot of overhead, that HTTP/1.1 did not had (besides other flaws) I think HTTP/2 was a little bit rushed, but I guess HTTP/3 will clean up some stuff.
HTTP2 has been created for big players. When you are a small team, benefits from it are not that interesting. Plus mandatory TLS is making boostraping a project unnecessary hard.
If you mean bootstrapping a webservice-project, I don't believe it's hard in any way. With offerings like Lets Encrypts, it's incredibly easy and takes no more than 20 minutes.
It's 20 minutes only if you are exactly in the regular use case and already know what you are doing. And it's 20 minutes for one thing. Imagine if each little detail of your setup would take 20 minutes.
And of course down the road you will have to figure out subdomains, cross domain requests, media streaming, embedding external unsecure content, debugging on mobile devices, etc.
And of course this will work only if you are on a private server, cause on the simplest hosting you won't have this.
You lost touch with the quantity of knowledge you accumulated over the years.
Most browsers block it at this point, and rightfully so. A single unencrypted include can be used to inject malicious content on a web page. Lots of people use internet over untrusted connections (open wireless) or partially trusted connections (ISPs injecting content).
>You lost touch with the quantity of knowledge you accumulated over the years.
Therefore we must lobby that our skill set is protected by law and that protocols and best practices are not allowed to change! Uh? Sorry man, you are going to have to evolve as the internet changes. Well, you don't have to, but when all your devices are encrypted and asking for a few bit coins to unlock, you will have been warned that it was going to happen.
Easy to say when your business is no tied to advertisers on which you have zero leverage.
> Therefore we must lobby that our skill set is protected by law and that protocols and best practices are not allowed to change!
Making TLS optional on HTTP2 would have not taken anything from you. You are just here in your ivory tower looking down on people that are clearly not as pure as your are.
Well sorry, but down there, among the humans, you have PHP dev that are using EasyPHP, people that don't know the command line, people using mixed content and have no control over it, people that can't spend hours to continuously learn new things every week, people that don't have private servers and so on.
And you know what ? I'm not even among them. I run my service on private servers with https. I use docker, letsencrypt and react or whatever is useful to me.
But giving trainings and having friends running side business, I know that it's not the only reality in IT.
So basically, right now, HTTP2 is just built to be hard for a huge part of the dev population, with no good reasons.
>Easy to say when your business is no tied to advertisers on which you have zero leverage.
Then it is time for your business to die. Sorry about your luck.
>Making TLS optional on HTTP2 would have not taken anything from you.
It would have reduced my security in the future when developers used it inappropriately.
>Well sorry, but down there, among the humans, you have PHP dev that are using EasyPHP, people that don't know the command line, people using mixed content and have no control over it, people that can't spend hours to continuously learn new things every week, people that don't have private servers and so on.
Then evolutionary forces far beyond them are going to push them to extinction. Adapt or die as they say. Your friends running side businesses need to realize they are on the public internet subject to grand forces far beyond their imagination. Malicious governments watching recording everything they can on people. ISPs injecting ads into customer data streams and selling browsing habits to third parties. Blackhats compromising machines to steal, or to spread trojans to client computers. Those things are real threats that the 'lowly humans' have to know about if they want to do business on the Internet and not be totally compromised. The fact that bad people exist just raises the cost of doing business.
Bootstrapping using letsecrypt requires an internet connection and a public facing service. I can say that 14 year old me, who had books, a pc and no internet would have been screwed by this.
If you're just using it locally, you can use a self-signed cert. Caddy generates one automatically by just adding "self_signed" to your config[1]. Or you can copy-paste a single OpenSSL or PowerShell command to generate one.
But this implies you can then deploy your caddy server. Which means a private server with root rights. Do you realize most devs don't even know server admin ? If you know the command line, you are not most dev. You are far, far away from the reality of enterprise software.
Tomcat and apache are still massively deployed. If you moved to nginx years ago, remember than it's still a new toy for a huge number of people. So caddy...
This is the closest thing to reasonable that anybody has suggested. I seem to be getting brigaded by the encryption cargo cult and while I agree that nothing public facing should not have encryption it's a different thing to force all developing students to learn TLS/SSL before they start actually learning to code.
I expressed dissatisfaction with the fact that learning to code for the web has a barrier to entry now that can be very difficult for various reasons. Learning To debug TLS issues is another problem in of itself which often takes a lot of years to fully understand.
A new programmer should be very aware of the tools that are available to make this easier. But 14 year old me with a php book and no internet did not have such luxuries and that was all I said.
The difference between 14 year old you and now, is that crappy php app you made then might have made it years before it was hacked. These days it could last just minutes.
The problem here is you are thinking about just yourself. Yes, it's hard to do learn all these things. It was especially hard back then because the the sources to learn were pretty crappy and taught a lot of bad things. That's part of the reason we have the 'exploit a day' internet we suffer from now.
If new web developers have to learn both protocol security and code development, it will probably mean less developers. Which is great for the ones that take the time to learn, they'll get paid more. It also is probably better for the customer, hopefully it will mean they are less likely to get served virus_porn_encrypt-your-computer.exe from your half baked website.
People downvoting you completely forgot how it was when they started. They don't see the huge gap between us now, experts, and the beginners arriving on the market. In the quest of making our field more professional, we failed to make it gradual improvements.
Pardon my ignorance, but how will the 14 year old you create a website without an internet connection and a public facing service?
If you can create one, presumably for your internal network, with no public facing parts, how exactly does letsencrypt "screw you"? You can create and distribute self-signed certificates today, just like you could the past 20 years.
But soon everybody will be. It's not for me. I will always be alright. But I know many people that will be left behind. On HN, users completely fail to relate to regular devs.
You are right. More and more propaganda these days to steer the crowd to things were only the top players benefit. HTTP 0.9/1.x with or without TLS (aka HTTPS) is still relevant. And making TLS mandatory is a very bad decisions, it will hurt every dev who had to touch lower level APIs or do performance optimization or debugging a lot. Sometimes it would help to think a bit father then ones nose is long.
>it will hurt every dev who had to touch lower level APIs or do performance optimization or debugging a lot
Cry me a river. Devs want to pump out as much code as possible without thinking about the security considerations. Who cares about the end user that's going to get pwned, right?
HTTP is being phased out. HTTPS certs are available for free. Companies can use self generated certs if needed. The days of exchanging unencrypted text over the internet and blindly expecting some evil or ignorant 3rd party not to tamper with it is long gone.
It will suck to have to resort to essentially MITM tactics on my developer workstation in order to decrypt local traffic. Yes, there are tools like Fiddler that can do this, but eventually it may be impossible to MITM traffic the way Fiddler does. That will make it more difficult to debug applications. There are also platforms like Pivotal Cloud Foundry that rely on being able to inject headers into HTTP messages as the messages traverse the platform (e.g., to support throttling); this will become more difficult if that traffic has to be encrypted.
HTTP/2 with forced TLS is, in effect, a binary protocol. Right now, developers all over the world enjoy the plain-text and hackable nature of HTTP 1.x.
I'm not sure why it would become impossible to do what Fiddler does. It's not like any major platform is going to stop letting you add root certificates.
I don't agree with the parent post (I think all traffic should be encrypted now that let's encrypt exists) but this is an interesting point.
For example, Google is making it as difficult as possible to install certificate authorities in Android.
Root certificates are encrypted as if they are client certificates, requiring the user to set up PIN or password protection on their lock screens. Removing this protection immediately disables the installed user certificates.
Google has changed the Android 7 API so that by default the user certificate store is not used to validate the validity of certificates (only the system store is used). Apps need to opt in to still support user certificates.
Of course this is all done to prevent malicious actors from performing MitM attacks on users. Still, I would not be surprised if Android P removes the user certificate store all together.
Cloud Foundry doesn't have a problem injecting headers, as HTTP traffic is plaintext inside the system itself. It's once it starts traveling across the public internet that encryption is needed. This does make it harder for network edge caches and for middleboxes, but that's not totally a bad thing either.
It's a common argument that HTTP/2 being binary makes it harder to debug. Sure, it might be tougher if I'm using Wireshark to look at raw TCP packets. But Chrome/Safari/Firefox/IE Web Inspectors (what most web devs use to debug HTTP requests) already parse the HTTP the same way as before.
So I've noticed exactly 0 difference in my workflow between HTTP 1 and HTTP 2.
<strike>None for netcat (it's a bare-wire tool, and doesn't support HTTP/1.1 either)</strike>
sorry, I must have misread "socat" as "netcat". Since socat's HTTP integration is an HTTP proxy on port 8080 it should be transparent due to using "CONNECT" for https.
