Can someone justify MQTT over HTTP and WebSockets?
Before you jump down my throat, I've used all three protocols to a fair degree. MQTT was the most painful, and without strong justification, what's the point? I always read a bandwidth usage justification, but if that's the case, someone should be able to tell me the number of overhead bytes saved using MQTT over WebSockets.
A quick comment based on our experience. We have been providing a proprietary messaging server based on WebSockets and HTTP Streaming for many years now, mainly used for financial trading, online betting, and aerospace. The increased popularity of MQTT in recent years pushed us to find some good points of contact between MQTT and our well established practices to deliver real-time data through the Internet. Simply tunneling MQTT over WebSocket didn't seem to be enough in some scenarios (perhaps corner cases, perhaps not, we don't know yet). In particular, what is completely lacking in MQTT (and in MQTT over WebSockets) is the ability to throttle and resample the data on the fly based on the available bandwidth.
To make a long story short, we decided to "add" this and other features to MQTT by creating a gateway that transparently remaps MQTT over our proprietary but open protocol (TLCP). The result is a solution that exposes very standard MQTT API (Mosquitto-like), but is much more Internet-friendly than MQTT over WebSockets (both for throttling and other reasons, which I don't list here to keep it short).
Decoupling between publishers and subscribers is the biggest one. Being able to store messages while consumer is down. Last will messages are very convenient. Ability to authorize on topic level.
All of that is very useful in my use case (networking and IoT equipment in the middle of nowhere in rough conditions that can go down a fair amount).
MQTT is interesting for IoT devices. Theses devices are low power, low memory, low speed, may have intermittent connections.
The alternative to MQTT wouldn't be HTTP or WebSockets, but actual sockets.
MQTT is simple to implements and doesn't require much performance. Many of its implementations don't support its QoS feature that retries because the embedded processor doesn't have enough memory.
Sure an HTTP client can be done, but you won't get the functionalities of MQTT and you'll have to support it yourself.
MQTT is always on and it's bidirectional. The client can both listen to the server and talk to it. You sure can do that using some kind of push/pull over HTTP but that's going to be costly. You can sure do HTTP long polling, but that's most probably contains code smell.
MQTT can be used simply to broadcast something, which is quite useful in the IoT world. You got a data that has changed, another client can easily get it and do something with it (either log it, show it, or change its behaviour based on it). Again something that you'll have to implements over HTTP.
> A HTTP client can be implemented by hand with just a few examples in a few hours.
If an implementation for it already exists (and most likely than not it does) you are better of with using it than rewriting it.
Sure you can't do it in hours, but MQTT is simple enough that there's already an implementation for you somewhere. Though seeing this source code, I'm pretty sure it can be pulled off in hours still [1].
This is a much better argument than your first comment.
For a persistent connection, you might want to move from HTTP to WebSockets, which also brings the bidirectional quality and some more of the pub/sub qualities. Though WebSockets' complexity compared to just HTTP is similar to MQTT.
Being able to implement the protocol yourself is less something I would do in production and more a comment on how easy a protocol might be to grok.
I wouldn't expect to implement my own WebSockets client. So this and the relative overlap of features puts MQTT and WebSockets in the same bucket for me (far more so than MQTT and a raw socket).
Summarizing my views: MQTT seems as opaque as WebSockets without the benefits of being built on a very common protocol (HTTP) and being used in industries beyond just IoT. The main benefits proponents of MQTT argue for (low bandwidth, small libraries) don't seem particularly true in comparison to HTTP and WebSockets.
> built on a very common protocol (HTTP) and being used in industries beyond just IoT.
For use case beyond IoT, sure use everything else, but MQTT is great specifically for IoT. It's like saying x86 is better than others 32 bits architectures used by microcontroller because it's used in industries beyond embedded. What is used in embedded make sense for embedded only. If your needs are beyond that, go beyond.
> I wouldn't expect to implement my own WebSockets client. So this and the relative overlap of features puts MQTT and WebSockets in the same bucket for me (far more so than MQTT and a raw socket).
I wouldn't consider even implementing a WebSockets client. I would consider porting the 600 lines of that library I linked though. Would you consider porting that librarie instead [1].
> don't seem particularly true in comparison to HTTP and WebSockets.
I use MQTT over my ESP8266 devices. That thing has 96 KB of RAM and 4 MB of flash memory. Sure Websocket can works on it, sure you can write an HTTP server on it, but in comparison, the library I linked is 600 lines of code, is simple to use and does everything that is needed of IoT really simply.
Even if you were to try to replicate that in WebSocket, you would still have to add a way to broadcast that information easily instead of simply having another client that subscribes to the same server and listen to the topic you publish to.
> What is used in embedded makes sense for embedded only
Certainly, with that attitude. Personally, I fight that mentality when I can. The reason to lean on technology that is used in multiple disciplines is to increase the pool of viable programmers. If it's the wrong tool, then it's the wrong tool. But otherwise use the tool that the most can use.
And yeah, that's micropython (because I love inviting the wrath of embedded developers). But that will run on your ESP8266 just fine. I use the D1 mini :)
You're definitely right on having to implement more of the pub/sub on your WebSockets server. I misspoke in saying WebSockets gives you those qualities.
> Certainly, with that attitude. Personally, I fight that mentality when I can. The reason to lean on technology that is used in multiple disciplines is to increase the pool of viable programmers. If it's the wrong tool, then it's the wrong tool. But otherwise use the tool that the most can use.
Which is my point, the right tool in that case is one that fit on the memory footprint of an embedded microcontroler.
> On porting libraries...well, the WebSockets client I play with is only 60 lines.
Theses 60 lines does almost nothing. You ignore the 240 lines inside protocol.py just beside. You also ignore the 600 KB binary required for MicroPython. Already you simply CAN'T run on an ESP8266 that's only embedded with 512 KB (so can't be run on any of my ESP-01 module).
That does make me think though that the Websocket protocol is quite simpler than I though, but if I were to use it, I would plainly do a socket connection instead which would be even more simple.
> I use the D1 mini :)
Sure with one big enough, but we are talking about embedded, where resources are scarce. Running Micropython is a luxury for many embedded developers.
Yeah, I did miss the import from protocol.py. I knew it couldn't be just 60 lines but got excited and hit enter. Regardless, to your earlier point, I don't dig into the protocol's details often and just rely on the library.
Micropython and this WebSocket client DO run on the ESP8266. As noted before, I'm using these libraries on a D1 mini. I don't know the specifics of the memory footprint as I stay away from applications where I'm up against the wall of my device's capabilities. But the point stands that there's plenty enough space on the ESP8266 for the glorious luxury of micropython.
(At first I thought you were asking about MQTT-over-websockets, but you're actually asking about MQTT vs websockets.)
I believe MQTT implementations tend to be smaller, which is why it's popular for IoT devices. Both HTTP and WebSockets are much more diverse specs, so a generally available implementation might have a lot of features the device never uses but still occupy disk and memory (multiple kinds of headers and their semantics, data encodings, etc).
Before you jump down my throat, I've used all three protocols to a fair degree. MQTT was the most painful, and without strong justification, what's the point? I always read a bandwidth usage justification, but if that's the case, someone should be able to tell me the number of overhead bytes saved using MQTT over WebSockets.