I really liked the "predecessor" DCOP better. Applications publish objects, others can call methods (one-shot) or functions (wait for response).
From DBUS' own (old) FAQ:
> D-Bus is a bit more complex than DCOP, though the Qt binding for D-Bus should not be more complex for programmers. The additional complexity of D-Bus arises from its separation of object references vs. bus names vs. interfaces as distinct concepts, and its support for one-to-one connections in addition to connections over the bus. The libdbus reference implementation has a lot of API to support multiple bindings and main loops, and performs data validation and out-of-memory handling in order to support secure applications such as the systemwide bus.
> D-Bus is probably somewhat slower than DCOP due to data validation and more "layers" in the reference implementation. A comparison hasn't been posted to the list though.
IMHO DBus suffers from the kind of overengineering that's been endemic in Desktop Linux in the last few years: DConf/GSettings (as a less flexible, IMHO unneccessary replacement for GConf), PulseAudio and NetworkManager (initially really bad, now work nicely, as long as you don't have to debug problems...), all the *Kit stuff, systemd, and so on. For me, libraries like this don't really solve problems on average, but cause regressions.
I think people who haven't hacked on making nice UIs for system features don't understand what problems these things solve. But people who have do, and that's why they code them.
Is there a simpler, less-engineered way? Probably, in the truism sense that all software sucks. But then, anyone could have coded this better way and made it work, and they didn't. So the current work has the advantage that somebody did it and it exists. I'll take that.
Knowing the problems solved here I actually think the current stuff is pretty good. Not flawless -it's software - but good. It does pretty much work. Go use 15 year old Linux if you want to replace nostalgic memories with a good dose of how much it sucked :-p
Agreed -- I remember some of the alternative ways to solve the same
problems, such as CORBA and various custom protocols, but they were
painful to use...
Now, early DBus was a bit lacking in that respect as well, but with
GDBus (GLib's DBus implementation) it's become as straightforward as
IPC can be, even in plain C; I've seen quite a few people new to the
technology that were able to get up to speed with it quickly.
It'll be interesting to see how people are taking DBus and pushing the
boundaries of what it can be used for, such as kdbus.
Interesting perspective. I thought I read somewhere that ZeroMQ was championing becoming apart of the Kernel. I work in the embedded space and not knowing much about DBUS other than it's some fancy IPC and ZeroMQ, we went with ZeroMQ just because we saw a clear path for what we needed get done. Granted all our IPC communication was between our own apps. but after reading your post I keep wondering what REAL problems does DBUS solve besides having a standard messaging format? Could someone explain why a simple messaging format couldn't be deployed on top of something like ZeroMQ?
This probably deserves a long blog post or something (maybe I already wrote it somewhere) but here's a teaser.
dbus is not mostly about IPC.
Linux desktops, including gnome, KDE, and those before them and alternatives to them now, use a "swarm of processes" architecture. This is as opposed to an alternative like smalltalk, Eclipse, Firefox, or Emacs where lots of plugins are loaded into one huge process.
Problems common in server side IPC which aren't as big an issue here: scalability; network partitioning; protocol interoperability.
Problems which are more of an issue: service discovery (can't just use DNS); tracking lifecycle of other processes; inherent singleton, stateful nature of hardware, the kernel, and user interfaces.
The main way dbus helps with this is the star topology with a daemon that can start on demand and track all the processes. IPC is then coordinated with this in such a way that race conditions can be avoided, for example you can start a service and send it a command without a race that your command arrives too soon.
Anyhow this is just enough to get an interested person tracking down the details, I'm not spelling it out obviously.
dbus handles a lot of cases where you have an unprivileged process like a GUI widget communicating information to a root process like a network manager. Because it's centralized and everything flows through the dbus process, a lot of it is about policy enforcement— ie, these methods can be called by this user or group, these other ones by this other group, etc.
Examples of the configuration which controls this kind of thing:
I'm not against any of these projects in particular (they all solve a specific problem), however as you describe, all the recent infrastructure being pushed on desktop and servers is poorly designed and initially badly implemented. I cannot even blame the distributions, as projects such as DBus/-Kit/etc were initially just part of other, unrelated, projects that got mass adoption.
The implementation got better (actually, systemd partially got rid of many -Kit components which never truly worked, which IMHO, doing us a favor), but the design is still wonky, still poorly documented, very difficult to debug as a result and understand as a whole.
The fact that these tools rely on self-describing protocols with one single implementation without documentation also results in very tight coupling.
It's also shifting the idea of having a cli interface to perform a task to a dbus interface. Often without any documentation of the dbus interface itself to begin with (or with a crappy cli wrapper).
How exactly is a CLI wrapper supposed to work without some kind of IPC?
Most components of the system have an associated daemon running and maintaining them.
You need some kind of way to communicate to that daemon.
DBUS is the most common and easiest way.
Most components of the system /are/ a daemon. Most of the terrible decisions in Linux infrastructure of the last 10 years have been making daemons where a protocol would would do the job more simply.
IMHO DBus suffers from the kind of overengineering that's been endemic in Desktop Linux in the last few years
It's not just Desktop Linux; I've noticed the overengineering trend in basically... all software. There's many contributing factors, probably one of the biggest being the second-system effect ("there were some problems with the old way of doing it so let's find a new way that fixes them - and all the problems that could possibly occur in the future, whether or not they may actually become problems"). I think YAGNI needs to be applied far more often than it currently is.
The solution to complexity should not involve adding more complexity.
The main problem is, DCOP was explicit about it being a quick hack.
While DBus featured a huge amount of bombast from the day one while being really slow on useful things. I think they did not even have a proper console client a few years into development and some shipping.
"due to data validation and more "layers""
You can't really have data validation and proper layers without a spec. You just can't, it would be an annoyance instead of useful feature.
Speed wasn't the right design goal for a mechanism used mostly for control purposes that simply didn't have to be fast. Least-common-denominator flexible implementation was the most important thing (threads or main loop, no dependencies, etc) in order to be adopted. And then validation was required to use it on system contexts not only trusted in-user-session contexts. It also had to swap in for both DCOP (KDE) and CORBA (GNOME). These were the choices that made it viable and why it became widely used.
Many years later people decided it would be nice to have performance too so they reimplemented with more assumptions that didn't used to be safe, and did the kdbus work. The result is still compatible with the original protocol because it was always possible to make a fast implementation.
This is how open source works. People do what they care about. When speed came to the top of the list people did it.
Adoption speaks for itself. Things are adopted when they are a viable solution. Others thought speed was the key and they were wrong; dbus was adopted precisely because it focused elsewhere. But speed wasn't precluded by the design and when people cared they solved it.
Well, DBus was politically pushed onto KDE back in the day, when there was a strong interest in making a success out of cooperation efforts in the Linux desktop space. IIRC, KDE was basically strong-armed out of DCOP.
Once it was established as "the official IPC on the two dominant Linux desktops", it would have been difficult to ignore it regardless of its actual quality.
In creating dbus, I talked to a bunch of developers at KDE, specifically those who worked on Qt and DCOP, about their requirements. Then I met the requirements they said I had to meet for them to use dbus. I did the same for developers at GNOME. Because dbus met the requirements that the actual decision-making developers had, and allowed them to do useful things, it was adopted.
There was no mechanism to strongarm anybody into anything. KDE and GNOME devs told many ideas and many people to take a hike over the years. dbus was simply a matter of figuring out what those developers wanted and focusing on solving the actual problem, rather than hypothetical or philosophical problems.
When dbus was adopted, remember, people had already been down many roads; ad hoc IPC mechanisms, ad hoc communication through files and timestamps, hacks over X11 protocol, DCOP, multiple implementations of CORBA, SOAP, ICE (the old X-associated one), etc. People had wrestled with this problem space a lot and they had some pretty developed ideas about how to do things ideally. dbus was about coalescing those ideas into running code, and that was successful and stuck for a decade-plus now.
People sometimes have a "wtf" reaction coming from Internet protocols or kernel concerns, and while there are some wtf-worthy details in any piece of software, lots of the time people just don't understand the problem. Just as GNOME and KDE both took years to understand it and flailed around with all those protocols that didn't work out well.
The chronology I remember is a bit different (GNOME needed a replacement for CORBA and came up with DBUS; KDE already had DCOP; then Freedesktop happened, with big players pushing to reduce mismatches so that they could ship both environments with less hassle; and KDE scored some and GNOME scored some (more), with DBUS being one of them), but I was just a random observer so I might have got it wrong.
Still, it's a fact that DCOP back then felt much nicer and more humane (I wouldn't know about speed). I haven't used Linux in a while so I honestly don't know now.
That seems to be the story of freedesktop.org in total.
It was, iirc, supposed to be a place for KDE and Gnome to hammer out some cooperative issues, so that programs related to each project would coexist better.
These days it seems like Gnome (or maybe i should say Red Hat?) is running the place, with KDE doing its own thing and glancing over the fence from time to time.
I should have mentioned here earlier, the original thought with dbus was that nobody would use libdbus directly; KDE would use an API more or less exactly like DCOP and GNOME would use soemthing like gdbus. For that reason, libdbus was lowlevel and flexible in a way that made it annoying to use - convenience API in libdbus would have been bloat.
What actually happened is that the high-level wrappers didn't get written for a long time because libdbus was just good enough that people didn't bother to work on alternatives. So that sucked, but it does seem to have finally been sorted out and libdbus is now on the way out AIUI.
Somehow, attempts to bolt message passing onto Linux never seem to be very good. Probably because the primitives underneath are a poor match.
QNX, which is a real-time microkernel, got message passing more or less right. You connect to a port of another process. Then you send with MsgSend, which sends a message of N bytes, and waits for a reply. So it's like a procedure call.
The receiving end (considered the server) does MsgReceive, which blocks waiting for work, gets the bytes, and returns a reply with MsgReply. That's QNX messaging.
Everything goes through this, including all I/O. It's very fast, and integrated with the CPU scheduler, so most message passes just transfer control to the other end without scheduling delay. This allows rapid tossing of control back and forth between processes without going to the end of the line for CPU time.
Because QNX is a real-time OS, there are some additional features. Messages are ordered by thread priority, so real-time requests are serviced ahead of non-real time. (This works well enough that running a compile or a browser doesn't impact hard deadline real-time work.) Any request can have a timeout, in case the other end has a problem. Finally, when a MsgSend from a high priority process goes to a lower-priority process, the receiving process gets the higher priority until the MsgReply, to avoid priority inversion.
Linux messaging almost always goes through unidirectional byte pipes of some sort. So you need a protocol just to figure out where the message boundaries are. D-Bus seems to be struggling with that. Building a call-like mechanism on top of unidirectional pipes means that callers do a write followed by a read. For a moment, between the write and the read, both sender and receiver are ready to run. This means a trip through the scheduler, or worse, starting the receiving process on a different CPU and suffering cache misses.
It's one of those things where the wrong primitives at the bottom cascade into layers of complexity above.
No, it's not a one-way communications scheme. It's a connectionless communications scheme, and no more vulnerable to spoofing than the connection oriented protocols.
X11 was a great success because it was a protocol that you could write X servers and X clients and X window managers against.
Nobody wants to do that anymore. There are two major forces to blame.
First, open source. A well designed protocol is much more work, and you can avoid it by just pointing to the open source'd implementation, as this article (hilariously) shows for the case of DBUS.
The second force is the adware/spyware model of web and app monetization. You don't want people to use their own clients against a protocol (email, usenet, web 1.0) because you can't serve ads as effectively and you can't run analytics on their every mouseclick, touch gesture and keypress.
The whole systemd debacle would be much defused if systemd had been a couple of well thought-out and stable protocols, much like X11, instead of a big source-blob of underspecified and ever-shifting implementation.
It's interesting that you mention X11. The Wayland developers recognised that X11 had not only become fat, which isn't inherently problematic, but that almost all of the old fat had become dead weight. X11 essentially became a horrific over-engineered, vaguely graphics related, IPC mechanism... just like DBus. Wayland is an RPC protocol.
I'm kind of hoping IPC mechanisms similar to those used by Wayland, which fell out of all the work on XCB (a clean X11 equivalent binary protocol for Xorg that, iirc libX11 is now built on top of), will ultimately be adopted by other projects. Interestingly they implemented RPC dispatch using libffi, which is pretty elegant.
The Wayland FAQ, in fact, has a rationale for avoiding DBus[0]. The core Wayland framework certainly hasn't suffered in terms of bloat or complexity by avoiding it. Go look at the code[1], and compare it to DBus[2]. Admittedly the topologies are different, but as far as I'm aware nothing prevents Wayland clients from establishing their own P2P communications.
> XCB (a clean X11 equivalent binary protocol for Xorg that, iirc libX11 is now built on top of)
I thought XCB implements the same protocol but provides another (more modern) API. Am I mistaken? I never wrote code using libx11 or libxcb. Well, not 100% true, I forked and improved a tiny project that touches X11 at two small points. Doesn't really count. I don't get any understanding of X11/XCB from that:
This is basically true. It's the same packets on the wire - the biggest difference is that for most packets libx11 waits for a response while xcb returns a token representing a promise.
In large part because programmers started to assume they could talk directly to the hardware, rather than the hardware being at the other end of some random network connection. This then makes a large part of X11 redundant, as it is there to make things network transparent.
Well, you certainly have a point, but I'm not sure you're right or not.
Pointing to a library instead of protocol isn't some recent fad. Say, this was the case with ALSA/libalsa. I guess OSS folks had ranted, yet ALSA is what we've ended with. So, it's certainly not something related to webapps and proprietary APIs. Well, I think in 2000s everyone who had some relation to FLOSS just hated such things. Microsoft's stuff had been a pain in the ass, probably more than they are now. Although, the webapp trend may have had influenced overall mentality with "aw, just link to this and you're good." Some sites, notably, Mega, had even switched to that model from a previous API-first approaches.
There are also cases where implementation-first approach has worked well, though. For example SQLite - while most users just link to the library, the format is well-documented, thought out and is quite sane. So, systemd can recover. I'm not keeping myself up-to-date with news on that, but I think I've heard (maybe, incorrectly, though) there are some efforts in that direction.
Protocols are not exclusively a good thing. People constantly try to reinvent email, but because any attempt to fix the problems requires modifying the SMTP and IMAP protocols and formatting you are fucked from the get go because you break protocol compatibility.
That is also why Wayland is having such a hard time. Every program and compositor is written against the X protocol and supplemental libraries, and any attempt to fix the protocol requires a backwards incompatible reimplementation.
So protocols are not a catch all solution - they are better than just undocumented spaghetti, but protocols imply bureaucracy and inflexibility, whereas dbus and systemd and gstreamer can move fast and break things without being beholden to a big beefy documented protocol standard to also break and try to get people to transition on.
Do you really think the world would be better place right now if email had been distributed as a library and not as a protocol anyone could implement? How could that possibly have worked? And how could it possibly have solved compatibility problems? It's not as if you can get everyone to upgrade libraries simultaneously.
The library becomes the de facto protocol. The difference is that there is a mindshare that updating libraries to stay current is less intrusive than having a protocol revision change break backwards compatibility.
I don't necessary think it would have been better, or even feasible, just that it is a situational quandary, not just the black and white "protocols always win". Protocols enable interoperability at the cost of rigidity and resistance to change. You should protocolize when you are are damn sure you aren't changing anything else, and there have been very few technologies where you can say the protocols conceived a decade ago are still problem free today.
> whereas dbus and systemd and gstreamer can move fast and break things
I think you will find that this attitude is exactly the opposite of what is expected at the plumbing layer. Move fast and break things is fine when no layers depend on you to remain stable (see also, the Web). Imagine how hellish life would be if Linus et al. took this attitude towards kernel development.
The kernel does not have a stable ABI, and none of the developers really commit to significant backwards compatibility - they just maintain LTS kernel tree branches for those that want to stay in one spot.
The same thing happened with the gstreamer 0.x and 1.x forks, except every Linux release is effectively a new GST release, albeit the APIs userspace uses against the kernel rarely break, it is a huge spread of APIs and one or another usually does change each release.
Can I use software written for an release (lets say 0.8.x) of gstreamer, and have it run correctly against a modern release (such as 1.2.x or 1.4.x)?
Because that works with th kernel, as it most certainly does have a stable userspace ABI. Internal stability is not a big concern unless you're trying to hide the source code (proprietary driver). Linus' guarantee about not breaking userspace is very widely known, so I'm sure you know this.
The GP was absolutely correct: while you have the luxury of being able to "move fast and break things" as a leaf app that nobody depends on[1], for most everything else "reliability" and "doesn't give me lots of extra work" are far more important than any other feature.
This is really just an extension of why it is considered a good idea to write maintainable, modular code instead of leaving people with a mess of spaghetti to debug.
[1] That changes if your userspace grows and people ARE depending on it.
"D-Bus is low-overhead because it uses a binary protocol" -
Alright, got it. It's binary.
"The protocol is a line-based protocol, where each line ends with \r\n. Each line begins with an all-caps ASCII command name containing only the character range [A-Z_], a space, then any arguments for the command, then the \r\n ending the line. The protocol is case-sensitive. All bytes must be in the ASCII character set."
Wait a second. This section describes a line-based ASCII protocol. Is this some other protocol?
"A nul byte in any context other than the initial byte is an error; the protocol is ASCII-only."
Ookay. So... it's ASCII only, except for the first byte?
"Returns auditing data used by Solaris ADT, in an unspecified binary format. If you know what this means, please contribute documentation via the D-Bus bug tracking system."
Oh, no, okay... unspecified binary format. THIS IS ACTUALLY IN THE "SPEC".
Sorry for yelling. I lost it a bit there. Not quite as much as the authors of the dbus "specification", though.
Here is the full context about the auth protocol in the spec:
Authentication Protocol
Before the flow of messages begins, two applications must authenticate. A simple plain-text protocol is used for authentication; this protocol is a SASL profile, and maps fairly directly from the SASL specification. The message encoding is NOT used here, only plain text messages.
In examples, "C:" and "S:" indicate lines sent by the client and server respectively.
Protocol Overview
The protocol is a line-based protocol, where each line ends with \r\n. Each line begins with an all-caps ASCII command name containing only the character range [A-Z_], a space, then any arguments for the command, then the \r\n ending the line. The protocol is case-sensitive. All bytes must be in the ASCII character set. Commands from the client to the server are as follows:
Yes, there's an auth protocol before the actual dbus protocol - there are two protocols in the spec. I think that's what you are missing.
Perhaps it's confusing but slow down and understand the tech before criticizing. It is not in fact an ASCII-only binary protocol. Other engineers do sometimes know what they are doing.
Let me get this straight. To you, there is nothing wrong with embedding a separate ASCII protocol for authentication inside a binary protocol? Of course I realize that the quoted text about ascii-only describes a subset of the full protocol. To do this is ridiculous and bad engineering.
I also note that you completely ignored "If you know what this means, please contribute documentation via the D-Bus bug tracking system." appearing in the specification. Seriously. This is not a specification. This is a poorly written description of an existing mess of a wire protocol.
Other engineers clearly do not know what they are doing.
I don't see the problem with having two protocols on one socket, as long as its defined how it works (how to switch over). HTTP for example supports switching to websocket.
The spec has always said it was informal and needed more work, for at least a decade now. Many people have implemented dbus and rewriting the spec hasn't been enough of a priority for any of them to do it. To me that says that while many are willing to say "it should be better" (including me) none of them are willing to say "and it's important enough to spend my next few
months on" (including me). But the beauty is that at any time someone is free to change that.
I think it's wrong to say something is must-have when it obviously by existence proof has not been must-have. And in fact a lot of tech that had the must-have failed. Say CORBA, which certainly had specs. They were just specs that specified the wrong thing. I'd rather have the (approximate, good enough) right thing with an informal but good enough spec (to a motivated reader giving it a chance), than pay a bunch of committee people to write down a design that failed to solve the requirements.
I would absolutely argue that websockets is a terrible protocol as well, and the only justification for it that I can see is that it was created as a hack on top of HTTP. It's certainly not the model of an ideal protocol design or the correct way to design things from scratch.
I also think it is perfectly fine to have a reference implementation without a formal specification in many cases, and I don't think specification-first necessarily leads to a better protocol. As you point out, CORBA is certainly horrible as well.
Just because no one has replaced dbus it isn't good enough. The ever-appearing mantra of "if you don't like it, write your own" is boring. It is perfectly reasonable to point out what is wrong with a protocol in wide(ning) use without immediately presenting a fully formed alternative. It could be that most people have more pressing concerns than individually fighting Red Hat and all the projects invested in dbus for control over Linux userspace.
There is no dichotomy here, the only choices are not silently accepting the protocols we have or "paying a bunch of committee people".
I don't think you're seriously proposing that anyone could just step in and redesign dbus from within the existing project structure at this point. Any changes would be fought tooth and nail by the people invested in it right now. Any replacement would necessarily be a completely separate project, and it would have extremely slim prospects of succeeding. It's not surprising that people aren't doing that, even though dbus is flawed.
We are stuck with a lot of terrible things. The only way any of them will be fixed is for enough people to get angry enough to do something about it.
In my experience (having submitted many patches) it is very hard to get patches accepted to fdo projects. It may be because the maintainers are salaried by RedHat, so they get new task assignments and are removed from maintainership of the packages. E.g here is a six year old patch lingering:
Someone certainly could write a bunch of patches to make the dbus specification better. But to get those patches included into the official dbus specification is a completely different problem. And you know that. Having spent a lot of effort in this thread defending the dbus spec as adequate, you wouldn't be the one spending your time integrating their patches.
You could see the blog post as a bug report. Pointing out weaknesses in the spec that could use improvement. Or you could see it as just another guy ranting on the internet.
If I had touched any of this stuff in years I might treat it that way in part (after expressing annoyance), but I'm not currently involved - my ssh key probably doesn't even work anymore, but if it did I'd still go through the current maintainers like anyone else because I haven't been keeping up with the latest.
The current people doing this kind of work deserve non-abusive informed criticism. It isn't OK to post a long diatribe of BS and then expect people to take it as if it were a helpful bug report. It isn't helpful. It's jackassery and unacceptable and this kind of abuse has done more damage to Linux than anyone knows.
It is entirely OK to ask questions or file bugs without sending patches. Just don't be an ass about it like this article was.
Alright. I didn't realise that I was talking directly to one of the designers of dbus, which makes some of the things I said unnecessarily harsh and perhaps even personal. Apologies for that.
I have to ask, though - why engage in a thread like this? Clearly, dbus is successful, it is being integrated into the kernel, it is used all over Linux user space by now and whatever flaws it has are clearly not impeding its use. So why bother arguing about the spec on HN?
I could sit down and discuss what an improved protocol might look like, but I don't even know if I agree that /any/ protocol that does what dbus does is the right approach, and either way, this is not the place to do so.
Why engage is a good question :-) I had the misfortune to see a link on Twitter and discover people were wrong on the Internet.
I do think there's useful stuff to learn and discuss here about software development and dbus itself if people dig in and understand it. Perhaps some bystanders will learn something.
I welcome improving and even disrupting and replacing dbus but I don't think the kind of criticism found in this article will lead to that.
I did learn a lot, thanks for commenting. It also seems like you agree with the premise of the article that there is confusing stuff in, what you called 'informal' spec, but explained why. So, thanks again.
No, I am just curious. It seems overly defensive. In game development, there was/is a guy named Derek Smart who is famous for popping up in every thread on the internet about one of his games and defending it relentlessly. The result is that every thread on the internet about one of his games devolves into endless flamewars. There are multiple examples of authors writing long responses to every single review of their books on amazon, resulting in a sort of game of insults between author and reviewers.
As the designer of something, you are too invested in it to handle someone going "this sucks!" well. My advice would be to sit back, recognize that the other person saying this is not invested to the same degree, is looking at the problem from the completely opposite perspective and is not familiar with the reasons behind every single compromise made, or why a certain feature seemed like a good idea at the time but turned out not to work in practice. Let someone else handle the defense.
It doesn't matter if there are valid criticisms to be made - inevitably, everyone describing the specification who weren't involved in writing it will misunderstand something, or leave something out, or quote something out of context or incompletely, or have completely different concerns in mind than the authors did - and as that author you are drawn to such things like a moth to the flame. But you have to resist the flame. After all, you wrote it, it does what you intended, you released it. If someone else wants something different, let them be unhappy and maybe if they are unhappy enough they will come up with their own thing.
Every one of hp's posts in this thread have been informative. Many of them were written as responses to cries arising from the mob with pitchforks and torches, but that doesn't reduce their utility. Given that both the original article, and your comment, made much of the "ASCII binary protocol", explaining the apparent contradiction certainly raised the signal to noise ratio here.
Well, I don't disagree with any of that. I wrote my comments in a bad mood and they came out harsher than I wish they had. I'm just thinking that if I was the author of dbus, I wouldn't engage the noise on HN. It was an honest question and I got an honest reply, and that's great.
Hey, all I did in my comment was to quote from the spec. My complaint (well, one of them) is that it's a "binary" protocol but it requires parsing just like an ascii protocol, it encodes data as ascii values and it even embeds a full ascii protocol in the auth sequence. (not to mention XML for reflection with an embedded type DSL in attributes, ...)
well, yes, but he's arguing that the documentation is bad, not that the dbus protocol doesn't exist.
The phrasing "The text protocol described in this document" (sic) can not easily be interpreted as "the thing described in this section, but not in the rest of the document".
I stand by my interpretation of incoherent editing.
"Immediately after connecting to the server, the client must send a single nul byte. This byte may be accompanied by credentials information on some operating systems that use sendmsg() with SCM_CREDS or SCM_CREDENTIALS to pass credentials over UNIX domain sockets. However, the nul byte must be sent even on other kinds of socket, and even on operating systems that do not require a byte to be sent in order to transmit credentials. The text protocol described in this document begins after the single nul byte. If the first byte received from the client is not a nul byte, the server may disconnect that client."
Oh, technically the dbus auth handshake is defined as not being part of the dbus protocol. That's not convincing.
If they are defined as separate protocols, then the argument that the spec is ridiculous because it contradicts itself on this point is based on a misunderstanding, whatever the merit (or demerit) for these things being separate protocols is.
Under Windows, COM and RPC are a dream compared to this mess. Interfaces are clearly-defined, versioned entities that support transmitting arbitrarily complex values (even ones containing circular references!) over a variety of protocols. It's also very fast: the "ncalprc" transport uses ALPC, which is the best IPC system I've seen on any platform. [1]
Using dbus, I feel like I have to type the same goddamn namespace name three or four times and that I never really feel like I've gotten it right.
Also, FOSS developers are in general completely obvious to interprocess race conditions. When you refer to something by an "id" and that "id" can be reused and recycles between subsequent calls, you've gotten it fundamentally wrong. I see developers make this mistake over and over in POSIXland.
[1] No, ALPC isn't like RPC-specific like kdbus: it's just a very, very good implementation of message passing over socket-like kernel handles.
That's not fair at all. I know it's en vogue to hate on some of the parts of the Linux desktop, but COM in Windows has a ton of warts all of its own. DCOM is brutally complicated to set up and operate properly. COM uses poorly-specified and difficult to parse TLB files. The IDL tool used to be brutally buggy and crashy. Don't get my started on the way that proxies worked.
Many of the critiques in this article could probably be applied to COM in many places. I imagine they might have ironed out some of the kinks, but it's got 10+ years on DBUS.
When COM was as old as DBUS is now, good luck finding the documentation for the more advanced uses outside of the books written by the third-party COM experts.
Interesting, because one of the things that attracts me to Linux over Windows is the use of well-defined interoperable wire protocols rather than RPC systems. This makes it much easier for the different ends of the communication to be written by unrelated teams without quite such coordination costs. Meanwhile RPC-style is much easier if the same team is writing both ends.
This is why there are a zillion IMAP clients and several servers but only one reliable Exchange client and server, the Microsoft one. The wire protocol is basically MAPI over DCOM, and this requires a server architecture identical to the MS one (and at much greater risk of copyright/patent problems).
GNOME have been able to produce such a bad wire protocol for dbus because they're writing both ends and happy to break compatibility regularly. It doesn't have to be this way and is a function of the project management style.
(There are some Linux clients for Exchange but no server, which is the wrong way round as it's much easier to get people to switch on the server rather than the desktop)
> because one of the things that attracts me to Linux over Windows is the use of well-defined interoperable wire protocols rather than RPC systems.
NFS is an RPC system. Both protocol-based and RPC-based systems can be well-documented and well-specified. Both kinds of system can be arbitrary nightmares. I don't think you're drawing a useful distinction. Both ecosystems have some services provided over RPC and some services provided over some kind of other message format.
> [RPC] requires a server architecture identical to the MS one
No it doesn't. It "requires" Exchange-style architecture in the same way that IMAP "requires" maildir storage. In the end, you have a message-passing protocol, and you can choose to respond to messages any way you'd like.
> GNOME have been able to produce such a bad wire protocol for dbus because they're writing both ends and happy to break compatibility regularly
They claim that the dbus wire protocol is stable and that individual dbus services will present broadly compatible interfaces. Dbus is for everyone, not just internal communication inside GNOME. That's what makes dbus being awful especially annoying.
It's a stylistic opinion rather than a hard distinction. Yes, in both cases you're exchanging messages and responses between turing-equivalent systems. However, the systems you end up with are different depending on which pieces are designed first and with higher priority.
Spot on. I can't say enough good things about COM; it's enabled some quite amazing things for me over the years and really top notch distributed systems that are easy to manage.
Careful! If you accidentally discover what dbus is for and why it is here, Lennart will immediately deprecate it and replace it with something even more bizarre and inexplicable! And we'll never interoperate and the year of the Linux desktop gets pushed back another decade or two...
Find myself reminded of Joel going over the Excel file format specs after MS released them, and found off hand references like "See Lotus 1-2-3 spec" or something of that nature.
So the documentation apparently sucks. OK, this is a bad sign, but can be fixed. Also, apparently the raw on-the-wire protocol is less than optimal. This is harder to change, but the implementation of kdbus has an excellent opportunity to fix this for itself and its associated client library.
The article is completely bullshit but it's good to know that people here trust the first random thing that they read on hackernews over reading the damn spec themselves.
As someone not very familiar with D-Bus I’d like to ask what’s exactly wrong with the XML configuration files? It just seems like something everyone takes for granted. From the examples linked in this thread[1] they seem pretty clear and readable to me.
This article is essentially "I didn't understand something after spending 15 minutes (or so) on it, and here are my criticisms of how I speculate this might work."
You know, fair enough. But if you as the reader want actual knowledge you can read the docs and code yourself and spend more than 15 minutes (or however long it was, but not long enough to have accurate info for sure).
There are hundreds of people and packages using dbus after many similar technologies were tried and didn't catch on. A curious person might ask why.
I created dbus and wrote the original spec. There is a nul byte first because some platforms require that to send credentials. Then a plain text protocol modeled on SASL for authentication. After that the binary message protocol begins.
Sorry if others could have done it better, but they didn't, and many tried. My way works and exists.
It's possible that others ways exist and are better but which simply haven't caught on. That said, they're not much use if nobody it using them and I and many others are thankful to you for your contribution.
I did stop, many years ago, in part because I was tired of this sort of thread. People who don't know the requirements and don't bother to learn anything go on ill-informed rants about other people's work. Hey, it's no longer officially my problem. But the people who are still working on improving the software still don't deserve the abuse.
> I did stop, many years ago, in part because I was tired of this sort of thread.
Reading this makes me sad. So this is why we can't have nice things in free software for end users! I, for one, will try to remember to stop and think before criticizing any free software project from now on.
they go on these rants because the documentation is so dam poor. The OP lists blatant contradictions. Your response further up would be quite helpful in the code and/or in the documentation. I work in the embedded space and often read data sheets with similar contradictions or errors. It's almost like the documentation has to be as good as the code and idea. What good is the best mouse trap if no one knows how to use it. Poor documentation is a pet peeve of mine too and it's the poorly documented DBUS stuff is why I never bothered to even try to learn more about it.
> R. S. H. P. is an American computer engineer and entrepreneur. He is known in the free software community due to his work on HAL, GNOME, Metacity, GConf, and D-BUS.
People like you are the toxic part of the community. If you don't like dbus or systemd then either come up with a better alternative or use whatever else you prefer. But attacking someone on a personal level because you don't like their stuff is just incredible toxic and annoying. Especially if you have nothing to show for yourself.
No one is showing up to your house and reformatting your computers. If you like the old stuff, keep using the old stuff. If you don't like the new stuff, don't upgrade to it.
Switching dbus to websocket/http instead of its own outer layer equivalent would get you about 1% done implementing dbus. It doesn't address or answer 99% of why dbus exists or what it does.
So it's a fine thing to consider (since websocket exists now) but I'd question the word "just" here.
It's like saying the way you'd implement an Amazon web service would be to "just use http." OK. Now what is the service? ;-) I hope that makes sense.
It's a mistake to view the problem solved here as "sending messages." The problem is all about the semantics of sending them and the lifecycle services provided by the central daemon.
Can you tell me something which dbus solves but something over http cannot? By this I mean in practice and not what it can possibly do. I have used it KDE and network manager and the like. They are all better off with rest style APIs actually.
That question is like asking "can you tell me something the Twitter API solves that http does not?" - it doesn't make any sense. They are distinct layers. One builds on the other.
http+websocket is a way to set up a full-duplex stream of messages where messages are anything you like.
dbus does have that part, but then it defines additionally what the messages actually look like in enough detail to bind them to method calls; it defines semantics such as guaranteed ordering and errors; it defines a central bus daemon; it adds broadcast messages over the daemon; it adds security features to allow mixing user and system domains; it adds a way to locate the bus daemon; it adds a way to launch and track the lifecycle of named processes; etc, a number of other APIs. It is not just a socket.
Could you implement a dbus-equivalent using http? Sure. But http does not include a "free" implementation of dbus, any more than it includes an implementation of Twitter.
dbus-on-http would have to define how method signatures and types map into http, and then it would still have to actually implement the daemon with its features and semantics.
http wouldn't make any material difference here; it would have some bikeshed-level pros and cons, but not change the system design in a material way.
How are REST and DBUS exclusive alternatives? DBUS is a protocol, REST is a protocol-agnostic architectural style. Why not use the REST architectural style when defining DBUS APIs?
From DBUS' own (old) FAQ:
> D-Bus is a bit more complex than DCOP, though the Qt binding for D-Bus should not be more complex for programmers. The additional complexity of D-Bus arises from its separation of object references vs. bus names vs. interfaces as distinct concepts, and its support for one-to-one connections in addition to connections over the bus. The libdbus reference implementation has a lot of API to support multiple bindings and main loops, and performs data validation and out-of-memory handling in order to support secure applications such as the systemwide bus.
> D-Bus is probably somewhat slower than DCOP due to data validation and more "layers" in the reference implementation. A comparison hasn't been posted to the list though.
IMHO DBus suffers from the kind of overengineering that's been endemic in Desktop Linux in the last few years: DConf/GSettings (as a less flexible, IMHO unneccessary replacement for GConf), PulseAudio and NetworkManager (initially really bad, now work nicely, as long as you don't have to debug problems...), all the *Kit stuff, systemd, and so on. For me, libraries like this don't really solve problems on average, but cause regressions.