QNX is one of the most ground breaking OSs that I've seen in real life and had the pleasure to work with for a good five years as my 'day job'. It was so good that everything else felt (and still feels) both sluggish and cumbersome by comparison. Imagine an OS that never freezes, one that allows you to restart device drivers and upgrade them on running systems. Use remote resources as if they are local. Control hardware in real time from the same machine that runs your desktop. What held it back is that the company that sold it (Quantum Software in Montreal, Canada) wanted to charge for every bit that left their premises and for a long time it looked as if that strategy was going to win them the race.
Then FREEBSD86 happened and suddenly a lot of router vendors had an alternative, then came Minix and finally Linux. Now none of those three are as good at what QNX does best: hard real time, message passing, clustering. So the fit for these other OS's to occupy QNX's niche was never a good one. And so Quantum stayed alive a little longer. I petitioned them for a 32 bit version when they were still fully behind their 16 bit OS and refused to even consider making a 32 bit version because their old product was selling well and because backwards compatibility must have been a hard nut to crack for them.
So, in about 2 years I wrote a kernel that lifted the userland mostly from minix and linux. Added a network stack and if not for some unfortunate affairs it would have been released as open source.
In a way I'm happy that never happened because I would have been the slave of the rest of the world maintaining that. It's one thing to push an application into the world, those things have a life-span. But an operating system will easily outlive its creators. The fact that Quantum was super aggressive when it came to IP also didn't help the chances of a release.
Every day your life is touched by 100's if not 1000's of instances of QNX still quietly chugging away somewhere in the belly of the systems that keep our infrastructure working.
I'd love to see a 64 bit open sourced version of QNX take over, there is really no substitute for an OS with a sense of time. But I'm afraid it will have to stay a dream, the stewards of QNX have never really dared to commit to open sourcing it for real, I've been asking them for long enough now that I feel that will simply not happen.
> Imagine an OS that never freezes, one that allows you to restart device drivers and upgrade them on running systems.
I'm curious how much of this depends on the driver behaving sanely at load and unload time, cleaning up after itself, gracefully handling unexpected initialization-time state, etc.
Graphics drivers on Linux are a particularly sad example of what I mean - for example while tinkering with an old PC with I-don't-remember-exactly-what onboard graphics chipset a little while ago, I noted how the framebuffer driver boasted that upon unload it would switch back to 80x25 (which it did! :D).
At the other end of the spectrum, while doing some KMS/DRM debugging on my old ThinkPad X61 at one point, unloading the driver would result in a frozen display - but not a frozen system, so my test cycle became "make; rmmod ...; cp .....; insmod ...". It worked absolutely perfectly (with just a ~250ms screen freeze), but it's kind of sad that this inelegance is the current state of affairs.
> Use remote resources as if they are local.
I am particularly frustrated by Linux's lack of IPC in practice. The fact that I can CONFIG_SYSVIPC=y - that that option exists - means nothing: nothing I know uses it, and I have no idea how to use it. OS-level IPC is not common knowledge. It's maddening.
It's sad the best option here is Erlang: that results in "yay message-passing" being restricted to large monolithic systems. I have wanted a message-passing/IPC-based command shell since forever. That's the one thing I wish I knew how to build.
> Control hardware in real time from the same machine that runs your desktop.
:) you know what this reminds me of? The real-world version of Symbolics Lisp machines. With those you could do anything you wanted, literally - they had no MMU, and I think (I might be guessing...) the memory map was static. Not quite Commodore-64, but the same ideas were there.
> So, in about 2 years I wrote a kernel that lifted the userland mostly from minix and linux. Added a network stack and if not for some unfortunate affairs it would have been released as open source.
Very very nice. Hopefully my question is answerable if I phrase it like this: is your code still in use out there, and if so, what sorts of things would I be doing (banking, using electricity, etc etc) if I were indirectly using it?
More directly, if you can actually provide more info about this kernel, I'm very curious.
> In a way I'm happy that never happened because I would have been the slave of the rest of the world maintaining that. It's one thing to push an application into the world, those things have a life-span. But an operating system will easily outlive its creators. (...)
I think I should mention AtheOS here - this was a little hobby x86 OS one guy did in his spare time. Everything, kernel up, all the way to a desktop with applications. Sound, video, I/O... file system... SMP... C++ GUI... POSIX... threads. Took 6 years (1994-2000). https://en.wikipedia.org/wiki/AtheOS
Eventually the project was abandoned, but in 2002 a small group of enterprising developers took AtheOS' GPL'd codebase (\o/) and built Syllable from it: https://en.wikipedia.org/wiki/Syllable_Desktop, https://syllable.org. They tidied up the UI a bit, added USB support and other things, and make slow progress forward.
Now, Syllable is certainly not QNX. But I wanted to say this: the person (or group) who does the sprint and gets to the point of "ok, it's out there now" doesn't have to be the same group that eventually maintains everything. It's an interesting idea to weigh up, because it instantly puts an upper bound on total development time. That time might be years, but 5-6 years is not eg 20 years. (Methinks it'll be interesting to hear Linus reflect on kernel development after he's retired and can say what he really wants to, and can step back far enough to consider all of his ~40+ years of involvement.)
Now, sure, with a "sprint then step back" approach it's impossible that there won't be disappointment and frustration with the way some things end up being done by the new gatekeepers, but if the implementation is sketched out sufficiently well and the time is taken to get the fundamentals (and the thousands of little things) correct, that could well mitigate the worst of that problem.
> I’m with Animats that it’s surprising that nobody is cloning QNX in these microkernel projects given it’s architecture is open. Past any patent protection, too, if they had any.
Such an approach would bring many benefits.
Being able to pick the license is the clear winner, of course. It would be an interesting nut to crack, because license changes (at least after public releases) are inelegant, and always run the risk of compromising confidence. This would take some interesting discussion: I personally hate the GPLv3, for example, but "GPLv3 or contact us" might be attractive to interests that need to draft a license from scratch (at least ideologically - of course a commercial license template would be filed away).
Another very attractive benefit I see is that everyone would be able to debate the finer points of QNX's existing design as implemented, and debate what parts of the design are worth copying and what should be reconsidered. Benchmarking different workload types as well as collecting anecdotes could be useful here.
Haiku (https://www.haiku-os.org/) has been in slow development since 2001, and only got to R1 in 2012. So, 11 years - for a small team of enthusiasts.
I guess the next question would be, is an open reimplementation of QNX a sane idea?
QNX is associated with a bunch of different ideas. When you say "open-source reimplementation of QNX that does X, Y and Z," you get to run with the existing context. So, rebuilding QNX could be sane in that regard.
But... Haiku's binary compatibility almost feels like a committee meeting for the sake of meeting now. Haiku is locked to GCC 2.95 (and had to backport a bunch of critical fixes in order for that to be viable), doesn't support BeOS R5 on PPC aka the BeBox (which is understandable, but notable), a bunch of private APIs aren't going to be implemented, and for all that, only a handful of applications work. (https://en.wikipedia.org/wiki/Haiku_(operating_system)#Compa...)
The question is, how set-in-stone are QNX's various APIs - ie, how much of a non-moving target are they, how well-documented is everything, how well-known are the bugs?
Obviously binary compatibility itself would be a good idea for an open-source reimplementation of QNX, there are quite a few more apps for QNX than BeOS. But the kind of apps running on QNX are the kind that you generally want to be able to leave running for a few years knowing there won't be a scheduling glitch anytime soon - the margins with QNX are a lot tigher than BeOS ever was.
:v
I need to go out now and I want to post this now instead of when I get back in a few hours, so I'll leave it there.
I can envisage that redoing QNX might be a "...dude. No. Too much work. Don't go there." in terms of the investment/reward, with all the details taken into account, but I don't know what all the details are, and with the little I do know it sounds like it could be well worth it.
> I'm curious how much of this depends on the driver behaving sanely at load and unload time, cleaning up after itself, gracefully handling unexpected initialization-time state, etc.
Well, drivers are just processes. So as long as they don't leave the hardware in a totally borked state a re-start of that process should re-initialize the driver and as long as that works I see no reason why the driver would not be able to restart and function.
> It's sad the best option here is Erlang: that results in "yay message-passing" being restricted to large monolithic systems.
And Erlang is soft real-time as opposed to QNX's hard real-time. For non-interactive backend stuff it doesn't matter that much but for front-end or hardware control it matters a lot.
> is your code still in use out there, and if so, what sorts of things would I be doing (banking, using electricity, etc etc) if I were indirectly using it?
I never released other than some very fancy routers where it was used under the hood without anybody being the wiser, but just about every industrial plant runs QNX somewhere, and quite possibly still the 16 bit version on ancient Compaqs.
> I guess the next question would be, is an open reimplementation of QNX a sane idea?
You're asking the guy whose front page post reads 'how to sort 2 tons of lego'? :)
I'm not always the best in making decisions when it comes to what makes economic sense. But I think the world of computing would be a much better place if we went microkernel. Stability and reliability and real time in my book easily trump performance.
> how set-in-stone are QNX's various APIs - ie, how much of a non-moving target are they, how well-documented is everything, how well-known are the bugs?
QNX is incredibly simple at the core. The hard part was to get the interrupt handlers work right, after that it was a walk in the park. But I do recall having a footswitch connected to the reset button because I got tired of diving under the table to reset the machine. No VMs back then...
> Obviously binary compatibility itself would be a good idea for an open-source reimplementation of QNX, there are quite a few more apps for QNX than BeOS.
I would not care about that at all, but I'm sure you could do a ABI layer if you really wanted to.
> "...dude. No. Too much work. Don't go there." in terms of the investment/reward, with all the details taken into account,
Fortunately when I started that trick I didn't know just how hard it was. And eventually QNX did come out with a 32 bit version (2 years later or so).
But an open source version of it would make big inroads into all kinds of embedded systems. But good luck getting the likes of NVIDIA to make a driver.
>> I'm curious how much of this depends on the driver behaving sanely at load and unload time, cleaning up after itself, gracefully handling unexpected initialization-time state, etc.
> Well, drivers are just processes. So as long as they don't leave the hardware in a totally borked state a re-start of that process should re-initialize the driver and as long as that works I see no reason why the driver would not be able to restart and function.
Yeah. Fair point. I think I was conflating OS architecture with buggy drivers there.
>> It's sad the best option here is Erlang: that results in "yay message-passing" being restricted to large monolithic systems.
> And Erlang is soft real-time as opposed to QNX's hard real-time. For non-interactive backend stuff it doesn't matter that much but for front-end or hardware control it matters a lot.
Ooooh. Good point. Now I think about it, hard real-time and message-passing is... I can see why people like it :) those two things together are something I've been trying to figure out for a very long time.
>> is your code still in use out there, and if so, what sorts of things would I be doing (banking, using electricity, etc etc) if I were indirectly using it?
> I never released other than some very fancy routers where it was used under the hood without anybody being the wiser, but just about every industrial plant runs QNX somewhere, and quite possibly still the 16 bit version on ancient Compaqs.
Nice. Very nice :)
(I noticed from a comment elsewhere that it was a 32-bit reimplementation when QNX was 16-bit, and that you dropped it in without anybody realizing. That is both insane and awesome.)
>> I guess the next question would be, is an open reimplementation of QNX a sane idea?
> You're asking the guy whose front page post reads 'how to sort 2 tons of lego'? :)
Oooh! That was you! That was a really impressive design. I found the "jet of compressed air" so reminiscent of "use a desk fan to blow the box off the conveyor" in that apocryphal "$2m consultant" story :)
> I'm not always the best in making decisions when it comes to what makes economic sense.
Sometimes economic sense isn't the best driver of things that become revolutionary ideas. Linux isn't overall a bastion of best practices nowadays (sadly) but it was started as "just another hobby project" that happened to go viral (in the 90s, no less).
But even before that, UNIX itself was purely academic and its designers retained the "information wants to be free" mentality, continuing to quietly release updates via tape even after the licensing wars began.
But besides all of that, sitting around all day and figuring things out without commercial focus (at least initially) - basically just having fun being creative - can be the most educational. Oh, and fun. :D (I've only just recently figured this out...)
> But I think the world of computing would be a much better place if we went microkernel. Stability and reliability and real time in my book easily trump performance.
I don't fully understand why Linus thinks that a monolithic kernel approach is honestly more efficient. I can't help but wonder if it was a case of "it was easier for me to do it that way, shhhhhh", or what. QNX is a microkernel that also offers hard real-time guarantees. I find it difficult to believe that microkernels do not offer practical performance in the face of such capability. But then again, I'm admittedly not an OS expert. (Although https://en.wikipedia.org/wiki/Talk%3ANanokernel - thanks Google for showing me the talk page instead of the article! haha - cleared things up nicely.)
I can very easily understand microkernels being more stable and reliable. (Does QNX have anything like Minix's reincarnation server?)
Regarding realtime, I've personally been particularly interested in UI design for as long as I can remember, and one of the things I've been trying to crack for ages is how to get rock. solid. animation. - with consistent fps regardless of what else is going on - and I've finally concluded that there are no alternatives to doing this from the kernel up.
Also - being able to do realtime hardware I/O from hacked-together Perl sounds like... yes please!
>> how set-in-stone are QNX's various APIs - ie, how much of a non-moving target are they, how well-documented is everything, how well-known are the bugs?
> QNX is incredibly simple at the core.
That's promising!
> The hard part was to get the interrupt handlers work right, after that it was a walk in the park. But I do recall having a footswitch connected to the reset button because I got tired of diving under the table to reset the machine. No VMs back then...
Wow, hah. (Hopefully this was on an embedded box with no/a skippable BIOS? :P)
Nowadays a decent i7 (a decent one) can get 200MHz emulated performance in Bochs, AFAIK.
FWIW, MARSSx86 is a fork of QEMU 0.14 with a 200kHz cycle-accurate simulator/instruction logger that kicks in either when you run a command in QEMU's console or in QEMU's launch parameters. So when it's off QEMU runs at full (emulated, non-virtualized) speed. https://github.com/avadhpatel/marss/graphs/contributors shows that development wound down ~halfway through 2012, with very minor updates since. But it's not dead.
Obviously these tools are not perfect, but seem like they might make "serious"/in-depth OS development a tiny bit more fun and accessible. And besides these there's also DOSBox's built-in (and eminiently-hackable) debugger that you can enable at compile time (IIRC).
I also remember stumbling on a little assembler-emulator for DOS that could "play" the assembly backwards, which I've always thought was quite cute. It's easiest to link to the last time I mentioned it (you'll see why): https://news.ycombinator.com/item?id=13807599
>> Obviously binary compatibility itself would be a good idea for an open-source reimplementation of QNX, there are quite a few more apps for QNX than BeOS.
> I would not care about that at all, but I'm sure you could do a ABI layer if you really wanted to.
Oh, okay. Hm.... could this ABI layer itself be a server to the microkernel? :D :D
>> "...dude. No. Too much work. Don't go there." in terms of the investment/reward, with all the details taken into account,
> Fortunately when I started that trick I didn't know just how hard it was. And eventually QNX did come out with a 32 bit version (2 years later or so).
Heh.
This is very very interesting, then. Depending on how long ago you did this (sounds like it was a while ago - not sure when QNX went 32-bit) you might remember quite a bit if you tried to do it again.
And if you did, you've already done the "re-do QNX, fixing all the less-than-ideal bits that couldn't be identified as such at design time" thing - so you'd be going over it a second time, with potentially even more refined results.
> But an open source version of it would make big inroads into all kinds of embedded systems. But good luck getting the likes of NVIDIA to make a driver.
Eheheheh. The situation is not nearly that negative now! :D
First, you target Intel. This is simply tedious: there are real specifications floating around, along with a full GPLv2'd implementation in Linux that "just" needs normalizing (basically, distilling KMS and DRM out of the picture). As you go along, you may be able to get some assistance - Intel recently said hello to FreeBSD: https://news.ycombinator.com/item?id=13848258, and I think you could describe QNX to be sort of on the same spectrum/track as an OS ecosystem in some ways, so that might translate to not-entirely-closed-doors where/when it would count. HD Graphics is a huge win to have available; many embedded systems (:D :D) don't have discrete GPUs, and the on-chip GPU specifies two video heads (IIRC), can handle 1080p+, and generally from the standpoint of "reimplement QNX" provides an overkill level of hardware capacity/capability. It would be more than enough to get in the door of making a serious MVP (that's actually usable), if you will.
With some kind of established thing going on with Intel, work would begin to try and target AMD, specifically the APUs. This is likely where the NDAs would start, but it would be viable to do so because a) AMD would be able to see the Intel-only situation and want to compete and b) I expect there would be some sort of commercial thing with the OS happening at this point (eg, the MVP attracting some bites, either from VCs or industry) helping things along.
From there, it'd be a matter of going "hey NVIDIA...?" and either waiting 2E14 years^W ^W for a port or getting specs to help out.
I realize I've naïvely just described a very high-level timeline involving tens of man-decades of development, a noteworthily large team, and probably a 7 figure budget. Sounds like a ridiculous amount of fun :)
I'll also admit that my experience with C is not quite where I'd like it to be, and I've never written about an OS before so I'd have no idea where to start.
But I'm game if you ever feel like doing this at some point. I was thinking of maybe making a website (hmm, what to call it - I have no idea) so people could have a hub/point of reference to point to and figure out what to work on and stuff.
That being said, no rush; this sounds like a fun idea, best to keep it that way. So TL;DR get in touch if you ever want to pursue this :P
(To clarify - I'm not saying you have to email to reply to this comment, just that my email's in my profile if you want to get in touch in future.)
QNX is one of the most ground breaking OSs that I've seen in real life and had the pleasure to work with for a good five years as my 'day job'. It was so good that everything else felt (and still feels) both sluggish and cumbersome by comparison. Imagine an OS that never freezes, one that allows you to restart device drivers and upgrade them on running systems. Use remote resources as if they are local. Control hardware in real time from the same machine that runs your desktop. What held it back is that the company that sold it (Quantum Software in Montreal, Canada) wanted to charge for every bit that left their premises and for a long time it looked as if that strategy was going to win them the race.
Then FREEBSD86 happened and suddenly a lot of router vendors had an alternative, then came Minix and finally Linux. Now none of those three are as good at what QNX does best: hard real time, message passing, clustering. So the fit for these other OS's to occupy QNX's niche was never a good one. And so Quantum stayed alive a little longer. I petitioned them for a 32 bit version when they were still fully behind their 16 bit OS and refused to even consider making a 32 bit version because their old product was selling well and because backwards compatibility must have been a hard nut to crack for them.
So, in about 2 years I wrote a kernel that lifted the userland mostly from minix and linux. Added a network stack and if not for some unfortunate affairs it would have been released as open source.
In a way I'm happy that never happened because I would have been the slave of the rest of the world maintaining that. It's one thing to push an application into the world, those things have a life-span. But an operating system will easily outlive its creators. The fact that Quantum was super aggressive when it came to IP also didn't help the chances of a release.
Every day your life is touched by 100's if not 1000's of instances of QNX still quietly chugging away somewhere in the belly of the systems that keep our infrastructure working.
I'd love to see a 64 bit open sourced version of QNX take over, there is really no substitute for an OS with a sense of time. But I'm afraid it will have to stay a dream, the stewards of QNX have never really dared to commit to open sourcing it for real, I've been asking them for long enough now that I feel that will simply not happen.