The editor relies on an impressive client-side library called ChainPad, which uses blockchains as inspiration for determining the authoritative document after conflicts or many simultaneous edits. Typically operational transformation algorithms and systems to manage conflicts are handled by the server, precluding the possibility for end-to-end encryption.
However, ChainPad runs completely on the client-side, and is oblivious to the underlying text, thus allowing us to encrypt text client-side before broadcasting to other participants and the broadcast server. This is the first major effort I've seen for a real time client-side collaboration algorithm, and its use of a blockchain type structure is ingenious.
Thanks for the explanation. Looks like its not just E2E encrypted but distributed as well. Very cool work!
However, you shouldn't need a blockchain to make a fully encrypted realtime collaborative editor. Here the blockchain is needed to make OT work properly because OT relies on a global ordering of operations. But algorithmically you can get around that either using CRDTs or modifying OT with a purge() function. (This lets you move between any valid linearisation of history, so you can transform by rewriting your history to look like the remote client before accepting their operation.)
I've been messing around with OT stuff for a few years. Feel free to pick my brain if you have any questions - I'm looking forward to seeing where you go with it! (Grab my email from a commit on github)
I asked about this in another thread. It seems like all the blockchain does in this scenario is allow colliding edits to be detected for a fast retry. In an environment with slow network I don't see how this would work well.
If that's the case, something like treedoc[1] is prescribed and it renders the blockchain part of this irrelevant unless we're concerned about sourcing edits (which treedoc or a similar CRDT would be entirely amenable to and even appreciate).
I was sort of hoping that there was something I didn't understand about the algorithm that helped here.
Thanks. Yeah I finally realized that's what people were talking about.
I don't entirely understand the appeal in 2017, it seems like it gives worse performance, worse guarantees, and is more complex than the equivalent CRDT.
Based on the design document for ChainPad, it seems like it would be more accurate to describe it as a fancy CRDT as opposed to a blockchain. It doesn't include mining or any of Bitcoin's incentive mechanisms, so it doesn't inherit either Bitcoin's security benefits or its scalability/efficiency shortcomings.
No! Blockchains guarantee total ordering of events, but CRDTs are designed so that ordering doesn't matter and so long as all updates reach all nodes in the system (in any order) everyone agrees on the result.
In some cases (e.g., a monotonically growing list) you can use both. But the way a blockchain handles conflict and the way CRDTs handle conflict is quite different.
Convergence, they get from the blockchain. So everyone eventually reaches the same content. But that content may delete a whole paragraph you just wrote, or duplicate it, or do any number of monstrous things, while your wifi was flaky.
Make this test: open a document in two tabs, write "my test", go offline, write "is done" at the end in one, and delete "my" in the other, go online, wait for synchronization. You may see: "my test is donetestmy test i". Intention not preserved. (Just to be clear: this isn't just a problem for offline; network latency is dangerous.)
So local changes (not yet sent to others) are rebased with an OT-style algorithm, although a seemingly faulty one. (OT is brutal to implement.) But it gets worse:
> If a Patch is rooted in a previous state of the document which is not the Authoritative Document, the patch is stored in case it might be part of a fork of the patch-chain which proves longer than the chain which the engine currently is aware of.
A blockchain is a type of Merkle tree. When the tree has two children, only the longest chain matters. So, you lose your work if someone forks the chain you were on and makes a ton of edits.
Admittedly, you wouldn't give edit access to an untrusted party anyway. But that merely makes the security guarantees of the blockchain irrelevant!
Reaching convergence is easy: you simply need total order. Use a Lamport timestamp and encrypt patches with your private key, and you get the same guarantees for a fraction of the cost and complexity of chainpad.
If you want intention preservation too, there has been great efforts in OT to ensure the operations you make are applied in their intended way, so no deletion or duplication of content. (CRDT actually gets it for free, along with peer-to-peer, lucky them.) End-to-end encryption is not incompatible with either OT or CRDT; just encrypt your patches locally, and others can decrypt them with your public key. (It's probably much, much easier to implement with CRDT though, just because peer-to-peer OT is not trivial.)
(Having read this, it probably comes across as more aggressive than planned. I am actually fairly impressed by chainpad: it is a fun and ambitious project. But I wouldn't trust it with data I don't want to lose.)
I've spent a lot of time implementing realtime sync algorithms too, and I strongly agree with your comment above. I hope the CryptPad devs haven't gone too far down the rabbit hole of the CS literature, instead of reasoning carefully from first principles.
It uses the same base as Cryptpad [0] (Chainpad) which is created by the same people that created Chainpad.
It has more functions like a presentation mode [1].
The edit resolution is done client side and the "server" only relays encrypted messages. Alternative transports are in works, including WebRTC. Everything is open source on [2].
One thing that scares me is that links like they should be shared. So I think people would just be sending these links to the other person over open email. But the link IS the doc, and people will share it incorrectly. So it doesn't hit the server with the doc on it... but it hits facebook, gmail or snapchat so the other person can get it.
(I'm part of the cryptpad team) This is indeed an issue, and on cryptpad we will one to tackle that issue in the future with other ways to share the key that does not involve copying it on unsecure channels. For now there is though a difference between dropping you URL in unsecure channels and having your content lying around unsecured at a hoster's site.. the first difference is that your hoster has your content but might not be able to read your snapchat
Putting my security hat on, why are you worried about the host being the attack vector more than in transit? The host could easily alter the javascript that is sent back to sniff the crypto token.
> Since the encryption key follows the # symbol, it is never sent to the server.
This doesn't seem like a safe idea to me. While it's true that browsers don't send the # part of a URL when fetching a page, you can use JavaScript to get the value (in fact that's what this webapp is doing).
So what is the mitigation if someone manages to XSS the website and start snarfing the encryption keys for everyone's messages?
I'm a bit confused about ChainPad. Why use a blockchain here? Wouldn't treedoc be enormously better?
I confess to being a bit uninformed about how chainpad resolves editing being done in the exact same area by multiple people all at once. Treedoc handles this elegantly and with good performance. Does ChainPad?
I'd test myself, but since I lack friends it's difficult to do correctly.
The back button seems broken so you might want to check on the History API. I understand you need a pure JS redirect for generating the encrypted URL so that would seem like the best way to go. Otherwise you could also do:
- Add a button in the main page that generates the page in-the-spot.
- Add a button in the middle page that generates the page when clicked.
I was about to ask for the same feature. A download link would be enough to start with, to open the text with Typora or other viewers (Typora is an editor.)
I looked at the Android app. It asks for access to contacts and accounts. I don't like that but maybe I'll use it with my own server (Rail 5, great for me!) Still, why does it need those permissions? Anyway, I'm on Android 7 so I can probably not allow them when asked. (I'm not able to fork and remove them, not in Android development.)
Source is available here: https://github.com/standardnotes/collab-editor
The editor relies on an impressive client-side library called ChainPad, which uses blockchains as inspiration for determining the authoritative document after conflicts or many simultaneous edits. Typically operational transformation algorithms and systems to manage conflicts are handled by the server, precluding the possibility for end-to-end encryption.
However, ChainPad runs completely on the client-side, and is oblivious to the underlying text, thus allowing us to encrypt text client-side before broadcasting to other participants and the broadcast server. This is the first major effort I've seen for a real time client-side collaboration algorithm, and its use of a blockchain type structure is ingenious.
More info on ChainPad here: https://github.com/xwiki-contrib/chainpad