As someone who is not a cryptography expert, is there any hope of using similar logic to train on encrypted data? Naively it seems like you could perform the same operations on the back propagation steps (or any other update algorithm you're using for non NN models) to arrive at the encrypted version of the parameter updates, which you could then decrypt to get the updated model. Am I missing something here?
You're quite correct that evaluating the back propagation could be done exactly the same as the forward pass. However, if you want to train for more than a handful of steps you'll have to use an operation called bootstrapping to periodically "refresh" the ciphertexts that encode the model. Bootstrapping is essentially evaluating the decryption circuit of HE using HE itself (with an homomorphically encrypted version of the secret key). The problem is that bootstrapping is much more expensive than the other operations in HE.
People have done very effective training on encrypted data using simpler models, like linear or logistic regression. See for example this work [1] from my colleagues at Microsoft Research.
Training is a lot tougher. Just doing one gradient update step isn't all that bad (although you may have to play with the loss function a bit, e.g. logit cross entropy is probably tough to evaluate). However, then you need to go and actually do all the steps and gradient updates, so you probably need some form of bootstrapping to be able to evaluate computations of that depth. Also, the use case is slightly less compelling. For training, you can probably get all the parties who have data to coordinate and evaluate an MPC more cheaply than you could with HE alone. I think it'll require a very compelling use case for somebody to go and think through what the best way to do it is and it'll probably depend on the specifics of the application (who has what data, and what are we willing to leak as we go along - e.g. it's a lot easier if you don't care about keeping the weights secret).
There are definitely compelling use-cases and there are people working on it (though not me). Developing tools/systems to handle sensitive data in a secure way is extremely expensive and time consuming. If you can create data collection and model training pipelines that can operate effectively with just encrypted data then you greatly reduce the number of vulnerabilities (e.g. fewer employees need to actually see the sensitive data and fewer points of attack on the system itself).
There are certainly a number of factors to consider besides data security when evaluating the practicality of such an approach but I just wanted to confirm that it was technically possible before getting in to any of that. Thanks for your response and the post, I knew almost nothing about HE before today.
