Yes. Though I think they stumbled on ReLU just from technical considerations and not taking nature into account?
That happens fairly often in engineering: looking around enough one can often find natural precedents for our artifices. But directly trying to copy nature did not lead to the plane or helicopter.
Wikipedia list this as the first paper involving ReLU in an ANN:
[1] R Hahnloser, R. Sarpeshkar, M A Mahowald, R. J. Douglas, H.S. Seung (2000). Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit.
That happens fairly often in engineering: looking around enough one can often find natural precedents for our artifices. But directly trying to copy nature did not lead to the plane or helicopter.