It might be true that latency can't be improved much by using an off-the-shelf FPGA but performance in parallel processing tasks can. For instance an FPGA might be cheaper than a GPU (in relative performance/$) in machine learning implementations because the FPGA architecture can allow for higher throughputs/less bottlenecks.