I think the major downside is the world is immutable. For the search algorithm to run in short enough times to render in real time the point cloud needs to be organised very carefully.

Part of the point cloud needs to move for animation, which involves re-indexing at least part of the world. Doing this once per frame is probably far too expensive (at least currently), which is why animation doesn't feature in their videos.

That doesn't mean this technology should be dismissed; most world data in 3D scene graphs is static, so it does have applications. I don't see it overtaking rasterised graphics entirely, but I think there is value in it.

I'm not an expert either, so take my comment with a healthy bucket of salt.