So aside from fire, water, rodents, bugs and children, what could possibly happen to a physical book?

Other than loss and theft of course.

Dust, and dust removal, and dust prevention, present risks to old books.

(http://www.nationaltrust.org.uk/what-we-do/what-we-protect/c...)

(http://www.nationaltrust.org.uk/servlet/file/store5/item3780...)

Some oppressive regimes will confiscate some books. I don't know what happens to that book afterwards.