I think that Occam's razor says, that if we have several theories that describe reality with the same accuracy, we should chose the most succint (least redundant).
I don't think of it in terms of redundancy, but of introduced entities. That is, if two theories have equal explanatory power, but one theory requires ten new concepts, but the other one requires one new concept, prefer the latter. This interpretation is, I think, the spirit of what he intended: http://en.wikipedia.org/wiki/Occams_razor#Ockham
That's a good way to put it. If you want to be mathematically rigorous about it, you could phrase it as a statement about probabilities: for any A and B, P(A and B) <= P(A).
