It's more than just grade inflation -- in high schools across the united states the quality of courses vary so much as to be incomparable. Suppose you and I are both taking the most advanced English classes our schools offer. If I get an A and you get a B, but you're in a high school with just 300 people in it and I'm in a high school with 3000 people, I have no idea which grade 'means' more.
Theoretically, those grades should "mean" the same, because they should be measures on an absolute scale. What matters ins the quality of the school, which is not directly tied to the number of students at it.
Agreed that the quality of the school is really the fundamental determiner of grades. In the original comment I considered adding in a suggestion that a larger school could support more levels of classes, but it felt clunky so I just left it out. Of course, schools can be good and bad for many reasons orthogonal to their size. A small school might be a magnet school drawing the top students of a larger area. Or it might be a rural area which has a hard time recruiting teachers.