You're confusing the last mile transmission capability with the cost of transferring the data in and out of the network.
Let's say you and I go in to business together and build a microwave wireless internet service. There is commodity equipment available that can push 100 mbps pretty easily. We buy a microwave transmission tower and central office from a company that is going out of business at a rate that cannot be beat. We price out our customer premises equipment so that we make half our money back on the setup fee, so setup costs are minimal.
We're now ready for business. We can provide 100 mbps internet service to customers with minimal equipment cost. Sweet!
Here's the rub. Unless our customers are content to transfer data only between themselves, we're going to need some outside connectivity. We need to peer with all the major networks. So we try to set up peering agreements with ATT, Level 3, and UUNet. The problem is, they have no need to transmit data over our networks, so we have a huge imbalance. We have to pay for every single bit our customers transmit.
It's time to rethink our 100 mbps bandwidth policy, is it not?
Just because we can transmit at 100 mbps to the customer, doesn't mean we can afford for them to use all that bandwidth all the time. There is a cost associated with transferring data over partner networks, and someone has to pay that cost. Comcast's residential pricing model does so in a way that is dead simple for consumers. If you don't fit that model, you should purchase business service.
We have to pay for every single bit our customers transmit.
What about all the bits your customers receive? You sound like you're at least somewhat familiar with peering agreements, so I'd like to know how that part works. If your customers are mostly receiving instead of sending, wouldn't those bits be paid for by the ones sending the bits?
For that matter, why can I get 200GB/mo at 50mbit/s plus a server for $20 from a VPS provider, and $.15/GB beyond that ($.10 if purchased in advance in the case of Linode)? Why do consumer ISPs need to charge so much more?
Based on your arguments in this and other posts, they have caps because they have to pay for the outgoing bandwidth. Why not just charge $.15/GB over the cap? Last mile infrastructure doesn't account for the difference, since it has a mostly fixed cost of construction and operation. Interfering with other oversubscribed customers can't explain it either, since everyone gets a fair and equal share of the pipe when the pipe is overloaded.
You're definitely making the best counter points in this discussion. Unfortunately, I'm only familiar with peering agreements in so much that I can read about them on Wikepedia and follow links. The peering agreements are private, so we don't know who pays who.
Your questions regarding the fees for bandwidth are especially relevant. More things I can't address, because I don't know Comcast's costs for transport or last mile.
When I say that consumers won't/don't want a metered internet plan, I'm basing this on what's in the market and basic consumer tendencies toward simplicity. Metered services have a tendency to make customers feel "nickel & dime'd". Put plainly, consumers will choose a flat rate service over a metered service, even if the metered service would result in a lower bill [1]. It's basic risk aversion.
I'm of the opinion that "free market" ISP solutions aren't working out all that well. The necessity of public land-use rights in order to reach the last mile severely limits competition in the ISP space. I think that a good first step toward fixing the problem would be forcing some more transparency on ISPs. I'd love to know what's going on behind closed doors. I'd love to know who pays who. I'd love to know just how Comcast's transport costs break down.
Until we know some of these details, it's mostly speculation (yes, especially on my part), but I think there are some reasonable assumptions to be made.
For that matter, why can I get 200GB/mo at 50mbit/s plus a server for $20 from a VPS provider, and $.15/GB beyond that ($.10 if purchased in advance in the case of Linode)? Why do consumer ISPs need to charge so much more?
The ISP has to maintain and upgrade thousands of miles of physical plant to deliver data to customer's homes in the geographic regions they cover. For a typical cable ISP for instance there's about 3-5 miles of copper between your modem and the the fiber node and typically dozens of miles between the fiber node and the provider's headend/datacenter where they uplink to the Internet -- which incidentally for smaller rural ISPs may itself be hundreds of miles away from competitive rates on data. Going back to the cost of maintaing/upgrading the infrastructure a small plant of only ~1000 miles typically requires at least a dozen full time employees simply to maintain it day to day and that's not including things like management, network engineers, customer service, technical support, equipment, contracted labour, customer installation, etc. On top of that you've got the cost of leasing access to the poles/underground conduits and the cost of powering the 3-5 miles of plant plus whatever operational costs you have to put gas into your fleet of vehicles, the cost of the vehicles themselves, insurance, etc. When you put this all together you're looking at some pretty huge overhead on the consumer side of Internet access plus whatever profit the company wants to make to justify all of the above. It's not a charity after all.
That still doesn't justify the 250GB cap. Once the line is installed and maintained, using it more or less doesn't change the cost of installing and maintaining it. At that point we're talking about bulk bandwidth, which if you live in a metropolitan area, cannot cost significantly more than what Linode is charging.
In the case of cable Internet you're dealing with a shared medium.
Your ISP may have a 10Gbit uplink in their headend but that does not extend to your home. In reality with 4 channel DOCSIS 3 bonding the speed available "on-the-wire" is about ~150Mbit/sec and it's shared between about 200 customers at minimum. (in reality it's usually a lot closer to 300-500 customers) So if 15 customers out of the 200 want to use 10Mbit/sec 24x7 there is no bandwidth left over for the other 190 customers.
Just to be clear this part is not strictly an infrastructure challenge as the physical cable plant is not inherently limited to ~150Mbit/sec. It simply depends how many DOCSIS channels are being sent out and how many customers share them. They originate from a CMTS in the provider's headend. A fully loaded CMTS sets you back about $500-$750k. So cable ISPs figured out capping data can save them quite a lot of money. The alternative is a) they make less money b) they charge the customer more. We know A certainly isn't going to happen in a non-competitive market and B is something most customers would not like so we get c) capping.
Telco or FTTH are a completely different beast though. Much harder to justify capping there. For telco it's usually because there is a limited uplink to the DSLAM. As of a few years ago it was not uncommon to have a few T1s feeding a DSLAM. FTTH is rarely capped because the fiber that goes to your home is capable of pushing 10Gbit/sec with the right optics. WISPS have the same issue as cable -- in fact WiMAX is very similar to DOCSIS and some WISPs actually do use DOCSIS over wireless at 10Ghz.
In the case of cable Internet you're dealing with a shared medium.
Your ISP may have a 10Gbit uplink in their headend but that does not extend to your home. In reality with 4 channel DOCSIS 3 bonding the speed available "on-the-wire" is about ~150Mbit/sec and it's shared between about 200 customers at minimum. (in reality it's usually a lot closer to 300-500 customers) So if 15 customers out of the 200 want to use 10Mbit/sec 24x7 there is no bandwidth left over for the other 190 customers.
I realize that the cable is shared, but remain unconvinced that a flat cap is in any way better than other solutions (such as dividing available bandwidth proportionally among active users, with higher-paying customers getting a larger slice). It shouldn't matter if someone's using their 10mbit/s 24/7, it should only matter if they're using it during peak hours. On the flip side, the "average" user who wants to stream Hulu, Youtube, and Netflix is likely to be doing their streaming at the same time as most other customers, causing more degradation to other customers' experience per hour of usage than the heavy downloaders. Capping the heavy users doesn't change the fact that the average users all want to stream video at the same time.
Also, how can the cable company possibly offer me the 50+mbit/s connection for which I currently pay over $100/mo if the 4 256-QAM downstream channels (totaling ~150mbit/s as you said) are shared by so many customers? Those channels only use 24MHz of the total spectrum.
I agree the flat cap isn't going to be the end of this. It just happens to be the easiest solution right now and the heavy users are low hanging fruit.
As for the 50Mbit on a 150Mbit bonding group -- that's he magic of oversubscription. It's amazing how little bandwidth people actually use. The number of heavy users who are doing 50Mbit/sec non-stop are really the fringe cases. It's actually quite hard to figure out how to max out a 50 or 100mbit/sec for long periods of time. The traffic patterns are extremely bursty. For example a single uncapped modem's traffic. (in MB/sec)
Let's say you and I go in to business together and build a microwave wireless internet service. There is commodity equipment available that can push 100 mbps pretty easily. We buy a microwave transmission tower and central office from a company that is going out of business at a rate that cannot be beat. We price out our customer premises equipment so that we make half our money back on the setup fee, so setup costs are minimal.
We're now ready for business. We can provide 100 mbps internet service to customers with minimal equipment cost. Sweet!
Here's the rub. Unless our customers are content to transfer data only between themselves, we're going to need some outside connectivity. We need to peer with all the major networks. So we try to set up peering agreements with ATT, Level 3, and UUNet. The problem is, they have no need to transmit data over our networks, so we have a huge imbalance. We have to pay for every single bit our customers transmit.
It's time to rethink our 100 mbps bandwidth policy, is it not?
Just because we can transmit at 100 mbps to the customer, doesn't mean we can afford for them to use all that bandwidth all the time. There is a cost associated with transferring data over partner networks, and someone has to pay that cost. Comcast's residential pricing model does so in a way that is dead simple for consumers. If you don't fit that model, you should purchase business service.