What are the practical uses of a self hosted LLM? Is it actually possible to approach the likes of Claude or one of the other big ones on your own hardware for a reasonable budget? I don’t know if this is something that’s actually worth it or if people are just building these rigs for fun or niche use cases that don’t require the intelligence of a hosted LLM.
Personal opinion, it's for fun with some internal narrative of justification. It doesn't seem like it would be cost effective or provide better results, as all the major LLM vendors benefit tremendously from economies of scale, and the monthly fees for these services are extremely reasonable for what you are getting. Going further, the cloud based LLM receive upgrades constantly while static hardware will likely lock you out of future models at some time horizon.
