Wouldn't it be yet another optimization to just include that loadScript function in the HTML? Reducing round trips to the server is a common technique in web site optimization.
I realize that script will be cached after the first time, but it's so small that it will hardly affect the performance of subsequent requests anyway.
Google conveniently hosts and distributes most of the common Javascript libraries through their CDN, and allows external developers to load them using this technique. http://code.google.com/apis/ajaxlibs/
It's probably faster to serve them off your own server due to HTTP keepalive.
Sure, making a new connection to google's CDN might be faster than making a new connection to your server, but it's definitely not going to be faster than sending a little more data over an existing connection. Not to mention the extra DNS lookup.
This is assuming that the script file and the page it's included in are coming off the same server. Most people who are tweaking performance at this level have probably already separated the serving of static and dynamic content.
This has nothing to do with googles hosted libraries but it is javascript loaded from google servers:
I've had both analytics and adsense slow down page loads quite a bit. Analytics tags should always go as far to the end of the page as you can manage, with adsense tags you're bound to put them where they display the ad. 3rd party javascripts are a pain, it's an extra point of failure that is not under your control.
If I'm not mistaken, at least Drupal has a module that allows you to host the google analytics JS files on your own server. This would lead me to believe that you can do that sans Drupal, though I've never attempted it (in either fashion actually.)
Not if you put it at the bottom. Yes, it does block the page, but hopefully your page will degrade gracefuly until the js is loaded. If it doesn't, then you want it at the top, where it will block. It will load slower, sure, but people won't have to see a broken site.
I realize that script will be cached after the first time, but it's so small that it will hardly affect the performance of subsequent requests anyway.