IMO, the problem isn't with "just editing a quick file" or not.
The problem is, IDEs are amazing for everything other than editing text. This includes editing source code, mind you. I find it much easier and funner to edit code in vim, since it has so many small things that make editing code much faster.
IDEs are generally much slower for the actual act of editing code than things like vim. They're just better because of everything else they do, which (sometimes) makes up for it.
Also, most people care a lot more about all the other things like code completion than they do about the small improvement you can get by editing in vim/emacs.
Define most people. Personally I spend the majority of my day editing text, so the small improvements are pretty important.
P.S. I have code completion, jump to file etc all working just fine in vim. On a codebase with 15000+ files, this is a hell of a lot faster in vim than it is in say, Eclipse.
The problem is, IDEs are amazing for everything other than editing text. This includes editing source code, mind you. I find it much easier and funner to edit code in vim, since it has so many small things that make editing code much faster.
IDEs are generally much slower for the actual act of editing code than things like vim. They're just better because of everything else they do, which (sometimes) makes up for it.
Also, most people care a lot more about all the other things like code completion than they do about the small improvement you can get by editing in vim/emacs.