I did exactly that and can't recommend it highly enough. I finished the CS major in my first 2 years and then spent the next two years taking "independent study" courses and just generally hacking and doing research for a few different faculty members.
Protip: you can take "freshman physics/chemistry" as a senior if it's not a pre-req for anything else you need. And by then, you'll have learned to study more effectively than when you were a freshman and it'll be ~2 hours a week of work.
Yeah, I did something like that -- put off a digital electronics lab course until my last semester. I was so engaged in my pet project by that point that I almost completely blew off the lab course. They were very generous to give me a D so I could graduate :-)
I just graduated and did a similar thing. I wrapped up everything but my thesis before my senior year and spent those two semesters taking fun courses I didn't need like Intro to Nanotechnology and rigorous philosophy classes. The only regret I have is that I didn't take those classes earlier and thereby missed out on upper divisions in those departments.
That said, at my university your major only takes up 1/3rd of your overall credit hours so I had tons of time for electives and courses far outside CS.
Protip: you can take "freshman physics/chemistry" as a senior if it's not a pre-req for anything else you need. And by then, you'll have learned to study more effectively than when you were a freshman and it'll be ~2 hours a week of work.