Absolutely not for a first time, you're right. For me though, as I became more comfortable with the altered psychedelic state of mind, being around people became much easier. I wouldn't want to be trapped at a party, but I've stopped in at them for 30-40 minutes, had fun, and then left to move onto the next thing.
The important part for me is to have options, whether that's a ride back home, a nearby park, someone to take a walk, an Ipod and some headphones, etc.
Shrooms and LSD is not a party drug. I repeat. not a party drug.