I agree. A friend of mine does a lot of research on Congress and says that getting rid of closed-door committees and committee votes has really increased the power of the parties for reasons you mention: it's hard to negotiate in good faith when people (e.g. lobbyists and party officials) are constantly looking over their shoulders. Earmarks and other things can give them ways to work things out when in private, compromising here and there, and coming up with a bill that will be balanced in the end.

I strongly suggest his research[0].

EDIT: Oh, and my friend[1] who does a lot of the research is a ex-NASA scientist, which is one of the reasons I also got excited about his research, as I studied electrical and computer engineering in college and was excited to see an engineering mindset applied to political dynamics.

[0]: congressionalresearch.org

[1]: https://congressionalresearch.org/JamesDangelo.html