I totally see your point and for a lot of people what you're saying will resonate (I can see this really catching on with gradstudents), but the question really boils down to: do you really want your engineers spending time learning these things?
From what I've seen - it's incredibly hard to find a good engineer that knows his stuff. They generally have PhDs and years of training in math and whatever particular field they work on. If you get one that is actually doing novel algorithm development, each one is a golden goose that will bring in revenue for the company for years to come.
By contrast if you offer a decent salary you can get a good C++ programmer with 3-4 years of experience to do all your C++ grunt work. (we hire CS students from the local University, and bring them up to speed within 6 months)
While not incredibly hard, the problem with learning "appropriate structure for long term maintainability and production use" is that it takes an awfully long amount of time. You need to go through a lot of trial and error and learning from your mistakes. You need to read programming textbooks, and keep up with the latest trends. It's actually a huge time investment.
> it's incredibly hard to find a good engineer that knows
> his stuff. They generally have PhDs and years of
> training in math and whatever particular field they work
> on. If you get one that is actually doing novel
> algorithm development, each one is a golden goose that
> will bring in revenue for the company for years to come.
Like ritchiea, I admit I don't have a huge breadth of experience, but I've worked with enough PhDs to say with confidence that, more often than not, they represent a net negative contribution to an engineering team. Novel algorithms aren't useful if they can't run in production, and the PhD's I've known, with one or two exceptions, lacked both the ability and desire to produce production-quality code. I grew to deeply resent those "engineers" who would read papers for half the day, make buggy commits to prototype repos that never got released, and refuse requests from both peers and managers alike to contribute to the team's extant backlogs.
Sounds like a CS PhD! hahaha (maybe Math?)
I want to preface by saying that for most work that goes on in Silicon Valley a PhD adds nothing.
I know what you're talking about.
However outside of SV there are a lot of places where that's simply not the case. What you have is basically a cross disciplinary collaboration where you just don't have the educational background nor the expertise to do the engineering yourself.
The resentment your describing is really the same resentment you can feel towards your boss -"All he does is tell us what to do. He's not in the trenches like us!" - and it's directly related to the amount of mutual respect. In essence, how hard do you feel the other person is working relative to you.
Your description of your coworkers seems to indicate that you didn't feel like they're pulling their weight, and that's definitely a big problem.
I've heard second hand that in the valley there are a lot of incompetent PhDs that use their educational background as leverage to slack off. People will often hire graduates solely based on a degree thinking that if someone has a Math PhD "they're smart, they can learn on the job, and they'll contribute in some magical way just by being there". But actually these guys have just spent 5 years in poverty getting an advanced degree and don't want to be a code monkeys with a bunch of fresh out of college 21 year-olds. And, oh yeah - reality check, no one cares about their degree in abstract algebra.
9/10 of these PhDs are crappy engineers and 9/10 times they are thrown at problems they don't really have a background in (so they can't even be crappy and regurgitate equations they're memorized).
What kind of work do you do where you find that PhD level education is requisite for good work and an algorithm designer is essential to your revenue? Hope this doesn't come off as accusatory, I'm very curious.
There is a whole world of programming outside of web frameworks and cell phone apps haha.
Think of the people that for example made the traction control system for your car. You think the same person that worked out the math in some simulation in MATLAB wrote the code that went on the controller circuit? It's not impossible, but it's not likely.
Closer to home, you can think of companies like Leap Motion, the Oculus folks, the Kinect people. Often companies that deal with video processing or control systems depend on a core set of algorithms to set them apart from the competition. The engineers behind it generally have a high level of understanding of linear algebra and often statistics and 15+ years of experience in a field.
It's very hard (though in many areas not impossible) for a programmer to learn that stuff to a level where they can innovate and contribute - so you end up relying on specially trained people who don't know about inheritance and how to use git hahaha
Where I work algo development is done by about 50% PhDs, 25% MAs and 25% BA/BS.
Thanks for replying. It's not so much that I don't realize there is a huge world of programming outside of web frameworks and cell phone apps, as it's nice to hear about those jobs. I don't get the opportunity very often.
I don't think the bar needs to be that high. Just write a decent API for your magic algorithm, choose decent names and types, tune the inner loops and grok the performance edge cases, write a few unit tests, and I think the 80/20 rule applies for anything much after that. A much better scenario for a handoff to another engineer than "here is my .m file, enjoy porting it to C++." They can refactor it up more and introduce interfaces, etc, from there.
These good engineers who are worth their weight in gold spend a lot of hours jumping through hoops to get Matlab to run in an acceptable time frame. And it makes it hard for anyone to figure out what they've done (with weird index tricks, for example).
From what I've seen - it's incredibly hard to find a good engineer that knows his stuff. They generally have PhDs and years of training in math and whatever particular field they work on. If you get one that is actually doing novel algorithm development, each one is a golden goose that will bring in revenue for the company for years to come.
By contrast if you offer a decent salary you can get a good C++ programmer with 3-4 years of experience to do all your C++ grunt work. (we hire CS students from the local University, and bring them up to speed within 6 months)
While not incredibly hard, the problem with learning "appropriate structure for long term maintainability and production use" is that it takes an awfully long amount of time. You need to go through a lot of trial and error and learning from your mistakes. You need to read programming textbooks, and keep up with the latest trends. It's actually a huge time investment.