This largely matches my experience - as a software engineer, I spend probably < 1% of my time doing math more complex than arithmetic and simple mathematical logic - but it's missing something crucial:
In a market economy, basically all returns come from marginal gains. The vast majority of your lifetime income will come from a dozen or fewer opportunities that you happen to be in a position to take advantage of, whether it's a new job offer or a high-profile project you volunteer for or a startup that takes off. You will qualify for those opportunities based on the skills you have that other people don't have. They will make money for the organization because of features or insights that your competitors lack. Your customers will buy it because it lets them do things that they couldn't otherwise do.
The stuff that you and everyone else spends 99% of your time doing is economically irrelevant. You probably still need to do it (though if you can program a computer to do it, you have a huge leg up on competitors), but it doesn't get you anywhere.
Ironically, this is one of those insights that a good understanding of math will give you. The common-sense understanding is that we should be teaching what the majority of people are doing; the data says that we should be teaching what the majority of people are not doing but desire the results of.
There is more to math than just calculations, algebra, geometry, etc. Logic, and reasoning through choices & the effects of those choices are vital math skills as well, and for many people in math, they don't manifest until one takes a few rigorous proof-based classes, where one often sees new approaches to arriving at concepts they know, and how to properly get to a conclusion from a particular point.
I wish these skills were emphasized more earlier in people's education, they are important skills for maximizing success IMO.
For software engineers specifically -- every time you are coding formalized programmatic logic, you are using math. If you took a logics and proofs course, it would help you formalize logic better. Every time you write a "for" loop you are essentially using summations.
There is a book called Concrete Mathematics and one of the primary authors is Donald Knuth, basically it's "Programmers math" and in my opinion, would make your average developer who completes the text into an exceptional developer.
>every time you are coding formalized programmatic logic, you are using math
Almost no one (outside of those who were friends with math majors) are actually familiar with the nature of proof-based mathematics. When you hold out "programming is math" the general public, policymakers, admissions offices, kids who might want to be programmers, etc. don't make the association to Analysis and Abstract Algebra, they make it to high school Algebra, Geometry, Algebra II/Trig, Precalculus, and (maybe, for the better kids at richer high schools) Calculus.
This has the effect of selecting programmers for their skill, patience, and diligence in executing the algorithms they're given. This is exactly backwards!
In my opinion, if you're looking for an analogy to a high school class, programming is chemistry. You're given a reasonable number of facts about the world, some lab and algorithmic procedures, and a problem. How to string together your intuitions, information, and procedures to arrive at an answer is entirely up to you.
What you speak of is a failure of mathematics education, frankly. I meet too many people with this viewpoint, and regularly have to explain that there's all sorts of math out there that would align with one of their skills. People are not taught mathematics from a problem solving interactive puzzle perspective, they're taught it from a rote computational perspective, removing all the sense of wonder and pleasure from doing things within a system of axioms.
I see the argument about chemistry, but I do think there's a very strong crossover of doing math and building software (in the large). In math, you begin with a set of base known statements. Whether starting from the axioms, or working within some system, you begin to build a vocabulary of the system, a feel of the way objects within the system interact, and an understanding of the limitations of the system you're currently in. Through the course of needing to get things done, you combine these various abstractions together to create more useful ones, constantly making your system a little bit more powerful. Often, taking steps back from your system and combining things into elegant ways gives insight into other patterns present in the system, and recasts solutions in slightly different lights with hints as to what they could have been, had you known what you know now. Then over the course of time, you begin to bring in new foreign systems, which you might not understand fully, and you use inelegantly. But again, working through issues, you build up that vocabulary and intuition to solve problems in your space, constantly putting tools and tricks into your belt on how things could fit together. Occasionally something will be insufficient in your tool belt, or a concept you weren't quite clear on will come up in your problems, causing you to go back, learn a thing, and come back armed ready to keep making progress.
To me, that all sounds like what I do when I'm building software.
>Almost no one (outside of those who were friends with math majors) are actually familiar with the nature of proof-based mathematics
Hmm...when I was a CS major, a discrete math course was required, and it was all proofs. I believe that is still common. I don't find the proof aspect that relevant to programming, but the concepts of discrete math, such as sets, graphs, definitely are.
As far as things that aren't programming go in relation to programming, contest math, with its focus on efficient and creative problem solving, is probably more similar to the daily experience of writing software than any course I took. The exception would be Math Modeling and other courses that actually required programming.
> This has the effect of selecting programmers for their skill, patience, and diligence in executing the algorithms they're given.
Our high school experiences couldn't have been more different.
In math class we were given the definition of a limit and derived all of the rest of high-school calculus collaboratively from there. Similarly for high-school linear algebra: couple of quick axioms and then all further theorems and algorithms derived from there.
Meanwhile in Chemistry class they'd give me a failing grade because I failed to memorize the exact charges of the various polyatomic ions and solving the rest of the problem using variables in place of the charges wasn't good enough.
You seem to describe a very different world of programming than the one I live in.
I have no dislike for logic, formal proofs, category theory and such: I can live in that world just fine. However, almost none of the problems I've ever had to dealt with in my 15 years programming made me wish for large amounts of that: I've used bits here and there, but going all in never made any sense.
The hard part of most tasks I had to do in professional programming is to gather enough information to even begin to write one of those formal specifications: Knowing what real business needs are, understanding how distributed systems fail, working around bugs, reading library code quickly and effectively, learning how to factor code in a way that would evolve well with changes in what the application needs to do... that's where the value is. 99% of that is talking to people and making educated guesses about the future: The implementation itself is trivial. With that, was able to get solid code working in languages that were ill equipped to handle any of the fun pieces of math that you'd need to write mathematically-minded programs.
If anything, the experience I've had working with people that were heavy on the math has been dismal. I worked with a rather well known dev who has published books, gives a lot of talks in FP circles, and is a lead dev in a huge mathematically driven library. We worked on on a rather novel distributed systems. He hid in a corner with a copy of Mathematica for a couple of months, and came back with a paper that proved a well performing solution to one of the things we had to do with our distributed system: Great!... Except, not so, because his proof only worked in a mythological environment where there is no latency between nodes. Under real conditions, the algorithm's properties led to unacceptable results, and he just hid behind the proof.
I don't have papers and patents behind me. He knows a lot more category theory than I do. But It took me a day to implement a completely unproven algorithm that had far better performance properties in a real case. This is not due to any special brilliance of mine, but because I sat down and approached the system like a scientist or an engineer would, instead of like a mathematician.
In my opinion, teaching the scientific method and the basics of engineering in other fields will serve your average programmer far better than being better at math ever will.
I disagree, as someone coming from a math background - from no formal CS training, but 4 years of math at a top graduate program, I fully believe that a large percent of my peers from there could be extremely successful developers. In fact, I ran into a fellow former peer once at a conference who joined Google after exiting academia - he is currently on the Angular team. To take myself as an example, I have been very successful as well, currently being a lead developer who mentors/helps other developers on a near daily basis.
Formal mathematics training isn't necessary to be successful - many of us know many people who have picked up the intuition that those formally trained understand at a deep level & have been able to use it to great advantage. However, I argue that more training in areas of logic & reasoning would serve people well in all facets of their lives, including helping people be successful in software development over the long term. I know that I use my knowledge I gained from decades of mathematics to code strong abstractions faster than just about everyone else I have worked with so far (I've made mistakes too, but nothing terribly costly & unrepairable) - it helps me nearly every day I code & learn new things.
I'll believe that there is a correlation of more than 0.5 to 0.7 between good at software development and good at abstract math. But hardly higher -- there are also some other talents needed, I don't know what.
I saw a few people that had no problems with the math courses in CS but that never really got the programming part. (No names or details, people might be recognized.) But this is of course anecdotal.
(My surprise over this was similar to when I realized that rationality and intelligence didn't have as high correlation as I expected. I know some intelligent people which are Truthers and believe in other conspiracy theories. Others can't reason when their emotions are involved.)
> But hardly higher -- there are also some other talents needed
I'm not going to deny the presence of some built-in preferences for formalized thinking among some subset of the general population, however the cognitive resources and preferences that would lead one to enjoy or excel at math, are about the same as those correlated with programming. e.g. a noticeable interest in understanding mechanical systems.
So given that set of preferences that would produce an interest in math/cs, if a person was raised by a mathematician parent, they'd likely be "good at math", but possibly still struggle with programming when they reach their CS classes in college. Meanwhile another person with the same set of preferences but raised by a hacker, might be good at programming by the time they reach college, but still have a hard time with math.
The issue is less about "can this CS person be good at math?" or vice versa, and more about "will this person be motivated to learn these complementary skill-sets?". Because it's not just a one-way street, mathematicians can also learn from thinking programatically, as shown by the growing popularity of automated theorem provers like Coq. And mathematics being helpful for programming is nowhere better epitomized than by the huge trend of nearly every popular language slowly adopting functional features (and even "optional" typing), because everyone is slowly discovering that mathematically sound programming principles actually are practical (just a bit more abstract and difficult to grasp at times).
Both of those examples should be pretty good evidence of these two fields being pretty helpful towards each other. The large show-stopping issue here making this relationship non-obvious, is education. If complex subjects that are already difficult get taught poorly, it's only natural that we don't expect people to make connections and think "oh hey, this [math/cs] concept I learned about might be helpful in this [totally unrelated] situation".
P.S.: as an anecdote, I just recently used my (minimal) category theory knowledge to change the way I take notes, so that I can smoothly use outlining tools (like org-mode) more like mindmaps when taking notes on complex topics.
> P.S.: as an anecdote, I just recently used my (minimal) category theory knowledge to change the way I take notes, so that I can smoothly use outlining tools (like org-mode) more like mindmaps when taking notes on complex topics.
I, and likely many others, would love to see a writeup of this if you ever felt inclined to write one.
I thought so too, it was surprising to see intelligent people struggle for years with the simpler parts of programming.
Maybe those guys were just unmotivated? Or they were hardworking idiots which had gotten a good math background by sheer will and hard work? It didn't seem so.
The alternative is the existence of basic cognitive talents you need for coding which aren't needed for math? Like for us color blind, that really can't appreciate some art?
None chimed in with support, so I'd guess my sample was unusual; a lot of the guys here should have teaching experience from academia.
(Category theory makes for better Org Mode? I'm curious, too!)
When all you have is a hammer, the world is full of nails. In my experience, knowing more math will allow you to tackle harder problems in a more sophisticated manner.
> Great!... Except, not so, because his proof only worked in a mythological environment where there is no latency between nodes. Under real conditions, the algorithm's properties led to unacceptable results, and he just hid behind the proof.
Then he did a bad job. His failure does not undermine the power of taking a formal approach to a problem. He should have accounted for this latency in his initial model. What he did is the equivalent of someone working on an autonomous car and assuming that a noisy GPS signal is good enough to do control with.
"When DEK taught Concrete Mathematics at Stanford for the first time he explained the somewhat strange title by saying that it was his attempt to teach a math course that was hard instead of soft. He announced that, contrary to the expectations of some of his colleagues, he was not going to teach the Theory of Aggregates, not Stone's Embedding Theorem, nor even the Stone-Cech compactification. (Several students from the civil engineering department got up and quietly left the room.)"
Keith Devlin argues that math training helps when programming, in an article (from 2003) in Communications of the ACM called "Why universities require computer science students to take math" (pdf) ftp://ftp.gunadarma.ac.id/.upload/Communication-ACM/September-2003/p36-devlin.pdf
Sample quote: "Once you realize that computing is all about constructing, manipulating, and reasoning about abstractions, it becomes clear that an important prerequisite for writing (good) computer programs is the ability to handle abstractions in a precise manner."
Logic, and reasoning through choices & the effects of those choices are vital math skills as well, and for many people in math, they don't manifest until one takes a few rigorous proof-based classes
A lot of people I meet don't understand logic and reasoning. A lot of people don't understand rigor. Many people have difficulty thinking systemically or have difficulty thinking about the 2nd and 3rd order effects of their actions. I mean, think about the way people drive and some of the things you've seen other drivers do! People also seem to be much more self centered, less civic minded, and less cognizant of details.
I wish these skills were emphasized more earlier in people's education, they are important skills for maximizing success IMO.
I think American kids would have a better understanding of 2nd and 3rd order effects if they were required to clean their own school bathrooms, like in Japan. (Disclosure: I had to clean my own school bathrooms, but did not go to school in Japan.)
I agree. The majority of engineers may not utilize higher level math. However, all engineers learn how to analyze a problem and attempt to achieve an unknown solution. The solution may include known components that already solve a particular problem, e.g. authorize website users with the Spring Security library. Yet the key is synthesizing these pieces into a new solution for the problem domain. Most mathematics teach students to solve known problems and memorize formulas. Students will get far more mileage by learning to problem solve through logic, reasoning, and synthesis. Not to mention developing the willpower needed to keep working through failures and the social skills to work with others towards solving a problem.
In the spirit of what you're saying - it's possible ten people will be taught undergrad level math and only one will use it - but economically, that one person's added production value could pay for the education of all ten and then some.
The article speaks about Sputnik and one of the world's richest men, mathemetician and retired hedge fund manager Jim Simons notes that he is one of the first students to have benefited from the post-Sputnik STEM student grants. He also mentioned how many people in the US were getting doctorates in mathematics in the year Sputnik went up - I'm looking at the list of names right now and it is much less than 300. So the additional NSF funding etc. that paid for his doctorate resulted in the billions his hedge fund made.
I wrote CAD software for a while and became an expert in Trig ... I have to say that the point where a branch of mathematics becomes intuitive is a real epiphany. I also used mathe pretty heavily working on QAM modulator designs for the cable television industry. And yet in almost thirty-five years of engineering I really only remember using Calculus once (to adjust some tables we used to calculate Bessel filter component values).
I think there are 3 fundamental issues that are coming together. The first is a discounting of the fact that programming and using Excel are in their own way doing higher level math. You're not doing the lower level grunt work but combining those techniques to do something greater. The second being that many problems aren't about mathematical truth so much as mop and bucket work. Designing your process for off hours communication isn't going to simplify into an elegant equation. However being really good at that sort of thing is important. The final thing is most people are lousy at recounting what they did/do. So it maybe that a significant number of people do use advanced math < 1% of the time but don't actually remember it.
>The vast majority of your lifetime income will come from a dozen or fewer opportunities that you happen to be in a position to take advantage of, whether it's a new job offer or a high-profile project you volunteer for or a startup that takes off
Is this really true? I'm still way too early in my career to know. But I'd have thought if you stick with the more "traditional" route, your income would be fairly consistent (as opposed to something similar to the startup lottery, for example).
Even if you stick with the traditional route, having a tiny initial advantage for your first job will have compounding results over your lifetime. This is significant enough that people who graduate college during a recession on average have a 9% lower salary during the first 10 years of working [0].
Many people can tell you about their "big break". That one success that seemed to cascade into other successes. Whether this is a real phenomenon or simply an illusion of memory, I don't know. It's quite possible that if that big event hadn't taken place, there'd have been a similar one just a little later, taking the person on a different, but equally pleasant path. I suggest you not trust individual narratives, only systematic analysis.
The opportunities are transition points, not isolated events. Like moving from 100k/yr salary to 400k or whatever. OP is saying (in my understanding) that the bulk of your outcome will be determined by the magnitude of those transitions.
He's talking about opportunities. For sure, if you keep doing what you're doing, you will have consistent income. But what if, one day, your boss came to you with a novel, difficult problem that no one in the company, and perhaps in your industry, had already solved? The argument goes, having an arsenal of advanced math at your disposal would make you more likely to solve that difficult, novel problem, and set you up to be handsomely rewarded.
You don't often get "handsomely rewarded" for doing what people consider to be your job, even if you do it much better than anybody else. The opportunities are when you get to jump to a different role, getting promoted or going to a different company or using your insight to start your own company.
That's one way of looking at it, but you don't just happen to have those skills. You get them from experience and practice - or doing well at the 99% that doesn't feel important.
This arguement reminds me of people who focus on a key, heroic play in sports, rather than all the good and bad that led up to it.
But if that's the case, the fact that we all learn the same thing(even something as powerful as math), doesn't sound ideal at all, and a multidisciplinary knowledge strategy should serve us better, right ?
The author addresses exactly this point. (Whether or not you agree with the author is a separate issue, but your comment seems to indicate you didn't reach that part of the post.)
In a market economy, basically all returns come from marginal gains. The vast majority of your lifetime income will come from a dozen or fewer opportunities that you happen to be in a position to take advantage of, whether it's a new job offer or a high-profile project you volunteer for or a startup that takes off. You will qualify for those opportunities based on the skills you have that other people don't have. They will make money for the organization because of features or insights that your competitors lack. Your customers will buy it because it lets them do things that they couldn't otherwise do.
The stuff that you and everyone else spends 99% of your time doing is economically irrelevant. You probably still need to do it (though if you can program a computer to do it, you have a huge leg up on competitors), but it doesn't get you anywhere.
Ironically, this is one of those insights that a good understanding of math will give you. The common-sense understanding is that we should be teaching what the majority of people are doing; the data says that we should be teaching what the majority of people are not doing but desire the results of.