I didn't know there was a deep learning gold rush. Maybe this explains the crazy number of stars on my libdeep library on Github, while there being no comments or issues raised.
Deep learning is not any sort of magic bullet. It may be marginally better than other machine learning methods in specific contexts, but I'm not convinced that there are going to be any deep learning tycoons or deep learning entrepreneurs (were there any SVM tycoons?). But I suppose as a buzz term "deep learning" is better than the meaningless "big data". Just replace the latter with the former in the marketing literature.
I've been noticing a slight trend among the same group of people that are likely to use the big data buzzword without really understanding it, to also use deep learning in the same way, to simply mean "gain business insights from [obviously big] data".
As far as I know the only gold rush is around marketing surrounding this and related buzzwords, i.e., it's the latest thing that your business absolutely must be doing to keep up with your competitors. That particular usage of course has as little to do with actual deep learning as the misappropriation of big data has to do with anything.
Fortunately this trend has been a bit slower to take off, presumably because whereas big data is a fairly nebulous concept, it's much easier to correct someone when they talk about deep learning quite wrongly.
That's the exact market I'm cashing in on. But rather than just be someone in the space who talks about it, I made it[1].
I think at the end of the day, the stuff talked about in the media may or may not have some merit (otherwise why would google or these other companies put resources in to it?)
Rather than read the blogspam, read the papers instead though. Try to understand the merits of what's going on and apply it to your use case.
Let me clarify: Techcrunch articles talking about papers being published.
Blogspam to me is just something that talks about the stuff at such a high level, there's no meat in it. Within that subset I'm talking about academic concepts where you can learn a lot more if you just read the papers.
Before I get bombarded here, I think it's best to clarify that the word blogspam is a term thrown around a lot around here.
When the press talks about an academic paper, they tend not to add much value in terms of actually explaining what's going on. I think people drawing their own conclusions from the original works is the better way to go. Obviously not everyone will agree with me here, but so be it.
Broadly speaking, the people who have significant, nontrivial expertise in the field is limited (to a first approximation) to
- students of Hinton/LeCun/Bengio/Ng, and/or
- members of Google/Facebook/Baidu's deep learning group.
An actual deep learning expert can have unrivalled access to infrastructure, excellent colleagues, data, and compensation, at any of Google/Facebook/Baidu etc.
Given this backdrop, it's less surprising that a significant number of the current crop of deep learning companies are unimpressive technologically - those with actual expertise have much better opportunities.
After watching 'Recent Developments in Deep Learning' by Geoff Hinton [1] a while ago I did come away believing that there does seem to be quite a bit of untapped potential in deep neural nets.
Do you feel that talk is overly optimistic/misleading? Or just that you don't think there's that much money to be made with it?
I've also seen that talk. In many cases I think the advantages which may be had from deep learning are likely to be marginal - a few percent improvement over the next best algorithm. Multi-layer neural nets already existed for a couple of decades, and deep learning is really just a refinement on that, making the training process faster and more robust to overfitting.
Do you know of any other system that gets ~14% error rate on the IMAGENET dataset? The next best system as recent as 2012 was doing ~26% error. That is a pretty big gap, especially considering IMAGENET is a million image data set. (btw did not see the talk)
About libdeep: Would be very useful if you had the man page inserted into the README. This way I can make a decision as to if I need this library without having to download/install.
It's not marginally better, it's significantly better in some areas. There were massive improvements on some benchmarks like speech recognition and image classification.
It's not another machine learning algorithm, it's a feature learning algorithm. You can feed raw pixels into an SVM, but you aren't taking advantage of the structure of the data at all.
After seeing the term "gold rush" get thrown around, I compared a 19th century gold rush to a (supposed) deep learning / data-based gold rush in terms of economic terms (e.g. rivalry and excludability): http://djwonk.tumblr.com/post/87513512919/economics-of-a-sup...
UPDATE: My commentary may have conflated "data analytics" with "deep learning". I probably should disentangle them.
The fact that features can be learned means we could see the emergence of machine learning methods that could be used by anyone with a technical background, not just ML experts.
That has already been the case for a long time. There have been many neural net libraries which don't require the developer to understand the details of the algorithms.
1. Machine learning will come to play a more important role in the future (not less)
2. Machine learning is a wide field with many areas of complexity and a wide variety of applications, many not yet discovered
3. Deep learning is an exciting area in machine learning research showing promising (state of the art?) results in several domains
Then I think a few conclusions follow:
1. New tools will address this market, both free and commercial.
2. Services will be a major part of this ecosystem (see history of databases, ERP, CRM) and those consultants both use and sell tools (applications)
3. Deep learning is interesting, quite possibly worth using, and changing rapidly. But it doesn't negate what came before it (or after).
So the answer here is all of the above models will be important, with the application of machine learning to everything really being the larger umbrella opportunity, and deep learning currently being an interesting avenue of research?
No, deep learning really does automatically (read - without much art) learn good features, that's its main purpose. Problem is that deep learning itself is not trivial to use.
WTF is 'the deep learning gold rush'? (Also, the HN title does not match the article title, which is: "Deep Learning Business Models".)
Would love a better explanation from someone than this article gives to support the notion of some kind of massive 'Deep Learning' market that is yet untapped.
> Deep learning requires very large training sets, and these shouldn’t be shipped around a lot.
To the extent that this is true, companies that offer these services may be driven to integrate more closely with customer data. This may involve custom in-house deployments or ways of getting the data in a cost-effective way from, say, Amazon S3, or wherever the data lives (HDFS, etc).
This leads me to speculate on an additional business model, "Behind The Firewall" Software Deployment. This could be somewhat different from the others suggested in the article: 1. Sell hardware; 2. Open source plus services; 3. Hosted API, “Deep Learning as a Service”; 4. Individual deep learning services.
I think there is something to be said for the "deep learning gold rush" I'm one of the ones trying to cash in myself by being an independent player in the space with my own distributed deep learning framework[1].
The goal of data accessibility can be solved by an abstraction layer that auto vectorizes (transforms in to matrices) the needed data at runtime, trains the nets on that particular mini batch of data, and continues on.
That's what I'm trying to do with a concept of a DataSetIterator[2]. This understands how to pull in the data, and handles all the logistics while the runtime only knows about DataSetIterators.
I'm also partnering with a former cloudera engineer in the hadoop space to take on in process YARN deep learning[3]. Data should not be moved. It should be processed and left where it is. I'll be interested to see the innovations in this space in the coming years.
I don't believe deep learning as a service is the way to go, I think behind the firewall deep learning apps will be the way to go here.
For many (most?) analytics tools, the application developer has to specify what gets recorded and what gets fed as an input to predictive analytics. I expect to see more tools that assume any data is fair game; e.g. all data is worth at least a quick look. These tools will probably figure out how to select what is interesting enough to examine more deeply.
Ersatz Labs is in the space, and we're in the process of launching as well [wait till tomorrow ;) ] At the end of the day, companies that focus on apps will be the ones that get bought or cash in. I don't think "machine learning" appeals to businesses. 30% more revenue because of a more accurate model sounds pretty good to me though.
Deep learning is not any sort of magic bullet. It may be marginally better than other machine learning methods in specific contexts, but I'm not convinced that there are going to be any deep learning tycoons or deep learning entrepreneurs (were there any SVM tycoons?). But I suppose as a buzz term "deep learning" is better than the meaningless "big data". Just replace the latter with the former in the marketing literature.