How do you load balance a Flask application since it's single threaded? For optimal performance shouldn't there be a single Flask instance (process?) for each request being processed? What kind of load balancers are usually used with Flask and how do they decide to spawn new Flask instances?
And why should Flask be used instead of asynchronous Tornado or Node.js?
Typically, one uses an application server container such as uWSGI for a production deployment of a Flask application.
uWSGI allows for brokering requests across multiple application processes and/or threads, and also allows for more interesting and complex scenarios such as zero-downtime deploys, "automatic" scaling of processes/threads based on load characteristics, to name a few.
> And why should Flask be used instead of asynchronous Tornado or Node.js?
Tornado's asynchronous request handling is quite nice, but it also forces you to ensure that every 3rd party system you interact with has an equivalent async-compatible library; otherwise you end up blocking a request while waiting for a response, thus defeating the whole purpose of Tornado. Most systems (e.g. Postgresql[1]) have async-compatible drivers, but it's not always the case.
As for why not node.js, well, perhaps a better question would be: How would Node be an improvement over Flask+uwsgi? What are the associated infrastructure/architecture/cultural costs?
> "automatic" scaling of processes/threads based on load characteristics
How is this done in detail? In the case of Flask does it create new processes or threads? How quickly does it create a new Flask process/thread to serve a new request?
EDIT: This website documents configuration settings for autoscaling: http://uwsgi-docs.readthedocs.org/en/latest/Cheaper.html - looks like it's based on various heuristics and algorithms, and in practice it's too slow to create a new process for each request, thus wasting resources
I would think that ideally the number of processes/threads would exactly match the number of live requests, otherwise you're either having too few instances to handle all the requests as fast as possible or having too many instances which means that more resources are used than necessary, thus increasing costs.
I think Node.js is an improvement because the architecture is better suited for scaling and all the dependencies are asynchronous by default. Of course, I think there are other reasons for Node.js (e.g. isomorphism, easier full-stack development etc.), but I guess that's another discussion, really.
A count of the flags that you can pass to uWSGI shows 976 different options. It's highly configurable. WSGI itself is a synchronous protocol, but (as icebraining pointed out) uWSGI can be run in async mode [0] which ends up looking much like node.js.
In terms of the ideal number of processes/threads, I'm not so sure number if live visitors is correct. It's going to depend on your various resource constraints but if requests are quick to service there's no problem with having other requests waiting in a queue.
I once was called in to fight a fire where the team said, "we don't understand, we have horizontal scalability, but adding more machines seems to be making things run slower!" - Erm, yeah, because your single db server is on its knees :-)
Edit to add that I've just looked through the cheaper docs you mentioned and that behaviour all looks pretty sane. You can have a good default number of workers and expand as required. Also, uWSGI forks workers after the app is loaded in memory so you get fast copy-on-write behaviour during worker start.
Of course, this doesn't change your app and its dependencies to work in an async-friendly way, but then again, Node.js callbacks are hardly transparent either.
Well, the difference is the entire ecosystem of nodejs is built around async. Fitting it into python is very hard, and you better have a really good reason for trying it. It just doesn't interact favorably with a lot of common python tools (Sqlalchemy being one example)
This. Flask can technically run in an async event loop (see:
http://docs.gunicorn.org/en/stable/settings.html). But gevent is a massive monkey-patch on top of Python standard libs to make them async friendly.
If you're hitting enough load where async really matters, just skip Python altogether. And please, please, don't use Node. Just use Golang or Scala for true multithreading, and you'll save yourself the headaches Javascript will bring for something that's not truly parallelized.
I've built with Flask, Tornado, and with Node/Express.
Flask/Python blows Node out of the water where performance is concerned. That shouldn't be a surprise to anyone.
Flask has a significantly faster ramp up time than Tornado as it is designed to be simple.
As far as load balancing/performance, you front end it with something like NginX.
You can process things out of band easily enough with Flask, but I wouldn't evangelize it so much to say that it's the best solution for every problem.
Personally, I like to keep things as simple and as logically separated as possible until it's absolutely necessary to change. Flask is one of those frameworks that doesn't present a lot of mystery.
My biggest problem with node isn't performance - for that it's fine for most use cases that I deal with. It's that you can get perl-level convoluted and sloppy very quickly and easily.
And my experience is generally also that V8 beats the pants off CPython in performance.
I suspect that part of the problem is that the node ecosystem encourages you to just "npm install" whenever you have a problem, and so you have 3-4 layers of third-party libraries doing minimal work (but consuming lots of CPU cycles) on top of your problem space. But if that's the problem - don't do that. Flask is a microframework; it's trivially easy to build a small microframework on top of the Node.js stdlibs. Node was meant as a scriptable high-performance C++ server, after all, and there's nothing stopping you from using it as one.
It has a range of 723 requests (CherryPy) for a plaintext hello world to 6.8Million (ulib) for plaintext. I looked at the Flask numbers and they don't compare with my own - I got between 3 and 4 times that when I benchmarked it myself on hardware that I'm sure was older and less powerful (and I didn't bother to turn off logging).
I looked at the node code and it does not use express - which could very well be the difference I am experiencing.
I'll tell you what though... I'm going to have a look at ulib for static files!
Flask is on there about 3 times, with different choices for ORM. It's at about 8% when used without an ORM. There's no sans-ORM configuration for Express, though, so still not an apples-to-apples comparison. I suspect the grandparent missed the other Flask configurations in the very long list.
> Flask/Python blows Node out of the water where performance is concerned. That shouldn't be a surprise to anyone.
I'm a huge fan of Flask and Python in general, and despise server-side Javascript (or Javascript anything, really). But I'm curious how you can make this claim. CPython lacks JIT, and gunicorn + Flask is going to run a process pool by default, with each process being single threaded and under GIL.
What's your set up then? PyPy and gevent gunicorn workers? Otherwise I don't see how CPython+Flask+gunicorn could beat Node performance wise out of the box. At best PyPy + gevent would approximate NodeJS performance, unless you can tell me otherwise.
Original post author here. On the single server level I've mostly used Flask behind uWSGI for load balancing, My experience with this has been overall positive (but uwsgi can be a bit of a pain to script deployments for, maybe it's better now though, not sure).
Then there's the further load balancing you can do behind something like an ELB instance in AWS to control load across multiple servers.
I don't know current status, but in the post you would use gevent or similar to async. Also you would put nginx and likely gunicorn in front of the app.
You typically use something like uwsgi or gunicorn to serve requests, and it's uwsgi & gunicorn that decide how many processes or threads to start. These typically sit behind a load balancer and / or a reverse proxy.
In terms of async Tornado / etc, it serves a different need. Like all wsgi servers, it is best to optimize Flask such that you serve a small number of concurrent requests that are fast to prevent a long queue waiting to serve web requests. With Tornado you have more flexibility in that particular area, but also another layer of complexity. Different tools for different use cases.
> Like all wsgi servers, it is best to optimize towards serving a small number of concurrent requests that are fast and end fast to free up the worker pool.
Well, most (or all) web applications I've worked with make database calls to serve a response, which (in case of Flask) locks the whole thread, and there's always the possibility that the database request is slow. That's why I prefer asynchronous web servers where I don't need to worry about this particular problem.
Modern web applications tend to be more like light proxies between the database and other (micro)services. Most logic has been transferred to the client, including templating. Serving static files should be done by Nginx in any case. Therefore I think it's rather critical that the web application does the serving of multiple requests well, and why I think the async architecture is the way going forward.
Well, most (or all) web applications I've worked with make database calls to serve a response, which (in case of Flask) locks the whole thread
I don't get this, when did Flask start connecting to databases? The only point of Flask is that it only provides an HTTP layer, nothing more. If you want to connect to a database, you have to import some other library (or use something from Python's standard library), which may or may not be blocking.
Flask started talking to databases the moment people started using it. It's a web framework. OP's comment was saying it's single threaded nature is a detriment.
Flask is just an HTTP layer, not an execution engine, so it doesn't really have a single- or multi-threaded nature. It's thread-safe, though.
That said, the default development server built-in (which uses werkzeug's underneath) is multi-threaded - just pass threaded=True to app.run(...) - and in production, you can deploy a Flask app using an evented framework like Tornado or Gevent: http://flask.pocoo.org/docs/0.10/deploying/wsgi-standalone/#...
There really isn't a reason, I don't believe, to use Flask over X. It's just as good, and it comes down to your needs.
To answer your first question: you use something like uWSGI. Flask, by default, stores sessions entirely client-side. You'll have to use something like Redis if you want to use more than one server (properly, anyway), but a Python app like this should easily be able to scale. It won't be Flask that stops you, anyway.
And why should Flask be used instead of asynchronous Tornado or Node.js?