Hi, my name is Julius and I performed the said benchmark while being an intern at FB.
Yes, we used opcode cache in PHP. I definitely don't consider myself to be an expert on PHP configuration. I used the default settings. I was obviously also very suspicious of the results that we got. To the extent, that I consulted a few people about what could be improved about the PHP setup. I remember opcode cache was the first thing I checked, since it's something everyone knows and talks about. It was turned on.
I will update the blog post first thing tomorrow (sorry guys, it's the middle of a night here in Poland, so don't feel like doing it right now).
The only other place where I could have possibly screwed up the results is compilation. I had to build PHP from source, since I wanted to have the "latest greatest" and CentOS has dated packages at times. Though again, I remember just following the official instructions.
If you would like to have the test re-run, you'll have to harass some real FB engineers, since the test was performed on a devbox that I don't have access too anymore. You're also more then welcome to try it yourself and post some alternate benchmarks.
I think I can promise to link to your results from the blog post.
I used the php.ini file that comes bundled with the sources. I just think after I ran the benchmark the first time, I enabled opcode cache explicitly just to be sure. I used the Wordpress setup that comes in the sources. Followed the on-screen instructions. That's it.
I tried tweaking the nginx setup, especially the tcp_nodelay & keepalive_timeout settings. It provided maybe 15% speed up, nothing dramatic. I probably didn't collect enough datapoints to be able to reliably tell. In the end, I sticked to the nginx defaults.
There are a couple of important features still missing from the initial implementation. The most notable is support for local UNIX sockets. At present, only less performant (however more flexible) TCP sockets are available.
This note hints at a missed opportunity to employ what some like to call "unix factoring": the listen(2) call can occur outside of your FastCGI worker program, which then only has to call accept(2) on the inherited descriptor, and implement the FastCGI protocol.
Why do it this way?
1. Beyond the accept(2) call, your worker program is now network agnostic.
2. Guess what, you can now test your worker program by feeding it FastCGI protocol data directly on stdin. Just add a mode that uses stdin instead of the fd returned by accept(2).
3. For easy TCP socket support, run your worker under something like this tiny program [1]. For easy Unix Domain socket support, run your worker under something like this tiny program [2]. Your worker program doesn't change because the networking concern has been separated out.
4. If you want, you can put a program which keeps a preforked worker pool between the the listener program and the worker program. This separates out the concurrency concern.
5. No real performance downside; listen(2) is called once for a long-running server process.
Please, if you do performance benchmarks, always run them with an opcode cache (PHP 5.5 + opcache would be the obvious choice). Without it your benchmark becomes completely meaningless. I also feel like HHVM devs should be aware of this, so likely the numbers are deliberately misleading.
Apart from that, having FastCGI support is of course nice.
Or at least say upfront if you are or not. I think it's not exactly unreasonable to compare "out of the box" to "out of the box", but it should be clear that's what you're doing.
Really I think this sort of thing should be encouragement to make the opcode cache no longer a default-off additional thing. There's no good reason to have a php deployment -- even one that's just for dev purposes -- that doesn't use an opcode cache.
I wish they would be implementing more of the runtime support in PHP itself. Why not make the FastCGI[1] code in pure PHP (like others have[2])? I understand if they don't feel PHP is the best language for daemons, but it would allow so many other developers to use, modify, and comprehend the implementation. I think HHVM suffers from a serious lack of dogfooding.
Miniscule improvements are worth large amounts of money at Facebook's scale. They have an excellent team of C++ and compiler developers working on HHVM, and they see no reason to waste time attempting something in PHP when they could make it run faster in native code without much more initial effort (and probably less effort over time).
I would hope that their JIT'ing VM could come close to native performance (and rarely faster as tracing JITs can occasionally be). It's about drawing the line somewhere. It's like choosing between Phalcon and Symfony. I understand their internal needs at Facebook, but considering the effort they are putting in to make non-Facebook codebases work I don't think it's too much of a stretch to implement some of the runtime in PHP.
Many of the older extensions that were imported back in the HPHPc days are in C since they were much easier to port from php-src if we didn't have to translate the language.
As for why this was't in PHP, two reasons. It was easier to stay in the language since our event loop code was already in C++, and most of the gains we get from writing library code in PHP comes from not having to cross a language barrier from userland code. The event loop is not the best place for that.
because PHP is "slow" and c is "fast", just about anything thats written in PHP such as FastCGI parser would be daftly slow compared to a counterpart written in c or c++. Most people will not be prepared to sacrifice perfomance just for the sake having everything in PHP
Has something changed? The last time I went to use hiphopvm it didn't even have proper support for php 5.4 (broken namespacing and traits, too I believe). That said, it was basically unusable.
(HHVM alum here). Sorry you had a tough time. What's changed is that time has passed, and the HHVM team has put a lot of effort into closing the compatibility gap. If you are aware of a bug with namespaces or traits, let us know at https://github.com/facebook/hhvm/issues
When did you try it last? We spun up a whole team (which I'm on) who's entire goal is to make all of the world's PHP code runable on HHVM. All of the top 20 PHP frameworks work on HHVM now.
I use HHVM in prod. I love it, well done :) I'm actually putting together an app dev "template" for composer to be used with HHVM, and a few compo sable libraries. Not a framework, but good libs and best practices.
hm cant seem to get it to work with Nginx under Ubuntu 12.04
I followed the instructions and the hhvm seems to be running fine but i get a NOT FOUND from nginx.
I basically just changed fastcgi_pass 127.0.0.1:9000; (was php5-fpm) before but the hhvm logfiles are still empty.
Any heads up ?
Trying this on a Symfony 2.4 project.
Yes, we used opcode cache in PHP. I definitely don't consider myself to be an expert on PHP configuration. I used the default settings. I was obviously also very suspicious of the results that we got. To the extent, that I consulted a few people about what could be improved about the PHP setup. I remember opcode cache was the first thing I checked, since it's something everyone knows and talks about. It was turned on.
I will update the blog post first thing tomorrow (sorry guys, it's the middle of a night here in Poland, so don't feel like doing it right now).
Cheers.