Wordpress HHVM Benchmark

[TOC]

I have a Wordpress site running for our brewing project, and I though it would be fun to see if a very basic benchmark would show any changes between two different ways of running PHP code.

My current setup is Nginx and PHP-FPM and I will test against Nginx and Facebooks Hip Hop VM.

Since I already got the site running with PHP-FPM I will not use anytime to explain how to set it up, both setups will be using pretty much standard config, not any special tweaking.

First, install ab:

apt-get install apache2-utils

The tests

The tests will be run from the same machine as the Nginx is hosted on.

The specifications for the server is:

Intel i7 2600 3.4 ghz
16gb RAM
Intel 330 120gb SSD

I will try to run the test with this different settings on each PHP backend:

Test 1 Concurrently (-c): 2 Number of requests (-n): 1000

Test 2 Concurrenty (-c): 5 Number of request (-n): 1000

Test 3 Concurrenty (-c): 200 Number of request (-n): 100000

Installing HHVM

My server runs Debian 7 Wheezy and I will, therefore, show how to install HHVM on this system.

Add the repositories and install hhvm

wget -O - http://dl.hhvm.com/conf/hhvm.gpg.key | sudo apt-key add -
echo deb http://dl.hhvm.com/debian wheezy main | sudo tee /etc/apt/sources.list.d/hhvm.list
apt-get update
apt-get install hhvm

And since HHVM gives us a nice install script which takes care of Nginx we just issue:

/usr/share/hhvm/install_fastcgi.sh

Running the tests

I actually got some interesting unexpected numbers. I guess that most of them can be changed with some tweaking.

Test 1

PHP-FPM:

Time taken for tests:   56.205 seconds
Requests per second:    17.79 [#/sec] (mean)
Time per request:       112.410 [ms] (mean)
Time per request:       56.205 [ms] (mean, across all concurrent requests)
Transfer rate:          401.71 [Kbytes/sec] received

HHVM:

Time taken for tests:   11.450 seconds
Requests per second:    87.34 [#/sec] (mean)
Time per request:       22.900 [ms] (mean)
Time per request:       11.450 [ms] (mean, across all concurrent requests)
Transfer rate:          1971.13 [Kbytes/sec] received

Test 2

PHP-FPM:

Time taken for tests:   30.708 seconds
Requests per second:    32.56 [#/sec] (mean)
Time per request:       153.540 [ms] (mean)
Time per request:       30.708 [ms] (mean, across all concurrent requests)
Transfer rate:          735.25 [Kbytes/sec] received

HHVM:

Time taken for tests:   5.225 seconds
Requests per second:    191.38 [#/sec] (mean)
Time per request:       26.125 [ms] (mean)
Time per request:       5.225 [ms] (mean, across all concurrent requests)
Transfer rate:          4319.43 [Kbytes/sec] received

Test 3

PHP-FPM:

Time taken for tests:   168.360 seconds
Requests per second:    593.97 [#/sec] (mean)
Time per request:       336.719 [ms] (mean)
Time per request:       1.684 [ms] (mean, across all concurrent requests)
Transfer rate:          960.51 [Kbytes/sec] received

HHVM:

Time taken for tests:   383.876 seconds
Requests per second:    260.50 [#/sec] (mean)
Time per request:       767.751 [ms] (mean)
Time per request:       3.839 [ms] (mean, across all concurrent requests)
Transfer rate:          5879.34 [Kbytes/sec] received

Note: When running the third test, I observed very high load during the HHVM test.

Conclusion

I was a little supprised how much HHVM outperformed PHP-FPM at the lower concurrencies, and I was very surprised that PHP-FPM actually won the high concurrency test. I guess that this can be turned around with a little tweaking. Both installs are without anything but stock install, both can probably do better.

I will also let the test site run HHVM from now on, I will not, at first, tweak anything, as I don't get any amount of hits that will need this.

An other interesting observation from my side is that the site actually feels snappier. This may only be in my head.