Update: There is a part #2 of the benchmark results.
This all began as a colleague of mine stated that Python was so damn slow for maths. Which really astonished me and made me check it out, as my father told me once that he was very satisfied with Python, as it was very maths oriented.
The benchmarks here do not try to be complete, as they are showing the performance of the languages in one aspect, and mainly: loops, arrays with numbers, basic math operations.
Note: Give your ideas and use-cases on what to benchmark, and I’ll try to implement it for you. I.e. “benchmark the languages for reading a file, then splitting it to tokens by white-space and finally outputting all unique elements and their count”.
Out of curiosity, Python was also benchmarked with and without the Psyco Python extension, which people say could greatly speed up the execution of any Python code without any modifications.
Here are the benchmark results:
| Language | CPU time | Slower than | Language version |
Source code |
|||
|---|---|---|---|---|---|---|---|
| User | System | Total | C++ | previous | |||
| C++ (optimized with -O2) | 1,520 | 0,188 | 1,708 | - | - | g++ 4.5.2 | link |
| C++ (not optimized) | 3,208 | 0,184 | 3,392 | 99% | 99% | g++ 4.5.2 | link |
| Javascript (nodejs) | 3,096 | 0,384 | 3,480 | 104% | 3% | 0.2.6 | link |
| Java | 8,521 | 0,192 | 8,713 | 410% | 150% | 1.6.0_26 | link |
| Python + Psyco | 13,305 | 0,152 | 13,457 | 688% | 54% | 2.6.6 | link |
| Python | 27,886 | 0,168 | 28,054 | 1543% | 108% | 2.7.1 | link |
| Perl | 41,671 | 0,100 | 41,771 | 2346% | 49% | 5.10.1 | link |
| PHP | 94,622 | 0,364 | 94,986 | 5461% | 127% | 5.3.5 | link |
The clear winner among the script languages is… Python. 
NodeJS JavaScript is pretty fast too, but internally it works more like a compiled language. See the comments below.
The times include the interpretation/parsing phase for each language, but it’s so small that its significance is negligible. The math function is called 10 times, in order to have more reliable results. All scripts are using the very same algorithm to calculate the prime numbers in a given range. The correctness of the implementation is not so important, as we just want to check how fast the languages perform. The original Python algorithm was taken from http://www.daniweb.com/code/snippet216871.html.
The tests were run on an Ubuntu Linux machine.
You can download the source codes, an Excel results sheet, and the benchmark batch script at:
http://www.famzah.net/download/langs-performance/
Update (Jul/24/2010): Added the C++ optimized values.
Update (Aug/02/2010): Added a link to the benchmarks, part #2.
Update (Mar/31/2011): Using range() in PHP improves performance with 5%.
Update (Jan/14/2012): Re-organized the results summary table and the page. Added Java.
Освен всичко Python е много по-четлив, според мен.
Nice little benchmark. I ran these tests on my iMac with Core i7 2.8 GHz:
While the choice between Perl or Python really doesn’t make any significant difference, to C++ from both, there is a massive gap.
More interestingly, I create a Javascript version of the algorithm and let it run on nodejs:
Which is roughly 9 times (900%) faster than Perl or PHP, and only 3 times (300%) slower than C++. Here is the Javascript version, click “show source” below to see the full source:
function get_primes7(n) { if (n < 2) { return []; } if (n == 2) { return [2]; } var s = []; for (var i = 3; i < n + 1; i += 2) { s.push(i); } var mroot = Math.floor(Math.sqrt(n)); var half = s.length; var i = 0; var m = 3; while (m <= mroot) { if (s[i]) { var j = Math.floor((m*m-3)/2); // int div s[j] = 0; while (j < half) { s[j] = 0; j += m; } } i = i + 1; m = 2*i + 3; } var res = [2]; for (var x = 0; x < s.length; x++) { if (s[x]) { res.push(s[x]); } } return res; } for (var i = 0; i < 10; i++) { var res = get_primes7(10000000); console.log("Found " + res.length + " prime numbers."); }I will keep a close eye on that Javascript environment
This was very insightful, thanks. You actually managed to astonish me – when I tried the nodejs Javascript on my computer, the same used for the above tests, it ran *faster* than the C++ version. Then I recalled that I could compile C++ with optimizations, and now things seem logical again. Though, nodejs seems to be very very fast, at least in this benchmark test.
Here are my nodejs results (using nodejs v0.1.101) which can be compared directly with the original results in my post:
Though I’m not quite sure if nodejs doesn’t actually do the same as C++ – compile the program, and then execute it. In contrast, the script languages don’t compile their code – they parse it to bytecode and interpret it on-the-fly, which is, of course, slower than a real one-time compilation.
P.S. I edited your comment, in an effort to make the layout a bit more readable. I hope you don’t mind…
Nodejs is v8 which tents to generate a lot of optimised machine code.
( see http://www.youtube.com/watch?v=hWhMKalEicY )
So (hopefully) the actual implementation is more like a JIT compiled machine code version of the program then interpreted bytecode. (Which is good performance wise)
Javascript development is REALLY REALLY interesting to follow. And most of the idea’s going into Javascript implementations like v8 but also the new Apple and Mozilla implementations in the war on having the fastest Javascript engine are very applicable to other (scripting) languages as well.
Javascript has the advantage of being the browser language which is in the middle of the browser wars and therefore has gotten many many resources and money and engineers thrown onto itself in the effort to make it faster. (It’s just too bad that the all implement old javascript versions and not use new versions which are more ‘Pythonic’)
This PHP version is a bit more optimized on code. There are some inefficiencies in the code above. The following is about 20% faster on my system.
function get_primes7($n) { if ($n < 2) return array(); if ($n == 2) return array(2); $s = range(3, $n, 2); $mroot = sqrt($n); $half = count($s); for ($i=0, $m=3; $m <= $mroot; $m = 2*++$i + 3) { if ($s[$i]) { for ($j = ($m*$m - 3) >> 1; $j < $half; $j += $m) $s[$j] = 0; } } return array(2) + array_filter($s); }Hm, very strange. The optimized code doesn’t run faster on my system… There are some very good ideas though:
* range() — faster with 5%
* “$number / 2″ vs. “$number >> 1″ — maybe slightly faster
* while() substituted with for() — no change
* “$i = $i + 1″ vs. “++$i” — no change
* array_filter() — slower (also returns 1 less prime numbers)
With your help, my tests show 5% faster PHP execution now. Thanks.
Some other comparisons with cython
On my machine the python script takes 28.7s normally and 13.9s with psyco.
Compiling the unmodified code with cython shaves almost 8s of running in 20.9s.
When declaring c types for i,m and j it goes down to 18.3s.
[...] For complete information about the test environment, please review the previous article. [...]
If you are comparing Perl performance for mathematical functions you should really be using the Perl Data Language additions to Perl.
PDL exists for this very purpose and hugely outperforms basic Perl.
If you compared PDL I think you would see a very different story.
Did you try running the test with caching enabled for PHP ? (APC for instance)
I’d be curious to see the results.
As far as I know and according to the Wikipedia article about PHP accelerators, they help only “to avoid the overhead of parsing and compiling source code on each request”. Since I’m doing only one request (script execution) but it does a lot of work inside it, without exiting the PHP script, the PHP interpreter spends most of the time working on the code algorithm, which is pretty short — only a couple of lines.
Just in case, I did test the PHP benchmark with and without APC enabled, and it made no difference on my computer.
Code optimizers are a different story. In theory they should make the PHP code run faster. This is what Zend claim for their well-known PHP Optimizer which for PHP 5.3 is now called Zend Guard:
My tests however shown no difference in performance. Other tests confirm my results (PHP compiler performance; Impact of Zend Optimizer on PHP Performance).
I’m not really sure though if Zend Guard does the same thing as Zend Optimizer did for PHP 5.2 and older. In PHP 5.2 we had an INI setting for optimization level, for example. With Zend Guard there are almost no settings and it could be that Zend Guard is only to decode Zend encoded PHP scripts. The Zend Server suite has a Zend Optimizer+ component which may be what Zend Optimizer for PHP 5.2 did, but Zend Optimizer+ is not shipped as a separate .so library, so I’m not going to try it.
Thanks for this insightful work.
Would be interesting to see a comparison with Java as well.
I’m not a Java developer but with the help of Google, I think I managed to “translate” the C++ code into Java. You can see the benchmark results in the updated table above.
Ivan – thanks so much !
So in this case it’s fair to say that JS in node is 2,5 times faster than Java as JS = 1/2 as fast as C++ and Java = 1/5 as fast?
You’re welcome.
Yes, we can say that but for this particular test case. This simple benchmark is pretty limited — review the second, bold paragraph in the beginning of the article. You need to test further for your particular setup (program algorithms, usage pattern, hardware and OS, etc).