—
There are newer benchmarks: C++ vs. Python vs. PHP vs. Java vs. Others performance benchmark (2016 Q3)
—
The benchmarks here do not try to be complete, as they are showing the performance of the languages in one aspect, and mainly: loops, dynamic arrays with numbers, basic math operations.
This is a redo of the tests done in previous years. You are strongly encouraged to read the additional information about the tests in the article.
Here are the benchmark results:
| Language | CPU time | Slower than | Language version |
Source code |
|||
|---|---|---|---|---|---|---|---|
| User | System | Total | C++ | previous | |||
| C++ (optimized with -O2) | 0.952 | 0.172 | 1.124 | – | – | g++ 5.3.1 | link |
| Java 8 (non-std lib) | 1.332 | 0.096 | 1.428 | 27% | 27% | 1.8.0_72 | link |
| Python 2.7 + PyPy | 1.560 | 0.160 | 1.720 | 53% | 20% | PyPy 4.0.1 | link |
| Javascript (nodejs) | 1.524 | 0.516 | 2.040 | 81% | 19% | 4.2.6 | link |
| C++ (not optimized) | 2.988 | 0.168 | 3.156 | 181% | 55% | g++ 5.3.1 | link |
| PHP 7.0 | 6.524 | 0.184 | 6.708 | 497% | 113% | 7.0.2 | link |
| Java 8 | 14.616 | 0.908 | 15.524 | 1281% | 131% | 1.8.0_72 | link |
| Python 3.5 | 18.656 | 0.348 | 19.004 | 1591% | 22% | 3.5.1 | link |
| Python 2.7 | 20.776 | 0.336 | 21.112 | 1778% | 11% | 2.7.11 | link |
| Perl | 25.044 | 0.236 | 25.280 | 2149% | 20% | 5.22.1 | link |
| PHP 5.6 | 66.444 | 2.340 | 68.784 | 6020% | 172% | 5.6.17 | link |
The clear winner among the script languages is… PHP 7. 🙂
Yes, that’s not a mistake. Apparently the PHP team did a great job! The rumor that PHP 7 is really fast confirmed for this particular benchmark test. You can also review the PHP 7 infographic by the Zend Performance Team.
Brief analysis of the results:
- NodeJS got almost 2x faster.
- Java 8 seems almost 2x slower.
- Python has no significant change in the performance. Every new release is a little bit faster but overall Python is steadily 15x slower than C++.
- Perl has the same trend as Python and is steadily 22x slower than C++.
- PHP 5.x is the slowest with results between 47x to 60x behind C++.
- PHP 7 made the big surprise. It is about 10x faster than PHP 5.x, and about 3x faster than Python which is the next fastest script language.
The tests were run on a Debian Linux 64-bit machine.
You can download the source codes, an Excel results sheet, and the benchmark batch script at:
https://github.com/famzah/langs-performance
/contrib/famzah 
February 10, 2016 at 9:21 pm
I wonder how PyPy stacks against CPython in those tests. Would you be interested in giving it a shot?
February 11, 2016 at 10:58 am
It’s lightning fast:
user 0m1.560s
sys 0m0.160s
Total CPU time of 1.660 seconds which makes it faster than NodeJS and very close to C++. Also 12x faster than the standard Python.
Python 2.7.10 (4.0.1+dfsg-1, Nov 20 2015, 19:46:58)
[PyPy 4.0.1 with GCC 5.2.1 20151028]
February 11, 2016 at 7:18 pm
Great! JIT for the win! Maybe add the results to the table so that we can brag about it ? 😛
February 12, 2016 at 9:37 am
Done. I was hesitating to do it because PyPy is not 100% exactly CPython. But in most cases it is a direct replacement, so you are right.
May 28, 2016 at 7:28 pm
One should bare in mind the way this benchmark works slants in statically typed languages favour. These benchmarks should be run as system services that respond to standard in and reply to standard out. Node must perform type inference before optimisation and execution. In reality systems built with these languages are long running services responding to requests. I would like to see the performance in a “real world” scenario. The most basic would be responding to standard input. This would provide a clearer picture of actual performance instead of compiler performance.
May 28, 2016 at 8:07 pm
The scripts are executing in a “long running” mode. The benchmark times include the interpretation/parsing phase for each language, but it’s so small that its significance is negligible. The math function is called 10 times inside each long running process, in order to have more reliable results.
May 30, 2016 at 5:26 pm
I would like to see the bench done using this in node.js
var start = process.hrtime();//put this at the top of the js file
var total = process.hrtime(start);//put this after the loop
‘total’ variable will contain a tuple array with position 0 being seconds and position 1 being nano seconds sum these values and you have the actual run time of the benchmark. Also node is now on version 6.2 not 4.2.
Dump sum of tuple to file or whatever
In fairness C++ bench should not include compile time
May 30, 2016 at 6:01 pm
In fact
var start = process.hrtime();
should probably be placed just before the bench loop not before the function definition as this would bring it as close as possible to ‘real life’ usage. Also the bench function should be called once or twice before running the loop to warm up the VM.
Similar steps should be taken for each language, taking in to account typical ‘real world’ configurations and each languages distinct strengths.
May 30, 2016 at 8:20 pm
Feel free to redo the tests using the current and your proposed setup, and let us know the outcome. This seems interesting.
July 4, 2016 at 1:55 pm
Interesting comparison. I have modified the php code to make it a bit faster in https://github.com/paul4156/langs-performance/blob/master/primes.php.
Also, a slower, but easier-to-understand (no more array key mapping) hash map solution is provided for comparison. c++ using unordered_map is slower than php (20 sec vs 14 sec).
Next to do is to compare the memory usage of each language.
July 27, 2016 at 12:06 pm
I got a contribution for the Go language: https://github.com/famzah/langs-performance/pull/1
Go performs as fast as Nodejs:
== JavaScript (nodejs) ==
real 0m4.365s
user 0m4.180s
sys 0m0.236s
real 0m4.301s
user 0m4.132s
sys 0m0.260s
v0.10.25
== Go (not optimized, default compiler) ==
real 0m3.819s
user 0m4.192s
sys 0m0.128s
real 0m3.222s
user 0m4.332s
sys 0m0.100s
August 25, 2016 at 12:08 am
For real Java testing, you should test time of execution of calculation method, not whole JVM start/stop + time of calculation. See: http://stackoverflow.com/a/3382969 . Probably related to other languages as well (unless you want to test runtime start time which probably doesn’t depend at all on a tested algorithm).
August 25, 2016 at 7:43 am
Hi Piotr,
That’s correct and already explained in the original article — “The times include the interpretation/parsing phase for each language, but it’s so small that its significance is negligible.” Furthermore, we run the calculation 10 times once the JVM has been started, and I believe that the time it takes is significantly bigger than the initial JVM start.
Unless Java 8 JVM start/stop time is incredibly huge, the test results should be valid.
I don’t have Java 8 to test right now, but here are the result for Java version “1.7.0_111”:
– JVM start/stop time (a program which only prints “Hello world”): 0.120 seconds total user + system
– The calculation benchmark (includes the JVM start/stop time): 10.824 seconds total user + system
So the absolute pure calculation benchmark time is 10.824 – 0.120 = 10.704 seconds CPU time. Therefore, the error is 1.1%. I consider this fair.
August 25, 2016 at 8:45 am
In my test (Java 7):
user+system: 12,504 [s]
calculation time: 3,476 [s]
Code:
public static void main(String[] args) {
long startTime = System.currentTimeMillis();
ArrayList res;
for (int i = 1; i <= 10; ++i) {
res = (new PrimeNumbersGenerator1()).get_primes7(10000000);
System.out.format("Found %d prime numbers.\n", res.size());
}
long stopTime = System.currentTimeMillis();
long elapsedTime = stopTime – startTime;
System.out.println("Elapsed time: "+elapsedTime+" [ms]");
}
September 5, 2016 at 10:24 pm
I’ve done some tests using Java 7 measuring the total CPU time it takes for the whole program to run:
– 10 loops take 7.5 CPU seconds (0.75 sec/loop)
– 50 loops take 30 CPU seconds (0.60 sec/loop)
– 100 loops take 54 CPU seconds (0.54 sec/loop)
– 300 loops take 156 CPU seconds (0.52 sec/loop) — 44% faster!
You are right to suspect that the JVM start/stop + optimization passes take a long time. My tests show that I must probably run the Java programs much longer (at least for 60 CPU seconds), in order to be able to benchmark the pure program running time.
Long story short — Java is 44% faster than what it is shown now in the summary table of this blog article.
More research should be done for the other languages as well. I mean that most (if not all) of them must be tried with more loops, in order to verify if their interpreter/initial-start time is negligible. Or maybe it will be better to simply adjust the loops count for each language to a number where the total test runs for at least 90 CPU seconds.
September 10, 2016 at 9:55 am
Hi Piotr,
I modified the benchmark method. Now all programs run for 90 wall-clock seconds. Even though the interpreter start/stop is included, it really is a very tiny part of the whole running time now.
Here are the new results: https://blog.famzah.net/2016/09/10/cpp-vs-python-vs-php-vs-java-vs-others-performance-benchmark-2016-q3/
To my surprise, Java 8 didn’t perform much faster as we expected. Maybe it gets slower as more and more loops are done, which also allocates more RAM.
November 11, 2016 at 4:55 pm
JVM has a “-server” option which defaults many options to performance settings (if I’m not wrong). So, it may preallocate heap or collect garbage efficiently to perform better. The “Server” Hotspot VM may perform better, too. Many different measurements might be encountered with the JVM’s XX options, I assume. Might worth to try.
November 11, 2016 at 6:09 pm
MT San, you are right and I didn’t know that. Luckily, “java” is already running in “-server” mode according to its help message on my Linux machines:
$ java -h Usage: java [-options] class [args...] (to execute a class) or java [-options] -jar jarfile [args...] (to execute a jar file) where options include: -d32 use a 32-bit data model if available -d64 use a 64-bit data model if available -client to select the "client" VM -server to select the "server" VM ... -hotspot is a synonym for the "client" VM [deprecated] The default VM is server, because you are running on a server-class machine.September 13, 2016 at 3:37 am
You chart is confusing. Your headers show C++ is slower than/previous but the values don’t match up with your conclusion. Nodejs shows it is 81% *slower*. And PHP 7 is not the best on the chart compared to C++…
September 13, 2016 at 7:58 am
Maverick, you need to elaborate.
Nodejs (2.040 seconds) *is* 81% slower than C++ (1.124 seconds). If you divide both times you get a coefficient of 1.81, which means that Nodejs needs 1.81 times more CPU time to complete the task. Expressed in percentage, 1.81 is 81% slower.
“PHP 7 is not the best on the chart compared to C++” — I don’t get it what you mean by this. It is the best *script* language. And by script language I mean a language which doesn’t need to be compiled before you execute it, like you do for C++, Java and Nodejs. Python + PyPy is a script language but it’s in gray and doesn’t count because PyPy does not fully support all Python code — it’s a optimization solution with some limitations.
September 13, 2016 at 4:10 pm
For instance you say nodejs got almost 2x faster. However Nodejs is 81% slower than C++ when **previously** it was only 19%. From those numbers it seems to me that nodejs became significantly slower!
September 13, 2016 at 5:52 pm
“Slower than previous” here means slower than the previous language in the current table. Not of the previous (last year’s) tests.
September 13, 2016 at 7:50 pm
That makes much more sense.
September 13, 2016 at 7:57 pm
🙂
November 12, 2016 at 5:23 pm
Thank you for doing these benchmarks!You did a great job there 🙂
Could you do the same with swift? I am currently learning Swift for mobile rather than Objective C or Java and I wonder if it is as performant as says apple…
November 12, 2016 at 5:35 pm
Hi, thanks. Unfortunately, I can’t code in all programming languages 🙂 It would be best if you make a fork of the GitHub repo, code your version in Swift, and then I can try to benchmark it.
November 23, 2016 at 7:48 pm
Lamer article. Why did you use ArrayList for java and Vector for C++ ?? Doesn`t you know how use LinkedList ??
November 24, 2016 at 6:51 am
Andrey, why don’t you show us all? 🙂
But please read the available information in the lamer article first. Like the “(see notes)” link for Java: https://github.com/famzah/langs-performance/tree/master/java-optimizations
December 13, 2016 at 6:17 pm
For me vectors and arrangements are prime because they have close mechanisms, not the same with lists. Not even all languages support it.
NOt becouse javers love linkedlist means we all have to use it 😀
December 12, 2016 at 5:17 am
There is two big problem.
In your article, `The clear winner among the script languages is… PHP 7.`, so JavaScript is not a script language?
Also, the stable version of Nodejs is v6, so do you think it’s reasonable that use the old nodejs version to test with the newest version of PHP?
December 14, 2016 at 1:06 pm
Hi. You are looking at the older results. There are newer one, as explained in the beginning of this page.
The used Node.js version there is “4.3.1” released on 2016-02-16. At the time of the last tests (Sep/2016), Node.js v6 was still not in LTS release (https://nodejs.org/en/download/releases/) and thus not yet “recommended for most users” (see what the Download page says regarding “Current” vs. “LTS” versions — https://nodejs.org/en/download/).
Last but not least in regards to language versions — I don’t posses unlimited free personal time, in order to constantly update everything 🙂 The whole project is open-source, so feel free to prove by your own test results that there is a performance benefit to update the Node.js interpreter version. And then when I can, I will do it and publish the results.
Regarding whether JavaScript is a script language or not. Of course, it is. The question is a bit open if it fits into the category “general purpose script language”. This is what I meant when I declared PHP7 as the winner. Somehow I imagine Node.js mostly as a script language used for server applications. It isn’t much used nor designed to be used synchronously and in command-line tool programs. Or was it?
January 23, 2017 at 4:10 am
why are you tampering with the PHP7 results you have 2 instructions extra:
error_reporting(E_ALL);
ini_set(‘display_errors’, ‘1’);
These should be set in the php.ini file and not in the code execution time as your code is currently set to.
January 23, 2017 at 8:57 am
Hi Martin, I don’t think that these two instructions make any difference in regards to the final benchmark results. They probably take something like microseconds to be executed.
May 6, 2017 at 6:02 pm
chart of this test, “total” values
May 31, 2017 at 9:23 pm
I miss Ruby here :-).
June 27, 2017 at 4:30 pm
>> The clear winner among the script languages is… PHP 7. 🙂
The clear winner among the script languages in your list is obviously… NodeJS? 😉
I did read your other comment to Jasin Yip, but seriously, just because you want to be PHP7 the winner by disqualifying Node as not “general purpose script language” does not make sense for me. You may dislike Node for whatever reason or not, but you cannot deny its wide spread use for everything. Its much more flexible to use than PHP ever will be.
Of course there are lots of command-line tools written in Node and there are even great libs for writing them. Just look at the whole grunt, gulp, bower tools. Also i would never write CLI scripts with PHP. The alternatives like python, perl, node or go (or even bash) are so much better fit than php.
Also you have embedded NodeJS in browsers which is used by tools like Atom.
So for me Node is much more general purpose than PHP is. I am not promoting that Node is better for your project than PHP or anything else. I am just clarifying my point when it comes to “general purpose” and flexibility between Node and PHP.
Thanks for the benchmark anyways, its always interesting to have new of them!
Cheers
July 8, 2017 at 10:55 pm
Hi, Chris. I no longer use this ambiguous summary sentence in the latest article.
P.S. I have no special attitude to PHP. I don’t even code in PHP 🙂
July 2, 2017 at 2:06 pm
Why do you run `while now – start < RUN_TIME` ? What is `RUN_TIME` ? Why not like `for _ in 0..<100` or something? Wont this `RUN_TIME` disrupt the speed of each program?
July 8, 2017 at 10:34 pm
Each language is tested for 90 wall-clock seconds. If it was “for 1…100”, very fast languages like C would finish too quickly, and slow languages would take a very long time to complete.
September 14, 2017 at 1:38 pm
An excellent comparison! you did great work on here!
November 9, 2017 at 1:58 am
“The clear winner among the script languages is… PHP 7”. I’d argue that it’s JavaScript, considering many IoT projects seem to use Node.js, so it has potential to grow fast for that use. Also, can’t it be considered a script language when used for Web front-end, and even when it’s JITed? The difference between JavaScript, PHP7 and Python is quite remarkable. Also that Java8 is slower than PHP7. Who knew?
November 10, 2017 at 11:09 am
You are right that there is too much ambiguity in what language can be considered a “script” one. I’ve already discussed this in previous comments.
Regarding which is slower or not — let’s not forget that every benchmark is just one use-case. Maybe in different situations things could change.
November 21, 2017 at 8:38 pm
1. step – PHP op-cache was enabled solving the IO issues
2. step – they did a great job with php7 reducing the exec time like twice … especially for OOP code
I tested it with some minimalistic version of Symhony 3 components:
sf3 hello world with php5.6 -> 30 ms
sf3 hello world with php7 -> 17 ms
pretty much a mazing keeping in mind it loads about 100 classes to boot the app
December 13, 2017 at 2:54 pm
Just tried to re-test some of these benchmarks on my Notebook. Here are the results:
Java8 (non-std): 999 ms
Nodejs (v8.2.1): 1485 ms
Java8 ( with VM argument -Xms512m) 1789 ms
Java8 4056 ms
Java9 results should be even better, but I cannot test this now.
Pingback: Whoever owns the metric owns the results — don’t trust benchmarks – Boris Gorelik
March 29, 2019 at 2:11 pm
$ time perl -e ‘for($x = 0; $x <= 1000; $x++){ 2**$x }'
real 0m0,007s
user 0m0,007s
sys 0m0,000s
$ time python2 -c '[ 2**x for x in range(1000) ]'
real 0m0,032s
user 0m0,028s
sys 0m0,004s
$ time python3 -c '[ 2**x for x in range(1000) ]'
real 0m0,046s
user 0m0,034s
sys 0m0,013s
June 20, 2019 at 10:13 pm
Hi. There is too few to be done inside a loop of just 1000 steps. Therefore, you’re mostly measuring the startup time of the interpreters. 9 years ago Perl was still the fastest 🙂 https://blog.famzah.net/2010/08/02/cpp-vs-python-vs-perl-vs-php-performance-benchmark-part-2/