Online education portals like Udacity and Coursera are really changing the world of remote learning in significant ways. By making free and high quality education accessible to a global audience, these platforms are opening up undreamt of possibilities for communities around the world to improve, grow, and prosper in the digital economy of the 21st century. Education at top tier colleges and universities has traditionally been a social and economic privilege, but now anyone can join in the learning revolution by sitting in virtual classrooms with the world’s best and brightest educators. Whether this involves learning how to code and build smart phone apps, or starting up a new business, or learning about public health literacy, the sky is the limit of what’s now possible.

Everything about Web and Network Monitoring

More PHP Performance Testing

by | May 02, 2012

PHP_performanceIn my last article, I tested 3 performance tips regarding PHP’s iteration and selection control structures. Let’s continue the testing today by examining some tips regarding output.Website Performance: PHPand similar articles list many PHP performance tips, but lists of tips sometimes contain spurious advice, or at least advice of minimal value. We really must measure the effectiveness of a tip in our own production environment before judging its worth.Test Harness for PHP offers us a simple little PHP program that we can use to test code-related tips. I use it here to evaluate the effectiveness of performance tips, but please remember that my results will differ from yours because the server I test on is different from yours in many ways.

Concatenating Output

One tip says to output a concatenated string with a multi-parameter echo statement instead of concatenating within a printstatement.Let’s test three options. Option #1 is the print statement, which uses the dot operator to concatenate the strings. Options #2 and #3 are the echo statement. Like option #1, option #2 uses the dot operator. Option #3 avoids concatenation altogether by using a comma to separate the five strings into five pseudo-arguments.The three options being tested are:

Option #1:
   print ($str1 . $str2 . "literal string" . $str3 . $str4);

Option #2:
   echo $str1 . $str2 . "literal string" . $str3 . $str4;

Option #3:
   echo $str1, $str2, "literal string", $str3, $str4;

$str1 is the uppercase alphabet; $str2 is the ten digits; $str3 is 587 random alphanumeric characters; and $str4 is the lowercase alphabet.

I couldn’t use my customary 1,000,000 iterations in this test because CPU time and quantity of output exceeded my system’s limits. I reduced the number of iterations to 10,000 instead, which means I had to multiply the results by 100 so I can compare them to the other tests I run. The chart below shows the results normalized to 1,000,000 iterations:

Option 1   Option 2   Option 3   Winner
--------   --------   --------   ------
  1800       2100       2200        1
  1500       1400       1300        2
  1500       1300       1400        2
  1500       1300       1400        2
  1000       1900       1400        1
  1300       1500       1400        1
  1500       1400       1300        3
  1500       1300       1400        2
  1300       1500       1400        1
  1500       1400       1300        3
  1600       1300       1300       2/3
  1500       1300       1400        2
  1500       1300       1300       2/3
  1500       1300       1400        2
  1300       1600       1300       1/3
  1000       1900       1300        1
  1500       1100       1600        2
  1600       1300       1300       2/3
  1500       1400       1300        3
  1500       1400       1300        3

Option #1 is the fastest 5.5 times, option #2 is the fastest 8.5 times, and option #3 is the fastest 6 times. [If an option shares first place with another option, both are counted as half-a-win, which accounts for the halves in the preceding sentence.]

It seems that all three options are fastest approximately one-third of the time, which points to the conclusion that there is no real difference between the options. We still need to be careful about this conclusion, though:

  1. Twenty trials may not be enough to provide statistical significance. It’s better than 10, but significance cannot be guaranteed.
  2. Because the number of repetitions was 10,000 instead of 1,000,000, each number in the above chart is ± 50. Since many of the above trials show a difference of only 200 between fastest and slowest, a potential error of 50 is a relative error of 25%, which is way too high to be acceptable.
  3. Tests were conducted on only one server. There is no evidence to suggest that these results can be obtained from any other server. [On the other hand, though, there is no evidence to suggest that they can’t.]

It looks like concatenating in a print statement or in an echo statement is about the same. This test provides no evidence to suggest that one is faster than the other.

Print vs. Echo

What if I get rid of the concatenation in the above test? Let’s test the print function against the echo statement without concatenation clouding the issue. Option #1 is a print statement that outputs a one-character string literal. Options #2 and #3 are echo statements. Option #2 uses parentheses; option #3 does not.

Option #1:
   print (".");

Option #2:
   echo (".");

Option #3:
   echo ".";

I alternated between two servers for these tests. Every second row belongs to the same server. Here are the results:

Option 1   Option 2   Option 3   Winner   Loser
--------   --------   --------   ------   -----
    3          5          6         1       3
    6          4          5         2       1
    4          5          5         1      2/3
    6          6          5         3      1/2
    3          5          7         1       3
    7          5          5        2/3      1
    3          6         16         1       3
    7          6          6        2/3      1
    3          5          8         1       3
    6          6          6
    3          6          7         1       3
    6          5          6         2      1/3
    3          5          7         1       3
    6          7          6        1/3      2
    4          5          7         1       3
    6          6          5         3      1/2
    3          6          5         1       2
    5          6          6         1      2/3
    3          5          6         1       3
    6          6          5         3      1/2
    3          6          5         1       2
    6          6          6
    3          5          6         1       3
    5          6          6         1      2/3
    3          6          8         1       3
    6          6          6

Let’s summarize the wins and losses for each option:

            Wins  Losses
            ----  ------
Option #1   15.5     5
Option #2    3.0     6
Option #3    4.5    12

At first glance, it seems like we have a clear winner here (option #1, the print statement). However, there are other things to consider.

I notice a pattern in the results: Option #1 is the winner in all the odd-numbered rows. Since all the odd-numbered rows are the results from one server, we really need to look at the two servers separately:

              Server #1      Server #2
              ---------      ---------
            Wins  Losses   Wins  Losses
            ----  ------   ----  ------
Option #1    13      0      2.5     5
Option #2     0     2.5      3     3.5
Option #3     0    10.5     4.5    1.5

It is better to conclude that option #1 (the print statement) is faster than the echo statement on one server, but may be the slowest option on the other server. The different results between servers may be due to the fact that they are running different versions of PHP and other software.

We can only conclude that this test needs to be run on our own production server. The averages we calculate from any other server (or set of servers) may be misleading. Our production server is the one that matters, so that’s the one we need to test on. Then, each time we upgrade PHP, the operating system, or any other supporting software, we need to run the test again.

Ignoring the 7th row as an outlier, the difference between the fastest and slowest option in each case is only a couple of milliseconds (per 1,000,000 iterations). If someone refers to this as a micro-optimization, perhaps they’re right. Is all the effort worthwhile?

Conclusion

Neither print nor echo can be shown to be faster than the other. When outputting multiple strings, no difference is observed. When outputting a one-character literal string, different servers give us different conclusions.Lesson Learned: PHP performance tests must be conducted on the server we intend to use for production. All the test results in the world are meaningless if they do not match the results obtained from our own production server.
Post Tagged with

Category: Website Performance | Tagged No Comments.

About Warren Gaebel

Warren wrote his first computer program in 1970 (yes, it was Fortran).  He earned his Bachelor of Arts degree from the University of Waterloo and his Bachelor of Computer Science degree at the University of Windsor.  After a few years at IBM, he worked on a Master of Mathematics (Computer Science) degree at the University of Waterloo.  He decided to stay home to take care of his newborn son rather than complete that degree.  That decision cost him his career, but he would gladly make the same decision again. Warren is now retired, but he finds it hard to do nothing, so he writes web performance articles for the Monitor.Us blog.  Life is good!