[转]A HOWTO on Optimizing PHP

Bottlenecks

(a) Networking

(b) CPU

(c) Shared Memory

(d) File System

(e) Process Management

(f) Connecting to Other Servers

Tuning Your Web Server for PHP

(a) Apache 1.3/2.0

(b) IIS Tuning

PHP4's Zend Engine

One of the secrets of high performance is not to write faster PHP code, but to avoid executing PHP code by caching generated HTML in a file or in shared memory. The PHP script is only run once and the HTML is captured, and future invocations of the script will load the cached HTML. If the data needs to be updated regularly, an expiry value is set for the cached HTML. HTML caching is not part of the PHP language nor Zend Engine, but implemented using PHP code. There are many class libraries that do this. One of them is the PEAR Cache, which we will cover in the next section. Another is the Smarty template library.

Finally, the HTML sent to a web client can be compressed. This is enabled by placing the following code at the beginning of your PHP script:

<?php

ob_start("ob_gzhandler");

:
:

?>

If your HTML is highly compressible, it is possible to reduce the size of your HTML file by 50-80%, reducing network bandwidth requirements and latencies. The downside is that you need to have some CPU power to spare for compression.

The PEAR Cache is a set of caching classes that allows you to cache multiple types of data, including HTML and images.

The most common use of the PEAR Cache is to cache HTML text. To do this, we use the Output buffering class which caches all text printed or echoed between the start() and end() functions:

require_once("Cache/Output.php");

$cache = new Cache_Output("file", array("cache_dir" => "cache/") );

if ($contents = $cache->start(md5("this is a unique key!"))) {

#
# aha, cached data returned
#

print $contents;
print "<p>Cache Hit</p>";

} else {

#
# no cached data, or cache expired
#

print "<p>Don't leave home without it…</p>"; # place in cache
print "<p>Stand and deliver</p>"; # place in cache
print $cache->end(10);

}

Since I wrote these lines, a superior PEAR cache system has been developed: Cache Lite ; and for more sophisticated distributed caching, see memcached (Added 28 Feb 2005).

The Cache constructor takes the storage driver to use as the first parameter. File, database and shared memory storage drivers are available; see the pear/Cache/Container directory. Benchmarks by Ulf Wendel suggest that the "file" storage driver offers the best performance. The second parameter is the storage driver options. The options are "cache_dir", the location of the caching directory, and "filename_prefix", which is the prefix to use for all cached files. Strangely enough, cache expiry times are not set in the options parameter.

To cache some data, you generate a unique id for the cached data using a key. In the above example, we used md5("this is a unique key!").

The start() function uses the key to find a cached copy of the contents. If the contents are not cached, an empty string is returned by start(), and all future echo() and print() statements will be buffered in the output cache, until end() is called.

The end() function returns the contents of the buffer, and ends output buffering. The end() function takes as its first parameter the expiry time of the cache. This parameter can be the seconds to cache the data, or a Unix integer timestamp giving the date and time to expire the data, or zero to default to 24 hours.

Another way to use the PEAR cache is to store variables or other data. To do so, you can use the base Cache class:

<?php

require_once("Cache.php");

$cache = new Cache("file", array("cache_dir" => "cache/") );
$id = $cache->generateID("this is a unique key");

if ($data = $cache->get($id)) {

print "Cache hit.<br>Data: $data";

} else {

$data = "The quality of mercy is not strained...";
$cache->save($id, $data, $expires = 60);
print "Cache miss.<br>";

}

?>

To save the data we use save(). If your unique key is already a legal file name, you can bypass the generateID() step. Objects and arrays can be saved because save() will serialize the data for you. The last parameter controls when the data expires; this can be the seconds to cache the data, or a Unix integer timestamp giving the date and time to expire the data, or zero to use the default of 24 hours. To retrieve the cached data we use get().

You can delete a cached data item using $cache->delete($id) and remove all cached items using $cache->flush().

New: A faster Caching class is Cache-Lite . Highly recommended.

In earlier section we have covered many performance issues. Now we come to the meat and bones, how to go about measuring and benchmarking your code so you can obtain decent information on what to tune.

If you want to perform realistic benchmarks on a web server, you will need a tool to send HTTP requests to the server. On Unix, common tools to perform benchmarks include ab (short for apachebench) which is part of the Apache release, and the newer flood (httpd.apache.org/test/flood). On Windows NT/2000 you can use Microsoft's free Web Application Stress Tool (webtool.rte.microsoft.com).

These programs can make multiple concurrent HTTP requests, simulating multiple web clients, and present you with detailed statistics on completion of the tests.

You can monitor how your server behaves as the benchmarks are conducted on Unix using "vmstat 1". This prints out a status report every second on the performance of your disk i/o, virtual memory and CPU load. Alternatively, you can use "top d 1" which gives you a full screen update on all processes running sorted by CPU load every 1 second.

On Windows 2000, you can use the Performance Monitor or the Task Manager to view your system statistics.

If you want to test a particular aspect of your code without having to worry about the HTTP overhead, you can benchmark using the microtime(), which returns the current time accurate to the microsecond as a string. The following function will convert it into a number suitable for calculations.

function getmicrotime()
{
list($usec, $sec) = explode(" ",microtime());
return ((float)$usec + (float)$sec);
}

$time = getmicrotime();

#
# benchmark code here
#

echo "<p>Time elapsed: ",getmicrotime() - $time, " seconds";

Alternatively, you can use a profiling tool such as APD or XDebug . Also see my article squeezing code with xdebug .

This case study details a real benchmark we did for a client. In this instance, the customer wanted a guaranteed response time of 5 seconds for all PHP pages that did not involve running long SQL queries. The following server configuration was used: an Apache 1.3.20 server running PHP 4.0.6 on Red Hat 7.2 Linux. The hardware was a twin Pentium III 933 MHz beast with 1 Gb of RAM. The HTTP requests will be for the PHP script "testmysql.php". This script reads and processes about 20 records from a MySQL database running on another server. For the sake of simplicity, we assume that all graphics are downloaded from another web server.

We used "ab" as the benchmarking tool. We set "ab" to perform 1000 requests (-n1000), using 10 simultaneous connections (-c10). Here are the results:

    # ab   -n1000 -c10 http://192.168.0.99/php/testmysql.php
This is ApacheBench, Version 1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/

Server Software:        Apache/1.3.20
Server Hostname:        192.168.0.99
Server Port:            80

Document Path:          /php/testmysql.php
Document Length:        25970 bytes

Concurrency Level:      10
Time taken for tests:   128.672 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      26382000 bytes
HTML transferred:       25970000 bytes
Requests per second:    7.77
Transfer rate:          205.03 kb/s received

Connnection Times (ms)
              min   avg   max
Connect:        0     9   114
Processing:   698  1274  2071
Total:        698  1283  2185

  

While running the benchmark, on the server side we monitored the resource utilization using the command "top d 1". The parameters "d 1" mean to delay 1 second between updates. The output is shown below.

    10:58pm  up  3:36,  2 users,  load average: 
    
      9.07
    
    ,
    
       3.29, 1.79
    
    
74 processes: 63 sleeping, 11 running, 0 zombie, 0 stopped
CPU0 states: 
    
      92.0% user
    
    ,  
    
      7.0% system
    
    ,  0.0% nice,  0.0% idle
CPU1 states: 
    
      95.0% user
    
    ,  
    
      4.0% system
    
    ,  0.0% nice,  0.0% idle
Mem:  1028484K av,  230324K used,  798160K free,      64K shrd,   27196K buff
Swap: 2040244K av,       0K used, 2040244K free                   30360K cached

  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
 1142 apache    20   0  7280 7280  3780 R    21.2  0.7   0:20 httpd
 1154 apache    17   0  8044 8044  3788 S    19.3  0.7   0:20 httpd
 1155 apache    20   0  8052 8052  3796 R    19.3  0.7   0:20 httpd
 1141 apache    15   0  6764 6764  3780 S    14.7  0.6   0:20 httpd
 1174 apache    14   0  6848 6848  3788 S    12.9  0.6   0:20 httpd
 1178 apache    13   0  6864 6864  3804 S    12.9  0.6   0:19 httpd
 1157 apache    15   0  7536 7536  3788 R    11.0  0.7   0:19 httpd
 1159 apache    15   0  7540 7540  3788 R    11.0  0.7   0:19 httpd
 1148 apache    11   0  6672 6672  3784 S    10.1  0.6   0:20 httpd
 1158 apache    14   0  7400 7400  3788 R    10.1  0.7   0:19 httpd
 1163 apache    20   0  7540 7540  3788 R    10.1  0.7   0:19 httpd
 1169 apache    12   0  6856 6856  3796 S    10.1  0.6   0:20 httpd
 1176 apache    16   0  8052 8052  3796 R    10.1  0.7   0:19 httpd
 1171 apache    15   0  7984 7984  3780 S     9.2  0.7   0:18 httpd
 1170 apache    16   0  7204 7204  3796 R     6.4  0.7   0:20 httpd
 1168 apache    10   0  6856 6856  3796 S     4.6  0.6   0:20 httpd
 1377 natsoft   11   0  1104 1104   856 R     2.7  0.1   0:02 top
 1152 apache     9   0  6752 6752  3788 S     1.8  0.6   0:20 httpd
 1167 apache     9   0  6848 6848  3788 S     0.9  0.6   0:19 httpd
    1 root       8   0   520  520   452 S     0.0  0.0   0:04 init
    2 root       9   0     0    0     0 SW    0.0  0.0   0:00 keventd

  

Looking at the output of "top", the twin CPU Apache server is running flat out with 0% idle time. What is worse is that the load average is 9.07 for the past minute (and 3.29 for the past 5 minutes, 1.79 for the past 15 minutes). The load average is the average number of processes that are ready to be run. For a twin processor server, any load above 2.0 means that the system is being overloaded. You might notice that there is a close relationship between load (9.07) and the number of simultaneous connections (10) that we have defined with ab.

Luckily we have plenty of physical memory, with about 798,160 Mb free and no virtual memory used.

Further down we can see the processes ordered by CPU utilization. The most active ones are the Apache httpd processes. The first httpd task is using 7280K of memory, and is taking an average of 21.2% of CPU and 0.7% of physical memory. The STAT column indicates the status: R is runnable, S is sleeping, and W means that the process is swapped out.

Given the above figures, and assuming this a typical peak load, we can perform some planning. If the load average is 9.0 for a twin-CPU server and assuming each task takes about the same amount of time to complete, then a lightly loaded server should be 9.0 / 2 CPUs = 4.5 times faster. So a HTTP request that used to take 1.283 seconds to satisfy at peak load will take about 1.283 / 4.5 = 0.285 seconds to complete.

To verify this, we benchmarked with 2 simultaneous client connections (instead of 10 in the previous benchmark) to give an average of 0.281 seconds, very close to the 0.285 seconds prediction!

    # ab   -n100 -c2 http://192.168.0.99/php/testmysql.php
 
[ some lines omitted for brevity ]

Requests per second:    7.10
Transfer rate:          187.37 kb/s received

Connnection Times (ms)
              min   avg   max
Connect:        0     2    40
Processing:   255   279   292
Total:        255   281   332

  

Conversely, doubling the connections, we can predict that the average connection time should double from 1.283 to 2.566 seconds. In the benchmarks, the actual time was 2.570 seconds.

When we pushed the benchmark to use 40 connections, the server overloaded with 35% failed requests. On further investigation, it was because the MySQL server persistent connects were failing because of "Too Many Connections".

The benchmark also demonstrates the lingering behavior of Apache child processes. Each PHP script uses 2 persistent connections, so at 40 connections, we should only be using at most 80 persistent connections, well below the default MySQL max_connections of 100. However Apache idle child processes are not assigned immediately to new requests due to latencies, keep-alives and other technical reasons; these lingering child processes held the remaining 20+ persistent connections that were "the straws that broke the Camel's back".

By switching to non-persistent database connections, we were able to fix this problem and obtained a result of 5.340 seconds. An alternative solution would have been to increase the MySQL max_connections parameter from the default of 100.

The above case study once again shows us that optimizing your performance is extremely complex. It requires an understanding of multiple software subsystems including network routing, the TCP/IP stack, the amount of physical and virtual memory, the number of CPUs, the behavior of Apache child processes, your PHP scripts, and the database configuration.

In this case the PHP code was quite well tuned, so the first bottleneck was the CPU, which caused a slowdown in response time. As the load increased, the system slowed down in a near linear fashion (which is a good sign) until we encountered the more serious bottleneck of MySQL client connections. This caused multiple errors in our PHP pages until we fixed it by switching to non-persistent connections.

From the above figures, we can calculate for a given desired response time, how many simultaneous HTTP connections we can handle. Assuming two-way network latencies of 0.5 seconds on the Internet (0.25s one way), we can predict:

As our client wanted a maximum response time of 5 seconds, the server can handle up to 34 simultaneous connections per second. This works out to a peak capacity of 34/5 = 6.8 page views per second.

To get the maximum number of page views a day that the server can handle, multiply the peak capacity per second by 50,000 (this technique is suggested by the webmasters at pair.com, a large web hosting company), to give 340,000 page views a day.

The patient reader who is still wondering why so much emphasis is given to discussing non-PHP issues is reminded that PHP is a fast language, and many of the likely bottlenecks causing slow speeds lie outside PHP.

Most PHP scripts are simple. They involve reading some session information, loading some data from a content management system or database, formatting the appropriate HTML and echoing the results to the HTTP client. Assuming that a typical PHP script completes in 0.1 seconds and the Internet latency is 0.2 seconds, only 33% of the 0.3 seconds response time that the HTTP client sees is actual PHP computation. So if you improve a script's speed by 20%, the HTTP client will see response times drop to 0.28 seconds, which is an insignificant improvement. Of course the server can probably handle 20% more requests for the same page, so scalability has improved.

The above example does not mean we should throw our hands up and give up. It means that we should not feel proud tweaking the last 1% of speed from our code, but we should spend our time optimizing worthwhile areas of our code to get higher returns.

The places where such high returns are achievable are in the while and for loops that litter our code, where each slowdown in the code is magnified by the number of times we iterate over them. The best way of understanding what can be optimized is to use a few examples:

Here is one simple example that prints an array:

for ($j=0; $j<sizeof($arr); $j++)
echo $arr[$j]."<br>";

This can be substantially speeded up by changing the code to:

for ($j=0, $max = sizeof($arr), $s = ''; $j<$max; $j++)
$s .= $arr[$j]."<br>";

echo $s;

First we need to understand that the expression $j<sizeof($arr) is evaluated within the loop multiple times. As sizeof($arr) is actually a constant (invariant), we move the cache the sizeof($arr) in the $max variable. In technical terms, this is called loop invariant optimization.

The second issue is that in PHP 4, echoing multiple times is slower than storing everything in a string and echoing it in one call. This is because echo is an expensive operation that could involve sending TCP/IP packets to a HTTP client. Of course accumulating the string in $s has some scalability issues as it will use up more memory, so you can see a trade-off is involved here.

An alternate way of speeding the above code would be to use output buffering. This will accumulate the output string internally, and send the output in one shot at the end of the script. This reduces networking overhead substantially at the cost of more memory and an increase in latency. In some of my code consisting entirely of echo statements, performance improvements of 15% have been observed.

ob_start();
for ($j=0, $max = sizeof($arr), $s = ''; $j<$max; $j++)
echo $arr[$j]."<br>";

Note that output buffering with ob_start() can be used as a global optimization for all PHP scripts. In long-running scripts, you will also want to flush the output buffer periodically so that some feedback is sent to the HTTP client. This can be done with ob_end_flush(). This function also turns off output buffering, so you might want to call ob_start() again immediately after the flush.

            function TestRef(&$a)
{
    $b = $a;
    $c = $a;
}
$one = 1;
ProcessArrayRef($one);

          

            function TestNoRef($a)
{
    $b = $a;
    $c = $a;
}
$one = 1;
ProcessArrayNoRef($one);