news, development

Multiprocessing with PHP5: Performance!

While reading on nodeJS  I was wondering if something similar is possible with PHP5 and started reading into threading and multiprocessing with PHP... and I started to like multi processing:

Why multiprocessing?

A typical PHP application is executed serial: each command after another. There are a lot of commands that takes long and there are a lot of commands that just waits for I/O (e.g. file operations, fetch operations, database operations). Why not doing something useful while another process waits?

Imagine a simple application, that:

  1. reads data from two database tables (2 x 1sec)
  2. resizes some images (0.5 sec)
  3. logs some events (0.5 sec)
  4. calls some backend system (2 sec)
  5. and finaly renders the view (0.1sec)

So it will take about 5.1 seconds!

The execution time could definitly be improved using threading to about 1.6 seconds! Easily by executing the two database calls parallel (1 sec) and wait till they are finsihed; executing 2. and 3. parallel (+0.5sec) and wait till they are finished; then starting 4. without waiting and doing the output directly. (+0.1sec)

How to use threads?

The idea was to simplify the usage of threads (or parallel processes to be more correct). I found a nice concept in the blog from Tudor (see here ) and created a first beta version of "Threadi" based on his work.
Threadi is a set of classes that should help building reliable multithreated PHP applications and it is my first git hosted project :-)

To see how Threadi works a simple example:


class Worker {
public function doSomething() {
sleep(2);
echo "worker output at ".@date('i:s').PHP_EOL;
}
}

$worker1 = new Worker();
$worker2 = new Worker();

$startTime = microtime(TRUE);
$worker1->doSomething();
$worker2->doSomething();
echo "Time needed in seconds: ". ( microtime(TRUE)-$startTime).PHP_EOL.PHP_EOL;

As you might imagine the script will run about 4 seconds.
Now do the same with using threads:

 

$startTime = microtime(TRUE);
//create two threads for the execution of the function and start them
$thread1 = new Threadi_Thread_PHPThread( array($worker1,'doSomething') );
$thread2 = new Threadi_Thread_PHPThread( array($worker2,'doSomething') );
$thread1->start();
$thread2->start();
// create a join point for both threads and wait till both are finished
$joinPoint = new Threadi_JoinPoint($thread1, $thread2);
$joinPoint->waitTillReady();

This script will run about 2 seconds only. It uses the PHP5 process control features ( de.php.net/manual/en/book.pcntl.php ).

 

Lets make a second example, using a Thread that is able to return the result of the child process:

 

$thread1 = Threadi_ThreadFactory::getReturnableThread(array($worker,'fetchAWebsite'));
$thread1->start('http://www.google.de');
$thread2 = Threadi_ThreadFactory::getReturnableThread(array($worker,'fetchAWebsite'));
$thread2->start('http://www.aoemedia.de');
$thread3 = Threadi_ThreadFactory::getReturnableThread(array($worker,'fetchAWebsite'));
$thread3->start('http://www.heise.de');


$joinPoint = new Threadi_JoinPoint($thread1, $thread2, $thread3);
$joinPoint->waitTillReady();

In this example the class Threadi_Thread_PHPReturnableThread is used, that in addition uses PHP shared memory functions ( www.php.net/manual/en/book.shmop.php ) to return the result of the child to the parent process.

And a last example:

 $service = new Threadi_ThreadService(); 
$service->callAndForget(array($backendFacade,'addOrder'));
echo 'Finished - and hope that the backendFacade will do its job :-)';

 

Where to get it

Threadi is really just a first beta - it needs to be proven in real projects first. You find the current version here: https://github.com/danielpoe/Threadi

Feedback is really welcome.

In order to run it your PHP needs SHMOP and PCNTL support. For macport users:

 

sudo ./port install php5 +apache2 +ipc
sudo ./port install php5-pcntl

 

Unfortunatly this all works only with CLI or with PHP as CGI module - It will not run as an Apache module.

blog comments powered by Disqus
  1. Dan 12.09.11 07:57

    In the examples you display all threaded examples. Yet you also supply a non threaded function. Though this in itself is without a linked example of its use. It would be appreciated to also add a example using the non-threaded variant as well. Just to round out everything your class can do =). Possibly even going so far as maybe sending me a example of this class with a non-threaded example. Would be appreciated. And thank you.

  2. Kurt 17.02.11 20:53

    I work at interactive intelligence and we deal a lot with Java. This seems so much faster. We definitely don't get that kind of processing time or multifunction.

  3. Daniel http://www.typo3-media.com 17.01.11 21:49

    Thx for the comment - I added a info to the blog that pcnt only works with PHP for CLI or CGI.

    Another fireAndForget Thread implementation could be http://framework.zend.com/manual/en/zendx.console.process.unix.overview.html

    I will add that

  4. Daniel http://www.typo3-media.com 17.01.11 21:08

    Hey
    A JoinPoint is a point in your process (most likely your parent process - but can be any other process as well) - were you want to wait for certain threads to be finished - before you go on to the next commands..

    They are used in UML activity diagrams (that can be used to draw multi process flows): http://www.agilemodeling.com/style/activityDiagram.htm

    I dont know how synchronizing threads in Java works - so I cannot compare it :-(

  5. Ingo http://www.ingo-renner.com 17.01.11 12:40

    Also related:
    http://techportal.ibuildings.com/2011/01/17/doing-the-heavy-lifting-gearman-and-magento/

  6. Tudor http://blog.motane.lu 17.01.11 11:18

    Nice, but does this work under Apache? My original implementation was meant for CLI apps only, I think that the fork() call might actually fork the whole Apache...

  7. Dawid http://typo3blog.pl 16.01.11 22:06

    Cool thing. One question. The join point. Is it somethink simillar to for example synchronizing threads in java ?

blogroll