Is there any way to make asynchronous PHP url requests?

Question

I need to make several requests (about 200 requests) for an external url, but I don’t want php to "wait" for the request to end.

Remembering that I want to avoid using functions like exec and things such that run on the command line, on account of shared servers not having support for such operations.

Is there any way to make asynchronous PHP url requests?

Answer 1

The library Guzzle in its latest versions has support for asynchronous requests.

Example taken from the documentation.

$requests = [
    $client->createRequest('GET', 'http://httpbin.org'),
    $client->createRequest('DELETE', 'http://httpbin.org/delete'),
    $client->createRequest('PUT', 'http://httpbin.org/put', ['body' => 'test'])
];

$options = [];

// Cria pool. 
$pool = new Pool($client, $requests, $options);
//envia todas as requisições e quando elas terminarem chama o callback complete.
Pool::send($client, $requests, [
    'complete' => function (CompleteEvent $event) {
           echo 'Completed request to ' . $event->getRequest()->getUrl() . "\n";
           echo 'Response: ' . $event->getResponse()->getBody() . "\n\n";
     }
]);

//continue seu código aqui.

In this case all requests will be executed in parallel without blocking

Answer 2

Just as a complement to the @Viniciuszaramella response, where he quotes the requisitions from Guzzle\Http.

Explanation of the Asymchronism of the Guzzle

Regarding the asynchrony regarding Guzzle requests, I realized that the same, when using the class Pool actually sends several requests at once background, having therefore their answers asynchronously.

In case, if we made a request with the Guzzle, normally without using the Pool, within a for from 0 to 200, expensive request would await the reply of the other to perform a request.

Example:

  $guzzle = new GuzzleHttp\Client;

  for ($i = 0; $i  < 200; $i++)
  {
       $guzzle->get('http://httpbin.org/get', ['query' => ['i' => $i]]);
  }

In the case of Pool, requests are sent "at once" - according to the configuration you have in the option concurrency, which is the number of requests made at a time in the queue.

$requests = function () {

    $guzzle = new GuzzleHttp\Client;

    for ($i = 0; $i  < 200; $i++)
    {
          yield $guzzle->get('http://httpbin.org/get', ['query' => ['i' => $i]]);
    }
};


$pool = new Pool($guzzle, $requests(), [
    'concurrency' => 50, // envia de 30 em 30,
    'rejected' => function ($exception, $index)
    {

    },

    'fulfilled' => function ($response, $index)
    {

    }
]);


$poll->promise()->wait();

I performed similar tests on both cases. The result was that, in the case similar to the first example, it took 45 to 50 seconds a request (enough to give that PHP time limit error). In the second case, only 5 to 7 seconds were used in the execution time.

So it really is an effective method.

Other Asynchronmentalism

Before Vinicius' reply was published, I had found on the internet to make asynchronous requests with PHP. However, when performing such an operation, PHP will not wait for the answer, but will only make the request while the code will run normally until the end.

function async_request($method, $url, $params, array $headers = [])
{

    $method = strtoupper($method);

    $sendData = in_array($method, array('PUT', 'POST'));

    $data = http_build_query($params);

    $parts = parse_url($url) + [
        'port'  => 80,
        'path'  => '/',
    ];

    $headers += [
        'Content-Type'   => 'application/x-www-form-urlencoded',
        'Host'           => $parts['host'],
        'Content-Length' => !$sendData ? 0 : strlen($data),
        'Connection'     => 'Close'
    ];

    $path = !$sendData ? $parts['path'] . '?' . $data : $parts['path'];

    $output_header = sprintf('%s %s HTTP/1.1', $method, $path) . "\r\n";

    foreach ($headers as $name => $value)
    {
        foreach((array) $value as $v)
        {
            $output_header .= "$name: $v\r\n";
        }
    }

    $output_header .= "\r\n";

    $handle = @fsockopen($parts['host'], $parts['port'], $_errno, $_errstr, 30);

    if ($handle === false)
    {
        throw new \RuntimeException($_errstr);
    }

    fwrite($handle, $output_header);

    if ($sendData) fwrite($handle, $data);

    fclose($handle);

    return $output_header;

}

You can test in this function as follows:

#index.php

assync_request('GET', 'http://localhost/target.php', []);

echo 'terminou';

#target.php

sleep(5);
file_put_contents('gato.txt', "Meow!!!\n\n", FILE_APPEND);

When you make the request, you will realize that the word 'terminou' will be displayed immediately. And after 5 seconds, is that the file gato.txt will be created. This is because php has executed a socket in the backgroud, which is responsible for requesting target.php.

That was the idea that I had initially about the asynchrony, but anyway the @Viniciuszaramella response served me better.

Answer 3

An old routine I developed:

class Background
{

    /*
    $cmd -> A linha de comando a executar.
    $opt -> Opção de parâmetros para o ambiente onde executa o script.
    */
    public static function Call($cmd, $opt = 'start')
    {

        if (stripos(php_uname('s'), 'windows') !== false) {
            /*
            Condições de parâmetros para ambiente Windows.
            */
            switch ($opt) {
                default:
                case 'start':
                    $prefix = 'start /B '; // Esse aqui é o padrão, pois é compatível com as versões mais recentes do Windows.
                    $sufix = '';
                    break;
                case 'nul':
                    $prefix = '';
                    $sufix = ' > NUL 2> NUL';
                    break;
            }
        } else {
            /*
            Opções para ambiente *nix. (isso inclui os-x)
            Normalmente o sufixo ` &` é compatível com diversas distribuições Linux. Esse parâmetro diz ao sistema operacional executar em background.
            */
            switch ($opt) {
                default:
                case '&':
                    $prefix = '';
                    $sufix = ' &';
                    break;
                case 'dev-null':
                    $prefix = '';
                    $sufix = ' > /dev/null 2>/dev/null &';
                    break;
            }
        }

        exec(sprintf('%s%s%s', $prefix, $cmd, $sufix));

        return null;
    }

}

define('PHP_PATH', '/local/do/binario/php');

echo 'start '.microtime(true);
Background::Call(PHP_PATH.' "/local/de/um/arquivo.php"');
Background::Call(PHP_PATH.' "/local/de/um/arquivo.php"');
Background::Call(PHP_PATH.' "/local/de/um/arquivo.php"');
echo PHP_EOL.'end '.microtime(true);

php file.

<?php

/*
Fecha o output, ou seja, faz com que o script que invocou, não fique esperando por uma resposta.
*/
fclose(STDOUT);

/*
Daqui em diante, pode fazer o que bem entender.
Claro, esteja ciente de que tudo aqui está sendo executado em ambiente CLI (Command Line Interface).

Vamos fazer um exemplo para simular e testar se está funcionando.
*/

file_put_contents('background.txt', 'start '.microtime(true).PHP_EOL, FILE_APPEND);
sleep(5); // espera 5 segundos
file_put_contents(BASE_DIR.'background.txt', 'end '.microtime(true).PHP_EOL, FILE_APPEND);

About the asynchronous execution routine

There is no process control. Basically an execution command is sent by the function exec(), that combined with some parameters ignores the return of a response. Thus, asynchronous executions.

Combinations vary by environment, so be aware that it is not enough to just copy the code and think it will work like magic.

In the method Call() class Background, just add new parameters if necessary.

Example:

            /*
            Opções para ambiente *nix. (isso inclui os-x)
            Normalmente o sufixo ` &` é compatível com diversas distribuições Linux. Esse parâmetro diz ao sistema operacional executar em background.
            */

            switch ($opt) {
                default:
                case '&':
                    $prefix = '';
                    $sufix = ' &';
                    break;
                case 'dev-null':
                    $prefix = '';
                    $sufix = ' > /dev/null 2>/dev/null &';
                    break;
                case 'dev-null2':
                    $prefix = '';
                    $sufix = ' /dev/null 2>&1 &';
                    break;
                case 'outro-nome-como-quiser':
                    $prefix = '';
                    $sufix = ' /algum/comando/diferente/que/funcione/num/ambiente/especifico';
                    break;
            } 

The script is simple, clean, easy to understand. It does not depend on third party libraries.