How to know the number of lines a large file has in php?

Question

How do I know the number of lines in a file using PHP?

I know there are functions like file, which returns all rows of the file in one array. We could just use one count, but the problem is I need to do this for a 60mb, and I don’t think it’s a good idea to use file in that case.

There’s some other way to do it?

How can I know, for example, how many lines there are in a 2gb, without bursting the PHP memory?

Is there any smarter way to count the lines of a PHP file?

Answer 1

The two methods ( fgets() and file()) use loop to read the file (which is inevitable). Either implicitly or explicitly there will be a loop going through all lines of the file.

But you just want to know the number of lines, so no matter the file size as you will only write a value. Do it:

$myfile = fopen("meuArquivo.txt", "r") or die("Unable to open file!");
while(!feof($myfile)) {
  $count++;
}
fclose($myfile);
echo $count;

Answer 2

You have to read it in Chunks data. Something like this:

$file = fopen("teste.txt",'r');
$count = 0;
while (!feof($file)) {
    $line = fgets($file, 4096); //provavelmente eu colocaria um valor maior, jamais menor
    $count++;
}
fclose($file);

I set a limit of 4096 bytes because you’re at risk if the file is too big and you don’t have enough line breaks to create Chunks small. This solution is not perfect. A better one would need a much more sophisticated algorithm.

I came to think of another who has problems too:

$file = fopen("teste.txt",'rb');
$count = 0;
while (!feof($file)) {
    $chunk= fread($f, 4096); //provavelmente eu colocaria um valor maior, jamais menor
    $count += substr_count($chunk, "\n");
}
fclose($file);

I put in the Github for future reference.

Line breaking can have more than one character and stay one character in one Chunk and the other in the next Chunk. Then you won’t tell.

Production-ready solutions would have to consider this and treat when it happens. This is easier to solve in the second algorithm. He still has the advantage of never filling his memory.

Do tests to evaluate the best size of the Chunk. I put 4K because it’s memory page size and the most common size of cluster of the file system. Smaller will be worse and tend to have more risk of cutting line by half disturbing the two algorithms. Bigger ones can give much better results. I would venture to say that the bigger the better, but it depends on the hardware, OS, usage pattern, etc. It can get good at testing and create some problem in use in production. If you could read the whole file it would always be the fastest and risk-free.

The Guilhermelautert raised a question of \n be just the line feed and therefore would not cause problem in the break. But in Windows the break is \r\n (I have no way to test). Of the two one or PHP considers the \n in code like full line break and happens what I said, or this code would not work properly on Windows, requiring the use of \r\n in the code to catch the break the right way, which would have the problem of dividing the break indicator line the same way.

• stream-get-line()
• fgets()
• fread()
• fopen()
• substr_count()

Answer 3

I, as a lover of OOP in PHP, would do this to the object SplFileObject

$file = new \SplFileObject('file.extension', 'r');
$file->seek(PHP_INT_MAX);
echo $file->key() + 1; 

I use the PHP_INT_MAX to point to the last line of the file, because SplFileObject implements SeekableIterator.

Hence, as the count of the lines begins by 0, I have to add +1 to bring the right value.

Another detail: How I’m using SplFileObject, the iteration of a large file would be done line by line, thus saving in memory and being able to count a giant file, without locking the script.

Answer 4

As lover of REGEX I propose:

$content = file_get_contents("file_name");          //  LE TODO ARQUIVO
$content = preg_replace('~[^\n]~', '', $content);   //  REMOVE TUDO QUE NÃO SEJA QUEBRA DE LINHA (\n)
print_r(strlen($content)+1);                        //  CONTA QUANTOS BYTES SOBRARAM, +1 POIS NO FINAL DO ARQUIVO NÃO TEM \n