Read several . txt files and remove duplicate information

Asked

Viewed 205 times

0

I have several files. txt with enrollments and some repeat themselves in different files and I would like to remove all the repeated ones.

The first solution I thought was to make a comparison between all the files using PHP or Nodejs, but I got a bit left behind and I don’t know if this would be the best solution.

Then I thought of throwing everything in a DB just to treat, because I need those license plates in files . txt, but I saw that DB would get huge, with many registrations and maybe it is not very viable because it is hundreds of files and thousands of registrations.

What would be the best solution to this problem? Either of these two above can solve?

php

1 answer

1


You can use the function file to open the file and play all its contents in an array.

Then just use the function array_unique to remove identical lines.

01.txt

valdeir
psr
naval
fuz. nav
valdeir
valdeir psr
stackoverflow

PHP code

<?php

$files = glob("*.txt");

$content = [];

foreach($files as $file) {
    $content = array_merge($content, file($file, FILE_IGNORE_NEW_LINES));
}

$contentUniq = array_unique($content);

var_export($contentUniq);

Output

array (
  0 => 'valdeir',
  1 => 'psr',
  2 => 'naval',
  3 => 'fuz. nav',
  5 => 'valdeir psr',
  6 => 'stackoverflow',
)

Browser other questions tagged php

You are not signed in. Login or sign up in order to post.