0
Hello, I am solving a question that is a routine in my life, grab csv and popular database.
I took a demand where I need to take the first 21 columns of 32 csv files and group everything into a single table, I used the following:
ini_set('memory_limit', '60000M');
$path = '/var/www/html/csv/';
$files = array('ACELERA2.csv','ATLAS.csv','BOLSA.csv','CADASTRO_DMA.csv','CADASTRO_INFRA.csv','CAPITAL.csv','CHEGAMAIS.csv','CPMF.csv','ECOOL.csv','ENERGIA.csv','ENQUADRAMENTO.csv','INOVACAO.csv','INTELIGENCIA.csv','JUROS.csv','MEU_NOVO_MUNDO.csv','MONITOR.csv','NEXCODE.csv','PESQUISAEXPORTACAO.csv','PREMIO_AGUA.csv','PREMIO_MERITO.csv','PROTHEUS.csv','RM.csv','RODADAS.csv','SCE.csv','SCN.csv','SCS.csv','SGC.csv','SINDI.csv','SMS.csv','SPVNE.csv','SR.csv' );
$insert = null;
foreach ($files as $file )
{
$rrow = null;
$fp = fopen($path.$file, "r");
if ($fp)
{
$head = fgetcsv($fp, 0, ";", '"');
$count = count($head);
$collun = '(';
//$collun ='CREATE TABLE '.substr($file,0,-4).' ('."\r\n";
for ($i=0; $i < 22; $i++) {
if ( $i !== $count )
{
$collun.= $head[$i].',';
}
}
$collun_ = substr($collun,0,-1).')';
while ( $data = fgetcsv($fp, 0, ";",'"') )
{
$row = '(';
for ($i=0; $i < 22; $i++)
{
if ( $data[$i] == null )
{
$insert = 'null';
}
else
{
$insert = $data[$i];
}
$row.= "$$".$insert."$$,";
}
$rrow.= substr($row,0,-1).')';
$sql = 'INSERT INTO pf '.$collun_.' values '.$rrow.' ON CONFLICT (cpf) DO NOTHING';
echo $sql;
sleep(3);
$dbconn = pg_connect("host=localhost dbname=xxx user=xxxx password=xxxx")
or die('Could not connect: ' . pg_last_error());
$result = pg_query($dbconn, $sql);
var_dump($result);
pg_close($dbconn);
}
}
}
The bug occurs as follows, for each Insert into it add 3 sets of the $rrow variable not closing the syntax correctly. I thank anyone who can collaborate.
What is the error message?
– rray
The error is that it is: Insert into Fp (...) Values (...) (...)(...); Without the comma between the values, and neither wanted the block Insert. The first block I did is even with 64gb it can’t allocate everything.
– Christopher Tavares