1
Hello everybody good morning, I am having problems with a update routine in the mysql database, I am reading a Bufferedreader containing more than 3,000,000 records and updating in mysql but when I am close to 1,200 first records the system presents the following error:
2015-08-21 20:27:12,512 ERROR Erro de IO: Connection reset
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at sun.security.ssl.InputRecord.readFully(Unknown Source)
at sun.security.ssl.InputRecord.read(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
at sun.security.ssl.AppInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.http.ChunkedInputStream.fastRead(Unknown Source)
at sun.net.www.http.ChunkedInputStream.read(Unknown Source)
at java.io.FilterInputStream.read(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
at java.util.zip.InflaterInputStream.fill(Unknown Source)
at java.util.zip.InflaterInputStream.read(Unknown Source)
at java.util.zip.GZIPInputStream.read(Unknown Source)
at sun.nio.cs.StreamDecoder.readBytes(Unknown Source)
at sun.nio.cs.StreamDecoder.implRead(Unknown Source)
at sun.nio.cs.StreamDecoder.read(Unknown Source)
at java.io.InputStreamReader.read(Unknown Source)
at java.io.BufferedReader.fill(Unknown Source)
at java.io.BufferedReader.readLine(Unknown Source)
at java.io.BufferedReader.readLine(Unknown Source)
at ar.com.qualaconsulting.AccountDataLoader.loadCF_Ids(AccountDataLoader.java:1387)
at ar.com.qualaconsulting.AccountDataLoader.preprocess(AccountDataLoader.java:1717)
at ar.com.qualaconsulting.TestDataLoader.main(TestDataLoader.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
The referenced code is this:
InputStreamReader csv = new InputStreamReader(results, "UTF-8");
BufferedReader csvReader = new BufferedReader(csv);
//Busca a primeira linha que é o cabeçalho
csvReader.readLine();
String curLine = null;
logger.info("Vai Percorrer arquivo CSV");
int totalArquivosAtualizados = 0;
while((curLine = csvReader.readLine()) != null){//loop vai começar na 2nd linha
try{
logger.info("SfID atualizado = "+totalArquivosAtualizados);
curLine = curLine.replace("\"", "");
String[] line = curLine.split(",");
String numCF = line[1];
String sfId = line[0];
logger.info("Update no MySql com ID da cadeia forçada. numCF:" +numCF +" sfId:"+sfId);
cadeiaForcadaDao.updateSfID(Long.valueOf(numCF), sfId);
totalArquivosAtualizados++;
logger.info("Update no MySql FINALIZADO.");
}catch(NumberFormatException e){
this.logger.error("Ocorreu um erro ao atualizar a cadeia: ", e);
}catch(Exception e1){
this.logger.error("Ocorreu um uma exeção genérica: ", e1);
}
}
if (csvReader != null) {
try {
logger.info("Fechando o arquivo csvReader!");
csvReader.close();
} catch (IOException e) {
this.logger.error("Erro de cono fechamento de arquivo .CSV =>" +e.getMessage());
}
}
logger.info("Total de SfIDs atualizados = "+totalArquivosAtualizados);
}
logger.info("Gravação finalizada");
}
} catch (ConnectionException e) {
this.logger.error("Erro de conexão ao SF: " + e.getMessage(), e);
} catch (UnsupportedEncodingException e) {
this.logger.error("Erro de 'Encoding': " + e.getMessage(), e);
} catch (IOException e) {
this.logger.error("Erro de IO: " + e.getMessage(), e);
} catch (InterruptedException e) {
this.logger.error("Erro de Interrupção: " + e.getMessage(), e);
}
The line referenced in the error (Accountdataloader.java:1387) is precisely:
while((curLine = csvReader.readLine()) != null){
which is when reading file, there is some limit of Bufferedreader or Mysql for various updates this way?
You are using Ibernate?
– adelmo00
No, I’m using jdbc/spring
– Ronaldo
You can post the code of these methods: ar.com.qualaconsulting.AccountDataLoader.loadCF_Ids() ar.com.qualaconsulting.AccountDataLoader.preprocess()
– adelmo00
@adelmo00 the loadCF_Ids() method is just above code, simplified because it does other searches to fill the Results before being added in the Inputstreamreader. the error happens after updating some records and could no longer pass to the next record in csvReader.readline(), I thought Inputstreamreader was corrupted but I used a lineNumberReader to check the number of lines and returned more than 3,000,000
– Ronaldo
Looking here on the web I found a tutorial to perform Spring Batch Insert from a .CSV. file See if you can help: http://www.mkyong.com/spring-batch/spring-example-csv-file-to-database/. For the error that occurred I haven’t found anything yet, as soon as I find answer put
– adelmo00
OK Thanks Delmo I took a look at your tutorial but it’s not what I’m looking for, the problem really is in reading the content of Bufferreader, but there comes a time when it finds this error and stops reading the rest of the file and consequently update in mysql. At first I thought it was an empty line in Buffer, but when I use the lineNumber it goes through all the lines using the same csvReader.readline(), so the doubt
– Ronaldo