Is there a size limit on the writerow of csv?

Question

I will create a CSV of 1,341 lines (with header). In Python 3 I used csv commands, but the file created has 1,282 lines

The line data is extracted from 1,340 Pdfs, from the metadata. I created a list and print to check, it is extracting everything right

The code of the CSV:

import csv

conjunto = open('emendas_autores.csv', mode='w', encoding='latin_1')

resultado = csv.DictWriter(conjunto, fieldnames=["Arquivo", "Autor", "Assunto", "Data_Criacao", "Data_Moficacao"])

resultado.writeheader()

def salva_csv():
    print("+")
    print (arquivo)
    print (author)
    print (subject)
    print (creation_date)
    print(mod_date)
    print("+")
    resultado.writerow({'Arquivo': arquivo,
                   'Autor': author, 
                   'Assunto': subject,
                   'Data_Criacao': creation_date,
                   'Data_Moficacao': mod_date})
    return

Then goes the iteration that extracts the data from the Pdfs and calls the function salva_csv() and each file

Please would anyone know what might be wrong to stop at 1,282 lines? There is size limit on the writerow of csv?

Answer 1

I don’t know exactly how the csv library works, but you could also use the pandas to read and save csvs (I don’t know if in your case it’s better).

You could do

import pandas as pd

csv = pd.read_csv('emendas_autores.csv')

dados = {
    'autor': ['autor1', 'autor2', 'autor3'],
    'assunto': ['assunto1', 'assunto2', 'assunto3']
}

df = pd.DataFrame(dados)

df.to_csv('nome_meu_csv.csv', index=False)