Load multiple concatenated CSV at once in Python

Question

Talk personal, quiet?

There is an easier way to load . csv concatenated at once instead of doing the procedure below?

z1 = pd.read_csv('arquivo1.csv')
z2 = pd.read_csv('arquivo2.csv')
z3 = pd.read_csv('arquivo3.csv')
z4 = pd.read_csv('arquivo4.csv')
z5 = pd.read_csv('arquivo5.csv')
z6 = pd.read_csv('arquivo6.csv')
z7 = pd.read_csv('arquivo7.csv')
z8 = pd.read_csv('arquivo8.csv')

objs = [z1, z2, z3, z4, z5, z6 ,z7 ,z8]

pd.concat(objs, axis=0)

Answer 1

The procedure is the same, but you can change it 1 line of code using the potential of MAP to apply the function to the whole file name matrix:

df = pd.concat(map(pd.read_csv, glob.glob('arquivo*.csv')))

Explore a little more map and reduce potential to work with matrices/lists, the combination of this will make your code smaller and more performative

I hope I’ve helped!

Answer 2

You can do it using glob

import glob
arquivos = glob.glob('arquivo*.csv')
# 'arquivos' agora é um array com o nome de todos os .csv que começam com 'arquivo'
array_df = []

for x in arquivos:
    temp_df = pd.read_csv(x)
    array_df.append(temp_df)

df = pd.concat(array_df, axis=0)