Import csv with pandas. Column values with semicolon. What to do?

Question

I’m importing CSV files with pandas to do a data wipe. And I’m finding a problem. Some values have ';' .

For example: df[0][1] = "STREET A ; "

I import csv through: df = pd.read_csv('GUAXUPE.csv', header = None, Sep = ',', encoding = 'utf-8')

And despite having placed the separator as the comma, the pandas recognizes the ' ' as a separator and adds a field where it should not, deregulating part of the table.

I have already searched on the internet, I read the part of the documentation of pandas that speaks of read_csv() and I found no answer. Someone has an idea ?

Answer 1

In the engine command add the engine parameter, it would look like this:

df = pd.read_csv('GUAXUPE.csv', header = None, sep = ',', encoding = 'utf-8',engine='c')

In the read_csv documentation it says that the Python engine automatically recognizes tabs, the C engine does not. I believe it can function.

follows the doc: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html

Answer 2

I solved the mystery! The problem was not pandas. But rather the way he was supervising the generated tables. When he opened the file. csv in libre office, the program initially asked to choose which characters would be considered as separator. And automatically, the semicolon was selected. I supervised the tables by the notebook jupyter, and it’s all right !

Answer 3

One way around is to join the column with the information to the left of ";" with the column containing the information to the right of ";". Would something like this:

df['dados_esquerda']+=";"+df['dados_direita']

In case you didn’t want to name the columns, I believe this could be done like this:

df.loc[:,1]+=";"+df.loc[:,2]

In the above example, I assumed that the second column contains the data to the left of the ";" and the third column the data to the right of the ";".

Later, you can delete the unwanted column with the del command:

del df[2] #ou del df['dados_direita']