Open text file in float value list

Question

I need to open some files I have here that are in txt format which is even less common and specific in data processing issues, these files consist of values from 0,000 up to 0,999 and also from -0,000 to -0,999, and each of these values is separated by the tab ' t'.

This is the original contents of the file (testdata1.txt):

0.4422  -0.1441 0.1334  0.2333  0.5541  0.1331
0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0

Just below I made this algorithm so I could read all the contents of the testdata1.txt file and replace the tabs with commas and put everything in a list:

with open('testdata1.txt') as fout:
    rows = fout.readlines()
    #testrow = ['0.4422\t-0.1441\t0.1334\t0.2333\t0.5541\t0.1331\n','0\t0\t0\t0\t0\t0\t0\t0\t0\n','0\t0\t0\t0\t0\t0\t0\t0\t0\n','0\t0\t0\t0\t0\t0\t0\t0\t0\n']     
    lista = [row.replace('\t',',') for row in testrow]
    new_one = [i.replace("'","") for i in lista]
    #lista_np = np.array(lista)
    #newfile.write(row.replace('\t',','))
    #float1 = lista_np[0]
    #avali = eval(new_one[0])
    print(new_one)

This is the original exit before the execution:

['0.4422\t-0.1441\t0.1334\t0.2333\t0.5541\t0.1331\n', '0\t0\t0\t0\t0\t0\t0\t0\t0\n', '0\t0\t0\t0\t0\t0\t0\t0\t0\n', '0\t0\t0\t0\t0\t0\t0\t0\t0\t']

And he’s giving this exit after the execution:

['0.4422,-0.1441,0.1334,0.2333,0.5541,0.1331\n', '0,0,0,0,0,0,0,0,0\n', '0,0,0,0,0,0,0,0,0\n', '0,0,0,0,0,0,0,0,0\n']

What I would like to do is take all these simple quotes out of each of the characters, and get these values 0.4422, -0.1441, 0.1334, 0.2333 to be individualized and convert each of them into float, so I’m looking to put all of this in one position: new_one[0] and leave each character separate, for example:

[0.4422,-0.1441,0.1334,0.2333,0.5541,0.1331,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

This file I created is just an abstract, I’m working on a bigger file than that only it would be too long to show all of it here. I tried using float(new_one) and by numpy.loadtxt but none of them worked causing the same error:

ValueError: could not convert string to float:

I’ve looked all over the internet and I couldn’t find anything to help with that, so now I’m stuck in that part.

Answer 1

You can read the file as CSV with the character delimiter \t:

import csv

def read_file(filename):
    with open(filename) as stream:
        reader = csv.reader(stream, delimiter='\t')
        for row in reader:
            yield from map(float, row)

This will return a generator that will consume your file already returning all values as float. If you want a list of all of them you can make:

valores = read_file('data.txt')

print( list(valores) )

That would display:

[0.4422, -0.1441, 0.1334, 0.2333, 0.5541, 0.1331, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]