Slow iteration with pandas

Asked

Viewed 19 times

1

I am using the following code to generate all chords of up to 6 elements, with 12 possible notes for each element. Then the amount of chords generated should be : (12*12*12*12*12*12) + (12*12*12*12*12) + (12*12*12*12) + (12*12*12) + (12*12) + (12) = 3.257.436 . Right ?

Well, according to my calculations, it’ll take about 30 hours to finish on my laptop. That is if the processing speed does not change with time ... I created a free virtual machine on google Clouds, and I left this code running there tbm (8 vCpus, 8gb of ram) but it’s been a few hours.

So I’m wondering if there’s a way to speed up the process. I couldn’t use the Vm with 16 vCpus. And I don’t know what I can do to improve the code.

def calculando_todos_acordes_e_diferencas():
    import pandas as pd
    import itertools                          
    anagrama=[]
    for i in range(1,13):
        anagrama.append(i)

    tst=[[[0],[0]]]
    df=pd.DataFrame(tst, columns=["notas","diferencas"])
    count_name=-1

    for qntd_notas in range(7):
        for i in itertools.product((anagrama), repeat=qntd_notas) :
            diferencas=[]
            count=-1
            for primeiro in i :
                count=count+1
        
        
                if i.index(primeiro) != len(i)-1 :
                    for segundo in i[count+1:]:
                        diferenca= segundo - primeiro
                        if diferenca < 0 :
                            diferenca=diferenca* -1
                        diferencas.append(diferenca)

          #  if len(df.index) == 100000 :
           #     count_name=count_name+1
            #    df=df.append({"notas":list(i),"diferencas":diferencas},ignore_index=True)
             #   df.to_csv("acordes e diferencas pt %s.csv" %(count_name), index=False)
              #  df=pd.DataFrame(tst, columns=["notas","diferencas"])

            df=df.append({"notas":list(i),"diferencas":diferencas},ignore_index=True)
    
    df.to_csv("acordes e diferencas TOTAL2.csv", index=False)
            #else:
            
     
calculando_todos_acordes_e_diferencas()
  • I’m a little confused... It wouldn’t be (12*11*10*9*8*7) + (12*11*10*9*8) + (12*11*10*9) + (12*11*10) ? I’m not a deep music connoisseur, but I believe that the chords have at least 3 notes. So I wouldn’t have (12*11) or (12). As for the number reduction in a unit, I thought: After choosing a note to assemble the chord, you cannot choose it again. This would greatly reduce processing time.

No answers

Browser other questions tagged

You are not signed in. Login or sign up in order to post.