Problem with Extractall not extracting exact occurrence

Question

I have the following Dataframe:

df = pd.DataFrame({'Texto' : ['é importante o sucesso', 'o dia está lindo']})

I have two names . txt palavras_positives.txt and palavras_neagtivas.txt. These two frames are opened and transformed into a list of strings as follows:

positive = pd.read_csv('palavras_positivas.txt', encoding='utf-8')
negative = pd.read_csv('palavras_negativas.txt', encoding='utf-8')

The . txt files are formed by a single column and each row has a single word. Both Dataframes are then transformed into string and used as in a search:

lista_positiva = positive['Positivas'].to_list()
pattern_positivo = '|'.join(lista_positiva)

The same process is effect for negative words.

So I use the pattern_positive and the pattern_negative in the following code:

df3['Positivos']=df3['Texto'].str.extractall(({pattern_positivo})").groupby(level=0).agg(','.join)

df3['Negativos'] = df3['Texto'].str.extractall(f"({pattern_negtivo})").groupby(level=0).agg(','.join)

My problem is this:

in the text: " success is important ", I can get the words "success" and "important" because they’re on the positive list. In the negative column I am extracting the word "impose" because this word exists in the negative list, so for some reason the code reads "important" and returns "impose".

My goal is to extract the exact occurrence in both extractall.

Can someone help me?

Answer 1

So what you want is to compare the words, try it like this:

import pandas as pd


data = {'Frases': ['Bom dia', 'Ok, vai lá', 'o sucesso é importante']}
df = pd.DataFrame(data)

negativ = ['mal', 'impor']
positiv = ['bom', 'ok', 'importante', 'sucesso']


def getByWordlist(row, wordlist):
    results = []
    for word in row['Frases'].split(' '):
        word = word.lower().replace(',', '')
        if word in wordlist:
            results.append(word)
    return results

df['Positivas'] = df.apply(getByWordlist, args=(positiv,), axis=1)
df['Negativas'] = df.apply(getByWordlist, args=(negativ,), axis=1)

print(df)

output:

                   Frases              Positivas Negativas
0                 Bom dia                  [bom]        []
1              Ok, vai lá                   [ok]        []
2  o sucesso é importante  [sucesso, importante]        []