How to Scrap with Python to compare prices on a shopping site?

Question

Comparing the price of the same product every minute and notifying me when the value drops 50%. I used a While >= but it is returning an error indicating that you cannot use >= between str and int. It ends up returning the price as a string.

I’ll show you the code:

https://github.com/ViniciusCavalcante2000/Scraping-de-pre-os/blob/main/scraping%20de%20pre%C3%A7os%20Python

from selenium import webdriver
from time import sleep
from selenium.webdriver.chrome.options import Options


chrome_options = Options()
chrome_options.add_argument('--hedless')
driver = webdriver.Chrome()
driver.get('https://www.magazineluiza.com.br/ssd-480gb-kingston-sata-rev-3-0-leituras-500mb-s-e-gravacoes-450mb-s-a400/p/220920100/in/ssdi/')
sleep(3)
produto = driver.find_element_by_xpath('/html/body/div[3]/div[5]/div[1]/div[2]/h1').text
preçoatual = driver.find_element_by_xpath('/html/body/div[3]/div[5]/div[1]/div[3]/div[2]/div[4]/div/div[2]/div/span[2]').text
preçoalvo = 300

while preçoatual >= preçoalvo:
    print(preçoatual)
    sleep(1)
print('preço alvo atingido')



____________________________________________________________________________






Código alterado após as sugestões do amigo:

from selenium import webdriver
from time import sleep
from selenium.webdriver.chrome.options import Options


chrome_options = Options()
chrome_options.add_argument('--hedless')
driver = webdriver.Chrome()
driver.get('https://www.magazineluiza.com.br/ssd-480gb-kingston-sata-rev-3-0-leituras-500mb-s-e-gravacoes-450mb-s-a400/p/220920100/in/ssdi/')
sleep(3)
produto = driver.find_element_by_class_name('header-product__title').text
preçoatual = driver.find_element_by_class_name('price-template__text').text
preçoatual = preçoatual.replace(',', '.')
preçoatual = float(preçoatual)
preçoalvo = 300

print(produto)
while preçoatual > preçoalvo:
    print(preçoatual)
    driver.get('https://www.magazineluiza.com.br/ssd-480gb-kingston-sata-rev-3-0-leituras-500mb-s-e-gravacoes-450mb-s-a400/p/220920100/in/ssdi/')
    sleep(3)
    produto = driver.find_element_by_class_name('header-product__title').text
    preçoatual = driver.find_element_by_class_name('price-template__text').text
    preçoatual = preçoatual.replace(',', '.')
    preçoatual = float(preçoatual)
    sleep(5)
print('preço alvo atingido')

Answer 1

If the problem is just the value being as string, just convert it to float.

preçoatual = driver.find_element_by_xpath('/html/body/div[3]/div[5]/div[1]/div[3]/div[2]/div[4]/div/div[2]/div/span[2]').text

# Para isso será necessário remover os caractere "." que estiverem na variável e
# depois pegar os caracteres "," e substituir por ".", assim será possível fazer a
# conversão sem gerar erros.
preçoatual = preçoatual.replace('.', '')
preçoatual = preçoatual.replace(',', '.')
preçoatual = float(preçoatual)

However from what I saw on the website link, you’re taking the product title instead of the value. So to make it easier and more certain that you’re getting the right element, I recommend you take the value by its class. To do this just use the function find_element_by_class_name() of selenium. The class of the element possessing the product price is price-template__text.

then just replace:

driver.find_element_by_xpath('/html/body/div[3]/div[5]/div[1]/div[3]/div[2]/div[4]/div/div[2]/div/span[2]').text

For:

driver.find_element_by_class_name('price-template__text').text

link to the function locate_element_by_class_name():

https://selenium-python.readthedocs.io/locating-elements.html#locating-Elements-by-class-name

Now another important detail to let you know, is that the way you’re comparing, I believe, is incorrect. You are storing only once the product price and soon after you use it for comparison on the block while. So you are comparing the same price of the product every time. To get around this, I suggest you change the operator of the while and then enter a function to update the page data and make the price query within the while, so the value in the variable will always be updated.