Scroll Down Python and Selenium

Asked

Viewed 711 times

0

I’m running a boot to collect information from Facebook for a political survey. However, when I collect the comments I’m not getting the Scroll Down of the page.inserir a descrição da imagem aqui

I’ve tried several code formats, but scroll down only happens outside the pop-up, IE, the main page on facebook and not the one where the image and comments appear. I have tried several ways, but without success - I leave below some codes I have tried.

the_last_in_list = driver.find_elements_by_xpath('//div[@class="_4swz _293g"]')[-1]
the_last_in_list.location_once_scrolled_into_view

for i in range(10):
    driver.execute_script("window.scrollBy(0,500)","")
    sleep(2)

page = driver.find_element_by_tag_name("html")
page.send_keys(Keys.END)

I’m grateful for your help/

python selenium web-scraping

  • Dude, I think you need to get Lenium to click first on the div that contains the comments, I inspected element here and found a form#u_r_2, have it clicked before sending the scrollBy

1 answer

0

It probably happened because the window you opened is'floating'. Find the div[@class=~~] and do:

opened_window = self.driver.find_element_by_xpath('//div[@class="isgrP"]')
        scroll = 0
        while scroll < 10: # Para dar 10 scroll down
            self.driver.implicitly_wait(30) # Dar um tempo para aguardar e 
                                              não dar erro
            self.driver.execute_script('arguments[0].scrollTop = arguments[0].scrollTop + arguments[0].offsetHeight', opened_window)
            scroll += 1

Browser other questions tagged python selenium web-scraping

You are not signed in. Login or sign up in order to post.