Python: Web Scraping in Dynamic Page

Asked

Viewed 384 times

2

I am researching on web scraping. I have found dozens of pages, however, in none of them found how to extract information from dynamic pages.

Well, I’m trying to get the value of the LTCBRL cryptocurrency directly on the following website: https://br.tradingview.com/symbols/LTCBRL/

Below is the picture of the screen... inserir a descrição da imagem aqui

The tag on which the value is: 

<span class="tv-symbol-header-quote__value tv-symbol-header-quote__value--large js-symbol-last">269.97</span>

I’d just like a tip on how to get the value.

The code provided by the colleague Jones Vieira was:

He informs that for it worked, however, for me present the error in the image below.

# -*- coding: utf-8 -*-
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
driver = webdriver.Chrome(
    chrome_options = chrome_options,
    service_args = ['--verbose']
)

driver.get('https://br.tradingview.com/symbols/LTCBRL')
wait(driver, 10).until(
    EC.presence_of_element_located((By.CLASS_NAME, 'tv-symbol-header-quote__value'))
)

value = driver.find_element_by_class_name('tv-symbol-header-quote__value').text
print(value)

inserir a descrição da imagem aqui

python-3.x
  • 1

    https://scrapy.org

  • you are trying to pick up the object that holds the value directly?

  • @RFL yes. I am. It is that value 269.99, within the <span class="tv-Symbol-header-quote__value tv-Symbol-header-quote__value-large js-Symbol-last">269.97</span>

  • I have researched so much and, I read in questions of other users that can be the issue of javascript.

  • Have you tried using Selenium? https://selenium-python.readthedocs.io

  • I already researched about, a little while ago. I’ll still need to learn how to use. I tried to get a code ready just for testing but it didn’t work.

  • If someone told me that the library Selenium or any other would certainly work, I would study it knowing that the solution would be there... the point is that I have read enough and, aimlessly I feel lost.

Show 2 more comments

1 answer

2


Try this code, I made it here quick to test and it worked:

# -*- coding: utf-8 -*-
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
driver = webdriver.Chrome(
    chrome_options = chrome_options,
    service_args = ['--verbose']
)

driver.get('https://br.tradingview.com/symbols/LTCBRL')
wait(driver, 10).until(
    EC.presence_of_element_located((By.CLASS_NAME, 'tv-symbol-header-quote__value'))
)

value = driver.find_element_by_class_name('tv-symbol-header-quote__value').text
print(value)

  • Dear Vieira Jones. Thank you for your attention. The code didn’t work for me. I updated my question by placing the code you reported and the error image. Thank you!

  • I installed the Chrome drive by the following link https://sites.google.com/a/chromium.org/chromedriver/downloads and it worked for your code. Thank you

  • Good that worked friend. I did a test on a project that I already owned the driver, for that reason I forgot to inform the need for it.

Browser other questions tagged python-3.x

You are not signed in. Login or sign up in order to post.