0
I’m trying to get data from the site: Thing Verse, but I don’t succeed.
I tried to put the html code here, but it would be too big... Anyway I search for the Xpath and find the elements I want to extract, but when running the python code nothing is returned.
Here are the Xpath:
//div[@class="SearchResult__searchResultItem--c4VZk"]
//span[@class="ThingCardHeader__cardNameWrapper--3xgAZ"]
I can find several different ways the elements with Xpath, but when I run the code returns nothing.
The code:
import scrapy
class ThSpider(scrapy.Spider):
name = 'th'
start_urls = ['https://www.thingiverse.com/']
def parse(self, response):
results = response.xpath('//div[@class="SearchResult__searchResultItem--c4VZk"]')
for result in results:
yield{
'title' : result.xpath('//span[@class="ThingCardHeader__cardNameWrapper--3xgAZ"]/text()').get
}