Most voted "scrapy" questions
Scrapy is a web framework to extract data needed from websites (Web scraping). It is open-source, written in Python and controlled using command line tools.
Learn more…60 questions
Sort by count of
-
0
votes1
answer35
viewsDict with repeated Python attributes
Good afternoon! I’m putting together a formdata for a post, formdata = { 'data': '', 'controle': 'ADMIN', 'g-recaptcha-response': recaptcha_response } for numero in nDams: formdata['nu_dam[]'] =…
-
0
votes1
answer52
viewsThe URL responds only with the open site
I need to scrape the information from this page here. In the developer tools, I found this link with the necessary answers. The problem is that the link with the answers opens only if the site is…
-
0
votes0
answers19
viewsHow to run a scrapy + Django on Runtime without getting error?
I have a scrapy with 2 Spiders that when running from the command line works perfectly without errors. However, when I call through the application occurs the following: 2021-02-09 12:31:39…
-
0
votes0
answers24
viewsPython - Scrapy - Return nested Json (Json’s List)
Hello, I’m having a problem generating a dictionary answer within another dictionary. I have a home page that contains 8 bimonthly programs, in these picked up their respective links and properties.…
-
0
votes0
answers20
viewsXPATH span within div
I’m trying to get data from the site: Thing Verse, but I don’t succeed. I tried to put the html code here, but it would be too big... Anyway I search for the Xpath and find the elements I want to…
-
-1
votes1
answer207
viewsHow to use scrapy on Asp.net pages
Well folks, my question is this:: I have to download the excel file of the product that contains the description "Maíz", Product type "Los Démas. En grano." and marketing "In bulk with 15 %…
-
-1
votes1
answer123
viewsPip install scrapy error 1
I’m trying to install Scrapy in vscode but every time there’s an error that I’m not able to solve, someone could help me? error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for…
-
-2
votes1
answer48
viewsWrite excel file within Scrapy project
I have a Spider that picks up the xlsx links, in Request I call the files and saved in: def save_file(self, response): f = open("teste.xls", "wb") f = write(response.body) f.close() But returns the…
-
-2
votes0
answers19
viewsHow to make a scrapy post in python?
I am learning the lib Scrapy in Python and I am having difficulty to perform a request with the POST method for the url. I’m trying to use the following code: form={"letraLocalidade":"", "ufaux":"",…
-
-3
votes1
answer55
viewsOnly json content between []
I have a question about a code I made using scrapy to collect data and send to a json file. The problem is that the file formatting is not as it usually is, so I found it strange, I’m in doubt if…