1
I am very beginner in the subject, can you help me? I am testing Spiders to seek bids. But I can’t return the items through the form.
I have the code below example:
import scrapy
from scrapy.http import FormRequest
class AmparoDeSaoFranciscoSpider(scrapy.Spider):
name = 'AMPARO_DE_SAO_FRANCISCO'
start_urls = ['https://amparodosaofrancisco.se.gov.br/portaltransparencia/?servico=cidadao/publicacaolicitacao/']
def parse(self, response):
yield scrapy.FormRequest.from_response(response=response,
formdata={'ctl00$body$hfAnoLicitacoes': '2020' ,'ctl00$body$hfMesLicitacoes': '04'},
callback=self.parse_item)
def parse_item(self, response):
orgao = response.xpath('//*[@id="dataTables-Licitacoes"]/tbody/tr[1]/td[2]').get()
processo = response.xpath('//*[@id="dataTables-Licitacoes"]/tbody/tr[1]/td[3]').get()
modalidade = response.xpath('//*[@id="dataTables-Licitacoes"]/tbody/tr[1]/td[4]').get()
autorizacao = response.xpath('//*[@id="dataTables-Licitacoes"]/tbody/tr[1]/td[5]').get()
objeto = response.xpath('//*[@id="dataTables-Licitacoes"]/tbody/tr[1]/td[6]').get()
yield {
'orgao': orgao,
'processp': processo,
'modalidade': modalidade,
'autorizacao': autorizacao,
'objeto': objeto,
}
This is not returning any value to me. I set the values: 2020 and 04 in the form, but the serial ideal already pull every year and months. Can you give me a light? abs!