Scrapy - Search for items in form

Asked

Viewed 32 times

1

I am very beginner in the subject, can you help me? I am testing Spiders to seek bids. But I can’t return the items through the form.

I have the code below example:

import scrapy
from scrapy.http import FormRequest

class AmparoDeSaoFranciscoSpider(scrapy.Spider):
    name = 'AMPARO_DE_SAO_FRANCISCO'
    start_urls = ['https://amparodosaofrancisco.se.gov.br/portaltransparencia/?servico=cidadao/publicacaolicitacao/']

    def parse(self, response):
        yield scrapy.FormRequest.from_response(response=response,
                                               formdata={'ctl00$body$hfAnoLicitacoes': '2020' ,'ctl00$body$hfMesLicitacoes': '04'},
                                               callback=self.parse_item)

    def parse_item(self, response):
        orgao = response.xpath('//*[@id="dataTables-Licitacoes"]/tbody/tr[1]/td[2]').get()
        processo = response.xpath('//*[@id="dataTables-Licitacoes"]/tbody/tr[1]/td[3]').get()
        modalidade = response.xpath('//*[@id="dataTables-Licitacoes"]/tbody/tr[1]/td[4]').get()
        autorizacao = response.xpath('//*[@id="dataTables-Licitacoes"]/tbody/tr[1]/td[5]').get()
        objeto = response.xpath('//*[@id="dataTables-Licitacoes"]/tbody/tr[1]/td[6]').get()
        yield {
            'orgao': orgao,
            'processp': processo,
            'modalidade': modalidade,
            'autorizacao': autorizacao,
            'objeto': objeto,
        }

This is not returning any value to me. I set the values: 2020 and 04 in the form, but the serial ideal already pull every year and months. Can you give me a light? abs!

No answers

Browser other questions tagged

You are not signed in. Login or sign up in order to post.