Crawler

Asked

Viewed 237 times

-1

Currently I develop Crawler in php using guzzle, and lately I’ve been looking if it would be possible to use javascript to create Crawler, using the user’s ip and such, you know if it is possible? if you don’t know why or if someone has tried?

javascript web-crawler
  • 2

    Javascript in Node.js yes, Javascript browser is not possible.

1 answer

2


As placed by Guilherme Nascimento in the comments, you can develop a Crawler in Javascript using Node.js. There are even libraries that help you, through a headless browser (no visual interface, only programmable), such as the Puppeteer. Thus, you only need to make a request for a web page and, with the HTML obtained, use Puppeteer to inspect it.

In the browser, on the other hand, it is not possible, since even if we have Domparser available, we would not always be able to make requests to other web pages, mainly due to security policies CORS.

Browser other questions tagged javascript web-crawler

You are not signed in. Login or sign up in order to post.