Help Chrome Puppeteer Paging

Question

I’m doing a Webcrawler, in the following stream

1. Access a search URL
2. Collecting the information
3. Execute pagination (here is the error)

Click to the next page, reload it (then I tried to force a Wait), but it still doesn’t work at all.

Follow a passage

Puppter.ts

export class PuppeteerChrome {
public browser : Promise;
public page : EventEmitter;

constructor() {
}

async initializeBrowser() {
    this.browser = await puppeteer.launch({ headless: false, ignoreHTTPSErrors: true, args: ['--no-sandbox', '--disable-setuid-sandbox']});
    this.page = await this.browser.newPage();
    this.page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36')
    this.page.setViewport({
        width: 1920,
        height: 1080
    });   
}

async navigateTo(url) {
    if (!url)
      throw new Error('Missing URL');
      return await this.page.goto(url, {
        waitUntil: 'networkidle',
        timeout: 180000
    });
}

destroy() {
    if (this.browser)
      this.browser.close();
}

async closeBrowser() {
    return await this.browser.close();
}

async pages() {
    return await this.browser.pages();
}

async executeScript(jQueryExpression) {
    try {
      const scriptResult = await this.page.evaluate(jQueryExpression);
      return scriptResult;
    } catch (err) {
      console.error(err);
      return {};
    }
}

async waitPageLoad() {
    let pageready =  false;
    do {
    let readyState = await this.executeScript('document.readyState;');
    pageready = readyState == 'complete' || readyState == 'interactive';
    } while (!pageready)
    return pageready;
} }

client ts.

//browser
this.chrome = new PuppeteerChrome ();

// Next Page
await this.chrome.executeScript(scripts["ScriptNextPageBrowser"]);

// tentztiva 2
this.chrome.executeScript(scripts["ScriptNextPageBrowser"]).then(() => {
   this.chrome.waitPageLoad();
});

Does anyone have any tips?

Answer 1

I ended up solving with a waitforTimeout() after the execution of the click.