Htmlunitdriver does not find field id - Selenium

Question

I am using Selenium to automate the browser user action. I am using the HtmlUnitDriver, not to open the browser and not have to point a .exe.

The page I am doing, will show the components according to values selected in combos. When I change the second combo it is that the problems begin, because it does not find the id of the next.

I realized that the HTML code at the moment is different and so does not find, I’m just not able to find a solution to it. I’ve used the wait on terms, thread.sleep, find the element by Xpath, none of this solved for me.

Code:

public static void main(String[] args) throws InterruptedException {

    HtmlUnitDriver driver = new HtmlUnitDriver(true);

    try{        

        driver.get("https://www.fazenda.sp.gov.br/guiasinternet/Gare/Paginas/Gare.aspx");

        driver.findElement(By.id("ReceitaTipo")).click();

         {
           WebElement dropdown = driver.findElement(By.id("ReceitaTipo"));
           dropdown.findElement(By.xpath("//option[. = 'GNRE']")).click();
         }

         //printar código fonte da pagina            
         System.out.println(driver.getPageSource());

         Thread.sleep(5000);

         driver.findElement(By.id("CodigoReceita")).click();

         {
           WebElement dropdown = driver.findElement(By.id("CodigoReceita"));
           dropdown.findElement(By.xpath("//option[. = '10002.1 - Energia Elétrica']")).click();
         } 

         //A partir daqui para de funcionar
         System.out.println(driver.getPageSource());         

         driver.findElement(By.id("CnpjCpf")).click();
         driver.findElement(By.id("CnpjCpf")).sendKeys("087.271.516-54");   

         driver.quit();

    }catch(Exception e){
        e.printStackTrace();
    }
}

Answer 1

I believe that this will not be possible with Htmlunitdriver at the moment, even in its latest versions and even with the

new HtmlUnitDriver(true);

Htmlunitdriver does not have such interesting support for running js, searching for stackoverflow you see that everyone who tries something of the kind, is facing problems.

In the specific page you are trying to believe that when you select a recipe code it runs some script, however through Htmlunitdriver this script execution is not working and the page does not change status.

My suggestion and temporary solution until Htmlunitdriver has better support is to use Chrome as headless, in which case you would depend on the executable but would not need to open the browser, follow an example where you has chromedriver.exe in your project folder:

private WebDriver getHeadlessChrome() {
    try {
        //Sugiro manter o service fora desse método, isso aqui é apenas um exemplo.
        ChromeDriverService chromeDriverService = new ChromeDriverService.Builder()
                .usingAnyFreePort()
                .usingDriverExecutable(new File("chromedriver.exe"))
                .build();
        ChromeOptions chromeOptions = new ChromeOptions().addArguments("--headless");

        return new ChromeDriver(chromeDriverService, chromeOptions);
    } catch (Exception e) {
        //Do something
    }
    return null;
}

Another tip, when working with dropdown, I suggest you use 'Select'' import org.openqa.selenium.support.ui.Select; follow the example of your select code:

WebDriver driver = getHeadlessChrome();

try{
    driver.get("https://www.fazenda.sp.gov.br/guiasinternet/Gare/Paginas/Gare.aspx");

    Select typeDropdown = new Select(webDriver.findElement(By.id("ReceitaTipo")));
    typeDropdown.selectByVisibleText("GNRE");

    Select codeDropdown = new Select(webDriver.findElement(By.id("CodigoReceita")));
    codeDropdown.selectByVisibleText("10002.1 - Energia Elétrica");

    driver.findElement(By.id("CnpjCpf")).click();
    driver.findElement(By.id("CnpjCpf")).sendKeys("087.271.516-54");
    driver.quit();

} catch (Exception e){
    e.printStackTrace();
}

Another detail, your question is very interesting to the community, since apparently, at the moment there is no answer to solve this problem with the requirements you need that are:

• Do not open the browser
• Do not need an . exe
• Have support due to javascript execution.