Search for e-mail or word within a webdriver site in python?

Question

Hello,

I am wanting to create a Bot to enter a website and search if that site has an email to contact and create an email list ... the problem is that this can all be very generic.. on some site may be in an id, on another may be in class ... I thought I would try to read all the HTML elements of the site and check if it has text with "@" and ". COM" to characterize as an email ... however I’m coming from VBA to python and I’m still catching ... in the documentation of Selenium it shows an example that searches a word inside the page, but when I turn it from error in the import of "PAGE"... someone could help me with a code to read all HTML elements and look for a word or help me solve this import Page issue, because searching Google I found nothing if this page is already installed in Selenium or if I have to install, which PIP INSTALL I have to use and etc ...

import unittest

from selenium import webdriver

import page

class PythonOrgSearch(unittest.TestCase):
    """A sample test class to show how page object works"""

    def setUp(self):
        self.driver = webdriver.Firefox()
        self.driver.get("http://www.python.org")

    def test_search_in_python_org(self):
        """
        Tests python.org search feature. Searches for the word "pycon" then verified that some results show up.
        Note that it does not look for any particular text in search results page. This test verifies that
        the results were not empty.
        """

        #Load the main page. In this case the home page of Python.org.
        main_page = page.MainPage(self.driver)
        #Checks if the word "Python" is in title
        assert main_page.is_title_matches(), "python.org title doesn't match."
        #Sets the text of search textbox to "pycon"
        main_page.search_text_element = "pycon"
        main_page.click_go_button()
        search_results_page = page.SearchResultsPage(self.driver)
        #Verifies that the results page is not empty
        assert search_results_page.is_results_found(), "No results found."

    def tearDown(self):
        self.driver.close()

if __name__ == "__main__":
    unittest.main()

Answer 1

Breno, I went looking for this tutorial that you started, it seems you skipped the reading of the session 6.2, she explains how to create the page.

https://selenium-python.readthedocs.io/page-objects.html

import page

The above code is trying to import a class that does not yet exist, so it is giving error.

On how to find an element that contains the @ and end with .with, this will be possible using Xpath and the findElement method.

Example looking for any element that contain @python org.:

driver.find_element_by_xpath("//*[contains(text(),'@python.org')]")

Example looking for any element that finish with @python org.:

driver.find_element_by_xpath("//*[substring(text(), string-length(text()) - string-length('@python.org') + 1) = '@python.org']")

Example looking for any element that contain @ and finish with .org:

driver.find_element_by_xpath("//*[contains(text(),'@') and substring(text(), string-length(text()) - string-length('.org') +1) = '.org']")