Pyautogui for virtual keyboard

Asked

Viewed 545 times

0

I am working on a project using Python 3.7 and the Pyautogui library.

I have a website that needs to login using the mouse to type on a virtual keyboard.

The problem is that the keyboard changes position with each click.

Does anyone know how to solve this problem?

html python selenium pyautogui

1 answer

1


I never got into the advanced part of this module, but I know how to do this:

PAG (easiest to call it that) also knows how to search for images on the screen, based on what it has available:

PAG.locateOnScreen('print.png')

If "print.png" is the key print *the print of the key must be in the same directory as yours. py, I believe there is another way without having the prints in the same path, but I didn’t get to the advanced of this module.

You can set the value of each key with the name of the photo, then using only the variable in the command:

a = 'a.png'
b = 'b.png'
PAG.locateOnScreen(a)
PAG.locateOnScreen(b)
...

Now you know how to find the key, now you have to click it. the command PyAutoGui.click() should click, but not on your image, yet...

 a = 'a.png'
 b = 'b.png'
 a_X,a_Y=PAG.locateCenterOnScreen(a)
 b_X,b_Y=PAG.locateCenterOnScreen(b)

 PAG.click(a_X,a_Y)
 PAG.click(b_X,b_Y)

Thus, it will search for the center of the image you provided, which will give its coordinates on your screen, then it will click on the coordinates that were provided by LocateCenterOnScreen()

But as your keyboard changes with each click, instead of searching for the coordinates of all the keys and then clicking, change your code to search and click, search and click. if possible, of a from time import sleep and put sleep(0.2) After each click to delay, the keyboard may take longer than your code. Also remember to replace the PAG for example PyAutoGui or whatever name you gave him. I hope that’s what you’re looking for, good luck!

Pyautogui Docs: https://pyautogui.readthedocs.io/en/latest/index.html

Edit1: it is likely that the site has a security against this, it is not difficult to make a program that locates the image. if you do, let me know that maybe I can help with that too.

Edit2: example of how it should look your directory

  • First I take print from the right screen ? Then I put this image inside my.py project.' ?

  • print of the key, type A, B, C, in png. saves the print in the same folder as your . py

  • for numbers is the same thing ?

  • yes, for numbers and letters, anything, just print a print with a name, within your code define a variable with its value being the name of your print, Example: d_key = "D_key.png"

  • ok. Thank you! :)

Browser other questions tagged html python selenium pyautogui

You are not signed in. Login or sign up in order to post.