GUI automation with python

Asked

Viewed 432 times

1

Next, I want to start a project to create a bot for a specific android game in python, for this, I use an emulator, Nox. My idea is to be able to work with the automation part of Gui (automatic clicks, press a button on the keyboard, etc) in a background way, in case, that could leave the window minimized, and go performing other tasks.

The library I found for the Gui automation python was the Pyautogui, but you can only make the clicks on the whole window if the pixels appear currently on the screen, which will not occur if the process is minimized

Is there any way I can do this process with the window minimized?

The following is an example of a very simple code to give an example

import os
import pygetwindow
import time
import pyautogui

title = "NoxPlayer"

os.startfile(r"C:\Program Files (x86)\Nox\bin\nox.exe")

time.sleep(5)

window = pygetwindow.getWindowsWithTitle(title)[0]
window.activate()
window.resizeTo(1280,720)
window.moveTo(0,0)

time.sleep(5)

posx, posy, height, wight = pyautogui.locateOnScreen(r"Images\Icon.png")
pyautogui.moveTo(posx + 10, posy + 10, 1)
python gui

  • can provide a minimum replicable example?

  • 1

    edited the question with an example

  • I don’t think I can, pyautogui automates the interface, if the window this minimized will not have graphical interface to be automated, maybe it works on a virtual machine, the script running inside it next to the application, then maybe you can minimize the window of the virtual machine

1 answer

1

Is there any way that I can do this process with window minimized?

No. It will be necessary to maximize the window so that image search methods (such as locateonscreen) can be used.

To maximize a window, I usually use the function below. The 'win_name' argument must be the name of the process/window to maximize.

def win_act(win_name):
    import win32gui
    import re

    class WindowMgr:
        """Encapsulates some calls to the winapi for window management"""

        def __init__(self):
            """Constructor"""
            self._handle = None

        def find_window(self, class_name, window_name=None):
            """find a window by its class_name"""
            self._handle = win32gui.FindWindow(class_name, window_name)

        def _window_enum_callback(self, hwnd, wildcard):
            """Pass to win32gui.EnumWindows() to check all the opened windows"""
            if re.match(wildcard,
                         str(win32gui.GetWindowText(hwnd))) is not None:
                self._handle = hwnd

        def find_window_wildcard(self, wildcard):
            """find a window whose title matches the wildcard regex"""
            self._handle = None
            win32gui.EnumWindows(self._window_enum_callback, wildcard)

        def set_foreground(self):
            """put the window in the foreground"""
            win32gui.SetForegroundWindow(self._handle)

    w = WindowMgr()
    for x in range(3):
        w.find_window_wildcard(f".*{win_name}*")
        # time.sleep(2)
        w.set_foreground()

Browser other questions tagged python gui

You are not signed in. Login or sign up in order to post.