Pick up mouse position on virtual machines

Asked

Viewed 668 times

4

I’m having a problem getting the mouse position in a virtual machine.

I have a macro program on a virtual machine that performs a series of loop actions, but it can happen from the catch action and the only way to detect this is with the mouse movement. I made a small system in Delphi that captures the mouse position and in case it stays more than 5 minutes in the same place I receive an email, but when I disconnect from the virtual machine and let the macro running in it, my system in Delphi stops capturing the position.

Does anyone know a method to solve this? It can be in any language.

  • Have you ever thought of, instead of capturing the mouse position, leaving a signal somewhere? Or incrementing a counter with each iteration of your loop... If you do this with a service, you can access the service remotely to see if your application is active.

  • The problem is that each loop of the macro can have an indefinite time, a loop can have time from 5 minutes to 20, everything depends on the particular case.

  • You can increment a counter in the service and in a client program make a request every five minutes (or other arbitrary interval). Think outside the box ;)

  • Why don’t you open a thread in the same macro program and checks the state of the mouse (or itself) through that thread?

  • @Felipe.Velar is not possible to modify the macro, is a system called Automate, I believe that make the system capture the position and send to my system is valid, vlw.

  • If user leaves the pc for 5 minutes as this detection?

  • @Bacco I am not connected to vm, so I would like to develop a system that identifies when the macro is stopped, but it has to be something that does not directly involve the operation of the macro, so an external program that captures the mouse, but from what I saw the system does not consider that the macro is moving the mouse.

  • What do you mean by "connected to the VM"? What virtualization program and guest operating system do you use? I say which version of Windows is virtualized?

Show 3 more comments

1 answer

1


I imagine you are using windows API to recover cursor position, but this function should stop working when you disconnect, end there is a "monitor" so it is possible to say the current position.

Not knowing the technology used in the macro, it’s also complicated to make affirmations, but I imagine that it uses the windows Apis to click buttons, type text, etc.

Well, this macro must have a mechanism, at each event, to determine whether or not it can continue with the actions.

Finally, a solution in this case, is to make that each time the macro is waiting for the next step of the loop, it counts the time and, in case of failure, run the program that sends the email.

Another approach, if you are monitoring a specific program, would be to monitor an element other than the mouse. If the macro goes through multiple screens, you could check the title of the current window with the windows API. Another possibility would be to capture the window image at regular intervals and check if anything has changed.

Ultimately, you can replicate the same logic that the macro uses to wait for the system to check whether or not it is waiting indefinitely.

However, if the macro is "dumb" in the sense that it has no checks, just goes acting in defined time intervals, you can think of another solution.

There is a tool called Sikuli which allows automation using images. For example, you capture the image from a button and then build a script to click that button through the image. Sikuli will look for the image of the button on the screen and can wait a while until that button appears. If the expected button does not appear, then you can capture the error. Sikuli implementations can be made in java or Python.


Updating

I did a search on the Apis related to the mouse cursor position. The mouse position is usually given relative to the current monitor, as Windows supports multiple screens. So when you disconnect all monitors the cursor can be disabled.

I assume you’re currently using the GetCursorPos. If so, try to check the GetCursorInfo, because it returns useful information about the cursor state and also about its position.

However, it seems that this information is not accessible by project decision. See this link, that says something like that:

You cannot get this information with the desktop locked. While it is locked, the security subsystem between takes over and your desktop is out of bounds.

  • Luiz, the problem is that when I am not connected it is as if the mouse does not move, it seems that the program that captures the position does not consider that the macro is moving the mouse.

  • @Mdomingues The program you made runs inside or outside the virtual machine?

  • It rotates inside the machine, capturing the current mouse position, if the mouse remains more than 5 minutes still it sends an email to me, warning that the macro is stopped.

  • @Mdomingues Certo, I had not committed myself to this detail. What is the virtualization technology that you use?

  • The virtual machines are mounted inside a linux server, I have no more details about, but each vm uses a Windows XP system, for better compatibility with the macro. I connect the vm’s using the program "Remote Desktop Connection Mananger", but after starting the macro disconnected me.

  • @Mdomingues I adjusted the response to reflect my current suggestions.

  • Thank you very much, I’m reading and as soon as possible put the outcome.

  • @Mdomingues I did some more research and added an update to the reply. Nothing very encouraging, however.

Show 3 more comments

Browser other questions tagged

You are not signed in. Login or sign up in order to post.