Use java to find an image on the screen

Question

How can I make java read an image on the screen, the intention is to make a bot, as soon as I see the img click on it, the img is not in a browser, website, or something from the web, but in a program.

Answer 1

Actually there is a solution well viable for it. You can implement the libraries Sikuli into your app Javato detect and interact with image elements on the screen.

This library is designed to automate user interface testing, but can accommodate your needs quite easily!

Here’s an example of a little code that does exactly what you want it to do:

Source

import java.net.MalformedURLException;
import java.net.URL;

import org.sikuli.api.*;
import org.sikuli.api.robot.Mouse;
import org.sikuli.api.robot.desktop.DesktopMouse;
import org.sikuli.api.visual.Canvas;
import org.sikuli.api.visual.DesktopCanvas;

import static org.sikuli.api.API.*;

public class HelloWorldExample {

 public static void main(String[] args) throws MalformedURLException {

       // Open the main page of Google Code in the default web browser
       browse(new URL("http://code.google.com"));

       // Create a screen region object that corresponds to the default monitor in full screen 
       ScreenRegion s = new DesktopScreenRegion();

       // Specify an image as the target to find on the screen
       URL imageURL = new URL("http://code.google.com/images/code_logo.gif");                
       Target imageTarget = new ImageTarget(imageURL);

       // Wait for the target to become visible on the screen for at most 5 seconds
       // Once the target is visible, it returns a screen region object corresponding
       // to the region occupied by this target
       ScreenRegion r = s.wait(imageTarget,5000);

       // Display "Hello World" next to the found target for 3 seconds
       Canvas canvas = new DesktopCanvas();
       canvas.addLabel(r, "Hello World").display(3);

       // Click the center of the found target
       Mouse mouse = new DesktopMouse();
       mouse.click(r.getCenter());
 }
}

May also consult:

How to use Sikuli in your Java projects