How to determine if a point of a swing component is visible on the screen?

Question

How do I determine if a certain point X within a swing component is visible on the user screen?

For example, let’s assume that the JComponent To was added to a window B (normally, but not necessarily, a JFrame).

• Use the method isVisible() is easy. If this is false, then the component is not visible and with that neither the point X inside it. The same goes for the window B.
• If the window B is minimized, so the component To is not visible nor the point X within it.
• If the component To is within a container hierarchy where one of them is not visible, so the component is not visible.
• If the component To is beyond the limits of your container, so obviously it is not visible.
• If the component To is partially outside the container, I can without difficulty compute the rectangle of intersection between the component and its containers to know if the point X is inside the container.
• Case the window B has been dragged so as to be partially off the screen, I can use the class Toolkit of AWT to pick up what are the user’s monitor(s) areas(s) and then use this as part of the intersection of rectangles.
• Inside the window B, there may be another component D which overlaps with the component To hiding it in whole or in part. This I can solve by listing all the components of each window and checking the z-order.

So far so good, I know it is possible. But there is one more issue that I cannot resolve:

• The window B may be partly or wholly hidden due to another window C of another application that is on top, possibly hiding in whole or in part the component To.

That last item, I don’t know how to do. Someone has an idea?

Oh yes, finally, let’s ignore that some windows or components may have transparent or translucent areas that can show the contents of other windows or components that are underneath. Let’s consider that all windows and components are completely opaque rectangles.

---

I tried to get my hands dirty and I got this from here:

Pointvisibility.java

package pixelvisibletest;

import java.awt.Component;
import java.awt.IllegalComponentStateException;
import java.awt.Point;
import java.awt.Window;
import java.util.Objects;
import javax.swing.SwingUtilities;

/**
 * @author Victor
 */
public class PointVisibility {

    private PointVisibility() {
    }

    public static boolean isPointVisibleInComponent(Component c, Point componentLocation) {
        Objects.requireNonNull(c);
        Objects.requireNonNull(componentLocation);
        Point p;
        try {
            p = c.getLocationOnScreen();
        } catch (IllegalComponentStateException e) {
            return false;
        }
        p.x += componentLocation.x;
        p.y += componentLocation.y;
        return isScreenPointVisibleInComponent(c, p);
    }

    public static boolean isScreenPointVisibleInComponent(Component c, Point screenLocation) {
        Objects.requireNonNull(c);
        Objects.requireNonNull(screenLocation);
        Component d = findComponentInScreenLocation(screenLocation);
        while (d != null) {
            if (d == c) return true;
            d = d.getParent();
        }
        return false;
    }

    public static Component findComponentInScreenLocation(Point screenLocation) {
        if (screenLocation == null) return null;
        return findComponentInScreenLocation(screenLocation, findWindowInScreenLocation(screenLocation));
    }

    public static Window findWindowInScreenLocation(Point screenLocation) {
        if (screenLocation == null) return null;
        // BUG Não consegue determinar se o ponto screenLocation está ou não obscurecido por outra janela.
        for (Window window : Window.getWindows()) {
            Point compCoords;
            try {
                compCoords = window.getLocationOnScreen();
            } catch (IllegalComponentStateException e) {
                continue;
            }
            Point relativeToWindow = new Point(screenLocation.x - compCoords.x, screenLocation.y - compCoords.y);
            Component inTheSameWindow = window.findComponentAt(relativeToWindow.x, relativeToWindow.y);
            if (inTheSameWindow != null) return window;
        }

        return null;
    }

    static Component findComponentInScreenLocation(Point screenLocation, Window window) {
        if (window == null) return null;
        Point copy = (Point) screenLocation.clone();
        SwingUtilities.convertPointFromScreen(copy, window);
        return SwingUtilities.getDeepestComponentAt(window, copy.x, copy.y);
    }
}

Mousevisibility.java

package pixelvisibletest;

import java.awt.Component;
import java.awt.MouseInfo;
import java.awt.Window;
import java.util.Objects;

/**
 * @author Victor
 */
public class MouseVisibility {

    private MouseVisibility() {
    }

    public static boolean isComponentUnderMouse(Component c) {
        Objects.requireNonNull(c);
        Component d = findComponentUnderMouse();
        while (d != null) {
            if (d == c) return true;
            d = d.getParent();
        }
        return false;
    }

    public static Component findComponentUnderMouse() {
        return PointVisibility.findComponentInScreenLocation(MouseInfo.getPointerInfo().getLocation(), findWindowUnderMouse());
    }

    public static Window findWindowUnderMouse() {
        for (Window window : Window.getWindows()) {
            if (window.getMousePosition(true) != null) return window;
        }
        return null;
    }
}

To test the two classes above, I created this class:

Pixelvisibletest.java

package pixelvisibletest;

import java.awt.Color;
import java.awt.EventQueue;
import java.awt.Graphics;
import java.awt.MouseInfo;
import java.awt.Point;
import javax.swing.JComponent;
import javax.swing.JFrame;

/**
 * @author Victor
 */
public class PixelVisibleTest {

    public static void main(String[] args) {
        EventQueue.invokeLater(PixelVisibleTest::go);
    }

    private static void go() {
        JFrame jf = new JFrame();
        jf.setTitle("teste");
        jf.setLayout(null);
        jf.setSize(400, 100);

        JComponent b = new JComponent() {
            @Override
            public void paintComponent(Graphics g) {
                g.setColor(Color.blue);
                g.drawRect(0, 0, getWidth() - 1, getHeight() - 1);
                g.setColor(Color.red);
                g.drawRect(5, 5, 3, 3);
                g.setColor(Color.green);
                g.drawRect(6, 6, 1, 1);
            }
        };
        jf.add(b);
        b.setBounds(15, 15, 20, 20);

        jf.setVisible(true);
        jf.setResizable(true);
        jf.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);

        Thread t = new Thread(() -> {
            try {
                while (jf.isVisible()) {
                    EventQueue.invokeLater(() -> {
                        Point mouse = MouseInfo.getPointerInfo().getLocation();
                        boolean r1 = jf == MouseVisibility.findWindowUnderMouse();
                        boolean r2 = b == MouseVisibility.findComponentUnderMouse();
                        boolean r3 = MouseVisibility.isComponentUnderMouse(b);
                        boolean r4 = PointVisibility.isPointVisibleInComponent(b, new Point(6, 6));
                        boolean r5 = PointVisibility.isScreenPointVisibleInComponent(b, mouse);
                        boolean r6 = jf == PointVisibility.findWindowInScreenLocation(mouse);
                        boolean r7 = b == PointVisibility.findComponentInScreenLocation(mouse);
                        jf.setTitle("teste " + sn(r1) + sn(r2) + sn(r3) + sn(r4) + sn(r5) + sn(r6) + sn(r7));
                    });
                    Thread.sleep(5);
                }
            } catch (InterruptedException e) {
                // Deixa a thread morrer.
            }
        });
        t.start();
    }

    private static String sn(boolean b) {
        return b ? "S" : "N";
    }
}

The method findComponentUnderMouse() class MouseVisibility determines which component is under the position where the mouse is. The method isComponentUnderMouse(Component) test if a given component is the component under the mouse. These methods work properly.

It turns out that not always the reference point I care about is the mouse position. It may be some other arbitrary position on the screen. For these cases I created the methods findComponentInScreenLocation(Point) and isScreenPointVisibleInComponent(Component, Point) in class PointVisibility. These methods are analogous to the two methods I mentioned above MouseVisibility with the exception that the reference point taken is not the mouse position. These methods DO NOT work properly.

In the test class, within the thread, seven conditions are tested, each referring to a class method PointVisibility and MouseVisibility. The result of each test is one boolean which is shown in the window title (S = yes = true, N = no = false). Running this class (method main), just drag the window to any place and come out moving the mouse to test the operation. The point (6, 6) corresponds to the green pixel drawn on the screen inside the red square.

The methods of the class PointVisibility are failing when the reference pixel is over the component, but that pixel is covered by another application window. The code of the class PointVisibility is not able to perceive the existence of windows from other applications. This does not occur with the class MouseVisibility because in the depths of the AWT, the method getMousePosition(boolean) ends up making a call to the method isWindowUnderMouse(Window) of the interface java.awt.peer.MouseInfoPeer. The implementation of this interface is internal, specific to each JVM and/or SO, and usually done in native code, so you can’t just imitate it using any arbitrary point instead of the mouse position.

My goal is to have a class implementation PointVisibility that is able to perceive when the reference point is obscured by some other application window. I don’t need much of the class MouseVisibility, I just put it here to be comparative.

Answer 1

I don’t know any solution using only Java, but I know that it is possible to do this using the Windows API, and I believe it is also possible to do this in several other toolkits (GTK+, Qt, KDE, etc...) using similar functions as the Windows API.

In Windows you can do as follows:

1. Use the function Getdesktopwindow to get Handle from the desktop;
2. Use the function Getwindow with the GW_HWNDFIRST parameter to get Handle from the top window;
3. Use the function Getwindowrect to get the coordinates of the Handle rectangle from the window;
4. Use the function Getwindow with the GW_HWNDNEXT parameter to get Handle from the next window;
5. Repeat steps 3 and 4 until you reach your window;
6. Use the data you got from the other windows to calculate which parts of your window are visible (don’t forget to look at the order of the windows);
7. Calculate if the point you want to check is in one of the parts of the window that is visible;